LLM-Powered Automatic Product Category Classification

Building an LLM-Based Product Category Auto-Classification System

Overview

We built a system that automatically classifies e-commerce product categories using an LLM. The project ran from November 26 to December 26, 2024 — about one month, across 48 commits — and involved experimenting with various approaches to achieve precise product category matching.

Problem Statement

Product category classification is a core challenge in e-commerce:

• Hundreds of thousands of products are listed every day
• Seller-assigned categories are frequently inaccurate
• Correct classification directly affects search quality and recommendation accuracy

To overcome the limitations of existing rule-based classification, we decided to leverage an LLM's natural language understanding capabilities.

Development Process

Phase 1: Core Architecture Design (11/26–11/29)

After initializing the project, we landed 13 commits in the first week alone to lay the foundation.

class CategoryClassifier:
    """LLM 기반 카테고리 분류기"""
    
    def __init__(self, model: str = "gpt-4"):
        self.model = model
        self.category_tree = self.load_category_tree()
    
    def classify(self, product_info: dict) -> str:
        """상품 정보로 카테고리 예측"""
        prompt = self.build_prompt(product_info)
        result = self.llm.invoke(prompt)
        return self.parse_category(result)

Phase 2: Prompt Engineering (12/02–12/10)

This was the most experiment-heavy stretch of the project. We tested a wide range of prompting strategies across 11 commits.

Core challenge: selecting the correct leaf node from among thousands of categories.

Approach 1: Hierarchical Classification

# Step 1: Select top-level category
major_category = classify_level1(product_name, product_desc)  # e.g., "Clothing"

# Step 2: Select mid-level category  
mid_category = classify_level2(major_category, product_detail) # e.g., "Clothing > Men's Clothing"

# Step 3: Select leaf category
final_category = classify_level3(mid_category, product_spec)   # e.g., "Clothing > Men's Clothing > T-Shirts"

Approach 2: Few-Shot Classification

Improve matching accuracy by providing examples of similar existing products along with their categories.

Approach 3: Embedding + LLM Hybrid

Use product name embeddings to narrow down candidate categories, then let the LLM make the final selection.

Phase 3: Performance Optimization (12/12–12/20)

We implemented a batch processing pipeline and cost optimizations for high-volume workloads.

# 배치 처리 파이프라인
async def batch_classify(products: list, batch_size: int = 50):
    results = []
    for batch in chunks(products, batch_size):
        tasks = [classify_single(p) for p in batch]
        batch_results = await asyncio.gather(*tasks)
        results.extend(batch_results)
    return results

Phase 4: Evaluation and Refinement (12/20–12/26)

The final 6 commits focused on measuring classification accuracy and making final adjustments.

• Accuracy metrics: Exact Match, Level-N Match (evaluated at each category depth)
• Error analysis: analyzing misclassification patterns and correcting prompts accordingly

Technical Insights

1. Category tree compression: a tree compression technique that efficiently fits thousands of categories into the LLM context
2. Cost optimization: using GPT-3.5 wherever feasible instead of GPT-4, and applying caching
3. Consistency: setting temperature=0 to ensure the same product always maps to the same category

Retrospective

This 48-commit project gave us a much deeper understanding of LLMs' classification capabilities and the importance of prompt engineering. In particular, hierarchical classification and the embedding-hybrid approach have since become recurring patterns we reach for in later projects.