CrossEncoder Reranker: Boosting Search Accuracy with Two-Stage Retrieval

Why Reranking Is Necessary

flowchart LR
    A[사용자 쿼리] --> B[Bi-Encoder]
    B -->|Top-100 후보| C[Cross-Encoder Reranker]
    C -->|Top-10 정제| D[최종 결과]

    style B fill:#e1f5fe
    style C fill:#fff3e0

Bi-Encoders (SentenceTransformer) are fast but have accuracy limitations. Cross-Encoders are slow but accurate. Reranking is the pattern that combines the strengths of both.

Cross-Encoder Implementation

The 1/13 commit Add initial implementation of CrossEncoder model and training utilities introduced the following:

from torch import nn
from transformers import AutoModel, AutoTokenizer

class CrossEncoder(nn.Module):
    def __init__(self, model_name: str):
        super().__init__()
        self.bert = AutoModel.from_pretrained(model_name)
        self.classifier = nn.Linear(768, 1)  # 유사도 점수

    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids, attention_mask=attention_mask)
        # [CLS] 토큰의 출력 사용
        cls_output = outputs.last_hidden_state[:, 0, :]
        score = self.classifier(cls_output)
        return score.squeeze(-1)

Reranker Training

Between 1/13 and 1/15, we added the reranker training code and ran training:

# Reranker 학습 예시
def train_reranker(model, train_pairs, epochs=3):
    optimizer = StableAdamW(model.parameters(), lr=2e-5)
    loss_fn = nn.BCEWithLogitsLoss()

    for epoch in range(epochs):
        for query, doc, label in train_pairs:
            inputs = tokenizer(query, doc, return_tensors='pt',
                             truncation=True, max_length=512)
            score = model(**inputs)
            loss = loss_fn(score, label)
            loss.backward()
            optimizer.step()

Bi-Encoder + Cross-Encoder Pipeline

sequenceDiagram
    participant U as 사용자
    participant BE as Bi-Encoder
    participant VS as Vector Store
    participant CE as Cross-Encoder

    U->>BE: 쿼리 인코딩
    BE->>VS: 벡터 유사도 검색
    VS-->>CE: Top-100 후보 문서
    CE->>CE: 정밀 점수 계산
    CE-->>U: Top-10 최종 결과

Results

After adding the reranker:

• MRR@10: 0.62 → 0.74 (+19%)
• nDCG@10: 0.58 → 0.71 (+22%)

The 1/21 fix_all commit cleaned up the full pipeline, and the final fix on 1/23 wrapped up the project.