Documents

Self-Reasoning Framework for Better RAG with LLMs

9 min readAug 5, 2024Feb 21, 2026

[LLM] Self-Reasoning Framework

Baidu published a paper on arXiv titled "Improving Retrieval Augmented Language Model with Self-Reasoning."

RAG (Retrieval-Augmented Generation), introduced to improve LLM performance, is a process that optimizes LLM outputs by referencing a trusted knowledge base outside the training data before generating a response. During this process, if the knowledge base lacks information relevant to the given query, or if incorrect knowledge is retrieved, hallucinations are likely to occur during response generation.

This paper aims to reduce such hallucinations by leveraging reasoning paths generated by the LLM itself.

These "LLM-generated reasoning paths" consist of three stages: 1) a relevance-aware process, 2) an evidence-aware selective process, and 3) a trajectory analysis process. Each is described in detail below with reference to the figure.

1) Relevance-Aware Process
DPR and Contriever are selected as the base retriever R to retrieve the top-k relevant documents. The LLM is then instructed to describe the relevance of each of these k documents to the query and explain its reasoning.

2) Evidence-Aware Selective Process
The LLM selects the documents it deemed relevant in the previous step and is instructed to automatically identify key sentence snippets from those selected documents. Again, the LLM is asked to output the reasoning for why each selected snippet can answer the query.

3) Trajectory Analysis Process
Finally, all self-reasoning trajectories from the previous processes are aggregated to form a chain of reasoning snippets, improving the overall performance of retrieval-augmented generation. Specifically, the LLM is asked to analyze the reasoning trajectories internally and ultimately output a concise analysis along with a short answer.

A simple implementation of this framework is shown below.

from transformers import DPRQuestionEncoder, DPRContextEncoder
from sentence_transformers import SentenceTransformer
import torch
import faiss
from langchain_huggingface import HuggingFaceEndpoint,

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-27b-it")
model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-27b-it",
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Load DPR model
question_encoder = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base")
context_encoder = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base")

# Load Contriever
contriever = SentenceTransformer('facebook/contriever')

# Document database (example)
documents = [
    "DPR은 Dense Passage Retrieval의 약자입니다.",
    "Contriever는 대조 학습을 사용하는 검색 모델입니다.",
    "검색 시스템은 정보 접근성을 향상시킵니다."
]

dpr_doc_embeddings = context_encoder(documents, return_tensors="pt").pooler_output
contriever_doc_embeddings = contriever.encode(documents, convert_to_tensor=True)

# Build FAISS index
dimension = dpr_doc_embeddings.shape[1]
index = faiss.IndexFlatIP(dimension)
index.add(dpr_doc_embeddings.cpu().numpy())

# The paper states that both DPR and Contriever are used, but does not specify the exact
# combination method, so scores are combined for a unified ranking.

def hybrid_retrieve(query, top_k=5):
    # DPR retrieval
    q_embedding_dpr = question_encoder(query, return_tensors="pt").pooler_output
    dpr_scores, dpr_indices = index.search(q_embedding_dpr.cpu().numpy(), top_k)
    
    # Contriever retrieval
    q_embedding_contriever = contriever.encode(query, convert_to_tensor=True)
    contriever_scores = torch.matmul(q_embedding_contriever, contriever_doc_embeddings.T)
    
    # Normalize scores
    dpr_scores = (dpr_scores - dpr_scores.min()) / (dpr_scores.max() - dpr_scores.min())
    contriever_scores = (contriever_scores - contriever_scores.min()) / (contriever_scores.max() - contriever_scores.min())
    
    # Combine scores (e.g., average)
    combined_scores = (dpr_scores + contriever_scores.numpy()) / 2
    
    # Determine final ranking
    final_indices = combined_scores.argsort()[::-1][:top_k]
    
    return [documents[i] for i in final_indices]

def format_retrieved_docs(docs):
    formatted_docs = []
    for i, doc in enumerate(docs, start=1):
        formatted_docs.append(f"문장{i}: {doc}")
    return "\n".join(formatted_docs)
    
qeury = '검색시스템은 무엇인가요?'

task_1 = f'''query: {query} 
		    documents : {format_retrieved_docs(hybrid_retrieve(query))}
            다음 query와 documents를 보고, query와 연관성이 있는 것으로 보이는 document를 고르고 
            그 이유에 대해서 서술하시오.
'''

input_ids = tokenizer(task_1, return_tensors="pt").to("cuda")

with torch.no_grad():
	outputs = model.generate(**input_ids)
output = tokenizer.decode(outputs[0])

task_2 = f'''query: {query} 
		    output : {output}
            query와 output을 보고 ouput의 연관성 있는 문장으로 부터 
			query의 답변을 하는데 유의미한 문장을 추출하고 그 이유를 설명하라
'''

input_ids = tokenizer(task_2, return_tensors="pt").to("cuda")

with torch.no_grad():
	outputs = model.generate(**input_ids)
output_2 = tokenizer.decode(outputs[0])

task_2 = f'''query: {query} 
		    추론 과정 1: {output}
            추론 과정 2: {output2}
            위의 추론 과정을 보고 query에 대하여 적절한 응답을 하시오 
'''

input_ids = tokenizer(task_2, return_tensors="pt").to("cuda")

with torch.no_grad():
	outputs = model.generate(**input_ids)
output3 = tokenizer.decode(outputs[0])
Tags
LLM