HormAS: Generative AI-Based Hotel Review Sentiment Analysis and Auto-Response System
Project Overview
In the hotel industry, online review management has a direct impact on revenue. However, reading, analyzing, and drafting responses for hundreds of reviews one by one is not feasible in practice. HormAS (Hotel Review Management Assistant System) leverages GPT-3.5 to automatically analyze the sentiment, emotion, and intent behind hotel reviews, and generates appropriate responses.
Technical Architecture
- Frontend: Admin dashboard built on Next.js 15
- Backend: Django 5.0.1 + LangChain 0.1.3 + GPT-4
- DB: PostgreSQL (AWS RDS)
- Deployment: Docker + Kubernetes
Core Feature: Comparative Study of Prompt Engineering Techniques
The academic contribution of this project lies in a comparative study of prompt engineering methodologies. We applied three prompting techniques to the same review analysis task and benchmarked their performance against each other.
1. Chain of Thought (CoT)
Explicitly requires the model to reason step by step:
리뷰를 분석해주세요. 다음 단계를 따라주세요:
1단계: 리뷰에서 핵심 키워드와 표현을 추출합니다.
2단계: 추출된 표현의 감성(긍정/부정/중립)을 판단합니다.
3단계: 고객의 감정(만족, 불만, 실망 등)을 파악합니다.
4단계: 고객의 의도(재방문, 보상 요구, 정보 공유 등)를 분석합니다.
5단계: 종합 판단을 내립니다.
2. Zero-shot CoT
A minimal approach that simply appends "Let's think step by step":
다음 호텔 리뷰의 감성, 감정, 의도를 분석해주세요.
단계별로 생각해봅시다.
3. Function Calling
Uses GPT-4's Function Calling feature to guarantee structured JSON output:
functions = [{
"name": "analyze_review",
"parameters": {
"type": "object",
"properties": {
"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]},
"emotion": {"type": "string"},
"intention": {"type": "string"},
"key_phrases": {"type": "array", "items": {"type": "string"}},
"confidence": {"type": "number"}
}
}
}]
Embedding Similarity Analysis
We converted the outputs of all three prompting methods into embedding vectors and computed cosine similarity scores. This allowed us to quantitatively compare how consistently each method produces its results.
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
# Convert each method's analysis output to an embedding
cot_embedding = get_embedding(cot_result)
zero_shot_embedding = get_embedding(zero_shot_result)
func_calling_embedding = get_embedding(func_calling_result)
# Compute pairwise similarity
similarity_matrix = cosine_similarity([
cot_embedding, zero_shot_embedding, func_calling_embedding
])
Findings
- CoT: Produces the most detailed analysis, but output format is inconsistent
- Zero-shot CoT: Analysis quality is comparable to CoT, but more concise
- Function Calling: Delivers the most consistent results via structured output, making downstream post-processing straightforward
Overall, Function Calling proved to be the most practical choice for a production environment.
Automatic Response Generation
Based on the analysis results, the system automatically generates a response to each review. A response_prompt_selector picks the appropriate system prompt tone depending on the review's sentiment and detected emotion.
- Positive reviews: Express gratitude + encourage a return visit
- Negative reviews: Apologize + commit to improvements + offer compensation
- Neutral reviews: Thank for the feedback + provide additional information
Retrospective
Looking back, this was research conducted at a time when agent-based AI was just beginning to take shape. Prompt engineering papers — CoT, Zero-shot CoT, and the like — were flooding the field, and LangChain had just made its debut. It genuinely felt like stepping into uncharted territory.
These things are commonplace now, but at the time there was something exciting about being at the edge of a frontier. That feeling made the work fun.