Contextifier HWP/HWPX Processor: Parsing Korean Documents

The Reality of HWP Document Processing

Building document-based AI systems for Korean enterprises requires HWP/HWPX support — yet almost no open-source libraries handle it properly.

HWP vs HWPX

graph LR
    subgraph HWP["HWP (구 버전)"]
        A[바이너리 포맷] --> B[OLE 컴파운드]
        B --> C[바이너리 파싱 필요]
    end
    subgraph HWPX["HWPX (신 버전)"]
        D[XML 기반] --> E[ZIP 압축]
        E --> F[XML 파싱으로 처리]
    end
    HWP --> G[Contextifier HwpProcessor]
    HWPX --> H[Contextifier HwpxProcessor]
    G --> I[통합 ChunkResult]
    H --> I

HWPX Processing Pipeline

HWPX is XML-based, which makes parsing relatively straightforward:

1. Decompress the ZIP archive
2. Parse XML to extract body text, headers, and footers
3. Extract tables and convert them to Markdown
4. Extract chart data
5. Extract images and run OCR

Refactoring Table/Chart Extraction to Inline

Early on, tables and body text were extracted separately, which broke context continuity. In v0.2.0, we refactored to an inline extraction approach so that tables are inserted at their original positions within the document.

# 리팩터링 전: 테이블 별도 추출
text_chunks = extract_text(hwpx)
table_chunks = extract_tables(hwpx)  # 위치 정보 없음

# 리팩터링 후: 인라인 추출
chunks = extract_with_inline_tables(hwpx)
# 테이블이 원래 위치에 Markdown으로 삽입됨

Chart Extractor

Both HWP and HWPX support chart data extraction. The underlying chart data is converted into a table format that LLMs can readily interpret.

Fixing Surrogate Character Issues

There was a bug where HWPX files containing Unicode surrogate characters caused errors when writing output files.

# fix: Handle surrogate characters when writing files
def safe_write(text: str, path: str):
    cleaned = text.encode('utf-8', errors='surrogatepass').decode('utf-8', errors='replace')
    with open(path, 'w', encoding='utf-8') as f:
        f.write(cleaned)

This fix was included in the 2026-01-27 commit.