AI Engineer Jang

Home About Project Book

Documents Graph Gallery

Documents

Loading documents in Inference.

AI Engineer Jang© 2026 HaRyeom Jang. All rights reserved.Archive

Home DocumentsAIInference

Inference

21 posts

1-10 of 21 documentsPage 1 / 3

GPU Memory Fragmentation in LLM Serving: Causes and Measurement

LLMInferenceGPUvLLM+5

Aug 18, 202614 min

Mixture of Experts vs Mixture of Tokens: Two Sparsity Strategies Compared

LLMInferenceGPUArchitecture+3

Aug 18, 20268 min

Why Chunked Prefill Exists: How Prefill Stalls Decode and the Fix

LLMInferencevLLMServing+2

Aug 18, 20268 min

KV Cache Quantization: Why It's Different from Weight Quantization

LLMInferenceGPUvLLM+1

Aug 17, 202612 min

Prefix Caching Deep Dive: Why TTFT Drops but Throughput Doesn't

LLMInferenceGPUvLLM+3

Aug 17, 202613 min

Why TTFT, TPOT, and p99 Latency Move Independently in LLM Serving

LLMInferencevLLMServing+2

Aug 16, 202610 min

Continuous vs Static Batching: Throughput Gains and When They Reverse

LLMGPUInferencevLLM+2

Aug 15, 202614 min

Mixture of Tokens vs MoE: Two Paths to Sparse Activation

LLMInferenceGPUArchitecture+4

Aug 15, 202610 min

LLM Serving Schedulers: Who Decides When Requests Hit the GPU

LLMInferenceservingGPU+2

Aug 15, 202613 min

Where Quantization Loses Accuracy: W4A16, W8A8, and FP8 Compared

LLMquantizationInferenceGPU+2

Aug 14, 202612 min

Inference — Documents | AI Engineer Jang