LLM Inference Optimization
High-performance LLM serving built on vLLM, covering KV Cache management, Speculative Decoding, Continuous Batching, and quantized deployment for low-latency online inference at scale.
Kyrie Chen
Hi, that's Kyrie
I write about machine learning, LLM systems, AI infrastructure, and the cities and journeys that stay with me after the trip is over.
Engineer, researcher, traveler, husband, and father. This site is where technical notes and personal essays live together.
About
Where scaling laws meet the open road
My technical writing traces every step of learning and building through school and industry; my travel and life essays capture the places and moments worth revisiting over time.
Here you'll find deep dives into LLMs, inference systems, training strategies, and production engineering — alongside travel stories about cities, coastlines, alleyways, and the everyday moments worth remembering.
- Machine Learning / Deep Learning
- LLM / AI Infra / NLP / CV
- Travel writing / long-form notes
Tech Stack
Skills & Tools
Python
C++
C
Java
PyTorch
Transformers
DeepSpeed
Megatron-LM
CUDA
MoE
vLLM
KV Cache
TensorRT
Triton
Docker
Kubernetes
LangChain
RAG
MCP
Memory System
Prompt Engineering
What I Work On
View all on GitHub →
Focus Areas
Distributed Training & Model Architecture
Large-scale distributed training infrastructure and model architecture research, including 5D Parallelism (DP/TP/PP/SP/EP), MoE sparse architectures, Scaling Law validation, and training stability tuning.
Agent & Memory System
Design and implementation of LLM Agent systems, including hierarchical Memory management for multi-turn conversations, Tool-use orchestration, MCP protocol integration, and RAG pipelines.
LLM Serving Infrastructure
Production-grade LLM serving platform covering model deployment orchestration, GPU cluster scheduling, Triton inference gateway, and end-to-end observability.
Blogs
View archive →
Selected writing
2026-08-13
大模型是怎么炼成的（一）：训练全景与数据工程
一个大模型从原始网页、代码仓库和论文，变成能对话、写代码、做数学题乃至调用工具的系统，中间到底经历了什么？2022 年，人们习惯用「Pretrain + RLHF」概括它；今天，更准确的描述是一组相互回流的模块：数据工程 → Pretrain → Mid-training → SFT → 偏好...
2026-07-01
FlashAttention：从 IO 感知到软硬件协同流水线
主流 LLM 训练与推理栈通常都会在硬件、dtype、head dimension 和 mask 形态满足条件时，调度到 FlashAttention 或同类 memory-efficient attention kernel；它不是任何输入上都无条件启用的单一开关。如果你需要训练超长上下文、...
2026-06-30
并行策略到底在切什么：训练 OOM 与 ZeRO/DP/TP/SP 的边界
在训练大模型时，OOM 并不总是同一种问题。最常见的误解是：只要把 ZeRO-3 开起来、把 DP 卡数加上去，或者多加几台机器，显存问题就应该线性缓解。但实际训练里经常会遇到另一类 OOM：某个 step 内部临时产生了一个巨大的局部张量，例如长序列下的 attention scores、a...
2026-05-12
Do Your Own Harness Engineering
Intro
Harness Engineering 不是一套有严格边界的标准。2026 年 2 月，HashiCorp 联合创始人 Mitchell Hashimoto 在《My AI Adoption Journey》中用 Engineer the Harness 概括了一类实践：让 ag...
2026-05-05
锦绣闽中，入梦三城
今年五一，想着能带小朋友去自驾可达的玩沙滩的地方，但长三角沿线要么去过，要么实在是海滩拿不出手，就想着要么干脆开远一些直达福建。于是就有了今年这趟福州-泉州-平潭的行程。
从我们家出发到福州单程的驾驶时间不算休息大概是 8.5 个小时。印象中从加州一号公路和大西北德令哈到敦煌线之后，好像单日...
2026-04-01
Claude Code 如何工作
2026 年 3 月 31 日，Anthropic 发布 @anthropic-ai/claude-code v2.1.88 时出了一个低级失误：Bun 默认生成的 59.8 MB .js.map source map 文件没有被 .npmignore 排除。几小时内，约 1,900 个 Ty...