I am an M.S. student at the University of Electronic Science and Technology of China (UESTC). My background spans systems programming and applied AI, with hands-on experience in C/C++, Python, Linux, deep learning, and LLM inference. I am currently focusing on LLM inference runtimes, model serving and scheduling, GPU kernels, and edge deployment.
My current research focuses on collaborative inference and scheduling optimization for small language models of different scales in resource-constrained edge environments. I am particularly interested in system-level trade-offs among inference latency, resource utilization, and service quality.
| Project | Description |
|---|---|
| LLM-Serve | A single-GPU LLM inference runtime exploring Paged KV Cache, continuous batching, chunked prefill, serving benchmarks, and EAGLE-style speculative decoding. |
| HunyuanOCR-ncnn | A pure C++17 runtime for deploying HunyuanOCR 1.5 with pnnx and ncnn, with optional Vulkan acceleration for the vision encoder. |
| llm-infer-kernels | CUDA and Triton implementations of common LLM inference operators with correctness tests, benchmarks, and optimization notes. |
| riscv-memcpy-memset-opt | RV64GC memcpy and memset optimizations using scalar unrolling, RVV 1.0, and Zicboz. |
| Community | Contributions |
|---|---|
 Mooncake |
Contributed allocator benchmarking and optimization work, including a merged size-class allocator fragmentation benchmark and an ongoing size-class-aware allocation strategy. |
 ncnn |
Develop and maintain HunyuanOCR-ncnn, a community implementation for deploying Tencent HunyuanOCR 1.5 through the pnnx/ncnn toolchain. |