Archive
Latest writing
4 blogs
Inside an LLM Inference Engine: From Request to Response with MTLQwen
What is it like to build an inference engine from scratch, which beats llama.cpp on Qwen 3.5.
Writing a high performant GEMV + dequant for Metal
Notes on writing a high performant quant linear kernel for Metal (hitting 90% of Metal's MBU).
From Standard Attention to Linear Attention Reducing memory footprint
Notes on differences between Linear and Standard attention
LLAMA 1B Optimizing Kernels
Notes on optimizing kernels for Llama 1B