Deep Learning Compiler
System Architecture
Tensor Program Generation
Megakernel Compilation
Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel (MLSys 2026) [Paper] [arXiv]
CMU
Introduce Event Tensor, a unified compiler abstraction that represents tiled-task dependencies as first-class tensors for dynamic megakernels.
Support both symbolic-shape dynamism and data-dependent dynamism, then lower the abstraction through the Event Tensor Compiler (ETC) with static and dynamic scheduling transformations.
Achieve state-of-the-art LLM serving latency, including up to 1.40x speedup on fused GEMM and Reduce-Scatter kernels, up to 1.23x on MoE workloads, and up to 3.5x lower engine warmup overhead.
Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programs (arXiv:2512.22219) [arXiv] [Code] [Homepage]
CMU
Introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance megakernel.
Use an SM-level graph representation and an in-kernel parallel runtime with decentralized scheduling to enable cross-operator software pipelining and fine-grained kernel overlap.
Reduce end-to-end LLM inference latency by up to 1.7x over kernel-per-operator serving systems while preserving flexible tensor-programming workflows.
General Tensor Program Generation
DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs (ICML 2026) [arXiv] [Code]
ByteDance Seed & PKU & THU & ZJU & SJTU
Introduce DITRON, a distributed tensor compiler with Core-, Device-, and Task-level tiling interfaces that map computation and communication onto the memory hierarchy of GPU clusters.
Reorder tiles through distributed swizzling to overlap computation with communication, and use task dependencies and scheduling to fuse registered Triton kernels into megakernels.
Support NVIDIA and AMD GPUs through hardware-specific SHMEM backends; report 6-30% speedups on isolated kernels and 5-30% gains in end-to-end vLLM inference over expert-tuned CUDA baselines.
AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures (ASPLOS 2022) [Paper]
Alibaba
Memory-intensive operators.
Acronyms
DSA: Domain-Specific Architecture
Last updated