> For the complete documentation index, see [llms.txt](https://paper.lingyunyang.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://paper.lingyunyang.com/reading-notes/conference/sosp-2026.md).

# SOSP 2026

## Meta Info

Homepage: <https://sigops.org/s/conferences/sosp/2026/>

Paper list: <https://sigops.org/s/conferences/sosp/2026/accepted.html>

### Acceptance Rate

15.9% (= 62 / 390)

## Papers

### Large Language Models (LLMs)

* LLM Training
  * Training Emulation
    * A Few GPUs, A Whole Lotta Scale: Faithful LLM Training Emulation with CrystalLLM
      * Alibaba & SJTU & Harvard & ZJU
  * RL Post-Training
    * Scaling Weight Transfer in LLM RL Training with Reference-Oriented Storage
      * UW-Madison & ByteDance Seed
* LLM Inference
  * Deterministic Inference
    * Axiom: Achieving Determinism in LLM Inference
      * MSR & UW
  * Offline Inference
    * Batched in Back: Characterizing and Optimizing Offline LLM Inference in Production with ACDC
      * SJTU & Alibaba
  * MoE Inference
    * Democratizing MoE LLM Decoding via Barrier-Free Expert Parallelism
      * USC & Google DeepMind & UT Austin
  * Multi-LLM Serving
    * Multi-LLM Serving at Production Scale \[[Code](https://github.com/Janus2026/Janus)]
      * PKU & JD & UCAS & USTC & SJTU
  * Workload Characterization
    * LMTracer: Fine-Grained and Real-Time Performance Profiling for Production LLM Systems
      * SCITIX & THU
  * End-to-End Serving
    * Taming Inference Workloads at Global Scale: Foundation Model Serving in SystemX
      * AWS
* LLM Applications
  * Agentic Systems
    * Skill VM: Write Once, Run Everywhere Efficiently \[[arXiv](https://arxiv.org/abs/2604.03088)] \[[Code](https://github.com/SJTU-IPADS/SkVM/)] \[[Homepage](https://skillvm.ai/)]
      * SJTU IPADS
      * Introduce **SkVM**, treating agent skills as programs and LLMs plus harnesses as heterogeneous processors to improve skill portability across models and agent runtimes.
      * Combine AOT compilation based on model-harness capability profiling with runtime JIT optimization, including code solidification and feedback-driven skill rewriting.
      * Evaluate across eight LLMs and three agent harnesses; improve task completion by about 10%, reduce token cost by up to 40%, and cut end-to-end latency by 19-50x through JIT-boost code solidification.
  * Agent Data Infrastructure
    * Supporting Agents on Streaming Data via a Forkable Shared Log
      * UIUC
  * Voice Applications
    * Scalable Context Orchestration for Serving LLMs Over Voice
      * SJTU & AgenticSys Labs

### Diffusion Models

* Diffusion Model Serving
  * DiFlow: A System for Micro-Serving Text-to-Image Diffusion Workflows \[[arXiv](https://arxiv.org/abs/2604.08123)]
    * HKUST & Alibaba
    * **Our work!**
    * Decomposes text-to-image diffusion workflows into independently managed model-execution nodes.
    * Enables per-model scaling, cross-workflow model sharing, and adaptive model parallelism for bursty serving workloads.

### Model Serving

* Multi-Model Serving
  * SANDHI: Fine-grained Merging for Memory Efficient Multi-Model Serving
    * GaTech & Microsoft
* Trusted Inference
  * Cardamom: A Multi-GPU Framework for TB-Scale Encrypted Inference
    * CMU & UT Austin & NVIDIA

### Deep Learning Compilation

* Agent-Based Kernel Generation
  * Guiding Agentic GPU Kernel Optimization with Data Flow Invariants \[[arXiv](https://arxiv.org/abs/2604.18616)]
    * HKUST & UCAS & UC Riverside & USTC & XJTU & THU & Stanford
* GPU Kernels and Compilers
  * Taming Dynamism on GPUs: Cross-SM Kernel Fusion via SM Cooperation and Just-in-Time Reduction
    * SJTU IPADS
    * Introduce **MorphKernel**, targeting dynamic and imbalanced GPU operators such as decoding attention and MoE expert activation.
    * Fuse multi-phase kernels across SMs by decomposing coarse phase barriers into sub-task dependencies with fine-grained SM synchronization and on-chip data exchange, reducing HBM traffic for intermediate results.
    * Provide a declarative programming abstraction over operators such as map, iter, and reduce, while the runtime dynamically decides scheduling, communication, and parallel partitioning from input shapes.
    * Use TMA-based send/recv primitives, GPC-topology-aware communication, and per-SM instruction queues; improve performance by 1.3x on average over real workloads.
* Kernel Verification
  * M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification
    * Penn State & ICT, CAS & PKU
* Wafer-Scale Compilation and Runtime
  * MeshRT: Compile-Time Governed Wafer-Scale Runtime for Low-Latency High-Throughput Inference
    * Edinburgh & MSR
  * Wavel: A Fast and Efficient Compilation System for Wafer-Scale Accelerators
    * Edinburgh & MSR

### Resource Management

* GPU Scheduling and Sharing
  * Beyond Utilization: Energy-Conscious GPU Sharing for Inference Serving
    * UT Austin
  * Efficient GPU Multitasking with Morphable Kernels
    * THU
  * Linux AGX: An Adaptive GPU eXtension to Linux Fair Scheduling for Physical AI and Robotic Systems
    * UC Riverside

### Model Storage and Data Pipeline

* Model Storage
  * TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models
    * UVA & Harvard

### ML for Systems

* System Optimization
  * TuxBot: Semantic-Aware Online OS Tuning with LLMs
    * Columbia & IBM Research