> For the complete documentation index, see [llms.txt](https://paper.lingyunyang.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://paper.lingyunyang.com/reading-notes/conference/atc-2026.md).

# ATC 2026

## Meta Info

Homepage: <https://atc.sigops.org/2026/>

### Paper List

* Accepted Papers: <https://atc.sigops.org/2026/accepted.html>
* Program: <https://atc.sigops.org/2026/schedule.html>
* Proceedings: <https://dl.acm.org/doi/proceedings/10.1145/3828523>

### Acceptance Rate

13.9% (= 135 / 973)

## Papers

### Large Language Models (LLMs)

* LLM Training
  * Memory and Data Pipeline
    * HeadFlow: Scaling Long-Context Training with Head-Group Activation Residency \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857140)]
      * SJTU & ZJU
      * Plans attention activation placement per head group across GPU memory, pinned host memory, and SSD, with recomputation as another option. Overlaps activation offload and prefetch with attention computation.
      * Supports full-parameter training of a 24B model at a one-million-token context on eight 96 GB H20 GPUs.
  * Scheduling and Parallelism
    * Efficient Training on Multiple Consumer GPUs with RoundPipe \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857063)] \[[arXiv](https://arxiv.org/abs/2604.27085)] \[[Code](https://github.com/thustorage/RoundPipe)] \[[Artifact](https://zenodo.org/records/23220089)]
      * THU
      * Dispatches pipeline stages round-robin across GPUs rather than pinning each stage to one device, reducing pipeline bubbles caused by uneven stages. Schedules weight transfers by priority and enforces consistency at layer granularity.
      * Runs low-rank adaptation (LoRA) of Qwen3-235B at a 31K sequence length on one server with eight RTX 4090 GPUs.
  * RL Post-Training
    * PrefixFlow: Training-Time KV Caching for Schedule-Level Prefix Reuse in LLM RL Training \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857100)] \[[arXiv](https://arxiv.org/abs/2606.01143)]
      * HKUST & Tencent & SUFE & HKU
      * Runs one forward pass over a shared rollout prefix, trains suffix microbatches against the cached keys and values, then runs one prefix backward pass using their accumulated gradients.
    * FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857118)] \[[arXiv](https://arxiv.org/abs/2602.09578)]
      * JD.com & Huawei & HUST
      * Uses a multi-agent execution graph to sample independent rollouts in parallel, overlap rollout and training across agents, and allocate training resources only to active agents.
  * Fault Tolerance and Rescheduling
    * MoECatch: Efficient Checkpointing and Consistent Recovery for Mixture-of-Experts Models \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857109)]
      * SEU & CUHK & Capital Normal University & Defense Innovation Institute & CityU
      * Prioritizes overdue experts under a fixed checkpoint budget, then ranks the rest by routed-token counts. During recovery, replays missed updates to bring stale experts up to the target version.
      * Checkpoints up to 2.5× as often and cuts time lost to failures by up to 22.86% in the reported experiments.
* LLM Inference
  * Workload Characterization
    * Model-as-a-Service in the Wild: Characterizing Production LLM Inference Clusters at Scale \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857044)] \[[Trace](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-v2026-maas)]
      * CUHK-SZ & Alibaba & SJTU
      * **Our work!**
      * **Best Paper Award**
      * Analyzes six months of online and offline inference workloads across more than 100,000 GPUs, finding GPU memory overprovisioning, cold models that stay resident, and an imbalance between prefill and decode resources.
  * Request Scheduling
    * CacheSched: Efficient LLM Request Scheduling for Hierarchical Prefix Caching \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857048)] \[[PDF](https://kyleshu.github.io/assets/pdf/atc26_CacheSched_paper.pdf)] \[[Code](https://github.com/LLMServe/CacheSched-artifact)] \[[Artifact](https://zenodo.org/records/23205571)]
      * PKU & UCLA & ZTE
      * Routes requests using predicted prefill compute and cache transfer costs, accounting for both prefix locality and worker load. Each worker admits host-memory prefix loads that fit within the active batch's compute window.
    * Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857087)] \[[arXiv](https://arxiv.org/abs/2605.06046)]
      * IIT Bombay
      * Uses reinforcement learning to balance larger decode batches against greater prefix sharing. A chunked hash tree identifies shared prefixes and selects matching requests.
    * Axon: Asynchronous Global KV-Aware Scheduling Across Distributed Instances for Agentic Workloads \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857095)]
      * UMacau & Great Bay University & Alibaba
      * Pools KV cache across serving instances and sends queries to the workers holding the relevant context shards. Schedules by cache locality and worker load, and restores missing fragments in parallel with GPU execution.
  * KV Cache Management
    * DittoCache: From CPU-Orchestrated to GPU-Centric KVCache for Efficient LLM Decoding \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857042)] \[[Code](https://github.com/USTC-MLSys/DittoCache)]
      * USTC & Institute of Artificial Intelligence, Hefei Comprehensive National Science Center & Huawei
      * Uses hash-based retrieval with a host-resident KV cache, then prefetches selected entries by layer. Uses CUDA graph replay to reduce CPU orchestration during decoding.
    * Octopus: Unlocking SSD Parallelism for Sparse KVCache Retrieval \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857061)] \[[arXiv](https://arxiv.org/abs/2603.17803)]
      * THU
      * Groups KV entries that are frequently accessed together and places clusters across SSDs with selective replication. Balances reads across drives to parallelize sparse KV retrieval.
    * DAG-KV: DAG-Guided KV Propagation for Structure-Aware Knowledge Reuse \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857074)]
      * SDU & Huawei
      * Caches each knowledge unit as a reusable KV token. Builds a query-specific directed acyclic graph (DAG) over retrieved units and propagates attention along its edges to recover dependencies for multi-hop reasoning without re-encoding the joint context.
    * KVLite: Efficient LLM Agent Scheduling via Head-Aware KV Sharing \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857081)]
      * NUDT
      * Identifies which attention heads are sensitive to context changes and recomputes KV only for those heads when agents reuse shared text. Uses the shared cache to pipeline work along the workflow's critical path.
  * Position-Independent Caching (PIC)
    * HYPIC: Position-Independent Caching for Hybrid-Attention LLM Serving \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857127)] \[[arXiv](https://arxiv.org/abs/2607.01299)]
      * Xiaohongshu & PKU & SJTU
      * Composes cached recurrent states for linear-attention layers using per-segment transition operators, and recomputes tokens near segment boundaries for full-attention layers. Parallelizes prefill of uncached segments across instances.
  * Prefill-Decode (PD) Disaggregation
    * ATP: Workload-Adaptive Scheduling for Goodput-Optimized Tensor-Parallel LLM Serving with PD Disaggregation \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857038)]
      * USTC & NUS & Anhui University & Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
      * Routes each request's prefill and decode to instances with suitable tensor-parallel degrees. Reassigns instances between the two pools as the workload mix changes.
      * On Mooncake traces, sustains 2.07× DistServe's goodput on average under the paper's latency targets.
    * ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857077)] \[[arXiv](https://arxiv.org/abs/2607.00466)] \[[Code](https://github.com/casys-kaist/ELDR)]
      * KAIST & MSR & Shanghai Xingyunzhili Artificial Intelligence Institute
      * Builds signatures from prefill expert activations, then sends each request to the least loaded decode worker among close signature matches. Grouping requests that use the same experts reduces weight reads during decode.
  * Serverless Inference
    * CacheFS: Peer-to-Peer Cooperative Caching for LLM Weight Distribution at Cloud Scale \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857016)] \[[Artifact](https://zenodo.org/records/23167350)]
      * ZJU & Alibaba
      * Exposes model weights as files and lets workers fill their caches from the backing store or peers. Supports broadcasting weights to preload workers.
    * gKSM: GPU Memory Deduplication at Driver Layer for Serverless LLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857017)] \[[arXiv](https://arxiv.org/abs/2610.25219)]
      * USTC & Minjiang University & CUHK
      * Deduplicates model weights across colocated inference instances. Uses GPU-side hashing and virtual-memory remapping to share identical pages while isolating writes.
    * Energy-Aware Scheduling for Serverless LLM Serving on Shared GPUs \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857091)] \[[arXiv](https://arxiv.org/abs/2606.30391)]
      * Pitt & HPE Labs
      * **Festina** coordinates request placement, GPU compute partitioning, and device power settings to cut cluster energy use under latency targets. Migrates workloads to consolidate demand onto fewer GPUs.
    * IgniteServe: Accelerating LLM Serving Engine Cold Starts via Fine-Grained Parallelization \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857102)]
      * HUST & UT Arlington
      * Represents engine initialization as a dependency graph. Overlaps independent tasks, parallelizes and reuses subgraph compilation, and builds CUDA graphs on demand.
  * LoRA Serving
    * InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857067)] \[[arXiv](https://arxiv.org/abs/2604.07173)]
      * SJTU & ByteDance & SMU & HKUST-GZ & NUS
      * Runs low-rank adapters on a shared server separate from base model inference. Sizes adapter capacity against latency targets and reduces the cost of remote adapter execution with GPU-initiated communication and specialized kernels.
  * Attention Mechanisms
    * SptcAttn: Accelerating Long-Context Attention with Diagonal Stripes using Sparse Tensor Core \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857028)]
      * SJTU & Enflame
      * Maps diagonal stripes of attention scores to the 2:4 pattern supported by sparse Tensor Cores, eliminating work within a tile while retaining the selected scores.
    * Mosaic: Efficient CPU–GPU Coordinated Block-Sparse Attention for Long-Context Decoding \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857033)] \[[Artifact](https://zenodo.org/records/23201742)]
      * Beijing University of Technology & MSR & Beijing Jiaotong University
      * Chooses KV budgets and block granularities per attention head, then schedules block-sparse attention from a shared queue to overlap CPU and GPU execution.
    * JetFire: Approximate-then-Correct Attention on Hybrid Memory for Long-Context LLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857090)] \[[Code](https://github.com/Xtra-Computing/JetFire)]
      * NUS & HUST & ETH Zürich
      * Computes approximate attention from a low-bit KV cache on the GPU, then corrects the contribution of the top-ranked tokens on the CPU using higher-precision KV entries.
  * Compression
    * TensorBridge: Bridging Microscaled FP4 Weights to FP8 Tensor Cores for LLM Serving \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857032)] \[[Code](https://github.com/revollllt/atc26-tensorbridge)] \[[Artifact](https://zenodo.org/records/23045885)]
      * HKUST-GZ
      * Keeps weights in NVIDIA's 4-bit floating-point format (NVFP4), then reconstructs them in the 8-bit format (FP8) inside a Hopper matrix multiplication kernel without a 16-bit intermediate.
    * HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857108)] \[[arXiv](https://arxiv.org/abs/2502.03589)] \[[Artifact](https://zenodo.org/records/22982680)]
      * UVA & VMware Research by Broadcom & UCL & Harvard
      * Reduces the bytes sent between prefill and decode by quantizing the KV cache and evaluating attention over the compressed representation.
  * Speculative Decoding
    * BMV: Breaking Monolithic Verification for Fast Speculative LLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857099)]
      * HUST & UNSW Sydney
      * Prunes low-confidence draft branches at intermediate target-model layers and reuses the computed state for the remaining branches, preserving the target model's output distribution through exact final verification.
  * Offloading
    * MoE-Lego: A Plane-Grained System for Dynamic-Precision MoE Offloading \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857064)]
      * SJTU
      * Stores each expert once as nested bitplanes and transfers only the planes needed when its precision changes. Groups routed work by target precision so GPU kernels can process multiple experts together.
  * Heterogeneous Deployment
    * Chorus: GPU–CPU Cooperative LLM Serving for Length-Skewed Decode \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857065)] \[[PDF](https://www.cse.ust.hk/~weiwa/papers/chorus-atc26.pdf)]
      * HKUST & Nokia Bell Labs & Alibaba
      * Keeps prefill and short requests on GPUs and moves selected long decode requests to CPU workers with Intel Advanced Matrix Extensions (AMX). Reclaims GPU KV pages so the GPU workers can admit more short requests.
  * Multimodal Inference
    * FlashCodec: Unleash All Decoding Resources for Low-Latency Video Decoding in MLLM Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857057)]
      * Wuhan University
      * Splits video into independently decodable groups of pictures and schedules them asynchronously across GPU video decoders to reduce preprocessing latency for multimodal inference.
    * Rocks, Pebbles and Sand: Modality-aware Scheduling for Multimodal Large Language Model Inference \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857088)] \[[arXiv](https://arxiv.org/abs/2603.26498)] \[[Code](https://github.com/muse-research-lab/rps-serve)] \[[Artifact](https://zenodo.org/records/23134685)]
      * IMDEA Software Institute & Universidad Politécnica de Madrid
      * **RPS-Serve** estimates each request's GPU admission cost and groups requests by resource demand. Uses priority scheduling with aging so heavy image or video requests do not block smaller text requests.
  * Trusted Execution
    * RS-Infer: A Fast and Secure LLM Inference System for Mobile Devices with Recallable Resource Isolation \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857080)]
      * SJTU
      * Separates resource management from access rights so the normal-world OS can manage memory and the neural processing unit (NPU) while Arm TrustZone protects their use during inference. Coordinates model caching and execution across models in mobile agent workflows.
  * Retrieval-Augmented Generation (RAG)
    * TwinCache: A Retrieval- and Reranking-Aware Caching System for Efficient RAG Serving \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857039)]
      * BUPT
      * Builds a small candidate set from similar past queries and their reranking results, skipping database retrieval when the cache hit passes validation.
      * Reranks promising chunks first, overlaps remaining reranking with chunked prefill, and stops once the leading evidence stabilizes.
  * LLM-based Applications
    * Scaling LLM Agent Tool Access: The MCP Gateway \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857007)] \[[arXiv](https://arxiv.org/abs/2607.15593)]
      * NJU & Alibaba Cloud & Fudan University & RMIT University & Politecnico di Milano & ZJU
      * Moves Model Context Protocol (MCP) adaptation, access control, and session-aware routing into a shared gateway. Retrieves relevant tools before presenting them to the model so each request can use a small subset of a large catalog.
    * DeltaBox: Millisecond-Scale Sandbox Checkpoint and Rollback for Stateful AI Agents \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857015)] \[[arXiv](https://arxiv.org/abs/2605.22781)] \[[Code](https://github.com/dongyunpeng-sjtu/deltabox)] \[[Project](https://dongyunpeng-sjtu.github.io/deltabox/)] \[[Artifact](https://zenodo.org/records/23202153)]
      * SJTU & Huawei
      * **Best Paper Award**
      * Records incremental filesystem and process-state changes to checkpoint and roll back an agent sandbox in milliseconds. Lets agents branch from intermediate states without copying the entire sandbox.
    * AgentMesh: Toward Redundancy- and Barrier-Free Dataflow in Multi-Agent Systems \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857023)]
      * PKU & Huawei
      * Reuses cached parts of tool results and fetches only new content. Prefills independent branches in parallel using the workflow's dependency graph, then streams chunks at a rate that overlaps production, transfer, and prefill.
      * Cuts end-to-end latency by up to 62% on deep-research and coding-agent workloads, with no statistically significant quality loss in the reported evaluation.
    * TraJudger: Structure-Guided Failure Attribution for LLM-Powered Multi-Agent Systems \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857043)] \[[PDF](https://netman.aiops.org/wp-content/uploads/2026/09/TraJudger.pdf)]
      * Nankai University & Alibaba & THU
      * Builds a dependency graph from agent trajectories, uses specialist judges to identify failure types, and traces dependencies backward to distinguish root causes from downstream failures.
    * Cloudless Tomorrow: A Memory-Centric System for Self-Improving Device-Cloud LLM Agents \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857092)]
      * HUST
      * **WorkflowPage** turns cloud fallback executions into reusable on-device workflow pages with a common execution prefix. Uses topology-aware prefetching and utility-based eviction within a bounded memory budget.
    * Make Graphs Great Again: Rethinking LLM Agent Architecture via Structure-Semantic Separation \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857104)]
      * HUST
      * **MotifAgent** compiles repeatable tool-call patterns into executable skills and invokes the LLM for decisions that require semantic judgment. Passes workflow structure to the serving engine for request scheduling and KV cache management.

### Distributed Training

* Communication
  * Towards High-Throughput Collective Communication via Harnessing PCIe and Dedicated Interconnects \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857136)] \[[Code](https://github.com/Networked-System-and-Security-Group/HyLink)]
    * BUAA
    * **HyLink** splits collective transfers across PCI Express and the dedicated accelerator interconnect. Adjusts the split at runtime as contention changes.

### Deep Learning Recommendation Models (DLRMs)

* Distributed Training
  * SOLO: Making Single-Local-Replica Storage Practical for Geo-Distributed Recommendation Model Training \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857035)]
    * USTC & ByteDance
    * Keeps one local copy of training data and uses the expected consumption order to repair unavailable files from another datacenter before they are needed. Places data by access patterns and shares preprocessed intermediates across jobs.
    * In ByteDance production, reduces storage cost per GPU by 45–55% while using less than 1% of inter-datacenter bandwidth.
  * EMDedup: Accelerating DLRM via In-Network EMbedding Layer Deduplication \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857124)] \[[Appendix](https://github.com/craftsvoice-eng/Appendix-of-EMDedup-Accelerating-DLRM-via-In-Network-EMbedding-Layer-Deduplication/blob/main/Appendix.pdf)]
    * ICT, CAS & Zhengzhou University & Huawei & SDU & UCAS
    * Deduplicates embedding lookup requests in the switch and multicasts shared responses to the requesting GPUs, reducing redundant traffic in embedding-layer all-to-all communication.

### Deep Learning Compilation

* Code Generation
  * A Novel AI-Native GPU Kernel Programming Language for Autonomous Kernel Tuning \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857041)]
    * Enflame & SJTU
    * Gives tuning agents independently editable primitives for structural choices such as warp specialization and pipeline staging. The compiler rejects infeasible schedules before GPU profiling and returns structured diagnostics for the next attempt.
* Tensor Program Optimization
  * Schedules Are Solvable Symbols: Tuning-Free Compilation of Tile Programs on Dataflow Architectures \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857138)] \[[arXiv](https://arxiv.org/abs/2609.29219)]
    * NUS
    * **Loom** derives legality and latency expressions from a hardware description and solves them to choose tile sizes, core mappings, and asynchronous pipelines without profiling each input shape.

### Resource Management

* Serverless Computing
  * HybridFaaS: Efficient GPU Serverless via Hybrid Autoscaling \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857036)]
    * HUST & UCSC
    * Scales inference by adjusting replicas, pipeline parallelism, and tensor parallelism. Groups GPUs with similar processing capacity to narrow the search for a placement that meets latency targets.
* GPU Clusters
  * Detecting and Localizing Network Faults in RoCE-Based AI Clouds under Restricted Telemetry \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857027)]
    * PKU & Huawei
    * **FACT** detects faults and localizes their causes using standard switch telemetry in RDMA over Converged Ethernet (RoCE) networks, including negative acknowledgments, packet counters, and throughput inferred from mirrored headers.
    * In Huawei's production deployments, localizes 99.95% of network faults within 10 minutes.
  * Workload Analysis of a Production BigData-Storage Cluster for ML and Data Analytics \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857020)]
    * THU & ByteDance
    * Combines cluster metrics and file metadata with per-disk traces from 60 machines in ByteDance's production storage system. Finds roughly log-uniform file sizes and local file popularity closer to lognormal than to Zipf–Mandelbrot.
    * Reconstructs representative traces from workload statistics to support storage evaluation.

### GPU Systems

* GPU I/O
  * The GPU Changes Everything: Rethinking GPU I/O Stack at Massive Scale \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857013)] \[[Code](https://github.com/ScaleXLab/ATC26-ShaperIO)] \[[Artifact](https://zenodo.org/records/23270957)]
    * HKUST-GZ
    * **ShaperIO** schedules GPU-initiated SSD requests on the device. Pipelines reads across hardware queues, merges writes, and adjusts write concurrency using throughput and tail-latency feedback.
  * GPipeANN: Taming Throughput-Latency Tradeoff in GPU-SSD ANNS \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857011)] \[[Code](https://github.com/ruc-system/quiver-ae)] \[[Artifact](https://zenodo.org/records/23074357)]
    * NJU & RUC & Tencent
    * Runs approximate nearest-neighbor search in a persistent GPU kernel that switches to a ready query while another waits on an SSD read. Tunes how many query contexts share a thread block to balance utilization against queueing delay.
* Checkpointing and Restore
  * FLASH: Fault-Led, Attested, Secure Hybrid-Key Checkpoint for Confidential GPU \[[Paper](https://dl.acm.org/doi/10.1145/3828523.3857076)] \[[Project](https://septlab.com/papers/flash.html)] \[[Artifact](https://zenodo.org/records/23199897)]
    * USC & Korea University & NVIDIA & Roblox & UC Riverside
    * Preserves ciphertext already produced by the GPU transfer engine and tracks which pages are dirtied while inference runs, limiting the pause to copying the final dirty pages.

## Acronyms

* KV: Key-Value
* LLM: Large Language Model
* LoRA: Low-Rank Adaptation
* MoE: Mixture of Experts
* PD: Prefill-Decode
* RL: Reinforcement Learning
