The paper optimizes LLM inference systems through queueing theory.
problem Efficient LLM inference for AI agents under various routing topologies.
method Developed a fluid-limit framework for multi-class batched processing networks under K-FCFS scheduling.
result Proved that work-conserving scheduling algorithms maximize throughput for LLM inference.