Tag: AI scheduling for large language model inference

An offline, uncertainty-aware reinforcement learning scheduler called URSA lifts SLO attainment in LLM inference serving to 90.4 percent across three ...