Same Cluster, 33 Points More Utilization: What Changed Was the Order

A new GPU allocator designed to optimize scheduling order significantly improved GPU utilization by up to 33 percentage points and increased priority-weighted output by up to 105% compared to a FIFO scheduler, without changing hardware. The allocator manages complex workloads including training, real-time inference, batch inference, and quantization by addressing the challenge of allocating resources to competing workload types with different requirements.