The challenge
Language-model services process many requests at once. Batching, shared caches, dynamic scheduling, and overlapping execution improve performance, but make energy use difficult to attribute to individual requests. System-level GPU counters capture total consumption without showing how requests and scheduling decisions contribute to it.
The work
We are building LLMargin to connect synchronized GPU-energy measurements with individual requests, scheduler activity, cache reuse, and overlapping work. The system preserves the execution context needed to explain where energy goes when requests share hardware.
We evaluate request-level energy estimates against models based only on requests or aggregate counters. This measurement work establishes the foundation for predicting how serving decisions change total energy use, including choices about batching, scheduling, and hardware settings.
Why it matters
We are turning GPU energy measurements into practical evidence for diagnosing regressions, comparing workloads, and improving serving decisions. The focus is on explaining where energy goes and identifying the changes that make inference more efficient.