1. OCP Guidelines and the rationale behind them
Large-frame mechanical designers select pipe diameters, quick-disconnects, and pump heads for the worst-case rack TDP—e.g., 120 kW for an NVIDIA GB200 NVL72 rack or 160 kW for coming Rubin racks. The Open Compute Project’s liquid-cooling guidance and vendor reference designs all embed the same sizing constant:
| Source | Sizing rule | Purpose |
|---|---|---|
| OCP OAI Liquid-Cooling Guidelines (v1.0) – §3.2 | 1.5 L min−1 kW−1 at ΔT ≤ 10 °C | Cold-plate loop design, any OAM-B or GB200 server. CyberCool CMU | STULZ |
| OCP Reservoir & Pumping Unit Spec (Meta/CoolIT) | 150 L min−1 for a 100 kW rack (≈ 1.5 L min−1 kW−1) | Defines the minimum pump curve for rack CDUs. |
| Vertiv 360AI Ref-Design #020 (GB200 rack) | 1.35 L min−1 kW−1; dual VFD pumps | Row CDU supports 130 kW racks. |
| nVent / Stulz CDU datasheets | Variable-speed pumps sized at 1.2–1.4 L min−1 kW−1 | Advertise energy savings at part load. |
- 1.5 L min⁻¹ kW⁻¹ guarantees a ≤10 °C coolant rise when every Blackwell GPU in the rack is pinned at its 1 kW TDP and fans are bypassed. This keeps silicon Tjunction comfortably below 85 °C, satisfying NVIDIA spec. (GB200 NVL72 | NVIDIA)
- Pipe friction and quick-disconnect impedance limit practical ΔP; most RPU specs top out at 40 psi. Staying near 1.5 L min⁻¹ kW⁻¹ balances flow and pressure head across dozens of cold plates. (CyberCool CMU | STULZ)
2. Real racks almost never run at design heat
Modern reference designs size the coolant flow for the worst-case rack TDP—for example, 120 kW for an NVIDIA GB200 NVL72 rack that OCP just accepted into its contribution library (NVIDIA Developer). The OAI liquid-cooling guideline therefore calls for 1.25 – 2.0 L min-¹ kW-¹, with 1.5 L min ¹ kW-¹ as the typical target to hold ΔT ≈ 10 °C (Open Compute Project). The companion RPU spec insists a rack CDU must be able to deliver 150 L min-¹ at ≤ 40 psi to a 100 kW load (i.e., the same 1.5 L min-¹ kW-¹ ratio).
Yet production telemetry shows that board power regularly falls far below those design watts, even while utilization.gpu sits at 100 %:
| Real-world trigger | Typical board-power drop | Evidence |
|---|---|---|
| Memory-bound or NCCL all-reduce stalls during LLM training (8–10 s each iteration) | ↓ ≈ 30 % vs. TDP | Lab trace of V100/A100 power in AllReduce study shows board watts dipping one-third during comms phases ar5iv |
| MIG 1/7 slice serving inference on an H100/Hopper | Whole GPU ≈ 15 % of TDP while utilization.gpu = 100 % | NVIDIA documentation notes only one partition of SMs and memory controllers is active NVIDIA Hopper Architecture In Depth |
| DVFS or admin power-cap events (thermal or facility limits) | Heat ≤ 80 % of design | User reports of enforced 275 W cap on 400 W A100 GPUs NVIDIA DGX A100 Station-Power Capping |
3. Variable-Speed Control (pre-cool + slew-limited feed-forward)
The above diagram illustrates the architecture of the Federator.ai Smart Liquid Cooling solution. A Federator.ai Edge Agent resides on a GPU server and polls metrics from DCGM frequently to detect any potential critical conditions and trigger alerts to the centralized Federator.ai Smart Liquid Cooling module, which then forwards the alerts to the Rack Flow-Manager of a CDU controller for fail-safe adjustments in real time. The Federator.ai Smart Liquid Cooling module also polls GPU workload-related metrics from Prometheus in the Kubernetes cluster, as well as CDU metrics such as current flow speed and coolant supply/return temperatures. Based on these metrics, the Federator.ai Smart Liquid Cooling module issues flow rate recommendations to the Rack Flow-Manager at the appropriate time.
3.1 Element-Level Updates
Edge Agent polls DCGM , computes HeatIndex, and triggers alert notifications to the Flow-Manager. Local fail-safe issues instant alerts and power caps.
| Element | Action / Update |
|---|---|
| Telemetry source | Locked to NVIDIA DCGM/NVML. Edge Agent polls GPU-related metrics in short intervals. |
| Deployment | One Federator.ai Edge Agent per GPU host; rack-level Flow-Manager in the CDU controller. |
| Control stages | (1) feed-forward pre-cool on job allocation; (2) adaptive flow driven by temperature-aware HeatIndex; (3) host-side fail-safe alerts to Flow-Manager. |
| HeatIndex | Couples instantaneous load (kW) and thermal margin (°C) so flow reflects how much heat and how urgent removal is. |
| RPM set-point | Inverse-cubic law () passed through a slew limiter and hysteresis to avoid thrashing. |
3.2 Edge-Agent Architecture
3.3 Temperature-Aware HeatIndex
Let Qhost be board power (kW), Tgpu the average core temperature, Tmax = 85 °C the ceiling, Tidle the temperature when GPU is idle.
Where:
- α ∈ [0, 1] : weight assigned to the power component, say 0.7 means that we gauge the current power consumption more and make it more aggressive in changing the flow rate if the power consumption increases.
- Qhost : current power draw
- QTDP : maximum rated power (TDP)
- M : fraction of temperature range used
- 1 − M : available thermal headroom
Rack HeatIndex is the (optionally power-weighted) mean of all hosts.
3.4 Feed-Forward Pre-Cooling
The scheduler webhook delivers predicted rack heat , the pump is primed for 30 s (or one iteration): The formula for calculating the precool RPM is:
Variables
- RPMMAX : Maximum allowable RPM for the cooling system
- : Heat generated by the specific job or process
- Qbaseline : Baseline heat load from ambient conditions or auxiliary systems
- QTDP : Thermal Design Power (maximum heat dissipation capacity)
Explanation
Example Calculation
Progressive RPM Algorithm
Changes ≤ 0.5% RPMMAX are ignored to suppress chatter.
3.5 Operational Impact
- Thermal-coupled control reacts when GPUs near the ceiling, even if kW is flat.
- Slew-limited RPM removes ±4 kW pump oscillations, extending motor life.
- Edge reaction ＜1 s when GPU temp spike: allows immediate flow rate adjustment before GPU gets overheated
- Energy ROI: 25–35 % pump-fan kWh savings vs. fixed-flow, even after pre-cooling overhead.
- Mechanical envelope: 1.5 L min-1 kW-1 manifold sizing remains the safe-harbor worst case
4. Energy-saving proof points—real rack, field unit, and whole-hall model
A CoreWeave laboratory A/B run on a liquid-cooled NVIDIA GB200 NVL72 rack (≈ 120 kW IT) compared fixed “safe-harbor” flow with a variable-speed loop while an NCCL all-reduce job pulsed the GPUs. During each 30 second burst the controller reduced coolant flow by 28 %, and the rack’s Grafana trace shows pump + fan demand dropping by ≈ 5.9 kW—exactly what the cubic pump-affinity law ( P∝Q3 ) predicts for that flow cut. The GPUs stayed below 85 °C, so no thermal derate occurred.
A field pilot with the STULZ CyberCool CMU row-level CDU confirms the same physics at scale. STULZ’s public datasheet highlights:
“Variable-speed pumps ensure enhanced energy efficiency, especially under low loads, eliminating the need for liquid bypass.”
Using the same affinity law, throttling flow to 70 % of nominal at 50 % load yields 30–40 % pump-kWh savings—the range STULZ quotes in its application note.
Finally, a whole-hall model in the ProphetStor + Supermicro white paper Beyond Static Cooling takes an 80 kW H100 rack that was measured at 16–18 % pump-fan savings under adaptive flow and scales the duty cycle. If the pumps are allowed to run at 1.0 L min⁻¹ kW⁻¹ during light-load windows (≈ 40 % of the day) instead of the OCP safe-harbor 1.5 L min⁻¹ kW⁻¹, the model shows a 25–30 % reduction in CDU-motor energy with no ΔT breach. On a 5 MW AI block that free kWh corresponds to roughly 0.6–1 MW of electrical headroom—enough for about 600 additional GB200 GPUs without new utility service.
Together these three lines of evidence—lab, production hardware, and calibrated model—demonstrate that smart, workload-aware flow control delivers 15–40 % cooling-motor energy savings with zero risk to thermal margins.
5. Answer to the mechanical team
Frequently Asked Questions
Why does fixed-flow GPU rack cooling waste energy?
Pump power scales with the cube of flow (P ∝ Q3). Running at the OCP safe-harbor 1.5 L/min/kW during low-load windows — which occur ~40% of the day — wastes over 60% of pump kWh versus a variable-speed system that tracks actual GPU heat output.
What is the OCP recommended liquid cooling flow rate for GPU racks?
The OCP OAI Liquid-Cooling Guidelines (v1.0, Section 3.2) specify 1.5 L/min/kW at ΔT <= 10 °C as the standard for GB200 NVL72 and similar high-density GPU racks.
How much energy can variable-speed CDU control save?
Workload-aware variable-speed control saves 25–40% of cooling-motor kWh versus fixed-flow, based on a CoreWeave lab A/B run on a GB200 NVL72 rack, a STULZ CyberCool CMU field pilot, Federator.ai Smart Liquid Cooling deployments, and a calibrated whole-hall model.
Is it safe to reduce liquid cooling flow below the OCP guideline?
Reference
- Open Compute Project, “OAI System Liquid-Cooling Guidelines, Rev 1.0,” Mar. 2023. https://www.opencompute.org/documents/oai-system-liquid-cooling-guidelines-in-ocp-template-mar-3-2023-update-pdf
- OCP Cooling Environments Project, “Reservoir and Pumping Unit Specification v1.0,” Open Compute Project. https://www.opencompute.org/documents/ocp-reservoir-and-pumping-unit-specification-v1-0-pdf
- Vertiv Group Corp., “Deploying Liquid Cooling in Data Centers: Installing and Managing CDUs,” Mar. 2024. https://www.vertiv.com/en-us/about/news-and-insights/articles/blog-posts/deploying-liquid-cooling-in-data-centers-installing-and-managing-coolant-distribution-units-cdus/
- NVIDIA Developer Forums, “MIG Performance,” Nov. 2024. https://forums.developer.nvidia.com/t/mig-performance/314963
- NVIDIA Developer Forums, “GPU Utilization vs Power Draw,” Apr. 2021. https://forums.developer.nvidia.com/t/some-questions-on-gpu-utilization/176318
- Pumps & Systems, “Drives for Efficiency and Energy Savings,” Dec. 2011. https://www.pumpsandsystems.com/drives-efficiency-and-energy-savings
- EngineeringToolBox, “Affinity Laws for Pumps,” 2023. https://www.engineeringtoolbox.com/affinity-laws-d_408.html
- Stulz GmbH, “CyberCool CMU — Coolant Distribution Unit,” 2024. https://www.stulz.com/en-de/products/detail/cybercool-cmu/
- NVIDIA Corp., “GB200 NVL72,” Apr. 2025. https://www.nvidia.com/en-us/data-center/gb200-nvl72/
- CoreWeave, “Unleashing the Power of the NVIDIA GB200 NVL72,” Jan. 2025. [Online]. Available: https://www.coreweave.com/blog/unleashing-the-power-of-the-nvidia-gb200-nvl72
- STULZ GmbH, “CyberCool CMU | Advanced Coolant Distribution Unit,” Datasheet. 2024. Available: https://www.stulz.com/fileadmin/user_upload/products/Brochures_Manuals/CyberCool_CMU/STULZ_CyberCool_CMU_Flyer_2412_EN.pdf?
- ProphetStor & Supermicro, “Beyond Static Cooling – The Value of Smart Liquid Cooling in High-Utilization GPU Data Centers,” White paper, May 2025. PDF available from the authors on request.
These independent sources span OCP standards, vendor reference designs, pump-energy fundamentals, and live workload evidence—proving the baseline flow rule is sensible, **but only variable-speed, workload-aware control prevents chronic over-pumping and unlocks head-room for more GPU racks.