The physical structure to house digital infrastructure for today’s and future workloads is not built the same way it was for yesterday’s workloads. The conversation starts with how to handle the heat generated by today’s highly dense computing systems, very powerful single workloads, and small clusters of workloads. Thus, in the end, the engineering of how to handle the heat generated by today’s and future workloads is central to the design and construction of the structure housing digital infrastructure for those workloads.
How cooling affects a data center build. Engineering choices determine the structural, ceiling-height, mechanical, piping, power, and air-distribution architecture of a server facility. In other words, they form the basis of the server housing facility’s blueprint It’s the core blueprint around which entire facilities are constructed.
The Shift from Air to Liquid Cooling
For decades, the use of raised-floor air cooling in server facilities was the norm. The cold air from the chillers was distributed under the raised floor via ductwork, then drawn up through perforations in the tiles above the server cabinets. This cold air was then drawn across the fronts of the servers in the hot aisles, where it mixed with the hot air from the rear of the servers.
Remember those old traditional server rooms where you would feel an icy draft off the floor and hear the sound of huge blowers? Those used to be the state-of-the-art facilities But air has physical limits.
When rack power densities exceed normal levels, for very high processing workloads, in particular, air cooling simply is not able to keep up. A lot of space is required for moving air. Massive amounts of air must be chilled, then blown through large amounts of computer equipment, generating massive amounts of heat in return. The large amount of space required for such air-cooling systems decreases the efficiency of a data center facility.
The need for liquid cooling to manage the high heat generated by extremely power-dense servers has spurred a revolution in how data center buildings are designed and constructed. Liquid cooling allows for the most efficient transfer of heat from chips and servers. This enables the highest server densities in data centers, allowing extremely power-dense servers to be efficiently managed. Note that liquid cooling significantly changes the construction priorities for a data center building.
Structural and Mechanical Impacts on Facility Construction
Once you decide to switch to liquid cooling systems in your data center, the change in physical requirements will be significant. The weight of fluid-filled loops, heavy heat exchangers, and large distribution units for cooling liquids will be much greater than that for corresponding air-based thermal systems. Therefore, the floor’s structural design will need to be capable of carrying greater point loads. This may require a reinforced concrete slab and special support beams.
Piping architecture – supply and return piping architecture for a liquid cooling facility is critical to get right. Often, smart sensors are placed along the return piping to enable monitoring of the cooling system. Liquid for cooling in a data center will also need to be contained, so the designer of the facility must incorporate a leak detection system and provisions for a ‘container basin’ to hold and treat the coolant in the event of a major leak And then there is the human element.
The physical labor of building a thermal system of this complexity is grueling work. Pipe layouts need to be carefully planned to accommodate supply and return lines, with smart sensors installed to monitor thermal conditions and fine-tune cooling precisely where it’s needed. Once installed, these systems require intricate work to route heavy fluid loops and connecting lines, which in turn requires careful consideration to contain any damage in the case of a leak. Leaks can cause serious damage in data centers and must be contained and/or drained as safely and efficiently as possible. Such work is grueling and complex and best installed by a highly qualified, experienced workforce. It is, in effect, a highly intricate mechanical and plumbing system that must be built and fully tested before a data center is powered up. This transition to advanced liquid cooling represents a positive evolution in data center staffing, shifting the industry toward a more specialized, highly skilled workforce essential to the reliability of today’s high-density environments.
Electrical and Spatial Power Considerations
As we consider how the choice of cooling technology affects the electrical architecture, we note that mechanical systems consume up to 40% of the facility’s power to run fans and chillers in air-based systems. High-efficiency liquid cooling systems, though heavy, consume far less power to cool servers of equivalent class. As a result, a larger fraction of the power budget can be devoted to high-value computing functions rather than to mechanical cooling systems. We consider the implications for the electrical architecture of the cooling choice, including transformer, generator, and UPS requirements.
When mechanical systems that previously used a portion of the power to cool the computer systems in the data center now require less power to cool them, more of the total power in the data center can be devoted to actual computing. However, to take advantage of this, the various electrical systems, such as power distribution, transformers, generators, and uninterruptible power supplies (UPS), must be reconfigured to leverage the increased efficiency of the cooling systems.
Space-efficient liquid cooling systems for computer facilities can also allow them to be built as compact or very large structures, with many computers densely packed within the same overall area of the facility. In contrast, space for lots of cooling air to move through the data center must be provided for air-cooled systems to prevent computer “hot-spots”. If insufficient space is provided for air movement, it can require significant effort and, thus, be very costly to provide adequate cooling for large groups of high-density computer servers in a building using air cooling for all computers in the facility. Also, more value will be gained from a given amount of space for a liquid-cooled system than for an air-cooled system of the same size. The value gained from a given amount of space is, for all systems, proportional to the amount of computing that can be done in the space. Hence, substantial computing can be done with space-efficient liquid cooling for high-density computer servers.
Water Usage and Environmental Sustainability
The cooling solution also defines the facility’s environmental footprint and determines the facility’s utility requirements. In the typical air-based thermal solution, evaporative cooling towers use millions of gallons of water per year. In areas with water scarcity, the typical air-based thermal solution for a data center can stall municipal permitting. It can also generate community resistance.
There are many closed-loop liquid-cooling systems on the market, as well as waterless air-to-air heat exchangers. These solutions can significantly reduce municipal water consumption. In many cases, it is possible to get a permit for a data center that uses dry cooling faster than for a data center that uses cooling towers, and in some cases it is even possible to put a data center in arid regions, where there is plenty of land and renewable energy but not enough water for evaporative cooling towers.
Designing sustainable facilities is no longer about making a good impression. Rather, it has become a necessary aspect of a company’s long-term operations as well as a matter for regulatory bodies around the world. A sustainable data center is quiet – even on a hot summer’s day – and doesn’t use up all the local water.
Designing for Future Scalability
Cooling technology, or more specifically, how it is implemented in data center builds, affects Facility Architecture in a very significant way, primarily because, in order to build a good data center, one has to anticipate the thermal evolution of future computing. Building a good data center architecture, in the first place, requires a number of blueprints and floor plans, and many of them must be studied in detail to understand all the aspects involved in a future-proof data center design.
A new form of facility design is being offered: ‘modular’ data centers. These can include, for example, ‘slab’ floors that are easily adaptable, very large utility corridors to contain future services, and ‘manifolds’ for pipework to allow, for example, the change from air to liquid cooling, or an increase in liquid cooling capacity, without having to undertake a complete refurbishment, and this all whilst the facility is operational.
Building with the future in mind – or, more specifically, with thermal evolution in mind – is a very forward-thinking approach to facility construction. While it may require more thought and upfront investment, in the long run it will be of great value to the facility owner and will last for many years to come.
Final Thoughts
In summary, advanced cooling systems have now become the primary driving force behind today’s modern data center architecture. In building a data center facility today, the thermal management of servers can no longer be an afterthought, as increasing computer power in smaller packages continues to drive higher and higher power densities. Those data center builders and operators who align their mechanical, electrical, and structural plans with the most advanced data center cooling technology from the very start of construction will build their data centers faster and operate them as efficiently as possible.
