Design Guidelines for a High Density Data Center:

Download this article in PDF.

Although actual power and cooling requirements seen by data centers don’t approach these loads indicated for equipment only, they do demonstrate the increasing power and cooling requirements of new data processing equipment. However, like all future projections, they need to be applied carefully.

If you had a data center fully populated with nothing but Blade or 1U servers and SAN’s all with maximum configurations, the watts per square foot requirements could reach the 200 to 300 watts per square foot level or even higher. However, most data centers today are a mixture of older legacy systems, some newer systems, areas of low or no load such as tape libraries and often aren’s equipment. As a t fully occupied due to the shrinking footprint of today’s result, most data centers don’t come close to seeing these loads. Despite the increases in power requirements for new equipment, the average watts per square foot load for data centers remains 30 to 40 watts per square foot. While portions of the data center are much higher, the overall average hasn’t changed dramatically in the last several years.

So how do you design a data center for these new higher density loads? I have some suggestions that will allow you to accommodate today’s technology in a cost effective manner. These suggestions include:

If you look at specific power requirements for servers shown in the chart below, the increases in power consumption, circuit requirements and cooling requirements becomes even more clear.

If you had a data center fully populated with nothing but Blade or 1U servers and SAN’s all with maximum configurations, the watts per square foot requirements could reach the 200 to 300 watts per square foot level or even higher. However, most data centers today are a mixture of older legacy systems, some newer systems, areas of low or no load such as tape libraries and often aren’t equipment. As at fully occupied due to the shrinking footprint of today’s result, most data centers don’t come close to seeing these loads. Despite the increases in power requirements for new equipment, the average watts per square foot load for data centers remains 30 to 40 watts per square foot. While portions of the data center are much higher, the overall average hasn’t changed dramatically in the last several years.

So how do you design a data center for these new higher density loads? I have some suggestions that will allow you to accommodate today’s and tomorrow's technology in a cost effective manner. These suggestions include:

Spread the load out: If you have a 10,000 square foot data center with 10 racks of high-density servers, spread them out over a larger area. Just because you can place the racks or cabinets 6 inches apart doesn’t’t mean you should do it. Separate the racks by at least a floor tile if not two. While it’s not as efficient from a floor space perspective, just taking this step can eliminate many of the cooling problems associated with these servers.

Set up hot and cold aisles: As you can see below, having the air intakes facing each other with perforated tiles in the middle aids in the flow of cooler air to the cooling fans for the equipment or cabinets. The aisles can be one or two tiles wide. Again, the further you spread the equipment out, the fewer cooling problems you’ll have.

Install high-density servers closer to the CRAC units: Static pressure under the raised floor is highest the closer you get to an air handler. I’ve seen data centers with CRAC units around the perimeter that installed their high-density servers in the center of the computer room where static pressure was the lowest. Putting the highest cooling loads closest to the source of cooling is a simple way to avoid cooling problems due to the lack of static pressure under-floor.

Don’t fully configure the racks: Just because you can fill a rack up to the top with servers doesn’t mean you should. Leave the top of the rack empty or use it for low load uses. There is a chimney effect with high-density servers whereby the servers lower in the racks get more of the cooler air while devices in the top of the rack tend to get warmer air. This is reflected in the failure rates for rack-mounted servers, with the servers highest in the rack having the highest failure rates.

Design your power and cooling systems to be expandable and flexible: Having systems that can be increased in capacity without shutting down the data center is critical in today’s Sizing the chilled water piping for this environment. Ultimate build out, leaving empty space in the electrical and mechanical rooms, leaving space on the raised floor for additional CRAC units, PDU’s are just some of the steps that need to be taken. Planning for the ultimate and ASTS’s capacity has to be built into the design from day one. Given the constant changes in technology, a properly designed data center has to be flexible enough to respond to those changes without impacting the operation of the data center.

Look at the environmental requirements of your equipment and factor it into the cost of ownership before making a purchase decision: Some manufacturer’s have responded to user complaints and have announced new servers with lower power and cooling requirements while providing comparable processing speed. Having servers with 80% of the processing capability, but half the power and cooling requirements might make more sense than having to double your existing power and cooling capacity. It would also put the manufacturers on notice that users are more interested in technology that doesn’t’ t require a massive capital expenditure to implement.

So what is the proper watts per square foot design for a data center being built today? How can you be sure that your data center has enough capacity for the equipment it’s completed in 12 to 24 months? A goods going to house when it’ starting point is to measure your current electrical load. While this doesn’t tell you what your future loads will be, it is at least a baseline for your design. Once you have the existing load information, review the power and cooling requirements for the technology currently being installed or on order in your data center. Knowing how quickly your company adopts new technology will also help you with planning for future loads. Assess how much unused space there is in your data center. If only 50% of the available floor space is in use and if 10 racks of servers are being replaced by 1 rack of high density servers, the watts per square foot requirements will not be as high as in a data center that is fully occupied.

Most data centers we are designing today are designed for somewhere between 70 and 100 watts per square foot of data center load, with expansion capabilities of 50 to 100%. This allows you to accommodate twice the average loads found in data centers today, (currently 30 to 40 watts per square foot), have the ability to double your capacity even further by adding equipment (ultimate capacity of 140 to 200 watts per square foot), and still do it in a cost effective manner. Since electrical and mechanical systems can be 70% of construction costs in a data center, installing equipment that isn’t needed, at least initially, is not a very cost effective approach. Remember, there have been equipment suppliers predicting 100 watts per square foot loads in data centers for 15 years now. Their predictions have yet to come true, but with the advent of these new high density servers, they may. Our challenge is to design data centers that can accommodate them if they do.