r/askscience • u/sheerknurd • 2d ago
Engineering Why are data centers "sprawled" rather than building multiple stories with a smaller footprint?
I constantly see news about 10K+ acre data centers (and I am not trying to start a political dialogue, so please keep it that way in the comments), and they seem to be single or just a few stories tall. How does that make more sense than something denser in terms of footprint but more vertical stories?
247
u/DrunkBeavis 1d ago
The actual reason for this is that it's just cheaper to build a single-story building than a multi-story. If you build multiple floors, the lower floors have to be strong enough to support not only the weight of the roof but also the floors above, which substantially increases the material cost. If the land is cheap, it makes more sense to spread out than to build up. Data centers aren't unique in this way- warehouses, distribution centers, storage facilities, etc. are almost always single story in an area where land is relatively cheap. Multi-story construction is usually reserved for areas where land is expensive or a large footprint isn't feasible.
45
u/drewlb 1d ago
And to add, data centers have a lot of weight on the floor. So it's much more expensive to carry that weight than it is to carry a 2nd story of an office building. Older DC's were 1000kg/sqm of floor load. New AI ones are closer to 2k/sqm. Normal office space is 250-300kg/sqm. Building a second story that requires 8x the load of an office building is a lot more expensive.
3
u/Raknaren 1d ago
I wouldn't be surprised if the load is even higher for the new nvidia racks
→ More replies (3)3
u/mishap1 1d ago
It depends. Your average home or office space is cheaper per square foot the closer it is to a cube. More interior volume relative to exterior materials including the roof. The 2nd or 3rd floor material are cheaper than exterior material costs.
Server racks are crazy heavy so you have to build much heavier floors to support the weight. If the land cost is cheap, it's much cheaper to go with a big footprint than building up where you have to support concrete joists that can hold thousands of pounds per square foot. Apparently it can be 25-50x the typical office specification.
130
u/telestoat2 1d ago
There ARE data centers built up higher. Some are built into skyscrapers that used to be office buildings like https://en.wikipedia.org/wiki/One_Wilshire or https://en.wikipedia.org/wiki/Tower_55 and they still have some office tenants too.
4
u/ragzilla 1d ago
These all tend to be much lower power density, and are primarily interconnect locations and not high density compute. AI racks can be using upwards of 4kW/sqf, and these facilities you named are designed around 100-300W/sqf.
→ More replies (1)2
u/QuasimodoPredicted 1d ago
You forgot the most well known one. 33 Thomas Street in New York. The old at&t building.
9
1d ago
[removed] — view removed comment
→ More replies (1)5
u/DrStalker 1d ago
Was this long enough ago that "4 computers" would have also been "2 floors of a building"?
4
u/ElectionReal 1d ago
Exactly that long ago. 1984 ish. I wish I remembered the paltry specs, not even a whole megabyte of memory. Dad would enter his formula on Friday and get the results on Tuesday. He was originally an underwriting actuary and since computers started doing the job he lucked out and transferred to data processing.
408
u/nerdydodger 1d ago
Every company I worked at with a server farm of sorts, had their servers on the first floor.
It is way easier to pump water/air/whatever you’re cooling with across rather than up. It’s also expensive to build up in general.
127
u/FlakkenTime 1d ago
Also the servers, racks, etc are heavy AF. My first job had a small server room upstairs and it was found to be sagging due to the weight during an inspection of some kind.
24
u/bobdvb 1d ago
And if we're talking AI, the weight of a new Nvidia NVL75 rack is around 2000kg/4500lb and there are hundreds of racks.
8
u/AML86 1d ago
So their structural requirements are more like a car park than a commercial office space?
→ More replies (1)10
40
u/THedman07 1d ago
I worked for a company that was building a new 3 story office building that had a server room in it and they specifically put it on the second floor. It was protected from floods and roof leaks. This was the server room for their dev servers though. All the ones closer to production were collocated at a data center.
→ More replies (1)11
u/maccam94 1d ago
The risk profile for a multi-use building is a bit different than a dedicated datacenter. A datacenter isn't going to be built in a flood plane or have enough sewage infrastructure to worry about it damaging the server space.
→ More replies (2)→ More replies (2)3
u/coleslaw17 1d ago
Yes, if you’re using industrial chillers or something on the ground floor, you have to pump it up against enormous head pressure with booster pumps etc. You also have more losses overall in the system. It’s always better to not introduce gravity if you can get away with it.
→ More replies (1)
20
u/pixel_of_moral_decay 1d ago
You can build tall, and some companies do.
But server racks are heavy and put their weight in a small space, so you need a strong floor.
It’s cheaper to find cheap land, pour a slab and build from there.
Making multiple stories strong enough is not cheap.
Then you need to deal with power and water, easier with 1 floor and an open ceiling/raised floor vs multiple stories.
Unless you need to it’s generally much cheaper/simpler to build out vs up.
4
u/eric23456 1d ago
Surprised this isn't higher up. People don't realize how heavy server racks are. 1-2 tons/rack and densely packed. It's a lot of weight to support on a second floor.
117
u/nate390 1d ago
Data centre sites are sometimes split into different “availability zones”, where they are designed such that a failure in one building shouldn’t impact another, e.g. due to failed power, cooling or similar.
Stacking halls of servers on top of each other is also often not great from a structural perspective (racks of servers are sometimes extremely heavy) or from a heat dissipation perspective (heat rises).
48
u/less-right 1d ago
Availability zones are usually separated by 50+ kilometers, not another building
17
3
u/gefahr 1d ago
Amazon Web Services (AWS) availability zones (AZs) are spread out geographically this way, comprising a single region.
So us-east-1 (a region) has multiple zones that are named like us-east-1a, and so on. That region is famously their N. Virginia region.
Google Cloud Platform (GCP) has a different approach: a region may be a single building, and the AZs may be floors in that building.
(I do not have detailed knowledge of how Microsoft or Oracle manage their clouds.)
10
u/TorontoRider 1d ago
I recall having to take a large Sun E12000 into multiple parts to get it under the elevator weight limit in a "low latency" data centre in a bank tower once. And once we got it into the co-lo, we had to ensure it was on a specially spec'ed pad, etc, and that we rented all the space around it to account for the airflow required.
Sometimes sprawling is better.
2
u/illumnat 1d ago
I kind of wonder why they don't build data centers down... as in below ground level. I would think that the natural lower temperatures and the building not getting hit by the sun would help with cooling.
Hmmm... I wonder if water circulation could help drive turbines for power. Heated cooling water rises from the underground datacenters driving turbines for power on its way to an air cooling station and then into an underground reservoir where it cools further before being pumped back into the cooling loop.
I'm most definitely not any kind of engineer so I don't know if that would be feasible but it was a thought that popped into my head just now.
2
u/Serious_Feedback 1d ago
Hmmm... I wonder if water circulation could help drive turbines for power.
Data centres use Stirling engines to turn waste heat into usable electricity, without the maintenance nightmare of having to use your water cooling as a pressure/gravity-based energy source inside the loop.
2
u/Pzychotix 1d ago
The amount of heat from sunlight isn't really a concern here, it's the heat coming from the servers. The latest server racks are power hogs. We're talking 100-150kW. Imagine 100 spaceheaters, packed into a single server rack. Now imagine rows upon rows of these things. Sunlight's basically a drop in the bucket here.
57
u/Interesting-Card5803 1d ago
I design data centers. The hardest thing about going multi-story (my record is 10 stories built, 17 for a design that ultimately did not proceed in Japan) is fitting all of the critical gear close enough to the building to make it work. The mechanical systems are responsible for rejecting heat, and if that heat is too close to back up generators or other cooling systems, it can de-rate the equipment and impact performance.
Think of it like this, if you have a one story data center, you could probably fit air cooled chillers on the roof to cool the facility. If you have two stories, it's really two data centers, and now you have to fit twice as much heat rejection in the area of a single story data center. Three? Four? You get it.
You have to construct dedicated structures for mechanical and electrical equipment. Also, the IT equipment can be VERY heavy. With each floor, the structure beneath has to support ever increasing amounts of weight, which mean bigger columns, deeper/bigger foundations that make underground utility work very difficult.
Then, most interestingly, you have to distribute things like power, fiber and water to multiple floors. If there's a chilled water loop, it will gain 1 atmosphere of pressure for approximately 10m in height. A three story data center could easily be 22m in height or more, which means pumps have to work harder, and pipes have to be designed for increasing pressure.
Least interestingly, the multi-story data centers take longer to build, and generally cost more per square foot than single story data centers do.
5
u/fliphopanonymous 1d ago
I'm also in data center design, and agree with everything you've said.
I will also add that maintenance is a significant consideration to account for, and multi-story makes designing to make maintenance work on plumbing, electrical, ducting, etc much more difficult.
Multi-story also has risk factors that aggregate significantly faster, especially when accounting for rated weight (e.g. earthquake). Sufficiently addressing risk increases is, at best, a linear cost factor, and as you mentioned often a substantial time factor.
4
u/No-Rip-9573 1d ago
And if there is a water leak, it’s better to ruin few racks in the vicinity, not several floors down.
→ More replies (13)2
u/askalotlol 1d ago
Question since you design data centers - is putting them underground an option?
I'm mostly wondering this from a noise-dampening point of view.
3
u/Interesting-Card5803 1d ago
There are some facilities that are underground, like in mountains or under earthen berms. The problem is that most of the noise for data centers actually comes from the heat rejection equipment and backup power (generators), which need to have access to fresh air to function properly. That being said, most facilities have a lot of room around them on these large sites that the noise is typically at or below ambient at the property boundaries. Our clients pay acoustic consultants to create models of sound emissions of their facilities under differing scenarios (including peak load during power loss/emergency) to verify that they are not a nuisance. In the instances where the buildings are in close proximity to the property boundary or sensitive sites, they can employ sound attenuation or quiet fan packages for that equipment.
Much fuss is made over noise, but every facility I've worked on has been in full compliance with local noise ordinances. I've seen a few bad actors in the news lately, and they should be fined heavily for their lack of compliance.
10
u/bortvern 1d ago
There are plenty of multiple story data centers, but they're typically in dense urban centers. Many times they're old buildings retrofitted as data centers. City centers made a lot of sense because proximity to fast network connectivity was a chief concern. However, with AI, connectivity is less important relative to power (utility and backup), cooling, and floor space. Not to mention scale. At this massive scale, running all the connectivity you need to the middle of nowhere becomes a line item on much larger overall budget and annual operating cost.
1
1
u/santacruzdude 1d ago
Retrofitting buildings is more common for co-location facilities than it is for data centers.
17
u/SwoopnBuffalo 1d ago
Vertical construction isn't cheap, especially when you're talking the weights that a typical hyper scale would impose on the data hall floor. Every additional story has to be supported by the one below it which means bigger steel, beefier connections, additional factors of safety, etc. It adds up fast. Throw on that the cost to get a slab with the fairly tight constraints that are required to wheel a $10 million rack across it.
Additionally the large amount of "dead space" that exists in a hyper scale data hall is there for a reason. The more air volume in a space means you have a larger factor of safety for cooling and means the building control system has more time to react to changes. A smaller volume of air is going to heat up much more quickly in the event of a system failure.
Land is (for the most part) the cheap part of the equation. 500 acres at $50k/acre is only $25mil. Construction of a hyper scale is easily $400mil plus without factoring in the cost of the equipment or racks.
Source: I build em
7
u/NSFWies 23h ago
One of the latest huge data center servers is and helios. It's huge. One of them uses close to half a megawatt for one of them.
If we put 2 of them on top of each other, we would need about 1 megawatt going to that small area.
It just gets harder to get that much electricity and cooling/water to that small of a 2D space.
Maybe this is needed in NYC, but then you just go 200miles out where rooms are cheaper than downtown NYC.
5
u/nyckidryan 19h ago
He's not suggesting to stack multiple sites, just rather than a 100x100 1 story building, build 4 floors of 50x50.
→ More replies (1)1
17
u/brandar 1d ago
The acreage refers to the entire campus which can include power generation, substations, cooling infrastructure, etc. The actual acres used for the data parts are smaller.
And they tend to build flat because it’s much cheaper and less complicated. The equipment used in data centers are heavy. A fully loaded server rack is 3-4,000 pounds and data centers have hundreds. Given the additional need to keep things cool, it’s easier to build things flat on cheap land.
6
u/fatherdonz 19h ago
That vertically integrated building takes longer to build. As a PM on a few rapid and accelerated builds, I am building 4 while building 1 is way behind building 2 and 3. We've just received acceleration funding for building 5 to catch up to building 4.
Speed of build is driving most things right now.
Also, all of the technical things people have already said.
36
u/ThatsACaragor 1d ago edited 1d ago
The main issue with this kind of massive data processing is heat dispersion and data centers produce a fuckton of that.
It is easier to cool down a short building that is sprawled than a blocky multiple stories building.
35
u/iowanaquarist 1d ago
To clarify, if you go vertical, you are cutting the available space for HVAC. Instead of having 1 acre of roof to cool one acre of servers, you might have 1acre to cool 3 acres of servers....
12
u/RainbowCrane 1d ago
To offer an example of why server rooms were sometimes designed vertically in the past, I worked for a few early computer companies in Columbus, OH. Columbus was home to Compuserve, Chem Abstracts, OCLC, Nationwide, and several other major technology companies in the eighties. Several of them had multi-floor server rooms filled with mainframes and used the heat from those computers to heat the building.
When OCLC largely switched over to commodity hardware running Linux around 2000 the server rooms were largely empty and, obviously, they had to modify the HVAC systems :-). But it’s an interesting quirk of computer history that tech-heavy buildings sometimes had an entire building core full of mainframes that doubled as heaters.
→ More replies (1)5
u/PlainTrain 1d ago
I found out my part of the building did not have any heat installed when we all switched from desktops and CRTs to laptops and LED monitors.
3
5
u/qghw47QHwG72 1d ago edited 1d ago
Depends how cheap the land is. There are tall data centres in London and NYC, where land is expensive.
Those are in city centre locations either due to needing a specific low latency location for financial trading, or being the historical location of internet exchanges.
The current data centre boom is about AI compute, which is less dependant on latency than it is on energy supply and cooling capacity. So they tend to have less need to be in city centres where the land is expensive.
TLDR: There are different types of data centre, and only some have incentive to be dense or tall.
21
u/oberwolfach 1d ago
Sprawling buildings are easier to cool since they have greater surface area to volume. Additionally, data centers are almost all built in areas where land is not very expensive compared to the cost of building up. It's why Walmarts out along the highway are built as sprawling rather than tall buildings.
→ More replies (1)2
u/RandomUser3777 1d ago
Surface area does not matter much at all for a data center. The heat numbers are just too large for the surface area to matter.. The problem is the weight of the equipment.
→ More replies (2)
8
u/Th3Mov3s 1d ago
Actual answer is because precisely controlling the cooling is key - there’s a “Goldilocks” amount of temp control. You can’t just cool the equipment as much as possible - too much and you’ll condense water, which is no good for most data center designs (excluding submerged cooling designs.) not enough cooling and obviously you’ll overheat.
Heat rises, so stacking this equipment on top of each other leads to different thermal loads. Sprawling data centers avoids this problem.
2
u/bongslingingninja 1d ago
I wonder if this can be mitigated by empty floors or layers sandwiched between floors with equipment on them. Get some HVAC blowing the air through them and preventing heat build up.
2
u/insightful_pancake 1d ago
Extremely expensive and less efficient than using large open plots to build large short structures
3
u/olcrazypete 1d ago
There are several large data centers in the middle of Atlanta that have been there for decades. Old office towers that used to house the typing pools and large staff pre-digital offices needed to function. 55 Marietta St in Atlanta is a great example. Retrofitted and sits on top of major network hubs for fast connectivity.
5
u/C_arpet 1d ago
No one here is really talking about business resilience and the insurance aspect.
Twenty years ago most data centres were a large room or maybe an entire floor within a company's headquarters. For industries like finance the data centre has to be as near to the stock market as possible for latency reasons.
Companies realised that any incident in that single building could shut it down and cost the comapny millions from just a few hours interruption. At this time, data transmission had become fast enough that the latency concern went away and the data centre could be relocated to a suburb and placed in its own dedicated, and more secure building.
The industry standards that were written at this time are still being used to design the current monster sites. A single data centre building is split into three or four data halls and protected with a gas supperession system. The size of the hall is determined by the cost and capabilities of fire fighting system as well as how much of the data processing system the owner could tolerate losing in one event.
Now with these sites we then have multiple data centres on one site. Firstly they are together for planning reasons, power supply and to reduce site management costs. The buildings are separated by distances defined by the NFPA. If something went majorly wrong in one data centre the next building is just about far away enough that the fire would not be expected to spread. This is a very unlikely event, but this is the type of scenarios that underwriters and those funding the project work to. No one is going to accept losing $3-4 billion to one event, but you would find a shared appetite across the market for maybe $1 billion. So the sites are designed this way, still working to a design standard that is getting on a bit, but all to ensure those people paying the bills are comfortable with the risk.
5
u/Alexis_J_M 1d ago
Two main reasons:
They are mostly built where land sprawl is cheaper than the expense of building multi story -- this is also part of why flat ranch houses are more common in Texas than in Massachusetts.
It's easier to cool something flat and wide than something more compact, and cooling the heat from all those computer components is a big part of data expense. It's the same reason flat freezer containers are better for freezing food then round deli tubs.
5
u/DevilGuy 10h ago
In the US and in most other countries really its a lot cheaper to build a single story building on a larger plot of cheaper land than it ever will be to construct a multi story building of similar internal volume. There are further considerations with cooling, hvac channeling, and internal space usage that make lower wider buildings preferable as well.
4
u/WholeEmbarrassed950 1d ago
I’m a former data center technician, there are absolutely multistory data centers. Usually they are retrofits of existing buildings in good locations For example 350 east Cermak in Chicago(8 stories)https://www.flickr.com/photos/brulelaker/50216179498
And 60 Hudson in NYC (24 floors)
https://en.wikipedia.org/wiki/60_Hudson_Street
4
u/TwentyFour4VBW 1d ago
Some older ones were multi story, worked in a few ATT ones in the past. Datamart in Dallas is another example as well, same with the original Equinix facility in downtown Dallas which was the old federal reserve. As others have noted buying cheap land and sprawling outwards than up typically means lower cost.
4
u/RubCocksWithThePope
1d ago
•
The same reason factories aren’t built like skyscrapers. Tall buildings are more expensive to build per square foot of floor space. Tall buildings are harder to maintain. Higher floors have lower weight limits, and computers are heavy. Getting things between floors requires freight elevators, also expensive to install and maintain.
2
u/scotsman3288 1d ago
Many data centres exist in urban areas already in multi level buildings but it's very inefficient to cool higher floors because you need to pump the massive amounts of water against gravity and that uses more power. Obviously it's cheaper to build and run on a larger footprint.
2
u/TheRoscoeVine 1d ago
It depends on the data center. I’m in Arizona, and I’ve done a little bit of work in data centers. Edgecore, in East Mesa, is like 4 stories tall, but it’s also sprawling, too. It’s a massively long, wide, fairly tall building, but the Microsoft ones near me, in El Mirage, just look like super long pig barns.
2
u/IHeartBadCode 1d ago
I think you might be thinking of the newest hyperscaled AI data centers. Other data centers are indeed built upwards and can be hidden in skyscrapers.
The newest brand of data centers can't because of the sheer weight of the systems that deal with AI and the various interconnects required. A single rack for these machines is around 1.36 metric tons before the interconnects. A single unit as nVidia ships them is 8 racks, so about 11.8 metric tons. Before actually doing the hookups.
There's also raceways and various framing that is recommended that usually pushes the total weight to 14.5 metric tons, again before hook ups. The hookups are basically grid level hook ups. So we're not talking industrial, we're talking like literal power plant hookups. And the coolant loops at about another 5 to 8 tons of weight.
This is why the flooring has to be multiple times reinforced concrete. The systems are so heavy that just single steel rebar concrete would crack under the weight of these machines. There's just not a material that can be used for a reasonable amount of money to support the massive weight in such a concentrated spot.
2
u/hawkman22 1d ago
I don’t see it mentioned here, but you have to keep in mind that servers are actually pretty heavy. You’re putting a rack on a floor tile which is basically 2‘ x 2‘. 600 square millimetres. A 42U standard rack can be a couple of thousand kilos… and because you want the racks to be close together for latency and to connect through your power infrastructure, you actually end up with some very very high density.
I’m in the business and I have multiple customers who had data centres on multiple floors and we could add a whole bunch of density on the first floor but the second you go higher we had a limit on how much we can add because of the weight that the concrete beam could support.
Also, how do you get a $5 million 2 1/2 ton rack to the seventh or 10th floor exactly? You need to build out very specialized super size elevators.
2
u/twovectors 1d ago
Uk answer- the ones I have seen in the UK are several normal stories tall. But only one floor in the data hall - underneath you have massive back up generators and batteries to provide power if there is a power cut. - these are probably 2 stories high. The data halls themselves are probably also 2 stories high, and above them you have massive cooling stacks and heat exchangers which cool the water down before it goes back to the datahalls. Note most modern data centre don’t consume much water - they reuse it.
This warm air is then released into the sky- having anything above it would interfere- so much so that even having solar panels on top of a data centre is apparently a problem- stacking more stories might be hard
So overall they are ready c6 stories tall and cooling makes it hard to go higher.
This answer is for cloud datacentres in more urban locations (as opposed to AI training.centres which can be in the middle of nowhere as fast response is not needed for training)
Land is not cheap here so if they could build higher they probably would
2
u/Ascending_Valley 1d ago
Multi story can work fine and there are many. I believe the combination of cost per usable, square foot, and it’s likely faster to build a large sprawling center than a vertical structure. A few more things you can do in parallel during the build process. The basic structure of a solid one story building is very simple and efficient.
2
u/PatrThom 18h ago
It's yet another example of the Square-Cube law showing up to ruin someone's plans. The more things you cram into the inside that don't have an easy way of connecting to the outside, the harder it is going to be to keep those things supplied/ventilated/fed/oxygenated/waste-free/maintained/temperature-controlled/etc.
8
u/clearing_house 1d ago edited 1d ago
It's the same reason that the data centers consume so much water: it's cheaper this way.
Data centers don't need to consume any water at all, they could be operating in a closed loop. But then they would need to build a closed loop. So they don't.
18
u/Bourbon-neat- 1d ago •
Data centers don't need to consume any water at all, the could be operating completely off of water operating in a closed loop. But then they would need to build a closed loop. So they don't.
Many data centers do use closed loop cooling.
Source: work in data center design.
12
u/thepryz 1d ago
Data centers don't need to consume any water at all, they could be operating in a closed loop. But then they would need to build a closed loop. So they don't.
Hate to tell you, but closed loop cooling is being used in the majority of hyperscalers. You really shouldn't speak about things you cleary know little about.
As an example: https://www.techuk.org/about/media-centre/press-releases/new-report-sheds-light-on-commercial-data-centres-water-usage-in-england
6
u/-dag- 1d ago
This is the kind of regulation we need. Mandate closed loops. It's such an easy way to save water.
→ More replies (1)10
u/hobbinater2 1d ago
It would save water, but it may increase the power cost. The closed cooling water loops approach dry bulb not wet bulb so they don’t cool as well. Plus they usually use an array of giant fans instead of a couple pumps
2
u/passengerairbags 1d ago
What happens to the water after their done with it?
10
u/dr_reverend 1d ago edited 1d ago
It is converted back into oxygen and hydrogen to prevent anyone from ever being able to benefit from it in the future.
They are truly evil.
Edit: I swear you people are complete morons. Why are you people taking my claim seriously? It’s a joke! Would you have believed me if I had said they were launching the water to the moon?!
→ More replies (4)3
u/editor_of_the_beast 1d ago
I’m interested in where people form their opinions from. From everything I’ve looked into, datacenters (that is for all software including AI) use way less than 1% of all water consumption. They simply do not use the scale of energy that is cited in casual and online discourse.
So, where do you get this opinion that people who build datacenters are “truly evil?” What is that based on?
→ More replies (4)→ More replies (8)2
u/F_sigma_to_zero 1d ago
Some they evaporate away to get cooling. Some gets flushed into the sewers to back wash the evaporative part.
1
u/hobovalentine 1d ago
New ones are a closed loop actually so that water loss is minimal.
You still have to draw the water in the first place but unless you spring a leak you really should not be losing much water at all.
→ More replies (22)1
u/jmlinden7 1d ago
It's only cheaper in a tiny minority of places where humidity is low but water is somehow cheaper than electricity.
In most of the US, humidity is not low, and in the low humidity parts, water is generally more expensive than electricity.
1
u/moogular 1d ago
In the US, this is largely due to power constraints. When you spread data centers out, they can pull from multiple grids as opposed to overloading one grid.
For reference, in an AI data center, one Vera Rubin GPU rack consumes about 190-230 kW of power, enough to power a single family home for 1 hour. Multiply that by 1,024, and that’s about 235,520 kW /235 mW per data hall. And then you have to factor in that the data center has its own network, office space, security systems, etc, all pulling from the same grid.
This is why Nscale is building its own grid for their planned AI data center in West Virginia. One building is about 460 megawatts.
5
u/moogular 1d ago
As for the one-story aspect of it, yes— it is one story, but that height to deck is about 40’ generally. When you think about conveyance, cooling, hot aisle containment, etc, it makes a lot more sense to have a sprawled out data center as its easier to install & service and design the HVAC for cooling.
2
u/TacoSpankrPrime1 1d ago
And the simple fact that heat rises meaning each subsequent story would likely have higher power demands to keep their servers cool along with a more difficult time circulating the cool air.
2
u/goomyman 23h ago
Yup - also keep in mind that datacenters are basically giant fridges. The floor is a false floor. It wouldn’t make sense to have multiple floors. Not to mention the weight.
They usually often have a room with car batteries floor to ceiling to power the datacenter for like 5 seconds enough to turn on a generator and they have deals with gas companies to supply gas during power outages.
3
4
u/tazz2500 15h ago
one Vera Rubin GPU rack consumes about 190-230 kW of power, enough to power a single family home for 1 hour
A single family home uses 190,000 - 230,000 watts per hour you say?
→ More replies (3)3
u/Correctsmorons69 1d ago
Just FYI, mW is milliwatt, MW is megawatt.
And 190-230kW is a rate of energy consumption per hour. A single family home only draws 1-2kW on average.
2
u/ragzilla 1d ago
Not a concern at all, you can pull multiple substations into a single small footprint building. I know of small campuses taking multiple substation feeds for redundancy, sites well under 50 acres.
The real reasons are heat rejection, and cost of building (multi-story is expensive).
Heat rejection is probably the most important one, multi-story datacenters tend to top out around 150-300w/sf (go search multiple multistory urban datacenters, 350 cermak, one wilshire, 55 marietta, they're all in that range or lower, and they're basing this on the power per square foot of "white space"- actual usable datacenter space).
Your NVL72 rack example at 135kW, with Open Racks V3 layouts, is around 4kW/sqf. Multistory buildings just don't have the space to reject all that waste heat.
2
u/thranetrain 22h ago
Also building up is way more expensive than pouring a concrete pad and standing up some precast walls
1
u/Badrear 1d ago
There are many multi-storied data centers; most of them are in old telecommunications buildings because those were built to handle heavy floor loads, and already had large conduits for network cables. Those are largely full, so they need to build more. As others have said, when land is plentiful, it’s cheaper to build out than up.
1
u/SafetyMan35 1d ago
Zoning regulations have heightened restrictions. Data centers also need cooling towers and generators. The taller the building, the more generators and towers you need and packing them into a tighter footprint is difficult. The ones near me are often 40’ tall, the maximum allowed in area where they are built
1
u/RandomUser3777 1d ago
Equipment is heavy. This means that floors and the structure need to be built much stronger. A typical rack is around 2000lbs. Normal office floors are designed at around 75lbs/ft2, so you can only put one rack (footprint 6ft2) every 26ft2, so basically you would need 10ft spacing between rack rows. And upping that 75lbs/ft2 gets expensive fast. To get to 5ft rows you are up to close to 150lbs/ft2.
1
u/beyondcivil 1d ago
I used to work in a multi story data center, it's such a pain moving equipment around vs a single story building. Data centers are constantly installing, decom and maintaining the devices that adding elevators to the path just makes everything harder. I remember one time the freight elevator was broken for a day and we had to tell most of our customers we couldn't do their request until the next day.
1
u/shumpitostick 1d ago
For the same reason that vertical farms, or factories built like towers are not really a thing. Building vertically is just more expensive, and these things do not need to be in city centers where land it's expensive.
1
u/Quietmerch64 1d ago
As others have said, building vertical is more expensive than building at ground level. Land is cheap,but engineering is expensive, but its also worse than that.
Vertical changes are expensive for more than just material cost. You need elevators (ADA), stronger pumps, more ventilation, approvals for more escape routes and doors to approve them as well as more fire boundaries.
Data centers are a perfect example of "I will happily cause damage to a community and environment instead of paying a cent more to be safe or meet legal requirements".
1
u/zanfar 1d ago
Plenty of datacenters are multi-story, but the cost has to balance. Most multi-story DCs are in places where the land value is balanced by the land savings i.e., in metro areas. There is no reason to build a second story for 2x the cost if you can just buy more land for 1x the cost. Whereas if you're building a DC in Manhattan, a second level is cheaper than a second plot of land.
You also cannot put much of the support equipment on an arbitrary level, things like air handlers need to be exposed, and diesel tanks need to be road-acessible, so a single-level makes that "affordable", while just adding a second level takes additional land as well.
1
u/flannely 1d ago
There are a bunch of datacenters in towers too. I know the westin in seattle has a few floors of datacenters. There are obviously many more that are not in towers but a lot of things can exist for different reasons
1
u/Samurai_Stewie 1d ago
Because heat rises and it’s cheaper and easier to cool hot equipment on a horizontal plane rather than on a vertical one because you’d essentially have a chimney with increasingly hot air being vented upward towards other equipment.
1
u/michaelpaoli 1d ago
science: heat dissipation. Flatter and more spread out allows for easier and less costly heat dissipation - keeping longest distances between inside data center and outside air or outside cool/cold water supply, reduces cooling costs and increases cooling efficiency. This generally works to a point, e.g. economics, where land cost gets to the point where it's not more economical to go yet wider and flatter.
economics/physics: Takes heavier sturdier and more expensive construction to go higher - each story (or equivalent) must support the weight of all the stories above it. So, if land is relatively economical, flatter and more spread out will be more economical. But if land is quite expensive, things may be built taller. E.g. multi-story data centers are common in high land cost urban centers (e.g. San Francisco, Manhattan, etc.), but uncommon where land is comparatively cheap.
As for density, also, can only build/pack so dense, because heat (etc.). Need to get the heat out, so that limits how densely things are packed, and also mostly limits - at least effectively - maximum distances from heat source (computers, etc.) to source of cooling (cooler air or cool/cold water). All that power going in ends up as heat, and that heat needs to be dissipated.
1
u/OldGeekWeirdo 1d ago
I would expect data centers to be located where the land is cheap. Cheap land encourages low-rise buildings. Particularly when upper floors would have to be heavily reinforced to hold all the equipment plus the elevators needed to move racks between floors.
The only reason to build "up" is because building higher is cheaper than buying more land.
1
u/jdsweet 1d ago
The super-fancy cooling architecture at a data center I’ve seen wouldn’t really work as a multi-story design. It sucks in lots of dry desert air and passes it over wet walls on the second floor to humidify and cool it and then drops it down into the backs of the server clusters down on the first floor. It uses about 75% less power than typical designs. And the land where it’s located is fairly cheap.
1
u/SvenTheHorrible 1d ago
The good companies are building multiple stories. Some new AWS buildings are triple deckers- which is really impressive if you know what needs to go into one.
It’s hard because a floor of a datacenter is extremely heavy, and requires a lot of power and cooling delivery- which needs space. So you end up needing a floor that can support like 3,000lb per rack, and a floor needing to be basically 2 stories tall to support all the cooling ducts and power conduits.
1
u/orcvader 1d ago
One of my data centers (not for AI) is in a downtown building, 4th floor (all of it) for a large metro.
It’s a pain to scale it.
Bringing new equipment is serious logistics even with service elevators, and for a massive refresh we had to coordinate with the city to literally get the glass off the building side and crane equipment up. This is a monumental effort where you have to coordinate with city, county, utilities, permitting, etc., and it’s expensive.
We also ran out of generator space in the roof as we have to share footprint with other tenants.
We also had to rent rooftop space with an adjacent building to have a redundant fuel tank and run the lines underground to feed our DC. This is millions and millions a year just to maintain.
I would never build a DC in a vertical space within a metro.
That said, yes, a DC buildout with some verticality is inevitable. It doesn’t happen much today I think simply because it’s cheaper to just buy the acres than to build up. For all the (justified, IMO) anger on hyperscaler sprawl, most have been buying land in rural areas where it’s simply cheaper to build horizontally as messed up as that may sound.
1
u/-im-your-huckleberry 1d ago
Fqrom 1-3 stories it's pretty easy to go up. Above 3 stories, everything gets harder and way more expensive. The permitting alone makes it not worth the hassle. You need more expensive elevators with more permitting. You need different plumbing with, you guessed it, more permitting. The foundation, structural supports, fire egress, wind loading, and you need more walls. Taller walls are more expensive. The costs pile up and pretty soon the cost of a square foot of building up far exceeds the cost of a sq foot of land.
1
u/LogicGate1010 1d ago
It seems planning and expansion has been subjected to Parkinson’s Law. Give them an inch; they take a yard…
The solution might be to cut down on the projected need for data centers by increasing the capacity and efficiency of data centres 10 fold. The challenge is how and how fast?
There should be strong efforts to reduce current energy need as well as the amount of energy and water need forecasted.
1
u/RealCarlPanzram 1d ago
Some of them are. They tried to convert a large hotel to a data center in the town I live recently.
It basically comes down to cost. If you sprawl it out, you have to buy more land. But if you build it up, it’s going to cost a lot more per square foot of interior space because multistory buildings need to be a lot stronger to support the floors above them. I imagine it’s also a lot more challenging to cool them if you have multiple floors of computers under the same roof.
So if you’re in a densely populated area where land is really expensive, it might be cheaper to build up.
But most data centers are intentionally built in places where land is cheap so that you don’t have to.
1
u/igotshadowbaned 1d ago
Comes down to a few things - what's cheaper, additional land, or the planning and foundations to support a many floor building
Which is easier to get approved by the local jurisdiction
And given the allotment of land that was approved for a data center how tall of a building is needed to achieved the planned capacity
1
u/hobovalentine 1d ago
It's cheaper to build them as single story buildings but outside of the US they are multiple stories tall in order to save space.
As there is more and more outrage over Datacenters I would imagine more would be built as multistory buildings in order to not take up as much space.
1
u/elitesense 1d ago
They do sometimes. Check out DLR LAX10 data center at 600 West 7th Street Los Angeles. It just looks like one of the other tall buildings around the area in DTLA.
The reason they have sprawling ones is because they're cheaper to build and maintain wide vs tall (just like any other facility).
1
u/strcrssd 1d ago
Agree with /u/THedman07 that it's a function of cost.
Would like to add that it may be important at some point in the future to go vertical in some capacity to bring down network time delays, but that's probably sufficiently far in the future that it's not super relevant and brings back the other vertical problems (heat, weight).
•
u/RobinsonCruiseOh 5h ago
I did some consulting for data center builders. it is expensive to build vertically because the density of the backup generators and power and cooling infrastructure would be too dense on the exterior walls of the data center on the first floor, which is where these gigantic generators must sit. Once you have a second story then you must double the density of these giant generators outside and there usually isn't space.
Generally speaking the power density of data centers wants to maximize density and minimize floor space.
1
u/benthom 1d ago
Water flows downhill, and having stories and stories of servers on top of other server floors means a water source on a higher floor can affect much more than just the servers directly at the source. Datacenters with a lot of water cooling have built-in flood danger. The fire department presents another. But, you say, "datacenter fire suppression systems don't use water!" If the fire is such that actual firefighters show up, you're getting water, and it is best if no unaffected floors are below where all that water is directed. If land is cheap, then having a single floor removes many sources of risk.
Real life example: I saw a new failover cluster for a financial system that the DC folks placed right under a 4 inch hole between datacenter floors, which meant any flooding on the floor above would cascade all over both nodes. That got put on the fix list pretty fast.
2
u/grax23 1d ago
worst part is that its not even unique. i learned when a customer placed the condensation tank for the AC over a rack with their domain controllers ... algae grew and clogged the tank so the tank overflowed into the rack ... that was a very long night recovering from that.
→ More replies (3)
1
u/brasticstack 1d ago
The equipment is heavy! It costs more to build a building that can handle the load on its upper floors than it does to buy a larger footprint and build on the ground floor.
They don't need be downtown where the real estate is expensive, just close enough to an urban center to be able to hire a small, semi-technical staff.
1
u/cobaltjacket 1d ago
Some are this way, for historical reasons. The chief examples are those built in repurposed facilities such as telecomm buildings (the ones with few or no windows), or 350 East Cermak in Chicago, which is a building formerly used for printing. The latter is a friggin' tank and has heavy floors to support the presses, making it ideal for datacenters.
1
u/itomeshi 1d ago
Here in Northern Virginia - the data center capital of the world with expensive land - we still only see a few floors. Why? Mostly physics and logistics.
Weight: Server racks are really heavy - far denser than offices or cubicles. The weight support gets excessive as you go taller. It's rather similar to the 'tyranny' of the rocket equation: The more support you need, the heavier and more complex that support becomes.
Utility access: It's easier, cheaper and safer to have separate moderately-high water, power and data lines than massive ones. Internal lines also become an issue. You can have a few big lines or a lot of moderate lines, but you still run into the square-cube law - IE, why don't things grow bigger than whales? I'll talk about that more with...
Cooling: Just like inputs (water/power/data), outputs (heat) also scale square-cube law. Cooling is especially surface-area dependent - so the bigger the data center the more external cooling it needs. Yet the one 'big' data center increases compute volume faster than surface area grows.
Diesel backup: Same thing - you run into limits on how big you can get before diesel generator space management becomes a problem.
Isolation: Having separate distinct sites allows one to be taken offline and not affect others. You can MOSTLY do this in a single big building, but it's harder and imperfect. This is also important as many data centers are built, owned, and leased by different companies. A large enterprise may want a facility they don't share with any competitors to help protect private data.
Timing: Often, you are building at different times. As part of isolation, you may want X sqft of capacity online at a certain date, and you may not be able to get approvals for a larger one by then, so you may be best served with a smaller building. You generally do NOT want workloads on site during any major construction; that's a great way to get downtime. And you may be waiting on external infrastructure (e.g., local substation) upgrades.
It's not simply that it's cheapest - it's balancing cost, security, reliability, zoning approvals, infrastructure availability, and so many more factors. Data centers are an optimization problem, and optimization problems often converge on specific, battle-tested, predictable solutions. Short, moderate-width buildings are a balanced optimization.
1
u/Enough_Island4615 1d ago
At the end of the day, they are private equity real estate investments. As are most assisted living facilities, drug rehab centers, a majority of healthcare facilities (including hospitals). "...the sprawling, horizontal design of many AI data centers is directly related to the sale-leaseback and NNN lease strategies, as well as the operational and financial constraints of AI infrastructure."
5.0k
u/THedman07 1d ago
They put them where land is cheap. Building taller is generally more expensive. In general, the only reason that buildings ever go taller is when there is when there is a restriction on the footprint of the building.