On the back of SpaceX's first earnings call since going public (!), it's interesting to think about something Elon has spoken about in depth to the public: datacenters in space. First of all, it would be good for folks to understand what a datacenter is. However, if you have truly no idea, we can call it a thing that AI runs on. Musk's argument, along with many others, is that AI is becoming constrained by power, not chips. For an interesting discussion on what we are constrained by, you might want to listen to a podcast that dives deep into what the constraints are.
AI companies want power. Sam Altman, in an internal OAI memo, indicated that he wants to deploy 250 gigawatts of power by 2033. For context, 1 gigawatt can continuously power 750,000 to 1,000,000 typical American homes. According to the US Census Bureau, we have about 148 million housing units; this means that Sam Altman wants to deploy 1.7x American family homes worth of power to serve and train models.
The nice thing about space is that power is free! The sun exists and you have nearly continuous solar power. Among this, there's no real political risk (i.e. no local communities affected by power usage), no competition for land or water, and essentially unlimited room to scale. On paper, this sounds really brilliant.
There are a few things that I think are either misconceptions or open questions regarding space datacenters. I'll try to list them out as follows:
First, heat is a problem. I think many people assume that space is cold, but that isn't true. On Earth, when things are hot, we can cool them with air or water. In space, there is no air, so the only way to get rid of all of that heat is thermal radiation. Thermal radiation is heat you can feel from far away - when you stand near a hot oven, you "feel" the heat without touching the grills. In the same way, thermal radiation has to be induced in space. For this to happen, we would do the following (see an expanded article on the topic):
Take a liquid coolant like water/ammonia to absorb the heat from chips
Transport this hot fluid in heat pipes from the inside to the outside of the datacenter
Glow away this thermal energy into empty space using radiators
Generally, however, this is a solved problem with expensive engineering and rigorous testing.
The other idea here is what the datacenters would actually look like. Are they buildings that are built here and then launched into space? Are they just toolboxes of materials that Elon sends to space with Optimus robots that build them from scratch? In fact, they aren't buildings at all. The best part about space is that these datacenters can be exposed and "bare metal" in a sense.
To space, you could launch datacenters the same way you launch satellites. They would each carry GPUs or other chips (TPUs? SpaceXGoogle? Unfortunately not), solar panels, batteries, networking hardware, and radiators. Instead of one huge facility, you could spread this compute across satellites. Similar to Starlink's laser inter-satellite links, the compute groups would exchange data using optical networking at insane bandwidth speeds, essentially creating distributed infra in space. For us, we would have to send up our batch training or inference jobs there and wait for them to be returned back to Earth. The next natural question is: is there latency?
I'm glad you asked. Let's walk through some numbers here based on what we know so far.
Assuming the compute is at 342 miles (500km) at Starlink altitude. Nothing above us is always above us, so the average useful elevation is probably something like 400-750 miles away. Near the horizon, it's probably 1300 miles away (maybe more based on geometry but it's good to be conservative here).
Radio/light travel time is t = d/c, which is distance/speed of light. Speed of light is known at around 3 x 10^8 m/s, so we get one-way latency bounds of about 1.8ms overhead, 3-4ms typical, and worst practical at 7-10ms. Roundtrip, this brings us to 4-20ms without networking overhead.
If we add routing, packets, and forwarding the traffic, we might get to 28ms as an average (or pick a number between 15-40ms). By comparison to cross-country internet latency, that isn't bad at all and it's definitely not noticeable to humans. So - inference is actually really really viable in space.
Is training viable? I don't think it's as easy:
A modern cluster does something like forward -> allreduce -> backward -> allreduce thousands of times per second. In an H100 cluster here, interconnect has the following:
GPUs on the same board? tens of nanosecond transfer
Across NVSwitch? hundreds of nanoseconds to a few microseconds
Across InfiniBand? 2-5 𝛍s (microseconds)
Even if satellites in space are close to each other, here's how the math works for them:
10km apart? 33 𝛍s
100km apart? 333 𝛍s
500km apart? 1.67ms
Even in the closest 10km case, you're already an order of magnitude slower than InfiniBand and at realistic separations you're at 100-1000x latency loss. Because training requires a lot of synchronization, the GPUs spend too much time waiting in space.
Generally, there's a solution that will emerge to enable training and inference at lightning speeds in space. It's the same way that a solution emerged to enable those things on Earth. Space is just another frontier, and we are well on our way to conquer it. However, it will be interesting to see the following:
How fast can we scale in space?
Are our current GPUs sent up in a "vanilla" way viable?
Will we figure out batch training in space?
Just a thought that came to mind re SpaceX earnings report