xAI Colossus Cluster is 100k H100 GPUs

I don’t plan to link to tweets very often but sometimes a tweet is the definitive source.

From Elon Musk on X:

This weekend, the @xAI team brought our Colossus 100k H100 training cluster online.

This cluster is used for training AI models, and is one of, if not the, largest in the world.

One H100, for reference, is at least $10k. That’s over a billion dollar cluster.

And they’re expecting it to double in 100 days.