xAI starts training 100k Nvidia H100s AI cluster for next-gen Grok

Generative AI firm xAI has started training its largest training cluster in Memphis with 100,000 Nvidia H100 AI chipsets for the next-generation Grok AI chatbot.

Elon Musk, founder of xAI thanked xAI, X (formerly Twitter), Nvidia, and other companies for supporting the development. He also revealed that the training had already started at 4:20 am local time.

The AI firm has been planning this upgrade since earlier this year. The plan received a boost with recent $6 billion series B funding. This fund will be used to advance the Grok Large Language Model (LLM) and other future products.

Grok is currently integrated into X social media sites and users need to purchase a premium or premium+ plan to use this chatbot. The new supercluster was announced a few months ago and it will be used to run Grok 2 and to train Grok 3 coming later.

Dell has also participated in this new data center with its custom supercomputer racks. These racks are designed with Nvidia software solutions for AI enterprises. However, only half of the total racks are designed by Dell and other companies, and the other half are developed in-house by xAI.

Dell supercomputer racks for xAI (Image Source – Michael Dell/X)

Using, H100 is the initial choice for the company because it is the current most popular all-around AI solution for LLM and machine learning. The company is preparing Grok 2 on this platform and it could release in August or September. The architecture is supported by liquid cooling technology provided by Supermicro computers.

The company will upgrade the architecture with 300,000 B200 chips and CX8 network. This cluster will be online by next summer. Using a new chipset will improve processing power and help the company to deliver better performance.

With these new advancements, the company will rapidly increase its energy consumption to fulfill all of its energy requirements.