xAI plans massive expansion with 1 million GPUs at Memphis facility

AI company xAI has planned a massive expansion to its supercomputer cluster in Memphis to install more than 1 million GPUs and compete against top players.

The Greater Memphis Chamber said on Wednesday that xAI is planning an expansion to the Memphis, Tennessee AI data center with at least one million graphics processing units (GPUs).

“When we announced six months ago that xAI would make Memphis its home for Colossus, we recognized it as our defining moment. Memphis has provided the power and velocity necessary for not just xAI to grow and thrive, but making way for other companies as well. We’re excited to welcome Nvidia, Dell, and Supermicro to the ‘Digital Delta’” said Greater Memphis Chamber President and CEO Ted Townsend.

Elon Musk founded xAI and has made major progress in the world of Large Language Models and generative AI with its chatbot named Grok. It was launched in November last year and has already received a few big upgrades in the past 1 year.

To increase capacity and improve performance, xAI has established a large facility in Memphis and recently enabled 100,000 Nvidia H100s GPUs to train the latest Grok models.

Dell supercomputer racks for xAI (Image Source – Michael Dell/X)

This facility has been built by the xAI team in collaboration with Nvidia, Dell, and Super Micro for server racks, networking technologies, and graphics cards.

It is revealed that the Supercomputer was built in just 122 days, which typically takes months or years to complete. On the other hand, the first rack onto the floor rolled out in just 19 days until training began.

Named Colossus, the supercomputer powers new Grok features including Generative images and image analysis. These two capabilities are now widely recognized in the world of generative AI.

That being said, the company has also improved the context length and increased the response rate for conversations.

Over the past two weeks, xAI has added numerous new features such as PDF analysis, you can check them here.

(source)