Elon Musk’s xAI Is Using 100,000 NVIDIA H100 GPUs To Train Grok 3 AI Model

Elon Musk has announced that the next-generation Grok 3 LLM model has been trained using an impressive 100,000 NVIDIA H100 AI GPUs.

It seems that every AI-focused company is racing to release advanced versions of their LLM models to meet the increasing demand for AI chatbots. Joining this competition is Elon Musk’s AI startup, xAI, which previously introduced the “Grok” series of LLM models. These models are specifically designed to serve as AI assistants for premium users on X.

The firm has released updated versions of the original model, and xAI is now gearing up for Grok 2. However, in his latest tweet, Elon Musk has started promoting the next-generation Grok 3 model, asserting that it will be significantly larger than its predecessors.

IMG 2299 1

The revelation about the hardware used to train Grok 3 is both surprising and exciting. Utilizing 100,000 units of NVIDIA’s high-end H100 AI GPUs for model training is expected to yield unprecedented results. To put this in perspective, it’s rumored that OpenAI’s GPT-4 LLM model was trained on 40,000 of NVIDIA’s older A100 AI GPUs. Given the advanced technology and greater quantity of H100s, the capabilities of the Grok 3 AI model are likely to be extraordinary.

In a previous tweet, Elon Musk mentioned that training LLMs using internet data is challenging and demands significant computing resources. He noted that the upcoming Grok 2 AI model, scheduled for release in August, addresses some of these issues, and the next-generation Grok 3 LLM model will further enhance these improvements, ultimately becoming a significant product for xAI.

IMG 2300

Additionally, Musk announced plans to acquire NVIDIA’s flagship Blackwell B200 AI accelerators for xAI, valued at approximately $9 billion, highlighting the substantial investment in this venture. With Grok 3 utilizing 100,000 units of NVIDIA’s H100s, the training costs alone are estimated to be around $3 billion, showcasing Musk’s commitment to pushing the limits with xAI.

It will be fascinating to see how xAI’s ventures contribute to the AI landscape. By all indications, Musk’s AI startup could be poised to become a major player in the industry.