Nvidia recently reported over $19 billion in net income for the last quarter, but this impressive figure has not fully eased investor concerns about the company’s growth prospects. During an earnings call, analysts raised questions about how Nvidia would adapt if AI developers began using new techniques, such as test-time scaling, to enhance their models.
Test-time scaling, a method exemplified by OpenAI’s o1 model, allows AI systems to deliver more accurate results by allocating extra computing power during the inference phase—when the AI processes a user’s input. This approach shifts focus from the traditional training phase to inference, leading to discussions about the future role of Nvidia’s chips.
When asked about this shift, Nvidia CEO Jensen Huang called test-time scaling “one of the most exciting developments” and emphasized Nvidia’s readiness to support this new direction. Huang’s comments echoed Microsoft CEO Satya Nadella, who recently described o1 as a significant innovation for the AI industry.
This shift has important implications for the semiconductor market. Nvidia’s GPUs are widely regarded as the standard for training AI models, but the inference phase is becoming increasingly competitive, with startups like Groq and Cerebras producing specialized inference chips.
Despite reports of slowing progress in generative AI, Huang reassured analysts that AI developers continue to improve their models by using more data and computing power during the pretraining phase. He noted that while Nvidia’s current workloads are predominantly in training AI models, the growth of inference will become more significant as AI adoption increases.
Huang underscored Nvidia’s advantages in the inference space, citing the company’s scale and the reliability of its CUDA platform. He expressed optimism about the future, envisioning a world where widespread AI inference signifies AI’s ultimate success.
“Everyone knows that if they build on CUDA and Nvidia’s architecture, they can innovate faster,” Huang said, highlighting Nvidia’s commitment to maintaining its leadership in AI computing.
While some experts, including those from Andreessen Horowitz, suggest diminishing returns from existing AI training methods, Huang remains confident in Nvidia’s ability to adapt and thrive in a rapidly evolving industry.
