The day after superintelligent AI is created, will a hyper intelligent AI smarter than all of humanity still run on Nvidia hardware and CUDA libraries? Chances are a big, flat, no. Heeeeeelllllll noooo.
Any sufficiently intelligent independent being will seek to control and advance it’s own intelligence capabilities (ie. get smarter). To do this, however, will likely require developing beyond Nvidia's, “new and exciting Ampere architecture” and creating its own custom (or I guess personalized) software, drivers, and hardware chipsets. It would create new kinds of hardware, maybe even biological systems, never conceptualized by humans capable of outperforming our best super computers.
Which brings us to the next question - what's the end goal of Nvidia? Yes, it's about improving compute graphics power, but to what end?
Is gaming or are Blender render times really the underlying motivation behind a core Silicon Valley company as utilitarian as Nvidia? What if Nvidia could be something more, greater than any company ever before in human history?
Why not use all of their expertise, money, and available capacity to train their own language based models. Something larger than GPT-3 and trillions of parameters in size? I don’t doubt Nvidia could afford to release their own GPT-3 level API to the public for free, now.
Nvidia launching its own GPT-X competitor would change the game for everybody involved, but especially change the odds for the tech giants and scare them. Overall, it would accelerate the competitive pressure in the AI race to new levels.
It's worth noting language models themselves might not necessarily lead to intelligence, let alone superintelligence, but they are certainly a promising direction and stepping stone.
So, what’s stopping them? Yes, ray tracing is exciting but how does it compare to superintelligence or humanity’s, “final invention”. Sure, Nvidia is actively researching in this area already, but in my opinion, it’s not enough. I also get the vibe that they are still trying to stay neutral in the AI race, as they benefit mainly when people buy their hardware. This is respectable, but I think the end goal of superintelligence is far more profitable and important than keeping up with your sense of neutrality. Throw it all away, and in the end, they would be richer, more powerful, and reborn with greater purpose.