For most people, NVIDIA means GPUs. But at WeAreDevelopers World Congress in Berlin, CTO Michael Kagan barely mentioned them.
Instead, he focused on the infrastructure behind AI: networking, energy, software, data centers, robotics, and the massive engineering challenge of scaling it all. NVIDIA’s framing was clear: modern AI is no longer just hardware – it’s an “AI factory.”
Kagan compared AI to past tech revolutions. Electricity came through a wall socket. Cloud computing put infrastructure on demand. Now AI is changing the interface again making computers accessible through natural language instead of code.
Before AI, only a few million people on Earth could operate computers by programming them. Now everybody can program the computer and run the AI, and if you don’t know how to do it, just ask AI. It will tell you.
In other words, users no longer need to understand the technical details behind computing. If they’re unsure how to do something, AI can often explain it or do it for them.
Kagan framed AI less as a software category and more as a new computing layer that expands access to technology. The computer stays the same – the interface between people and machines is what’s getting much simpler.
One of the most interesting parts of the discussion focused on Mellanox, the networking company Kagan co-founded in 1999 before NVIDIA acquired it in 2020.
Mellanox was originally built for large-scale cloud computing, helping connect servers across massive data centers. But as AI workloads grew, that networking layer became even more important because the challenge was no longer just building individual processors, but making thousands of them work together efficiently.
That’s why Kagan says modern AI systems should be viewed as single computers made up of huge numbers of GPUs spread across racks and data centers:
The computer is not a box under the table anymore.
That made the Mellanox acquisition strategically important: it gave NVIDIA the networking needed to connect GPUs and train larger AI models at scale.
Kagan repeatedly used the term AI factory because, in his opinion, it better describes the role modern AI infrastructure plays.
Traditional data centers store data and process requests. AI infrastructure does something different: it uses huge amounts of data and power to train models and run inference across applications. “AI factories take the energy and the data and convert it to intelligence”, says Kagan.
He compared the process to a power plant generating electricity. It’s a metaphor, but it captures how NVIDIA sees AI infrastructure: as a production system where intelligence is the output.
When asked what an AI factory looks like, Kagan said:
The first thing you notice when you go to the AI factory is cables.
Those cables connect hundreds, thousands, and eventually millions of GPUs into one computing environment. NVIDIA breaks that challenge into two parts: scale-up, which links GPUs with NVLink, and scale-out, which connects those systems into massive clusters.
As those systems grow, networking becomes essential. Every processor has to stay in sync, and NVIDIA says lowering the cost of generating AI tokens remains one of its key goals.
AI performance used to improve mainly as hardware improved, but Kagan says the challenge is now more complex.
A big reason is the rapid growth of inference. Traditional computing follows a simple pattern: a user sends a request, the computer processes it, returns a result, and waits for the next instruction. Agentic AI works differently, with models constantly exchanging information with software tools and other services while completing tasks. That creates far more communication inside the data center.
Those interactions happen much faster than humans can issue requests, while Moore’s Law is slowing down. That means smaller and faster transistors alone are no longer enough to meet demand.
That is why building larger processors alone is no longer enough to keep up with growing AI workloads. According to Kagan, CUDA has become one of NVIDIA’s biggest long-term advantages because it allows developers to fully exploit the company’s hardware:
Chips without software are just expensive sand.
He explains that NVIDIA wants to give developers a stable platform while the hardware keeps evolving. Too much general hardware can be inefficient, but too much specialization can quickly become outdated. CUDA gives developers the flexibility to support new workloads without starting from scratch.
Power is another major limit. Kagan said electricity is now one of the biggest constraints on new AI data centers. NVIDIA is working on ways to connect multiple sites over long distances so they can act like one system, with training done where power is available and inference closer to users.
The session concluded with advice for young engineers: rather than recommending a specific programming language or AI framework, Kagan encouraged students to develop strong foundations in mathematics, physics and chemistry before specializing.
He also reflected about his childhood curiosity, saying he used to take new toys apart just to see how they worked. If he were starting school today, he said he’d seriously consider studying digital biology, since understanding the human body is still one of the most fascinating engineering challenges:
The most complicated machine that is out there is a human.
Even though the conversation touched on GPUs, networking and AI infrastructure, Kagan kept coming back to one idea: progress in AI depends on understanding complex systems. Whether that’s millions of processors, distributed data centers or the human body, engineering starts with curiosity and a desire to figure out how things work.
