NVIDIA’s Co‑Design, CUDA Moat, and Token Factories Shape AI

Large‑scale AI problems no longer fit inside a single computer accelerated by one GPU. To achieve speedups greater than the number of added machines, algorithms must be refactored, data and models sharded, and pipelines distributed. Amdahl’s Law reminds us that the portion of work that cannot be parallelized caps overall speedup, even with massive compute. NVIDIA’s response is “extreme co‑design,” an effort that optimizes every layer of the stack—architecture, chips, systems, system software, algorithms, and applications—while also integrating power, cooling, networking, and storage. The goal is to deliver more tokens per second per watt than the previous generation, even as AI workloads consume gigawatt‑scale power.

From Accelerators to CUDA

NVIDIA began as an accelerator company, but a narrow application domain limited its R&D impact. The strategic decision to embed a programmable parallel platform—CUDA—on GeForce GPUs consumed significant profits but created a massive install base. An install base of millions of PCs gave developers a reason to target CUDA, even if end users did not actively use it. This strategy turned NVIDIA into a computing platform company, leveraging programmable pixel shaders, IEEE‑compatible FP32, and Cg as stepping stones toward CUDA’s success.

Leadership, Vision, and Manifesting the Future

NVIDIA’s organization mirrors its environment, with Jensen Huang’s direct staff of about 60 people and decision‑making that relies on collective problem solving rather than one‑on‑one meetings. Huang gradually shares the potential of new ideas, shaping belief systems over time so that major announcements—such as the acquisition of Mellanox or an all‑in bet on deep learning—receive near‑universal buy‑in. Keynotes at GTC are used to align industry partners and employees around a shared vision. The company views its success as a national asset, contributing tax revenue, re‑industrialization, and technology leadership for the United States.

AI Scaling Laws and Emerging Blockers

Four scaling laws structure NVIDIA’s view of AI progress:

1. Pre‑training scaling – originally limited by high‑quality data, now overcome by synthetic data generation.
2. Post‑training scaling – focuses on data quality and augmentation.
3. Test‑time (inference) scaling – the “thinking” phase, far more compute‑intensive than training.
4. Agentic scaling – AI agents spawn sub‑agents, creating teams that multiply capability and generate data that feeds back into the other three laws.

The industry has moved from a data‑limited regime to a compute‑limited regime, and intelligence is expected to continue scaling with compute.

Power, Supply Chain, and Energy Solutions

Extreme co‑design also targets energy efficiency, aiming for continual improvements in tokens per second per watt. The grid, built for worst‑case demand, often leaves excess capacity idle. NVIDIA proposes contractual arrangements and data‑center architectures that allow graceful degradation of service during peak grid stress, enabling utilities to segment power delivery. Supply‑chain bottlenecks—such as EUV lithography at ASML, CoWoS at TSMC, and HBM at SK Hynix—are addressed through close collaboration with CEOs of partner firms, influencing investments in low‑power memory and high‑bandwidth interconnects. NVIDIA’s supply chain spans roughly 200 companies.

The AI Factory Paradigm and Token Economics

Computing is shifting from a retrieval‑based, file‑oriented model to a generative, context‑aware system that produces “tokens” in real time. These tokens are treated as commodities, comparable to iPhones in market impact, with potential pricing around $1,000 per million tokens. Data centers become “factories” that directly generate revenue‑producing output, turning computation from a storage unit into a product‑generation unit. The exponential demand for token factories is expected to drive a hundred‑fold increase in computation’s share of global GDP.

NVIDIA’s Moat: CUDA Install Base and Ecosystem

The single most valuable asset is the CUDA install base, built on GeForce GPUs and trusted by millions of developers. This base creates a virtuous cycle: developers build software for CUDA, users adopt it, and NVIDIA continues to improve the platform. The ecosystem integrates NVIDIA’s architecture vertically (chips, racks, pods) and horizontally (clouds, supercomputers, edge devices, cars, robots, satellites), ensuring that a single architecture underpins a wide range of products and services.

Global Landscape: China, Open Source, and Modality Diversity

China contributes roughly half of the world’s AI researchers and drives rapid innovation through open‑source collaboration and provincial competition. NVIDIA’s vision for open‑source AI models—exemplified by the 120‑billion‑parameter Nemotron 3 Super—aims to diffuse AI across industries and countries. The company also emphasizes that AI extends beyond language to biology, chemistry, and physics, requiring specialized models for each modality.

Partnership with TSMC

TSMC’s success stems from a blend of cutting‑edge technology, high throughput, and world‑class customer service. NVIDIA’s long‑standing relationship with TSMC, built on trust rather than formal contracts, has produced tens to hundreds of billions of dollars in business over three decades. The partnership leverages TSMC’s ability to orchestrate the dynamic demands of hundreds of global customers while delivering high yields and reliability.

Future Value and the Nature of Compute

NVIDIA’s growth is portrayed as “extremely likely and inevitable” because the shift to generative computing creates new markets rather than stealing existing share. Tokens produced by AI factories become valuable commodities, and the world will need an exponential number of such factories. The company envisions revenue potentially exceeding $3 trillion, a scale not limited by physical constraints but by the breadth of opportunity and the ability to scale the supply chain. GTC events are expected to make this future more concrete for investors and partners.

Economic Impact and Computational GDP

Computation’s contribution to global GDP is projected to become 100 times larger than today. As AI factories turn tokens into revenue, NVIDIA’s role as the primary infrastructure provider positions it to capture a substantial share of this growth. The company’s supply chain, shared by about 200 partners, is designed to handle the anticipated surge in demand for compute power.

AI Agents as the “iPhone of Tokens”

Agents represent the fastest‑growing application in history, likened to the “iPhone of tokens.” They continuously interact, report task completion, and request new assignments, effectively forming a self‑organizing workforce of AI sub‑agents. OpenClaw and Claude Code are highlighted as early examples of this paradigm, enabling programmers to “talk” to a laptop as if it were a human colleague.

Leadership, Resilience, and Pressure Management

Success is attributed to working harder, enduring suffering, and systematically breaking down problems into manageable parts. Jensen Huang emphasizes self‑discipline (“Did you do it? … stop crying about it”) and the importance of sharing worries to reduce psychological low points. Public reasoning, open‑mindedness, and tolerance for embarrassment are presented as essential traits for navigating near‑death moments for the company.

Evolution of Work and Skills in the AI Era

GeForce introduced many users to NVIDIA during their teenage years, later transitioning them to CUDA, Blender, and Autodesk. AI tools like DLSS 5 illustrate how generative AI can augment creative workflows without replacing core artistic intent. The definition of coding is evolving toward specification and architecture definition, expanding the pool of people who can describe software from 30 million to potentially 1 billion. Employers increasingly value AI expertise across all professions, from accountants to electricians, as AI automates mundane tasks and elevates the creative and strategic aspects of work.

Defining Intelligence vs. Humanity

Intelligence is described as a commodity—perception, understanding, reasoning, and planning—that can be measured and scaled. Humanity, by contrast, encompasses character, compassion, and lived experience. While AI can recognize emotions, it does not feel them. The distinction underscores the belief that intelligence can be commoditized without diminishing the value of humanity.

Mortality, Legacy, and Knowledge Transfer

Continuous knowledge transfer—passing on information, insight, and skills in every meeting—is emphasized as the preferred method of preserving legacy. The desire to “die on the job” reflects a commitment to relentless contribution rather than traditional succession planning.

Hope for Humanity

Optimism is expressed in humanity’s capacity for kindness, generosity, and compassion. Expectations include ending disease, drastically reducing pollution, and achieving short‑distance travel at the speed of light via humanoid spacecraft. Understanding the biological machine and cracking theoretical physics are viewed as achievable within the next five years, reinforcing confidence in a brighter future.