Nvidia Neotron 3 Ultra Review: Speed, Openness, and Coding Limits

Nvidia has released Neotron 3 Ultra, a new free and open AI model that has elicited a mix of delight, disappointment, and confusion. While benchmarks are available, practical testing reveals its true capabilities and limitations.

Initial Impressions: Speed and Coding Challenges

The model is remarkably fast, described as "blazing fast." However, initial coding experiments were problematic. When tasked with writing a light simulation program, a black screen resulted. Attempts to fix it yielded the same outcome. Manual debugging revealed mistakes, and even after correction, the output was unsatisfactory. A significant issue was the sheer volume of code generated—over a thousand lines for a task that a human-written solution (from the presenter's research) accomplishes in about 250 lines, rendering a functional scene.

Similar issues arose when attempting to generate a real-time strategy game, again resulting in a black screen or a minimal output like a single square. In contrast, Deepseek 4 Flash produced a much more impressive result with the same prompt.

Improvements and Alternative Use Cases

After reporting issues to Nvidia, some improvements were made, but the model still isn't ideal for complex coding tasks. However, its speed makes it excellent for other applications:

• Fixing broken installations: From the terminal, it performs excellently.
• Quick experiments: It can rapidly whip up small experimental code snippets.
• Organizing files: It handles file management tasks efficiently.

The presenter found themselves using it more and more for tasks other than challenging coding, highlighting its utility in various non-coding scenarios.

Openness and Licensing

Neotron 3 Ultra stands out for its openness:

• Weights are open.
• Research paper: The methodology behind its creation is openly published.
• Training data and recipes: These are being released for redistributable parts.

The licensing is particularly noteworthy. While Nvidia previously used a proprietary license (rated 7/10, allowing derivative works and commercial use with attribution and stricter patent grants), Neotron 3 Ultra uses the Open MDW license. This is essentially Apache 2.0 tailored for machine learning weights, rated 9/10. It allows almost everything, with the caveat that suing for infringement results in loss of the license. This is considered a significant improvement for open-source AI.

Running Neotron 3 Ultra

While the model is completely open for download and use, running it locally presents a challenge due to its size:

• Parameters: It has 550 billion parameters.
• Memory requirements: This necessitates hundreds of gigabytes of GPU memory, making local execution difficult for most users.
• Context window: It boasts a 1 million token long context window, which is beneficial for handling large codebases or extensive documents.

The presenter plans to use it on cloud platforms like Lambda GPU Cloud due to these resource demands.

Limitations and Future Desires

Neotron 3 Ultra is a text-only model and lacks vision capabilities. The presenter expressed a strong desire for a multimodal version.

Architectural Innovations

The model's efficiency and speed are attributed to several key innovations:

• Mixture of Experts (MoE): Although it has 550 billion parameters, only about 10% are active per token. This means specialized "mini-brains" are activated as needed, making it more efficient than running the entire model.
• Mamba layers: These layers address the "memory problem" in traditional AI systems. Instead of constantly re-reading information, Mamba layers process data once, taking highly compressed notes and discarding filler words. This allows for efficient processing of massive datasets.
• Low-precision numbers (NVFP4): The model uses low-precision numbers, reducing the computational load during operation.
• Parallel token generation: Instead of predicting tokens one by one, it uses multiple heads to draft several future tokens simultaneously, contributing to its speed.

Conclusion

Neotron 3 Ultra represents a significant step forward in open AI models, offering blazing speed and impressive openness, particularly in its licensing. While it struggles with complex coding tasks, its utility for other applications, combined with its innovative architecture, makes it a valuable addition to the AI landscape. The presenter emphasizes the importance of open science and open models for advancing humanity.