DeepSeek 4 Pro: Open‑Source Model with Faster Generation

DeepSeek has released DeepSeek 4 Pro, a new version of their AI model, which is a significant improvement over its predecessor, the preview version. This new iteration, identified as 0813, demonstrates a much better understanding of 3D structures, as illustrated by its ability to accurately render a Rubik's Cube, unlike the flash version which showed many missing parts and blackness. The quality of its output is described as "inching closer and closer to fable quality."

Key Features and Availability

DeepSeek 4 Pro's weights are available for free under an MIT license, which is a major advantage for the AI community. This means anyone can run the exact same model at their own cost. While hosting it at home requires substantial hardware, several third-party hosts offer access to the model, competing on price. DeepSeek itself also offers hosting, though they have recently increased their prices significantly (2.5 to 5 times the previous rates).

Architectural Improvements

Despite using the same underlying architecture as the previous version, DeepSeek 4 Pro shows massive improvements in performance. This advancement is attributed to sophisticated post-training techniques.

Specialist Models and Distillation

A key innovation involves the creation of several specialist models during post-training, focusing on areas like mathematics, coding, and agentic work. These are not "experts" in the mixture-of-experts sense (which are small pieces within a single neural network) but rather separately trained model checkpoints.

The process then involves "distillation," where more than 10 of these specialist models act as "teachers" to train one final "student" model. The student model learns by comparing its proposed actions with those of the teachers and then adapting its internal structure to align more closely with the teachers' responses. This multi-teacher approach significantly enhances the student model's capabilities.

Faster Generation

DeepSeek 4 Pro also incorporates a novel technique for faster generation. Instead of predicting one token at a time, it drafts several tokens ahead, a method that is significantly more efficient than previous approaches. This results in up to 78% faster generation for V4 Pro, offering a real and measurable speed-up in practical use.

Impact and Future Implications

The rapid development and release of DeepSeek 4 Pro highlight the fast-paced nature of AI research. The underlying research paper for this technology was published only six weeks prior to its widespread availability, demonstrating how quickly cutting-edge research can be implemented and made accessible.

The open-source nature of DeepSeek 4 Pro, with its MIT-licensed weights, is seen as a significant step forward for open science and research. It empowers users to run the model without restrictions or fear of being downgraded to different models based on specific keywords. This accessibility is expected to push other frontier AI labs to innovate and provide even better solutions more quickly.

The speaker also plans to discuss DeepSeek's "no agent harness," which is described as a novel and powerful design.

Running AI Models with Lambda

For those interested in running AI research papers, training models, fine-tuning existing ones, or performing inference for text-to-image or video generation, Lambda.ai is recommended. It provides powerful Nvidia GPUs, enabling users to reproduce AI research papers, test ideas, and run DeepSeek chatbots or agents quickly and reliably.