2.8 trillion-parameter model is just the beginning: Kimi K3 is taking large-model competition toward “AI-making-AI”

Author: Climber, CryptoPulse Labs

On July 16, MoonShadow (Mysterious Dark Side of the Moon) officially launched its next-generation open-source model Kimi K3. The model has 2.8 trillion parameters, a 1 million Token context window, natively supports visual understanding, and uses technologies such as Kimi Delta Attention and Attention Residuals.

This is the world’s first open-source model in the 3 trillion-scale category. Although Kimi K3’s overall performance still lags behind the strongest closed-source models such as Claude Fable 5 and GPT-5.6 Sol, it has demonstrated leading-edge levels across multiple benchmarks, and MoonShadow claims that its overall performance is consistently better than other tested models.

More attention is that Kimi K3 also independently completed the design of a chip. For a large model, starting to try designing the hardware needed to run AI may be even more worth attention than the 2.8 trillion parameters themselves.

I. Behind 2.8 trillion parameters: the large-model race is shifting from scale to efficiency

In the past few years, the most easily understood metric in the large-model industry has been parameter count. From tens of billions to trillions, and then to the trillion-scale, parameter scale has almost become an important symbol for measuring model capability.

But once model scale reaches 2.8 trillion parameters, the real question is no longer “how big the model is,” but: how is such a massive model trained? How many parameters are required to participate in computation for each task? How can the model gain stronger capability while controlling operating costs?

The answer provided by Kimi K3 is further expanding a sparsity-based architecture.

According to MoonShadow, Kimi K3 adopts Mixture of Experts, an MoE architecture. The model has 896 expert modules, but for each task it only activates 16 of them.

This means the model can have huge knowledge capacity, but it doesn’t need to call all parameters every time. Like a super organization with 896 specialized departments—when facing different problems, it only needs to mobilize the most relevant 16 departments.

The core value of this architecture is that the model’s total scale and the cost of single-run computation can be separated.

In the future, the large-model competition may not be about who has more parameters, but who can effectively call more parameters at a lower cost.

Another core innovation of Kimi K3 is Kimi Delta Attention (KDA). In the traditional Transformer architecture, when handling ultra-long text, computational complexity and memory pressure increase significantly. The goal of KDA is to improve the efficiency of the model when processing long-sequence information.

At the same time, Kimi K3 introduces Attention Residuals, a mechanism of attention residual connections. Traditional models typically pass information gradually layer by layer—information accumulates into later layers—but this can also introduce redundancy and attenuation.

Attention Residuals tries to let the model jump across different depths, selectively calling information from earlier stages.

If traditional models’ information flow is like a river running from the starting point to the destination, then Attention Residuals is more like building an information retrieval system along the way—allowing the model to re-call information from different depths depending on the task.

MoonShadow states that compared with Kimi K2, Kimi K3 achieves about a 2.5x improvement in overall scaling efficiency.

This indicates that the AI industry is moving from “the bigger the better” to “how to turn larger scale into higher efficiency.”

The significance of Kimi K3 is not only releasing a 2.8 trillion-parameter model, but also further raising the upper limit of open-source model scale.

In the past, open-source models were often seen as pursuers of closed-source models. Now, open-source models are trying to prove that extremely large-scale models can also be made public, studied, and further developed by others.

II. From chatbots to digital workers: Kimi K3 targets complex work

If 2.8 trillion parameters is the easiest-to-spread label for Kimi K3, then its true product direction is actually long-horizon tasks.

In the past, AI assistants mostly answered questions. Users asked questions, and the model provided answers. When asked to write code, it returned code. When asked to summarize an article, it generated a summary.

But real-world complex work often cannot be completed with a single round of question-and-answer.

A researcher may need to read papers, organize data, build models, run experiments, analyze results, and then write a report. A programmer may need to read a large number of files, understand the project structure, modify code to run tests, locate errors, and iterate continuously.

These tasks share common characteristics: long cycles, many steps, and large amounts of information—and they also require adjusting the next actions based on intermediate results. This is exactly what Kimi K3 is trying to solve.

In a case shown by MoonShadow, Kimi K3 completed a task in computational astrophysics research. By reading and cross-validating more than 20 papers, it carried out numerical computations, evaluated hundreds of equations of state, found inconsistencies in published formulas, and generated more than 3,000 lines of Python code and an interactive HTML dashboard.

The official said this task took about two hours, whereas under traditional conditions it might take one to two weeks for an experienced research team to complete.

This does not mean AI can replace researchers. The most important part of scientific research is often proposing questions, judging hypotheses, and interpreting results.

But Kimi K3 demonstrates an important shift: AI is gradually moving from helping humans complete a single step to autonomously completing an entire workflow. This is the difference between the Agent era and the traditional chatbot era.

Traditional chatbots solve whatever you ask and then answer. Agents solve what you tell them as the goal: they break down the task themselves, call tools, execute steps, check results, and continuously correct.

Kimi K3’s 1 million Token context window is important in this process.

For large code repositories, research reports, corporate materials, and complex project documentation, if the model can understand more information at once, it means it doesn’t need to forget the context as frequently and users don’t need to repeatedly provide background.

At the same time, Kimi K3 natively supports visual understanding, allowing AI to form a more complete closed-loop workflow.

For example, after AI writes code, it can check the webpage results when running; after AI creates a PPT, it can check the page layout; after AI generates content, it can also use visual feedback to judge the result.

In the past, AI was more like writing code with its eyes closed. In the future, AI can form a loop of understanding the task, generating results, observing results, finding problems, and modifying results.

MoonShadow also extends Kimi’s capabilities to scenarios such as Kimi Work, Kimi Code, and Kimi API, targeting research, documentation, slides, tables, dashboards, and complex programming tasks respectively.

In the future, the AI with real commercial value may no longer be the model that answers the most questions, but the one that can complete the most work.

Traditional software requires users to learn complex operating processes. The goal of an AI Agent is to connect search, databases, programming, data analysis, and office tools so that users only need to describe the final goal.

This means competition in the software industry in the future may no longer be about who has more tools, but who has a stronger AI execution system.

III. What deserves the most attention is not the model, but the fact that AI is starting to design chips

The most shocking part of Kimi K3 might be that it independently completes chip design.

According to information disclosed by MoonShadow, during a 48-hour autonomous run, Kimi K3 used open-source EDA tools and the Nangate 45nm process library to complete chip design, optimization, and verification for a small model tailored to its own architecture.

This does not mean Kimi K3 can already independently commercialize mass production of modern advanced-process AI chips. The 45nm process has a huge gap compared with today’s most advanced AI accelerators, and chip production from design to mass production involves complex IP, process, manufacturing, packaging, and supply-chain systems.

But this attempt still has important significance, because chip design is not just writing code—it requires handling multiple stages such as logic design, synthesis, placement and routing, timing analysis, power optimization, and physical verification.

In the past, AI in the chip industry was mostly used to assist engineers with local tasks, such as optimizing placement, predicting timing, and discovering design flaws.

What Kimi K3 shows is another possibility: AI is no longer only using tools, but starting to autonomously organize tools to complete a full engineering workflow.

This is very similar to the development path of AI writing code. Earlier AI could only generate a small snippet of code. Later it could write full programs. Then it could read codebases, run tests, and fix bugs. Now, AI is starting to try designing the hardware needed to run AI.

This could form a new AI self-enhancement loop: AI helps design stronger chips, stronger chips train stronger models, and stronger models help design the next generation of chips.

Even more noteworthy is that Kimi K3 also demonstrates the ability to autonomously develop a GPU programming system.

According to MoonShadow, Kimi K3 developed MiniTriton, a compact compiler system similar to Triton, including its own intermediate representation layer, optimization workflow, and PTX code generation workflow.

This shows that the boundaries of AI capability are expanding from using software to creating software tools.

In the future, the model itself may directly participate in chip optimization, compiler development, operator adaptation, and system tuning—this may be Kimi K3’s most important strategic value.

It is not just a model product; it is exploring an AI-native R&D mode. From model to compiler, from algorithm to chip, from data to applications, AI is gradually becoming part of the broader infrastructure.

Of course, chips designed autonomously by AI still require strict verification, and research results generated by AI also need review by professionals. Errors are also possible when AI autonomously executes complex tasks.

But Kimi K3 has already released an important signal: AI is gradually becoming a subject that participates in creating the next generation of AI, rather than merely an object being created.

Conclusion

The release of Kimi K3, on the surface, is a model upgrade—but beneath it, it represents a change in the logic of large-model competition.

From larger parameter scale to more efficient architectures. From answering questions to completing complex work, and then to autonomously developing compilers and designing chips—AI is increasingly participating in creating the next generation of AI.

2.8 trillion parameters may just be a number. What is truly worth paying attention to is that AI is starting to try designing its own future.