In the face of open source, OpenAI has no moat: Today, the Android moment of the big model is here

Original source: Zhiwei

Image source: Generated by Unbounded AI‌

In the past six months, OpenAI has amazed everyone with GPT.

People generally believe that the birth of GPT is the “iPhone” moment of the new era, it will become the door key of the new era of AI large models just like the iPhone brought people to the era of mobile Internet.

However, in the booming process of the ** mobile Internet, not only the achievements of the iPhone, but also the achievements of Android and the major Android camp mobile phone manufacturers behind it. **

Even, from the perspective of shipments alone, the contribution of the Android camp to the mobile Internet era seems to be greater.

**And now, the “Android moment” belonging to the AI big language model is coming. **

On July 18 local time, in the early morning of Beijing time, Meta released the latest generation of open source large model Llama 2.

According to the public data on Meta’s official website, the Llama 2 model series released this time includes variant models with three parameters of 7 billion, 13 billion and 70 billion.

Llama 2 has been trained with 2 trillion tokens and has more than 1 million manually annotated data. Compared with Llama 1, Llama 2 has 40% more training data and twice the context length.

After this upgrade, according to the published papers of Meta, although Llama 2 is still inferior to GPT-3.5 in various large-scale model tests, in comparison with the running scores of the current open-source large-scale model, it has already achieved a level that is generally slaughtered. Performance.

Comparison of running scores of Llama 2 with GPT and PaLM

Comparison of Llama 2 and other open source large model running scores

Perhaps you will feel scornful when you see that Llama 2 is inferior to GPT-3.5, after all, the latter has evolved to version 4.0.

However, you have to understand that when Android was first launched, it was also very crotchless, and open source has allowed the Android camp to compete with the iPhone now. **

Therefore, the biggest highlight of the release of Llama 2 is actually:

**Meta goes a step further on the open source basis, allowing the model to be freely used commercially. ** (Products with more than 700 million monthly active users need to apply for commercial permission separately, but few companies can meet this standard)

Yang Likun, winner of the Turing Award and chief scientist of Meta, also said on Twitter that this will change the pattern of the big language model market:

The editorial department of Zhiwei also contacted Zhiyuan AI Research Institute, which is quite influential in both the academic circle and the open source community. Their evaluation is:

**Open source is the only way. It is not unreasonable to say that the release of Llama 2 is the “Android moment”. Using open source to occupy the market is a competitive strategy. **

Corresponding to Llama 2’s commercial open source, OpenAI’s attitude towards open source has always been ambiguous.

Open source, or the open source movement, officially began in the late 1990s. Participants believed in open source code, information sharing, and free use of software.

The rise of Android and the popularity of GitHub are all inseparable from open source. The Linux system was also born in this wave of movement. Today’s routers, switches, smart washing machines, smart rice cookers, switches, servers and other devices are equipped with almost all kinds of Linux systems.

Including the successful landing of the American Mars rover a few years ago, it also symbolizes that Mars has become the second planet where the number of Linux computers exceeds that of Windows.

It is no exaggeration to say that open source today has changed the collaboration and innovation model of software and changed the technology landscape.

But in the field of large language models, the leader OpenAI seems to run counter to the Open in the name.

Even though the founder of OpenAI, Sam Altman, has repeatedly mentioned in public that he will open the source code of GPT-3 and GPT-4 in the future,** this is still a blank check so far. **

Even after Musk withdrew from OpenAI, he also publicly bombarded OpenAI many times, dissatisfied with OpenAI’s huge profits and gradually closing the source.

Sutskever, the chief scientist of OpenAI, responded that premature open source will make OpenAI lose its leading position in technology, and the power of the large language model is huge, and open source may have security risks.

Musk questioned whether OpenAI is not Open

Unlike OpenAI’s timidity in the face of open source, Meta is a staunch supporter of open source, and has promoted the historical process of AI to a certain extent.

And Meta’s decision today must have something to do with Meta’s chief scientist, Yang Likun.

As early as 2014, Zuckerberg realized the prospect of AI. After DeepMind was finally acquired by Google, Zuckerberg turned around and found Yang Likun, a bigwig in the AI academic field.

Yang Likun, who had worked at the famous Bell Labs, was one of the developers of the “Convolutional Neural Network”. At that time, he was teaching in New York.

According to VOX reports, in order to get Yang Likun, a great horse, Zuckerberg promised that scientific research results must be open source, the laboratory must be built in New York, and the experimental team does not need to consider profitability.

Yang Likun has achieved a lot during his tenure, such as developing the world-renowned AI framework Pytorch, improving GAN (generative confrontation network), launching the large language model Llama and AI image model SAM and so on.

**As agreed in those years, these projects have all been open source. **

Under the wave of large language models, the Llama large language model is also very popular in the open source community.

In open source communities such as Hugging Face, there are all kinds of magically modified alpacas (Llama’s Chinese translation is alpaca).

Including Stanford’s Alpaca, UCB’s Vicuna… Various modified models based on LlamA have emerged.

In many large model running score lists, under GPT-3.5 and GPT-4, almost all alpaca family slaughter lists.

In addition, Yang Likun does not agree with OpenAI’s closed-source reasons for security reasons.

In his view, the only way to make an AI platform safe, good, and useful is to open source it. **

In other words, it is dangerous for technology to be in the hands of a few people. Only by allowing the power to supervise AI to evolve at the same time can AI be controlled as much as possible.

At present, only open source can do it for the time being.

In general, in the wave of large language models led by OpenAI, the open source army led by Meta is crazily attacking cities and land in the open source community.

On the other hand, the conflict between open source and closed source is gradually heating up.

In May of this year, a memo “accidentally” leaked by a Google researcher put the battle on the table.

The editorial department of Zhiwei briefly summarized the key points mentioned by Google researchers in this memo:

① Open source AI is encroaching on the territory of Google and Open AI;

②Small models are more competitive than large models;

③ Data quality is far more important than data quantity;

④Google can’t beat open source;

⑤ Google needs an open source ecosystem more than open source needs Google.

To put it briefly, this eloquent article with thousands of words reveals a core message:

** In the face of open source, neither Google nor OpenAI has a moat. **

So, looking back at the Llama 2 released this time, Meta directly removed the restrictions on commercial use, which is really another army for the closed-source camp.

The open source Llama2 is not only free, but also allows developers to adjust it by themselves, so as to meet the low-cost and personalized needs of most commercial companies.

However, in the end, who is the winner between open source and closed source, the Zhiwei editorial department feels that it is not easy to draw a conclusion.

After all, open source is not a panacea. There are good and bad developers, and some public security risks, etc., are all problems that open source often encounters.

In the final analysis, the competition between open source and closed source is a different competition strategy. One pays more attention to expanding the market and setting standards, and the other pays more attention to profitability and leading technology.

For example, Android and iOS, Linux and Windows may not really win or lose in the end, but find their own reasonable positioning under different needs.

However, there is no doubt that Llama 2 has stirred up the entire market and has caused subtle changes in the ecology:

① Microsoft, one of the shareholders of OpenAI, has become the preferred partner of Llama 2 this time, and Microsoft seems to be betting on both sides.

② Meta announced that Llama 2 will be able to run on Qualcomm chips, which is bound to be a challenge for Nvidia, the overlord of AI chips.

In short, the drama in the big language model circle is getting more and more complicated and interesting.

If the muddy water caused by Llama 2 makes the AI era come sooner, please remember that this is not only the credit of Meta, but also the credit of the open source community.