History and Development of ChatGPT
From the Previous Article, we already know that GPT (Generative Pre-trained Transformer) needs a lot of data training to ensure the correct operation of the model. So, how did the well-known GPT-3.5 or GPT-4 develop? Or, what has GPT experienced in the development process?
The development of the GPT model can be traced back to 2018 and has undergone several versions of improvements and extensions. The following is the release time, number of parameters, and main features of each version of GPT:
GPT-1: June 2018
Number of published parameters: 117 million
Features: GPT-1 is the first GPT version, which adopts Transformer-based autoregressive language model architecture and has achieved excellent performance in various natural language processing tasks through unsupervised pre-training and supervised fine-tuning methods.
GPT-2: February 2019
Number of published parameters: 1.5 billion
Features: Compared with GPT-1, the number of parameters of GPT-2 has been dramatically improved, and the generation ability of the model has also been significantly enhanced. GPT-2 can generate high-quality articles and stories, but its generative solid power raises ethical and safety issues. Therefore, OpenAI adopted a phased strategy when releasing GPT-2.
GPT-3: May 2020
Number of published parameters: 175 billion
Features: The number of parameters is more than 100 times that of GPT-2, and its performance is close to the human level on many tasks. GPT-3 performs well on natural language understanding, generation, and summarization tasks. However, the large model size makes training and inference costs high. To allow more people to use GPT-3, OpenAI launched the OpenAI API based on cloud services.
In 2022, based on GPT-3, openAI will use a more extensive data set and a more complex model to train a large-scale language model GPT-3.5. Based on GPT-3.5, the language model trained for dialogue is ChatGPT. It was released on November 30, 2022. Two months later, monthly active users exceeded 100 million, becoming the fastest-growing number of users in history—consumer applications.
GPT-4: March, 2023
Number of parameters: 1.6 trillion
Features: The number of parameters of GPT-4 is 16 times that of GPT-3. The accuracy of GPT-4’s answers is significantly improved, has a higher image recognition ability, and can generate lyrics and creative texts to achieve style changes. In addition, the text input limit of GPT-4 has also been increased to 25,000 characters, and there are more optimizations for languages other than English.
There is a shocking rumor: GPT-4 passed the Turing test!
GPT-4 passing the Turing test means that it has achieved such a high level of performance on natural language processing tasks that it is difficult for humans to distinguish whether humans write the text it generates. This proves that GPT-4’s ability to understand and develop natural language has approached or reached human levels. It has high application potential in many fields, such as automatic question answering, machine translation, article summarization, natural language generation, etc.
At the same time, this also shows that GPT-4 can improve the efficiency and quality of natural language processing, bringing more possibilities for future development. At the same time, it is also possible to understand the picture, not to talk about the content that is not allowed and the sensitive content.