GPT-4

GPT-4 (Generative Pre-trained Transformer 4) is a large multimodal model from OpenAI that can accept both image and text inputs and output text. A transformer-based model pre-trained to predict the next token in a document, GPT-4's post-training alignment results in improved performance in terms of factuality and adherence to desired behavior. Although GPT-4 remains less capable than humans in many real-world scenarios, the model demonstrates human-level performance on a range of professional and academic benchmarks such as the bar exam.

GPT-4 accepts text or image prompts, allowing the user to specify any vision or language task. It generates text outputs (e.g., natural language, code, etc.) in response to the input provided. The text input and output can be up to 25,000 words, over eight times more than ChatGPT (only 3,000), allowing for long-form content creation use cases including extended conversations and document search and analysis. GPT-4 can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, and learning a user’s writing style. The model is capable of understanding images and expressing logical ideas about them, generating captions, classifications, and analyses. GPT-4 can also be augmented with test-time techniques developed for text-only language models, including few-shot and chain-of-thought prompting.

Introducing GPT-4

OpenAI announced GPT-4 on March 14, 2023. Initially, only the model's text capabilities were made available via ChatGPT Plus and the company's API, with a waiting list. Upon release, OpenAI stated it is working closely with a single partner to prepare the image input capability for wide availability. The company also released an open-source framework for automating the evaluation of AI models, called OpenAI Evals, allowing users to report shortcomings in the model. After the release, Microsoft confirmed its AI-enabled Bing, released on February 7, 2023, was already running an early version of GPT-4.

In July 2023, OpenAI opened GPT-4 API to all paying API customers and announced a deprecation plan for older models of the Completions API, which will retire in early 2024. Between March (release of GPT-4) and July 2023, millions of developers requested access to the GPT-4 API. After July 2023, the waiting list was removed, allowing all existing API developers with a history of successful payments to access the GPT-4 API with 8K context. OpenAI plans to further open access to new developers from the end of July 2023 and begin raising rate limits.

Continuing the research path from GPT, GPT-2, and GPT-3, OpenAI's deep learning approach leverages more data and more computation to build increasingly sophisticated and capable language models. A core component of developing GPT-4 was developing infrastructure and optimization methods that behave predictably over a wide range of sales. GPT-4 was the first in the GPT series that OpenAI could accurately predict aspects of its performance. These predictions were based on models trained with 1/1000th or less the computational power of GPT-4. GPT-4 was trained on Microsoft's Azure AI supercomputers, and training finished in August 2022. OpenAI spent six months between training the model and releasing it, making GPT-4 more aligned and safer. This involved more human feedback, including submissions from ChatGPT users and the help of over fifty experts in the field. The model will be regularly updated as it is more widely used. GPT-4 is 82 percent less likely to respond to prompts asking for disallowed content and 40 percent more likely to offer factual responses, compared to GPT-3.5 based on OpenAI's internal evaluations.

The performance difference between GPT-3.5 and GPT-4 becomes apparent when the complexity of tasks reaches a sufficient threshold. GPT-4 is more reliable, creative, and able to follow more nuanced instructions. GPT-4 outperforms ChatGPT (which is based on GPT-3.5), scoring higher in approximate percentile across a range of tests. This includes achieving 90th percentile in the uniform bar exam compared to 10th percentile for ChatGPT and 99th percentile in the Biology Olympiad compared to 31st for ChatGPT. OpenAI published the two model's performances on a variety of benchmarks. No specific training was provided for these exams. Vision refers to the inclusion or not of GPT-4's image input capabilities.

Performance of GPT-3.5 and GPT-4 for a number of exams.

GPT-4 has known limitations that OpenAI is working to address. These include social biases, hallucinations, and adversarial prompts. These are similar to limitations found in earlier GPT models. The model is still not fully reliable, providing incorrect information and making reasoning errors.