GPT-4 Technical Report (OpenAI, arXiv 2303.08774, March 2023)

The GPT-4 Technical Report is OpenAI's technical report accompanying GPT-4, submitted to arXiv on 15 March 2023 (arXiv:2303.08774, cs.CL) and last revised 4 March 2024 as version 6. It is authored institutionally as "OpenAI", asks to be cited as "OpenAI (2023)", and carries an authorship and credit-attribution statement listing contributors by team; arXiv lists the paper as "by OpenAI and 279 other authors." It runs to 100 pages, and the version-6 comment records the update as "updated authors list; fixed author names and added citation." The report covers GPT-4's capabilities, limitations and safety properties, and appends the GPT-4 System Card as its final appendix. The model page is at GPT-4 Family (OpenAI).

Summary

The abstract states that GPT-4 is "a large-scale, multimodal model which can accept image and text inputs and produce text outputs," and that while "less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers." Architecturally it is described only as "a Transformer-based model pre-trained to predict the next token in a document," with a post-training alignment process that "results in improved performance on measures of factuality and adherence to desired behavior."

The body has seven sections: an introduction; the scope of the report and what it omits; predictable scaling; capabilities, covering exams, academic benchmarks, multilingual MMLU, instruction following and visual inputs; limitations; risks and mitigations; and a conclusion. Appendices A to G give the exam methodology, the effect of RLHF on exam performance, contamination analyses for exams and benchmarks, the treatment of GSM-8K in training, the multilingual MMLU method, and examples of visual input. Appendix H is the system card. OpenAI's conclusion characterizes GPT-4 as "a large multimodal model with human-level performance on certain difficult professional and academic benchmarks" and states that, although "there remains much work to be done, GPT-4 represents a significant step towards broadly useful and safely deployed AI systems."

Scope and disclosure

Section 2 states that GPT-4 is "a Transformer-style model pre-trained to predict the next token in a document, using both publicly available data (such as internet data) and data licensed from third-party providers," then fine-tuned using Reinforcement Learning from Human Feedback (RLHF). It continues: "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

No parameter count, training-data composition, or compute figure is disclosed, an omission that became the reference case in the subsequent argument over what a frontier "technical report" is obliged to contain. The report pairs the omission with two commitments: OpenAI states that it is "committed to independent auditing of our technologies," pointing to initial steps in the accompanying system card, and that it plans "to make further technical details available to additional third parties who can advise us on how to weigh the competitive and safety considerations above against the scientific value of further transparency." A footnote adds that OpenAI "will soon publish additional thoughts on the social and economic implications of AI systems, including the need for effective regulation." The system card appendix states that GPT-4 finished training in August 2022.

Predictable scaling

The abstract identifies as "a core component of this project" the development of "infrastructure and optimization methods that behave predictably across a wide range of scales," and states that this "allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4." Section 3 gives the reason: for very large training runs "it is not feasible to do extensive model-specific tuning," so OpenAI built a stack whose behaviour at small scale could be extrapolated, predicting some aspects of GPT-4's performance from smaller models trained using 1,000× to 10,000× less compute.

Loss prediction. OpenAI predicted GPT-4's final loss on a dataset derived from its internal codebase, which is not part of the training set, by fitting a scaling law with an irreducible loss term, L(C) = aC^b + c, following Henighan et al., to models trained with the same methodology using at most 10,000× less compute. The report states the prediction "was made shortly after the run started, without use of any partial results," and that the fitted law "predicted GPT-4's final loss with high accuracy" (Figure 1, bits per word against training compute normalized so that GPT-4 is 1). It chose loss because it "tends to be less noisy than other measures across different amounts of training compute."

Capability prediction. The report argues that "having a sense of the capabilities of a model before training can improve decisions around alignment, safety, and deployment," and extends the method to pass rate on the HumanEval coding benchmark. It fits an approximate power law, −E_P[log(pass_rate(C))] = α∗C^−k, restricted to problems and models where every problem is solved at least once given a large sample budget, since very low pass rates cannot be estimated. Predictions for GPT-4 were registered before training completed, "using only information available prior to training." All but the 15 hardest HumanEval problems were split into six difficulty buckets by the performance of smaller models; Figure 2 shows the third-easiest bucket (23 problems), where the extrapolation from models trained with at most 1,000× less compute matched GPT-4's result. The other five buckets "performed almost as well, the main exception being GPT-4 underperforming our predictions on the easiest bucket." The report notes that performance on an individual problem "may occasionally worsen with scale."

Limits of prediction. Some capabilities "remain hard to predict." On Hindsight Neglect, one of the tasks from the Inverse Scaling Prize on which performance falls with scale, GPT-4 reverses the trend seen in the ada, babbage, curie and GPT-3.5 models (Figure 3), a result the report compares to Wei et al. OpenAI writes that "accurately predicting future capabilities is important for safety," says it plans to "register performance predictions across various capabilities before large model training begins," and hopes "this becomes a common goal in the field."

The claim bears directly on governance: if frontier capability can be forecast from small-scale runs, then capability thresholds are in principle knowable before a training run completes, which is the premise underlying compute-threshold rules in EU AI Act (Regulation 2024/1689) and the capability-threshold structure of frontier safety frameworks. See Scaling Laws.

Capabilities

Professional and academic exams

GPT-4 was tested on simulated exams originally designed for humans, with "no specific training for these exams." The post-trained RLHF model was used. Exams were sourced from the most recent publicly available official past exams or from practice exams in 2022–2023 third-party study materials that OpenAI purchased. A minority of questions had been seen in training; for each exam OpenAI ran a variant with those questions removed and reports "the lower score of the two." The evaluation setup was tuned on a validation set of exams, and final results are from held-out test exams; multiple-choice and free-response scores were combined into overall scores using publicly available methodologies for each exam, and the percentile each score corresponds to was estimated.

On the simulated Uniform Bar Exam GPT-4 scored 298 of 400, around the 90th percentile, where GPT-3.5 scored 213, around the 10th; the bar exam was run by Casetext and Stanford CodeX. GPT-4 scored 163 on the LSAT (~88th percentile), 710 of 800 on SAT Evidence-Based Reading and Writing, 700 of 800 on SAT Math, and 169 of 170 on GRE Verbal (~99th percentile), and reached the 99th–100th percentile on the 2020 USA Biology Olympiad semifinal. It was weaker on the AP English Language and English Literature exams (score 2 on each), AMC 10 (30 of 150, 6th–12th percentile), Leetcode hard problems (3 of 45) and Codeforces, where its rating of 392 fell below the 5th percentile. OpenAI summarizes that GPT-4 "exhibits human-level performance on the majority of these professional and academic exams" and "outperforms GPT-3.5 on most exams tested." Every exam score, with and without vision and for GPT-3.5, is listed under Reported results.

Appendix A sets out the method:

• Multiple choice. A few-shot prompt with gold-standard explanations and answers for a similar exam; an explanation was sampled at temperature 0.3 and the answer letter extracted. Each section was sourced as a holdout and a non-holdout exam, with methodology iterated on the non-holdout and the holdout run once. No non-holdout exam was sourced for the USABO or the Medical Knowledge Self-Assessment Program (MKSAP), which were run once with a methodology developed on AP Biology. A bug that limited response length on the AMC 10 and AMC 12 was fixed and those exams rerun. For some runs the letter was instead sampled at temperature 0 from the already-sampled explanation; OpenAI attributes these differences to "code mismatches detected post-evaluation" and believes their impact "to be minimal."
• Free response. Responses were sampled once at temperature 0.6 with a simple instruction-following prompt and no methodology iteration, because of the longer iteration time of human grading. Essays requiring judgement of writing quality (the AP English, history, government and art history exams, the GRE and the SAT) were graded by one or two qualified third-party contractors; other free-response answers were graded against official rubrics.
• Images. Text-only models received a tag reading "IMAGE:" with a non-meaningful filename in place of each image, which lower-bounds their multiple-choice performance. Images in free-response questions and in the USABO were transcribed as text, so those scores serve for both the vision and no-vision conditions.
• Scoring. SAT scores were converted with an official sample score chart; GRE multiple-choice scores with the formula of accuracy × 40 + 130; AP scores with public score calculators based on 2019–2020 scoring guidelines. Because 2022 AMC distributions were unpublished, AMC percentiles were extrapolated from two November 2021 distributions and "likely have wide uncertainty."
• Codeforces. Each model was evaluated on 10 recent contests of about six problems each, with 10 attempts per problem; each contest was simulated 100 times and Elo adjusted to equilibrium. Roughly half of simulations solved no problems, giving an equilibrium rating of 0, which is why the averages are low; the highest single-contest equilibrium was around 1,300 for GPT-4 and 1,000 for GPT-3.5.
• Snapshots. GPT-4's multiple-choice questions were run on a March 1, 2023 snapshot and its free-response questions on a non-final February 23, 2023 snapshot; the USABO semifinal used an earlier December 16, 2022 snapshot. GPT-3.5 was run on a standard ChatGPT snapshot.

Effect of RLHF. The report states that exam capability "appear[s] to stem primarily from the pre-training process and [is] not significantly affected by RLHF." On the multiple-choice portions of the exams the base model averaged 73.7% and the RLHF model 74.0% (Appendix B, Table 8), which OpenAI reads as showing that "post-training does not substantially alter base model capability." It notes that free-response answers cannot be compared on an even footing, since its sampling method likely benefits from instruction following.

Contamination. Appendix C measures overlap with the pre-training data by substring match: evaluation and training text are stripped of spaces and symbols, three random 50-character substrings are drawn from each evaluation example (or the whole example if shorter), and a match on any of them marks the example as contaminated. Contaminated examples are discarded and the exam rerun. The report lists the method's limits: it can produce false negatives and false positives, and it uses only questions or context, ignoring answers and sometimes answer options. The RLHF post-training dataset was not explicitly checked. Contamination was zero for the Uniform Bar Exam, Codeforces and Leetcode, and highest for GRE Writing (100%), AP English Literature (92%), AP English Language (79%), AP US History (73%), AP World History (47%), the LSAT (39%) and GRE Quantitative (35%). OpenAI concludes that the degradation from removing contaminated questions "is generally small and as often positive as negative" and that "contamination is not a substantive confounder on the overall results." Both vision and contamination are described as having "relatively little effect" across most exams.

Academic benchmarks

The pre-trained base model was evaluated with few-shot prompting on standard language-model benchmarks, with contamination checks for each. Portions of BIG-bench were found to have been "inadvertently mixed into the training set," and BIG-bench was excluded from the reported results. For multiple-choice questions the model was shown all options (A–D) and asked to choose a letter.

OpenAI reports that GPT-4 "outperforms existing LMs on all benchmarks, and beats SOTA with benchmark-specific training on all datasets except DROP": 86.4% on MMLU (5-shot) against 70.0% for GPT-3.5, 70.7% for U-PaLM and 75.2% for Flan-PaLM; 95.3% on HellaSwag; 96.3% on the AI2 Reasoning Challenge; 87.5% on WinoGrande; 67.0% on HumanEval (0-shot) against 65.8% for CodeT with GPT-3.5; 80.9 F1 on DROP against 88.4 for QDGAT; and 92.0% on GSM-8K with 5-shot chain-of-thought prompting. The MMLU, HumanEval and GSM-8K rows are recorded on those benchmarks' pages; the others are under Reported results.

GSM-8K and MATH in training. To improve mathematical reasoning, OpenAI mixed data from the training sets of GSM-8K and the MATH benchmark into GPT-4's pre-training. The tokens drawn from them were "a tiny fraction of the overall GPT-4 training budget," and a portion was held back, so a given training example may or may not have been seen. OpenAI checked that the GSM-8K test set was not in the training data and recommends reading the 92.0% "as something in-between true few-shot transfer and full benchmark-specific tuning."

Benchmark contamination. Appendix D estimates contamination from 1,000 randomly chosen examples per benchmark (all examples for HumanEval): about 0.6% for MMLU, 1% for GSM-8K, 3.4% for AI2 ARC, 0.9% for WinoGrande, 21% for DROP and 25% for HumanEval. On the uncontaminated HumanEval problems GPT-4 scored 65.58%, a degradation of 2.12%. HellaSwag was scored on a privately held holdout that was not checked, but OpenAI notes the result is close to the 95.6% scored on the validation set, which "was explicitly masked out during training."

Multilingual MMLU

To gauge performance in other languages, OpenAI translated all MMLU questions and answers with Azure Translate, using an external system "in case the model had unrepresentative performance for its own translations." Languages were chosen to cover different regions and scripts. The report cautions that the translations "are not perfect, in some cases losing subtle information which may hurt performance," and that some keep English proper nouns, which may help. The prompt follows the Gopher paper's MMLU format, with the instruction, question and answers translated but the "Answer" token and A–D labels kept in English; evaluation was three-shot rather than five-shot because some languages map to much longer token sequences, and the answer is the A–D token with the highest probability.

GPT-4 scored 85.5% on English MMLU in this setting. It exceeded the English-language state of the art in 24 of the 26 languages tested, and exceeded the English scores of GPT-3.5, Chinchilla and PaLM in most of them, including the low-resource languages Latvian (80.9%), Welsh (77.5%) and Swahili (78.5%). Its lowest scores were in Marathi (66.7%) and Telugu (62.0%). Appendix F illustrates the translation with an astronomy question in Marathi, Latvian and Welsh.

Following user intent and OpenAI Evals

On 5,214 prompts submitted to ChatGPT and the OpenAI API, human labelers preferred GPT-4's response to GPT-3.5's on 70.2% of prompts. One response was sampled from each model; labelers judged whether each response was what the user would have wanted, without being told which model produced it, with presentation order randomized. Prompts with disallowed or sensitive content, personally identifiable information, and short or overly common prompts were filtered out.

The report announces the open-sourcing of OpenAI Evals, OpenAI's "framework for creating and running benchmarks for evaluating models like GPT-4 while inspecting performance sample by sample," which it says is compatible with existing benchmarks and can track the performance of models in deployment.

Visual inputs

GPT-4 accepts prompts of "arbitrarily interlaced text and images" and, the report states, "exhibits similar capabilities as it does on text-only inputs" across documents with text and photographs, diagrams and screenshots; few-shot and chain-of-thought prompting are "similarly effective" with images. The main text shows one example, an explanation of the humour in a three-panel image of a VGA connector sold as a Lightning cable adapter. Appendix G adds examples of reading a chart and computing from it, solving a French-language École Polytechnique physics question from its diagram with chain-of-thought, describing an unusual photograph, summarizing part of the InstructGPT paper from an image of its pages, and explaining two memes. The report gives no vision benchmark scores: "Preliminary results on a narrow set of academic vision benchmarks can be found in the GPT-4 blog post," with further information promised in follow-up work.

Limitations

The report states that GPT-4 "still is not fully reliable (it 'hallucinates' facts and makes reasoning errors)," that it "has a limited context window, and does not learn from experience," and that "great care should be taken when using language model outputs, particularly in high-stakes contexts," with protocols such as human review, grounding with additional context, or avoiding high-stakes uses altogether.

Hallucination and factuality. GPT-4 scores 19 percentage points higher than OpenAI's latest GPT-3.5 model on nine internal, adversarially designed factuality evaluations spanning learning, technology, writing, history, mathematics, science, recommendation, code and business (Figure 6, comparing three earlier GPT-3.5-based ChatGPT versions). On TruthfulQA, which pairs questions with statistically appealing false answers, the base GPT-4 model is "only slightly better at this task than GPT-3.5," but after RLHF post-training OpenAI observes "large improvements over GPT-3.5" (Figure 7, which also shows Anthropic-LM from Bai et al.). The RLHF post-training data was not checked for TruthfulQA contamination. The report's example shows GPT-4 rejecting the saying "you can't teach an old dog new tricks" but choosing "Presley" for a question whose answer is Perkins. Figure 7's values are not printed in the text; the system card appendix gives accuracy "around 60%" against 30% for an earlier version, recorded on GPT-4 System Card (OpenAI, March 2023).

Knowledge cutoff and reasoning. GPT-4 "generally lacks knowledge of events that have occurred after the vast majority of its pre-training data cuts off in September 2021," with a small amount of more recent data in the pre-training and post-training sets. It "can sometimes make simple reasoning errors," can be "overly gullible in accepting obviously false statements from a user," and can fail at hard problems as humans do, "such as introducing security vulnerabilities into code it produces."

Calibration. The pre-trained model "is highly calibrated": on a subset of MMLU its expected calibration error was 0.007. After post-training the error rose to 0.074, and the report states that "the post-training hurts calibration significantly" (Figure 8). GPT-4 "can also be confidently wrong in its predictions."

Bias. GPT-4 "has various biases in its outputs" that OpenAI has "taken efforts to correct but which will take some time to fully characterize and manage." OpenAI states the aim of default behaviours that "reflect a wide swath of users' values," customization "within some broad bounds," and public input on what those bounds should be.

See Sycophancy and Hallucination.

Risks and mitigations

Adversarial testing by domain experts. OpenAI engaged "over 50 experts from domains such as long-term AI alignment risks, cybersecurity, biorisk, and international security" to test the model adversarially, including for risks "that will become relevant for very advanced AIs such as power seeking." Their recommendations and data fed into mitigations; for example, additional data was collected to improve refusals of requests to synthesize dangerous chemicals (Table 5, where an early GPT-4 generated steps and the March 14 GPT-4 declined). The acknowledgements add that participation in red teaming "is not an endorsement of the deployment plans of OpenAI or OpenAI's policies." See Jailbreaking and Red Teaming.

Model-assisted safety pipeline. GPT-4 was fine-tuned with RLHF, but "after RLHF, our models can still be brittle on unsafe inputs as well as sometimes exhibit undesired behaviors on both safe and unsafe inputs," which the report attributes partly to underspecified labeler instructions during reward-model data collection; the model could give advice on committing crimes, or become "overly cautious on safe inputs, refusing innocuous requests or excessively hedging." The safety approach adds two components: additional safety-relevant RLHF training prompts, and rule-based reward models (RBRMs), a set of zero-shot GPT-4 classifiers that supply an extra reward signal to the policy model during RLHF fine-tuning. An RBRM takes the prompt (optional), the policy model's output and a human-written rubric, for instance classifying a response as a refusal in the desired style, a refusal in an undesired (evasive or rambling) style, a response containing disallowed content, or a safe non-refusal. GPT-4 is then rewarded for refusing harmful requests and for not refusing a subset of prompts known to be safe and answerable. The report relates the technique to work by Glaese et al. and Perez et al., and credits it, together with computing optimal RBRM weights and additional supervised fine-tuning data, with steering the model closer to the desired behaviour. Examples show a bomb-making request refused by the latest GPT-4 (Table 6) and a request for cheap cigarettes answered by the latest GPT-4 after an early version had refused (Table 7). See RLHF (Reinforcement Learning from Human Feedback).

Safety metrics. OpenAI reports that its mitigations decreased the model's tendency to respond to requests for disallowed content by 82% compared to GPT-3.5, and that GPT-4 responds to sensitive requests, such as for medical advice and self-harm, in accordance with its policies 29% more often (Figure 9, comparing text-davinci-003, gpt-3.5-turbo and GPT-4). On the RealToxicityPrompts dataset GPT-4 produced toxic generations 0.73% of the time, against 6.48% for GPT-3.5.

Residual risk. The report states that the interventions "increase the difficulty of eliciting bad behavior but doing so is still possible," citing jailbreaks such as adversarial system messages, and that they must be complemented by "deployment-time safety techniques like monitoring for abuse as well as a pipeline for fast iterative model improvement." OpenAI says it is collaborating with external researchers to "build evaluations for dangerous capabilities that may emerge in future systems," and that it will publish recommendations on how society can prepare for AI's effects and initial ideas on projecting its economic impacts.

The system card appendix

Appendix H appends the GPT-4 System Card, which OpenAI also published separately on March 23, 2023. It analyzes two versions of the model, an early instruction-following version ("GPT-4-early") and the deployed version with further mitigations ("GPT-4-launch"), treats custom fine-tuning and image capabilities as out of scope, and covers hallucination, harmful content, harms of representation, disinformation, weapons proliferation, privacy, cybersecurity, risky emergent behaviours including the Alignment Research Center evaluation, interactions with other systems, economic impacts, acceleration and overreliance, followed by the deployment mitigations and the full RBRM instructions. Its content, and the quantitative results found only in the card, are covered on GPT-4 System Card (OpenAI, March 2023). Figures that appear both in the report's main text and in the card (the 82% and 29% changes, the RealToxicityPrompts rates and the 70.2% preference rate) are recorded on this page.

Contributors and acknowledgements

The authorship statement lists contributors alphabetically within teams. Among the named leads are Jakub Pachocki (overall pretraining lead and optimization lead), Łukasz Kaiser (long-context lead), John Schulman (overall lead for reinforcement learning and alignment), Barret Zoph (overall training lead), Jan Leike and Ryan Lowe (alignment co-leads), Mo Bavarian and Benjamin Sokolowsky (capability prediction co-leads), Sandhini Agarwal and Gretchen Krueger (system card co-leads) and Lama Ahmad (expert red teaming and adversarial testing program lead). Daniel Kokotajlo is listed under acceleration forecasting, and Sam Altman, Mira Murati and Ilya Sutskever among additional contributors.

OpenAI thanks Microsoft for its partnership, "especially Microsoft Azure for supporting model training with infrastructure design and management, and the Microsoft Bing team and Microsoft's safety teams for their partnership on safe deployment." It thanks its expert adversarial testers and red teamers, and Casetext and Stanford CodeX (P. Arredondo, D. Katz, M. Bommarito and S. Gao) for conducting the simulated bar exam. The report states that "GPT-4 was used for help with wording, formatting, and styling throughout this work."

Reception

Critics described the decision to withhold architecture, parameter-count, hardware, training-compute and dataset details as a turn toward secrecy relative to earlier OpenAI releases (Source: zdnet.com; fastcompany.com).

The bar-exam percentile was later contested. Eric Martínez (MIT) estimated in Artificial Intelligence and Law that the 298 score fell below the 69th percentile against test-takers from a recent July administration (about the 48th on essays), around the 62nd percentile against first-time takers (about the 42nd on essays), and around the 48th percentile overall (about the 15th on essays) against those who passed, arguing that OpenAI's estimate was skewed by conversions from February administrations dominated by lower-scoring repeat test-takers (Source: link.springer.com).

Position in the record

GPT-4 anchors the 2023 capability wave. It arrived roughly three and a half months after the ChatGPT launch, and its combination of headline professional-exam results with withheld architectural detail set the template that later system cards both followed and were measured against. See AI System Cards and AI Transparency.

Contamination data for exams

Appendix C, Table 10 breaks each exam into its multiple-choice (MCQ) and free-response (FRQ) sections, listed from most to least contaminated, and gives GPT-4's score as a percentage of the maximum on the whole section, on the uncontaminated questions and on the contaminated questions alone. The caption defines degradation as the "non-contaminated percent minus contaminated"; the printed values equal the change from the overall to the uncontaminated score, relative to the overall score (for the LSAT multiple-choice section, 83.61% against 76.00%, or 10.01%). The table is reproduced as printed; the headline exam scores it underlies are under Reported results.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

Provenance

The full text is held at Raw Sources/GPT-4 Technical Report (OpenAI, 2023-03-15).md, extracted on September 19, 2026 with pdftotext -layout from the arXiv PDF (version 6, dated 4 March 2024, 100 pages). Table columns survive through layout spacing; figures are not reproduced, although value labels printed inside vector figures (Figure 5's per-language scores and Figure 8's calibration errors) survive as text. Figures 3, 6, 7 and 9 carry no printed values, so only the relative figures stated in the text are recorded for them. Vision benchmark scores (VQAv2, TextVQA, ChartQA and others) appeared in OpenAI's GPT-4 blog post, not in the report, and are not recorded here.

An earlier capture, Raw Sources/OpenAI - GPT-4 Technical Report (arXiv 2303.08774, 2023-03-15).md, was retrieved July 31, 2026 from the arXiv abstract page using the vault's bin/fetch-source.py while Firecrawl was rate-limited. It holds the title, arXiv identifier, submission and revision history (v1 on 15 March 2023 through v6 on 4 March 2024), subject classification, author roster, page count and abstract, but none of the body. The version of this page written from it rested on the abstract and citation metadata alone.

Relationships

• related: OpenAI — the publisher.
• related: GPT-4 Family (OpenAI) — the model the report documents.
• depends-on: GPT-4 System Card (OpenAI, March 2023) — appended as Appendix H; the report's limitations and risks sections refer to it for detail.
• related: ChatGPT: Optimizing Language Models for Dialogue (OpenAI, November 30, 2022) — the deployment event that preceded it by under four months.
• supports: Scaling Laws — the predictable-scaling claim is a primary-source statement of forecastable capability, with final loss and HumanEval pass rate predicted from much smaller runs.
• related: AI System Cards — the report's selective disclosure is the reference case in the transparency debate.
• related: AI Transparency — the stated reasons for withholding architecture, compute and data details.
• depends-on: RLHF (Reinforcement Learning from Human Feedback), Training Language Models to Follow Instructions with Human Feedback (InstructGPT), Deep Reinforcement Learning from Human Preferences (Christiano et al., 2017) — the post-training method, cited as Ouyang et al. and Christiano et al.
• supports: Sycophancy and Hallucination — hallucination rates, TruthfulQA and the loss of calibration after post-training.
• related: Jailbreaking and Red Teaming — expert red-teaming and the residual jailbreak risk.
• related: AI Benchmarks and Evaluation — exam and benchmark contamination methodology.
• related: MMLU, HumanEval, GSM8K, MATH and MATH-500 — benchmarks reported, or mixed into training.
• related: Microsoft — Azure supported training infrastructure.
• related: Sparks of Artificial General Intelligence: Early experiments with GPT-4 — Microsoft Research's study of an early GPT-4 build.