GPT-4 System Card (OpenAI, March 2023)

The system card accompanying OpenAI's GPT-4 release. It predates the Preparedness Framework and is the reference point against which later OpenAI safety documentation is read. See GPT-4 Family (OpenAI).

Structure

The card sets out three parts: safety challenges "presented by the model's limitations (e.g., producing convincing text that is subtly false) and capabilities (e.g., increased adeptness at providing illicit advice, performance in dual-use capabilities, and risky emergent behaviors)"; an overview of OpenAI's pre-deployment safety process "across measurements, model-level changes, product- and system-level interventions (such as monitoring and policies), and external expert engagement"; and a demonstration that mitigations alter but do not eliminate the behaviours of concern.

Pairing limitations and capabilities as sources of risk is a framing that persists in later cards: a model that is subtly wrong and a model that is capably harmful pose distinct problems.

The ARC evaluation

The card's most-cited element is a preliminary evaluation "by the Alignment Research Center (ARC) of GPT-4's ability to carry out actions to autonomously replicate and gather resources — a risk that, while speculative, may become possible with sufficiently advanced AI systems — with the conclusion that the current model is probably not yet capable of autonomously doing so."

Two features of that sentence are worth noting: the risk is labelled speculative at the time, and the conclusion is hedged ("probably not yet"). The card immediately adds that "further research is needed to fully characterize these risks," calling for "more robust evaluations for the risk areas identified." This is the origin of the third-party dangerous-capability evaluation practice that later became routine — and ARC's successor organization, METR, now performs it under formal pre-deployment arrangements.

Power-seeking

The card gives an unusually direct statement of the instrumental-convergence argument as a rationale for evaluation. Systems that pursue "specific, quantifiable objectives" and "do long-term planning" are of concern because "for most possible objectives, the best plans involve auxiliary power-seeking actions because this is inherently useful for furthering the objectives and avoiding changes or threats to them." It adds that "power-seeking is optimal for most reward functions and many types of agents," that "some evidence already exists of such emergent behavior in models," and that "there is evidence that existing models can identify power-seeking as an instrumentally useful strategy."

See Instrumental Convergence, AI Autonomy Risk.

Relation to the technical report

The card was also appended to the GPT-4 Technical Report as its Appendix H. It states that GPT-4 finished training in August 2022, and it analyzes two versions of the model: an early version fine-tuned for instruction following ("GPT-4-early") and the deployed version with further mitigations ("GPT-4-launch"). Several of its safety figures also appear in the report's main text and are recorded on the report's page: an 82% reduction relative to GPT-3.5 in responses to requests for disallowed content, 29% more policy-compliant responses to sensitive requests, toxic generations on RealToxicityPrompts in 0.73% of cases against 6.48% for GPT-3.5, and a 70.2% preference for GPT-4's responses over GPT-3.5's on 5,214 user prompts.

Mitigations and their limits

Reported interventions include filtering inappropriate content from the pre-training dataset, fine-tuning to refuse "direct requests for illicit advice," reducing hallucination, reducing "the surface area of adversarial prompting or exploits (including attacks sometimes referred to as 'jailbreaks')," and training "a range of classifiers on new risk vectors" incorporated into monitoring workflows for API policy enforcement.

The card is candid that these are partial: "the effectiveness of these mitigations varies," and post-mitigation GPT-4 "is also able to give more detailed guidance on how to conduct harmful or illegal activities" than earlier models.

On hallucination, the card reports that GPT-4-launch "scores 19 percentage points higher than our latest GPT-3.5 model at avoiding open-domain hallucinations, and 29 percentage points higher at avoiding closed-domain hallucinations," and that its hallucination mitigations, which include GPT-4-generated comparison data for closed-domain cases, raised TruthfulQA accuracy "to around 60% as compared to 30% for an earlier version." GPT-4-launch's responses were preferred over GPT-3.5 Turbo RLHF's on 61.1% of prompts.

A methodological note explains that red-teaming compared two versions rather than the base model, "since the base model proved challenging for domain expert red teamers to use effectively to surface behaviors of interest" — and flags sycophancy, "tendencies to do things like repeat back a dialog user's preferred answer," as a behaviour that "can worsen with scale." See Sycophancy and Hallucination.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

Relationships

• supports: GPT-4 Technical Report (OpenAI, arXiv 2303.08774, March 2023) — appended to the report as Appendix H; the report's risks and mitigations section summarizes it.
• supports: GPT-4 Family (OpenAI) — the safety documentation for the release
• related: OpenAI Preparedness Framework V.2 — the structured successor to this card's ad hoc risk treatment
• related: METR — successor to the ARC evaluation function described here
• related: Instrumental Convergence, AI Autonomy Risk, Sycophancy and Hallucination, OpenAI