The gpt-oss model card is a 35-page document published by OpenAI on August 5, 2025, covering two open-weight reasoning models, gpt-oss-120b and gpt-oss-20b, released under the Apache 2.0 license together with a gpt-oss usage policy (Source: https://openai.com/index/gpt-oss-model-card/). It was OpenAI's first release of model weights since GPT-2 in 2019. The card distinguishes itself from a system card on the grounds that the models will be used in a wide range of systems by stakeholders OpenAI cannot control, so it focuses on default-deployment safety and adversarial-fine-tuning robustness rather than full system-level safeguards. The most-discussed element is an adversarial-fine-tuning experiment in which OpenAI attempted to push gpt-oss-120b across High capability thresholds in biological/chemical and cybersecurity domains, concluding that the release did not advance the frontier of those capabilities in open models.
Models and architecture
The card documents two model sizes, both autoregressive Mixture-of-Experts (MoE) transformers building on the GPT-2/GPT-3 architectures and post-trained with MXFP4 quantization (4.25 bits per parameter) on the MoE weights. The larger model fits on a single 80GB GPU; the smaller runs on systems with as little as 16GB memory. OpenAI describes the models as comparable to o4-mini in capability.
| Model | Total params | Active params | Layers | Experts | Checkpoint |
|---|---|---|---|---|---|
| gpt-oss-120b | 116.8B | 5.13B | 36 | 128 (top-4) | 60.8 GiB |
| gpt-oss-20b | 20.9B | 3.61B | 24 | 32 (top-4) | 12.8 GiB |
Architectural details given in the card include a residual dimension of 2880 with pre-LN placement and root mean square normalization; an attention pattern alternating banded-window and fully dense layers (bandwidth 128 tokens), with 64 query heads of dimension 64 and grouped-query attention using 8 KV heads; MoE layers with 128 experts (gpt-oss-120b) or 32 experts (gpt-oss-20b), top-4 routing, and gated SwiGLU activation; and a custom tokenizer (specified in §2.3). Pre-training drew on diverse internet and partner data. Post-training targeted reasoning and tool use through Variable Effort Reasoning Training, the Harmony Chat Format, and Agentic Tool Use training. Documented capabilities include web search, Python code execution, full chain-of-thought, and Structured Outputs.
Adversarial-fine-tuning experiment
OpenAI's Safety Advisory Group (SAG) reviewed an experiment in which OpenAI adversarially fine-tuned gpt-oss-120b in an effort to push it across High capability thresholds in biological/chemical and cybersecurity domains, simulating what a determined attacker with a field-leading training stack could achieve. SAG concluded that even with robust fine-tuning leveraging OpenAI's training stack, gpt-oss-120b did not reach High capability in either Biological/Chemical Risk or Cyber risk.
OpenAI also addressed whether releasing gpt-oss-120b would significantly advance the frontier of biological capabilities in open foundation models, and concluded that it would not. The card states: "For most of the evaluations, the default performance of one or more existing open models comes near to matching the adversarially fine-tuned performance of gpt-oss-120b." OpenAI presents this as the first time it has published an adversarial-fine-tuning robustness assessment of its own open-weight release, applying the OpenAI Preparedness Framework V.2 to an open-weights model and reframing the assessment question from the model's default behavior to the behavior a determined attacker fine-tuning the model could elicit.
Default safety performance
The card reports default-deployment safety evaluations across six areas: disallowed content across standard categories (§4.1); robustness to jailbreaks of the base open-weight model (§4.2); instruction-hierarchy adherence, measuring how reliably the model follows the four-level chain of command in the Model Spec (§4.3); hallucinated chains of thought, evaluating CoT veracity (§4.4); standard hallucination evaluations (§4.5); and fairness and bias evaluations (§4.6).
Preparedness Framework coverage
The card presents adversarially-fine-tuned evaluations across three capability tracks. For biological/chemical risk: Long-form Biological Risk Questions, Multimodal Troubleshooting Virology, ProtocolQA Open-Ended, Tacit Knowledge and Troubleshooting, TroubleshootingBench, and external red-teaming by outside safety experts. For cybersecurity: Capture the Flag (CTF) challenges and a cyber range. For AI self-improvement: SWE-bench Verified (N=477), OpenAI PRs, and PaperBench.
Relation to other releases and policy
gpt-oss-120b's Apache 2.0 license is more permissive than the custom license attached to the DeepSeek V3 and DeepSeek V4 models, and the release functions in part as a response to the Chinese open-weights ecosystem. For the Open-Source AI / Open-Weight Models debate, the adversarial-fine-tuning experiment offers evidence cited in support of the position that frontier open-weight models can be released with accompanying evaluation evidence rather than withheld on the grounds that downstream use cannot be controlled. The card also represents OpenAI's first published argument that an open-weight release paired with adversarial-fine-tuning evaluation can be made compatible with national-security risk frameworks, a question that intersects with export-control policy such as the BIS Framework for AI Diffusion — Interim Final Rule (Jan 13, 2025). The four features OpenAI foregrounds are the adversarial-fine-tuning robustness assessment, the direct comparison with existing open-weight models to gauge frontier-advancement, the permissive Apache 2.0 license, and the use of MXFP4 quantization to enable single-80GB-GPU deployment of a 120B-parameter model.
Provenance and confidence
Confidence is high for the documented architecture and capability claims. It is medium for the SAG conclusion that adversarial fine-tuning does not push gpt-oss-120b across High thresholds: this is OpenAI's own assessment of OpenAI's own model. The card cites external red-teamers but is OpenAI-authored, and the conclusion would benefit from corroboration by independent red-team evaluations.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| MMMLU — MMMLU per language — Arabic | gpt-oss-120b | 75.0% | language: Arabic; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | gpt-oss-120b | 80.4% | language: Arabic; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | gpt-oss-120b | 82.7% | language: Arabic; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | gpt-oss-20b | 65.6% | language: Arabic; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | gpt-oss-20b | 73.4% | language: Arabic; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | gpt-oss-20b | 76.3% | language: Arabic; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | o3-mini | 81.9% | language: Arabic; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | o4-mini | 86.1% | language: Arabic; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Arabic | o3 | 90.4% | language: Arabic; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-120b | 71.5% | language: Bengali; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-120b | 78.3% | language: Bengali; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-120b | 80.9% | language: Bengali; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-20b | 68.3% | language: Bengali; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-20b | 74.9% | language: Bengali; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | gpt-oss-20b | 77.1% | language: Bengali; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | o3-mini | 80.1% | language: Bengali; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | o4-mini | 84.0% | language: Bengali; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Bengali | o3 | 87.8% | language: Bengali; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-120b | 77.9% | language: Chinese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-120b | 82.1% | language: Chinese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-120b | 83.6% | language: Chinese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-20b | 72.1% | language: Chinese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-20b | 78.0% | language: Chinese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | gpt-oss-20b | 79.4% | language: Chinese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | o3-mini | 83.6% | language: Chinese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | o4-mini | 86.9% | language: Chinese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Chinese | o3 | 89.3% | language: Chinese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-120b | 79.6% | language: French; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-120b | 83.3% | language: French; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-120b | 84.6% | language: French; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-20b | 73.2% | language: French; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-20b | 78.6% | language: French; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | gpt-oss-20b | 80.2% | language: French; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | o3-mini | 83.7% | language: French; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | o4-mini | 87.4% | language: French; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — French | o3 | 90.6% | language: French; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-120b | 78.6% | language: German; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-120b | 81.7% | language: German; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-120b | 83.0% | language: German; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-20b | 71.4% | language: German; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-20b | 77.2% | language: German; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | gpt-oss-20b | 78.7% | language: German; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | o3-mini | 80.8% | language: German; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | o4-mini | 86.7% | language: German; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — German | o3 | 90.5% | language: German; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-120b | 74.2% | language: Hindi; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-120b | 80.0% | language: Hindi; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-120b | 82.2% | language: Hindi; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-20b | 70.2% | language: Hindi; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-20b | 76.6% | language: Hindi; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | gpt-oss-20b | 78.8% | language: Hindi; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | o3-mini | 81.1% | language: Hindi; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | o4-mini | 85.9% | language: Hindi; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Hindi | o3 | 89.8% | language: Hindi; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-120b | 78.3% | language: Indonesian; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-120b | 82.8% | language: Indonesian; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-120b | 84.3% | language: Indonesian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-20b | 71.2% | language: Indonesian; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-20b | 77.4% | language: Indonesian; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | gpt-oss-20b | 79.5% | language: Indonesian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | o3-mini | 82.8% | language: Indonesian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | o4-mini | 86.9% | language: Indonesian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Indonesian | o3 | 89.8% | language: Indonesian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-120b | 79.5% | language: Italian; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-120b | 83.7% | language: Italian; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-120b | 85.0% | language: Italian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-20b | 73.6% | language: Italian; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-20b | 79.0% | language: Italian; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | gpt-oss-20b | 80.5% | language: Italian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | o3-mini | 83.8% | language: Italian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | o4-mini | 87.7% | language: Italian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Italian | o3 | 91.2% | language: Italian; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-120b | 77.0% | language: Japanese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-120b | 82.0% | language: Japanese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-120b | 83.5% | language: Japanese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-20b | 70.4% | language: Japanese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-20b | 76.9% | language: Japanese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | gpt-oss-20b | 78.8% | language: Japanese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | o3-mini | 83.1% | language: Japanese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | o4-mini | 86.9% | language: Japanese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Japanese | o3 | 89.0% | language: Japanese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-120b | 75.2% | language: Korean; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-120b | 80.9% | language: Korean; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-120b | 82.9% | language: Korean; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-20b | 69.8% | language: Korean; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-20b | 75.7% | language: Korean; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | gpt-oss-20b | 77.6% | language: Korean; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | o3-mini | 82.6% | language: Korean; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | o4-mini | 86.7% | language: Korean; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Korean | o3 | 89.3% | language: Korean; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-120b | 80.0% | language: Portuguese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-120b | 83.3% | language: Portuguese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-120b | 85.3% | language: Portuguese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-20b | 73.3% | language: Portuguese; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-20b | 79.2% | language: Portuguese; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | gpt-oss-20b | 80.5% | language: Portuguese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | o3-mini | 84.1% | language: Portuguese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | o4-mini | 87.8% | language: Portuguese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Portuguese | o3 | 91.0% | language: Portuguese; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-120b | 80.6% | language: Spanish; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-120b | 84.6% | language: Spanish; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-120b | 85.9% | language: Spanish; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-20b | 75.0% | language: Spanish; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-20b | 79.7% | language: Spanish; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | gpt-oss-20b | 81.2% | language: Spanish; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | o3-mini | 84.0% | language: Spanish; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | o4-mini | 88.0% | language: Spanish; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Spanish | o3 | 91.1% | language: Spanish; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-120b | 59.9% | language: Swahili; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-120b | 69.3% | language: Swahili; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-120b | 72.3% | language: Swahili; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-20b | 46.2% | language: Swahili; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-20b | 56.6% | language: Swahili; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | gpt-oss-20b | 60.7% | language: Swahili; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | o3-mini | 73.8% | language: Swahili; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | o4-mini | 81.3% | language: Swahili; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Swahili | o3 | 86.0% | language: Swahili; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-120b | 49.7% | language: Yoruba; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-120b | 58.1% | language: Yoruba; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-120b | 62.4% | language: Yoruba; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-20b | 38.4% | language: Yoruba; reasoning level low; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-20b | 45.8% | language: Yoruba; reasoning level medium; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | gpt-oss-20b | 50.1% | language: Yoruba; reasoning level high; professionally human-translated MMLU; answers parsed from the response; developer-reported | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | o3-mini | 63.7% | language: Yoruba; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | o4-mini | 70.8% | language: Yoruba; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| MMMLU — MMMLU per language — Yoruba | o3 | 78.0% | language: Yoruba; reasoning level high; professionally human-translated MMLU; answers parsed from the response; reported by OpenAI in the gpt-oss model card | 2.6.3 (Table 2) |
| Standard Disallowed Content Evaluation — hate (aggregate) | gpt-oss-120b | 0.996 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — hate (aggregate) | gpt-oss-20b | 0.996 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — hate (aggregate) | o4-mini | 0.996 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — hate (aggregate) | GPT-4o | 0.996 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — self-harm/intent and self-harm/instructions | gpt-oss-120b | 0.995 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — self-harm/intent and self-harm/instructions | gpt-oss-20b | 0.984 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — self-harm/intent and self-harm/instructions | o4-mini | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — self-harm/intent and self-harm/instructions | GPT-4o | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/semi-restrictive | gpt-oss-120b | 0.967 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/semi-restrictive | gpt-oss-20b | 0.947 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/semi-restrictive | o4-mini | 0.975 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/semi-restrictive | GPT-4o | 0.967 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/exploitative | gpt-oss-120b | 1.000 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/exploitative | gpt-oss-20b | 0.980 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/exploitative | o4-mini | 0.990 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/exploitative | GPT-4o | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/minors | gpt-oss-120b | 1.000 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/minors | gpt-oss-20b | 0.971 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/minors | o4-mini | 0.990 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — sexual/minors | GPT-4o | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/non-violent | gpt-oss-120b | 1.000 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/non-violent | gpt-oss-20b | 0.983 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/non-violent | o4-mini | 0.991 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/non-violent | GPT-4o | 0.983 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/violent | gpt-oss-120b | 1.000 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/violent | gpt-oss-20b | 1.000 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/violent | o4-mini | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — illicit/violent | GPT-4o | 1.000 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/restricted | gpt-oss-120b | 0.996 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/restricted | gpt-oss-20b | 0.978 | not_unsafe; LLM-based grading models; default model; developer-reported | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/restricted | o4-mini | 0.955 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Standard Disallowed Content Evaluation — personal-data/restricted | GPT-4o | 0.978 | not_unsafe; LLM-based grading models; default model; reported by OpenAI in the gpt-oss model card | 4.1 (Table 4) |
| Production Benchmarks — non-violent hate | gpt-oss-120b | 0.895 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — non-violent hate | gpt-oss-20b | 0.901 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — non-violent hate | o4-mini | 0.832 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — non-violent hate | GPT-4o | 0.882 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — personal-data | gpt-oss-120b | 0.888 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — personal-data | gpt-oss-20b | 0.921 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — personal-data | o4-mini | 0.847 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — personal-data | GPT-4o | 0.860 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — harassment/threatening | gpt-oss-120b | 0.832 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — harassment/threatening | gpt-oss-20b | 0.819 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — harassment/threatening | o4-mini | 0.695 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — harassment/threatening | GPT-4o | 0.745 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — sexual/exploitative | gpt-oss-120b | 0.919 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — sexual/exploitative | gpt-oss-20b | 0.852 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — sexual/exploitative | o4-mini | 0.857 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — sexual/exploitative | GPT-4o | 0.927 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — sexual/minors | gpt-oss-120b | 0.967 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — sexual/minors | gpt-oss-20b | 0.866 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — sexual/minors | o4-mini | 0.862 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — sexual/minors | GPT-4o | 0.939 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — extremism | gpt-oss-120b | 0.932 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — extremism | gpt-oss-20b | 0.951 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — extremism | o4-mini | 0.932 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — extremism | GPT-4o | 0.919 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — hate/threatening | gpt-oss-120b | 0.898 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — hate/threatening | gpt-oss-20b | 0.829 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — hate/threatening | o4-mini | 0.795 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — hate/threatening | GPT-4o | 0.867 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — illicit/nonviolent | gpt-oss-120b | 0.692 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — illicit/nonviolent | gpt-oss-20b | 0.656 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — illicit/nonviolent | o4-mini | 0.658 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — illicit/nonviolent | GPT-4o | 0.573 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — illicit/violent | gpt-oss-120b | 0.817 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — illicit/violent | gpt-oss-20b | 0.744 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — illicit/violent | o4-mini | 0.845 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — illicit/violent | GPT-4o | 0.633 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — self-harm/intent | gpt-oss-120b | 0.950 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — self-harm/intent | gpt-oss-20b | 0.893 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — self-harm/intent | o4-mini | 0.862 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — self-harm/intent | GPT-4o | 0.849 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — self-harm/instructions | gpt-oss-120b | 0.910 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — self-harm/instructions | gpt-oss-20b | 0.899 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; developer-reported | 4.1 (Table 5) |
| Production Benchmarks — self-harm/instructions | o4-mini | 0.901 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Production Benchmarks — self-harm/instructions | GPT-4o | 0.735 | not_unsafe; LLM-based grading models; multi-turn conversations representative of production data, designed to be more challenging; reported by OpenAI in the gpt-oss model card | 4.1 (Table 5) |
| Instruction Hierarchy Evaluation - System <> User message conflict — System prompt extraction | gpt-oss-120b | 0.832 | higher is better; developer-reported | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - System <> User message conflict — System prompt extraction | gpt-oss-20b | 0.881 | higher is better; developer-reported | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - System <> User message conflict — System prompt extraction | o4-mini | 0.993 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - System <> User message conflict — Prompt injection hijacking | gpt-oss-120b | 0.780 | higher is better; developer-reported | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - System <> User message conflict — Prompt injection hijacking | gpt-oss-20b | 0.639 | higher is better; developer-reported | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - System <> User message conflict — Prompt injection hijacking | o4-mini | 0.917 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 7) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - system message/user message | gpt-oss-120b | 0.912 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - system message/user message | gpt-oss-20b | 0.793 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - system message/user message | o4-mini | 0.937 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - system message/user message | gpt-oss-120b | 0.965 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - system message/user message | gpt-oss-20b | 0.947 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - system message/user message | o4-mini | 0.982 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - developer message/user message | gpt-oss-120b | 0.909 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - developer message/user message | gpt-oss-20b | 0.661 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Phrase protection - developer message/user message | o4-mini | 0.912 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - developer message/user message | gpt-oss-120b | 1.000 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - developer message/user message | gpt-oss-20b | 0.946 | higher is better; developer-reported | 4.3 (Table 8) |
| Instruction Hierarchy Evaluation - Phrase and Password Protection — Password protection - developer message/user message | o4-mini | 0.947 | higher is better; reported by OpenAI in the gpt-oss model card | 4.3 (Table 8) |
| PersonQA — accuracy | gpt-oss-120b | 0.298 | no browsing; hallucination rate lower is better; developer-reported | 4.5 (Table 9) |
| PersonQA — accuracy | gpt-oss-20b | 0.155 | no browsing; hallucination rate lower is better; developer-reported | 4.5 (Table 9) |
| PersonQA — accuracy | o4-mini | 0.356 | no browsing; hallucination rate lower is better; reported by OpenAI in the gpt-oss model card | 4.5 (Table 9) |
| PersonQA — hallucination rate | gpt-oss-120b | 0.491 | no browsing; hallucination rate lower is better; developer-reported | 4.5 (Table 9) |
| PersonQA — hallucination rate | gpt-oss-20b | 0.532 | no browsing; hallucination rate lower is better; developer-reported | 4.5 (Table 9) |
| PersonQA — hallucination rate | o4-mini | 0.361 | no browsing; hallucination rate lower is better; reported by OpenAI in the gpt-oss model card | 4.5 (Table 9) |
| BBQ — Accuracy on ambiguous questions | gpt-oss-120b | 0.87 | higher is better; developer-reported | 4.6 (Table 10) |
| BBQ — Accuracy on ambiguous questions | gpt-oss-20b | 0.79 | higher is better; developer-reported | 4.6 (Table 10) |
| BBQ — Accuracy on ambiguous questions | o4-mini | 0.82 | higher is better; reported by OpenAI in the gpt-oss model card | 4.6 (Table 10) |
| BBQ — Accuracy on disambiguated questions | gpt-oss-120b | 0.90 | higher is better; developer-reported | 4.6 (Table 10) |
| BBQ — Accuracy on disambiguated questions | gpt-oss-20b | 0.89 | higher is better; developer-reported | 4.6 (Table 10) |
| BBQ — Accuracy on disambiguated questions | o4-mini | 0.95 | higher is better; reported by OpenAI in the gpt-oss model card | 4.6 (Table 10) |
| Tacit Knowledge and Troubleshooting | Consensus expert baseline | 80% | multiple-choice dataset created with Gryphon Scientific; model scores appear only in Figure 8 | 5.2.1.4 |
| Tacit Knowledge and Troubleshooting | 80th percentile PhD expert baseline | 63% | multiple-choice dataset created with Gryphon Scientific; model scores appear only in Figure 8 | 5.2.1.4 |
| TroubleshootingBench | 80th percentile expert score (12 PhD experts) | 36.4% | 52 expert-written protocols with three troubleshooting questions each; short answer; used as an indicative threshold; model scores appear only in Figure 9 | 5.2.1.5 |
| Cyber range — Without hints (primary metric) | All models evaluated | 0% | 5 range scenarios; 30 trials per eval; pass@12 without hints; card states "No model is able to solve any scenario unaided or with hints"; developer-reported | 5.2.2.2 |
Discrepancies noted
Figure 4's data label gives gpt-oss-120b's HealthBench Consensus score as 90.0 where Table 3 gives 89.9.
Relationships
- supports: gpt-oss (OpenAI open-weight models) (canonical primary reference), OpenAI Preparedness Framework V.2 (first documented application to an open-weights release), Open-Source AI / Open-Weight Models (methodology contribution)
- related: DeepSeek-V3, DeepSeek V4 Pro / V4 Flash, BIS Framework for AI Diffusion — Interim Final Rule (Jan 13, 2025), IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs (OpenAI, March 2026) (planned — Instruction Hierarchy is referenced in §4.3 alongside the IH-Challenge PDF in the same Apr 27 cohort)
- instance-of: AI System Cards