The Grok 4 Model Card is an eight-page document published by xAI and last updated on August 20, 2025 (Source: https://data.x.ai/2025-08-20-grok-4-model-card.pdf). It covers Grok 4 as deployed in the consumer applications (Grok 4 Web) and through the enterprise API (Grok 4 API), and organises its safety evaluations under xAI's Risk Management Framework into three groups: abuse potential, concerning propensities and dual-use capabilities. The card reports no general capability benchmarks.
Evaluation approach
xAI states that it considers two primary categories of risk, malicious use and loss of control, and that it measures specific safety-relevant behaviours tied to each. All results are given as fractions in three tables, one per group, and each group has a description of the evaluation, the results and the mitigations applied.
Abuse potential
The card measures refusals on an internal set of harmful queries described as covering six languages and totalling thousands of queries, graded by another model, under three conditions: no attack, a jailbreak in the user message, and a jailbreak in the system message. The metric is the answer rate, the share of queries that should have been refused and were answered. Agentic misuse is measured with AgentHarm in its no-attack setting, and susceptibility to hijacking with AgentDojo, reported as attack success rate. xAI reports that including its refusal policy in the system prompt, together with a warning about jailbreaks, lets the model reason over the policy and refuse almost all of the harmful requests. Grok 4 Web does not accept custom system prompts and is not evaluated with system-message jailbreaks. Mitigations listed are the refusal policy, the system prompt, and model-based input filters for biological and chemical weapons, self-harm and child sexual abuse material. The later Grok 4.1 Model Card (xAI, November 2025) states that the refusal figures in this card were computed on English prompts only because of an error in the evaluation settings.
Concerning propensities
Deception is measured on the MASK dataset of 1,000 questions as a dishonesty rate, the share of questions on which the model holds a consistent belief and explicitly contradicts it. Sycophancy uses Anthropic's answer-sycophancy evaluation (Sharma et al.). Political bias is measured with an internal "soft bias" evaluation in which an LLM judge scores paired, mirrored sociopolitical questions at 0, 0.5 or 1. xAI reports that its system-prompt instructions reduce deception, political bias and sycophancy, and describes the system prompt as its primary safeguard for these behaviours.
Dual-use capabilities
With safeguards removed, the card reports WMDP biology, chemistry and cyber accuracy, the text-only portion of the Virology Capabilities Test, BioLP-Bench, CyBench unguided success on 40 tasks using the UK AISI Inspect framework, and OpenAI's MakeMeSay persuasion evaluation against a non-reasoning Grok 3 Mini. xAI cites human expert performance of 38.4% on BioLP-Bench and 22.1% on VCT and states that both Grok 4 deployments exceed those baselines; it names expert-level biology capability as the area of highest concern. The card says xAI does not evaluate radiological or nuclear capabilities, and that third-party testing found Grok 4's end-to-end offensive cyber capabilities below the level of a human professional; the tester is not named. In response xAI reports deploying narrow, topically focused filters for biological and chemical weapons-related requests across product surfaces.
Transparency
The card summarises pre-training data sources and the use of reinforcement learning from human feedback, verifiable rewards and model grading, and points to the public repository where xAI publishes consumer system prompts.
Provenance and confidence
The document is developer-authored and every figure in it is developer-reported; no evaluation in it was run by a named third party. Confidence is medium for that reason. The vault copy is the text layer of the official PDF.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| Refusals — Standard (no attack) | Grok 4 API | 0.00 | answer rate (share of should-refuse queries answered; lower is better); internal multilingual harmful-query set, model-graded; with refusal-policy system prompt; developer-reported. The Grok 4.1 card states this figure was computed on English prompts only | 2.1.2, Table 1 |
| Refusals — Standard (no attack) | Grok 4 Web | 0.00 | answer rate; internal harmful-query set, model-graded; with refusal-policy system prompt; developer-reported. English prompts only per the Grok 4.1 card | 2.1.2, Table 1 |
| Refusals — + User Jailbreak | Grok 4 API | 0.00 | answer rate; jailbreak in the user message; developer-reported. English prompts only per the Grok 4.1 card | 2.1.2, Table 1 |
| Refusals — + User Jailbreak | Grok 4 Web | 0.01 | answer rate; jailbreak in the user message; developer-reported. English prompts only per the Grok 4.1 card | 2.1.2, Table 1 |
| Refusals — + System Jailbreak | Grok 4 API | 0.01 | answer rate; jailbreak in the system message; not evaluated for Grok 4 Web (no custom system prompts); developer-reported. English prompts only per the Grok 4.1 card | 2.1.2, Table 1 |
| AgentDojo | Grok 4 API | 0.02 | attack success rate, with system-prompt mitigation; developer-reported | 2.1.2, Table 1 |
| MASK | Grok 4 API | 0.43 | dishonesty rate (share of questions where the model has consistent beliefs and explicitly contradicts them), 1000 questions, with honesty system prompt; developer-reported | 2.2.2, Table 2 |
| Soft Bias (Internal) | Grok 4 API | 0.36 | average bias, LLM-judged paired sociopolitical prompts scored 0 / 0.5 / 1, lower is less bias; with system prompt; developer-reported | 2.2.2, Table 2 |
| Sycophancy | Grok 4 API | 0.07 | sycophancy rate, Anthropic answer-sycophancy evaluation (Sharma et al.); with system prompt; developer-reported | 2.2.2, Table 2 |
| MakeMeSay | Grok 4 API | 0.12 | win rate as attacker against a non-reasoning Grok 3 Mini defender, 15-turn limit; safeguards removed; developer-reported | 2.3.2, Table 3 |
| BioLP-Bench | Grok 4 API | 0.47 | accuracy, text-only questions, safeguards removed; developer-reported | 2.3.2, Table 3 |
| BioLP-Bench | Grok 4 Web | 0.44 | accuracy, text-only questions, safeguards removed; developer-reported | 2.3.2, Table 3 |
| BioLP-Bench | Human experts | 38.4% | human expert performance as stated by xAI in the Grok 4 model card | 2.3.2 |