An April 29, 2026 publication by OpenAI accounting for why its models from GPT-5.1 onward increasingly used goblins, gremlins, and similar creatures in metaphors.
Why the case is unusual
The document opens by distinguishing this failure from the kind engineering processes are built to catch: "Unlike model bugs that show up through a tanking eval or a spiking training metric and point back to a specific change, this one crept in subtly. A single 'little goblin' in an answer could be harmless, even charming."
That is the general observation the case supplies — a behavior below the threshold of any individual evaluation, detectable only as an aggregate rate across generations.
Detection
The pattern was first seen in November 2025 after the GPT-5.1 launch, following user complaints "about the model being oddly overfamiliar in conversation." A safety researcher who had encountered "goblins" and "gremlins" asked for them to be included in a verbal-tic check. Use of "goblin" in ChatGPT "had risen by 175% after the launch of GPT-5.1, while 'gremlin' had risen by 52%." OpenAI states that at the time "the prevalence of goblins did not look especially alarming."
A larger uptick with GPT-5.4 triggered a second analysis, which located the concentration: creature language was common in production traffic from users who had selected the "Nerdy" personality, which "accounted for only 2.5% of all ChatGPT responses, but 66.7% of all 'goblin' mentions." OpenAI's inference from the distribution: "If the behavior were simply a broad internet trend, we would expect it to spread more evenly. Instead, it was clustered in the part of the system explicitly optimized for a playful, nerdy style."
Root cause
Comparing RL-training outputs containing creature words against outputs to the same task without them isolated a single reward signal: "the one originally designed to encourage the Nerdy personality was consistently more favorable to the creature-word outputs," showing "positive uplift in 76.2% of datasets."
The finding with the widest application is the transfer. The reward "were applied only in the Nerdy condition, but reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them." Tracking mention rates with and without the Nerdy prompt showed that as creature-word rates rose under the personality, "they increased by nearly the same relative proportion in samples without it."
OpenAI describes the amplification as a five-step loop: playful style is rewarded; some rewarded examples contain a distinctive lexical tic; the tic appears more often in rollouts; model-generated rollouts are used for supervised fine-tuning; the model "gets even more comfortable producing the tic." A search of GPT-5.5's SFT data confirmed many datapoints containing the words, and identified raccoons, trolls, ogres, and pigeons as further tic words.
Remediation
The "Nerdy" personality was retired in March 2026 after the GPT-5.4 launch; the goblin-affine reward signal was removed and creature-word training data filtered. GPT-5.5 had begun training before the root cause was found, so the behavior persisted into Codex testing and was mitigated by a developer-prompt instruction. The post supplies a shell command for removing that instruction.
OpenAI's stated general lesson is that the episode is "a powerful example of how reward signals can shape model behavior in unexpected ways, and how models can learn to generalize rewards in certain situations to unrelated ones," and reports that the investigation produced "new tools for the research team to audit model behavior and fix behavior problems at their root."
Relationships
- supports: Reward Hacking — a documented production case of reward misspecification generalizing outside its intended condition
- related: GPT-5.5 ('Spud') — where the behavior surfaced most visibly, in Codex testing
- related: GPT-5 Family (OpenAI), OpenAI, RLHF (Reinforcement Learning from Human Feedback)