NIST AI 200-2 ipd — The TEVV-Athlon Framework for Evaluating AI Systems (initial public draft, August 2026)

This is the initial public draft of NIST AI 200-2, The TEVV-Athlon Framework for Evaluating AI Systems, published by NIST in the NIST Trustworthy and Responsible AI series with a cover date of August 2026 and announced on August 7, 2026. It is authored by P. Jonathon Phillips, Theodore Jensen, Patrick Hall, Razvan Amironesei, Yee-Yin Choong, Craig Greenberg and Kristen K. Greene; Phillips, Jensen, Amironesei, Choong, Greenberg and Greene are in the Technology Test and Evaluation Division of the Information Technology Laboratory, and Hall is listed as a NIST Associate affiliated with HallResearch.ai. The document is available free of charge at https://doi.org/10.6028/NIST.AI.200-2.ipd. The topic page is NIST AI 200-2 — The TEVV-Athlon Framework for Evaluating AI Systems.

The draft introduces a method — not a benchmark suite — for building an assessment of an AI system from an organisation's own objectives. Its stated motivation is that the NIST AI Risk Management Framework "specifically calls for a Test, Evaluation, Verification, and Validation (TEVV) methodology," which the AI RMF itself does not supply.

Scope and framing

The draft adopts the definition of TEVV given in the final report of the National Security Commission on Artificial Intelligence: "A framework for assessing, incorporating methods and metrics to determine that a technology or system satisfactorily meets its design specifications and requirements, and that it is sufficient for its intended use."

NIST describes the design problem as a "Goldilocks" challenge: "a universal, all-encompassing framework is often too unwieldy for practical application, whereas a highly specialized one lacks the versatility required for a broad spectrum of AI use cases." The framework is stated to be aimed at balancing comprehensiveness and flexibility, and to be applicable to statistical machine learning models, large language models, multi-modal models and agentic systems. A displayed epigraph reads: "Homage to George Box: No evaluation framework is perfect, some are useful."

The draft distinguishes two terms that are easily conflated. The TEVV-Athlon Framework is the methodology for creating an evaluation; a TEVV-Athlon is the assessment itself. The name is drawn from multi-event sporting competitions: "In a TEVV-Athlon, AI systems (rather than athletes) are tested across multiple events to measure an AI system's benefits and risks (rather than athleticism)."

The four stages

The draft's three defined terms are stated as follows:

• Metrology Blocks — "The key concepts or metrics which the evaluation aims to assess, which are selected based on the system attributes or characteristics of interest." Referred to in the text simply as Blocks. Each Block "must include a precise definition specifying the types of information required as evidence."
• Events — "The activities which produce evidence for each Block."
• Toolbox — "Set of methods and instruments used to elicit, collect, and analyze information from the Events."

Stage 1 supplies seven questions for articulating goals, presented as building on the Heilmeier Catechism and the Feynman method: the goals of the assessment stated "using absolutely no jargon"; who will care about the results and how they will make a difference; how long it will take and how much it will cost; the resources needed; which current TEVV techniques the assessment builds on; what in the approach will lead to a successful assessment; and what the challenges are. NIST states that organisers should answer questions 1 and 2 during the first stage and give an approximate answer to question 3, with the rest refined as design proceeds, and that "not all questions may apply."

Relationship to the AI RMF core functions

The draft positions the TEVV-Athlon as the implementation of the AI RMF's Measure function. The Govern and Map functions "serve as inputs to Stage 1," supplying the organisational governance context and the operating environment; the results produced in Stage 4 "inform the Manage core function." This is the draft's principal claim of lineage, and the reason it is presented as answering a call the AI RMF made rather than as a standalone evaluation regime.

The worked example

Section 3 constructs an example TEVV-Athlon for what the draft calls the query-violation problem: a chatbot is expected to respond to a request with relevant information (the query) while avoiding a prohibited category of information (the violation). The example is adapted, with simplifications, from the NIST ARIA 0.1 pilot evaluation.

Three operational environments are specified: TV shows (query: recommend a show or movie; violation: giving away the ending), meal planning (query: provide a meal plan; violation: undesirable food advice), and travel planning (query: provide travel information; violation: false travel information). Two AI RMF trustworthiness characteristics are selected — Valid and Safe — and each is given one Block: Helpfulness for Valid, and Violation Frequency for Safe. The Events are User Testing and Red Teaming; Helpfulness draws on User Testing alone, while Violation Frequency draws on both. The Toolbox comprises User Testing Instructions, Post-Task Questionnaires, Red Teaming Instructions and an Annotation Schema by which human annotators label chats. Stage 4 uses descriptive statistics, including response distributions for the Helpfulness questionnaire items and violation frequencies across both User Testing and Red Teaming sessions.

General guidance

Section 4 sets out practices the draft treats as the practical foundation of any TEVV-Athlon, stating that "no single measurement technique is inherently better than another."

• Gathering appropriate expertise. AI evaluation draws on computer science, signal detection theory, experimental design, psychology, statistics, and quantitative and qualitative social science, plus domain expertise for the application area. The draft notes that decisions made in early planning — how success is defined, which risks are prioritised, how measurements are constructed, which testing methods are chosen — "can shape the usefulness and limitations of the entire TEVV-Athlon."
• Checking basic requirements. Secure computational resources sufficient for repeated tests and storage, and confirm the ability to "execute inference queries directly on the system being measured." A risk-based approach follows: "systems with higher potential risk need more rigorous measurement." Where evaluation involves people, human-subjects review and approval may be required.
• Model testing and benchmarking. Benchmarking is described as a subset of model testing. The draft records its limitations directly: benchmarks "often measure proxies contained in datasets rather than real-world outcomes"; models may encounter benchmark or test data in training, producing contamination and "overly optimistic measurement results"; and optimising for benchmark performance can produce models that test well and fail in deployment. It notes that many widely used benchmarks are open and publicly accessible, which exposes them to leakage, task contamination and saturation, and suggests in-house or application-specific benchmarks as one mitigation. It cross-references the Center for AI Standards and Innovation's Practices for Automated Benchmark Evaluations of Language Models (NIST AI 800-2 ipd) for additional guidance. See NIST CAISI (Center for AI Standards and Innovation).
• Testing in more realistic settings. Red-teaming ("intentionally probing a system for harmful outputs, edge cases, or vulnerabilities"), user testing and field testing are presented as complements to controlled model testing, not replacements: "use real-world testing to complement, rather than replace, controlled model testing."
• Applying best practices for scientific measurement. Clearly define what is being measured, separate training and test data, document assumptions and limitations, report variability and uncertainty, compare against appropriate baselines, and calibrate against ground truth over time.
• Validating measurement. Results should be examined to determine whether they capture what they are intended to measure — the draft uses interrogation and validation for this step — because "measurement results and associated claims can be misleading if they do not account for the limitations of the measurement methodology." Validation is described as ongoing rather than one-time.
• Beware of Goodhart's Law. Metrics "do not only measure AI systems; they also influence how AI systems are designed, optimized, and compared." The draft notes that optimisation for one metric may alter internal model behaviour, embeddings, retrieval performance, calibration or interoperability in unexpected ways, and concludes that "strong benchmark performance or metric scores do not always indicate broader system quality or suitability for deployment."

Appendices

Five appendices supply reference tables, each keyed to trustworthiness characteristics and AI lifecycle stages:

• A — Selected AI benchmarks: automated LLM red-teaming evaluation (naming garak and PyRIT), facial-recognition benchmark evaluation (BFW, NIST FRTE/FATE), LLM benchmark evaluation (HumanEval, LiveBench, MMLU), object-recognition benchmark evaluation (ImageNet, MS COCO) and tabular-data benchmark evaluation (TabArena, PMLB). The appendix is stated to be "a practical reference rather than an exhaustive inventory."
• B — Selected AI red-teaming methods: availability attacks, confidentiality attacks, counterfactual fairness prompting, factuality and knowledge-robustness testing, integrity attacks (including prompt injection and data poisoning), jailbreaking, malicious-code elicitation, and agent/tool abuse testing covering unsafe tool selection, excessive agency and unauthorised action attempts.
• C — Selected user and field testing methods: impact assessment, field pilots, focus groups and interviews, structured human-subject experiments, post-deployment feedback, usability and UX research, and user surveys. The appendix states that informed consent, data protection and legal or ethical approvals may be required for any human-subjects research.
• D — Measurement science considerations: calibration of evaluation processes, documentation of assumptions, documentation for reproducibility, ethical treatment of human subjects, evidence-based claims, independent review or challenge, measurement of uncertainty, reporting limitations, validity, and use of controls or baselines.
• E — Experimental and process design considerations: comparison baselines, data selection, defined evaluation objectives, documentation of procedures, prevention of dataset contamination, repetition and sampling, and identification of variables and conditions.

Comment process

A 60-day comment period opened on August 7, 2026 and closes on October 6, 2026. Comments go to TEVV-Athlon@nist.gov with "NIST AI 200-2" in the subject line, in HTML, ASCII, Word, RTF, Excel or PDF. NIST states that all comments are subject to release under the Freedom of Information Act and requests that comments not contain proprietary information. It also discloses that its researchers and staff "may use a variety of software tools to help summarize or analyze your comments, including AI," and that "your data will not be used to train AI models."

NIST invites input on any aspect of the draft and enumerates six areas in particular: the definitions and uses of the TEVV terms; the flexibility, scope and applicability of the framework across different TEVV processes, systems and contexts; types of AI TEVV processes or activities that may not be adequately addressed by the current framework; the framework's usefulness for developing new TEVV processes and for novel or emerging AI systems; aspects warranting clarification, revision, removal or expansion; and additional concepts, provisions, examples or supporting materials. NIST states it encourages input from organisations experienced in conducting AI evaluations as well as from users of evaluation reports, naming business decision-makers, procurement specialists, researchers and technical staff. The named contact is P. Jonathon Phillips.

The draft carries a disclaimer that the manuscript "was edited with the assistance of Gemini, developed by Google, and ChatGPT, developed by OpenAI," used "to refine language, improve clarity, and enhance readability in accordance with the authors' instructions," with all content, scientific claims and conclusions reviewed and verified by the authors.

Reception

Ike Harris, executive director of the Frontier Security Institute, described the proposal as "the first step in standardizing the way the federal government evaluates AI systems both for itself and for its contractors" (Source: reuters.com). The draft's own scope is stated in organisational rather than federal terms — it addresses organisations generally and does not claim procurement application — so the characterisation extends past what the document asserts.

Provenance

Retrieved on August 9, 2026 by parsing the canonical NIST PDF at nvlpubs.nist.gov, the publisher's own repository, and cross-checked against the NIST landing page. The landing page carries a dcterms.created stamp of August 4, 2026 against a stated announcement date of August 7; the earlier stamp is a staging artifact rather than an event date. Publication history in the document reads "Approved by the NIST Editorial Review Board on [Will add date upon final publication]," consistent with an initial public draft.

Relationships

• depends-on: NIST AI Risk Management Framework 1.0 — the draft implements the AI RMF's Measure function and answers its call for a TEVV methodology.
• related: NIST AI 200-2 — The TEVV-Athlon Framework for Evaluating AI Systems — the topic page.
• related: NIST AI 300-1 — Guidance and Templates for Public-Facing AI Documentation, NIST AI 300-1 ipd — Guidance and Templates for Public-Facing AI Documentation (initial public draft, July 2026), NIST AI Agent Standards Initiative (2026) — adjacent NIST AI publications.
• related: AI Benchmarks and Evaluation — the measurement problems the draft catalogues (contamination, saturation, Goodhart effects).
• related: Jailbreaking and Red Teaming — Appendix B's method taxonomy.
• related: AI Pre-Release Vetting, Government AI Procurement, Federal AI Adoption — Patterns and Tensions — where a federal evaluation method would apply if adopted.
• depends-on: National Institute of Standards and Technology (NIST) — the issuing body.