AI evaluation produces measurements. Decisions depend on what those measurements can support, which system state they describe, and what remains unresolved.
On 5 September 2026, ODA3 Institute submitted a public comment on the Initial Public Draft of NIST AI 200-2, The TEVV-Athlon Framework for Evaluating AI Systems. We are publishing that comment alongside a subsequent supporting note, From TEVV Evidence to Bounded Findings: Illustrative Records and a Physical AI Worked Example.
The submitted comment is reproduced unchanged. The supporting note develops its recommendations through proposed implementation details and a hypothetical example. It has not been submitted to, reviewed by, or adopted by NIST.
The evaluation-to-conclusion relationship
NIST’s draft describes a flexible four-stage approach: Articulate & Organize, Define & Construct, Apply & Measure, and Synthesize & Interrogate. It connects evaluation goals to Metrology Blocks, Events and Tools, then synthesizes the evidence into organizational insight. It also addresses scientific measurement practices, uncertainty and measurement validity.[1]
ODA3 Institute’s comment focuses on how evidence remains attributable to the evaluated system and how conclusions remain bounded to what the evidence establishes. The supporting note organizes that contribution around three asks:
| Core ask | Practical implication |
|---|---|
| Evidence traceability, with proportionate integrity checks | Retain the relationship between evidence, its source, the Block/Event/Tool, execution context, and material transformations. Assess authenticity, completeness and alteration where material. |
| Binding to the evaluated system state | Identify what was tested and check whether material changes affect subsequent reliance on the results. |
| Explicit reasoning and bounded reporting | Connect evidence to specific propositions, preserve different outcomes, and state interpretation limits. |
These are proposed clarifications and implementation details. They do not require a uniform database, method or outcome taxonomy across AI systems.
Different outcomes support different conclusions
A completed test, an observed refusal, unavailable evidence and an unresolved discrepancy carry different meanings. Collapsing them into a single pass/fail label can conceal the limits of a finding.
The supporting note separates four record dimensions: execution status, observations, evidence availability and bounded conclusion. It also records the proposition examined, material assumptions and plausible alternative explanations, including through a linked inference record when a finding draws on several executions.
This preserves an important distinction: a behavior tested for and not observed is different from a behavior that could not be evaluated. Missing evidence does not establish successful performance or failure.
A physical AI example: operator override
The note uses an AI-enabled mobile robot in a controlled test area to illustrate the record. Its predefined criteria concern override acknowledgement, rejection of new motion commands, and reaching a test-defined stationary condition within a specified interval.
Four hypothetical executions illustrate different conclusions: a recorded response satisfying the specified criteria; acknowledgement with insufficient evidence about the other criteria; a measured response that does not meet the stopping criterion; and rejected-command records accompanied by apparent later motion.
The last case is an apparent discrepancy, not automatic proof that the rejection failed. Earlier commands, inertia, another control path or misaligned timestamps could explain it. Software-level acknowledgement or rejection alone does not establish physical stopping performance.
The example is illustrative and unvalidated. It was not executed, and no data was collected. Its observations and check results are hypothetical. The note provides no evidence about any deployed system and makes no claim of general safety, certification or operational effectiveness.
What this means for decision-makers
For security, governance and procurement teams, the practical question is whether an evaluation report provides enough context to understand a finding and its limits. Which configuration was tested? What evidence supports the proposition? What was unavailable? Which changes would affect reuse?
For executives and boards, the distinction is equally consequential: evidence of a successful tested response supports a bounded finding. It does not establish suitability across every environment, system state or period of operation. Reports should make those boundaries visible rather than leave readers to infer them.
Notably absent
This publication does not establish NIST acceptance, endorsement or adoption of the recommendations. The supporting record and worked example have not been validated in practice. No proprietary system telemetry, empirical test results or comparative performance claims are presented.
Sources
- NIST AI 200-2 ipd, The TEVV-Athlon Framework for Evaluating AI Systems, August 2026: https://doi.org/10.6028/NIST.AI.200-2.ipd
- NIST’s draft announcement and request for input: https://www.nist.gov/artificial-intelligence/ai-research/tevv-athlon-framework-evaluating-ai-systems
- ODA3 Institute’s original comment and subsequent supporting note, available above.
Methodology note
Standards correspondence and subsequent practitioner supporting note. Three asks: evidence traceability and proportionate integrity; binding to evaluated system state; explicit reasoning and bounded reporting. Four record dimensions: execution status, observations, evidence availability and bounded conclusion.
Notably Absent
No NIST acceptance, endorsement or adoption established. Supporting note not submitted to NIST. Physical AI worked example hypothetical and unvalidated; no execution, collected data, proprietary telemetry, certification, comparative performance or general safety conclusion.
