UK defence policy retains MOD accountability for AI assurance while suppliers may control key records. Buyers need proportionate, authorised ways to examine claims and record what remains unknown.
Executive takeaways
- Specify decision-grade evidence early. Identify material acceptance claims, the system configuration, evidence owners, review permissions and delivery milestones before contract commitment.
- Treat protected evidence as a review-design problem, not automatic failure. Use lawful alternatives where suitable, and record claims that remain unexamined.
- Carry evidence into operation. Establish change triggers, exceptional-condition testing and links to existing safety, security and ethical assurance decisions.
A supplier summary does not make the buyer’s decision
An acquiring authority can receive a substantial supplier dossier without knowing whether its intended configuration was evaluated under relevant operational conditions. A summary cannot replace the buyer’s decision about what was tested, which records an authorised reviewer examined, and which risks remain unresolved.
The UK Ministry of Defence (MOD) states that AI assurance is a shared activity with suppliers, but responsibility for assessing compliance with Joint Service Publication (JSP) 936 remains with MOD [1]. ODA3 Institute’s central recommendation is therefore practical: make the usability of assurance evidence an explicit procurement deliverable, scaled to mission consequence rather than document volume.
UK policy links supplier confidence to proportionate assurance
JSP 936 Part 1 is the principal MOD directive on dependable AI [2]. The distinctions in its requirements matter:
| Directive provision | What the published text says | Procurement implication |
|---|---|---|
| Paragraphs 198, 200–202 | Externally acquired AI must attract appropriate confidence; supplier competence, third-party software assurance arguments and training/test-data provenance carry specified requirements | Define what evidence will support those obligations for the intended use |
| Paragraphs 203–204 | Contracting should consider through-life access to data, algorithms and models; assessment of foreign export-control restrictions should protect operational freedom of action | Negotiate feasible rights and access rather than assume automatic source disclosure |
| Paragraphs 196–197 and 209 | Security analysis must address poisoning and adversarial interference; assurance must cover foreseeable excursions beyond the operational design domain | Examine behaviour under relevant adversarial and off-nominal conditions, not only nominal accuracy |
Paragraph 125 also requires analysis of human–AI allocation across modes of operation to address accountability and control [3]. Paragraph 206 recognises that some assurance risks cannot be tolerated; where they can, the assurance activity must be commensurate with risk [3]. None of this creates a universal procurement checklist for other countries.
The Defence Science and Technology Laboratory’s Secure by Design Problem Book discusses how intellectual property and export restrictions can constrain security-evidence sharing [4]. It is a defence cybersecurity research-problem publication, not a measured study of AI procurement failures. Such restrictions can be legitimate, but their existence should not silently transform an unexamined supplier statement into confirmed assurance.
Resolve restricted-evidence cases without assuming unrestricted access
ODA3 Institute proposes a proportional sequence for an acquisition team:
- Describe the claim and materiality. Link each acceptance claim to configuration, intended use, failure consequence, operating environment and applicable criterion.
- Establish the lawful disclosure boundary. Identify what is releasable, what an appropriately authorised reviewer may examine under protection, and what remains withheld with a documented reason. Classification, export licensing, intellectual property and national-security approvals require case-specific legal and security decisions.
- Select a feasible examination route. Options for consideration include a buyer-authorised on-site inspection, independently witnessed tests, reproducible black-box challenge tests, or appropriately controlled third-party review. Cryptographic attestations and trusted-execution-environment evidence may help establish selected integrity properties if technically applicable; they do not prove AI safety, test representativeness or control effectiveness by themselves.
- Record the residual decision. Where evidence cannot be inspected, identify the affected claim as unexamined. The accountable authority may seek more tests, limit use, defer acceptance or reject the capability according to the governing rules and risk tolerance.
These are candidate patterns, not MOD-approved substitute procedures. A supplier should know which claims, records and derived review outputs will be requested, who may rely on them, and how commercially sensitive material will be protected. A buyer should avoid asking a new entrant for more documentation than the system’s risk and decision justify.
Six questions for an evidence-usability review
| Buyer question | Evidence an authorised reviewer could examine | What it can establish—and what it cannot |
|---|---|---|
| What exact system is being accepted? | Model/build identifiers, deployed settings, change and release records | Configuration traceability, not future behaviour |
| Does testing match intended use? | Test plans, representative environments, limitations and exception logs | Observed results within tested bounds, not all mission conditions |
| What happens outside design conditions or under attack? | Adversarial/security evaluations, poisoning analysis, failure-mode and recovery tests | Tested resilience and failure responses, not immunity to adversaries [3] |
| Do humans retain defined accountability and control? | Human–AI function allocation, override tests, operator procedures | Design and observed control evidence within scope, not legal conclusions [3] |
| What remains inaccessible? | Disclosure schedule, export/classification constraints, authorised review log | Which claims could be examined; withheld records remain unknown |
| How will evidence survive change? | Update notices, regression tests, versioned system cards, retention and re-evaluation terms | Agreed continuity process, not proof it will be followed |
The level of examination should reflect consequence, operational context, foreseeable misuse and supplier maturity. The matrix structures a decision; it does not impose one evidential threshold on every AI application.
Examination needs authority, competence and a defined reliance boundary
Before appointing an independent examiner, a buyer should establish technical and domain competence; independence and disclosed conflicts; lawful permission to receive the material in the relevant jurisdiction; suitability of facilities, personnel and data handling; and who may rely on the work product. Buyer commissioning can reduce some supplier incentives but does not itself establish independence. Clearance, export authorisation and secure facilities cannot be inferred from an examiner’s technical credentials.
An examiner without protected access may still assess public or unclassified materials, synthetic tests, proposed handover completeness and the scope of missing evidence. It cannot claim to have inspected restricted originals. No defence clearance, protected-system access or completed examination by ODA3 Institute is claimed here.
Put evidence into existing decisions—not a parallel paperwork system
MOD guidance already supports documented lifecycle material, including version-controlled system cards [5]. JSP 936 also addresses AI ethical assurance and auditable records for senior decision-makers [3]. Evidence handovers should therefore feed the relevant system card, safety and security case, risk acceptance record and ethical assurance statement where applicable. The exact reporting chain depends on the MOD organisation and programme; these artefacts are not interchangeable.
| Acquisition point | Accountable role | Evidence decision |
|---|---|---|
| Early business-case / requirements stage | Programme accountable lead and technical authority | Identify material AI claims and proportionate review depth; use the programme’s actual gate terminology |
| Tender and contract design | Commercial, legal and security leads | Agree disclosure categories, reviewer authorisations, change notification and access rights |
| Before acceptance | Authorised acceptance authority | Record examined claims, exceptions and explicitly unexamined dependencies |
| Through-life changes | Technical and contract owners | Reconcile deployed configuration with evidence; trigger proportionate re-evaluation |
These milestones are recommendations, not claims that every MOD acquisition follows an identical Outline Business Case or Initial Gate process. Specific retention periods, notice deadlines and facility arrangements require contractual and security determination; this article prescribes none.
A cross-sector evidence question, not a defence insurance conclusion
ODA3 Institute’s broader research asks whether independently examined AI-control evidence could inform underwriting of AI-related risks. Defence capability acceptance is not insurance underwriting, and no defence-specific insurer evidence requirement, premium outcome or coverage decision was verified here. Technical failure, financial loss, legal liability and insurance coverage remain separate questions. This research proposition is not needed to accept the procurement recommendations above.
Notably Absent
No classified defence AI system, acquisition dossier, contract, protected review, acceptance decision or programme outcome was examined for this article. The sources do not quantify how frequently supplier records are unusable, establish that the alternatives proposed here improve cost or delivery time, or demonstrate that remote attestation substitutes for original testing. Neither ODA3 examiner clearances nor MOD appointment are claimed. No external jurisdiction’s procurement regime or specific export-licensing determination was assessed. No insurer requirement or coverage outcome was examined. These missing outcomes are research questions, not evidence that a particular supplier or programme failed.
The acceptance question
Before approving a capability, the accountable authority should be able to identify the evaluated configuration, the material claims, the records examined under proper authority and the uncertainties accepted. The operational objective is not unlimited disclosure; it is a defensible relationship between the decision and what could actually be checked.
References
[1] UK Ministry of Defence. “Responsibility for assuring AI in Defence.” AI Practitioner’s Handbook, published and updated 27 February 2026. https://www.digital.mod.uk/policy-rules-standards-and-guidance/aiph/responsibility-for-assuring-ai-in-defence (accessed 23 October 2026).
[2] UK Ministry of Defence. “JSP 936: Dependable Artificial Intelligence (AI) in defence (part 1: directive).” GOV.UK, 13 November 2024. https://www.gov.uk/government/publications/jsp-936-dependable-artificial-intelligence-ai-in-defence-part-1-directive (accessed 23 October 2026).
[3] UK Ministry of Defence. JSP 936 Part 1, version 1.1, November 2024, especially paragraphs 87, 99, 125, 196–197 and 198–209. https://assets.publishing.service.gov.uk/media/6735fc89f6920bfb5abc7b62/JSP936_Part1.pdf (accessed 23 October 2026).
[4] Defence Science and Technology Laboratory / UK MOD. “Secure by Design Problem Book.” GOV.UK, 24 April 2025. https://www.gov.uk/government/publications/secure-by-design-problem-book/secure-by-design-problem-book (accessed 23 October 2026).
[5] UK Ministry of Defence. “Documenting, storing and sharing system cards.” AI Practitioner’s Handbook. https://www.digital.mod.uk/policy-rules-standards-and-guidance/aiph/development/guidance-on-system-cards/documenting-storing-and-sharing-system-cards (accessed 23 October 2026; publication date not independently confirmed for this revision).
Methodology note
Source-bounded policy analysis; distinction between supplier assertions, lawfully examined evidence, unexamined claims and acceptance authority. No classified examination conducted.
