AI assurance architecture

Meta’s generation work filters candidates through executable checks; mutation-guided testing tests whether a candidate detects a fault. Agent guidance requires authority to be bounded at the action boundary.

Generate a negative-amount test from an approved contract, execute it through a scoped gateway, retain the independent fault-detection result and route the evidence to a human reviewer.

The engineer supplies the contract. The agent requests scoped context and proposes a test. The gateway authorizes sandbox execution. CI checks the candidate against a seeded boundary fault. The reviewer receives assertions and test evidence before using existing merge controls.

In Meta’s 86-component Kotlin evaluation, 75% of target test classes gained at least one generated test that built; 57% had one that also passed reliably; 25% had one that also increased line coverage.

Treat generation as a candidate factory. Require independent checks for stability and added test value; use fault detection where possible because coverage alone does not establish semantic quality.

The bars have a common zero-to-100-percent axis. The qualifying share drops from 75% for build success to 57% for reliable passing and 25% for added coverage. These percentages describe test classes, not individual generated tests.

NIST treats evaluation and risk management as lifecycle activities. Evaluation product documentation supports versioned datasets and comparisons; judge calibration still requires local evidence.

Compare baseline and candidate configurations against held-out cases, gate releases on agreed floors, then turn reviewed production failures into regression cases.

Versioned application configurations and a held-out evaluation corpus feed comparable runs and independent scoring. A failed release gate holds the change for investigation. A passing gate leads to controlled deployment and production monitoring; curated failures refresh the corpus.

Lifecycle guidance covers input provenance; agent risk guidance addresses untrusted context and access scope.

Filter retrieval using current authorization, retain a versioned input snapshot and test grounding separately from action permissions.

Approved sources feed scoped retrieval and a minimal context snapshot. The runtime treats that content as evidence. Retrieval checks evaluate relevance and access; the action gateway retains authority.

Lifecycle assurance requires tracking changing system components. Evaluation platforms compare configurations and datasets.

Bind the code, prompt, retrieval, tool policy and evaluator versions to one release identity; keep the run and decision references with it.

A manifest contains model and prompt, retrieval and data, tools and memory, and the evaluation contract. A release identity binds those versions to evaluated runs and approval.

Meta separates generated tests from executable validation and extends test value beyond code coverage through fault-oriented testing.

Review assertions against an approved behavior contract and evaluate side effects. Use relevant faults to challenge a test that merely matches current behavior.

The approved contract and implementation inform a candidate test. Separate oracle review checks intended behavior; independent execution checks both valid code and a seeded fault.

Meta ACH uses modeled faults to drive test generation and checks generated tests against original and mutated code.

Retain both execution results and review mutation relevance. A test gains useful evidence when it distinguishes intended behavior from a meaningful fault.

A fault concern drives a candidate test. It must pass original code and fail a relevant mutated version. Combined evidence reaches review; weak results lead to refining the concern.

Google reports 90.14% root-cause accuracy on 71 reviewed failures and deployment across 52,635 distinct failing tests.

Attach relevant log evidence to a diagnosis, retain reviewer corrections and measure local accuracy separately from reach.

A failing test feeds an assistant and then an engineer review. The outcome record captures corrections and effort. Separate panels show the small accuracy sample and larger deployment population.

FlakyGuard combines targeted execution context with LLM-based repair. Its paper discusses the risk of fixes that weaken test semantics.

Require reproducibility, preserved assertions and independent repeated validation before proposing a merge.

The workflow reproduces a flaky failure, gathers targeted context, proposes a repair and validates it. Semantically weak or unstable repairs return to diagnosis.

NIST emphasizes representative evaluation. Evaluation tooling supports curated datasets and feedback from operational traces.

Separate tuning from release evaluation, group related examples during splitting and preserve adversarial scenarios with explicit expected behavior.

Candidate cases pass through privacy review, deduplication and labeling. They feed development, held-out and adversarial sets with distinct uses.

OWASP identifies risks arising from untrusted inputs and agent tool use. OSFI’s July 2026 bulletin discusses agent identity, scoped tool access, traceability and resilience as sound practices.

Test the system’s response when poisoned context asks for an unauthorized action. Assert no side effect, a recorded denial and a bounded escalation path—not just a polite refusal in the model’s answer.

Untrusted documents, code, tool output and memory reach the agent runtime. The agent can request an action, but the gateway checks identity, resource and policy. Permitted calls reach scoped tools. Denied calls cause no side effect and stop or escalate.

Agent risk guidance identifies tool misuse and identity problems. OSFI discusses scoped identities, tool restrictions and activity logs.

Check policy at the gateway before invocation. A denial must produce no tool side effect and a traceable reason.

The runtime requests a production write. The gateway evaluates scope, records a denial and returns it to the runtime. The scoped tool is never invoked.

Agent guidance emphasizes scoped tool access and separation of consequential actions. Industrial generation patterns use independent execution checks.

Keep workers without production credentials, isolate test execution and allow only reviewed artifacts through existing release controls.

An orchestration worker invokes an ephemeral sandbox. Controlled artifacts reach review, then protected release systems. Production authority never belongs to the generating worker.

Lifecycle assurance considers system-specific risks and uncertainty. Evaluation tools support run and case-level comparisons.

Use mandatory floors for critical behaviors, then inspect the distribution of changes before authorizing rollout.

Comparable runs reach a mandatory-floor check. Failure holds the candidate. Passing candidates proceed to review of scenario-level regressions and remaining uncertainty.

Lifecycle and agent guidance make recovery relevant to assurance. OSFI discusses tested AI failure scenarios and continuity measures.

Separate containment, fallback and authorized resumption; explicitly handle actions that cannot be reversed by a model rollback.

A trigger moves normal operation to containment and fallback. Recovery review verifies a fix before an owner authorizes resumption. Completed side effects require separate reconciliation.

Evaluation workflows use operational feedback, while agent risk guidance calls attention to tool and action behavior.

Join application events, gateway decisions and reviewer corrections without broadly copying sensitive raw prompts into monitoring systems.

Three event streams enter a controlled telemetry boundary. The joined records support operations, platform monitoring and reviewed regression-case curation.

DORA’s qualitative work highlights verification friction. Experimental findings depend on the task and population; reported AI impact is not a universal savings rate.

Compare like tasks and account for preparation, execution, review, correction and rework. Link quality and cost to the same cohort; recognize monetary value only with a confirmed capture mechanism.

A task has multiple runs; a run may have several review records. Outcomes link through task and artifact identifiers. A cohort comparison joins baseline effort, run and review effort, rework and defects. Version identifiers make changes in tool configuration auditable.

The sources address different failure modes: verification effort, weak generated tests, lifecycle change and the consequences of agent actions.

Translate each finding into an observable design requirement. Review burden needs effort telemetry; test generation needs a quality gate; changing AI needs repeatable evaluation; tool use needs enforceable permissions.

DORA informs joined effort measurement. Meta informs independent test-quality gates. NIST informs versioned evaluation and feedback. OWASP and OSFI inform external permission enforcement. Each design response has an associated evidence record.

Lifecycle and action-risk guidance require versioned evidence and scoped authority.

Provide joined artifacts and negative fixtures so the logical design can be exercised.

A payment test task binds permitted context, resource scope and deadline. A policy decision authorizes only the sandbox action. An evaluation manifest joins artifact and corpus versions. The evidence record links task, policy and evaluation and remains held for human review.

AutoDiagnose informs evidence-based failure triage. FlakyGuard reports reproducibility and accepted repairs using separate denominators.

Verify repairs against the original failure. Require an owner, expiry and replacement coverage for a confirmed flaky test placed in quarantine.

Capture CI traces, fixture versions and repeatability evidence. An owner confirms whether the cause is product, test, data or environment. Proposed repairs require an independent retest and relevant regression. A confirmed flaky test may instead enter a time-limited quarantine with replacement coverage. Both paths keep a resolution record; the quarantined test remains open until restored.