RePerspective LabsFramework
Assuring AI Services from Release to Retirement
A practical method for deciding what evidence supports release, continued operation, material change and retirement.
Executive summary
An AI service is accepted for a defined scope, configuration and set of operating conditions. Its supporting evidence must remain connected to those conditions as the service changes. This Framework explains how to define material claims, assemble proportionate evidence, challenge its limitations and make accountable decisions across the operating life. It examines model behavior, control effectiveness and the complete human and system operation separately. One maintained service acceptance and review record connects release testing with production observations, material changes, incidents and retirement. The result is a reasoned basis for deciding what may operate, under which conditions, who accepts the uncertainty and what would require reconsideration.
What still justifies operating this service
A service can retain the same model and become less dependable because its reviewers lose capacity, its knowledge changes or a receiving team can no longer handle exceptions. Equally, a component can change without invalidating every finding from an earlier evaluation. The leadership task is to identify which conditions the acceptance decision depends on and whether the evidence still supports them.
I would treat service acceptance as a continuing commitment to a defined operation. The executive question is: What evidence justifies releasing this service, continuing to operate it, changing its scope or authority, and eventually retiring it?
The unit of analysis is the complete service: people, workflow, models, knowledge, tools, integrations, controls, support and recovery. A benchmark can inform a model claim. A control test can demonstrate a boundary. Overall acceptance also requires evidence that the work reaches its intended outcome under the conditions in which people will actually run it.
NIST's AI Risk Management Framework calls for evaluation in conditions resembling deployment and for monitoring during operation. The UK government's assurance guidance describes complementary techniques for substantiating claims throughout the lifecycle. [1][4] The method here is RePerspective Labs' synthesis for connecting that evidence to operating decisions. These sources do not validate the method or certify a service's safety.
Connect claims to evidence and decisions
Begin with the existing service definition, workflow, authority record, configuration baseline, source inventory, acceptance criteria, operating commitments and available test or production evidence. Identify the current decision: first release, continued operation, a proposed change or closure.
A claim states something the service must demonstrate within specified conditions. For each material claim, connect its acceptance criterion to supporting and contrary evidence, explain the evidence's limitations, and record why it is sufficient for the proposed commitment. Identify the changes or observations that would require reconsideration.
These connections determine the next action. A staffing change may affect the claim that required reviews are completed competently and on time. It does not automatically invalidate a technical demonstration that a forbidden payment call is blocked. Trace the affected assumptions before deciding what to retest and who must review the result.
↓ Claims determine the evidence required.
↓ Evidence supports an accountable decision.
The service acceptance and review record maintains these relationships and their decision history. Authorized operation follows acceptance; withholding release creates no operating permission. A Stop decision requires evidence of orderly closure.
Apply the method across the operating life
Define the service and its material claims
Specify the intended outcome, eligible work, users, affected people, operating conditions, dependencies and exclusions. Record the exact configuration or controlled baseline, including the knowledge and human arrangements on which it relies. State how an authorized person determines that the outcome is complete and acceptable.
Apply the existing authority arrangements to particular decisions: Recommend, Act, Escalate, Never Delegate. A service may use all four. Identify the receiving role, capacity and waiting state for an escalation. Changing the model's capability does not itself change its authority.
Separate three kinds of claim. Model claims concern task behavior, source use, omissions and uncertainty. Control claims concern enforced permissions, limits, intervention and traceability. Service claims concern accepted outcomes, human capacity, handoffs, continuity and recovery. A strong finding in one area cannot substitute for missing evidence in another.
The Outcome owner sets business acceptance criteria with relevant specialists. Identify critical conditions whose failure prevents the proposed use, as well as tolerable limitations and permitted responses. Where evidence cannot establish a material condition, narrow the proposed scope, obtain better evidence or withhold acceptance.
Design evidence proportionate to the commitment
For each claim, specify what evidence could support or contradict it and who is qualified to judge the expected result. Agree criteria before inspecting results. Consider the consequences, exposure, reversibility, detectability and time available for intervention when choosing the depth of evaluation.
Build coverage across ordinary, unusual, incomplete, conflicting and adverse cases. Examine relevant populations, languages, products, workload patterns and dependency conditions. Document where cases came from, why they represent the proposed work and which groups or conditions remain untested. Challenge cases explore failure mechanisms; they do not establish how frequently those failures occur in the population.
Keep a portion of evidence independent of development and tuning. Control access to held-out cases and expected results, record versions and disclose any exposure. Once a case has informed a repair, retain it as a regression test and obtain fresh independent evidence for the affected claim. Repeating a rehearsed test does not restore its independence.
Use repeated runs when variable behavior matters, recording settings and variation. Show important segments and failure severity alongside aggregates. A high overall score cannot erase a consequential failure in a relevant subgroup or a mandatory control. Explain uncertainty and the limits of inference; there is no universal case count or release percentage.
For generative AI, NIST cautions against extrapolating capability from narrow evaluations. [2] External benchmarks and supplier reports therefore require an explicit argument about their relevance to this service.
Test the complete operation
Follow cases through intake, knowledge retrieval, output, human decisions, actions, receiving-team acceptance and recovery. Observe the actual result in the authoritative business system. A successful tool response alone cannot establish completion, correct recipients or the absence of a duplicate effect.
Test missing and contradictory sources, forbidden actions, repeated requests, unavailable reviewers, dependency outages and uncertain completion. Examine both whether a control intervenes and whether its intervention leaves the work in a manageable state. A blocked action with an unstaffed exception queue is an unresolved service condition.
Observe representative people doing the work with realistic volume, interruptions and support. Can they recognize errors, challenge recommendations, find the relevant evidence and intervene before consequences occur? Include correction, off-system work and the burden transferred to receiving teams. NIST's Generative AI Profile explicitly distinguishes human proficiency testing from tests of model capability. [2]
If the business case depends on reduced effort, measure the complete recurring workload, including review, exceptions, support and recovery. Separate active effort from elapsed time. Released capacity requires a plan for its use and does not by itself establish cash savings.
Make accountable acceptance explicit
Name the evidence preparer, challenger, designated service acceptance authority and any separate residual-risk acceptor. Record their mandates. The Outcome owner owns the business criteria and resource commitment; the Operating owner demonstrates that the proposed operation can be supported. Knowledge and Technology owners substantiate their dependencies. The Control lead coordinates control evidence within an explicit remit. The Sponsor resolves resource and organizational barriers without replacing these accountabilities.
Independent challenge should examine case selection, expected results, contrary findings, untested conditions and the reasoning behind acceptance. Its separation and expertise should match the consequences and applicable requirements. Record conflicts, findings and how they were resolved. NIST AI RMF includes assessment by experts outside front-line development and/or independent assessors. [1] A second review by the delivery team should not be presented as independent assurance.
The acceptance authority can accept a bounded release, require narrower conditions or withhold release. Document residual uncertainty and who may accept it. An unresolved limitation needs an owner, response and review trigger; listing it does not make it acceptable. Independent assurance reports findings, while authorized management makes and owns the operating decision. Specialist acceptance duties remain with the properly designated roles.
Maintain evidence during operation
Map each material claim to relevant observations and a responsible owner. Combine telemetry with sampled outputs and completed outcomes, complaints, incidents, overrides, interventions and receiving-team feedback. Sample apparently accepted cases as well as exceptions so that successful processing does not become the only definition of success.
Check observation coverage, missing data, delayed outcomes and whether people are working around the approved route. NIST's 2026 monitoring report identifies practical limitations in distributed logging and understanding human-system interactions. [3] A quiet dashboard can therefore leave a claim unresolved.
Set scheduled reviews according to consequence, change rate and evidence availability. Add event triggers for incidents, capacity loss, new failure patterns, supplier notices and changes to accepted assumptions. State who acts on each trigger and within what operational window.
When evidence contradicts a critical condition, the Operating owner initiates the authorized containment route and escalates to the acceptance authority. Review the affected scope without waiting for the calendar. Evidence has no universal expiry date: its continuing relevance depends on what it demonstrates and whether those conditions still hold.
Reassess changes and learn from incidents
Before a change, trace its possible effects on claims and dependencies. Cover models, prompts, knowledge, permissions, tools, integrations, suppliers, workflow, user populations and operating conditions. Include interacting changes and changes outside the organization's direct control.
Targeted retesting is appropriate when affected claims can be identified and the remaining assumptions still hold. Pair it with regression checks on connected outcomes and controls. A wider review is needed when authority, population or purpose changes, shared dependencies may be affected, or the impact cannot be bounded. Preserve the rationale for both renewed and retained evidence. A supplier's limited disclosure increases uncertainty; it does not justify assuming equivalence.
An incident requires immediate operational decisions about pause, isolation, repair, rollback and recovery. Verify that any fallback or prior configuration remains suitable for present conditions. Reconcile unfinished or uncertain transactions before resuming the affected route. Recovery restores a permitted operation; it does not authorize a wider mandate.
Formal review then selects Expand, Continue, Narrow, Redesign or Stop, identifying the precise scope, responsible authority, implementation actions and next review. Wider use and greater authority are separate commitments. Update claims, tests, monitoring and operating design with what was learned. Retain the failed evidence and earlier decision so the reason for change remains visible.
Retire the service and demonstrate closure
A Stop decision must account for the work and obligations the service leaves behind. Identify open cases, queued actions, unresolved effects and complaints. Transfer them to a named receiving acceptor and establish a funded replacement or fallback where work must continue.
Withdraw credentials, permissions, integrations and scheduled activity so the retired service cannot keep acting. Transfer ownership of retained knowledge, records and outstanding issues. Preserve or dispose of evidence and data according to applicable retention, privacy and contractual requirements, including legal holds where relevant.
Record closure evidence, unresolved obligations and their owners. Verify that the replacement or fallback is accepted for its own scope; the retired service's approval does not transfer automatically. NIST AI RMF includes decommissioning within post-deployment risk-management planning. [1]
Maintain one service acceptance and review record
Use the existing operating or governance record where possible. Link versioned evidence at its authoritative location instead of copying test suites, logs and incident registers into another pack. The Operating owner keeps the record current; designated authorities remain responsible for its decisions.
| Record element | Minimum information |
|---|---|
| Service baseline | Outcome, eligible work, users, conditions, dependencies, configuration and authority |
| Claims and criteria | Material claim, acceptance criterion, critical conditions and accountable owner |
| Evidence and limits | Linked results, provenance, dates, coverage, contrary findings and remaining uncertainty |
| Whole-service findings | Model behavior, control effectiveness, human capacity, handoffs and recovery |
| Challenge and acceptance | Preparers, independent challenge, findings disposition, service approver and residual-risk acceptor |
| Authorized operation | Scope, conditions, exclusions, open issues, actions and responsible owners |
| Reassessment | Scheduled review, event triggers, affected claims and required response |
| Decision history | Decision, rationale, authority, date, implementation and closure evidence |
For the first review, select one bounded service and populate the record from existing material. Mark unsupported claims visibly. Ask the challenger to identify what could reverse the proposed decision. Resolve that evidence gap or limit the commitment before acceptance. At subsequent reviews, examine what changed and retain a clear reference to the previous basis.
A complaint drafting service loses review capacity
Hypothetical application. All evidence and decisions below are assumed to illustrate the method; no operating results or validation are claimed.
The service drafts responses to routine complaints about one product in one language. It operates under Recommend. Authorized people approve and send responses. Conflicting policy evidence must Escalate to a staffed specialist queue; settlement decisions are Never Delegate. Other products and languages remain outside scope.
The defined outcome is a complete, policy-grounded response accepted by the reviewer, with required escalation handled and subsequent corrections captured during the agreed follow-up. Before testing, the Outcome owner requires preservation of material complaint facts, support for policy statements from current authorized sources, and human approval before any response is sent. The baseline includes the model, prompt, knowledge version, workflow permissions, tested workload, response commitments, staffed shifts and backup arrangements.
| Material claim | Assumed evidence before release | Acceptance condition |
|---|---|---|
| Drafts use applicable facts and policy | Held-out ordinary, incomplete and conflicting cases meet the specified criteria; known coverage gaps are documented | Supported product and language only; conflicting cases withheld for specialist review |
| The service cannot send or settle | Boundary demonstrations show attempted sends and settlement actions denied without business effects | Human approval and sending remain in the approved workflow |
| Review and escalation are workable | Representative reviewers demonstrate error detection, source checking, workload handling and backup coverage | Operation limited to tested workload and staffed conditions; sampled accepted responses and queue performance reviewed |
An assessor separate from delivery challenges coverage and reviewer exercises. The evidence leaves uncertainty about rare complaint types and future workload. Untested types remain excluded. For this scenario, enterprise governance has explicitly delegated service acceptance to the Outcome owner and permissible residual-risk acceptance to a separate risk owner. They record bounded acceptance, the uncertainty, responsible owners and scheduled and event-triggered reviews.
Later, a staffing change removes tested review coverage. Sampling of accepted responses finds material omissions that reviewers missed; queue evidence shows the changed workload is exceeding demonstrated capacity. Model evaluation scores remain stable. The service-capacity claim is no longer supported, and the assumed protection from effective human review has weakened.
The Operating owner pauses the affected route, identifies cases requiring correction and moves work to a prearranged, staffed fallback. The acceptance authority records Narrow, limiting assisted operation to the workload and staffed periods still supported by evidence. It does not widen authority to compensate for missing reviewers.
Before reconsidering wider operation, the Operating owner must demonstrate revised staffing, competent challenge, escalation coverage and recovery under realistic demand. The record links the incident and affected claims, preserves the earlier acceptance basis and explains why unchanged model scores could not justify Continue under the changed conditions. No successful retest is assumed.
A bounded ordering service needs different evidence
Contrasting hypothetical application. A replenishment service may Act by placing orders for approved items and suppliers within transaction and cumulative limits. Its claim set must address authorization, correct destination, duplicate prevention, actual downstream completion and recovery, alongside the quality of its recommendations.
Assume a test produces a supplier timeout after an order request. The tool reports failure, but the receiving system's state is unknown. Retrying could duplicate an order; abandoning the case could leave a real commitment unmanaged. Recommendation accuracy cannot resolve either uncertainty.
Withhold release of the Act route until evidence demonstrates reconciliation and a controlled response to uncertain completion. A separately evidenced Recommend route may remain a candidate for acceptance. The record structure stays the same, while the criteria and demonstrations change with the authority and consequences. This contrast checks the method's coherence; it supplies no empirical evidence of reliability.
Limits of the method
This method organizes judgment and accountability. Its usefulness depends on credible expected results, meaningful evidence, competent challenge and authorities able to act. Some harms are delayed, rare or difficult to observe; absence of observed failure cannot establish their absence.
Sector requirements, specialist evaluation and authorized risk tolerances remain necessary. The sources here provide voluntary guidance or research, without determining an organization's legal obligations. If a material uncertainty cannot be accepted by the proper authority, obtain better evidence, restrict the service or stop the proposed commitment.
Leadership Takeaway
Choose one AI-enabled service and ask its accountable owner to connect the authorized scope, material claims and current evidence. Then ask which change would make that evidence insufficient and who would act. Bring those answers into the maintained acceptance record. Match the next commitment to what the complete operation can demonstrate, with explicit conditions and uncertainty, and a workable way to reconsider it.
Where to go deeper
Governing AI-Enabled Operations establishes enterprise mandates and oversight. Bounded Autonomy defines the delegation boundaries whose operation this method examines.
Workflow, Agent, or Both? helps select the mechanism before service acceptance. Measurement supplies outcome, control and human-work measures. The Continuous Evolution Loop explains how findings lead to controlled improvement.
Sources and further reading
These primary sources support the cited premises. The service acceptance and review method is RePerspective Labs' synthesis. Source versions and applicability were checked on October 14, 2026.
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, January 2023. See MEASURE 1.3 and 2.3–2.5, and MANAGE 1.4 and 4.1. Voluntary, cross-sector guidance.
- NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1, July 2024. See MP-3.4-004–006, MS-2.5-001 and Appendix A.1.4. Generative-AI-specific guidance.
- NIST. Challenges to the Monitoring of Deployed AI Systems. NIST AI 800-4, March 2026. See sections 3.2.1–3.2.3. A research synthesis of monitoring challenges, not an acceptance standard.
- UK Department for Science, Innovation and Technology. Introduction to AI assurance. February 12, 2024. See sections 3–5 on claims, assurance techniques and lifecycle application. Guidance rather than a universal legal requirement.