From Pilot to Operating Rhythm: Deploying AI into Real Work

Prove the complete workflow

Start with the outcome the business needs and the work required to deliver it. For a service operation, that might mean resolving requests accurately, within service commitments, at a cost and workload the organization can sustain. Faster drafting contributes only if the complete path to resolution improves.

A six-month field experiment involving 7,137 knowledge workers across 66 firms found individual time savings from AI assistance, but no detectable changes in the quantity or composition of workers’ tasks from providing individual access. The study examined digital work patterns; it did not directly measure overall business performance. Dillon and colleagues, Shifting Work Patterns with Generative AI, version 4, November 2025.

My reading is that useful individual gains can coexist with an unchanged system of work. Leaders still have to decide how saved effort changes service capacity, handoffs, responsibilities, or the employee’s working day.

Choose a bounded workflow and establish its baseline before expansion. Follow cases from arrival through completion, including waiting, review, rework, and downstream correction. Compare similar case types and operating periods, and account for changes in staffing or demand. Otherwise, selecting easier work can make the deployment look better than it is.

Name the Outcome owner and agree what would justify wider use. Those conditions should include outcome quality and the ability to handle failures, alongside speed and cost.

Make the pilot’s hidden work visible

Hypothetical example: a service-request team.

An operations team trials an assistant that drafts responses to routine requests. During the trial, delivery specialists select suitable cases, find missing context, and repair weak drafts. Reviewers see polished outputs, while that preparation is absent from the reported handling time.

Under ordinary staffing, the same arrangement could move effort into verification and correction. Faster drafts might create a review queue. Staff might continue a separate manual record because the official workflow does not capture what they checked. A deployment can look productive at one step while becoming more demanding overall.

The proposed change is to place the draft, approved supporting information, review decision, and final response in the existing case workflow. Define which requests are eligible. Route ambiguous or unsupported requests to a named human queue, with enough capacity to handle them. Before live use, establish who may pause the capability, how pending work will be recovered, and how the service will continue through an approved fallback.

Staff should rehearse the situations where they must challenge, correct, or bypass a draft. Managers must make room for that work and explain how released time will be used. Rewarding speed while penalizing appropriate escalation would undermine the intended operating model.

The benefit remains a hypothesis: less total effort per resolved request, with acceptable quality and a manageable review burden. Test it through observed handling time, sampled response quality, repeat contacts, and exception queues. This example reports no achieved savings.

Include recurring support, knowledge maintenance, evaluation, and technology costs in the economics. Initial learning costs are legitimate investments, but they should be visible. Time released becomes capacity only when the organization can use it; cash savings require a separate, credible change in expenditure. Specialist support may remain essential. It needs an explicit role, capacity, and budget.

Give the operating rhythm a decision to make

An operating rhythm connects evidence from daily work to people authorized to act on it. It should fit existing management routines wherever possible.

For a busy service operation, the following is a starting design:

Measure adoption against work that is actually eligible for the capability. Frequent logins reveal little about successful completion. Examine reasons for bypasses and overrides: they may expose a design problem or show that staff exercised appropriate judgment. Sample accepted outputs too, because an error that nobody challenges will not appear in the exception count.

NIST’s July 2024 Generative AI Profile connects monitoring after deployment with user feedback, override, incident response, recovery, and change management. Its guidance reinforces the need for an operating response to what monitoring reveals. NIST AI 600-1, MANAGE 4.1.

Each review should produce an owner, an action, and a way to verify whether the action helped. Changes to knowledge, workflow, or the AI capability should be evaluated before wider release. Capturing feedback does not authorize the system to change its own rules.

Use 30–60–90 days as decision checkpoints

A 30–60–90 day rhythm can structure deployment attention. The periods below are illustrative management checkpoints. Required controls apply from the start, and a date passing does not establish readiness.

Days 1–30: Establish a dependable, limited service

Use a defined scope with representative users and inputs. Observe the complete workflow, resolve early friction, and check whether staff can handle exceptions using the intended support routes. Test consequential failure scenarios in a controlled setting, including loss of a required system or knowledge source. A quiet month provides little evidence about rare failures.

Days 31–60: Test the next operating condition

If the initial conditions are met, introduce a deliberate change in volume, team, or case mix. Changing one major dimension at a time makes emerging problems easier to diagnose. Watch whether review demand, quality, and service commitments still hold.

Treat wider use and greater AI authority as separate decisions. Adding users to an assisted workflow does not justify allowing the system to send responses or take consequential actions independently.

Days 61–90: Test the support model the business will sustain

Reduce temporary project intervention toward the agreed ongoing support model and observe what happens. Retain the specialist support the service is designed to require. Check whether operating teams can resolve routine issues and obtain help through normal channels.

If performance depends on continuing exceptional intervention, make that dependency explicit. The next decision may be to fund it, redesign the workflow, retain a narrower scope, or stop. Record the evidence needed for another decision and its review date.

Agree pause conditions before expansion and rehearse the response. Disabling AI does not reverse actions already taken; recovery must address pending work and any affected cases.

This is the work of Phase 4, Deploy into Operations, in the Transformation Operating Map: establishing a functioning service and generating evidence for the next decision. Formal ownership transfer and the longer-term improvement loop are developed in the related Insights below.