Our Banking Client / A illustrative banking scenario story / Edition v1.24.0

A payment retry.
A different way to deliver quality.

In this QE testing scenario, a customer sends CAD 100, sees a timeout and tries again. Across five squads, 75 offshore QA staff need to prove that the money moves once while much testing remains manual and vendor test environments are scarce. Follow the release and the work behind it.

Our Banking Client is an illustrative digital payments scenario. Staffing, systems, hours and targets are planning inputs, not verified client results or industry benchmarks.

The QA organization

Manual & domain testing: 45 people
Automation engineering: 15 people
Data & environments: 8 people
QA leads & coordination: 7 people

Manual QE dominates: 75 offshore QA staff across five squads, including 45 manual and domain testers. Test environments are limited, backend services are not virtualized, and testing waits for the vendor’s available test environments and booked slots. Teams also contend for shared test data and resets.

Some Java API, Selenium and Appium scripts and partial CI exist, but much execution, rerun coordination and release evidence remain manual. More QE engineers alone do not create additional test-environment capacity. The pilot must address environment access, dependency control and repeatable execution alongside AI assistance.

Five squads cover payments, web, mobile, onboarding and settlement. An eight-person QA pilot team comes from this existing population: one lead, four domain testers, two automation engineers and one data/environment engineer. Product, development and platform support contribute separately.

01 / QE testing scenario

Timeout and retry: one transfer expected

PAY-142 is an illustrative QE testing scenario, not a reported client incident. Simulate a timeout after the provider accepts a transfer, then retry. Inject a duplicate-posting defect to check that tests reject two debits and require one transfer ID, one balanced journal and correct balances.

1Customer
2Payment API
3Provider stub
4Journal
5QA assertions

QE test scenario · ready

A duplicate posting must fail the test

The injected test defect creates two journals. QA expects one. Start the story or step through it at your own pace.

Illustrative failure: release blocked

Two journals produce payer CAD 800.00 and recipient CAD 200.00. Expected: one journal, CAD 900.00 and CAD 100.00.

QE testing scenario with an injected defect, not an observed client failure. No live payment service, test runner or customer data is used. Each journal has two balancing entries. No fees, FX or unrelated activity. The arrows show sequence, not traffic volume.

One release / Nine connected steps

What AI contributes from requirement to release

Follow PAY-142 through approved intent, test design, developer unit tests, executable API/UI tests, fixtures, execution, investigation, repair and release evidence. Each step names the AI output and the person who checks it. This is a proposed walkthrough, not a claim that a bank has already deployed it.

Nine steps, one payment scenario. AI drafts or recommends; engineers review; CI executes; release owners decide.

Three responsibilities: AI drafts and recommends. Existing automation provisions environments and executes tests. People approve behavior, changes and release decisions. Unit testing is developer work alongside the QA workflow; it adds no hours to the existing eight-stage effort model.

01 / Understand the change

Before: Copy the story; chase ambiguous retry rules.
Input: Approved PAY-142 revision + payment contract
AI assistance and execution boundary: AI drafts questions about a timeout after acceptance. The product owner confirms that the same key and payload return the original transfer.
Sample output: AC-142: retry must preserve the transfer ID and create no second posting.
Accountable reviewer: Product owner + domain QA

02 / Design test cases

Before: Assemble spreadsheet cases and check gaps by hand.
Input: AC-142
AI assistance and execution boundary: AI proposes normal, timeout, duplicate callback, concurrent retry and conflicting-payload cases. Domain QA defines expected results independently.
Sample output: CASE-142-R: timeout after acceptance; retry same key; expect one transfer.
Accountable reviewer: Domain QA

03 / Draft unit tests

Before: Developers handwrite unit tests; QA finds missing behavior later.
Input: AC-142 + handler source
AI assistance and execution boundary: AI drafts a unit test for the idempotency handler. The developer reviews the fake repository and verifies repeated calls do not create a second transfer. This does not prove integration.
Sample output: JUnit: create twice with the same key; verify one repository insertion.
Accountable reviewer: Application developer

04 / Build API / UI tests

Before: Handwrite requests, selectors, assertions and helpers.
Input: CASE-142-R + reviewed test patterns
AI assistance and execution boundary: AI drafts API tests and selected UI checks. Reviewers reject a status-only assertion and require the original transfer ID, one journal and agreed balances.
Sample output: PR-142: API assertions for transfer ID, journal count and balances; UI pending/retry state.
Accountable reviewer: Automation engineer + domain QA

05 / Prepare data and stubs

Before: Find usable accounts; book vendor slots; wait for shared resets.
Input: Known balances + provider behavior
AI assistance and execution boundary: AI drafts seed scripts and timeout mappings. Engineers validate fixtures and the provider model; containers and CI create repeatable runs. Real-provider checks still need vendor slots.
Sample output: FIX-142: payer 100000, recipient 0 minor units; STUB-142: accept then timeout.
Accountable reviewer: Data / platform engineer

06 / Select and run tests

Before: Run manual regression packs, collect screenshots and queue reruns.
Input: PR-142 + FIX-142 + STUB-142
AI assistance and execution boundary: AI recommends affected tests or requests an authorized pipeline. Azure Pipelines and test runners execute approved tests. Required payment coverage cannot be silently omitted.
Sample output: RUN-142: immutable build, test, fixture and stub versions; raw assertion results.
Accountable reviewer: DevOps + application QA

07 / Investigate failures

Before: Compare logs across systems and reconstruct the failure.
Input: RUN-142 failing assertion + trace
AI assistance and execution boundary: AI groups related failures and proposes duplicate posting as a cause, linking the assertion and journal evidence. A person confirms the diagnosis; AI cannot rewrite the failing verdict.
Sample output: DEF-142: expected 1 journal; observed 2 in the injected-defect example.
Accountable reviewer: QA lead + application developer

08 / Repair and retest

Before: Coordinate fixes, reset shared data and wait for another test slot.
Input: Confirmed DEF-142 + reviewed fix
AI assistance and execution boundary: AI can propose a repair. The developer reviews the production change; test expectations stay fixed. CI performs a fresh run and challenges the tests with the duplicate-posting variant.
Sample output: RERUN-143: corrected build passes; deliberately faulty variant must still fail.
Accountable reviewer: Developer + QA reviewer

09 / Review release evidence

Before: Reconcile screenshots and spreadsheets; chase approvals.
Input: RUN-142 + RERUN-143 + integration evidence
AI assistance and execution boundary: AI drafts a release summary and suggests reviewed defects for regression. Owners inspect required coverage, real-provider checks and unresolved gaps before deciding.
Sample output: PACK-142: retain the failed run, repaired run, review and remaining release gates.
Accountable reviewer: Release owner + QA lead

Technology mechanisms: Copilot test drafting, service virtualization, and CI test-result publishing. These sources support the mechanisms, not the illustrative client outcomes.

Follow one artifact chain, including the failed run

Every reference below belongs to the same authored PAY-142 example. The duplicate-posting defect is deliberately injected to challenge the test. Original failures remain linked after a repair; actual integration evidence is still required.

An illustrative failure-to-fix trace. A deliberately faulty build must fail; a passing retest alone is not release approval.

Outcome checks: critical-scenario coverage, first-pass usable candidates, injected-fault detection, environment waiting, feedback time and evidence completeness. Compare manual to modernized QE first; then measure the additional effect of AI on equivalent packs, including review and rework.

The leadership adoption roadmap

Four layers of AI-assisted QE

Start with a person using Copilot, connect one repeatable workflow, delegate bounded tasks where justified, and scale proven services across squads. Select a layer to inspect its payment example and prerequisites.

Shared foundations · start on day oneContext · data · reliable tests · CI · QE modernization · owners

Layer 4 supports layers 1–3. Scale useful assistance without having to delegate more actions.

Explore the AI adoption roadmap →

02 / Build once, reuse across squads

The proposed shared QA platform

Read the top row left to right, then the lower row right to left. Blue is system context or execution, purple is AI assistance, amber is human review and green is evidence. Arrows show work handoffs.

This is the target to build and prove; backend service virtualization and isolated runs are not available in the scenario baseline. Context adapters bring approved stories, test examples and relevant defects into the AI workbench. Reviewers approve artifacts before existing pipelines execute them. The platform team owns the shared integration and fixtures; each application team owns its expected behavior.

Inspect the technology choices and integration work →

Architecture change

Before / after: the payment test system

Start with manual QE, limited shared environments and non-virtualized backend services. Vendor test-environment availability caps parallel testing, while shared data resets delay reruns. The pilot target adds repeatable execution, validated virtual services and retained evidence. Real-provider validation still needs vendor slots. This comparison describes a scenario baseline and proposed target, not verified client infrastructure.

Before / scenario baseline

Manual QE team → manual API/UI runs → few shared environments and test accounts. Backend services are not virtualized. Vendor test environments limit parallel runs; logs and screenshots are collected manually.

After / pilot target

AI draft + human review → pinned CI runner → isolated payment service and PostgreSQL. Versioned WireMock models reproduce faults; raw assertions and run IDs are retained.

Target: virtualized service tests reduce dependence on vendor slots. Real-provider and settlement checks remain separate gates.

Present the technical architecture comparison →

Quality, reliability and adoption

Outcomes beyond hours

Use three primary measures to judge whether the pilot earns trust. Inspect the definitions below for supporting proof, owners and limits.

Catch payment faults

Catch every critical seeded fault

Reviewed fault scenarios and independent balance assertions

Cover critical journeys

Exercise every critical scenario

AI drafts edge cases; domain QA approves the scenario matrix

Explain each release decision

Complete every decision record

CI captures evidence; AI drafts a source-linked summary

Supporting proofTrust repeat runsValidate real integrationsEnable the next squad

Baseline: Not recorded · Observed after: Not recorded · Targets proposed for pilot agreement.

Business outcomes to validate: fewer customer-impacting payment defects, fewer emergency fixes and more predictable releases. Track these across comparable releases; the pilot measures alone do not establish those effects.

Primary Catch payment faults

Proposed acceptance: Catch every critical seeded fault.

Measure: Correctly detected viable critical payment faults / all approved viable critical faults in the seeded catalogue. Unexecuted faults remain in the denominator; report separately.

Evidence: Versioned fault catalogue, clean-build control run, injected-build results, transfer and journal assertions; show caught / eligible counts.

Owner: Domain QA lead + payment service owner · Contribution: Shared contribution

Keep the measure honest: The clean control must pass and the injected fault must fail the intended assertion. Approve and log invalid or equivalent fault exclusions; infrastructure errors are not detections.

Primary Cover critical journeys

Proposed acceptance: Exercise every critical scenario.

Measure: Approved critical scenarios with reviewed assertions and a valid execution result / all approved critical scenarios in the frozen pilot matrix. Skipped or blocked scenarios are incomplete.

Evidence: Requirement-to-scenario matrix and execution IDs for retry, concurrency, changed payload, delayed callback and reconciliation cases; publish covered / approved counts.

Owner: Product owner + domain QA lead · Contribution: Shared contribution

Keep the measure honest: Coverage means exercised behavior, not a green result. Report failures separately; do not remove difficult scenarios or accept AI-generated assertions without domain review.

Primary Explain each release decision

Proposed acceptance: Complete every decision record.

Measure: Pilot release-candidate decision packs containing every required, resolvable evidence field / all pilot release candidates submitted for a decision. Missing packs remain in the denominator.

Evidence: Schema-validated packs linking requirements, source commit, environment and fixture versions, raw results, exceptions, reviewer and decision; show complete / submitted counts.

Owner: Release owner + QE lead · Contribution: Shared contribution

Keep the measure honest: A complete pack may recommend rejection. AI summaries must agree with retained raw results; completeness never overrides a failed mandatory gate or human release authority.

Supporting Trust repeat runs

Proposed acceptance: No unexplained critical flakes.

Measure: Test cases with mixed pass/fail results / all cases completing the agreed unchanged-configuration repeat protocol. Publish incomplete cases / all selected cases separately.

Evidence: Pre-agreed repeat count, first-attempt and retry outcomes, build/data/stub versions, error classification, and mixed-result / completed plus incomplete / selected counts.

Owner: Automation lead + platform engineer · Contribution: QE foundation

Keep the measure honest: A retry pass does not erase the first failure. Investigate critical flakes; expose blocked or incomplete runs and keep mandatory coverage unchanged.

Supporting Validate real integrations

Proposed acceptance: Verify agreed provider behaviors.

Measure: Approved provider behaviors verified against the real provider sandbox / all provider behaviors assigned to the agreed integration check. Report stub outcomes separately.

Evidence: Contract version, provider-sandbox run IDs and captured behavior, stub-to-provider mismatch log, unresolved exclusions, and verified / agreed behavior counts.

Owner: Integration lead + provider relationship owner · Contribution: QE foundation

Keep the measure honest: Passing stubs proves only the simulated scenario. Sandbox checks do not establish production resilience; document inaccessible behaviors and retain separate end-to-end acceptance.

Supporting Enable the next squad

Proposed acceptance: Second squad runs independently.

Measure: Journeys independently configured, run and evidenced by the second squad / all journeys assigned to its agreed reuse trial. Record pilot-author interventions separately.

Evidence: Second-squad commits and CI runs, template versions, completed handoff checklist, support log and independently completed / assigned journey counts.

Owner: QE platform owner + second-squad lead · Contribution: Shared contribution

Keep the measure honest: Agree what independent means before the trial. Hidden pilot-author fixes or omitted mandatory checks cannot count as successful reuse; assign ongoing support ownership.

Separate modernization from the AI increment

Existing workflowObserve current tests and evidence.
Modernized QERepeat the same scope with reliable foundations.
AI on that foundationHold the workflow constant; add reviewed assistance.

Measure the existing workflow, the improved QE foundation, and AI assistance on that foundation. Hold scenario scope and acceptance criteria constant, record configuration changes, and retain human review effort. Seeded-fault detection is a pilot test of test effectiveness; it does not establish a reduction in production defect escapes.

Baseline and observed after results are not recorded. Agree the scope, denominators and targets before the pilot. These are proposed acceptance criteria for Our Banking Client, not measured client improvements or industry benchmarks. Report QE modernization and AI contributions separately.

Measurement basis: PIT: Basic concepts · Playwright: Retries. Definitions and acceptance criteria above are authored for the pilot.

Present the executive outcomes → · Inspect the technical measures →

03 / Break down the work

AI assistance at every workflow stage

Before / after: manual, modernized and AI-assisted QE

Establish the baselineActive work · vendor waits · coverage gaps

Prove repeatabilityEnvironment readiness · reruns · raw evidence

Isolate added AI valueCandidate acceptance · review · rework

Compare the same payment scope. Modernization removes execution bottlenecks; assess incremental AI value against that repeatable baseline.

Measure modernization and the AI increment separately

Record three observations of the same eight-stage QA scope. All inputs start unknown. Include every attempt, failure, review and correction. Developer unit work is outside this denominator. The recording explains the comparison method; it does not read your entries.

Modernization: Not yet estimable

Incremental AI: Not yet estimable

Existing workflow · Not recorded

Scope / release-pack IDAcceptance criteria versionApplication buildEnvironment versionFixture versionProvider or stub versionTest or manual procedure versionAccountable reviewerEvidence reference Work hours (exclude review and rework)Review hoursCorrection / rework hoursOperating hoursOne-time setup hoursCompleted comparable packsTotal attemptsFailed attempts Hours include failures and incomplete work; attempts and packs remain separate.

Enter the observation and its evidence to compare.

Modernized · AI off · Not recorded

Enter the observation and its evidence to compare.

Same foundation · AI on · Not recorded

Enter the observation and its evidence to compare.

Recurring hours per completed comparable pack include work, review, correction and operation. Setup is retained separately and is not silently amortized. Every comparison matches scope, acceptance criteria and application build. The AI comparison also requires the same application build, environment, fixtures and provider version as the modernized control. Entered data is unverified; these are effort differences, not cash savings or proof of better quality.

Choose a stage to inspect the manual work, AI assistance and a concrete sample artifact. Samples are authored illustrations, not a live payment integration.

1. Requirements

“What happens when the provider accepts the payment but the customer sees a timeout?”

How the work changes: inputs, assistance, owners and tools

Input: PAY-142 story, API contract and past payment defects
Current manual work: Clarify what a retry means and chase answers across time zones.
AI assistance: Draft ambiguity questions and identify affected payment journeys.
Output: Versioned acceptance criteria and an open-question log
Owner and human check: Product owner + domain QA. Product owner confirms that a retry uses the same key and returns the original transfer.
Example stack: Azure Boards + a proposed context connector

Authored sample · PAY-142 / rule-r3

Requirement: retry an accepted payment after a timeout.
Key: same request key + same amount → same transfer ID.
Expected: one journal; payer 90000, recipient 10000 minor units.
Open question: key-retention period requires product agreement.

Product owner + domain QA checks: Expected financial behavior is agreed independently of the generated test. Open questions remain visible.

Handoff: Approved rule-r3 → domain QA; unresolved rules return to the product owner.

Inspect the prerequisites for requirements →

2. Test design

“Two concurrent requests with one key must create one transfer and one balanced journal.”

How the work changes: inputs, assistance, owners and tools

Input: Approved behavior, prior scenarios and defect history
Current manual work: Copy cases between spreadsheets, fill gaps and review duplicates.
AI assistance: Draft positive, boundary, retry and concurrency cases with requirement links.
Output: Reviewed test matrix with independently defined expected results
Owner and human check: Domain QA lead. Include same-key/different-payload, parallel retry and duplicate callback cases.
Example stack: Azure Test Plans; Gherkin if already used

Authored sample · PAY-142 / scenarios-r2

C1  Timeout → same-key retry    → one transfer
C2  Concurrent same-key calls   → one transfer
C3  Same key, different amount  → reject conflict
C4  Repeated provider callback → one posting

Domain QA lead checks: The matrix includes independent outcomes, concurrency and changed-payload behavior.

Handoff: Reviewed scenarios-r2 → automation and fixture owners.

Inspect the prerequisites for test design →

3. Test data

“Payer starts at CAD 1,000.00; recipient at CAD 0.00. Transfer CAD 100.00.”

How the work changes: inputs, assistance, owners and tools

Input: Account schema, balance rules and fixture conventions
Current manual work: Find eligible accounts, prepare balances and coordinate shared data resets between squads.
AI assistance: Draft seed scripts and scenario combinations using synthetic records.
Output: Versioned fixtures with isolated accounts and known starting balances
Owner and human check: Test-data engineer. Verify relationships and exact balances before running tests.
Example stack: Proposed PostgreSQL fixtures + Testcontainers for service tests

Authored sample · PAY-142 / fixtures-r4

run_id: sample-142
currency: CAD
payer_start: 100000 minor units
recipient_start: 0 minor units
amount: 10000 minor units
reset: isolated accounts recreated per run

Test-data engineer checks: Balances and account relationships are valid; separate runs do not share mutable fixtures.

Handoff: fixtures-r4 + account IDs → the environment preflight.

Inspect the prerequisites for test data →

4. Environment

“The stub accepts a transfer, delays its response and later returns a callback.”

How the work changes: inputs, assistance, owners and tools

Input: Build ID, deployment manifest and dependency contracts
Current manual work: Check deployments, book scarce vendor test-environment slots and wait for shared resets. Backend services are not virtualized.
AI assistance: Summarize readiness failures and draft stub mappings for engineers to implement and validate.
Output: Reproducible test run with pinned build, data and stub versions
Owner and human check: Platform engineer + QA. A deterministic preflight checks health, schema and fixture readiness.
Example stack: Existing Azure Pipelines + proposed WireMock and isolated test environments

Authored sample · PAY-142 / environment-r2

app_build: payment-example-a1
fixture_rev: fixtures-r4
provider_model: accepted-then-timeout-r2
readiness: app + database health checks required
retention: evidence stored outside disposable resources
cleanup_owner: platform on-call

Platform engineer + QA checks: The provider substitute reproduces the intended failure. The real provider and settlement still need separate checks.

Handoff: Pinned manifest + usable environment → CI runner.

Inspect the prerequisites for environment →

5. Automation

“Assert the transfer ID, journal count and balances, not only HTTP 200 or a success toast.”

How the work changes: inputs, assistance, owners and tools

Input: Reviewed scenarios, seed tests and repository conventions
Current manual work: Write selectors, API requests, assertions and reusable helpers.
AI assistance: Draft tests in the existing framework and propose maintenance patches.
Output: Reviewed pull request linked to the accepted test matrix
Owner and human check: Automation engineer. A deliberately broken duplicate-posting variant must fail. No weakened assertions or silent skips.
Example stack: REST Assured + JUnit; Playwright for the new web journey

Authored sample · PAY-142 / candidate-r1 → review-r2

AI candidate r1: assert response.status == 200
Reviewer: insufficient; this can miss a duplicate posting.
Revision r2: assert same transfer ID on retry
             assert journal count == 1
             assert balances == 90000 / 10000
Challenge: deliberate duplicate-posting variant MUST FAIL

Automation engineer checks: A candidate cannot weaken required assertions or skip the deliberate-defect check. This is sample pseudocode.

Handoff: Reviewed commit → CI; rejected candidates return to the author.

Inspect the prerequisites for automation →

6. Execution

“Execute retry tests at the API layer, then check the customer-facing status in the browser.”

How the work changes: inputs, assistance, owners and tools

Input: Approved tests, pinned build and ready environment
Current manual work: Run regression packs by hand alongside partial scripts, collect screenshots and queue reruns for the next available environment slot.
AI assistance: Recommend affected tests and summarize run evidence.
Output: Runner results, traces and mandatory-suite completion record
Owner and human check: Application QA team. Keep the mandatory payment suite while validating selection quality.
Example stack: Azure Pipelines; existing Selenium and Appium suites retained

Authored sample · PAY-142 / sample-run-142

Candidate: review-r2
Correct behavior fixture: required assertions PASS
Deliberate duplicate variant: journal count 2 → FAIL (expected)
Candidate disposition: challenge detects the known defect
Required suite: other cases must still complete
Evidence: manifest + assertions + logs

Application QA team checks: These are illustrative receipts, not executed results. Detecting the injected defect does not establish complete coverage.

Handoff: Original receipt + artifact links → QA triage and reviewer.

Inspect the prerequisites for execution →

7. Triage & retest

“A correct UI toast with two ledger journals is a product defect; changing the assertion is unacceptable.”

How the work changes: inputs, assistance, owners and tools

Input: Failed assertions, logs, traces, changes and environment state
Current manual work: Compare failures, recreate shared data and coordinate fixes and retests around vendor slots and time zones.
AI assistance: Cluster related failures and draft evidence-linked defect or repair proposals.
Output: Reviewed defect disposition, fix and fresh rerun evidence
Owner and human check: QA lead + developer. Separate product defects, flaky tests and environment faults. Recheck original intent after a patch.
Example stack: Playwright traces + pipeline artifacts + Azure Boards

Authored sample · PAY-142 / sample-defect-17

AI draft: duplicate journal observed after retry.
Evidence: sample-run-142 / assertion journal_count=2
Uncertainty: root cause not established by this assertion.
Human disposition: developer investigates transaction handling.
Retest: new build + fresh run; old failure stays retained.

QA lead + developer checks: The diagnosis is traceable and uncertainty is preserved. A suggested fix does not close the defect.

Handoff: Named developer + reproducible failure → fix, fresh retest and domain review.

Inspect the prerequisites for triage & retest →

8. Release evidence

“PAY-142 → reviewed test → build → fixture → run → defect disposition → sign-off.”

How the work changes: inputs, assistance, owners and tools

Input: Results, open defects, waivers and requirement coverage
Current manual work: Reconcile screenshots, logs and spreadsheets by hand, then chase owners for release status.
AI assistance: Draft the release summary with links and unresolved evidence gaps.
Output: Versioned evidence pack and accountable release decision
Owner and human check: Release owner + QA lead. Missing evidence remains unknown. A failed money-movement assertion blocks this case's release.
Example stack: Azure Test Plans reports + pipeline artifact store

Authored sample · PAY-142 / sample-release-review

Requirement: rule-r3
Scenarios: scenarios-r2 / all required cases accounted for
Test + fixture + build: linked in run manifest
Defects: dispositions + fresh retest links required
Provider / settlement: separate evidence required
Decision: HOLD while required evidence is missing

Release owner + QA lead checks: The release owner resolves required checks and exceptions. An AI summary cannot override a missing result.

Handoff: Reviewed evidence + explicit decision → accountable release owner.

Inspect the prerequisites for release evidence →

Before the pilot

The platform is an assumption to prove

Adoption decisionRequired evidence accepted · owner accountable · foundation work funded

Business & peopleExpected outcomes · reviewer capacity · offshore handoffs · measured baseline

Runnable engineeringInfrastructure · CI/CD · test data · dependency control · testable applications

Supported servicesFrameworks & devices · AI access · run evidence · platform ownership

Authored adoption assumptions. Validate each application and workflow; unknown requirements remain open. Assumptions and evidence →

The target requires a repeatable test environment, usable pipelines, controlled provider behavior, isolated data, approved AI access and reviewers with time to act. The manual baseline has limited environments and no backend service virtualization; these target capabilities must be established, not treated as already available. Each missing dependency needs a named owner, an evidence check and funded work before the affected AI workflow can proceed.

Inspect Our Banking Client's twelve adoption assumptions →

04 / Test the proposal against reality

Maturity changes scope and effort

The profiles below illustrate effort; they do not approve adoption. Validate the workflow prerequisites separately. Select an illustrative delivery profile. The model updates all eight stages and shows whether usable capacity covers setup effort.

RequirementsVersioned stories; some ambiguity

DevOpsCI available; deployment partly manual

Cloud & environmentsLimited Azure/vendor slots; shared reset

Application testabilityPayment API testable; no backend virtualization

AutomationJava automation usable; UI pack noisy

Offshore deliveryDomain expertise strong; automation skills uneven

Pilot payment APIs and one web journey. Add synthetic fixtures, provider stubs and evidence links alongside AI assistance.

Next evidence gate: Two comparable assisted release packs, stable mandatory tests and measured review effort.

Baseline

300h

Assisted target

182h work40h

Purple = review. Hours for both approaches share the same scale. Platform operation and setup appear in the capacity calculation below.

Modelled person-hours per bounded release test pack
Stage	Baseline	Assisted work	Review	Assisted total
Requirements	20	12	4	16
Test design	40	20	8	28
Test data	30	20	4	24
Environment	25	20	2	22
Automation	55	30	10	40
Execution	80	52	4	56
Triage & retest	35	20	6	26
Release evidence	15	8	2	10
Total hours	300	182	40	222

33 usable hours per pack

78h gross capacity − 12h platform operation = 66h net capacity. Redeploying 50% yields 33h. Recovering 480h of setup requires 15 comparable packs, rounded up.

Adoption prerequisites are unverified. All hours are conditional assumptions. Assisted work and review are separate, non-overlapping buckets. The baseline includes its ordinary review and rework. Setup includes integration, coaching and supporting staff time. Additional tool, cloud and vendor charges are excluded. This is capacity arithmetic, not financial ROI or an isolated AI productivity estimate. The release pack uses a 10-business-day planning window; no shorter release cycle is claimed.

Read the measurement method and limitations →

05 / Turn the discussion into deliverable work

Observed time framework for the API pilot

A framework for collecting actual API pilot timings. The windows below are illustrative planning inputs, not observed client durations.

Observation status: not yet recorded

Planning window · weeks 1–2
Observe the baseline

Record API test preparation, active effort, waits and rework for one comparable pack.
Evidence gate: Reviewed API scope + usable baseline
Planning window · weeks 3–8
Run the API pilot

Record setup separately; measure review, execution, triage and retest over repeated packs.
Evidence gate: Required assertions pass + comparable run evidence
Planning window · weeks 9–12
Confirm repeatability

Measure second-team setup, support effort and repeat runs before choosing the next scope.
Evidence gate: Reusable pattern + evidence-based next step

Record actual dates and elapsed time; track active person-hours, waiting reasons, review and rework separately.

Readiness controls the pace: environment, data, virtualization, CI and reviewer gaps can extend a phase. Advance on evidence, not the calendar.

Use the pilot time observation record →

06 / Present the story

Two audience briefings

The Executive deck focuses on strategic vision and shared investment. The technical deck walks through API contracts, timeout and retry sequencing, code-level assertions, CI configuration and evidence schemas.

ExecutiveOur Banking Client: strategic vision21 slides · 25–30 minutes Technical SDs & leadsOur Banking Client: engineering blueprint33 slides · 35–45 minutes

Open the deck for a full presentation view.

Open Our Banking Client: strategic vision ↗

Brainstorm questions

“Where do those 75 QA staff spend time, and where are they waiting?”
“How many vendor test environments and concurrent slots do we actually have, and which tests wait for them?”
“Which AI layer fits our next workflow, and which capabilities should the platform team make reusable?”
“Which delivery commitment would use any capacity the pilot returns?”

Follow up by walking one recent release with the QA lead, an application developer and a platform owner. Bring the failing test, its data, the environment booking history and the reset and rerun handoffs.

Sources and the limits of this case

Primary documentation reviewed 7 September 2026. Sources support the technology mechanisms and the importance of delivery foundations. They do not validate Our Banking Client's scenario staffing, hours, payment rules or outcomes.

S1 · DORA 2025: State of AI-assisted Software Development
AI adoption depends on the surrounding delivery system. This informs the readiness profiles; it supplies no savings rate for this case.
S2 · Microsoft: Azure Test Plans overview
Manual and automated testing, traceability and reporting. The case's AI context connectors are proposed integration work.
S3 · Playwright: Test Agents
Planner, generator and healer examples inform the browser workflow. Review and rejection of skipped or weakened tests are proposed pilot requirements.
S4 · Playwright: Trace Viewer
Inspect recorded actions, snapshots and network evidence when diagnosing browser failures.
S5 · WireMock: Service virtualization
Simulate payment-provider behavior for controlled dependency tests. Stub fidelity still needs integration validation.
S6 · REST Assured
An example Java API testing framework that fits the team’s existing Java skills in this scenario.
S7 · Testcontainers for Java
Disposable container dependencies for service tests; requires a compatible container runtime. It does not provision the entire fintech estate.
S8 · Appium documentation
An example automation stack for existing mobile tests. Device availability and app observability still affect delivery.
AL1 · GitHub: Writing tests with Copilot
Supports examples of generating and reviewing unit and integration tests. It does not establish client productivity or test completeness.
AL2 · Microsoft: Publish Test Results v2
Supports publishing JUnit and other result formats and attachments. The AI diagnosis adapter and workflow described here are proposed additions.
AL3 · GitHub: About Copilot cloud agent
Supports the GitHub-hosted agent example and its customization. It does not establish a ready-made integration with the client's Azure stack.
AL4 · GitHub: Copilot Agents application card
Describes agent capabilities and limitations, including the need to review and test generated changes. Our adoption gates are an authored QE proposal.
QO1 · PIT: Basic concepts
Supports mutation testing as a check that tests detect introduced faults and distinguishes detected, surviving, equivalent and invalid mutations. The payment fault catalogue and pilot acceptance criteria are authored proposals, not PIT benchmark results.
QO2 · Playwright: Retries
Defines passed, flaky and failed retry outcomes and supports preserving first-attempt results. The unchanged-configuration repeat protocol and target are authored pilot measures, not a Playwright industry benchmark.

Broader industry research · Dictionary · Implementation notes

Three clocks, three scopes

Engagement phase	Planning duration	What it covers
Sponsor alignment	2 weeks	Scope, outcome charter, funding and authority
Baseline and readiness	4–6 weeks	Includes measurement and foundation discovery
Capped pilot	8–10 weeks	Includes setup and repeated execution; agree the exact window before starting
Limited validation	1–2 quarters	Repeat the result across releases and teams before scale

Measurement protocol: 3 baseline weeks and 8 pilot weeks including setup, within the phases above. One extension may last up to 4 weeks. Freeze the actual window before assignment.

API teaching example: weeks 1–2: observe the baseline; weeks 3–8: run the api pilot; weeks 9–12: confirm repeatability. These windows are a separate planning example, not the approved measurement protocol or an observed client duration.

Readiness can extend the plan. A narrow assisted workflow can begin sooner when its own prerequisites are met; an unresolved environment or provider dependency blocks broader execution.

The QA organization

Timeout and retry: one transfer expected

A duplicate posting must fail the test

What AI contributes from requirement to release

Follow one artifact chain, including the failed run

Four layers of AI-assisted QE

The proposed shared QA platform

Before / after: the payment test system

Before / scenario baseline

After / pilot target

Outcomes beyond hours

Catch payment faults

Cover critical journeys

Explain each release decision

Separate modernization from the AI increment

AI assistance at every workflow stage

Before / after: manual, modernized and AI-assisted QE

Measure modernization and the AI increment separately

1. Requirements

2. Test design

3. Test data

4. Environment

5. Automation

6. Execution

7. Triage & retest

8. Release evidence

The platform is an assumption to prove

Maturity changes scope and effort

Observed time framework for the API pilot

Observe the baseline

Run the API pilot

Confirm repeatability

Two audience briefings

Brainstorm questions

Sources and the limits of this case

Three clocks, three scopes