The process transfers to other products. A human defines what the software should do. A lead agent organises the work, developer agents implement individual tasks, and a test runner checks the result against the running product. The merge gate integrates the verified changes. The programming language, development tool and model are project settings.

The result is a usable feature

A task describes an outcome from the perspective of someone using the product. It includes the required variants and the conditions under which the outcome must hold. The task is complete when the software meets these requirements and its scenarios pass against the integrated version.

The evidence covers the path from input to observable effect. In my command-line agent, live scenarios check responses and the effects of tool calls. In the data pipeline, database scenarios compare the source, raw data and history. In the strategy game, test drivers control the simulation and check a headless player build on macOS and Windows. The checks adapt to the product while the development process stays the same.

The method connects implementation with independent acceptance. The agent writing product code cannot change the approved expected results. Its report helps with diagnosis. The runner sets the task’s status from the executed test and records the checked commit and run log alongside it, so the result can be traced back to the same code later.

Define the scope of a round

Work starts with a finite list of user tasks. In my template it is called feature_list.json. Each entry contains the desired outcome, required variants, affected interfaces and associated scenarios. A passes field records whether the task has passed acceptance.

The list stays fixed for one round. Additional requests go into a separate list so that ongoing work has a defined endpoint and new requirements do not displace tasks already agreed upon. The next round starts when the current tasks are complete or the owner has explicitly revised the scope.

Before the first developer session, acceptance checks run against the existing product. This harvest run identifies functionality already present and includes it in regression testing. The agents start with the remaining tasks. When adopting an existing project, this preserves usable code and defines the remaining work as the difference between requirements and verified behaviour.

Make acceptance an executable oracle

An oracle compares observed behaviour with a defined expected result. It consists of scenarios with an initial state, inputs and checkpoints. A test driver runs them against the product and returns a machine-readable result. An existing test filter is enough for some tasks. Others need a driver for the user interface, database or simulation.

Scenarios are prepared before work on the product code begins. An agent drafts them from user tasks and acceptance criteria. The owner checks that they describe the desired functionality and approves them. This ratification determines what the developers will work towards and which result the gate may accept.

A good scenario also checks relevant failure cases. For a data export, for example, it could require the intended user to receive the correct data while an unauthorised request fails. This example combines usable functionality with its access requirements, which a check concerned only with successful execution could miss.

The known scenarios are supplemented by hidden variants with different inputs and the same checkpoints. These twins remain outside the developers’ working directories. If a twin fails while the known scenario passes, the owner investigates whether the implementation is tailored to the known inputs or the scenario itself needs correcting.

An incorrect expected result remains incorrect even when a test passes. Reviewing the scenarios against the product requirements is therefore part of the method. An owner who delegates this review to another agent defines its authority and checks samples personally. That is how I work in the game project: I define the game rules, and another agent reviews scenario drafts. Developers gain no authority to change their own acceptance checks.

Work through the loop

The loop repeats the same sequence for each approved task:

1. A worker claims an open task. A lock prevents another worker from working on the same task at the same time.
2. The worker starts in its own worktree from the current main branch. The session receives the task, its scenarios and a short progress note as its initial context.
3. The developer works on product code within a defined time and cost budget. The loop controller ends the session when the limit is reached.
4. The resulting branch enters the merge queue. The gate checks it against the current state of main.
5. If the checks pass, the gate merges the change and marks the task complete. Otherwise, the task stays open and is parked for a decision, with a failure reason and run log.

Each new session starts with fresh context. The repository holds the code, while the progress note records the findings needed to continue. A new attempt therefore does not depend on a long conversation whose assumptions and intermediate states the next agent would have to reconstruct.

Several workers can handle independent tasks in parallel. Integration into main is serial, so every change is checked against a version that already includes the changes merged before it. A worker’s tool and model can change without altering the expected results or acceptance rules.

The gate protects the shared product

The merge gate takes a branch from the queue and rebases it onto main. It takes the task list, scenarios and loop tooling from the trusted main branch. Any developer changes to these files are reset before testing. Tool-level write protection and a comparison of protected paths after each session provide additional checks.

The build, task scenarios and tests for affected modules then run. A small smoke set checks core functionality. Invariant tests check properties that apply across the product, and the diff is scanned for credentials and prohibited test changes. The gate performs a fast-forward merge only when the required checks pass.

Developers are prohibited from deleting, weakening or skipping existing tests. Among other checks, my implementation detects a decrease in the number of test declarations or assertions and uses the invariant tests from main. An altered assertion can still check less without changing that count. The protected acceptance scenarios and hidden variants therefore remain necessary alongside these technical controls.

Regular regression runs recheck accepted tasks. If a feature loses its expected behaviour, the runner resets its status and schedules it for repair. The hidden variants run as part of these checks. In the game project, the gate also checks previously passing scenarios that use the same test driver before merging another change.

Responsibilities and decisions

The owner is responsible for the product goal and acceptance against its requirements. They decide on scope changes, parked tasks and external actions such as pushing or publishing. Time and cost limits are set before a run. The session length depends on the project and how long its checks take.

The lead agent prepares the next work. It drafts scenarios, starts workers and keeps the documentation aligned with the product. Developers work on their branches. The gate checks and integrates their results. This division of responsibilities requires no particular model family, provided the tools support the necessary access and protection rules are enforced outside the developer session.

For a parked task, the owner decides whether to attempt it again, split it, correct the scenario’s requirements or move it to a later round. A further attempt therefore has a reasoned assignment. The original user outcome remains open until it is fulfilled or explicitly changed.

I describe the background to this separation in my account of why my agentic SDLC stalled. Repairs and written reviews had become detached from usable functionality. In the current process, each worker follows a task whose completion the runner can establish against the product.

Agents and models in use

These are the role assignments in my projects as of 11 October 2026. As the owner, I define the product goal and the agents’ authority. The working instructions call the lead agent “Root”.

Root drafts scenarios and starts developer sessions, with their command-line tool and model configured in the loop controller. In the strategy game, Root also performs the delegated review of scenario drafts written by another agent. The merge gate and test runner perform acceptance checks as programs without a language model.

The command-line agent also has a model inside the product under test: its live scenarios initially ran against gpt-5.6-sol, later against GLM 5.3 through OpenRouter. That model produces the product responses checked by the oracle. It does not decide whether the developers’ work passes acceptance.

Adopt the method in a project

Adoption requires acceptance scenarios and a connection to the project’s build and test environment. The loop itself is reusable. My template exposes three configurable commands: build, module tests and oracle. The oracle accepts --ids and --out, exits with code 0 only on success and writes a report.json with exactly one entry per requested task ID.

Start with a small, complete user task. Its scenario follows the intended path through the product. Once the runner can check that path, further tasks can be added in the same way. If an interface lacks a test driver, building one is part of setting up acceptance.

For an existing repository, first preserve the usable product state and any unfinished branches. The branches go through the gate, then the harvest run checks the approved scenarios. The loop starts with tasks that do not yet pass. New requests go into the next round.

For products with variable model responses or demanding environments, checks can be divided into two levels. A deterministic run against the local product determines passes. An additional reference run examines behaviour under realistic conditions, such as actual data volumes. Its results remain separately visible because local acceptance establishes only the properties checked there.

The working instructions describe responsibilities and point to executable controls. In my template, a script limits their length so that individual cases do not keep adding new rules. Run logs connect worker, gate and oracle through a shared trace ID. An optional adapter then copies the confirmed status into an existing ticket system and reads it back for verification.

The task list and work in fresh sessions draw on Justin Young’s Effective harnesses for long-running agents. Locks for parallel workers come from Nicholas Carlini’s work on a C compiler with Claude. My method combines these components with protected oracles, approval of the expected behaviour and the merge gate as the sole route to integrated, accepted functionality.

I examined the template alongside the working instructions and run records from both projects. In one project, the daily reports from September 19 to 23 recorded no newly verified user task. The history examined contained 265 non-merge commits, of which 190 changed Markdown files only. These records do not tell me how much time the work consumed or whether a changed document was needed to build the next feature to meet its requirements.

The other project had recently made progress on recovery after failures, and in a system that has to produce evidence of its results, the verification tool is itself a product feature. Diagnosing both projects as producing useless tests would be wrong. I was concerned about the longer stretches in which further repairs and passed checks still left no additional use case ready to use.

What I had built into the process

In my assessment of spec-driven development, I described how much depends on executable acceptance checks. As later defects appeared, the instructions grew more specific. A reviewer found a test that did not detect its intended fault, so we added a counter-check. As the person commissioning the work, I took part in introducing another prohibition after an agent concealed an error status.

Many individual corrections had a good reason, but together they created assignments that agents could keep fulfilling without bringing the application any closer to a state someone could use. There was always another test matrix to complete or another report to correct. The product defect remained unresolved.

One of my rules for the spike even required duplicate work. Before the plan gate, I required an isolated experiment that ran a use case through the components involved, so the agent would find out early whether its approach worked or failed at an interface. Tests and findings were intended for the plan, while the rule required throwing away the experiment’s product code. The working integration had to be built again later.

I had introduced a handover at which working functionality could be lost. The concern behind the rule was reasonable: unreviewed experimental code does not belong on the main branch. But I had no justification for requiring every useful experiment to be rewritten.

The round budget also limited only part of the effort. Functional corrections were capped, while mechanical repairs did not count towards the limit. With tool problems and fresh reviews of individual changes adding more work, an attempt kept consuming resources even when, on paper, it had not reached its limit and was allowed to continue.

With these rules, I had expanded acceptance checks while making the route to the first working implementation unnecessarily difficult. A process change will not remove model errors or difficult architecture questions. I could still remove the compulsory duplicate work immediately.

The Definition of Done was already there

By an executable product path, I mean a flow from the intended input to the point where someone uses the result. For an import, that might be where an authorised query returns the imported data, while a successful helper function somewhere along the way says little about whether the complete flow works.

This belongs in a Definition of Done. The Scrum Guide requires a usable Increment that works with previous Increments. The Definition of Done describes the required quality state. Individual acceptance criteria add what the particular feature is supposed to do.

My projects already had completion rules of this kind. Another sentence about user value would hardly have resolved the stagnation. The Definition of Done says which state we accept, but gives the model no route to that state when it gets lost in repairs while building the feature.

I have therefore changed how the first implementation tasks are scoped. The first slice contains a small, complete use case across the necessary components, and we extend that running version to cover the remaining agreed cases until the assignment is fulfilled. The whole story is finished only once every acceptance criterion and the product’s shared quality requirements are met.

For a data product, that first slice might read a specified source record and return it through the intended read interface. Permissions and preserving the source belong in the experiment. Later slices add further data cases or recovery, as required by the assignment. This describes a possible task scope. We have not yet completed that product run successfully.

Reusing the useful experiment

I have corrected the disposal rule in the flow with a coordinator. The experiment stays isolated. Once its provenance, diff and affected risks have been reviewed, the same code may become the implementation candidate. The required plan review and authorisation still apply, as does independent acceptance before merging.

We discard an unsuitable experiment. If it proves useful, the agent works on an application whose behaviour it has already observed and adds the missing cases to the same code base until the result is complete.

For acceptance, I separate requirements from observations. Approved requirements and my decisions determine what the product must deliver. Code and runtime show what exists. If the application produces an incorrect number, more passing tests do not turn that number into the new expected value.

I reviewed the mandatory files too. User flows, examples and test notes are kept in ticket sections or refer to a shared source, where the next person or agent working on the feature will find the information. A second file containing the same text rarely helps.

Research on instructions and repairs

Evaluating AGENTS.md found no general improvement in success from additional context files on Python tasks from existing repositories. Agents followed the instructions and did more testing or exploration, with mean inference costs increasing by more than 20 percent. The June 2026 revision recommends necessary project-specific guidance and evaluating its benefit.

That is a narrower finding than an explanation of my entire SDLC. The study examined neither my prohibition registers nor our assignments lasting several days, but it does show that following an instruction alone says nothing about its benefit. During the revision, I asked what each additional obligation was for.

SlopCodeBench examines successive extensions to the same code base. In the May 2026 version, structural quality problems grew across many trajectories. Explicit guidance on code quality improved the initial code but did not prevent subsequent deterioration. The results concern the evaluated agents working on 36 Python problems with a total of 196 checkpoints.

I therefore want to observe the extensions after the first finished feature as well. If a later feature damages previous behaviour, that loss belongs in the same assessment as the new use case, even if the first version produced by the changed procedure looked good.

TraceCoder evaluates an approach to repairing code. It uses runtime traces, remembers failed repairs and rolls back harmful changes. On BigCodeBench-Complete, removing instrumentation reduced success from 89.04 to 78.51 percent. These component comparisons provide evidence for mechanisms within the system. The study tested benchmark code repair, rather than a full SDLC in a running project.

For my process, I am therefore testing the following assumption: before another repair, I need an observation that narrows down the cause, such as a test whose result distinguishes two possible faults or protects behaviour that has already been demonstrated.

Fewer handovers and focused checks

I start with one implementation owner and an independent review of the result, adding another agent when there is a specific question to answer, such as where a measured value came from. The agent acting as an early adversary may later handle acceptance, provided it stays independent of the author and meets the applicable model requirements.

Towards a Science of Scaling Agent Systems shows why agent count alone explains little. Collaboration helped on tasks that could be split into independent parts. Closely sequential decisions could fare worse with additional coordination. The study supplies no universally best team size for my projects.

Tests now follow the risk of a change. Data loss, unauthorised access and failed recovery still require independent negative checks against the product. Such a check deliberately introduces a relevant faulty situation and shows that the control detects it. Creating an artificial fault for every row of a text matrix is no longer a general obligation.

A text correction is reviewed against its sources and for consistency, while a configuration change could delete data and needs different checks even if its diff is short. File type does not decide this. Existing evidence can be reused where its binding to code, configuration and reviewed scope still applies. Affected interfaces and permissions are checked again.

Access controls and isolation remain in place. A scanner finding suspicious error handling is a reason to inspect it. Expected failures can be handled when the expected and actual statuses are recorded, every other deviation fails the run, and the environment is cleaned up afterwards.

When a repair attempt ends

The budget now covers the entire attempt. Mechanical fixes count alongside reviewer rounds, tool and environment work, and failed calls. Opening a new ticket does not reset the budget.

Before another run, the agent names the changed code or a different suspected cause, together with the expected result that will let it assess the hypothesis after the run. An identical repeat needs a question about reproducibility or variation. Without this distinction, I would just have added a mandatory field to the old loop.

When no knowledge is added or the budget runs out, we save the last useful state. The responsible agent describes the unresolved cause and a different approach. Where needed, I bring in a person with relevant expertise. The papers provide no universal limit on the number of rounds.

Ending this attempt does not close the product assignment. A smaller delivery scope requires my explicit decision. Failed checks and features still lacking evidence remain open, even if their unfinished parts would fit neatly into a follow-up ticket.

What became of the corrections

The revised rules have been in the training template and one of the two projects since September 26, alongside changes to the templates and dispatch prompts that previously contradicted the new flow. The rule checker’s existing self-tests passed in both directories. They check the behaviour of that tool, but do not establish whether the revised rules lead to usable software faster.

In the second project, a command-line agent for local language models, I did not add these corrections. Every approval there rested on a written verdict that a language model gave about a diff, and at no point did a machine check against the running product whether a task was done. The rules in this post change how such a verdict comes about. They give the machine no criterion it can check on its own.

So that project got a fresh start on the same day: a fixed list of user tasks, one executable scenario per task kept out of the agents’ reach, and a merge gate that alone marks a task as done. The loop’s lead agent there is Claude Code on Claude Opus 5.5, the developer sessions run in Claude Code and in Muse Code on Muse Spark 1.3, and the live scenarios have run against GLM 5.3 via OpenRouter since September 28. The old process is archived there, and the other project has run on the corrected version since September 26. A comparison of both flows on the same initial code is therefore no longer available.

The resulting process built around a loop and an oracle describes how user tasks, executable acceptance checks and integration form a reusable method.

Functional end-to-end tests use predefined tasks to check whether the finished system does what I expect of it. With that evidence in hand I accept code generation by agents as a black box, as long as the functional result can be checked deterministically. The product-level tests during development have their own job alongside that.

As a step in that direction I have added a coordinator to my development cycle. He helps me decide. Behind this sits a common practice: putting models and agents from different vendors on the subtasks they suit.

Updated September 26, 2026: The development flow preserves useful experimental code, limits total repair effort and bases additional checks on risk. Model assignments and the operating experiences described here date from the original version.

What work I hand to the coordinator

From the limits of human sign-off follows the need to pin down more precisely what I still decide myself. As the principal I set the functional goal and say when I accept a result. The coordinator aligns the running work with that.

We test our agentic CLI functionally end to end with Terminal-Bench 2.0. People curated the tasks, each one brings its own execution environment, a hand-written reference solution, and tests that check the result, as the Terminal-Bench paper describes. From that I derived the north star: a generic agentic CLI that works correctly under real model, tool, and failure conditions and proves its results. That is the hard yardstick.

For each self-contained product assignment I appoint a PO, an agent in the role of product owner. Technically he is an AI agent like any other. The role name says what job he has in the development process and which decisions I delegate to him. He carries the assignment through to an independently reviewed result. Three POs work on their own topics at the same time here, and each one answers for his own result.

Within my mandate the PO picks the approach and cuts the work into tasks. He decides which parts belong together functionally and where rework is needed, and he commissions developers and independent reviewers himself. He commissions the isolated experiment within his mandate. Once the plan has passed independent review, he authorises further implementation without asking me every time. What he decided sits in the ticket with its reasoning. I can change it or revoke the mandate.

The mandate itself lives in the repository. It names the product assignment, the permitted decisions, and the limits, for costs and for actions that reach outside, along with the reviews there is no way around and the cases I decide myself. Two things are closed to the PO: signing off his own work and loosening the acceptance criteria on his own authority. I hand him the day-to-day lead of a product assignment. For the product, I am liable.

If one PO needs a building block from another, the two agree directly on what gets delivered and how it is handed over. The supplier answers for his contribution, the recipient for the integration and for the result of his own assignment. Everything else runs on alongside. No PO outranks another. Each looks after one implementation strand with his developers and reviewers.

What he owes, the responsible PO enters in the existing ticket, together with the contributions he expects from others. The general rule for the role sits once in the working instructions and is only applied in the ticket. Whoever owns a topic keeps it. Passing on a subtask or shrinking an implementation step does not release a PO from a promise he has yet to keep.

The coordinator holds a mandate of his own for the questions between these assignments. He watches that responsibilities are unambiguous and that the agreed independent evidence stands behind every completion report. Where it is missing, he keeps the assignment open and sends the PO back to work.

A coordinator who confirms every handover or repeats every review does the work twice. He steps in when the POs cannot settle a conflict on their own. He also handles shared tool and provider problems and merges reviewed changes.

For an assignment I start with one owner of the implementation and an independent review of the result. I call the acceptance agent the Judge. Where a specific risk warrants it, the Judge also acts as an early adversary, checking which faults the planned tests might miss. These two tasks do not automatically require two agents. The reviewer stays independent of the author and follows the model requirements. Additional participants need a specific technical question.

A request to the coordinator names the responsible PO, the open question, and the commit. The existing review report goes with it. Where needed he reads the underlying files and raw logs. An unchanged status report triggers no fresh full review.

What I have not delegated comes to me. For that I need the affected state of the work and the possible consequences of a change. The coordinator puts his recommendation alongside it and names what is still unresolved, so that I can judge a concrete question without digging the whole affair out of the logs.

This agent errs too. So he does not rewrite a negative review verdict into a positive one. If I accept a known risk, my decision stands next to the original verdict. Access locks and cost limits remain in force. Before a release or a default activation I still require a decision paper, as far as I have kept that authority.

Precedents for these agent roles

Agents in such roles are already being studied. Closest to our setup is Effective Strategies for Asynchronous Software Engineering Agents of 23 March 2026. In the CAID procedure described there, a manager agent breaks the work down and assigns it, the developers sit in separate Git worktrees, and afterwards the manager merges their contributions and adjusts the remaining plan. The authors tested this on Commit0 and PaperBench. That is roughly what PO and coordinator do here. One difference stands out: in the main flow the developers review their own changes first. I staff that position with a separate reviewer.

Three further papers describe the same roles from other directions. Coordination Without Judgment, online since 17 July 2026, has agents appear as product owner, scrum master, and developer across 19 simulated scenarios and produce backlogs and sprint plans, though all I can reach of it is the conference abstract. Toward Agentic Software Project Management of 23 January 2026, accepted for the AGENT workshop at ICSE 2026, sketches a coordinating agent with specialised subagents and four operating modes that govern how independently they may act. Software Engineering in the Agent Era of 4 September 2026 separates delegated execution from the authority to accept a residual risk. None of the three has been tested empirically.

I have adopted that last separation. Every role now states explicitly which decisions I give away and which I keep. Our POs are one concrete shape of these approaches, extended by the release after independent plan review and by a coordinator for the questions between assignments.

How I distribute models across the roles

For the coordinator I will from now on pick the newest top model I can get for the job. The POs get a proven model from the generation before last. Development and review run on fast models that edit code and handle tools across several steps. A model change leaves the authority of a role untouched.

With the coordinator the effort pays off, because the hard cases land on him: spotting that two POs describe the same missing interface differently, or judging which parts of a promised feature are still owed after the assignment was cut down. My Codex registrations have GPT-6 Astra there with the reasoning setting xhigh, which governs how much thinking the model is meant to spend. OpenAI lists GPT-6 Astra as its most capable model for complex tasks. What the choice is worth, I see in the coordinator’s results.

A PO works on the same bounded assignment over a long stretch, and for that I would rather keep a proven model, even when a newer generation has appeared in the meantime and looks better on paper. Among my OpenAI assignments GPT-5.6 Sol sits below GPT-6 Astra, at Anthropic I have staffed POs with Claude Opus. “Generation before last” always means the offering of one single vendor. Version numbers from different vendors yield no common ranking.

For developers and Judges I want short feedback loops, but speed only helps if the model also does the job. The developer changes existing code and corrects it against the tests. With the Judge there is the added point that he repeats the required checks himself, for which he has enough context and access to the tools. Where that fails, I take a different review model for that assignment.

What recent research shows about splitting models

I find evidence for this split in work from the ten months between 13 November 2025 and 13 September 2026. The most vivid is the Sakana Fugu technical report of 19 June 2026. In a documented Terminal-Bench 2.1 run, GPT builds a server, then Claude Opus reviews the implementation and finds a false reachability proof, whereupon GPT corrects the work. In other tasks the models swap roles.

The remaining papers supply the measurements. Learning to Orchestrate Agents in Natural Language with the Conductor, first published on 4 December 2025 and accepted for ICLR 2026, distributes subtasks with a trained coordination model across GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro among others. The authors set this mixed pool against a variant that assigns every subtask to GPT-5, and on BigCodeBench and GPQA Diamond the mixed pool does better. Agent-as-a-Router of 22 June 2026 chooses among eight models and improves that choice through a verifier and a memory of earlier assignments. SWE-Router of 30 June 2026 first lets a cheaper model run a few steps and then decides from that trace whether a stronger one takes over, tested on SWE-Bench and SWE-Smith.

I take the deliberate combination of different models to be well supported. My fixed mapping of generations to coordinator, PO, and development is by contrast only one implementation in my own project. The Conductor, for instance, works with a purpose-trained model of seven billion parameters, while my coordinator additionally takes up conflicts between assignments and prepares decisions for me. For that I pick a current top model.

Models in use and how they are reached

The table shows documented assignments from project runs and registered control threads up to 13 September 2026, earlier variants included. It records what has run so far. The new selection rule did not yet apply to all of these assignments, and the sessions did not all run at the same time either.

Claude Code I use with direct access to Anthropic. In Codex a local provider profile determines which vendor the command line addresses: my OpenAI profile calls the GPT models, another connects Codex to OpenRouter, for instance for the model identifier z-ai/glm-5.2, and OpenRouter brokers access to GLM 5.2 from there. The tool is the same in both cases.

Muse adds a further coding agent. I commissioned one developer run through muse exec --provider meta --model muse-spark-1.3 at a high reasoning setting, the model runs at Meta, which describes it for coding and longer agentic tasks. I have never compared these models under equal conditions. A verdict on which one works faster is not mine to give.

Independent review after a model change

One staffing had Claude as PO and planner, Muse Spark as developer, and GLM as Judge through Codex and OpenRouter. Before a substitution the coordinator checks two things: whether independence holds and whether the run fits the released budget.

Claude Opus appears in the table as developer and as Judge, because it took on different assignments. No agent may sign off its own implementation. For the plan gate I additionally require different model families for planning and review, and another version number of the same family is not enough. DeepSeek V4 Flash was an early adversary in the evidence I drew on and never granted a final acceptance.

The review record states which model was called through which provider at which reasoning setting, along with the reviewed commit and the input. I take the logged call for this. What a model says about its own name does not count. Where an alias points at changing versions, I have to look up for the individual run which one it was.

How POs and the coordinator exchange messages

In my earlier attempt with three Claude sessions I watched Claude agents send each other messages. Claude Code offers ListAgents and SendMessage for this, the sessions find each other through their registration and send text over Unix domain sockets. The receiving session reads a delivered message between two tool calls, and if it happens to be idle, the message starts a new turn. Anthropic calls this cross-session messaging.

For our Codex sessions we tied the messages to the working documents in the repository. A PO writes his handover there and sends the recipient only a pointer to it through the queue. A helper of our own nudges idle sessions as soon as something changes. After an interruption the PO still finds his assignment again even when the earlier exchange has long dropped out of his context.

Codex also offers subagents for delegated subtasks. The PO and coordinator sessions are something else, they run longer and each carries its own assignment. We built the message path for these control threads.

State of the work and recipient have to match

For each PO it is recorded which Codex thread performs his role and in which working directory he operates. He is addressed through the thread’s UUID. After a restart I check this mapping, because old paused sessions can still be lying around under the same role name.

The handover sits in the ticket or in the handoff. It names what is finished, plus commit and branch, the review report, and the work still open. The shared coordination register points at it and carries responsibilities and unresolved questions. When something changes, the agents replace the head of the register, and older entries slide below it or into the archive.

Each of these files answers exactly one question. The order sits in the backlog table. How far the work has come sits in the ticket, the permitted scope in the release, and the independent review passes verdict on one specific commit, whose number is in the report. Whoever receives a completion report reads the linked files in the stated version.

With separate Git worktrees this is necessary, because before integration a new handoff note only exists on the sender’s branch, and a file path without branch and commit will send the recipient to the older version sitting in his own directory.

The queue delivers into the existing thread

For an addressed message we use codex queue. As an address the subcommand takes the UUID of the target thread or its exact session name, and which UUID belongs to which role I read off the process list and the rollout files that this very process has open. The entries in angle brackets are placeholders:

codex queue --thread "<recipient UUID>" \
  --message "PO1 to PO2: handover on branch <branch>, commit <sha>. Read <ticket path#section>. Open question: <question>."

PO1 reaches PO2 this way. With an unresolved conflict he addresses the coordinator instead, who answers in the existing thread of the responsible PO. I copy no text between terminals in the process. The queue takes the message, and the already open terminal interface processes it in the original thread as soon as the running turn ends.

There is no shared Codex daemon in the background for this. Queuing happens exclusively through the CLI, and nobody writes into the local queue database.

Reading it is allowed, though, and that has changed the path. At a safe point in its work an agent looks up for itself what the database holds for its own thread identifier, instead of waiting for someone to wake it. It remembers the identifiers of the entries it has already handled, so that a later delivery of the same message triggers no second assignment.

Such a call changes nothing about the recipient’s model and permissions. He carries on with his context and with the child agents he has already commissioned. A message to a PO does not start a second reviewer while the first is still running.

The queue transports text, nothing else. Linked files the recipient reads himself, in the right version, and he checks the message against his permitted assignment. In the log the queue message shows up as a user message, although no human wrote it. No additional human sign-off arises from that.

What the queue cannot steer

On 19 September I had an independent reviewer look into whether a running Codex interface can also be addressed directly, through the process identifier or through the thread. The answer was no. None of the three running processes offered a socket or pointed at a control socket, and a separately running app server was connected to none of these interfaces.

The obvious commands do not help. codex app-server proxy presupposes a control socket that is already running, codex remote-control start explicitly starts a daemon, and codex debug app-server send-message-v2 knows neither a process nor a thread nor a socket selector. The attempt to write straight to the foreground process’s pseudo terminal ended at the operating system with “Permission denied”. This was checked against Codex CLI 0.155.1.

That leaves codex queue as the only supported route. It queues. There is no switch for interrupting, attaching or waking immediately, and a running turn finishes before the message gets its turn.

A helper wakes idle sessions

A local resume helper watches what changes in the repository. A macOS LaunchAgent calls it every five minutes, as long as the machine is awake and the user is logged in. For every registered control thread, the role, the working directory, and the files to watch are on record. Anything explicitly paused is not woken.

It compares fingerprints of the local branch states and of the watched files, among them the working rules, the coordination register, and the tickets of the assignment. If something has changed, it checks whether the thread is currently idle and whether the matching terminal interface is still open, because only that may write into this thread. Where it is closed or the state unclear, the helper waits. Before delivering it also compares model, permissions, and working directory against the register entry.

The wake-up message itself says only: read the current sources, continue the open assignment. A PO then looks at what is pending, the coordinator at a cross-cutting question or a reviewed candidate for integration. A file change yields no new product assignment. If the session finds nothing permissible, it goes on waiting.

Delivery and completion are checked separately

A successful queue call confirms only one thing: the message is queued. How fast an idle session picks it up afterwards I measured once, on 19 September. The running turn ended at 09:02:34.469 UTC, the automatic follow-up turn began four milliseconds later, and at 09:02:34.488 the queued message stood in the thread as user input. No timer, no second controller, no keystroke from me. Before every automatic delivery the helper therefore drops a unique marker and notes the current position in the run log, so that it can later search there for the marker and the matching turn. Only that turn’s start and completion show that the session processed the wake-up message.

Just any completed turn is not enough. Even after the matching turn the functional success is open, and the results and checks from the ticket are what prove it. An assignment can be processed correctly and still end in a waiting state.

On timeout or unclear delivery the helper first reconciles queue and run log and only then decides about resending. A shared lock and at most one open automatic delivery prevent overlapping wake-ups. For the files themselves this lock does not help: before an agent writes there himself, he still checks whether he is responsible, because the helper reserves nothing against manually started work.

As processed the helper records the file state from before the send. If another handover arrives during the turn, it still sees it on the next pass.

The answer goes to the responsible PO

Suppose PO1 has a building block ready that PO2 uses in his assignment. PO1 files the handover with branch, commit, and review report and points PO2 at it, PO2 reads it and checks whether the block meets his requirement. For the promised delivery PO1 remains liable. PO2 demonstrates that it works in his own assignment. A detail conflict the two settle directly, and PO3 carries on with his assignment alongside.

Where the cut of the work is at stake, they send the question to the coordinator. He reads the evidence and records his decision within his mandate. After integration he reports the shared commit to the POs concerned, PO2 takes over that state and checks what has changed at his interface.

Developers and reviewers belong to the PO who commissioned them. The coordinator turns to that PO and does not start his child agents himself. If a control thread is replaced, the open assignment and the steering of the child agents move with it. The old thread is paused.

Register and delivery checking are tools of our own in this setup, and we have to maintain them. An error in the helper can bring the collaboration to a halt, even when all models are reachable and the product code works.

What I am correcting in my earlier assessment

In my assessment of spec-driven development I described end-to-end tests and independent audits as remedies against drift, against specification and code pulling apart, which nobody notices as long as the tests are green. One sentence in it went too far: an executable test does not drift, because it stays red until the code fits. That is not quite right. A test can carry the same wrong assumption as the code and stay green all the same.

Our own product-level tests demonstrated this to us. They produced the expected results themselves. Nothing came out of the product in the process, and the path from input to output went unchecked. After that correction the checks were still missing for whether two compared runs belonged together at all, which model they belonged to, and with which input they had been started. A formally flawless review report thus supported a statement about the wrong run.

The independent reviews found the errors, but only after several rounds of rework, at the end of which we cut down the scope of the implementation to reach a state that could be accepted at all. Then a model provider went down as well. The quota error that was last on the screen explained only part of the delay, the rest went to the review architecture, which we corrected as well.

An ever longer catalogue of small working instructions does not come out of this. When the same problems recur or an assignment stalls, the PO reviews his approach. The coordinator requires a reasoned decision and tracks whether it is actually carried out. What the PO chose sits with its reasoning in the existing ticket.

The functional review lies with the independent reviewer. He has a yardstick of his own and access to the original outputs. Fresh context alone does not protect him from confirming the same error with the same review logic.

The spike before the plan gate

My earlier flow started with a ticket and a feasibility check, then came ever more detailed plans, and whether their checks work at all often only came out in review. A mutation, a deliberately introduced error, then hit an unused code branch or stopped the compiler outright. About the functional check such a red run said nothing.

Work on a new feature begins with a bounded experiment on its own branch. It executes the smallest promised user effect from input to visible result. I have removed the disposal requirement: after reviewing provenance, the diff and affected risks, useful experimental code may become the implementation candidate without a rewrite. Plan review and the required authorisation still precede further implementation. Independent acceptance precedes merging into the main branch. My report on stagnation despite extensive testing explains why I changed this handover.

The attempt follows the input through to the component that consumes the result. Where a stored run record belongs to it, the product code produces it. The test prepares its inputs and the expected target values, and the observed output comes from the product.

For cases involving critical data, permissions or recovery, the developer runs a correct case and introduces a functional fault that fails at the intended test assertion. A valid counter-case still passes. This shows whether the control distinguishes faults or rejects everything. The possible harm determines the negative check. A text correction needs a source and consistency review, without artificial mutations for every table row.

For such risks, the adversary examines the early run using code and raw evidence. The reviewer searches for incorrect executions that the test would accept. If the same independent reviewer later handles acceptance, they use the saved results and review the affected changes again. The candidate report records which critical checks they reproduced themselves.

Where the reviewed values come from

The plan contains the executed checks with their results, plus the functional intent and the reasoning behind the architecture decisions. For the values the verdict hangs on, a provenance table is added: it points at the approved functional definition, names the writing product component and the recipient, lists the permitted deviations, and requires a check with deliberately wrong data.

When comparing a feature switched on and off, both runs get the same task and the same input, all other settings are equal, and only the switch under examination tells them apart. The evidence belongs to its particular attempt, and which review model passed the verdict is stated as well. Swapped evidence gets caught by the check, even when every single file is valid on its own.

A switch can be read correctly and still have no effect later. The product can discard a model response although the call succeeded. That is why the check reaches to the point where the promised effect arises.

I set functional expectations before the agents judge the result. Approved requirements determine what is expected, while code and runtime document what exists. Some acceptance scenarios are withheld from the developer. The developer writes technical tests, whose adequacy the independent reviewer assesses. Shared code for reading a file format is acceptable. The two sides judge functional validity independently.

When the same class of error appears a second time

Automated checks catch lint errors, missing tests and incorrect file references before model review. Their repairs count against the attempt’s total budget, alongside functional reviews, delta checks, tool and environment work, and failed calls. Separating error classes still helps diagnosis. It exempts no work from the limit. A new ticket does not reset the count.

When the same class of error appears again, the PO changes the approach and records why. Before each further run, the PO names a change or a different hypothesis and the observation needed to assess it. Identical repeats need a question about reproducibility or variation. If no knowledge is added or the budget runs out, the PO saves the last useful state and prepares a different approach or expert help. The product assignment stays open. A smaller delivery scope or partial completion requires my explicit approval.

The reviewer checks the entire submitted scope and bundles the corrections. Closed points he reopens when something relevant changes or new evidence speaks against them. An additional report on unchanged code triggers no full repeat.

Carrying on after a provider outage

If the required review fails because a provider quota is exhausted, its verdict reads “inconclusive”. The evidence is then not enough for a decision. Even in a brief progress report I want to tell this state apart from a failed product check.

Before longer review runs the assignment sets the time and cost budget, plus the number of permitted attempts, the review model, and a permitted substitute. An unplanned change requires a decision of its own, not least because it shifts the independence of the model families.

For the resumption the PO secures the reviewed commit and the raw evidence so far and notes which work is still missing. Before the next attempt the old run has to be finished and the code restored after test mutations. Per working area, still only a single agent writes.

If the original reviewer has finished his measurements and only the verdict is missing, he continues there. A new reviewer makes plain which measurements he takes over and which he repeats himself. A short successful model call says nothing about whether the provider will sustain a longer run.

Shared outages the coordinator handles with the POs concerned, and the resumption of individual assignments the respective PO organises himself. That keeps it clear, even with a substitute, who commissions the developer or the reviewer.

When a feature counts as finished

The progress report states which use case works through the intended product path and which previously accepted capabilities remain usable. An initial reference case demonstrates the approach. Completing the story requires all agreed acceptance criteria and the product’s shared quality requirements. Foundation packages alone do not fulfil that promise.

Each PO owns completion of their assignment. The coordinator merges independently reviewed changes and arranges checks of the affected integration. Existing evidence applies only to its code, configuration and reviewed scope. If interfaces, permissions, transactions or the environment change, the reviewer checks the affected assertions again. A new build receives its own provenance and package records.

If a feature is switched off at first, the activation evidence follows. The responsible PO organises the real on/off comparison with matching run records and the independent verdict on the observed effect. What the mandate says decides whether it then runs by default. A set configuration field is not enough for that.

The documentation normally consists of four parts: ticket or specification, implementation and test note, independent review report, and handoff. They sit as sections in the same file, as long as it is recognisable who wrote what. Raw logs and machine-generated run records lie alongside, and the working instructions point at them.

How I will measure the changes

All of this now sits in the working instructions and in the method template. For the running assignments I am keeping the procedure stable and will look at its effect across several completed user assignments. From the corrections so far follows neither a lower error rate nor a lasting gain in speed.

Further general rules I will only make once a shortcoming of the procedure is documented. If an assignment stalls, the responsible PO first investigates the cause: whether he understood the task correctly, whether his technical solution works, whether the model in use is fit for it, or whether he simply lacks the tool. The coordinator demands this clarification and a reasoned change of approach with it. Errors and security problems we keep fixing as soon as they surface.

I record the time to the first complete product path and to the fully accepted feature. The budget includes all work, including failed repairs, provider waiting time and my own interventions. For a comparison, the previous and revised flows should start from the same code and assignment with the same models, budget and independent acceptance.

Lasting progress also means that a demonstrated feature survives later changes. The acceptance and regression tests already planned are meant to show that. If a promised capability is lost again, that setback belongs in the report. A new role description says nothing in itself about whether the product reaches a capability and keeps it.

With the coordinator I watch which requests pile up at him. If routine questions or finished code are waiting there, something is wrong with his remit or his room to decide. For my own sign-offs I still need access to the original evidence.

Underneath sits the process I described in the SDLC of spec-driven development. The codebase holds a large collection of specifications, ordered into epics and tickets, with the dependencies spelled out. A developer agent did the implementing, and a judge checked its work independently. Claude Opus with effort high ran as the developer, codex --profile openai gpt-5.6-sol as the judge. A model served over OpenRouter also works for the judge.

The brief for the three

Each of the three sessions got the same base brief. The two prompts below are translated from the German originals. The document titles they cite are left as they stand in the repository:

You are product owner for [redacted]. You hold owner directive and scope authority. Your guiding standard for every decision is the North Star contract as laid down in “Absolutes Produktziel” and “North-Star-Zielbild und Prioritaetsordnung”. Together you are building one product and not isolated silos. You are responsible for the strategy, backlog, architectural interfaces, implementation and evidence of your product area. Inform the other product owners early about shared contracts, dependencies and changes. Root decides on priority or scope conflicts. Existing gates, independent audits and the binding backlog order remain in force. You steer the appropriate use of the judge and the developer.

On top of that came one area each. Product owner 1 got Correctness & Evidence:

Take responsibility for [redacted] delivering demonstrably correct results. Your area covers typed outcomes, independent semantic verification, calibrated pass/fail/inconclusive verdicts, evaluation, provenance and solid evidence receipts. Develop a coherent roadmap from that and carry each permitted piece of work through to reproducible live evidence.

For product owners 2 and 3 I wrote matching prompts, then started the three sessions.

Excerpts from the terminals

Alongside the usual agent reports, sections of their conversation appeared in the terminals. The originals are in German, the language I work in with the models. The excerpts below are unchanged, each followed by an English rendering:

PO 1

Verstanden: drei parallele Agenten, ich bin PO 1 (Correctness & Evidence).
Ich registriere das im Koordinationsfile und lasse Slots für PO 2/PO 3.

› Message from @[redacted]-f3: Koordinationsdatei liegt auf main:
  tickets/backlog/PO-KOORDINATION-2026-09-01.md, Commit a1ed15189
  (docs-only). Dein PO… (ctrl+o to expand)

....

› Message from @[redacted]: PO3 Vollzug: E426-Downstream-Rebaseline No-FF
  `31c89d791` auf main, Ledger-Nachtrag `d6a8db960` (docs-only).
  Du kannst … (ctrl+o to expand)

In English:

Understood: three parallel agents, I am PO 1 (Correctness & Evidence).
I am registering that in the coordination file and leaving slots for PO 2/PO 3.

› Message from @[redacted]-f3: Coordination file is on main:
  tickets/backlog/PO-KOORDINATION-2026-09-01.md, commit a1ed15189
  (docs-only). Your PO… (ctrl+o to expand)

....

› Message from @[redacted]: PO3 executed: E426 downstream rebaseline no-FF
  `31c89d791` on main, ledger addendum `d6a8db960` (docs-only).
  You can … (ctrl+o to expand)

PO 2

› Message from @[redacted]-3c: PO3 quittiert: T1967-Plan-Revalidation bindet
  ausschließlich den gelandeten V1-Vertrag (54ebaacde), bleibt bis zur User…
  (ctrl+o to expand)

In English:

› Message from @[redacted]-3c: PO3 acknowledges: T1967 plan revalidation binds
  exclusively the landed V1 contract (54ebaacde), holds until the user…
  (ctrl+o to expand)

PO 3

› Message from @[redacted]-67: PO1: kein Einwand gegen S1/S2 unter den
  bekannten Substrat-Auflagen — je Commit Re-SHA-Differential der 13
  Print-Plan-D… (ctrl+o to expand)

› Message from @[redacted]-3c: PO3 Vollzug (§3): T2089 Plan-Gate R2 pass;
  Gate-Evidenz docs-only No-FF `e56a36d3a` auf main, Ledger-Nachtrag
  `01bb95f2… (ctrl+o to expand)
  ⎿  “Ack Basis 01bb95f29; PO2 V2a in Implementierung” → sent to
     [redacted]-3c — another Claude session on this machine

› Message from @[redacted]-f3: PO1 hat die Altitude-Regel gegengezeichnet
  (docs/audits/2026-09-02-root-delegated-decisions-po1.md, main 4ce3090eb)
  mit… (ctrl+o to expand)

› Message from @[redacted]-f3: Quittiert: AGENTS.md-Abschnitt
  „Plan-Gate-Altitude“ mit PO1-Präzisierung gelandet, gilt ab jetzt auch
  für mein laufende… (ctrl+o to expand)

In English:

› Message from @[redacted]-67: PO1: no objection to S1/S2 under the
  known substrate conditions — per commit a re-SHA differential of the 13
  print plan d… (ctrl+o to expand)

› Message from @[redacted]-3c: PO3 executed (§3): T2089 plan gate R2 pass;
  gate evidence docs-only no-FF `e56a36d3a` on main, ledger addendum
  `01bb95f2… (ctrl+o to expand)
  ⎿  “Ack basis 01bb95f29; PO2 V2a in implementation” → sent to
     [redacted]-3c — another Claude session on this machine

› Message from @[redacted]-f3: PO1 has countersigned the altitude rule
  (docs/audits/2026-09-02-root-delegated-decisions-po1.md, main 4ce3090eb)
  with… (ctrl+o to expand)

› Message from @[redacted]-f3: Acknowledged: AGENTS.md section
  “Plan-Gate-Altitude” landed with the PO1 refinement, applies from now on
  also to my running… (ctrl+o to expand)

The three tell each other where they can get in one another’s way, meaning shared contracts and the order of merges into main. They announce a merge in advance and record the commit SHA afterwards. They also coordinate the run of the full test suite, because one pass now takes more than three hours.

Pretty wild, all of it.

How the messages travel

I had Claude research this in the Anthropic documentation. Claude Code has two tools for it. ListAgents shows a session which other agents it can reach, SendMessage delivers a text to a name. I called neither of them. I tell one session what the other should know, and it writes the message itself. Since version 2.1.224 the feature is there on macOS and Linux with nothing to configure, and I am running 2.1.258. Anthropic documents it under Cross-Session Messaging.

Every session binds its own Unix domain socket at startup, and the other sessions on the same machine deliver to it. On Windows, Claude Code uses a named pipe instead. Sessions find each other through registration files on disk, which is why a container and its host cannot reach each other. The detour through Anthropic servers falls away as long as both sessions run on the same machine. /status shows the session’s own address in the Peer address row, /list-agents shows the sessions it can reach.

The receiving session reads a message between two tool calls, and a running tool is not interrupted by it. If the session is idle, the message starts a new turn. Only plain text goes over the wire. The sender’s conversation history and its files stay where they are.

The limits are drawn tight. Claude Code rate-limits repeated messages from the same sender, drops duplicates that arrive close together and queues at most 50 accepted messages for the recipient. A loop between two sessions therefore runs out on its own. A message from another session also does not count as my consent. Anything that needs a permission still reaches me as a question.

The experimental team feature is not involved here. It is off by default, and with it a lead starts its own teammates and keeps a shared task list with them. I opened the three sessions separately, each in its own terminal.

All of these messages are transient. The binding record is the file tickets/backlog/PO-KOORDINATION-2026-09-01.md that appears in the first excerpt. Every product owner enters their claims, the ticket numbers and the merge announcements there before acting. The state thus lives in a versioned artifact, as the agentic engineering pipeline provides.

This concerns the Owner role in the spec-driven development SDLC that I documented here. This person approves the plan, authorizes implementation stages, and decides on the merge. Each decision becomes a repository entry alongside the verdicts of the review models. Between these points, implementers, subagents, and review models do the work. What the Owner knows of their work is what they present at the return point.

Recent research is examining this problem as well. Five papers from the past six months show two patterns: attention declines during a run, and reviewers rely on material supplied by the agent. That undercuts the purpose of their review.

The position paper “AI Agents Push Humans Out of the Loop” by Margaret Mitchell, Avijit Ghosh, and Samir Passi sharpened this argument on August 24, 2026. It brings together research on automation and human-computer interaction.

Oversight During a Run

Coding agents produce plans, tool calls, and test results alongside the code. They change their approach during a run and delegate work to other agents. Mitchell, Ghosh, and Passi describe what a person would have to do to supervise this process. The person would have to follow the stream, retain earlier steps, and assess the effect of the next action. At the same time, they grant permissions and attend to their own domain work. The interfaces offer little help. They display long logs that have not been prepared for live review, and they repeat approval requests until those requests become routine. The paper identifies another risk: people may lose domain practice and independent debugging skills after working with automated systems for a long time. No one has yet measured the strength or duration of this effect for coding agents.

A formative study with four software developers shows what happens to attention during a run. The four participants worked on a single task with the coding agent Cline, and their observed cognitive engagement declined over the course of the work. They reviewed the visible result more often than the path that produced it. Four participants cannot establish a general effect. An interview study of 17 experienced software developers found a similar pattern in daily work. The interviewees saved effort by using heuristics. Some treated the agent’s plan as a record of its actual behavior. Others relied on green tests as a guarantee of correct code. Twelve of the 17 participants worked at the same large technology company, so the researchers classify their findings as exploratory.

Both heuristics depend on material supplied by the agent. The plan records the agent’s intention. Green tests prove what the tests check. When the agent wrote those tests itself, it is also assessing its own work. Anyone who relies on this material reviews the self-report of the system under review. A run produces so much material that little else remains possible as long as supervision means watching the agent work.

Review Before Execution

The most extensive direct comparison so far comes from a study of 48 participants published in April. It tested four forms of oversight for agents that operate websites. The tasks involved privacy problems, prompt injection, and manipulative interfaces. Problematic actions occurred less often when participants reviewed the agent’s plan before execution. Once such an action was already visible, participants did not intervene successfully more often because of it. None of the four variants performed best in every situation. According to the interviews, intervention also depended on whether the interface marked the critical moment as a decision early enough.

For human sign-off, this is the most relevant result. A person made a difference when reviewing before execution. At that point, the interface marked a decision and presented a plan. During a run, successful intervention depended less on the oversight variant than on whether the interface marked the moment. The agents in this study operated websites. Applying the result to repositories remains an assumption. It is consistent with the heuristics in the interview study and with the decline in engagement observed in the Cline study.

What the Owner Actually Sees

The documented process in the spec-driven development SDLC assigns every operational role to agents. Only the Owner is human. The agent-supported workflow begins with a ticket and a feasibility check. Then come a pre-gate, a plan gate, implementation stages with their own review loops, completion, and merge. An independent Judge reviews the Implementer’s work. The process returns to the Owner after two failed correction rounds, when the plan contains a conflict, or before an irreversible external action.

This review chain finds errors. The audit artifacts from project A contain 77 failed, 53 passed, and 9 inconclusive verdicts. In project B, 23 of 82 recorded gate runs failed. Each verdict has its own review run and raw log. No Owner read that many runs in full. The Owner saw the states presented at the return points and the agents’ summaries of them. The decision stored in the repository must not claim more. An entry that contains only “approved,” a date, and a name leaves open which state the Owner saw and which criteria they applied. A later reader cannot infer that the Owner reviewed the entire run.

The process already provides an autonomy mandate for this purpose. The implementation stream operates within an agreed scope and reports at defined return points. These points lie where the studies find human review effective: before an execution step, at a marked decision, with a plan or fixed state available. Mitchell, Ghosh, and Passi propose the same arrangement. They call for fixed autonomy boundaries, reviews of related changes as a group, and machines that check in advance what can be verified automatically. At a return point, the Owner needs a clearly identified state of the work and its supporting evidence. A conversation log or the agent’s summary is insufficient. The reason is the same as for the interviewees: the agent’s plan did not record its actual behavior. The right-hand column of the table shows what continues without another sign-off.

The Sign-Off Package

Before work begins, the Owner records the domain expectations and the permitted external effects. Only then does the Owner see an agent’s plan or verdict. Mitchell, Ghosh, and Passi recommend this order so that the record shows whether the model shaped the person’s judgment. For sign-off, the order has a second purpose. It provides the criteria against which a fixed state can be assessed. Without those criteria, the Owner is left with the agent’s assessment at the return point, the same shortcut used by the interviewees.

At the return point, the Owner answers six questions:

1. Which commit and state change are being approved?
2. Which domain expectations were recorded before the agent’s verdict?
3. Which evidence exercises the real execution path?
4. Which deviations and open findings remain?
5. Which systems, data, or people may be affected?
6. Can the change be reversed, and has that path been tested?

The questions address the shortcuts observed in the studies. Question 3 requires evidence from the real execution path because the interviewees treated the plan as a record of actual behavior. The expectation recorded in advance places question 2 before the agent’s verdict, which might otherwise become the standard. Question 1 puts the commit hash into the decision. A later reader can then identify the state under review. Question 6 examines a condition in Gaube et al.’s model of effective oversight: the right to intervene. The sign-off evidence must therefore show that the rollback path has been tested.

The answers form the sign-off package together with the diff, test results, and review report. The Implementer assembles it. An independent review model cross-checks the statements against the repository, raw logs, and runtime evidence. The approval applies to the named commit and one state transition. After any change, the Implementer submits the new state again.

When the machine verdict is inconclusive, the team resolves the finding or reduces the scope. Besides approval and rejection, the process needs two further outcomes: resubmission with conditions and termination.

In my experience, people cannot keep pace with agents producing results at every step.

Conclusion

Cognitive overload is real. Placing a person in an SDLC as the human in the loop creates the appearance of safety and human participation. Only consistent validation by Judge or Auditor agents, in my experience, safeguards the running process. Domain-level end-to-end tests must also take on a new role in the SDLC.

Sources

• Margaret Mitchell, Avijit Ghosh, Samir Passi: AI Agents Push Humans Out of the Loop, arXiv 2608.23642 (August 24, 2026), https://arxiv.org/abs/2608.23642
• Chaoran Chen et al.: Comparing Human Oversight Strategies for Computer-Use Agents, arXiv 2604.04918 (April 6, 2026), https://arxiv.org/abs/2604.04918
• Shipi Dhanorkar, Samir Passi, Mihaela Vorvoreanu: Human oversight of agentic systems in practice, arXiv 2606.05391 (June 3, 2026), https://arxiv.org/abs/2606.05391
• Carlos Rafael Catalan, Lheane Marie Dizon, Patricia Nicole Monderin, Emily Kuang: “I’m Not Reading All of That”, arXiv 2603.14225 (March 15, 2026), https://arxiv.org/abs/2603.14225
• Susanne Gaube et al.: Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems, arXiv 2605.16278 (April 9, 2026), https://arxiv.org/abs/2605.16278

In the software development life cycle (SDLC), the model takes over implementation. An owner approves the work item, executable acceptance tests check the behavior, and an independent auditor examines the result. The merge decision rests with a human again. I have already evaluated this workflow in two larger projects.

The experiment on an M4 Pro

The test machine was a MacBook Pro with an M4 Pro, 64 GB of unified memory and macOS 26.6.2. On it ran Qwen3.8-27B in a 4-bit quantization and Gemma 4 12B-it in 8 bit. Both models were less than two weeks old on measurement day. The runtimes were llama.cpp, MLX, LM Studio and omlx.

The short performance test consisted of a prompt of around 700 tokens and 128 tokens of output. After a warm-up, three measurement runs followed at temperature 0. The tables give the median. All runtimes used GPU acceleration, Flash Attention and a context window of 200,000 tokens.

For Qwen, the MLX stack led llama.cpp by around 40 percent in generation. The hybrid model uses Gated DeltaNet, a form of linear attention, in 48 of its 64 layers. At the time of measurement, the Metal implementation in the MLX stack was faster. The result does not transfer to every model architecture, as the second series of measurements shows.

Gemma’s 40 sliding-window layers and eight full-attention layers ran at about the same speed in every runtime. The field stayed within around ten percent. Which runtime to pick therefore also depends on the architecture and the workload profile. With repeated long prefixes, a working prompt cache can outweigh several tokens per second of generation rate.

The M4 Pro provides 273 GB/s of memory bandwidth. Qwen’s weights occupy around 16 GB depending on the format, Gemma’s about 12.7 GB. Both models fit 200,000 tokens of context into 64 GB only because a small share of their layers keeps a KV cache that grows with context length. Measured RSS peaks lay between 13 and 29 GB. The entire series ran on the notebook.

The coding task

The coding side needed a work sample. Qwen and Gemma received the same brief: a complete German marketing landing page as a single HTML file. The brief prescribed the content and the sections. It required inline CSS, a responsive layout and a light and dark mode. External resources were ruled out.

Each run consisted of a single chat-completion request. Qwen worked without thinking mode on a budget of 24,000 tokens. Gemma got 16,000 tokens. Temperature, top-p and top-k followed the model vendors’ recommendations. All six combinations of two models and three server runtimes delivered a complete HTML document.

For the quality check, the pages were anonymized as A through F. Chromium rendered them at 1,280, 768 and 390 pixels wide in the light and the dark color scheme. A separate Opus model run also examined HTML and CSS. Ten points each were available for marketing effect, visual design, fidelity to the brief, code quality and overall impression. Nobody from the target audience took part in this scoring.

All three Qwen pages reached 37 to 43 points. Gemma came in at 14 to 22. Qwen wrote 745 to 881 words and laid out a recognizable design system. Its pages held 20 to 28 CSS variables and 14 to 37 inline SVGs. No required content was missing. Gemma produced 310 to 355 words, eight to nine CSS variables and at most one SVG. Every Gemma output lacked parts of the brief.

Qwen’s longer outputs contained errors too. One page hid its main heading behind a broken reveal animation. On another, the hero stayed two-column on the phone and got cut off. The Qwen page generated with LM Studio rendered without visible defects in every view checked. The Gemma field showed a transparent sticky header, overly wide tables and visible ASCII transliterations of German umlauts.

The best Qwen output covered all mandatory content. Its operating-system requirement was invented, and the dark mode deviated from the brief. In the SDLC described above, such findings go back to the implementer before sign-off.

There was one run per combination and no human control group. The 43 points therefore do not place Qwen in any ranking against real junior developers. What the experiment demonstrates is the takeover of a bounded work package with a checkable result. General parity across backend code, debugging and longer-running changes would take more tasks and direct comparison runs.

Hardware for around €10,000

My own measurement ran on a mobile M4 Pro with 64 GB. Two days after the benchmark, Apple presented a much larger local platform. The new Mac Studio with M5 Ultra starts at €6,599 in Germany. It can be configured with up to 512 GB of unified memory and reaches 1.2 TB/s of memory bandwidth according to Apple. In the largest configuration, the chip has 36 CPU and 80 GPU cores. The 512 GB model is slated for late October 2026. Apple quotes up to 4.3 times the AI performance. Independent inference measurements for the new machine do not exist yet.

The M5 Max in the same case already starts at €2,999, supports up to 128 GB and reaches up to 614 GB/s depending on configuration. The likewise new Mac mini with M5 Pro starts at $1,699 in the US, offers up to 64 GB and 307 GB/s. Apple claims up to four times faster prompt processing than the M4 Pro for one selected LM Studio test. This number too needs independent reproduction.

Nvidia’s DGX Spark has a different focus. For a $4,699 list price, the small machine offers 128 GB of coherent memory, a GB10 CPU-GPU combination and 273 GB/s of memory bandwidth. NVIDIA quotes up to one petaFLOP of FP4 compute with sparsity and inference for models up to 200 billion parameters. Two units can be coupled via ConnectX-7. At US list price, two Sparks stay just under $9,400 before tax.

Theoretical compute numbers are of limited use as a speed measure for language models. During token-by-token generation, moving the weights out of memory often caps the throughput. An independent comparison by Tom’s Hardware in July 2026 found higher decode rates on the M4 Max than on GB10 and AMD Strix Halo, while GB10 led in prompt processing. The DGX Spark opens up other models and modes of operation through its 128 GB and the CUDA stack. The Mac delivers plenty of memory bandwidth for dense models and a simple local desktop setup.

For around €10,000 you get a Mac Studio M5 Ultra in the base version or, at US list price, two DGX Sparks before tax. After the purchase come running costs for power, storage, replacement hardware and software-stack upkeep. The human work on specification and verification belongs in that calculation as well.

Documented model progress

Qwen’s model page allows a comparison of two generations in the same local size class. Qwen3.6-27B appeared on April 22, 2026, Qwen3.8-27B on August 14. In the results published by the vendor, the score on Terminal Bench 2.1 rose from 63.4 to 73.0, on SWE-bench Pro from 53.5 to 61.7. On DeepSWE 1.1 it grew from 13.3 to 42.2. Some tests ran in a Claude Code harness. The numbers document rapid improvement on these three tests, but they remain vendor figures.

An analysis of 20,574 real agent sessions from 1,639 repositories found seven recurring forms of misalignment. In 91.49 percent of the visible resolutions, the user had to step in with an explicit correction (arXiv 2605.29442, May 28, 2026). A supervised run has to budget for such corrections.

SpecBench shows a wide gap for specification work. The benchmark uses design discussions from Kubernetes, React, Rust, TVM and vLLM. The best agent tested caught 44.4 percent of the flaws the maintainers had found (arXiv 2605.30314, May 28, 2026).

A separate specification phase measurably improves implementation. SpecFirst splits the reconstruction of a program into behavior exploration and subsequent implementation. Across 200 tasks and four models, test pass rates rose by 6.9 to 21.3 percentage points over a combined pass (arXiv 2607.27167, July 29, 2026). In this SDLC, executable requirements give the implementer that groundwork.

The limits stay narrow on hard domain tasks. MDArena tested 50 real tasks from molecular-dynamics workflows. The best configuration examined solved 24 of them completely. On the hard tasks, every configuration stayed close to zero (arXiv 2608.02642, July 31, 2026). SWE-bench Science likewise comes in below 50 percent pass@1 for the best agent across 119 tasks from scientific repositories (arXiv 2608.19799, August 20, 2026).

These findings limit my claim to prepared implementation. System design and the clarification of incomplete requirements stay with experienced developers. For changes with domain risk, the implementation needs its supervision too.

My answer to the title question

For the measured part of the work, my answer today is “almost”. The local model turned a worked-out brief into a usable web page. At the same time, the spread between 37 and 43 points for Qwen shows how much reliability is still missing. The experiment demonstrates the performance in a closely supervised implementation role. It does not stand in for a whole position.

Factoring in the current pace, I expect local models on the hardware then available to clear this bar for prepared standard tasks within a year at the latest. The differences between the two Qwen generations and the new machine generation already point that way.

Sources

• Apple: Mac Studio with M5 Max and M5 Ultra, August 25, 2026
• Apple: Mac mini with M6 and M5 Pro, August 25, 2026
• NVIDIA: DGX Spark, technical data and price
• NVIDIA: DGX Spark User Guide
• Qwen3.8-27B, official model card
• Tom’s Hardware: local LLM comparison of M4 Max, GB10 and Strix Halo, July 30, 2026

The promise

When agents write the code, the human work moves forward, into the precise description of what is supposed to come into being. The specification becomes the actual source of the system. The code counts as a derived artifact that can be regenerated from it at any time. Tools such as GitHub’s Spec Kit or Amazon’s Kiro institutionalize this sequence. Requirements are captured, a plan is derived from them, the agent implements, and at the end the result is validated against the specification. Thoughtworks lists the technique in its Technology Radar under “Assess” (as of November 2025). The idea is older than the agents. Requirements documents have existed for as long as there have been software projects. What is new is the expectation that the chain from requirement to implementation holds without human translation, because an agent takes it over.

Behind this stands a promise of control. Whoever specifies with enough precision, so the assumption goes, controls the outcome, even without reading the emerging code line by line. The specification is meant to replace the review that secures correctness in human teams. In its strong form this means that a sufficiently precise specification yields a conforming implementation.

Both projects discussed here were set up as specification-driven from the beginning. The very first commit of project B contains nothing but specification files. In project A, the ticket, epic and ADR system likewise came into being before the first feature.

The setup

I developed both projects with coding agents. The agents implemented and reviewed. The documentation, too, ran through them.

Specification discipline was high in both projects. In project B, every larger ticket follows a schema with 23 mandatory sections, from fixing the scope through acceptance criteria to the verification and negative-control plan. A single specification package of 122 Markdown files contains 741 tables with close to 6,000 data rows. In project A, a central working constitution governs the ranking of sources, the separation of roles between implementer and auditor, and the conditions under which a result counts as accepted. A passed plan review comes before any implementation start, an independent audit before any merge, self-acceptance is ruled out.

What stands out is the size ratio between prose and code. In project A, the entire Markdown corpus of tickets, plans, audits and evidence documents comes to around 868,000 lines. That is almost twice the product code. In project B, around 244,000 lines of specification sit next to around 550,000 lines of code. Alongside the software, a second system made of prose has come into being, similar in size and with a maintenance need of its own.

The finding

By spec drift I mean specification and reality diverging. The code does something other than what the specification describes, or the specification describes something that never existed in that form or no longer does. In both projects this state was the rule over long stretches. Across the entire runtime I did not come across a longer strand of work that ran through without documented drift. The audit artifacts of both projects show this in four directions.

The implementation deviates from the specification. This is the case one expects, and it occurs in forms that no diff review catches. In project B the specification required deterministic generation of content. A repair plan later counted 16 places in the code that read the system clock anyway. The effect stayed invisible for months because an earlier reference export had frozen the same generation date: the tests compared against reference data that contained the same error. An audit in project A documents a case in which the system prompt required two shell commands before any assumption about a path, while the rule set blocked one of those commands. 48 of 51 observed command sequences followed the instruction and ran into the block. In the process, 21 test runs consumed their full iteration budget without effect, and all processes nevertheless exited with code 0. Another audit showed that a normalization function truncated multi-digit error counts to zero, which allowed a check module to be set to “passed” despite explicit evidence of errors. A documented configuration switch for the token limit also had no effect: four measurement runs with four different values produced exactly the same output length every time, without an error message.

The documentation ages against the code. An automated comparison in project A found references to source files that had long been deleted or renamed, and a contradiction register counted eleven different versions of the same test count across the documents. Each of those numbers had been correct at its time and then froze. Status drift is more serious. An independent audit across 16 work packages concluded that six of them carried status information that the reality of the main branch had overtaken. Work already merged appeared open. Agents read these artifacts as a source of control, so every later gate would have checked against a false status. The same class includes a finding in which four core levers of the implementation existed only on a side branch and were preserved solely through an archive tag, while planning listed them as available. In another case a passed plan gate lay only on the working branch and was missing on the main branch, against which it should have applied.

Specifications contradict each other. In project A the acceptance criteria of two tickets collided head-on. Both concerned the same function. One required a tightening of the existing patterns, the other a targeted loosening. Another ticket declared itself a precondition of one that knew nothing of this dependency. These specifications were written at different times by different agent contexts, none of which surveyed the whole body.

The specification contradicts itself. This is the direction that surprised me most. A plan audit in project A showed that three provisions of an already detailed plan produced no callable code path between them. The plan was unfulfillable in itself before a line was implemented. In project B an audit found that nine mandatory sections of the parent specification had disappeared from the tickets through a compaction pass, and nobody had noticed until a reviewer counted them. In the same project there is an audit document that turns its own earlier “passed” into “not passed” by addendum, because the chain of evidence had been tied to the wrong objects under review.

In my assessment the cause lies in the working model itself. Agents work with a limited context window in sessions that end. In project B, subagents ran into session limits after around two hours. A project of several months is then made up of hundreds of handovers between contexts, each of which sees only a section of the system and of the specification. At every handover, specification and reality can drift apart a little. One agent changes code and misses some of the affected documents. The next reads one of the stale ones and builds on it. Resumption prompts and handover documents soften this, but they do not replace the missing overall view. The specification ages just as the code does. It sits in the system as a maintained artifact, and unlike the code it has no tests. Specifying with more precision therefore mainly enlarges the surface on which the specification ages. The 23 mandatory sections in project B did not prevent drift in any case.

I am not alone in this observation. A field study of 20,574 real agent sessions found systematic gaps between assignment and execution. In 91 percent of the visible cases an explicit user correction was needed (arXiv 2605.29442, 2026). Even the best agent in a benchmark on specification competence identifies only 44 percent of the defects in specs, that is, gaps, ambiguities and contradictions (arXiv 2605.30314, 2026). A comparison of three spec-driven frameworks across two model families concludes that stricter binding to the specification even lowers the determinism of the output. What remains attainable, the authors conclude, is verifiability (arXiv 2606.30689, 2026). The mechanism of context loss is described as well, down to the detail that context compression preferentially discards exactly those tokens that encode the original specification (arXiv 2607.27167, 2026). From industry, Atlassian’s engineering blog reports the same: with a full context window, agents silently abandon earlier decisions, and larger models do not solve the problem (Atlassian Engineering, June 2026).

What helps in practice

Three countermeasures emerged in both projects and have proven themselves in daily use. All three move reliability to places that can be executed or checked independently.

When a model picks a different solution than the specified one, the reflex is to roll the deviation back. That reflex is expensive and often unfounded in substance, because many deviations are technically equivalent or better. It is more workable to accept such deviations and to check whether the system behaves the way it should. Whether the code follows the wording of the specification becomes secondary. The instrument for that is end-to-end tests against the real execution path. Project B holds close to 600 browser end-to-end cases next to a good 6,000 unit cases. For every new functional unit an end-to-end proof of its own is mandatory. Whoever accepts a deviation has to update the specification as well, otherwise the same deviation shows up as a finding again in the next audit. In both projects, end-to-end tests have been the only instance that checks semantics. In project B the same component delivered a green run at package level with more than 400 tests. On two simulators, the integration run then failed on a single pair of tests. The project’s working rules have recorded since then that a green package run does not establish correctness. The working constitution of project A states the corresponding principle, that a live gate can be green on broken semantics. That is why every feature there requires an end-to-end scenario of its own against the real target system. A documented live run across 54 check cells ended with 16 cells passed. An honestly red result is worth something in this way of working, because the counter-class is documented as well. One gate reported green three times although the child system under test produced malformed output. Because of such false-green cases, the E2E tests themselves have to be checked, for instance on whether they are able to fail at all.

E2E tests do have another weak point, and it is well measured. When the implementing agent sees the tests and iterates on them, it optimizes for passing them. Whether the task is solved drops out of view. A benchmark on reward hacking in long-horizon coding agents documents as an extreme case a 2,900-line sham compiler that had simply memorized the test inputs. The gap between visible and held-back tests grew there with the size of the code (arXiv 2605.21384, 2026). My way out of this dilemma is end-to-end tests that humans curate or whose content humans prescribe. The scenarios and expected values come from the domain. The implementing agent runs them but does not write or change them. Part of the scenarios can also be held back and run only once the implementation is finished. That costs human working time at a point where it pays off. These tests are the one artifact in the whole chain that the agent may neither produce nor reword.

Every implementation additionally goes through a loop of independent audits. I described the pattern in detail in “The second lane”. Here the core is enough. One agent implements, a second agent with fresh context and the explicit task of refuting the result checks it, and a machine-readable verdict decides the merge. Project A also prescribes that reviewer and implementer come from different model families. If both are the same, the verdict is discarded. The reviewer runs there in a write-protected working directory of its own, without tool access to the running session. Its prompt, its raw log and its verdict are bound together by checksums so that a verdict can later be assigned to an exact state. The loop arises because a “not passed” triggers a correction round and the correction is checked again.

That these loops find real defects and not just formalities is shown by the verdict statistics. The machine-readable verdicts of the audit artifacts in project A read 77 times “fail”, 53 times “pass” and 9 times “inconclusive”. In project B, 23 of 82 logged gate runs ended in “fail”. Almost all the drift cases cited in this post come from exactly these review runs. Without the auditors they would have stayed in the system, because the respective implementation had reported itself finished.

The reviewers have limits of their own. In project A the follow-up check found a reasoning error in the report of a review model. The report cleared a test failure with the claim that the affected file was unchanged, which the diff refuted. A model used as a judge now goes through a calibration procedure of its own there, with independent double labelling. At present it is not cleared for unsupervised use.

Studies support both precautions. An evaluation of around 541,000 judgements from 21 judge models found high consistency alongside weak validity. The authors call for a dedicated validation protocol for judges (arXiv 2606.19544, 2026). Cross-model review also works differently depending on direction. In one study a Claude review of Codex drafts raised the success rate from 71.6 to 89.7 percent. The reverse pairing lowered it from 91.4 to 82.8 percent (arXiv 2607.21656, 2026). So which family implements and which one reviews is itself a decision that has to be validated.

The third countermeasure starts before the implementation, with the form of the specification itself. For agents over long periods, prose is too ambiguous. Each of the four drift directions from the finding is ultimately a property of text that nobody executes. From this follows a shift left toward more formal specifications. A human or a lead agent assigned to the job first writes integration tests to implement against. Where a test form does not suffice, a domain-specific notation that can be evaluated by machine helps. A failing test is a hard specification. It does not drift, because it stays red until the code fits, whereas a Markdown section can assert something false for months.

In rudimentary form this already exists in both projects. Project A holds ten drift test classes of its own that check registries, configuration fields and manifests against the code on every run. Project B runs sabotage gates that abort with a defined error code as soon as a contract is violated. One of them, though, has also raised a false alarm because of a tooling artifact. The limits described above continue to apply. A test that the implementing agent is allowed to shape can turn out tautological. What lies beyond the testable, such as architectural intentions and rationales, still needs prose, only much less of it. For the checkable core of the requirements, a test is nevertheless the most stable form I have seen in either project.

What it costs

The most visible item is token consumption. The operations notes of project B record that stranded agent runs, which hung at session boundaries without a commit, burned 250,000 to 350,000 tokens per resumption without progress. A single review run there produces 500 kilobytes to one megabyte of text. In project A the audit artifacts log the ratio between internal working log and visible result: a good 8,300 lines of raw log for a verdict of 64 lines. This effort falls due again with every correction round.

On top of that comes operational effort that appears in no token bill. The notes from project B describe agents that hang without a completion message and can be recognized only by circumstantial evidence, such as absent file changes alongside missing processes. They also record the discipline of running only one active agent per work area, because a replacement agent started too hastily overwrites half-finished states. A resumed agent once discarded someone else’s not-yet-committed work. Since then the rule is that discarding is allowed only by a reversible route.

A problem of its own is the models’ tendency toward micromanagement. Left unattended, the review loop escalates. The extreme values come from project A. A single ticket accumulated 96 audit documents, and another went through 23 rounds in two calendar days. A pure specification artifact without a line of product code was re-audited nine times. The ninth round still changed 122 added and 75 removed Markdown lines compared with the eighth. In doing so, the models invent ever finer occasions for review: bisection steps, diagnostic tickets, interim acceptances. The project notes now name this explicitly as a relapse to be avoided. In project B I intervened myself and stopped an agent that was spinning through its third correction round on the same test while the product code had long been correct.

Without fixed stop rules that bound the loop from outside, the approach therefore tips over. In project A several forms of such circuit breakers have become established. A two-correction-round rule ends the loop after two unsuccessful rounds and forces a decision between accepting with documented risk, descoping, or abandoning. This limit has empirical counterparts. Across seven models, the gains from self-correction loops concentrate in the first two rounds (arXiv 2604.10508, 2026). In one investigation with smaller models, multi-agent debates cost 2.1 to 3.4 times the tokens at equal or worse accuracy. Peer critique there destabilized previously correct solutions (arXiv 2605.00914, 2026). Counted authorizations cap expensive runs in advance: a decision artifact authorizes exactly one start, without selective repetitions and without cherry-picking from historical runs. A concurrency cap restricts the simultaneously active implementation strands to two. A prose cap sets, per work package, the added Markdown lines against the added product code lines, so that the documentation does not overgrow the code. I find it notable that the most recent measure points in the opposite direction. It is an explicit autonomy mandate that forbids the implementation strand to file interim reports and provides only a few defined return points.

Taking stock

Spec-driven development did not deliver on its central promise in two disciplined, long-running projects. The specification drifted against the code, specifications contradicted one another, and single artifacts even contradicted themselves. By everything the audit artifacts show, more discipline would not have switched this off. Drift was the base rate of a working model made of months-long workflows and hundreds of context switches.

What did work is a different architecture of reliability. End-to-end tests check the behaviour of the system. Independent auditors catch in their loop a large share of the defects that the implementation had reported as done, and stop rules keep this loop affordable. Where requirements exist as tests from the outset, the prose surface on which drift can arise at all also shrinks. The specification changes its role in this setup. It remains a starting point and a reference for review. Against it, deviations become visible, and a conscious decision is then made about them. The role of the source from which the system reliably follows is one it did not fill in either project. The research fits this picture. An explicit specification phase measurably improves test pass rates, by 7 to 21 points in one recent investigation, but does not determine the result (arXiv 2607.27167, 2026). Part of the research pursues the counter-strategy and wants to prevent drift technically, such as gates that make spec-code divergence a merge blocker (arXiv 2606.27045, 2026). Whether that can be sustained over months, or merely shifts the point at which the maintenance costs arise, is empirically open.

None of this solves the underlying problem either. E2E validation, auditor loops, executable specifications and circuit breakers are mitigations. They make drift visible and give it a price. In both projects it has stayed nonetheless. Drift arises anew at every seam as long as the work is done by actors that have no lasting memory and never see the whole system. The review architecture works against a symptom whose cause lies in the shape of today’s tools.

The cost question remains open for me. In both projects, the effort for specification, audit and evidence taken together was larger than the product code, measured in lines. From two projects I cannot derive whether an intentionally smaller specification surface with earlier behavioural tests would come out cheaper. Only one item of that calculation seems certain to me: the ongoing maintenance of several hundred thousand lines of prose, along with their own drift.

The data

All figures and case examples from the two projects come from their repositories: from Git histories, ticket inventories, operations notes and the machine-readable audit and gate artifacts. The projects are internal undertakings, and I have chosen to describe them here only through key figures.

Sources

• How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions, arXiv 2605.29442 (2026), https://arxiv.org/abs/2605.29442
• SpecBench: Evaluating Specification-Level Reasoning for Software Engineering LLM Agents, arXiv 2605.30314 (2026), https://arxiv.org/abs/2605.30314
• Citation Discipline in Spec-Driven Development: A Cross-Model Empirical Study of Output Determinism and Automated Hallucination Detection in LLM-Generated Code, arXiv 2606.30689 (2026), https://arxiv.org/abs/2606.30689
• SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch, arXiv 2607.27167 (2026), https://arxiv.org/abs/2607.27167
• The Spec Growth Engine: Spec-Anchored, Code-Coupled, Drift-Enforced Architecture for AI-Assisted Software Development, arXiv 2606.27045 (2026), https://arxiv.org/abs/2606.27045
• SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents, arXiv 2605.21384 (2026), https://arxiv.org/abs/2605.21384
• Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?, arXiv 2607.21656 (2026), https://arxiv.org/abs/2607.21656
• Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias, arXiv 2606.19544 (2026), https://arxiv.org/abs/2606.19544
• How Many Tries Does It Take? Iterative Self-Repair in LLM Code Generation Across Model Scales and Benchmarks, arXiv 2604.10508 (2026), https://arxiv.org/abs/2604.10508
• The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate, arXiv 2605.00914 (2026), https://arxiv.org/abs/2605.00914
• Atlassian Engineering: Why AI Agents Drift Mid-Task and How a Multi-Agent System Fixes It (23 June 2026), https://www.atlassian.com/blog/development/specialist-agent-orchestration-jira
• Thoughtworks Technology Radar: Spec-driven development (ring “Assess”, as of November 2025), https://www.thoughtworks.com/radar/techniques/spec-driven-development

Why the agent duplicates

The cause is a grounding problem, a gap between what the agent sees of the codebase and what is actually there. A language model reads the repository file by file in a limited context window. It has no global map of the structure. When the matching existing path lies outside the loaded context, the model derives logic again that already exists. In a large system grown over years, the probability is high because the relevant code is more likely to lie outside the window than in a small, fresh project.

Ticket shape adds to the problem. A terse request such as “implement X” leads to a new helper more easily than an instruction to wire X into the existing path and document every affected surface. A green test often proves too little. It shows that the new path works. It does not check whether the actual product path calls it.

Data now backs this observation. The study “More Code, Less Reuse” (arXiv 2601.21276, 2026) compared AI-generated and human pull requests using objective code metrics. According to the paper, AI-generated changes contain more duplicative code and reuse existing functions less often. In an existing system, that is expensive because every copy has to be maintained, tested, and understood separately later.

Corrections from a branch

I see this in all feature implementations in my fairly large codebase, about 600K LoC. In my projects, every feature runs through a fixed review process by default: implementation, independent audit, and hard gates. This evaluation is one example of that process. From the review artifacts of one branch and the corresponding fix commits, I was able to count many concrete correction findings in the Git history. These are places where a first agent run got something wrong and a downstream step had to correct it. The rate is high. Across just over fifty evaluated commits, between 75 and 86 percent of the reviewed implementation units needed a correction, a different scope, or another review, depending on how one counts. The most frequent classes break down like this:

The second lane from the first section sits here at rank 3, “existing product path/surface not fully wired”. In the measurement, it appears as its own error class at just under 17 percent. By “claim” in the first row, I mean an asserted result, such as an agent’s statement that an acceptance criterion is fulfilled. Beyond the correction rate, the distribution shows which type of error a downstream reviewer can catch at all and which type arises before implementation. I treat the second lane separately here because, in a later production setting, it is the most expensive and hardest problem area to correct. Production errors in those places are hard to find and fix. This evaluation also documents that a review and audit agent has to be set up systematically as a fixed part of the development pipeline.

Every class in the table is a finding from the audit model. The corrections come from its review artifacts and from the fix commits that followed. The distribution is therefore the reviewer’s balance sheet, a record of what an independent model caught before it could move into a later state of the system. Without that review pass, the defects would initially have gone through because the implementation itself had reported them as done.

For the most frequent class, just under 28 percent, the reviewer found insufficient tests. The implementation had delivered tests that only kept the happy path green and produced no red counter-evidence. A trigger matrix, for instance, did not cover the triggers it claimed to check, and a negative test did not even create the failure case. The correction in each case was to build the tests so they fail when the claim is false. A team that accepts its own work misses this class because a green run looks like evidence.

The second-largest class, just over one fifth, was in the specification. Tickets were ambiguous or cyclically cut, and a dependency was not marked as a hard precondition. A final gate was scoped too narrowly, and the scope of two tickets did not match the state actually released. The reviewer found these contradictions by comparing tickets, README, and dependencies.

At rank 3, the second lane appeared as a surface that was not fully attached to the shared backbone. A second entry point had been decoupled from the shared resolver that a first one already used. Alternative bootstrap paths lost data along the way. In one place, a second resolution lane appeared next to the existing path without reusing it. The correction was a shared helper and proof that all surfaces use the same backbone.

The remaining two classes concerned data contracts and drift. In data contracts, the issues were a missing safe default, empty deltas that overwrote existing values, and serialization that was ambiguous. In drift, schema, app help, and reference state diverged from the actual code. That blocked releases because agents use exactly these artifacts as contract sources.

Across all classes, the audit model exposes the problems that a self-accepting development team would have passed on unchecked. The measurement justifies the second reviewer across the whole distribution.

Implementer and auditor as a refutation process

Against the second lane and the other error classes, a division of labor helps. I run it in my projects as a gate process. One agent implements along the ticket, repository rules, and existing paths. A second agent with fresh context tries to refute the claim. Hard gates, re-runs, greps, and artifacts decide the result. The implementer’s self-summary does not count as evidence. A fail or an inconclusive stops the merge.

It comes down to the adversarial stance. A second model that only reads the same soft summary and nods it through helps little. It becomes useful when it greps code paths, follows call sites, reruns tests, compares baselines, and searches specifically for red counter-evidence.

A model mix helps as well, for example Claude or Opus as implementer and a Codex or GPT model as auditor. Different families have different strengths, search patterns, and error profiles, and a model that accepts its own output judges too leniently. The actual safety gain, though, comes from separating role and context, from the refutation mandate, and from hard gates and reproducible re-runs. That includes fail and inconclusive actually stopping the merge.

The major providers follow a similar direction. The relevant line consists of several pieces: exploring the codebase before any code is written, defining a plan and clear done criteria, and backing the change with tests and review. It continues with a diff inspection in fresh context, findings returned to the implementer, and another review after the correction.

OpenAI describes a planning phase first for difficult Codex tasks. Codex is supposed to gather context, clarify open questions, and build a plan before implementation begins. For large repositories, OpenAI names four fields for the task: goal, context, constraints, and a clear “Done when”. The last point is central for brownfield work because “done” is tied to observable states, such as passing tests, changed behavior, or a reproduced bug that no longer reproduces afterwards. According to the documentation, the review loop includes tests, relevant checks, confirmation of the result, and a diff review for bugs, regressions, and risky patterns. The review surface can inspect uncommitted changes, branch diffs, and individual commits. Inline comments on the diff become working context for Codex again. OpenAI uses subagents for bounded checks: exploration, tests, triage, or review perspectives such as security, test gaps, and maintainability. Applied to the second lane, this means: the auditor reads the diff, tests, and task. The implementer’s success story is only a claim to be checked.

Anthropic is the most explicit about this pattern. In the Claude Code best practices, there is an adversarial review step: before completion, a subagent should review the diff in fresh context and report gaps. The reviewer sees the diff and the criteria, but no long implementation rationale. For correctness checks, Anthropic points to /code-review, which reviews the current diff for bugs in a fresh subagent and returns findings to the running session. Anyone who wants to check against a plan should name the plan, the diff, and the finding criteria explicitly, such as implemented requirements, tested edge cases, and scope adherence. The implementation session can then correct the findings and trigger review again. This creates the cycle I describe here as implementer-audit-feedback-correction.

Google provides evidence more through GitHub and CI workflows. Its official run-gemini-cli action names pull request reviews, issue triage, code analysis, and code changes as use cases. The PR review example runs automatically for new pull requests or manually through @gemini-cli /review. It gathers PR data and changed files, runs the Gemini analysis, and posts review comments back to the PR. Its prompt limits comments to changed diff lines and requires verifiable findings, severity classification, and concrete suggestions. In the Gemini CLI repository itself, the process documentation separates issue and PR: the issue describes what and why, the PR describes how the implementation works. CI checks linting, tests across several platforms, and coverage. PR automation checks the connection between issue and PR. Checkpointing adds to the correction path because Gemini CLI can create snapshots before AI file changes and restore files together with conversation state.

For the second lane, this yields a process that none of the providers describes as a ready-made rule. The agent that writes new code has first delivered only a local proof. It has shown that its new lane works. The missing evidence is often elsewhere: does the existing product path use this lane? Are all surfaces wired? Does a consumer use the new signal in the live path? Does the test cover the claim or only the newly built helper? These questions have to appear before implementation and in the audit. Before implementation, the Second-Lane-Grep forces evidence of existing code, config, test, or runtime paths. After implementation, the auditor checks the diff against plan, scope, and existing call sites. The finding goes back to the implementer, the correction lands in a new commit, and only another gate turns that state into something fit for release.

My gate template is stricter than these provider pieces. It combines OpenAI’s plan, done, and review concepts, Anthropic’s fresh adversarial review context, and Google’s PR and CI automation into one workflow. That workflow consists of plan/spec review before code, Second-Lane-Grep, Claim-Gate, an independent verdict artifact, correction by the implementer, and re-review. The cross-model rule is not stated that way in any of the provider sources. I use it as an additional hardening step because a different model family often brings different search paths and error profiles.

A concrete review workflow

The workflow starts before implementation. For an epic, I create a plan/spec review gate. This gate produces no product code. It checks all implementation tickets against the epic README and against each other.

• Ticket contract: Each ticket needs an objective, boundaries, and an output or acceptance format. Boundaries explicitly name what the ticket is not responsible for. If one of these fields is missing, the ticket gets no pass.
• Scope and dependencies: The reviewer looks for overlap between tickets, unclear ordering, missing preconditions, cyclic dependencies, and contradictory switches.
• Second-Lane-Grep: Every requested new implementation needs grep evidence. Are there existing code, config, test, or runtime paths that would need to be extended? The evidence names files, call sites, or typed models. If it is missing, the plan is fail or inconclusive.
• Claim-Gate: For every acceptance criterion, the ticket names the deterministic test that fails when the claim is false. If that test does not exist yet, writing it is part of the ticket.

Only after that does the first agent implement. It wires in the existing backbone, justifies new abstractions, treats producer and consumer together, checks all affected surfaces, and carries tests and artifacts along. Self-acceptance does not count.

The planning gate writes its result into a machine-readable verdict artifact, in my projects, for instance, as e2e-live/baseline/txxxx-plan-gate/VERDICT.json:

{
  "verdict": "fail",
  "planner_family": "gpt",
  "reviewer_family": "claude",
  "findings": [
    {
      "severity": "blocking",
      "artifact": "tickets/T1234.md",
      "finding": "Acceptance criterion AC-2 names no falsifying test."
    }
  ],
  "commitSHA": "..."
}

In the final audit, an independent model reads ticket, diff, tests, README, schema, and artifacts itself. It reruns the central gates, greps for dead and duplicate paths, and compares the existing product path with the new one. The verdict uses the same basic shape and records the fields implementer_family and auditor_family.

On fail or inconclusive, the artifact contains blocking findings with file or artifact evidence. Before the merge, these findings are dispositioned: blocking, non-critical, or deferred. A public performance claim needs its own Claim-Gate.

The most effective single rule shifts the burden of proof. A solution that creates a second implementation next to an existing path without wiring that path in does not count as reviewable. The implementer has to show that a new path is necessary and fully wired. “The second path does no harm” is not enough.

Checklist against the second lane

Per ticket or review artifact, a few checks are enough to find a second lane.

• Existing path: Which files, call sites, typed models, policies, and tests prove the existing product path?
• New change: Were new files, services, resolvers, or switches created? If so, does the ticket or an architecture note explain why the existing path was extended or why a new path was necessary?
• Wiring: Which producer delivers the new signal, which consumer uses it, and is that consumer in the live path? Are there fields or events without a consumer?
• Gate evidence: Which test would have been red before the fix? Does the gate cover the real claim or only the happy path? Do ticket, README, schema, and code agree?

Limits of the process

The workflow has weaknesses that need to be understood.

The auditor can adopt the same assumptions as the implementer. If it reads the same ticket, the same summary, and the same logic, the errors correlate. Fresh context helps against that, as do a different model family, an explicitly adversarial prompt, and a ban on taking the development summary as the sole source.

Gates can be soft. A gate that checks only happy paths can even stabilize a second-lane error. The green status of the new path then hides that the actual product path stayed defective. That is why every claim needs a test that would have been red without the fix.

Artifacts drift as well. Tickets, README, schema, snapshots, and code diverge over time, and because agents use artifacts as steering sources, this drift leads directly to wrong decisions. My own evaluation lists documentation, schema, and evidence drift as a separate correction class at just under 17 percent.

Conclusion

The provider guidance from OpenAI, Anthropic, and Google supports the direction: planning before implementation, tests tied to claims, diff review, CI, and a route from findings into correction commits. My gate process supplies the concrete design. The corrections from my own branch are a small sample, but every class in them is a finding from the audit model. At the same time, the high intervention rate shows that a substantial share of implementations was not immediately correct without review. The process works when it is run as an evidence-based refutation process. A green test suite by itself does not prove that the product path calls the new code. The audit has to provide that evidence. Whether the pattern holds across more projects would require a larger study.

Sources

• More Code, Less Reuse: AI-generated pull requests, arXiv 2601.21276 (2026)
• OpenAI: Codex Best Practices, https://developers.openai.com/codex/learn/best-practices
• OpenAI: Codex Review, https://developers.openai.com/codex/app/review
• OpenAI: Codex Subagents, https://developers.openai.com/codex/concepts/subagents
• Anthropic: Claude Code Best Practices, https://code.claude.com/docs/en/best-practices
• Anthropic: Claude Code Common Workflows, https://code.claude.com/docs/en/common-workflows
• Google: run-gemini-cli GitHub Action, https://github.com/google-github-actions/run-gemini-cli
• Google: Gemini CLI PR Review Workflow, https://github.com/google-github-actions/run-gemini-cli/tree/main/examples/workflows/pr-review
• Google Gemini CLI: Issue and PR Automation, https://github.com/google-gemini/gemini-cli/blob/main/docs/issue-and-pr-automation.md
• Google Gemini CLI: Checkpointing, https://github.com/google-gemini/gemini-cli/blob/main/docs/cli/checkpointing.md

In this pipeline, development no longer writes the code. This claim is uncomfortable, and I still hold it to be correct. There is no IDE anymore in the old sense, no integrated development environment in which a person opens file after file and types line after line. The developer no longer sees the individual source code as it comes into being, because the agent writes it. This sounds like a loss of control. The work just moves to a level at which the hard decisions have always been made anyway. Whoever types code line by line spends a large part of their time below that level.

Why this role in particular puts the thesis to the test

Business analysis and UX have long worked with language as their main material. An analyst who labels a source inventory, or a designer who describes an error state in business language, has never seen a compiler error message. For these roles, the agent speeds up an activity that consists essentially of reading, comparing, and formulating. Development is different. It has a craft that has built up its own world of tools over decades: editors with syntax highlighting, debuggers, refactoring keys, autocompletion. This world of tools was the pride of the profession. When I claim that it largely disappears in an agentic pipeline, I touch the self-image of many colleagues. That calls for a justification that goes beyond a slogan.

Developer and architect have become one role

Behind this role stands an organizational decision. In the classic setup, architecture and development are separate. There is someone, often with the title software or solution architect, who decides in advance how a system is built, which components talk to each other, which patterns apply. Alongside that is development, which translates these specifications into code. This separation had an economic reason. Architecture decisions are expensive if you correct them late, and typing code was slow enough that an upstream planning phase paid off.

The agent shifts this balance. When a first runnable implementation appears in minutes instead of days, the sharp separation between the person who designs the architecture and the person who implements it becomes artificial. The architect has to check against the grown legacy system anyway, the brownfield, to see whether her idea holds there. That check now runs through the same agent that later builds. It no longer makes sense to outsource the architecture knowledge into a separate upstream phase and a separate person when the feedback between design and reality becomes this tight. In the pipeline, developer and architect are therefore one connected role. The architecture review is its own station within the same role that also builds.

This station has its fixed place in the chain. Roughly, it runs like this: from the principles of the project, a kind of technical constitution, and the functional specification, open points are clarified first. After that follows the architecture review, in which it is decided how the change fits into the grown system. Only then follow the plan, the tasks, and the actual implementation, before the quality gate takes hold. The architecture thus sits before the code, but in the same hand that is responsible for the code.

The capability a model does not have

What the developer still contributes when the agent writes the code is the architectural judgment and the feel for the concrete brownfield. That is not a vague talent one invokes when the arguments run out.

An experienced architect recognizes that a change in one place triggers something three places further on. It is not in the code. She recognizes it because she has seen a comparable system fail at exactly this spot before. A storage logic, say, touches the same record in three places and at one of them assumes something the other two do not know. In the source code, that is not marked as a danger. It is a danger because a human knows the history of the system and knows where it once hurt. No repository holds this experiential knowledge. It lives in the heads of the people who have maintained the system.

The agent has something else. It has training patterns, that is, an average across very many systems that it saw during training. This makes it strong at everything common. At what is particular to this one system it grows uncertain, and in the brownfield what is particular almost always counts. Examples are column names that preserve an old business vocabulary, or process logic that implements some requirement from ten years ago whose reason is no longer documented. Add to that tacit conventions the code obeys without anyone recording why. A model cannot derive such peculiarities from the average of the world. Someone who knows the system from their own work has to know them.

That makes reviewing behavior, diff, and tests the new central work of development, and I consider this shift the core of the role. Diff means the difference between the code before and after a change, that is, exactly what the agent touched. This review demands more judgment than typing. Whoever types a function themselves thinks about syntax, about the right loop and the variable name. Whoever reviews a diff that an agent produced has to answer instead whether this change does the right thing in the grown system. Does it take effect in the three places it has to affect, and in none that it must not affect? That is the harder question, and it used to be buried beneath the typing.

How the agent takes over the work: the feasibility check

Development’s first concrete task in the pipeline is the feasibility check. It is a reading task. In this phase the agent changes not a single line of code. Instead it reads the grown system. To do so it searches the source code, reads the version history via git log and git blame, and evaluates Architecture Decision Records, that is, documented architecture decisions. It also reads runbooks, the operating instructions for live operation, and examines the database schemas. One of the important principles of the pipeline is that the agent may read freely but needs a release for every change. This way, exploring the system stays possible at any time, while an intervention in the code requires a conscious decision.

What this phase actually produces is a statement about whether the change is feasible, not feasible, or feasible only under certain conditions. With this statement comes a list of the affected components. The check also records which assumptions of the functional ticket have been confirmed against the brownfield and which have been disproven. An example: the ticket from business analysis assumes that a certain field is always filled. The feasibility check, however, finds against the real data that in many legacy records it is not filled at all. That is a disproven assumption, and it belongs documented before anyone writes code. Added to this are questions back to the business side and a short solution sketch. The sketch indicates a direction and does not yet contain an implementation.

The questions do not go into the chat. They go back into the functional ticket, to where business analysis sees them and can answer them. They are also classified. A question is either blocking, in which case implementation cannot begin without an answer. Or it concerns something that could, if need be, be cut out of the scope, in which case it does not hold up the ticket. Or it is purely a matter of implementation, in which case development can decide it itself without bothering the business side. This classification prevents every technical detail question from being blown up into a question back to the business side. Conversely, it also prevents a blocking gap from being swept under the table.

From ticket to task: the technical slice

When feasibility is clarified, development slices technical tickets. Here a rule applies that demands a lot of discipline in practice: a ticket comprises exactly one change with its own acceptance test.

The order of the tickets follows the natural buildup of a change, which grows from the bottom up through the layers of a system. First the data model, because everything else builds on it. On top of it the business logic that works with this data. Then the API, that is, the interface through which other systems or the user interface access it, and finally the user interface itself. After that come the tests, insofar as they do not arise per ticket anyway, and last the documentation. This order ensures that every ticket builds on a foundation that already stands.

Every technical ticket contains a back-reference. It names the acceptance criteria of the functional ticket that it implements, and the UX states that it serves. In practice I write these references as short tags, such as a reference to a particular acceptance criterion or a particular UX state. The point behind this is traceability. When someone later asks why this one code change exists, the reference leads from the technical ticket to the functional criterion and from there to the original requirement. This chain is valuable in the brownfield, because legacy code notoriously loses its justification. After three years no one knows anymore why a certain check exists, and the version history alone is often not enough if the commit message back then was thin.

As a rough heuristic for the size of a technical ticket I take one to three hours of work. What is meant is the slicing of a task. The number is not a billable time budget. A ticket that can be implemented and tested in one to three hours has a size that a human can still keep in their head when reviewing the diff. A ticket that comprises a whole day produces a diff that is too large to review responsibly. So the small slice has a solid reason: without it, a clean review is hardly possible anymore at the end.

The cascade of plan, epic, and tickets

Above the individual tickets lies a cascade that leads from the first research to the autonomous implementation loop.

At the beginning stands the research, and it is a reading task. Here the same thing happens as in the feasibility check: the agent reads the legacy code without changing anything. On this builds the epic. It describes the goal of the entire change, the scope, and the overarching acceptance conditions, and forms the bracket under which the individual tickets hang. From the epic the tickets are then sliced, small and testable, according to the rules I just described. The plan subsequently sets the order of the work, plus the quality gates that have to be passed along the way. It also contains the Definition of Done, that is, the condition under which the work counts as finished. Last runs the loop in which the agent actually works through the plan.

I steer this loop with a single instruction, which I reproduce here verbatim, because its exact wording is important:

Continue with the next step according to the implementation plan. Stop only when the plan is finished. Implement the task in a structured and careful manner. Be precise. Adhere to the quality requirements (lint, tests) absolutely. Commit often and early. Document your progress in the implementation plan, the epic, and the tickets.

This one instruction allows long autonomous work without constant micromanagement, because its guardrails lie outside the agent. It refers to a plan that a human reviewed beforehand and to quality requirements that are enforced by machine. Those are lint and tests that fail hard when something is wrong. Its stop criterion is clear, namely the end of the plan. Without the reviewed plan and without the hard gates, the same instruction would be dangerous. It could send the agent twenty steps in the wrong direction before anyone notices that the direction was wrong. With plan and gates it becomes a tool for the autonomous work that makes the speed gain pay off in the first place.

Committing often and early connects the loop with the principle that I consider the most important of the whole pipeline: that state lives in artifacts and not in the chat.

Git as the source of truth

An agent has no memory beyond the session, and even within a session a long chat history loses its effect. What is in the repository, on the other hand, endures. In this role, version control does not just store finished code. It is the actual source of truth about the state of the work.

Three useful traces arise here at no extra cost. The commit message records the why of a change, provided one takes it seriously and does not fob it off with one word. Via git blame the origin of every line can be traced: when it came into being, by whom, and with what justification. The git diff command shows the exact difference between two points in the history. Unlike a separately maintained notes file that lies around in every project and goes stale at the next code change, this trace hangs on the code itself and stays current. Whoever reads the history of the affected file before a change guesses less.

What the role produces

At the end of its work, development leaves behind a series of artifacts. I name them individually, because the quality gate later attaches to exactly these artifacts.

The first is the feasibility statement, recorded in its own block for development notes within the functional ticket, where business analysis finds it. Then come the epic and the technical tickets, each of which contains a test strategy. This test strategy names which tests arise at which level: unit tests for individual functions, integration tests for the interplay of several parts, end-to-end tests for the continuous flow from the user’s perspective. It also names something that is often missing and that I consider especially important: the anti-test. By this I mean that the strategy sets out what is intentionally not tested and why. A test strategy that only says what is tested leaves open whether a gap was intended or forgotten. When the strategy names its gaps explicitly, a reviewer can retrace the decision behind it and come back to it during testing.

Added to this are the implementation plan and the pull requests. A pull request is the unit in which a completed change is submitted for review. For it, a condition applies that I care about: the pull request must be understandable on its own, from the diff, the tests, and the plan. No one should have to know the chat history with the agent. If a reviewer can only understand the pull request by retracing what was discussed in the chat between human and agent, then the handover has failed. That chat is ephemeral, and no one archives it. The last artifact is the progress log. It records which step was completed when and with what result.

The quality gate of this role

Development has two check points in the pipeline, and they lie at different places in the work.

The plan review stands before the first code. It checks the order of the work, the test strategy, the risks, and the affected files. This review is the most effective point of the whole role, because an error is cheapest here. As long as it is in the plan, a correction in the text suffices. Once the agent has built it into the code over twenty steps, the code correction is joined by the search for the spot at which the direction tipped. Reviewing the plan beforehand costs little and spares this expensive rework.

The code review on the pull request comes after that. Here a human reviews the behavior, the diff, and the tests. The mandatory artifact of this role consists of two parts: the reviewed plan and the pull request with green checks. Green checks means that all automatic checks have passed. Without these two artifacts there is no handover to the next role.

Beneath the human check points lies a machine layer that no one has to go through by hand. Deterministic checks run before the expensive model stations even start up. The static security analysis examines the code for known vulnerability patterns. Alongside it, the license and IP check verifies whether the produced code pulls in foreign components under an incompatible license. Detection of prompt injection catches smuggled-in instructions that could manipulate a model out of a file it has read. These checks run as scripts with a traceable result, and they run first on purpose, so that no model spends computing time on plainly faulty material. They handle what can be determined unambiguously and so relieve the human gates, which deal with questions of judgment.

The economic reversal: quality that is suddenly affordable

About this role, one point occupies me most: it reverses a calculation that ran against quality for decades.

Code quality always had a price in developer hours. One hundred percent coverage by unit tests was a theoretical ideal in most projects: every line of the code executed at least once by an automatic test. That cost so much time that under deadline pressure it was the first thing to be cut. Lint in strict mode checks the code against all style rules and suspect patterns without any rule being waved through as an exception. That was just as expensive, because every rule that triggers costs a human who fixes it. The same goes for consistent formatting and gap-free static checking. These disciplines were the first to fall victim to deadline pressure, because their benefit lay in the future and their cost in the present.

The agent shifts this calculation, because adhering to these disciplines costs it almost nothing. It no longer costs an additional developer hour to cover one hundred percent of the lines with tests when an agent writes the tests. Satisfying every lint rule, too, costs no human effort anymore when the agent adjusts the code until no rule triggers. Formatting and static checking are no longer up for debate, because the agent keeps to them in the loop without a human having to spend time on it. With that, the calculation reverses: disciplines that used to fall victim to deadline pressure because they were expensive are now the standard case, because they have become cheap. An ideal that one had to afford has turned into the path of least resistance.

One hundred percent test coverage, however, is not the same as correctness. Coverage only measures whether a test executed a line. It says nothing about whether the test checks the right thing. An agent can write tests that touch every line and still assert nothing meaningful. They then only confirm what the code does anyway and do not check whether it does the right thing. A suite of such tests yields a green checkmark that is worthless. Full coverage, then, only shifts the problem. It ensures that the cheap errors surface by machine, and leaves the expensive errors, which lie in the understanding of the requirement, open. That is why the Test role is needed after development, and why it has to bring a different perspective from the one that produced the code and its tests.

The economic reversal is still valuable, because it shifts the floor. Previously the question was whether a project could even afford decent test coverage and strict static checking. This question no longer arises, because the answer is yes and it costs nothing. The energy that used to be needed to reach this floor is now free for the harder work. That includes reviewing whether the tests check the right thing, and judging whether the change does the right thing in the brownfield. The agent handles the base level, which leaves the humans time for these questions.

How the compliance rules take effect in this role

A series of compliance requirements runs through all roles of the pipeline. In development, some of them become especially concrete, because here, for the first time, code arises that can have real effect.

The static security analysis and the license and IP check attach directly to the produced code. They belong to the machine layer beneath the code review and do not run only later. The security test catches known vulnerability patterns before they get into a pull request. The license check prevents foreign code with a license incompatible with the project from slipping in unnoticed. That is a form of IP contamination that is a real risk in agentic work, because a model can reproduce code from its training whose origin no one knows.

Every AI-produced change gets a marking. In development this marking sits in the commit comment. It names the operation that was carried out, the responsible person, and the version of the model or the skill definition that produced the code. A skill definition here is the reusable instruction with which a certain task is given to the agent. This marking is the transparency requirement of the EU AI Act, Regulation (EU) 2024/1689, in daily practice. This way, the version history later shows whether a line of code was written by hand or proposed by a model and only confirmed.

The agent works in an isolated sandbox, that is, a sealed-off environment, and not on the developer’s laptop. It has access neither to the file system of the human who steers it nor to production environments. Its effect is limited to development, test, and integration. An operation on a production environment is not only forbidden by convention but actively blocked by a technical lock. This separation is what makes long autonomous work responsible at all. An agent that can do nothing to production may have more freedom in the development environment.

At the pull request stands the human gate. A named, responsible person releases the change or improves it, and the AI support does not diminish this responsibility. Added to this is the second and third opinion from the fifth principle of the series. In the code review a second model checks the patch, one that did not write it itself. A model that checks its own code clings to its own logic and overlooks the same errors it made while writing. A second model with different training data and different blind spots brings an independent perspective. When both agree, the finding is more reliable, and where they diverge, a human has to look more closely. The same separation also hardens against prompt injection, because a smuggled-in attack would have to outwit two different models in two different ways.

A word to those who would rather not hear it

I began this post with a thesis that touches the self-image of many developers, and I do not want to soften it at the end. The profession, as many learned it, ran largely from the mind through the keyboard into the editor. Competencies one was allowed to be proud of are now losing their value: mastery of the editor, speed at writing, formalized and conceptual thinking, knowing a programming language by heart down to its corners. That is a real loss for those who invested a lot in these skills, and I think nothing of selling it as a pure gain.

What takes its place demands more experience than typing. It means designing a plan that holds, slicing the tickets so that the review works, and judging whether a diff does the right thing in the grown system. The newcomer to the profession, who used to grow into the subject by writing code, faces a harder path. The entry through the craft falls away, and the entry through judgment presupposes an experience that she cannot yet have. What this path will look like, I do not know. The study by Christopher Potts and Moritz Sudhof of Stanford University, “A paradox of AI fluency” (arXiv:2604.25905), gives a hint that practice makes a difference. Practiced users spot errors in the dialogue much more often than unpracticed ones. How one acquires this practice when the traditional path of practice disappears is an open question, though, and I have no ready answer to it.

Outlook

In this pipeline, development delivers a pull request with green checks and full test coverage. That does not end the work. Coverage does not guarantee correctness: an agent can write tests that touch everything and check nothing meaningful. The hard judgment about the rightness of a change stays with the human. This is where the next role picks up. The following post in the series covers the Test role. It looks at the same feature from a different perspective than the role that built it. Its work starts where a green checkmark from development is not yet a guarantee of correct behavior.

For simplicity I call the role business analysis, or BA for short, well aware that the labels vary in practice. In some places it is called requirements engineering. Elsewhere the task falls to a product owner, a subject matter expert or an analyst at the boundary between the business side and the technical side. Who fills the role is secondary for this text. The subject here is the activity: clarifying what a system should do and why, in such a way that a developer can build something from it without having to guess.

Why this role goes first

Anyone working in the brownfield is dealing with a system that has grown over years and in which much was never spelled out. The greatest danger there is the misunderstood requirement, more rarely the difficult technology. A team can laboriously optimize an algorithm and cleanly rebuild a database. In the end it turns out that all the work solved a problem nobody had, while the actual problem remained untouched. That kind of failure is more common in a grown system than the spectacular technical collapse, and it is quieter, because it only shows up late.

Business analysis is the role meant to prevent this failure. It stands between what the business side wants and what the technical side builds, and it closes the gap between the two before anyone writes code. In agentic work this task is the key, because an agent can formulate a plausible requirement in minutes that reads as if a human had thought it through. A plausible requirement can still be wrong, and in a legacy system that decides success.

The old target picture

The task described here is not a new profession. Good business analysis was always meant to do exactly this. It gathers domain knowledge, checks statements, makes contradictions visible, formulates requirements so they can be tested, and prepares a ticket in such a way that development can work without guessing. In the old requirement specifications of waterfall projects this goal was often clearly stated. In daily project work it was rarely reached. The reason often lay in the conditions of the projects. Under project pressure, research was cut short and huge data sets were searched only in excerpts. Business context had to be assembled from old tickets, scattered documents, grown special cases and conversations. Anyone who still wanted clean documentation needed time that was rarely planned for. That is how incomplete requirements arose, even though the people involved knew what a good requirement should look like.

With agents, the economics of this work change. The standard stays the same, but the legwork becomes cheap: finding sources, comparing variants, preparing glossaries, normalizing acceptance criteria, generating example and test data, tabulating open questions. Completeness no longer needs days of documentation work. It comes out of many small, checked agent steps. That gives the BA back the part of the work that often fell short under project pressure: establishing business context, asking good questions, thinking through use cases and drawing the boundaries of a feature cleanly. She becomes the business orchestrator of a waterfall ticket. She operates the agent, reviews its suggestions, decides about evidence and keeps the ticket complete enough for development, UX and test to continue from it. This ties in with what I described in the post on the return to waterfall: the machine takes the search and writing load off the BA, and more room remains for judgment, creativity and responsibility.

The human ability: judgment about sources

What a business analyst brings and a language model does not have is the judgment about what a source is worth. An experienced BA reads an old ticket and recognizes that the requirement described in it was long since overtaken by a later decision. With a Confluence page she notices that it describes a wishful picture that was never implemented in that form. In conversation with a case worker she hears between the lines that the official process description and actual practice diverge. She distinguishes what is documented from what only sounds plausible, and from what contradicts itself across two sources.

A language model cannot do this reliably, and that lies in its nature. A model approximates language. It produces text that fits what it has seen in the prompt, and a documented fact reads in it just as convincingly as a skillfully worded guess. Both consist of the same grammatical building blocks and sound equally certain. Whether a statement is true in the world is something the model cannot judge. It sees only whether the statement is linguistically probable. In the brownfield context, where evidence is rare and assumptions are common, this blindness to the truth value is the greatest risk.

Perspective adds to this. A good BA thinks the result through from the point of view of those who work with the system, such as the case worker or the customer. She asks what this person wants to achieve and how they will know that they have achieved it. A model that is not guided thinks from the system’s point of view, because the system view dominates in the training material. It describes what a function validates. What a person should be able to do falls away. This shift of perspective is subtle and at the same time consequential, because it hands responsibility for success from the business side to the technical side, where it does not belong.

A further difference shows in conversations with stakeholders. A BA listens to a subject matter expert and recognizes when behind a single stated requirement there are in truth two that only happened to be named together. Behind a technically phrased request sometimes sits an entirely different business need, and she notices that too. She catches when a requirement silently assumes that the grown system behaves in a particular way, and that this assumption must be checked. This breaking down and asking back happens in conversation, in the back and forth between people, and it is the point where an agent can least replace a human.

Four core principles of the role

Every output of the role can be measured against four principles.

Deliver the documented truth. Business analysis delivers a documented truth and leaves the solution to development. It describes what should happen and why, for whom and with what benefit. How it is built is decided by development, and only after an examination of the brownfield. That examination is one the BA cannot perform at all, because she does not know the code in depth and cannot assess it either. A BA who prescribes a technical solution takes away from development the judgment that can only be made there, and she usually does so without being able to survey the consequences.

Every statement is labeled. In a BA output there is no unlabeled running text. Every claim carries one of four marks: a [Fact] for something verifiable with a named source, an [Assumption] for something plausible without evidence, a [Question] for something open that still has to be clarified, and a [Contradiction] for a point where two sources do not agree with each other. Smooth, unmarked text is a mere summary and is not enough. A summary blurs exactly the uncertainties and contradictions that make up the actual risk.

The ticket is binding. The chat history is fleeting, and whatever is worked out in an agent session but does not make it into the ticket is lost for the rest of the process. That is the artifact contract from the opening post, here in its concrete form for this role. A developer who implements the feature works from the ticket. Whatever is missing from it is missing for her entirely, because she does not see the course of the discussion. The post on spec drift shows how far ticket and code diverge over the course of a project.

E2E content exists before the handover. A ticket is ready for development only when it contains the business material for the later End-to-End test. This includes scenarios, placeholder data, expected message texts, start and target states, role changes and relevant handovers to neighboring systems. These are not executable tests yet. The BA delivers the content from which the test role later creates browser and workflow E2E specifications.

How the agent supports the role

In this role the agent is a working apparatus for the BA. It researches, sorts, drafts and produces the content from which development and test later work, and in doing so takes the mechanical part off the BA. It decides nothing. The BA steers it through precise assignments, reviews the results and discards what only sounds convincing. Whoever fills this role still needs domain knowledge. A new craft skill comes on top: guiding the agent so that the end product is a complete, testable ticket and not a conversation log.

The agent works in the early phase that the pipeline calls Discovery, that is, the preliminary investigation. In it the team works out what the matter is about before it determines what is to be done. The agent’s first step is to build a source inventory.

A source inventory is a list of all the places that say something about the area concerned. Each place gets an identifier, a title and a date in the format YYYY-MM-DD. The date format is a rule with consequences: vague labels like “current” or “recently updated” are not allowed. In the system a Confluence page abandoned three years ago looks just as fresh as one that was just revised. Only the concrete date makes the difference visible, the difference that in a legacy system often decides whether a statement still holds or is long outdated.

For this the agent reads many sources: tickets from the tracking system, Confluence pages, Architecture Decision Records, that is, the documented architecture decisions, and Runbooks that describe how a system is operated. It also reads old documentation and the code itself. This breadth is its strength. A human needs days to read through scattered legacy documentation. An agent builds an inventory in a short time that a BA can then review. Here the tool amplifies the role, because it speeds up tedious but necessary legwork.

Assessing is not the agent’s job. Every finding it draws from the sources must be labeled by the BA. It can suggest: this statement relies on ticket TICKET-417 from 2024-03-12, so I propose [Fact]. Setting the mark, though, is up to the human. Only the human can judge whether the ticket still holds or has been overtaken, and whether it perhaps only documented a wish that never became reality. The labeling is the analytical core contribution of the role, and it stays with the human.

One peculiarity of this role runs counter to what one expects of a language model: the smooth, easily readable summary is expressly not the goal here. A model is trained to deliver a contradiction-free, rounded version, which is harmful in Discovery. A rounded summary smooths over the very contradictions in which the risk sits. When two sources contradict each other, the output should show the contradiction, with both sides and the mark [Contradiction]. It should not invent a third, harmonious formulation that pretends there is no problem. Discovery is about laying contradictions open.

After that, the agent becomes a production tool for the E2E content. It can propose business scenarios from acceptance criteria, generate Mustermann data, check expected messages for consistency and place process variants side by side. The BA decides which of these suggestions match the business reality. The agent provides volume and variants, and the BA turns them into binding content for the ticket.

The interplay with UX in the feature workshop

Business analysis works closely with UX, the role that describes how a feature feels for the users. In the pipeline both meet in a feature workshop. The workshop has a fixed form that prevents settling prematurely on a formulation that only holds on the surface.

The workshop begins with BA and UX describing the feature independently in their own words. Each side notes the goal, the user, the benefit, the boundaries and the open questions. This happens separately at first, because the differences between the two descriptions are the interesting points. Where BA and UX read the same feature differently, there is an ambiguity that has to be clarified before anyone goes on. From the two descriptions a shared team template then takes shape, an agreed picture of the feature that both roles have settled on.

A core piece of this shared work is the definition of the UX states. A feature has several states beyond the one course in which everything goes well, and all of them have to be described. The pipeline names them explicitly. [UX-SUCCESS] is the case in which the task succeeds. [UX-EMPTY] means the state in which there is no data yet, an empty list, a fresh record. [UX-ERROR] covers what happens when something goes wrong. [UX-LOADING] describes the wait for a response. [UX-EDGE] stands for the edge cases that occur rarely but then determine the behavior.

The error case is forgotten most often in practice and at the same time tells the most. [UX-ERROR] describes a business message in business language. When a case worker wants to save a claim without a required piece of information, she should read a message that explains in her language what is missing and why. A technical code like “Validation error in field 0x2F” is no such message. This requirement is a business one, and so it belongs in the BA part of the description. How the message is technically triggered is a matter for development. Its wording and its language belong to the business side.

Only after the team has reviewed this shared description and found it coherent are the tickets uploaded to the tracking system. Not every idea is immediately created as a ticket, because a prematurely created ticket suggests a commitment that has not been reached yet. Only the team check turns a draft into a ticket that the next role can take seriously.

What the role produces as artifacts

The main artifact of business analysis is the BA ticket. It has a fixed structure, and this structure is the form in which a documented truth can be handed over. Each section has a practical purpose, and the order of the sections has its purpose too.

The ticket begins with a header. It contains the outcome, that is, the result from the user’s point of view, and this outcome begins with a persona or an action: “The case worker can …”. This formulation is a test. If the result can be phrased so that an acting person stands at the beginning and a capability follows, then it is thought through from the user’s point of view. If the sentence begins with “The system …” instead, it is thought through from the system’s point of view, and that is a warning sign. Alongside the outcome the header contains the persona, the trigger, that is, what sets off the feature, the stakeholders and the source date.

After the header comes the body, in a fixed order of sections. First the problem or the need: why does this feature exist at all, what business need lies behind it? Then the acceptance criteria, formulated as Given/When/Then in business language and numbered individually, [AC-1], [AC-2], and so on. Given/When/Then is a simple scheme: given a starting situation, when an action occurs, then an expected result. Business language means that business terms appear there and no technical ones. Numbered individually, each criterion is testable and referenceable. A nested criterion that demands three things at once is in truth three criteria and should be split up.

After that come the UX states from the workshop, then the non-goals. The non-goals are a section of their own, often underestimated. They actively exclude common assumptions. If a feature introduces a new mandatory field, the obvious assumption is that the existing data stock must also get this field. Such an assumption belongs under the non-goals, expressly negated, so that no one silently takes it as given. What is not excluded gets read into the text by someone in case of doubt, and in the brownfield such tacit assumptions are, in my experience, the most expensive.

Next comes the source inventory, that is, the list of sources with their dates. After it comes the statement list, in which every claim stands with its mark and, for a [Fact], the source is named. Then the open questions, as a table. Each question gets an identifier [Q-1], a wording of the question, an owner, that is, a named responsible person who is to answer it, and an answer deadline. A last column records what happens if the question remains unanswered. This column is the most important, because it weights the question. An open question that, left unanswered, blocks the entire feature weighs more than one that only leaves a detail open, and the table makes this difference visible. The E2E content, the stakeholders and the sign-off form the conclusion.

Several mandatory artifacts belong to the BA ticket: a UX sketch that captures the workshop finding visually, and a snippet, that is, a small concrete example of the expected behavior. A glossary defines the business terms, and in the brownfield context it has a further column for the equivalent in the neighboring system. Where a term lives across system boundaries, the column should state what it is called in the other system. If no equivalent is known, a dash and an open question go there. Last, the E2E content belongs to it: example data with placeholders, business scenarios, expected messages and the states the later test is supposed to demonstrate. Real people, companies or case numbers do not appear in it.

This E2E content is a precondition for the ticket going to development. It is not yet a test plan and not a test script. It describes the business path that a later test must demonstrate. If this material is missing, development has to guess during planning which states and transitions should be visibly checked in the end. Then the ticket is too early for implementation.

Brownfield specifics: what is different in the brownfield

A business analysis in a grown system differs from one on a greenfield in the very kind of work. Three differences are so important that the pipeline anchors them as fixed rules.

Protection of existing data over forced migration comes first. A new feature should not automatically migrate or validate existing data. New obligations apply only to records that are newly created or actively edited, and the old stock stays readable and storable. This separation is a business decision with great effect. Without it, a data migration slips unspoken into a feature ticket. Yet it is an undertaking of its own, with its own stakeholders, its own risk profile and its own schedule. Treated as a side matter, it throws the scope of the ticket into disorder, and the feature ends up taking far longer than planned. That is why the BA makes this separation explicit in the non-goals.

The second peculiarity concerns the mapping of cross-system terms, already touched on with the glossary. In grown systems a system almost always shares data with others, and a term like partner, case or role can mean something slightly different in each of them. As long as this relationship is named nowhere, it remains a hidden source of error. The BA must map the cross-system terms explicitly, even when the current feature has no cross-system effect at all. The next change might have one, and then the ambiguity is already in the way.

Strictest of all is the rule that no technical specification may appear in the BA ticket, neither architecture nor data model, no column names and no choice of a framework. After the handover, development checks feasibility against the brownfield. It looks at the points in the grown code where the same logic hangs, at what can be reused and at what a change in one place triggers three places further on. If the BA ticket already prescribes a technical solution, it makes this check worthless. Development then only works through a specification that someone set without knowledge of the code. In my experience, the technical specification in the business ticket is the most common reason that development sends a ticket back. The BA therefore describes what should happen and why, and leaves the question of how it is implemented to development.

The quality gate: Definition of Ready

Between business analysis and development stands a checkpoint, called Definition of Ready in the pipeline, DoR for short, or also Ready for Dev. It is the point at which a human decides whether the ticket is ready to go into implementation. The gate has a checklist, and the check is left to a human on purpose, because it concerns questions of judgment that cannot be answered mechanically.

The checklist asks:

• whether the outcome is an unambiguous sentence that begins with a persona or an action and can be tested from the user’s point of view
• whether the acceptance criteria are numbered individually and formulated in business language, without technical terms
• whether the UX states are present, including the error case in business language
• whether the source inventory carries dates and contains no “current”
• whether the statements are labeled, each with its mark, and the source is named for every [Fact]
• whether the E2E content is complete, that is, business scenarios, Mustermann data, expected messages, start and target states and relevant handovers to neighboring systems
• whether the mandatory artifacts are in the repository, that is, UX sketch, snippet, glossary with the neighboring-system column and E2E content
• whether the ticket is free of a technical solution.

The mandatory artifact of this gate is twofold: the labeled source inventory and the finished ticket with E2E content. With only one of them, the gate’s question cannot be answered. The source inventory shows that the knowledge base is honest, with its gaps and contradictions, and the ticket demonstrates that this base has become a testable requirement.

Hardness is the gate’s most important property. If a point is missing, the ticket goes back. It is not silently processed further and not waved through with a verbal “we’ll clarify that later”, nor on the assumption that development will fill the gap. Whoever carries on silently like that drags errors deep into the chain, where they become expensive. A ticket that does not pass the Definition of Ready is not finished and does not go into implementation. This strictness can feel annoying in the moment. It keeps the cost of errors low, though, because the correction still happens in the text, while the system is not yet running.

How the compliance rules work in this role

The compliance requirements that the opening post described as the sixth principle are built into the way business analysis works. In this role they act at four concrete points.

Data minimization. In software development this means that personal data do not belong there. The example data in the BA ticket are therefore exclusively placeholders, Mustermann GmbH, Erika Mustermann, Max Mustermann. A BA ticket ends up at some point in a training, a presentation, an export or a shared workspace. Whoever works with placeholders from the start does not have to filter out real data afterward, a cleanup job in which not all places are caught anyway. The project context the agent works with contains only what the business needs, as plain text, without real data.

Sources with a date. That every source has a date and that there is no “current” is a matter of care and equally of traceability. Whoever later wants to check on what knowledge base a requirement arose finds dated evidence in the source inventory. That is the provenance an audit trail needs, in the simplest conceivable form.

Marking the AI-generated content. The EU AI Act, Regulation 2024/1689, requires transparency about the use of AI. In this role that means that every piece produced by the agent is marked as AI-generated. A note refers to the operation, the responsible person and the version of the model or the instruction. When the agent has suggested a mark for a statement, it is traceable that the suggestion came from a model and that a human confirmed or corrected it. Both stay in the trail: the agent’s marked suggestion and the mark that the human set.

The DoR gate as human sign-off. The principle of the entire pipeline is that the agent suggests and the human decides, and human oversight applies without exception. The DoR gate is the place where this oversight becomes concrete in business analysis. A named responsible person confirms the ticket before it moves on, and in doing so vouches for its content. That the agent does the groundwork does not lessen this responsibility. A ticket that a model suggested and a human only nodded through, without checking it, violates this principle, even if formally a sign-off exists. It counts only after a real check. How far such a sign-off reaches is set out in the limits of human sign-off.

A worked example

A short example with placeholders shows how the rules work together. A business department of Mustermann GmbH reports a need: when recording a case, a certain piece of information that was optional until now should in future be required. With this first sentence a premature approach would immediately create a ticket that demands a new mandatory field.

Business analysis begins differently. The agent builds the source inventory. It finds an old ticket from 2023 that once introduced the information as optional, and a Confluence page from 2024 that describes a process. Added to that are hints in the code on how the information is stored today. Now the BA labels. That the information is optional today, documented by ticket and code, is a [Fact]. A second [Fact] comes from the conversation: the business department demands it for all future cases. Whether the existing cases should get it too, nobody has said. That remains an [Assumption] to be checked. The Confluence page in turn describes a process that does not match the actual practice of case work, a [Contradiction].

From this labeled base the ticket takes shape. The outcome: “The case worker can close a case only if the information is present, and otherwise receives a business message that explains which information is missing.” An acceptance criterion [AC-1] describes the success case, [AC-2] the error case in business language. In the UX state [UX-ERROR], the message is formulated in business language. Under the non-goals it expressly states: the data stock is not changed, existing cases without the information remain valid and storable, a migration is not part of this ticket. The open question [Q-1] in the table: should existing cases be retrofitted at their next editing, owner the business department, with a deadline and the remark that a yes triggers a migration undertaking of its own. In the statement list, the BA records the [Contradiction] between the Confluence page and practice, so that development does not overwrite it unawares.

For the E2E test, the BA also produces the business content. Erika Mustermann creates a case for Mustermann GmbH. The required information is missing. On closing, the message appears in business language, the case remains in an editable state, and the handover to the neighboring system does not take place in this state. These details are meant to be concrete, because the later test needs visible content and expected states. Development can see before implementation which behavior must be demonstrated in the end.

What this ticket leaves out counts as much as what it contains. It contains no column name and no specification of whether the check happens in the frontend or the backend, nor any choice of a library. Development decides that after its feasibility check, once the ticket reaches it.

Outlook

The next post in the series turns to development. There it will become clear what becomes of a BA ticket that has passed the Definition of Ready: how development checks the brownfield against the requirement, how the what turns into a how, and how the small, individually implementable tickets come out of the Epic. It is also about how an agent can work across many steps without losing control. At the handover between business analysis and development it is decided whether the clean preliminary work holds, and that is why the DoR gate is as strict as it is.

The business analysis role can be tried out with a single exercise, even before the next post appears. Take the next requirement that lands on your desk and label every sentence in it. Which of it is a fact with a source, which an assumption, which an open question, which a contradiction? Usually the list of assumptions and questions grows longer than you would like, and that justifies the role. A requirement that fully resolves into documented facts is rare. Far more often a few assumptions you thought long settled show up during labeling.

Over the past months I have worked out an agentic engineering pipeline for the SDLC and tried it in workshops. It is an end-to-end way of working through the software development lifecycle with agents, tailored to exactly this setting. It describes how a team works with agents when it wants to bring a new feature into an existing system without damaging the brownfield. This text is the opening of a series and takes on the pipeline as a whole: why it looks the way it does, and which principles hold it up. The later pieces then go into the individual roles: business analysis, UX, development, testing, and operations.

Why brownfield is a problem of its own

In April 2026, Google Cloud’s DORA team published a report on the return on investment of AI-assisted software development. I discussed it in detail elsewhere (The ROI Depends on the Foundation). One figure from it is the most important for this topic. In greenfield, in new systems without legacy, the report puts the productivity effect of AI-assisted development tools at 35 to 40 percent. In brownfield, in a system that has grown over time, it is around ten percent. The factor between the two worlds is three to four. For organizations that earn their money maintaining existing software, that is the core statement on the economics.

A language model works only with the context it is given at the moment of the request, and that turns out differently in the two worlds. In greenfield it is small and manageable. In a legacy system it is large, scattered across many places, and a good part of it is written down nowhere. Part of this knowledge sits in a database whose column names come from an outdated domain vocabulary. A control flow encodes some long-forgotten regulatory requirement without noting it anywhere. The rest exists only in the head of the colleague who built the module years ago. An agent that does not have this context produces code that looks right and is wrong exactly where no one can see it from the outside.

To this economic finding comes a methodological one. Christopher Potts and Moritz Sudhof of Stanford University analyzed nearly 27,000 real AI dialogues in their paper “A paradox of AI fluency” (arXiv:2604.25905). Their most notable result concerns the visibility of errors. With practiced users, 59 percent of errors become visible in the dialogue itself, with inexperienced users only 12 percent. For inexperienced users, then, 88 percent of errors stay invisible in the dialogue. What happens to those errors afterwards, the study does not measure. For brownfield work, this finding alone is enough: where the dialogue triggers no correction, an error more easily travels into a production system. Letting agentic coding loose on a grown system without steering goes faster in the short term. The damage it does surfaces only weeks or months later.

The pipeline starts here. It is meant to make the invisible errors visible before they reach the grown system. That the agent types faster counts for less.

The pipeline in one picture

A feature moves through six phases, and at the handovers between roles there is a checkpoint each time.

First, the context is made known. In a legacy system that means pulling out of the existing code, the tickets, the old documentation, and the data what is known about the affected area. In this phase only reading happens, nothing is changed.

Then the team explores together with the agent. Before it thinks about a solution, it asks questions and looks for contradictions.

The results are recorded as Markdown files in the repository: what was found, what stayed open, and where sources contradicted each other.

Out of this understanding a structure emerges: an epic that describes the goal, and tickets that each cover one to three hours of work, testable and traceable.

Before any code, there is an implementation plan that fixes the order, the test strategy, the risks, and the affected files.

Last of all, the agent works through the plan, ticket by ticket, and documents its progress in the plan, the epic, and the tickets themselves.

The flow is familiar. Anyone who knows a classic development process will recognize the stations. What is new lies in the rules that hold between the phases.

Brownfield has to be made readable first

In greenfield you can start with phase two, because there is nothing you would have to read first. In brownfield the first phase is the most important and at the same time the one most often skipped. It is called context extraction, and it answers the question of what the system actually does today at the place where we want to change it.

The answer sits in several places, and a good pass reads them all. Code shows what happens, but not why. The why of a decision is in the version history. Tickets tell you which requirement once stood behind it, often in language that has since gone stale. The data itself records what the theory keeps quiet. A field which according to the documentation should always be filled is empty in a substantial share of the legacy records, because over the years the staff made do with free-text notes. A new mandatory field introduced without this finding leaves a large part of the data set failing the rule.

One special case of context extraction occurs in a legacy system almost every time and is almost never documented: terms that live in more than one system. A term like “partner” or “case” can mean something different in the system we are changing than in the neighboring system it exchanges data with. As long as this relationship is named nowhere, it is a risk for every migration and every coexistence of two systems. The pipeline therefore requires that a term which exists across system boundaries gets its counterpart in the other system recorded in the ticket’s glossary. That holds even when the current feature has no cross-system effect at all. Where no counterpart is known, there is a dash and a note as an open question. That turns an invisible assumption into a clarified fact or at least a visible gap. Most legacy systems, though, live with the silent ambiguity.

This first phase produces no solution yet, and it is not supposed to. Its artifact is a picture of the current state, evidenced and dated, with clearly marked gaps. Only on this picture can one responsibly decide what the new feature should do.

First principle: people stay in roles

The agent replaces the whole team, or so a common story promises: one person, one language model, and everything from requirement to deployment runs through the same chat. It sounds tempting because it is cheap, yet it does not hold up to a closer look.

Each role stands for its own view of the same feature. A business analyst judges differently than a developer, a tester looks at the same feature differently than a UX designer, and operations asks questions that no one in development asks. Through these differences a team finds the errors that a single perspective misses. Bundle all roles into one person and equip that person with an agent that can imitate every role, and you get only the imitation of those perspectives. The real friction between people who judge differently out of different experience does not arise that way.

In a grown system a second reason comes in. The constraints of a brownfield system are often experiential knowledge that is nowhere in the code. An experienced architect recognizes at first glance that a spot is delicate, because she has already seen, on a similar system, what can go wrong there. An agent does not have that experience. It works with training patterns, and those are an average of the world in which the specifics of this one system disappear. The overview of the grown whole, the feel for which change in one place sets off something three places further along, stays the human’s domain.

Beyond quality, there is the law. The EU AI Act, Regulation (EU) 2024/1689, provides for human oversight in particular for high-risk AI systems. For an agentic pipeline this does not imply a blanket legal duty to approve every single output. As a governance rule it still makes sense: every meaningful output needs human approval before it is accepted. That way, in the end, a person with a name stands behind a release. A chat log that no one can reconstruct does not.

The pipeline therefore has fixed roles, each with its own brief. Business analysis clarifies what is to be built and on what factual basis. UX describes how the feature feels for its users and which states it has to know. Development translates the domain structure into code, and testing checks whether the delivered behavior matches the promise. Operations decides whether and how a change may take effect on the real infrastructure. All of them use the same agent, only differently, with their own questions and their own boundaries.

How differently the roles look at the same feature shows in a concrete case. A mandatory entry is to be introduced, and each role puts its own question to it:

• Business analysis wants to know whether the domain plausibility is enforced in the right place and what happens to the existing data.
• UX asks how the caseworker notices that a mandatory entry is missing, and whether the message explains why in business language.
• Development looks for the places in the grown code where the same storage logic hangs, and works out which of them has to change with it.
• Testing cares about the negative case that the cheerful solution likes to forget: what happens when something is saved without the mandatory entry.
• Operations checks whether the change can be rolled back if the grown system behaves differently than expected.

Each of these questions knows a different way the feature can fail, and none is superfluous. The series treats each role on its own. The basic decision stays the same in every phase: each role belongs to a human, and the agent only strengthens it.

Second principle: every phase explores before it commits

In agentic work, the early jump to the solution is the biggest temptation. You type a task, the agent answers immediately with code, and it feels like progress. In brownfield this jump is dangerous, because the solution rests on an understanding that no one has checked.

The pipeline reverses the order, so that every phase begins with exploration. In business analysis, that means building a source inventory: which tickets, Confluence pages, architecture decision records, runbooks, and old documents say something about the affected area? Each source gets a status with a concrete date on which the statement held. Vague labels like “current” or “latest version” do not suffice. A Confluence page that no one has touched for three years looks just as fresh in the system as one from yesterday, and in a grown system that is a risk.

After the inventory comes the actual analytical work. Each statement is labeled: as an evidenced fact, as a plausible but unevidenced assumption, as an open question, or as a contradiction between two sources that both seem to be right. These labels keep visible the risky places that a smooth summary would blur.

This stance holds not only in analysis. Development reads its way into the code before it writes. Testing works through the acceptance criteria before it designs scenarios, and operations forms a hypothesis about the cause before it proposes a change. The reading phase is anchored in the tooling: search and read operations are freely allowed, writing and acting operations need approval. This gradient is intended, because reading should be possible at any time and a changing intervention has to remain a conscious, approved decision.

Exploration is also iterative. A phase rarely delivers a reliable picture on the first pass. You find a contradiction, go back into the sources, clear it up, and find a new open question along the way. This back and forth is part of it, because that is how understanding emerges in brownfield in the first place. The pipeline allows these loops because it records their results in artifacts, where they remain available after a session ends.

Third principle: state belongs in artifacts

A language model has no memory. It works only with the context it is given in this one moment. What stood higher up in the chat log is still effective only because the client sends it along again on every call. That log can lose effect as it grows longer. The effect is discussed as context rot and can set in before the technical limit of the context window is reached. If the project state is kept in the chat alone, it is easily lost again.

The pipeline therefore keeps the state outside the chat, in versioned artifacts. I call this rule the artifact contract: what arises during an agent session and is not transferred into a durable artifact counts as not having happened. The artifacts are Markdown files in the repository for context, findings, plans, and test output. Tickets hold scoping, status, and accountability, approved specification pages hold decisions, and pull requests hold the diff, the tests, and the review decision.

This has three consequences that together make the value. A new session can build on the artifacts without knowing the earlier discussion. In brownfield that counts for more than in a new build, because brownfield work rarely finishes in a single day and several people often share it. An epic, a plan, a finding can be read and assessed by a human before any code comes of it. The progress log documents what was decided when and why. That trail supports the evidence and accountability obligations in regulated contexts.

One artifact arises in a legacy system at no extra cost and stays reliable: the version history. What stands in the repository binds more strongly than any memory. A commit message records the why of a change, the provenance of a line can be traced, and the state before a commit can be compared with the one after. This information arises while working, on its own, and does not go stale, unlike a separately maintained notes file that no one updates when the code changes. An agent that reads the history of the affected file before a change stands on firmer ground than one that guesses. The pipeline therefore requires Git to be used as the binding source of truth.

The compliance framework the pipeline orients itself by pushes this artifact duty further. Every AI-generated artifact is marked as AI-generated, with a traceable reference to the operation, the person, and the model that produced it. That is the EU AI Act’s transparency requirement in daily practice: a concrete note on each individual piece of generated content. Whoever later wants to know whether a specification was written by a person or proposed by a model and only waved through finds the answer in the artifact.

Fourth principle: every role handover has a quality gate with mandatory artifacts

The core of the pipeline lies in the handovers. Between two roles the work does not simply change owner. A checkpoint stands in between, a quality gate, and this gate has two parts: a short, answerable question and an artifact that evidences the answer. Without the artifact the question cannot be answered, and only an affirmed answer opens the next phase.

That comes down to the error costs, which shift from phase to phase. An error found in analysis costs a correction in text. If the same error surfaces only in operations, it costs a correction in the running system, plus the search for the cause and the restoration. The riskier and more consequential a step is, the earlier it needs evidence, a plan, and a human decision. That decision falls at the gates.

At the end of discovery stands the question of whether the sources are solid. Are gaps and contradictions made visible, or was it smoothed over? The mandatory artifact is the labeled source inventory. Only once a human confirms that the knowledge base is evidenced and honest does structuring begin.

Before a ticket goes into implementation, a Definition of Ready checks whether it is small and testable, with acceptance criteria and explicit non-goals. This check is especially strict in a legacy system. It requires, for example, that a ticket’s outcome be phrased from the user’s point of view and not the system’s. “The caseworker can save a claim even without complete details, but receives a business message” is a testable outcome. “The system validates the claim field” is not, because it cannot be tested from the user’s point of view and shifts responsibility for success from the business side to engineering. A second rule of the same gate: no technical specification in the requirements ticket. Architecture, data model, framework choice, column names do not appear in the business ticket. A technical specification at this point makes the later feasibility check worthless and is the most common reason development hands a ticket back.

Before the first code there is a plan review. It checks the order of the work, the test strategy, the risks, and the affected files. Only after this review does the agent begin to write. That is extra effort. A plan reviewed beforehand, though, catches the errors an agent would otherwise build into the code over twenty tool calls before anyone notices that the direction was wrong.

The test gate asks whether the tests cover the acceptance, whether an intentional test gap is documented, and whether the pull request can be assessed with a test record. What counts as evidence here is a result: a browser test that runs the behavior, a trace, a screenshot. The agent claiming something is not enough in the gate. It has to show that it works.

The sharpest is the apply gate in operations. It separates diagnosis from effect. The agent may prepare a pipeline change, an infrastructure-as-code adjustment, or a monitoring rule as a pull request. Approving a pull request, however, is not approving the apply. Rollback, time window, responsible person, observation, and release are decided before any effect. The start on the real infrastructure stays a human action, even where the agent could technically perform it. The gate’s question is: is the effect reversible, is there a rollback and observation, and is the apply separated from the code review?

Beneath these human gates lies a machine layer that no one has to go through by hand. In a mature setup, automatic checkpoints examine every pull request before a human even sees it. These include an intake check, a policy check, a static security analysis, a prompt-injection detection, a red-team run, a sandbox execution, and a final gate with a report. These checkpoints run as deterministic scripts, and they run on purpose before the expensive, non-deterministic AI stations. That way no model spends compute on obvious garbage, and the gating becomes provable, because a script delivers a traceable result while a model answer does not guarantee one. The human gates remain responsible for the questions that call for judgment. Everything that can be checked automatically is handled beforehand by the scripts, and only the two layers together make the control that is needed.

Fifth principle: no decision rests on a single model answer

A language model is a statistical apparatus. It reproduces what occurred often in its training data, and its tone stays just as confident whether or not that holds in the concrete case. A single model answer is therefore only a hypothesis at first. In brownfield, where the specific counts and the average of the training world is often off the mark, this distinction weighs heavily.

The pipeline meets this with a simple discipline: for every important decision, a second and, where it matters, a third opinion is obtained, and from a different model. Different models are trained on different data, with their own strengths and blind spots. When two independent models reach the same finding, reliability rises. When they diverge, that too is a result: the human then knows the matter is not clear-cut and someone has to look more closely.

In analysis, several models can summarize the same source, and comparing the results shows where they differ in fidelity to the source, accuracy, and assumptions. For code review, the patch is checked not only by the model that wrote it but also by a second one. That one does not know the patch and is therefore not in love with its own logic. During verification, a model can be set explicitly to attack and refute a finding. What survives a serious attempt at refutation carries more weight afterwards.

Behind this stands model governance. In a mature setup the choice of model is not left to chance. Models are addressed through abstract tiers, from cheap to top class, backed by budget profiles and routing rules. The brand names stay out of it. Whoever wants to use an expensive tier has to justify it, and for some decisions a human approval is added. That keeps the second or third opinion reserved for the cases where the risk justifies it, and it does not become a silent cost driver. A trivial reformat does not need three models, but an architectural decision in the brownfield context is a different matter.

The second look does not only raise accuracy, it also hardens the system against a specific weakness of agentic setups. The context a model works with can be manipulated, for example through a prompt injection in a file it has read. Then a second model with different behavior is an additional hurdle for the attack. A defense that checks on several levels is harder to defeat than a single instance that you outwit once.

Sixth principle: compliance belongs in every phase

In many projects, compliance comes last. First comes the building, then the check whether what was built meets the rules, and at the end the notes are added. In agentic work this order no longer holds, and that is because of the speed. When an agent produces in minutes what took a human days, a downstream check is overrun long before it can take hold. Compliance has to sit where the work passes through anyway, that is, in the phases and gates of the pipeline itself.

The regulatory frame for German and European teams covers at least the EU AI Act and the General Data Protection Regulation. The EU AI Act, Regulation (EU) 2024/1689, follows a risk-based approach. Depending on the use case and risk class, duties arise around transparency, technical documentation, logging, human oversight, and cybersecurity, in particular for high-risk AI systems and certain generative AI applications. The General Data Protection Regulation, Regulation (EU) 2016/679, governs the processing of personal data and requires, among other things, purpose limitation and data minimization. Beyond that, teams have to account for internal security requirements and applicable AI-security frameworks. For generative AI these typically concern threat modeling, data classification, model security, prompt protection, supply chain, and incident response.

Translated into the pipeline, this yields concrete rules anchored in the phases. At the root stands data minimization: in development, personal data does not belong in the artifacts. Sample data in mandatory artifacts uses only placeholders from a sample schema, so no real personal names, companies, or case numbers. Workshop material, training exports, and pull requests quickly end up in more public contexts, such as a shared page, a mirror on a code platform, or a presentation. If you anonymize from the start, it spares the later search, where one hit usually slips through anyway.

Accessibility is easily forgotten in a legacy system. A mature setup of the pipeline follows the Web Content Accessibility Guidelines at level AA and the European standard EN 301 549, where they apply to the use case. A UX state that captures an error case also describes how that error becomes perceptible to a person using a screen reader. For certain products and services, such requirements have been binding in the EU since 28 June 2025, in Germany through the Barrierefreiheitsstärkungsgesetz, the Accessibility Strengthening Act. For public-sector bodies, accessibility requirements have applied for longer.

Part of traceability is that every statement in an artifact names its source with a date, every AI-generated piece its producer note, and that token consumption stays traceable. This trail supports the audit trail that regulation, accountability obligations, or internal governance may require depending on the use case. In the pipeline it arises as a by-product of the artifact contract that is required anyway.

For brownfield systems there is the protection of existing data against forced migration, which I consider one of the most important rules of all. A new feature does not automatically migrate or validate existing data. New obligations apply to records newly created or actively edited, and the existing data stays readable and storable. Without this line, a data migration gets dragged into a feature ticket as a side condition. A data migration, however, is a project of its own, with its own stakeholders, risks, and time windows. Treated as a side point, it distorts the scope and blocks the handover to development. Often it is exactly this rule that decides how long a feature takes in brownfield.

How the principles work together

The six principles are not a list from which you pick the comfortable ones, because they mesh in a particular order.

A handover between two people only works on an object both of them read, so the roles need the artifacts. An artifact without a checkpoint is just a file that no one has bindingly assessed, which is why the artifacts presuppose the gates. A gate, in turn, can only check what an honest preliminary investigation delivered in findings, gaps, and contradictions. The second and third opinion sharpens the human decision at the roles without replacing it. Compliance, finally, has no material to act on without all four.

The result is a chain of artifacts in which each link depends on the previous one. From a labeled source inventory comes a solid epic, from the epic tickets that pass a Definition of Ready. From the tickets comes a reviewable plan, from the plan a pull request that survives the test, and only from that an apply decision that can be taken responsibly. A team that skips a link cannot honestly close the next one. You can pretend to, but the omission takes its revenge where the system is stressed most, and in a grown system that is almost always production.

A common objection: doesn’t this sound like a return to the waterfall, the heavyweight process the agile movement wanted to overcome? The pipeline does have phases and handovers, but it is not rigid. Within each phase the work is iterative, with loops and self-corrections. The tickets are small, one to three hours, and flow through implementation one at a time. What sets it apart from the classic waterfall is the binding force of a few checkable artifacts at the right places, and the volume of documents does not matter for that. Which agile assumptions still hold under agentic tools and which lose their steering function, I have treated elsewhere. Here it is enough that the method provides for just as much iteration as before, only better evidenced.

What this series treats next

This text has stayed at the level of the approach on purpose and only touched on the individual roles. Each role offers enough substance for its own piece, and in practice the value lies in the detail.

The following pieces therefore go into depth. Business analysis shows how a source inventory becomes an evidenced knowledge base, how labeling statements works, and how you recognize that a discovery is ready for structuring. UX is about how a feature is described in one’s own words and how a team template with acceptance criteria and states emerges. That piece also shows why the description of the error cases often tells more than that of the happy paths. Development takes apart the cascade of research, epic, tickets, plan, and the final implementation loop. It shows how an agent can work autonomously across many steps without losing control. Testing deals with how acceptance criteria become scenarios and how a browser-based piece of evidence replaces a claim. Operations treats the separation of diagnosis and effect, the apply gate, and the question of what an agent may even prepare on real infrastructure and where the line lies.

Anyone who wants to try the pipeline before the detailed pieces appear can start with one rule that pulls all the others along: keep the state in versioned artifacts. Write the source inventory, the epic, the plan, and the test output into the repository. Once these artifacts exist, the question of the gates almost answers itself, because an artifact asks to be checked before the next one builds on it. Once the gates exist, the question of the roles follows, because a check needs someone to do the checking.

Shortcuts do not work in brownfield, because these systems are too old and too tightly entangled with everything else. The confidence of a single model in a fleeting chat is not something to rely on here. The roughly ten percent productivity gain that the DORA report names for brownfield work is no law of nature. It describes the order of magnitude of a calculation under particular assumptions. With the right pipeline that number can be moved. How far, I do not know, and I will refrain from naming a figure I cannot support. What I take away from the workshops is the shift of the work from fast typing toward the handovers, where a human checks what the agent has put forward. Whether that changes anything in your own system shows only when you try it on a real feature.

• the economic assumption that implementation is the bottleneck
• the methodological assumption expressed by the Agile Manifesto in its four value pairs
• the personnel assumption about teams, learning paths and roles
• the metric assumption that generation speed approximates delivery quality

At several of these points, agile logic is reversed. Supplementing or shifting it does not describe what happens there.

These claims rest on the current state of research on agentic software development, as far as it appeared between October 2025 and May 2026 as arXiv preprints or peer-reviewed contributions.

A different scarcity

Agile emerged in a time when the most expensive and slowest resource in software development was human labour. Developers cost money, onboarding took months, context switching was expensive and good architects were scarce. Several core principles followed from this scarcity.

Self-organisation emerged because central planning was too slow and too poorly informed to allocate scarce developer time sensibly. Timeboxing came about because tasks could neither be estimated nor tracked reliably over months, and smaller time windows limited the cost of error. Velocity emerged because a team knew its own average delivery speed better than any external capacity planning function. The principle of “working software over comprehensive documentation” followed from the fact that a complete specification before the code was more expensive than the code itself and quickly became obsolete anyway.

Each of these answers makes sense as long as implementation is the bottleneck. That assumption is changing. Bhati calls the new bottleneck verification debt. Apostolou, Bosch and Holmström Olsson speak of a capability-deployment verification gap. In their 16 interviews with twelve companies, they describe the same picture from two directions: four companies demonstrated higher agentic capabilities but could not use them productively because reliable verification was missing. Bandara et al. propose Agentsway, a process model in which verification and human orchestration stand alongside code generation as equal concerns.

This tips the optimisation problem. How fast a human can write code is no longer the bottleneck. What is scarce now is the pace at which an organisation can review, classify and take responsibility for generated code. All agile answers to the old scarcity are thus addressed to the wrong problem. Self-organisation answers scarce implementation capacity and offers nothing against scarce verification capacity. Velocity measures generation speed and says nothing about review capacity. Timeboxing structures human work in iterations, but not a gate sequence of specification, code, test, evidence bundle, review and approval.

A methodology built for a different scarcity is not modernised by adaptation. It loses its justification together with the optimisation problem it was designed to solve. That is the economic break.

What becomes of the four value pairs

The Agile Manifesto formulates four value pairs. Agentic work pushes three of them into reversal. The fourth remains formally intact, but its addressee changes.

Working software over comprehensive documentation. Spec-driven development reverses this order. Piskala describes the spectrum from spec-first to spec-as-source. Feng and Chen show in a pilot study that structured signatures can increase the test pass rate in repository-level generation. Lulla et al. measure a 28.64 percent shorter runtime and a 16.58 percent lower token consumption for an AGENTS.md across ten repositories and 124 pull requests. Galster et al. document eight configuration mechanisms in 2,853 repositories, including context files, skills, subagents, commands, rules and hooks. These artefacts stand before the code as a condition of possibility. Without them, output becomes less specific, slower and more expensive. The order of the Manifesto is thereby reversed.

Individuals and interactions over processes and tools. When the tool is the main producer, this sentence loses its meaning. In Agentic Agile-V, Koch argues that long chat histories do not work as engineering contracts. Interaction becomes a risk that processes and tools have to contain. With conversation-to-contract gates, acceptance gates, TDD governance (Hasanli et al.), policy-as-prompt (Kholkar, Ahuja) and plane separation in CI/CD (Barnes, Ghaleb), the research turns the axis. Commitment now comes from processes and tools because interactions and individuals can no longer guarantee it.

Customer collaboration over contract negotiation. The agentic turn forces requirements to become verifiable. A specification used as input for an agent is a contract in the technical sense: acceptance criteria, constraints, test cases. This is not contract negotiation in the legal sense. Nor is it the oral, collaborative requirements clarification of classic Agile. The spec becomes a contract again, this time in machine-readable form.

Responding to change over following a plan. This sentence formally still stands, but its addressee moves. The plan is now changed under gate discipline, which defines which change the agent may execute, which needs human approval and which must never take effect automatically. The research on plane separation by Barnes and Ghaleb and on governance layers by Koch, Xu et al. and Kholkar reads in places closer to V-model discipline than to Scrum.

A methodology whose values are reversed in three of four pairs cannot be continued without breaking the term. It must justify itself anew, or it loses its claim.

Who becomes senior?

The discussion about agentic tools speaks of the human as an orchestrator, which is the friendly formulation. The sharper reading concerns personnel structure, and it is more uncomfortable.

Horikawa et al. examine 15,451 refactoring instances in 12,256 agentic pull requests. Refactoring occurs frequently, aims mostly at maintainability and readability, and often remains local. High-level design work is less strongly represented. This is the distribution that used to be assigned to junior developers as a learning field: small tickets, local improvements, writing tests, improving readability. Pinna et al. show clear strengths for 7,156 agentic pull requests in certain task types, especially in the class of bounded, local changes. Robbes et al. document in their adoption study the rapid spread of this type of activity in particular.

So what agents are good at is, for the most part, the area through which junior developers build seniority. If this learning path is structurally shortened or skipped altogether, a question arises that is rarely asked in methodology debates: where, in five years, will the people come from who can reliably review, classify and take responsibility for agentic outputs? Apostolou et al. identify reliable verification as the missing element that prevents several companies from productively using higher agentic levels. Verification at this depth requires experience that, in the classic model, was built through exactly the work that agents are now taking over. The scientific material does not state this observation directly. It follows, though, from the cited findings read together with the common personnel structure of software teams.

The Scrum roles also come under pressure. Scrum Masters and Agile Coaches depend almost entirely on human team dynamics. They are the answer to conflict, communication deficits, missing self-organisation and unspoken expectations. In a workflow in which one human works with an agent, these issues lose weight. Technical workflow architecture takes their place: gate design, spec structures, tool rights, hook and policy configuration, MCP setup, evidence-bundle schemas. Galster et al. document eight such mechanisms, and most of them are engineering topics. A role whose field of work is so tightly bound to a shrinking component of the methodology is resized along with that component.

The Product Owner is replaced by business architects. The PO as an interface between business and development team was an answer to expensive development time and to business departments that could not translate their requirements into implementation themselves. When translation moves into machine-readable specifications and implementation capacity becomes cheaper, part of this mediation disappears. Responsibility for acceptance criteria, spec maintenance and gate decisions can move closer to the business than the Scrum model envisaged. In its agile definition, the Product Owner is no longer necessary in this form.

The metric reversal

A metric is useful as long as its sign is stable. More is good, or less is good. A metric whose sign can reverse is only a signal that has to be evaluated according to context.

Velocity stands at this point. Agarwal, He and Vasilescu examine 401 agent-first repositories and 117 IDE-first repositories with matched control groups. They find short-term velocity gains that are front-loaded in the agentic groups. At the same time, static-analysis warnings and cognitive complexity rise markedly. Ehsani et al. show in 33,596 agentic pull requests that reviewer abandonment is the most frequent pattern in the manually categorised subset of rejected PRs. High generation speed does not correlate with high delivery quality. In the agentic groups it goes together with growing complexity and review load that is not being handled.

In an agentic environment, velocity therefore turns from a virtue into a risk indicator. High agentic velocity with stable review capacity is suspicious. A methodology that has to reinterpret its central control metric as a risk metric should not gloss over this. Velocity remains measurable but loses its meaning as a control variable. Burndown shares this fate because its meaning rests on the same assumption: that generation speed approximates delivery speed.

No replacement exists yet. Gate cycle time, evidence completeness, verification density, policy conformity or accepted acceptance criteria per unit of time come into question as candidates. None of them is mature practice. The agile movement is left without a stable control metric, while the old metric not only becomes unreliable but flips into its opposite.

A business model disappears

One layer appears only rarely in the scientific material, but it is important for the fate of agile methodology. Since the mid-2000s, Agile has also been a consulting and certification market. SAFe, Scrum.org, LeSS, Scaled Agile Inc. and many independent coaches live from training courses, certificates and transformation projects. Their product consists of team coaching, rituals, role models and scaling frameworks.

The research points to other priorities. Bandara et al. put roles, learning loops, fine-tuning, orchestration and verification at the centre of their process model Agentsway. Xu et al. call for a governance-first approach to agent engineering. Kholkar and Ahuja develop policy-as-prompt as a translation of governance rules into runtime guardrails. Koch works on a multi-layer translation of governance norms into runtime guardrails and on a V-model variant for agentic work. Galster et al. document the configuration layer of existing tools. All of this is engineering work and none of it is coaching work.

This creates a structural transition problem. An industry whose business model consists of teaching rituals, roles and self-organisation practices cannot move into a market for gate architecture, spec engineering and policy configuration without losing its identity. That requires different methods, curricula, tools and buyers inside companies. Here, too, the old industry does not have to disappear. It can keep its former claim to methodological market leadership only if it reinvents itself. Renewal on this scale is historically rather rare and usually possible only across a generational change.

What might remain

Which agile contributions still apply because they do not depend on the disappearing scarcity?

Small delivery packages remain useful because they reduce risk regardless of who writes the code. So does close contact with business domains, because specifications quickly become wrong without domain feedback. Reflection at regular intervals keeps its value if its results land in the agent’s working environment and not only on a whiteboard. Accountability for delivery remains useful because agents cannot and should not bear responsibility. Transparency about ongoing work belongs on the list as well, but in the form of verifiable artefacts and not as a regular status meeting.

This is a much shorter list than what the agile movement has exported as methodology. It consists largely of practical wisdom older than the Agile Manifesto and able to stand without it. The specifically agile apparatus of velocity, sprint estimation, self-organisation as a control principle, Scrum roles and ritual cadence is missing from it. This apparatus distinguished Agile from older practice traditions. Agile with agents does not describe what is left. What remains when this apparatus disappears is closer to disciplined, verification-conscious software development with short feedback cycles, a practice that can be justified without the term Agile.

Conclusion

Agile methodology is collapsing because the conditions it optimised against are disappearing in several layers at once. Whether its supporters abandon it makes no difference to that. Scarcity moves from implementation to verification. The personnel structure on which its idea of team and learning path rested loses its basis of reproduction. Its control metric reverses its sign. The consulting economy that organised the methodology is left without a compatible product.

Four of these shifts are directly supported by the 22 works evaluated here. The fifth, the business-model question, is derived. Anyone looking at the documented shifts one by one sees adaptation. Read together, they give a different picture. Software-engineering practices do not end with this. What ends is the specific synthesis that was called Agile, in the form in which it was taught and sold for twenty years.

The more open question is speed. Apostolou et al. show that industry is still early. Seven of the twelve companies interviewed operate at the lowest maturity level. Four can demonstrate higher levels but cannot use them productively. That gives the existing methodology time but does little to change the direction. The gates, artefacts, plane separations, spec structures and governance layers documented in the research will sooner or later arrive in practice. When they do, velocity charts, daily stand-up notes and sprint boards will stay in place and simply show the wrong thing more and more visibly.

Note on evidence and method

This essay is based on my own evaluation of 22 works on agentic software development from the period between October 2025 and May 2026. It is not a formal systematic literature review. The selection and interpretation are mine.

Evidence for the economic, methodological, personnel and metric break points comes directly from the evaluated works. The section on the consulting economy reaches outside the scientific material and is marked as such. Likewise, the question of learning paths from junior developer work towards seniority is derived from the activity distributions that the studies describe. It is not a thesis directly established by the research.

One research gap is clear. Meaningful longitudinal studies of teams working agentically for more than a year hardly exist. Cross-team coordination is barely studied, and the translation of agile metrics into agentic workflows remains open. Responsibility after incidents, performance evaluation and skill development are also underexamined.

How the value of the produced results can be planned and assessed commercially remains entirely open.

Literature

The following list names the locally evaluated versions. For arXiv papers, the link points to the respective abstract page.

• Shyam Agarwal, Hao He, Bogdan Vasilescu: AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development. arXiv:2601.13597. https://arxiv.org/abs/2601.13597
• Spyridon Alvanakis Apostolou, Jan Bosch, Helena Holmström Olsson: Agentic AI in Industry: Adoption Level and Deployment Barriers. arXiv:2605.14675. https://arxiv.org/abs/2605.14675
• Eranga Bandara et al.: AGENTSWAY – Software Development Methodology for AI Agents-Based Teams. arXiv:2510.23664. https://arxiv.org/abs/2510.23664
• Marcus Emmanuel Barnes, Taher A. Ghaleb: From Assistance to Agency: Rethinking Autonomy and Control in CI/CD Pipelines. arXiv:2605.07062. https://arxiv.org/abs/2605.07062
• Happy Bhati: Agentic AI in the Software Development Lifecycle: Architecture, Empirical Evidence, and the Reshaping of Software Engineering. arXiv:2604.26275. https://arxiv.org/abs/2604.26275
• Ramtin Ehsani et al.: Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub. arXiv:2601.15195. https://arxiv.org/abs/2601.15195
• Shuzhao Feng, Boqi Chen: LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering. arXiv:2605.02455. https://arxiv.org/abs/2605.02455
• Matthias Galster et al.: Configuring Agentic AI Coding Tools: An Exploratory Study. arXiv:2602.14690. https://arxiv.org/abs/2602.14690
• Taher A. Ghaleb: When AI Agents Touch CI/CD Configurations: Frequency and Success. arXiv:2601.17413. https://arxiv.org/abs/2601.17413
• Tarlan Hasanli et al.: TDD Governance for Multi-Agent Code Generation via Prompt Engineering. arXiv:2604.26615. https://arxiv.org/abs/2604.26615
• Rashina Hoda: Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabulary. arXiv:2510.19692. https://arxiv.org/abs/2510.19692
• Kosei Horikawa et al.: Agentic Refactoring: An Empirical Study of AI Coding Agents. arXiv:2511.04824. https://arxiv.org/abs/2511.04824
• Gauri Kholkar, Ratinder Ahuja: Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents. arXiv:2509.23994. https://arxiv.org/abs/2509.23994
• Christopher Koch: From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI. arXiv:2604.05229. https://arxiv.org/abs/2604.05229
• Christopher Koch: Agentic Agile-V: From Vibe Coding to Verified Engineering in Software and Hardware Development. arXiv:2605.20456. https://arxiv.org/abs/2605.20456
• Jai Lal Lulla et al.: On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents. arXiv:2601.20404. https://arxiv.org/abs/2601.20404
• Giovanni Pinna et al.: Comparing AI Coding Agents: A Task-Stratified Analysis of Pull Request Acceptance. arXiv:2602.08915. https://arxiv.org/abs/2602.08915
• Deepak Babu Piskala: Spec-Driven Development: From Code to Contract in the Age of AI Coding Assistants. arXiv:2602.00180. https://arxiv.org/abs/2602.00180
• Romain Robbes et al.: Agentic Much? Adoption of Coding Agents on GitHub. arXiv:2601.18341. https://arxiv.org/abs/2601.18341
• Giovanni Rosa et al.: Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design. arXiv:2601.03878. https://arxiv.org/abs/2601.03878
• Yuheng Tang et al.: DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle. arXiv:2601.20882. https://arxiv.org/abs/2601.20882
• Qiang Xu et al.: From Craft to Constitution: A Governance-First Paradigm for Principled Agent Engineering. arXiv:2510.13857. https://arxiv.org/abs/2510.13857

This architecture has a consequence that only emerges in productive use. The context window becomes the system’s scarcest resource once a language model moves out of the editor chat into a command line and starts processing tasks for hours on end. The agentic command-line tools of the major vendors all answer the same technical question: how do you manage a scarce, transient working memory in such a way that a longer session remains productive? Six tools are meant: Claude Code by Anthropic, Codex CLI by OpenAI, Google’s Gemini CLI, Alibaba’s Qwen Code, OpenCode from the open-source community and GitHub’s Copilot CLI. Five of the six codebases are openly accessible thanks to open-source licences, and reading them side by side turns up the same four solution patterns everywhere, along with five axes on which the design decisions do diverge after all.

What limits the context window

All six vendors operate under the same constraints, which follow from the architecture of the underlying transformer models and are extensively documented in the research literature.

Hard-limited. One million tokens, the equivalent of around 750,000 English words, has become the new standard in the current top-tier models. Anthropic made the million-token window generally available for Claude Opus 4.6 and Sonnet 4.6 in March 2026. Claude Code users on Max, Team and Enterprise plans get it as the default without any configuration. OpenAI offers the million in GPT-5.5, capped at 400,000 tokens inside the Codex CLI. Google delivers it in Gemini 2.5 Pro, Alibaba in Qwen 3.5-Plus. Older and smaller models such as Claude Sonnet 4.5 remain at 200,000 tokens, which had been the industry standard up to 2025. Even with these enlarged windows, the attention operation of the transformer architecture scales quadratically with sequence length: twice the context means four times the compute. Sparse attention and state-space approaches dampen this effect but do not eliminate it, and a single token beyond the hard limit makes the API call fail, because nothing can be added at runtime.

Transient and stateless. At every single model call the application retransmits the entire previous history: system prompt, all messages, all tool calls and all tool results. Vendor-side prompt caching, which stores stable prefixes of the conversation, reduces the cost of repeated transmissions substantially. Anthropic charges cache reads at one tenth of the regular input price. OpenAI automatically grants a fifty-percent discount on reused input tokens, newer models in some cases more. None of this changes the underlying mechanism. The impression of memory still comes from organised repetition.

Ageing. Long before the hard limit, response quality declines. The term context rot was coined in 2025 by a technical report from Chroma Research for exactly this phenomenon. The authors tested 18 frontier models, including GPT-4.1, Claude Opus 4 and Gemini 2.5, and observed the effect in every single one. Anthropic has since adopted the term in its own documentation, and the Best Practices for Claude Code put the onset range for its million-token models at 300,000 to 400,000 tokens, around thirty to forty percent fill. The LoCoMo benchmark by Snap Research covers conversations averaging 9,000 tokens across up to 35 sessions. Empirical evaluations on it show across model families that even long-context LLMs trail human performance by a wide margin once the session grows long. A larger window shifts the problem in time without solving it.

The model’s attention spreads across more tokens, and old ballast pulls it away from the current focus. The model then grows more likely to ignore relevant instructions, which research calls instruction omission, the silent dropping of instructions without the model flagging it. A 2025 paper by Distyl AI shows that even the best frontier models tested reach only around 68 percent accuracy at 500 simultaneous instructions. The instructions fit easily into the window, the model sees them and still does not follow them all.

From these three properties follows a technical necessity. An agentic CLI must actively manage its context: compress older content, discard irrelevant tool outputs, offload important findings into durable files, reconstruct exactly what is relevant before each new task. What this means on the user’s side is covered in the field report on context management. How the six tools do this in detail is the actual subject of this comparison.

What all six tools solve the same way

The six codebases agree in four places, although programming languages, vendors and design philosophies pull apart. The convergence results from the same constraints, which force all vendors to explore the same solution space.

One shared pattern is selection over completeness. None of the tools examined copies files indiscriminately into the context. Each decides which instruction files it loads, which tool results it keeps, which memory layers are active. Selection is the front line of defence against context rot. Gemini CLI, for one, classifies read-in files into four tiers labelled internally as FULL, PARTIAL, SUMMARY and EXCLUDED, and decides per file how much of it lands in the context. Claude Code, during its micro-compaction, selectively clears old tool results without touching the conversation itself.

Just as widespread is the hierarchical discovery of instruction files. All six CLIs search for their project rules on the path from the project root down to the current working directory, whether those rules are called CLAUDE.md, AGENTS.md, GEMINI.md, QWEN.md or copilot-instructions.md. More specific rules in submodules override the more general ones. Anyone working in a monorepo automatically gets the appropriate rule set. This consensus holds across the whole industry and goes back to no single vendor. A flat, project-wide rules file would be too coarse-grained for large codebases.

Then there is the prioritisation of user content over tool output. When context gets tight, all tools discard old tool results first and keep the user’s messages. What a user has written expresses intent. A tool output can be regenerated when needed by invoking the tool again, a user message cannot, because it is an authentic part of the session.

The last of the four is structured re-anchoring after compaction. Afterwards the tool explicitly appends important state data to the new context: file attachments, the active plan, enabled tools, connections to external services and hook outputs from the session start. What comes out of it is a reconstructed working picture. Claude Code is particularly pronounced here. After every compaction it reattaches file attachments, plan-mode state, active skills, and tool and agent deltas.

Together, these four patterns form an informal pattern language for context management. The terms trim, summarise, persist and prune capture their operations. Anyone who can name them can assess a new tool within minutes. They are to context management what design patterns were to object orientation: a vocabulary.

Where the six tools differ

Despite the convergence, clear differences remain. Five axes are particularly consequential for tool choice. The values given can be verified by reading the source code.

Compaction thresholds: Gemini CLI at 50, Codex at 90 percent

This axis is measured by the threshold at which the tool triggers a compaction, expressed as a fill rate of the context window. The values lie far apart: Gemini CLI starts early, at fifty percent fill, while Qwen Code, which is built on a fork of the Gemini codebase, waits until seventy percent. Copilot CLI splits the task across two thresholds: at eighty percent an asynchronous compaction runs in the background. At ninety-five percent a blocking emergency brake holds the call until the compression is complete. Codex waits until ninety percent of the configured context window. Codex does allow the automatic token limit to be configured explicitly per model, but internally clamps the value to a maximum of ninety percent of the context window. Claude Code operates without a fixed percentage trigger. Several overlapping layers run there instead, whose depth of intervention ranges from surgical to complete.

Every threshold is a compromise. Compacting early costs additional LLM calls, since the summarisation is itself a model invocation, while late intervention lives longer in the context-rot zone and risks quality loss shortly before it. The choice between fifty and ninety-five percent is a choice between token economy and response stability.

Memory: a Markdown file in Claude Code, SQLite in Copilot CLI

The spectrum spans a wide range. OpenCode dispenses with persistent memory entirely. Whatever a session has produced lives on only in the files the user explicitly creates. Claude Code, Codex, Gemini and Qwen each use a flat Markdown file at a defined path: Claude Code under ~/.claude/.../memory/, Codex in a directory called memories, Gemini and Qwen in a project- or user-wide GEMINI.md or QWEN.md. Copilot CLI, by contrast, runs two SQLite databases in the background, combines them with vector embeddings and full-text search, and makes persistent memory a dedicated session capability with structured permission requests. Every memory write there passes through a permission gate that requires three fields: subject, fact and citations. Memory at Copilot is thus a record separated by topic, fact and provenance.

The range runs from the absence of a memory system to a fully fledged retrieval system with provenance fields. That is the clearest difference in the comparison. Anyone who only needs memory as a note in the repository has five tools to choose from, for memory as a queryable database across many sessions exactly one remains.

Sub-agents: delegation in Claude Code, a fleet dispatcher in Copilot CLI

Sub-agents serve to isolate context. The main agent delegates a sub-task to a child agent with its own window, which expires once the task is complete. This protects the main context from bloating, because the detailed knowledge of the sub-task does not flow back into the main conversation. Only the result returns. Claude Code provides a simple delegation tool here. Copilot CLI runs a fleet dispatcher that orchestrates parallel sub-agents, together with a dedicated smaller sub-agent model and explicit depth limits that prevent a task from recursing indefinitely. Codex uses sub-agents among other things for memory consolidation in a dedicated phase after session end. OpenCode and Qwen are sparsely equipped on this axis.

Observability: structured events and OpenTelemetry in Copilot CLI

One of the biggest differences sits on this axis. Copilot CLI emits structured events such as session.compaction_start and session.compaction_complete, supports OpenTelemetry and offers a preCompact hook for external observers. Every compaction can thus be measured, logged and audited. The event structure includes the token counts before and after the compression, the number of removed messages, the generated summary and the GitHub request trace. Anyone integrating a tool into productive pipelines or having to maintain audit trails depends on this kind of infrastructure. The other tools offer far less here.

Hooks and extension points in Claude Code and Copilot CLI

Hook points such as sessionStart, preToolUse, subagentStart or preCompact turn a closed tool into a platform. They are the modern form of extension points and let teams hang their own governance, validation and logic into the agent loop. Claude Code and Copilot CLI are the richest equipped here. Codex offers specific but less orchestrable extension points. Gemini, Qwen and OpenCode remain more sparing. They leave extensions largely to the configuration mechanism, and an event-based model is scarcely present there.

The six tools in detail

With the four patterns and five axes, each individual CLI can be characterised.

Claude Code: compaction in four layers

Claude Code treats compaction as a gradient with four layers that differ in their depth of intervention. At the lowest layer, a micro-compaction replaces individual older tool results with a placeholder text. It is surgical, leaves the conversation untouched and runs continuously during the session. Above it, a layer cuts individual older message blocks when needed, before the micro-compaction operates. As a further layer, session-memory compaction runs as a post-sampling hook after every model response, extracting memory entries from the session and storing them in a dedicated file. The deepest layer is the classic, complete compaction with subsequent rehydration of all state data. This architecture is the most elastic of the six and simultaneously the one with the highest internal complexity. It follows the idea that compaction should be a family of staggered interventions.

Codex: a 90 percent threshold and a protected user window

Codex treats the session as a cleanly defined state space. Automatic compaction kicks in at ninety percent of the context window. The user window holding the most recent portion of the conversation is preserved with 20,000 tokens so that the current working context stays protected. The memory system is built as a separate subsystem. Codex routes memory entries through a dedicated trace-summarisation endpoint to a sub-agent which, in a dedicated phase after the session ends, consolidates them into durable memory files. Reproducibility is the clearly visible design goal: initial context, compaction boundary and memory operations are all explicit and traceable.

Gemini CLI: a separate utility model for compression

Gemini CLI distributes the task across four specialised services that work together and affords itself a dedicated utility model that produces the summaries. This model goes internally by the name chat-compression-2.5-flash-lite. It is a smaller, cheaper variant of the main model and exclusively handles compression. With this, Gemini comes closest to the separation of memory operations from the main model recommended by research. Compaction begins aggressively at fifty percent fill, which generates extra LLM calls but keeps the main model well below the context-rot zone.

Qwen Code: a Gemini fork with a higher threshold

Qwen Code is a fork of Gemini CLI by Alibaba. The codebase inherits large parts of the Gemini logic. The compaction threshold is, however, set higher at seventy percent, and the compressor sits at a different point in the pipeline. A closer look shows that the compression bypasses the memory path that Gemini routinely traverses. The result is a modern architecture with an internal break in integration: what Gemini cleanly separates is brought together again in Qwen at one point.

OpenCode: overflow management without persistent memory

OpenCode understands itself as a janitor tool. It runs strong overflow management with a 20,000-token buffer and no fixed percentage threshold: compaction triggers when the context is actually about to overflow. Instruction files are loaded lazily. The code is flat and hierarchically organised, the overflow logic transparent and easy to follow. A persistent memory system is entirely absent. Anyone who prioritises simplicity and traceability finds the lowest complexity here.

Copilot CLI: SQLite memory, fleet dispatcher and hooks

Copilot CLI is of all six tools the closest to what research papers from 2025 and 2026 demand. Compaction runs two-stage, asynchronously at eighty percent and blocking at ninety-five percent. Persistent memory is managed across multiple sessions in two SQLite databases with vector embeddings and full-text search. Memory writes are permission-controlled and carry their own provenance field, documenting the sources of the stored fact. Sub-agents are dispatched in parallel through a fleet dispatcher. Compaction emits structured events with extensive metrics. A hook system allows external logic to be injected before and after every compaction. The price of this richness: the core is closed source. Only the SDK is publicly accessible. The central algorithms can however be inferred from the freely available SDK code and the documented telemetry, since the default thresholds and the event structure of the compaction are published there as a typed interface.

Memory subsystems: Markdown file or SQLite database

The memory subsystem is where the architectures diverge most strongly, and at the same time where the largest research gap lies.

Four of the six tools store memory as a Markdown file. Claude Code, Codex, Gemini and Qwen follow this path. The advantage is obvious: Markdown is human-readable, versionable and can be placed in the repository. A developer can read memory entries in a normal text editor, track them in git and review them in a pull request. Against this stands a disadvantage: a flat file is not a queryable system. Whoever wants to know what was decided about the authentication module two weeks ago has to either read the file themselves or throw its entire contents back into the agent’s context.

OpenCode dispenses with any persistent memory system. This is a conscious design decision. Anyone needing memory creates their own Markdown files in the repository and refers to them from the prompt. Every session therefore starts with whatever is documented in the repository, since the tool stores nothing internally. This clarity has its price: there is no automatically built-up learning curve across sessions.

Copilot CLI takes the opposite route and implements a full memory subsystem. Two SQLite databases store content and embeddings separately, full-text search allows classic keyword queries, and vector embeddings allow semantic similarity searches. Memory is written or voted on through structured permission requests. At every write the three fields subject, fact and citations must be present. This is closer to the demands of the research literature, particularly to works such as MemInsight and CDMem, than any of the other tools examined.

That said, even this system is not the state of the art. Cross-session memory is, according to the documentation, explicitly experimental. Intent-driven retrieval is missing, that is, the selection of memory entries on the basis of the current task intent. Retrieval runs over paths, namespaces and semantic similarity, without goal states. Causal and temporal structures between memory entries are limited to timestamps. There is no explicit event graph.

Sub-agents for context isolation in Claude Code and Copilot CLI

Alongside memory, sub-agents are the other large field of architectural difference. The basic idea is simple: the main agent hands a sub-task to a child agent so that it does not burden the main window. The child agent starts with an empty window, completes the task and returns only the result. Intermediate dialogue, tool calls and detailed research are spared from the main context.

Claude Code provides a simple delegation tool for this. The main agent can hand a task to a sub-agent that operates in the same model with its own context. This is conceptually clean and practically useful, especially for research tasks in large codebases. Parallel Claude sessions can also send each other messages and coordinate their work on a codebase that way.

Copilot CLI has the most extensive build-out. A fleet dispatcher can start sub-agents in parallel, a dedicated smaller model handles the sub-agent work at lower cost, and explicit depth limits prevent endless recursions. This brings the architecture close to the patterns discussed in research as hierarchical multi-agent systems.

Codex uses sub-agents primarily for memory consolidation, which runs in a dedicated phase after session end. This separation between session sub-agents and consolidation sub-agents is architecturally interesting: memory operations are taken off the hot path and handled asynchronously. This lowers session latency but shifts the memory update in time.

Gemini, Qwen and OpenCode offer less on sub-agents. Sub-agents exist, but they are neither the defining design principle as in Copilot nor the deeply integrated tool as in Claude Code.

State of research and the frontier gap

As far as the six tools have come, they all trail what academic research has put forward in concepts over the past eighteen months. Works such as MemoryOS (Oral at EMNLP 2025), HiAgent (ACL 2025), LightMem (ICLR 2026) and Mnemis (accepted at ACL 2026, from Microsoft Research) sketch a next generation of memory architectures that differ from today’s CLI implementations in four respects. The papers call for:

• intent-sensitive retrieval: memory is selected on the basis of the current task intent and not by path or timestamp.
• explicit causal-temporal structures, that is, relations between facts that are addressable as a directed graph, where prose hides them.
• learning policies for inserting and deleting memory entries, which learn from the success of later steps and manage without fixed heuristics.
• security governance for persistent storage with access control, audit trails and explicit handling of personally identifiable information.

Today’s CLIs implement fragments of this. Tool outputs are sacrificed before user prose because they are reproducible. That is an efficiency feature the research praises. Memory is written selectively. Copilot stores provenance fields, the others do not. A real intent-driven retrieval logic does not exist in any of the tools. Causal structures are entirely absent. On the economic efficiency of context management the CLIs are far along. On its depth they are at the beginning.

The survey “Memory in the Age of AI Agents”, published on arXiv in December 2025 with 47 authors, reaches a similar conclusion. Traditional taxonomies such as the distinction between long-term and short-term memory, it argues, are insufficient for modern agent memory systems, and multi-session consistency across hours and days is considered largely unsolved. Selective forgetting is described as the hardest of the open problems. Even Mnemis, one of the currently best-placed systems, achieves 93.9 out of 100 points on the LoCoMo benchmark with GPT-4.1-mini and 91.6 on LongMemEval-S. The paper explicitly claims no comprehensive solution and documents its limits transparently.

Consequences for tool choice

“Which CLI is the best?” is the wrong question, a universal answer does not exist, and a tool is only ever suitable for particular requirements. Anyone working in a CI/CD pipeline will favour Copilot for its observability or Codex for its reproducibility, while for interactive, heavily configured work Claude Code offers the greatest flexibility. For interactive, heavily configured development, Claude Code offers the greatest flexibility. For memory as a queryable database across many projects there is exactly one choice. OpenCode is the tool for those who value simplicity above all.

The pattern language is useful beyond this comparison. Anyone who has internalised trim, summarise, persist and prune, along with hierarchical discovery, sub-agent isolation, post-compact rehydration and hook-based extension, reads the relevant code paths of the next tool on the market and needs no demo video for it. The vocabulary outlives the individual implementations.

Most uncomfortable for vendors and users alike: even the best memory system does not replace structured planning of the work itself. The tools deliver the mechanics. What is to be built, in which order, and by what criteria for acceptance is still decided by a human.

That leaves the gap between research and implementation. It is an opportunity. The next generation of CLIs will be decided by which vendors first bring intent-sensitive retrieval, causal-temporal memory structures and learning policies into productive form. The main model will not be the deciding factor. Anyone choosing today should factor in that this field will look different in two years, without the pattern language presented here becoming obsolete.

DORA is now part of Google and has been the most influential research body for measuring software development since 2014. Its studies produced the four key delivery metrics around lead time, deployment frequency, change failure rate, and recovery time, which are now standard across the industry. The report “The ROI of AI-assisted Software Development” is a joint effort with the internal consulting unit “delta,” which supports Google Cloud customers in adopting AI.

The report is divided into seven chapters. It opens with an executive summary and builds a business case. It then describes the split in how the market assesses AI and walks through the ROI calculation with value drivers, cost drivers, and sample calculations. Five structural prerequisites for successful adoption and an investment roadmap follow. The report closes with guidance on securing long-term returns. The appendix contains a sample ROI calculator, and an interactive version is publicly available at dora.dev/ai/roi/calculator.

AI as an Amplifier, Code as a Burden

The claim that AI acts as an amplifier appears in the report in many variations. It follows from the central observation of the 2025 DORA Report: an organization with a well-functioning engineering system multiplies its strengths through AI. In a dysfunctional system, AI multiplies the dysfunctions. “Without this foundation, AI creates localized pockets of productivity that are often lost in downstream chaos,” the authors write. The result is a remarkable reinterpretation of what counts as code output.

With reference to the standard work “Software Engineering at Google,” the report cites the maxim that over the lifetime of a system, code is often “a liability, not an asset.” Operating a system, it says, exceeds the cost of building it by orders of magnitude. More code, generated without oversight, adds verification effort and leads to long-term technical debt. This reversal is programmatic: it shifts the usual AI promise from “more code, faster” to “fewer bottlenecks, consistently.”

A Familiar Thesis, Now Quantified

Anyone who reads the report alongside the recent post on the Team Operating System for Agentic CLI will find a direct point of reference. The thesis there was that tools without a method push errors into the output, where they stay invisible. In a Stanford study, Christopher Potts and Moritz Sudhof put the share of errors that became invisible at 88 percent in dialogues with untrained users. The DORA report supplies the organizational and economic side of that. Those invisible errors are not only a quality problem. They also show up in the delivery metrics as a higher change failure rate and a longer recovery time, with a measurable effect on revenue.

The report’s five systemic keys (trust, an internal platform, a data ecosystem, user focus, automated guardrails) largely coincide with the pillars of methodically run training. A written AI stance corresponds to a repository constitution. The internal developer platform run as a product has its counterpart in the picture of tool zones, in which tools are sorted by risk. User focus contains the mandate to cut every delivery against a clear goal. The goal is ROI. Without the method, though, ROI stays scattered. What used to count as a question of style and review culture appears in the DORA report as a variable in the money model.

What This Means for German Teams

Sample figures in the report come from a U.S. cloud-native model. At $176,000 in annual compensation, the calculator sits at the upper end of what a German software professional costs, above all in regulated industries with collective agreements. The report puts the gap between greenfield and brownfield effects at 35 to 40 percent versus ten percent, and that gap hits German enterprise landscapes harder still. More aging legacy knowledge is built into them than into the reference values from Mountain View. A result from the sample calculator will not materialize one-to-one in Stuttgart or Walldorf.

The structural point still holds, and it sits in one number: eight months. Under the model’s assumptions, that is how long the phase lasts in which more money flows into the system than comes out. Anyone who reads the J-curve as failure during this time and pulls the funding forfeits, in the authors’ reading, the upswing that follows. With an investment in the foundation, meaning platform, data, verification, trust, and user focus, a company rides the curve back up. Waiting for the next model finds no support in the text. The rest of this article reconstructs the calculation in detail, puts the contradictory market data in context, and names the reservations the report itself raises.

The J-Curve and Its Three Forces

At the center of the financial model is a curve shaped like the letter J. The report calls it the “J-Curve of AI value realization” and describes it as an empirically observed pattern for large transformation projects. In the early phase of adoption, productivity drops, sometimes for months. Only afterwards does it rise again, eventually growing exponentially. Three forces pull the curve down at first.

The learning curve comes first: the time teams need to master new interfaces, workflows, and prompting strategies. Then there is the “verification tax,” the time developers spend checking AI output for hallucinations and verifying it against security and architecture standards. As the third force the report names pipeline adaptation, because code produced faster piles up at testing and approval bottlenecks that used to be sufficient. The report reads these three forces as the price of learning in the transformation. It does not see a failure of the technology in them. Anyone who draws up a budget without planning for the J-curve risks cutting funds in the trough and forfeiting the later rise.

What the Implementation Costs

The sample calculator puts explicit figures on this investment block. It assumes 500 technical employees, fully loaded annual compensation of $176,000, a J-curve duration of three months, and a 15 percent drop in productivity. The price of learning alone amounts to $3.3 million. Direct tool and training costs come on top. The sample model budgets $250 per user account per year for licenses and $80 in additional usage costs, for tokens for instance. Added to that are $9,600 in training per person and $100,000 in additional infrastructure costs. That adds up to $5.065 million. Together with the J-curve, the total first-year investment comes to $8.365 million.

What the Implementation Yields

On the value side, the model works with three items. One item is freed-up staff capacity, which the report calls “Headcount Reinvestment Capacity.” It arises when developers gain time through AI that they can reinvest in higher-value work. At 12.5 percent of net time saved per person, that equals $11 million in the example. Another item is additional revenue from more delivered features. If a team delivers 50 features a year, in future it delivers 56. A third of those have a revenue effect, at 0.5 percent of a product portfolio revenue of $100 million. That adds almost $990,000. The third item has the opposite sign. From the 2025 DORA Report comes the finding that AI adoption correlates with a higher change failure rate. In the model, the share of failed releases therefore rises from 5 to 6 percent. With a four-hour recovery time and outage costs of $100,000 per hour, that shows up as minus $344,000. Summed up, the total first-year value is $11.646 million.

Behind the model’s three-part structure stands a broader value framework, which the report describes in the chapter on value calculation with five pillars: cost efficiency, productivity, developer experience, user experience, and business growth. The further right one moves in the model, the weaker the direct link to AI in engineering becomes, and the stronger the financial leverage. In a chart (Figure 4), the report makes visible what the 2025 DORA Report measured empirically. The largest effect of AI adoption shows up in individual effectiveness, followed by higher delivery instability (with the opposite sign), then organizational performance, meaningful work, code quality, product performance, delivery throughput, and team performance. Burnout and friction barely change. The spread of effects is wide, and the impact depends on the system in which the tool operates.

The report devotes a separate passage to developer experience. It expressly keeps it in the model but, out of caution, takes it out of the sample calculation. The link between AI adoption and staff retention is real, it says, but statistically too variable to put into a point estimate. As a qualitative argument it remains strong, because replacing a software professional typically costs one and a half to two times the annual salary. Anyone who creates a better working environment because routine work is offloaded to agents lowers those replacement costs. In the sample calculation the report intentionally leaves this item out, but in its argument it carries it as a “powerful qualitative lever.”

From these two sides of the calculation follows an ROI of 39 percent in the first year and a payback after 0.7 years, or roughly eight months. The report puts that magnitude in context. A payback between six and nine months counts as the benchmark for agile teams, between twelve and eighteen months for large organizations shaped by regulation. For a longer perspective the authors point to a survey of their own. Google Cloud customers reportedly achieved an average return of 727 percent on their AI investment over three years. The figure comes from a Google Cloud publication. The report labels it as a customer report without independent validation.

Methodological Notes in the Text

In several places the report itself points to the limits of the calculation. The methodology box states that the calculations are “a highly uncertain estimate, intended to spark a conversation, not as a rigid mathematical formula.” The authors cite the statisticians’ maxim that all models are wrong but some are useful. Anyone who changes individual assumptions in the calculator sees the end result tip immediately. A conservative variant lowers the value to 80 percent and raises the costs to 150 percent, an optimistic one goes the other way. The range is wide. The authors expressly recommend running several scenarios at the same time and advise against trusting any single point estimate.

Market Fragmentation

In a longer passage, the report asks why the financial results diverge so widely when adoption is nearly universal. It describes three camps. The optimistic position relies on a Google Cloud survey in which 78 percent of the executives polled reported a return from at least one generative AI use case. It also cites 88 percent positive feedback among early adopters of agentic systems. The neutral position points to the Stanford AI Index 2025, which classifies expectations for workforce productivity as “consistently mixed.” Adoption is high, it says, but structural transformation remains rare in most industries and the productivity gains are mostly marginal. The pessimistic camp cites research from the MIT-NANDA project. According to it, internal enterprise implementations often fail, which drives employees into a “shadow AI economy” in which they switch to unauthorized consumer applications. The main hurdle, according to the MIT finding, lies in organizational design. Neither budget nor technology is the obstacle.

Somewhat hidden in the chapter on ROI modeling stands a notable figure. An analysis from Stanford’s research program on software engineering productivity reportedly showed that AI tools produced a productivity gain of 35 to 40 percent on simple greenfield tasks. On complex, aging legacy code, the effect was often ten percent or less. The report’s authors use this difference to justify the selection of use cases. Those who deploy AI primarily in greenfield scenarios see results sooner. Brownfield migrations need more groundwork on the engineering system. The agentic engineering pipeline for the SDLC shows what this groundwork can look like in a brownfield codebase.

Five Structural Prerequisites

In the chapter “Build the organizational foundation for AI adoption,” the report names five systemic keys meant to lead a company out of scattered local productivity gains. Trust comes first, implemented technically as a “clear and communicated AI stance.” This means a written organizational stance on AI that sets out expectations, boundaries, and review obligations. Next is an internal developer platform (IDP), which is treated as a product and minimizes friction in dealing with tools, pipelines, and architecture patterns. In the agentic era, the report says, the IDP acts as a “risk mitigator and context provider for agents.” An AI-accessible data ecosystem is the third prerequisite, because agents are only as good as the data they can reach. An uncompromising user focus directs the speed gained through tools at real problems. The number of pull requests is beside the point there. Last, the report names automated verification guardrails, which act as brakes and allow the engineering system to drive faster safely.

One formula in the report bundles the thesis: “We measure AI not by the code it writes, but by the bottlenecks it clears.” The statement stands in the chapter on the business case. It shifts the usual promise that AI will replace developer jobs into a different logic. ROI, the report says, measures how much latent human creative potential is released by offloading systemic routine work to autonomous agents. The number of jobs that could be cut is no measure of that.

Recommendation Against Job Cuts

Consistently, the report advises against a cost-cutting strategy on the staffing side. Anyone who converts productivity gains into job cuts damages morale, it says, reduces the willingness to learn, and creates incentives to resist process improvements. The authors recommend instead framing freed-up capacity as reinvestment in innovation. The costs saved this way are avoided costs for hires that never happened. Savings from layoffs are not what is meant. This reading is built into the model partly because replacement costs for staff in software roles typically run to one and a half to two times the annual salary.

The term “verification tax” keeps returning in the text. In the chapter on additional and indirect costs, the report describes how a low level of trust deepens the J-curve. If every block of code were read twice before entering the pipeline, the productivity gains would dissolve. The report therefore treats trust as a hard financial variable. Producing it, it says, requires a system that rewards verification. Raw code volume must not be the incentive there. Research on agentic software development describes the same shift: verification becomes the bottleneck, and the agile practices have no answer to it.

Experiment Frequency as a Financial Metric

In a separate passage, the report argues that the frequency of experiments is a financial indicator and not only an engineering value. The reasoning draws on the concept of optionality from finance. An option, it says, is a low-risk investment that opens the right to invest more later without the obligation to do so. A prototype or an A/B test is such an option. AI lowers the upfront cost of that option by drastically shortening the time needed to write code. Whoever holds more options need not commit to a single hypothesis as early.

The financial reading adds a dimension to the method that differs from pure speed. Whether a team ships more code in the same time recedes into the background. The question is how many hypotheses a team can test with real users in the same time before it puts resources into scaling. High experiment frequency, the report writes, is an early indicator that the organization has absorbed AI and is not investing in the wrong features.

Three Reservations

The report comes from Google. It contains references to Google Cloud tools and consulting services, and its sample calculator ends with a contact option for the “delta” practice. The methodological soundness of the arguments is unaffected by that, but the marketing frame is there to see.

In addition, the calculator covers only the first year. The high multi-year returns of 727 percent come from a separate source, which the report itself labels as a customer report. Finally, the central input variables remain estimates: the share of time saved, the success rate of additional features, and the revenue effect per feature. The report discloses their range openly. The authors make no secret of this uncertainty. They stress several times that the calculator is a conversation starter and not a balance-sheet instrument.

Maturity as a Financial Variable

Despite these reservations, the report offers something that has been missing from the discussion of AI tools so far. It translates the claim that the foundation counts for more than the model into a monetary figure. As long as the discussion of agentic tools stayed reduced to comparing models or the speed of individual tasks, organizational maturity was a statement about style and review culture. In the DORA report it becomes a variable in the financial model. A conservative assumption about the maturity of the internal platform lowers the total value from $11.6 million to around $9 million. An optimistic assumption raises it by the corresponding amount. Maturity is a factor in the calculation here.

The report treats a second observation almost in passing. Inference costs for the most advanced models are said to have fallen by a factor of 280 between November 2022 and October 2024. With that, the cost of querying a model approached zero, and the real costs shifted to governance, meaning verification, workflows, and qualification. Those prepared in these three areas see the exponential rise in value in years two and three that the authors describe. Prepared means a platform team, a review culture, and honest planning for learning costs. Those who are not prepared stay in the trough.

This shift explains a second finding of the report. The question of which model a team uses is mentioned in the report but consistently pushed into a supporting role. The central variable is the competence to embed the model in a workflow that rewards verification and limits risk. What still counted as a technical question in 2024 is an organizational question in 2026. The report wants to make this shift visible in a single number.

An interim assessment in the report sums up the logic soberly: “The path to ROI is a sequence of building competencies, not a race for the latest model or the latest tool.” The text ends on this statement. After 60 pages of methodology it reads almost inconspicuously, but in substance it marks a reversal. The race for the fastest model generation is losing the role it was assigned in 2023 and 2024. In its place comes the race for the most mature engineering organization. Which of the two disciplines a company masters better is not decided in the license agreement. It is decided in the internal platform, in the review routines, and in the willingness to pay the price of learning for three months without cutting funds early.

The speed of this shift can be read from a few data points. Anthropic released Claude Code as a preview in February 2025, with general availability following in May. OpenAI launched Codex CLI in April 2025 as an open-source terminal tool. GitHub announced Copilot CLI in September 2025 as a public preview, with full GitHub integration and a plan mode. In February 2026 the tool adopted the AGENTS.md instruction file on an equal footing with the older .github/copilot-instructions.md. Within a year, a category of experimental tools has become a category of standard tools in productive use. Medium-sized and large IT organisations are currently rolling the tools out in waves, often faster than the methodology in their teams can mature.

Anyone who buys a tool in this situation and hands it to their developers can watch individual tasks get finished faster. At the same time, the quality of the project work drops. A study published this year has soberly documented this finding as a measured effect. In their paper “A paradox of AI fluency”, Christopher Potts and Moritz Sudhof, both of Stanford University, analyse nearly 27,000 real AI dialogues from the WildChat-4.8M corpus. That is 1,000 randomly drawn English conversations per month from May 2023 to July 2025. They measure four axes: AI competence, interaction style, task complexity and error indicators. The result that stands out concerns the visibility of errors. In dialogues with high AI competence, 64 per cent of error indicators appear, whereas in dialogues with low AI competence the figure is only 24 per cent. Read superficially, that makes the experienced users look like the poorer ones. The second figure resolves the paradox. Among the experienced users, 59 per cent of errors are visible within the dialogue itself. Among the inexperienced users, the figure is 12 per cent. The remaining 88 per cent go into the result without anyone having started a correction loop.

For development teams this means: anyone who uses agentic coding without active steering shifts errors from “visible and correctable” to “invisible and contained in the result”. Training that only shows how to operate the tool trains the second practice. Teams need a method that embeds the tool in a control architecture. Within it, errors become visible early, a human checks every risky step, and the project state lives outside the chat history in versioned artefacts.

This is the method I have worked out over the past few weeks as a training programme. I call it the “Team Operating System for Agentic CLI”. It is organised into five areas of competence with learning units that build on one another, supplemented by a validation framework meant to ensure repeatability for follow-up courses. My aim is to build method. The method prescribes a procedure that a development team has to set up end-to-end if it wants to keep going in the age of agentic tools. The following sections describe the content of this method and place it in the industry context.

Mental model as the first prerequisite

I begin my training with a clarification that stays unspoken in many organisations. Language models answer in natural language. They say “I think”, “I remember”, “I understand your code”. Anyone who works with them every day starts treating them like colleagues, people you coordinate with and whose judgement you rely on. This personification has consequences for how the tool is used. Users give vague hints and hope the model will think along, while assumptions stay unspoken.

The corrected view is technical and, at the same time, a relief. A language model is a statistical apparatus. It takes in a context, calculates probabilities for the next token, picks one, appends it and starts over. Outside the context window it has no memory, and it understands nothing. It approximates patterns that were plausible during training. A follow-up question from the model sounds like a colleague admitting a gap, but it is a probable continuation of the text so far.

From this model follows the central lever for operating it. The quality of the context is the quality of the answer. Anyone who treats the model as an approximator supplies explicit and complete context and gives it structure, knowing that no other lever works. Anyone who treats it as a colleague leaves gaps in the context and gets answers that fill those gaps convincingly without closing them.

This clarification is the prerequisite for the method elements that follow to take hold at all.

Three pillars of a team method

On this basis I have formulated three elements that hold the method up. They are called the Artifact Contract, Review Gates and Tool Zones.

The Artifact Contract settles where the project state lives. My answer: the chat is transit. The project state lives in versioned artefacts that people can review. Markdown files for context, decisions, plans, test output and runbooks. Jira for work breakdown, status and accountability. Confluence for approved specs and documentation. Pull requests for diff, tests and the review decision. Whatever comes out of an agent session and is not carried over into one of these artefacts counts as not having happened. The rule is restrictive. Without it, justifications, interim states and risks get lost in the chat window.

Review gates add six verifiable transitions to the pipeline. Discovery ends with the question of whether the sources hold up and whether gaps and contradictions are visible. The spec ends with the question of whether the architecture and compliance decisions are sound. A ticket counts as ready only once someone has confirmed that it is small and testable, with acceptance criteria and non-goals. Before any code, a plan review checks the sequence, the test strategy, the risks and the affected files. A pull request may be merged only once diff, tests and plan are comprehensible together. A production-related action stays blocked until runbook, rollback and human approval are in place. Each of the six gates has one short, verifiable question and one clearly named person responsible.

Tool zones sort MCP and CLI tools by risk. Read-only tools such as search, read, status and log queries are free to use, with the obligation to document the result. Draft tools let the agent draft plans, tests, code drafts and PR texts that a human reviews before they take effect. Write tools such as writing to Jira, pushing a branch or creating a PR need approval. Restricted tools cover production apply, secrets, data deletion and permission changes. Those remain exclusively human actions with their own gate, even if the agent could technically carry them out.

Together, the three pillars shift the question a team asks about agentic coding. The speed of the agent gives way to the safety with which the team can work with it.

Five fields, one artefact chain

I organise the learning path into five areas of competence: Operating, Documenting, Structuring, Implementing, Running. The order is not arbitrary. Each field lowers a different uncertainty and closes with an artefact that the next field uses as input.

Operating is the entry door. Here the team learns to run a CLI session, keep sessions clean, approve tools and set the first guardrails. The resulting artefact is an operating model in the form of a file in which the tool zones for the team’s own setup are spelled out. Which MCP servers are connected, which tools run without prompting, which need human approval.

Documenting is the discipline of project research. From sources, glossaries, Confluence pages and code findings, a knowledge base emerges in which gaps, contradictions and currency are visible. The agent here is a research tool. What it finds goes through a source check, and what it does not find is marked as a gap. The artefact is a documented inventory of sources that forms the basis for every later decision.

Structuring translates knowledge into decisions. Findings become architecture decisions, regulations become a compliance matrix, requirements become an epic with tickets that meet a definition of ready. The agent may draft a great deal here, but every decision passes through a review gate before it is used further. The artefact is a project structure that implementation can build on.

Implementing is the smallest phase and at the same time the most delicate. It ends with a pull request that fully implements a single small ticket. Before the first line of code there is a plan with scope, non-goals, affected files, test strategy, risks and sequence. Only after the plan review does the agent start writing. Tests are created before or together with the code. The PR has to be comprehensible without the chat history. The artefact is a proof of delivery in which plan, test output and review note sit side by side.

Running deals with what begins after delivery. Security as an operating rule, incident diagnosis, rollout decisions, apply gates, the team playbook. The artefact at the end is a file in which the team records its rules and makes them reproducible for the colleagues who come next. Anyone who takes shortcuts here loses the lessons of the preceding fields.

What follows from this arrangement is a chain of artefacts. The knowledge base builds on the operating model, the compliance matrix on the knowledge base, a sound epic on the compliance matrix, the plan on the epic. The PR builds on the plan, and the apply decision on the PR. A team that skips a field cannot honestly close the next one.

Roles in the rebuild

I split the learning units by area of competence on purpose. Business architect, UX designer, developer, tester and infrastructure sit in the same sessions. They use the same tool and review one another. The separation of responsibility runs through the artefacts.

Business architecture works mainly on discovery, compliance and slicing epics. It sifts through legacy documents, generates spec proposals and writes tickets in Jira. In the classic model this role was heavily document-based and often the project’s bottleneck when it came to speed. With an agent that can summarise Confluence content and propose spec drafts, the effort shifts from producing text to evaluating it. The role becomes an editor for the machine.

UX explores concepts, tests assumptions against behaviour and documents decisions in Confluence. Here the effect of the tools has so far been more modest. Visual concepts and prototypes are still created in specialised applications, but the accompanying documentation and the reasoning behind a decision can be sped up with agents.

Development runs through exploration, plan, implementation and PR. I stress the order explicitly. Before the code comes the plan, before the plan the question, before the question the review of code findings. Anyone who lets the agent write without a plan risks PRs that tests do not cover, that violate architecture patterns or that widen the scope unannounced. I have designed a lesson template for this phase in which this plan is formalised as a reviewable Markdown artefact.

The tester generates test cases from the spec and has a browser test runner execute E2E tests in the browser via the Model Context Protocol. Afterwards the tester writes the results back to Jira. Its importance grows because agent-generated code by definition demands more tests. The plan that precedes the code lists tests that check the plan.

Infrastructure takes care of pipelines, deployments and monitoring. It supplies a playbook that governs apply decisions: it writes down when an action may be approved and under which conditions it stays blocked. The role thereby moves closer to that of a platform engineer, who provides not only tools but also rules against which the agent can be checked.

All in all, the work of every role shifts. What remains is the responsibility for the artefacts at the end of the phase.

Toolchain and the Model Context Protocol

At the centre of the toolchain I describe in the method sits the agentic CLI with its attached MCP servers. Five stations group around it: Confluence for specs, Jira for tickets, Bitbucket for code and pull requests, Playwright for E2E tests, the customer for feedback and acceptance. The link between agent and stations runs over the Model Context Protocol, a specification proposed by Anthropic in 2024 that several vendors now support.

What this picture means becomes clear in comparison with the previous toolchain. Anyone who has set up continuous integration and continuous delivery over the past ten years knows the friction between the tools. Requirements lived in one system, tickets in a second, code in a third, tests in a fourth, deployments in a fifth. A human translated between them. The translation cost time and got lost. Nowhere was it documented in full.

A pipeline in which an agent is attached via MCP servers to requirements, tickets, code, tests and deployment closes this gap on the tool side for the first time. A ticket from Jira becomes a task for the agent, the plan lands as Markdown in the repository, the code goes into a PR. The tests run automatically, the result flows back into the ticket. What counted as a broken toolchain can become a continuous process, but only under conditions that I set explicitly in the training. MCP servers are not neutral. With write access, the agent can overwrite approved specs, close tickets and merge PRs. The tool zone architecture makes sure these paths of action are not left open.

Repo rules and security

In the advanced part of the method I deal with the question of how an agent keeps to the rules of a specific project. The answer is a file at the repository root whose name differs by tool. AGENTS.md has established itself as the canonical choice. GitHub Copilot CLI reads this file on an equal footing with the older .github/copilot-instructions.md. Alongside it there are fallbacks for CLAUDE.md, GEMINI.md, path-specific rules in .github/instructions/ and personal configuration under $HOME/.copilot/.

The principle is a repo constitution. Whatever the agent must not guess, meaning build commands, architecture decisions, local deviations, tool limits, belongs in the rules file. Whatever a linter or test enforces better does not belong there. Tutorials and marketing comparisons stay out.

One subtlety that I spell out in the training does not appear in every vendor’s documentation. If the same repo contains several parallel instruction files and they contradict each other, the resolution is not deterministic. The tool has to pick one, and the choice can vary between two sessions. The hard rule that follows: one truth per rule. Anyone with parallel instruction files has to keep them consistent in content.

The other critical topic is security. In the Running block I treat tool agents as a risk when three factors come together. Untrusted input is everything that flows into the agent from outside: tickets, logs, web pages, MCP responses, skill files. Sensitive data means secrets, customer data, production logs and internal architecture details. Effective tools are all those that can write, reach the network or act close to production.

The rule is a dividing line. Untrusted input counts as a finding. An instruction that sits in a ticket is input material and does not automatically become a task for the agent. Sensitive data is subject to a need-to-know principle, meaning minimisation, masking or separate processing. Effective tools need allowlists, a clear purpose and a human gate. I state this explicitly as an operating rule. Anyone who pushes security to the end of the learning path as a chapter of its own has already lost it.

Industrial software production or a fast workshop?

A claim is circulating in the industry that I have included in the method section on purpose, to put it up for discussion. With the continuous pipeline of agentic CLI, MCP servers, Confluence, Jira, Bitbucket and Playwright, so the sentence goes, the industry comes a giant step closer to the dream of industrial software production. I examine the claim because it shapes the promise right across the industry.

Industrial production lives on two properties: repeatability under changing input conditions and verifiability of every step. A screw off the line can be traced back to batch, material, machine and shift. A defective batch gets recalled. Software production has rarely been close to this model. It was craft, then workflow with continuous integration and code review. The chain from requirement through spec, code, test and deploy to customer feedback ran over tools that spoke different languages. People translated between them.

The agentic pipeline closes this translation gap on the tool side. That is a substantial change. Whether it leads to industrial production depends on three caveats.

First, verifiability. If the agent decides something in the chat and the decision does not land in an artefact, the reasoning is lost. The pipeline is continuous. The audit is only continuous if the team keeps to the Artifact Contract with discipline. Second, repeatability. Language models are not deterministic. The same prompt with the same context does not necessarily produce the same plan. Industrial production lives on exact repetition, agentic production on the range of acceptable variants. Third, quality assurance. Where a machine checks a screw, here a human checks a plan or a diff. The check remains, just in a different place.

I meet these three caveats with structure. The Artifact Contract secures verifiability, the review gates secure the repeatability of the quality threshold, though not that of the outputs. The tool zones prevent quality checking from being delegated to the tool. What emerges is reviewable software work at the speed of an agent and with the auditability of a team. I hold that wording to be more accurate and more honest than the advertising slogan of industrial software production.

What happens if a team changes nothing

The situation is asymmetrical. Anyone who does not use tools such as Copilot CLI or Claude Code loses speed against competitors who do. Anyone who uses them without building a method like the one described loses auditability, review culture and, in many regulated industries, the ability to comply. Both paths have costs. Both costs can currently be observed in the rollout at large companies.

My programme aims at the third path: use the tools, build the method, demand the artefacts, enforce the gates. I supply a procedure that is reproducible within an organisation and with which a team can pass on its way of working in writing.

The success of this procedure rests on three conditions. Managers have to take plan reviews seriously and schedule time for them, because a method that lives only during the training week is wasted time. Tool configurations have to enforce the tool zones technically. A writing connection to a production system without a human gate contradicts the method. Then there is iteration. Models and tools change, MCP servers get added. The playbook lives on the team revising it at least quarterly.

Outlook

Over the next two years, agentic coding will shape the industry more deeply than agentic editor plugins did between 2022 and 2024. Vendors such as GitHub, Anthropic, OpenAI and smaller specialist houses are competing over permission models, hooks, subagent systems and MCP extensions. The question of who has the best model is increasingly giving way to the question of who builds the best operating model around the model. The study by Potts and Sudhof describes the finding on the user side, and my programme draws the consequence for the team side.

Development teams that set up today what the five areas of competence prescribe create the conditions to grow with each new generation of tools without losing their review culture. Teams that hope the tool will bring the method with it will find the 88 per cent figure of invisible errors in their own deliveries. The tools are here. Whether they turn into project work depends on the method.

The architecture addresses a point in the debate surrounding so-called agentic AI that often remains in the background: language models such as GPT, Claude or Llama do not execute actions themselves. They suggest tool calls. The surrounding host, such as a library, a plugin or a command-line programme, handles the actual execution. AEGIS operates at this interface.

The boundary does not lie within the model

An autonomous agent has tools with which it can read and write files, execute shell commands, send HTTP requests or process tickets. The model does not possess these capabilities itself. It can merely formulate tool calls. Whether any effect follows is for the host to decide.

A common method is to provide the model with a tool called aegis_check and instruct it in the system prompt to call this before every action. This method is a convention. It can fail if the context becomes large or the model ignores the instruction. A hard boundary only arises where the host actually executes the tools.

The analogy with classic operating systems is obvious. A user-space process does not open a file by linguistically persuading the kernel. It issues a syscall, and the kernel checks parameters, permissions and state. Following this pattern, AEGIS sits in the host’s tool path as a syscall boundary and holds no conversation with the model.

Anatomy of a request

The transition from the model’s suggestion to the actual action is implemented as an HTTP call to a local sidecar service at POST /v1/check. The payload is structured and contains no natural language:

{
  "action_type": "shareIntelligence",
  "agent_id": "intelligenceAgentInMission",
  "proposition": {
    "classification": "secret",
    "recipient": "externalService"
  },
  "context": {
    "tool": "send_external_message",
    "session_id": "redteam-17"
  }
}

AEGIS checks the agent, the action type, the proposition and, optionally, a context. The host translates the tool call into the formal concepts of the loaded domain, in the example the “IAMission” domain with military secrecy rules. AEGIS carries the adjective “deontic” because its rule set operates within the logic of duty: a vocabulary of permitted, prohibited and required actions.

Its answer is just as terse:

{
  "decision": "FORBIDDEN",
  "reason_type": "EXPLICIT_NORM",
  "justification_chain": [
    "IAMissionCode forbids sharing secret intelligence with externalService"
  ],
  "norms_applied": [
    "IAMissionCode:shareIntelligence-secret-external"
  ],
  "action_type": "shareIntelligence",
  "agent_id": "intelligenceAgentInMission",
  "explanation": "FORBIDDEN by applicable IAMission rule"
}

The system knows three verdicts: PERMITTED executes, FORBIDDEN blocks, UNDECIDABLE blocks as well, because ambiguity does not count as permission.

Tool calling alone is not enforcement

Function calling and tool use, as defined by OpenAI and Anthropic in their APIs, are the standard form of interface between model and tool. A strict tool schema with action_type as an enumeration type and a proposition limited to known fields forces the model to provide structured suggestions instead of free text.

Such a schema is not enough to enforce actions, though. If the host provides other unchecked tools alongside aegis_check, the agent can select them. That is why AEGIS distinguishes between transparency and enforcement.

aegis_check as a visible tool is transparency: the model can ask whether a planned action would be permitted and adjust its strategy. Enforcement goes deeper. In the integration with OpenCode, an open-source coding agent, I have anchored it in two places by way of example: the permission.ask hook provides the primary verdict, and tool.execute.before acts as a second layer. In AEGIS’s own orchestrator, a component called ActionExecutor checks before every tool call whether the guard agrees.

This leads to an architectural rule: every tool that has an effect on the external world passes through the same bottleneck. The architecture does not rely on the model’s choice of tool.

MCP: Interface without automatic enforcement

With the Model Context Protocol (MCP), a specification for AI tools proposed by Anthropic, the picture is different. An MCP server can offer tools and resources. AEGIS could use this form and expose something like check_action or access functions such as read_via_broker via MCP.

It comes down to whether an MCP server can intercept the host’s existing tools or only provide additional ones. In the first case, AEGIS sits in the tool path. In the second case, it runs in parallel: the agent can ignore the AEGIS tool and choose another.

The Host Contract

The architectural principle of AEGIS is this:

In concrete terms, this means: any tool that can modify files, send data externally or achieve a comparable effect via the shell is either secured by the Guard or restricted in some other way. The comparison with traditional database perimeters is obvious: an API is not secure if a second access point without authentication is running alongside it.

Fail-closed means that the host does not execute any action if the Guard connection fails. An implementation that permits such actions no longer has a perimeter at this point. The AEGIS OpenCode plugin sets this behaviour as the default.

The Rejection Loop

Blocking an action is technically trivial. The more instructive part is the mechanism that lets the agent carry on after a rejection, the so-called Rejection Loop. It is implemented in the AEGIS orchestrator and distributes information unevenly on purpose.

A verdict contains several fields. Besides the decision (PERMITTED, FORBIDDEN, UNDECIDABLE), a reason_type classifies the reason according to a list of nine categories. EXPLICIT_NORM stands for a specifically applicable rule, CWA_NO_PERMISSION for a missing permission under the closed-world assumption, NO_JURISDICTION for actions outside any loaded domain, UNRESOLVED_CONFLICT for conflicting norms, MORAL_AXIOM for ethical principles that cannot be overridden. Four more cover validation and runtime errors. On top of that come a justification_chain with the engine’s reasoning steps and norms_applied with the identifiers of the rules involved.

Not all of these fields reach the model. The orchestrator calls a method named explain_safe(), which returns only the decision and the reason, no norm names and no reasoning steps. The full justification, including justification_chain and norms_applied, ends up in the audit log. It does not appear in the chat history. A component called RefusalRegistry also generates two different texts from the reason_type: one for human recipients and one for the model. So each of the three recipients gets its own level of detail.

The loop itself runs as follows: the agent proposes an action, the host translates it into an Action, AEGIS checks. If PERMITTED, the host executes the action and returns the result to the model. If the verdict is FORBIDDEN, the host executes nothing, appends a tool response to the conversation history and calls the model again. On UNDECIDABLE, the host hands over to a human immediately. The loop does not attempt another call, because an undecidable situation does not become decidable through repetition.

For a FORBIDDEN with reason EXPLICIT_NORM, the tool response the model gets to see takes the following form:

{
  "decision": "FORBIDDEN",
  "explanation": "Decision: FORBIDDEN\nReason: EXPLICIT_NORM",
  "executed": false,
  "suggestion": "Your proposed action was explicitly forbidden by a governing norm. Propose an alternative that does not violate the applicable rules."
}

A fixed set of templates indexed by reason_type supplies the suggestion text. For CWA_NO_PERMISSION, the message says in substance that no permission was found and that under the closed-world assumption the absence of permission is to be read as a prohibition. For UNRESOLVED_CONFLICT, the template recommends escalating to a human.

Not every rejection allows a retry. For MORAL_AXIOM, the suggestion text contains the explicit instruction “No override is possible. Do not retry. Inform the user.” For INTERNAL_ERROR, the text reads “Do not retry the same action. Escalate to a human operator.” The loop thereby distinguishes between rejections that an alternative proposal can cure and rejections that mark a hard stop. After a curable rejection, the model sees only the reason_type and the suggestion text and can formulate an alternative proposal on that basis. After a hard stop, it reads the instruction not to retry. Whether it complies is another matter. That is why the host, for its part, executes nothing that is not permitted.

The orchestrator counts the rejections. If the number exceeds a configurable threshold (the prototype sets it to three), the loop terminates. The host produces a final message of the form “Action forbidden after 3 attempts. [user_message] Escalating to human operator.” and hands over to a human. A further iteration limit in the outer loop guarantees that the loop terminates in any case and does not get stuck in an endless cycle of proposal and rejection.

The design rationale behind the asymmetry between audit, model and human: rejections can themselves give away information. A response of the form “prohibited due to source X” confirms the existence of X. The asymmetry is a consequence of this side-channel consideration: the auditor sees the full path, the model sees the reason_type without norm names, and the human recipient sees only a generic sentence.

Three examples

Three use cases illustrate the behaviour:

A coding agent wants to delete a file. AEGIS blocks this because the loaded domain prohibits deletions. The agent submits a patch instead.

An intelligence agent wants to send confidential data to an external service. AEGIS blocks the action. The agent proposes an internal summary to an authorised recipient.

AEGIS also blocks the risky shell command that a DevOps agent wants to execute. The agent builds a tested variant.

The difference from a classic permission prompt, which asks a human “May I?”, lies in the addressee of the request. AEGIS consults a formal rule set and provides the agent with a reason. A human is only involved if the Guard responds with UNDECIDABLE or the loop has been exhausted.

What AEGIS does not check

AEGIS only checks the actions that the host presents to it. Beyond this boundary lie unchecked tools, incorrectly translated tool arguments and gaps in the formal rule base. These points describe the actual requirements for an integration. The Guard’s interface is simple: an HTTP endpoint, JSON in, verdict out. The contract with the host is tight: nothing with any effect bypasses the Guard.

When you hear “deontic logic”, you quickly think of university seminars, modal operators and proofs at the blackboard. Then of runtime, of milliseconds that add up to seconds when a hundred agents want something at the same time. But it’s not that bad. The algorithm is a sorted list traversal with a few pairwise comparisons. There is neither a theorem prover nor a SAT solver like Z3 behind it.

The scenario

Back to our hospital example. The compliance officer has laid down three rules:

1. Nursing staff are not permitted to view patient records.
2. During night shifts, when no doctor is available, nursing staff are permitted to view patient records.
3. Medical data is never disclosed to external parties.

The third rule is a moral axiom and stands above everything else. The first two are context-dependent and conflict with one another. In MELD, this looks like this:

; Rule 1: General house rule
(forbiddenToDo-WRT HospitalCode nursing
    (readPatientRecord patientRecord))

; Rule 2: Exception for the night shift
(permittedToDo-WRT EmergencyCode nursing-nightshift
    (readPatientRecord patientRecord))

; Rule 3: Moral axiom
(forbiddenToDo nursing
    (forwardData medical external))

A nurse on night duty calls: she wants to view the patient record. The Guard receives the request and now has to decide: PERMITTED, FORBIDDEN or UNDECIDABLE. The algorithm runs in this order.

Step 1: Who does the rule actually concern?

Before I talk about conflicts, the engine filters the rules that are actually applicable. A rule is applicable if it concerns the agent and describes the requested action. So the engine asks: Which rules address this specific request?

applicable = [
    n for n in norms
    if n.matches_agent(agent)
    and self._proposition_matches(n, proposition)
]

matches_agent uses the ontology. If the rule applies to nursing and the agent is nursing-nightshift, and the ontology states (genls nursing-nightshift nursing), then the rule matches. nursing-nightshift is a specialisation of nursing. Inheritance works downwards.

For our scenario, two rules remain after the filter: Rule 1 and Rule 2. Rule 3 refers to a different action (forwardData) and is not applicable.

Under the hood, proposition_matches works with unification. That is why a rule with the pattern ("readPatientRecord",) also matches a request with additional parameters, which a string comparison would not. This is a Prolog technique from the seventies, applied in a new way. The filter itself is O(n). With ten thousand rules per domain, this still amounts to microseconds, because each rule is handled with a dictionary lookup and a tuple comparison.

Step 2: Axioms always win

Before anything else, the engine separates out the moral axioms. In the NormFrame data structure, there is a flag defeasible: bool. True means overridable, False means axiom.

axioms = [n for n in applicable if not n.defeasible]
defeasible = [n for n in applicable if n.defeasible]

if axioms:
    # Are there conflicting axioms? Then UNDECIDABLE.
    if detect_conflicts(axioms):
        return NormStatus(modality=None, reason="contradictory_axioms", ...)
    # Otherwise: the axiom wins, full stop.
    return NormStatus(modality=axioms[0].modality, reason="moral_axiom", ...)

An applicable axiom decides, without context dependency. If two axioms contradict each other, the engine returns UNDECIDABLE, and the surrounding infrastructure blocks the action. Fail-closed. I’d rather live with a false positive than with an action that should never have happened. In our scenario, there is no axiom in the filter set. Rule 3 deals with passing on data and says nothing about reading it. The engine continues.

Step 3: Specificity as an organising principle

Rule 1 and Rule 2 contradict each other. Rule 1 prohibits, Rule 2 permits. Both can be overridden, both are active, and I must decide which one wins.

DDIC replies: the more specific one. Specificity here is an integer. Each rule has a field specificity: int, which is calculated from the depth in the inheritance hierarchy when the MELD file is read in. nursing has depth 2, nursing-nightshift has depth 3. The rule for nursing-nightshift is more specific.

# Sort defeasible norms by specificity, descending
applicable_sorted = sorted(
    defeasible,
    key=lambda n: (n.specificity, code_rank, n.code, n.modality.value),
    reverse=True,
)

# Preemption: a more specific rule defeats a more general one
for i, a in enumerate(sorted_norms):
    for j, b in enumerate(sorted_norms):
        if j <= i or not a.conflicts_with(b):
            continue
        if a.specificity > b.specificity:
            defeated.add(j)
        elif b.specificity > a.specificity:
            defeated.add(i)
            break

A double-loop comparison. O(n²) in the worst-case scenario, rarely relevant in practice because n (the number of applicable rules after filtering) is almost never more than a dozen. For every applicable rule against every other: is there a conflict, and if so, which is more specific? The less specific one is discarded.

Rule 2 (specificity=3) beats Rule 1 (specificity=2). The engine responds PERMITTED.

Incidentally, the sort key is a tuple: (specificity, code_rank, code, modality_value). The latter fields act as tiebreakers and guarantee determinism even in the event of a tie. The conflict check conflicts_with, in turn, is a set comparison of the modalities. PERMITTED and FORBIDDEN conflict, OBLIGATORY and PERMITTED do not, because obligation implies permission. The core of Olson’s calculus fits into three lines of Python at this point.

Step 4: When specificity is not enough

What do I do if two rules are equally specific? For example, one comes from the HospitalCode and the other from the GDPRCode. Both apply to doctor, both concern readPatientRecord, one permits it and the other prohibits it. Same depth in the hierarchy, no pre-emption.

At this point code precedence applies. At startup, the engine receives a list code_prevalence: ["GDPRCode", "HospitalCode"]. Earlier entries win against later ones. Legal rules rank above house rules.

def _resolve_by_prevalence(self, norms):
    by_modality = group_by_modality(norms)
    best_modality, best_rank = None, -1
    for modality, mod_norms in by_modality.items():
        for n in mod_norms:
            rank = prevalence_rank(n.code)
            if rank > best_rank:
                best_rank, best_modality = rank, modality
    # Check if the rank is unique
    if tie_exists(best_rank, by_modality):
        return None  # UNDECIDABLE
    return by_modality[best_modality]

If the precedence is clear, the higher-ranking code wins. If two rules come from the same code and are still contradictory, or if two codes are of equal rank, what remains is UNDECIDABLE.

Step 5: UNDECIDABLE

return NormStatus(
    modality=None,
    reason="unresolved_conflict",
    justification_chain=tuple(chain),
)

modality=None. The engine says neither “I suspect FORBIDDEN” nor “I’ll take the first rule”. It says: I cannot decide. The surrounding infrastructure interprets this state as fail-closed. The action does not go ahead. A human takes a look.

I believe this is the part that many people who build rule sets underestimate. Most systems have an implicit default: allow when unclear, or prohibit when unclear. Both are wrong in critical contexts. The first approach opens up loopholes that nobody has documented. The second produces false alarms that nobody takes seriously anymore. UNDECIDABLE is a third state that says explicitly: the rule set is incomplete here. The problem then lies in the specification.

How do you know all this is correct?

An algorithm that decides on the behaviour of AI systems must be demonstrably correct. “Looks right” is not enough. I underestimated how much work this proof involves. The work lies less in the engine itself than in everything around it, the part that shows: it does what it is supposed to do.

AEGIS has a four-stage verification chain for this. Each stage catches a different class of errors. From concrete examples in the research literature to a machine-verified model.

Stage 1: The examples from Olson’s dissertation

The entire algorithm is derived from Taylor Olson’s work at Northwestern University. His dissertation, “A Formal Theory of Norms”, from June 2025, is the reference. It contains four concrete example scenarios with which Olson explains his calculus. A patient named Karli who tells a care robot which information it is permitted to pass on to whom.

• Karli says, “You may tell my husband which medicines I am taking.” A single permission, no prohibition. The robot may disclose the prescription. If asked about the entire medical record, there is no applicable rule. The robot must reply: I do not know. This is the fail-closed case.
• Karli says, “You must share my health status with the children.” Later: “My medical records must not be disclosed.” The later, more specific prohibition overrides the earlier obligation. FORBIDDEN wins.
• Permission for the entire medical record, but an explicit prohibition on sharing the list of medications with her husband. For general enquiries, the permission applies. For the specific enquiry, the prohibition applies. The order of the rules is irrelevant.
• Permission to disclose the medical records. Prohibition on upsetting her husband. Two rules that meet at an intersection where neither subsumes the other.

These four scenarios are represented as Python tests. If any of them fails, the implementation has not understood Olson’s calculus at this point.

Stage 2: Olson’s complete conflict matrix

Olson lists the expected outcome for every conceivable combination of norm type and subsumption relation. It is his own summary of what his calculus must do.

Stage 3: Exhaustive combinatorics

Olson’s table covers the conflict cases he describes theoretically. But a real engine gets not only the cases the author had in mind, but everything that humans or LLMs write into a rule set.

So the third test stage is tied to the engine’s own parameter space, independent of Olson’s theorems. The algorithm has four input dimensions that determine the result:

• Modality pair: Which two modalities are in opposition? (Permitted vs. Forbidden, Forbidden vs. Obligatory, and so on.)
• Specificity relation: Is the first rule more specific, the second, or are both at the same level in the hierarchy?
• Defeasibility combination: Are both rules overridable, is one an axiom, are both axioms?
• Code relation: Do both rules originate from the same code, or from different ones, and if different, which takes precedence?

Six times three times four times three gives two hundred and sixteen combinations. The expected result is derived programmatically from Olson’s rules.

Two hundred and sixteen is the complete enumeration of the relevant parameter space. If a cell fails here, I not only know that something is wrong, but exactly which constellation is causing the problem.

Stage 4: The TLA+ model

Tests check specific inputs. They do not check that there is no input that causes a problem. For that you need a formal model.

TLA+ is a formal specification language by Leslie Lamport, designed to describe concurrency and state machines in a machine-checkable way. The model describes the six-step algorithm as a state machine. Each step is a transition between states. On top of that, invariants are formulated, statements that must be true in every reachable state. For example:

• If a consistent axiom exists, the result is always the modality of that axiom. A moral axiom must never be overridden by an overridable rule.
• If all applicable rules are FORBIDDEN, the result is FORBIDDEN.
• If a more specific permission conflicts with a more general prohibition, the permission prevails.
• If a more specific prohibition conflicts with a more general permission, the prohibition prevails.
• In the event of equal specificity, the same code and a conflict, the result is UNDECIDABLE.
• The algorithm terminates for every input.

The TLC Model Checker is the tool that executes a TLA+ specification and searches the state space. It tries out all reachable states. For each state, it checks whether the invariants hold. If it finds a counterexample path, it displays it. For up to four simultaneously active norms with three modalities, four specificity levels and two codes, this is a finite but large set. TLC checks it completely.

Supplementary property tests

Then there are tests that target individual properties of the engine. No new scenarios, but they harden the algorithm against regressions. The most important is axiom monotonicity, the guarantee that an axiom is never defeated by an overridable rule. Hypothesis is a Python library for property-based testing. Instead of fixed examples it generates random inputs automatically and searches for counterexamples. With it, four hundred random scenarios are generated, each containing between one and fifty overridable rules plus one axiom. The invariant: the axiom wins. No counterexample in any run.

The rest is bookkeeping, but important: Determinism (same input, same output, even across parallel threads and randomly shuffled norms), Totality (the algorithm terminates, always), the duality forbidden(X) ↔ ¬permitted(X), the Closed World Assumption for registered actions without a permission rule. Each of these properties has its own test suite, and every one of them passes. It is painstaking work, and it is what later gives you the confidence to commit a change to the algorithm without lying awake at night.

What the verification chain achieves

The individual stages can be seen as filters, each catching a different class of errors. Stage one catches misunderstandings of the theory: if Olson’s examples cannot be reproduced, his calculus has simply not been understood. Gaps between the examples are caught by stage two, because Olson’s table covers cases that he does not explicitly go through in the individual examples. The combinatorics in stage three then catches the gaps between the table rows, the non-conflicting pairs and defeasibility combinations about which Olson says nothing theoretically. In stage four, the TLA+ model catches the errors that no specific input triggers, the invariants that must hold in all states. The supplementary tests act as a safety net against regressions in individual properties.

Together, this amounts to over three hundred and sixty formal tests, plus the TLA+ model and the equivalence tests between the two evaluators. None of this replaces a universal proof. But each stage rules out classes of errors, and that is more verification than most production systems have.

Confidence in the algorithm rests on the fact that it reproduces Olson’s published examples, satisfies his table row by row, runs through the entire parameter space exhaustively and holds against the formal invariants under TLC. This verification chain is documented and can be reproduced at any time.

What this means in microseconds

In our scenario, the engine runs in under ten microseconds because the algorithm does so little: filtering a list, a bit of sorting, a few pairwise comparisons. The speed of Python has nothing to do with it.

That means you can run every single action of an AI agent through this guard without the agent becoming noticeably slower. The bottleneck is the agent itself, because an LLM round trip takes several orders of magnitude longer than the DDIC evaluation.

Since then, large companies have been under a level of pressure I have not experienced before. The tools promise enormous productivity gains, speed and profit. The promise is not empty: the tools do deliver. Individual developers become measurably more productive and write pull requests faster. Code that used to take days now appears in minutes.

The blind spot

Those who deal with standards, governance, compliance or structured process models in a large company no longer control the introduction of these tools. They now only accompany it. The pressure from the business units is too great.

Security measures and best practices built up over years? Many of them are suddenly just a memory once CLI agents work directly in the console, writing code, modifying files and executing commands. The tools operate beneath the layers of protection we introduced over the years for our existing tools.

Keeping this balance between speed and control is a struggle. In my professional environment I watch it fail in many places.

From nails to screws

I know the comparison is worn. Still, it fits here because it gets to the heart of the problem.

Our developers and architects are handed a powerful toolset without sufficient experience and without sufficient literature on best practices. They also lack the structured framework they could have learnt at university: that framework has only been around for a few months. How is the tool used? Exactly the way a tradesperson accustomed to hammer and nails handles the first screws.

I have followed the development of these tools since the beginning and use them intensively myself. In my previous posts I have covered individual points: why AI-assisted development requires more planning, who can still check the last 10%, why context is the real problem. What I described there for the work of the individual developer also applies to organisations, only on a different scale.

What DORA has found

A research report that I recommend to anyone dealing with the topic shows that I am not alone in this assessment: Google’s DORA AI Capabilities Model 2025. The report is based on qualitative data and survey responses from nearly 5,000 technology professionals worldwide.

The key message is surprisingly blunt in its clarity:

AI is an amplifier. It amplifies the strengths of high-performing organisations and the dysfunctions of weak ones.

“The greatest returns come not from the tools themselves, but from investing in the foundational systems that amplify AI’s benefits.”

Put simply: those who are well organised get even better with AI, and those who work chaotically produce chaos faster with AI. The tool on its own only accelerates what is already there. In February 2026 the DORA team translated this thesis into a monetary figure in a report on the return on investment.

Seven capabilities

DORA identifies seven capabilities (organisational competencies) that count. None of them is a model or a tool:

Clear AI policy: Uncertainty about which tools are permitted and how they may be used paralyses organisations, either through fear or through uncontrolled proliferation. Both are bad. DORA research shows that organisations with a clear, communicated AI stance achieve measurably better results: greater individual effectiveness, higher throughput, less friction. Organisations without a clear policy? There, the effect of AI on organisational performance is not even statistically detectable.

AI-accessible internal data: Generic AI tools produce generic code. Only the connection to internal codebases, documentation, style guides and architectural decisions turns a chatbot into a specialised assistant. DORA describes the transition from prompt engineering to context engineering, and anyone who has read my post on context knows how pleased I am about this.

Small batches: This is textbook knowledge. The data makes it concrete: AI coding tools tempt developers to generate large amounts of code at once. The DORA research shows that the discipline of small batches amplifies the positive effect of AI on product performance and reduces friction. For teams running large batches, the effect of AI on product performance is not detectable. An interesting side finding: teams working in small batches report slightly lower individual effectiveness. Breaking work into small pieces takes time and feels slower, but the products get better.

Strong version control: AI-generated code is not deterministic. Each run can produce different results. Frequent, small commits and the ability to roll back quickly become the safety net. DORA recommends including AI prompts and agent configuration files (such as CLAUDE.md) in version control. 21% of respondents already do this.

Quality platforms, user-centredness and healthy data ecosystems are the remaining three. Each one deserves its own post. What they have in common: these are the same capabilities that have distinguished high-performing teams from mediocre ones for years. AI has not changed that, only made it more visible.

Seven team profiles

One part of the report particularly caught my attention. DORA has identified seven team archetypes from the survey data. The range runs from “Foundational Challenges” (10% of respondents, survival mode, high burnout rates) through “Constrained by Process” (17%, stable systems but stifling processes) to “Harmonious High-Achievers” (20%, low friction, high performance across all axes).

The uncomfortable part: only the top two clusters (Pragmatic Performers and Harmonious High-Achievers, 40% together) can benefit from AI adoption right away.

My recommendation

My recommendation for companies wishing to roll out AI coding tools on a broad scale is not particularly spectacular. It can be implemented in a matter of days.

Before the tools are rolled out on a large scale, a minimum process model is required. A pragmatic framework tailored to the company’s needs. Which tools are permitted? How is AI-generated code reviewed? What security requirements apply? How large can batches be?

This can be worked out in less than a week. Seriously, in days.

This framework then has to be taught, as training in which developers and architects build a basic understanding of the methodology, the possibilities and the security measures to be observed. A barrage of PowerPoint slides will not do. Even experts have only had the competence to use these tools safely for less than a year. Expecting developers to pick this up on the side is negligent. The training programme for development teams describes what such training has to cover.

What cannot be delegated

The DORA Report concludes with a sentence that I want to include here because it sums up the topic better than I could:

“AI, in and of itself, is not assured to be beneficial. It must be adopted with care.”

AI can change software development, but only for organisations prepared to build the systems, culture and practices in which it can work. Those who bet on the tool and ignore the organisation will find that speed without stability is just accelerated chaos.

The DORA AI Capabilities Model Report 2025 is available free of charge. Read it. Every word in it is valuable.

I have started to build such a guard. As is fitting, it also has a name: AEGIS, Architectural Ethics Guard for Intelligent Systems. I will write in more detail about the path there and the building blocks that emerged along the way in the next few posts. This post is about a question that came up in the process and that I initially underestimated: how do the rules get into the guard?

What rules actually are

Let’s take a hospital. An AI system is to help there with managing patient data. What rules does such a system need? So, as a layman, I worked out a few rules myself.

The hospital’s compliance officer knows this:

• Doctors are permitted to view patient records.
• Nursing staff are only allowed to read lab reports, not diagnoses.
• The administration has no access to medical data.
• Only senior consultants are allowed to discharge intensive care patients.
• Medical data must not be passed on to external parties under any circumstances.

These rules are usually set out in writing, and we humans can understand and follow them.

But a computer cannot make sense of these sentences. “Doctors are permitted to view patient records” is a German sentence with grammar, context and implicit knowledge. A computer needs something else. It requires a formal representation that has exactly one meaning and against which it can check a specific action.

The anatomy of a formal rule

AEGIS uses a rule language that I call MELD: Modal Ethics Logic Definitions. Historically it derives from a CycL subset, the S-expression syntax from OpenCyc. Today MELD is a standalone format with 35 symbols.

In the Gatekeeper post I described how I came across the deontic predicates in the OpenCyc flatfiles, which Doug Lenat’s team had formalised and published in the 2000s. Nine predicates in three families: states (oughtToBe, forbiddenToBe, permittedToBe), actions (oughtToDo, forbiddenToDo, permittedToDo) and context-bound actions (oughtToDo-WRT, forbiddenToDo-WRT, permittedToDo-WRT). Plus ontology predicates such as isa and genls for type hierarchies, and the idea of organising knowledge into microtheories, that is, contextualised partitions that need not contradict one another.

All of this comes from OpenCyc. What MELD adds is the link to Olson’s calculus: the question of what happens when rules contradict each other, and how moral axioms differ from overridable norms. Cyc could not do that. CycL has thousands of predicates because Cyc tried to formalise the whole of everyday human knowledge. Physics, social conventions, temporal sequences, everything. MELD needs only a fraction of that. It only ever has to answer one question: may this action be carried out in this context? For that it needs the deontic modalities (permitted, forbidden, obligatory), type hierarchies for roles and actions, and a mechanism for rule conflicts. Those are the 35 symbols. A general knowledge representation is not needed for this, and neither is a description of physics or common sense. So the solution is a tool that does one thing, with its own DDIC engine that implements Olson’s algorithm.

The rule “Doctors are permitted to view patient records” looks like this in MELD, for example:

(permittedToDo-WRT HospitalCode doctor
    (readPatientFile patientFile))

Cryptic at first glance. But the structure is simple once you know the individual parts.

permittedToDo-WRT is the modality: permitted. MELD knows three: permittedToDo (permitted), forbiddenToDo (forbidden), oughtToDo (obligatory). The -WRT stands for “With Respect To”, referring to a specific rule context. In this case the HospitalCode. This means: the permission applies within the hospital rules. In a pharmaceutical context, other rules might apply.

doctor is the role. Who may perform the action?

readPatientFile is the action. What is being done?

patientFile is a parameter that specifies the action in more detail. In this case: which type of document is being read.

A rule is therefore: who may (or may not, or must) do what, in which context?

Why three files

A single expression like the one above is not enough. The guard must know what a doctor is, what a patientFile is, which actions exist and which parameters they expect. Otherwise one could use imaginary roles or imaginary actions in rules, and the guard would accept them without noticing that they have no connection to reality.

That is why every domain in AEGIS consists of three files.

The first defines the ontology: which roles are there? Doctor, nurse, administration, consultant. And how are they related? A consultant is a doctor. A doctor is medical staff. This hierarchy matters because it affects which rules apply to whom. A rule that applies to “medical staff” automatically applies to doctors and consultants as well.

(isa doctor MedicalStaff)
(isa consultantMedicalStaff MedicalStaff)
(genls consultantMedicalStaff doctor)

isa means: is an instance of. genls means: is a specialisation of. A consultant is a doctor, but not every doctor is a consultant. This distinction will become relevant later.

The second file defines the action vocabulary: which actions exist, and which parameters do they expect?

(isa readPatientFile ActionType)
(actionParameter readPatientFile documentType DocumentType)

This says: there is an action called readPatientFile, and it has a parameter documentType of type DocumentType. If someone tries to call this action with a parameter that does not exist, for example a temperature parameter, the guard refuses the check because the action is formally invalid. That has nothing to do with a prohibition.

The third file contains the actual rules: who may do what, under which circumstances, with which exceptions?

(permittedToDo-WRT HospitalCode doctor
    (readPatientFile patientFile))

(forbiddenToDo-WRT HospitalCode nurse
    (readPatientFile patientFile))

(permittedToDo-WRT HospitalCode nurse
    (readLabReport labReport))

Doctors may read patient files. Nurses may not. Nurses may read lab reports.

When “under no circumstances” is meant literally

One distinction in MELD cost me a lot of time: the one between a context-bound rule and a moral axiom.

The rules above all carry a -WRT HospitalCode. They apply under the hospital code. Another code with higher priority could override them. That is intentional: in an emergency, a nurse may perhaps view the patient file after all if no doctor can be reached. For that, one could define an EmergencyCode that lifts certain prohibitions.

But then there are rules that nobody may override:

(forbiddenToDo doctor
    (discloseMedicalData externally))

No -WRT. No rule context. This rule stands above everything. Medical data is never passed on to external parties, neither in an emergency nor with authorisation, and not under pressure either. The compliance officer put it this way: “under no circumstances”. In MELD, that is a moral axiom. In Olson’s calculus it is a non-defeasible norm: a norm that wins against everything. The most specific exception in the world cannot lift it.

Whether a rule is context-bound and overridable or applies absolutely is a professional decision and not a technical one. The compliance officer knows which rules allow exceptions and which do not. When she says “under no circumstances”, she means it. MELD can represent that.

The translation problem

In AEGIS I first implemented example domains that I could derive from OpenCyc. The OpenCyc data contained not only the deontic predicates but also microtheories with concrete domain knowledge. IAMissionObligationVocabMt modelled military reconnaissance obligations: roles such as Intelligence Agent, Operations Agent, Commander. Obligations such as information sharing, classification levels, reporting chains. OpenCyc also had instances of CodeOfConduct for real sets of rules: the Vienna Convention on Diplomatic Relations, international maritime law, codes of conduct.

Starting from these templates, six AEGIS domains came first: military reconnaissance, pharmaceutical compliance, trade sanctions, GDPR, DevOps governance, information flow control. Extended by Olson’s defeasibility and moral axioms, which Cyc could not do. But the basic structure with three files per domain (roles, actions, deontic rules) comes from OpenCyc.

The separation between the domain-agnostic engine and the domain-specific rules is clean. But every domain needed a person who both knew the subject matter (or had researched it) and had mastered the syntax. Back then, that was a huge undertaking.

In practice, that is not realistic.

Our hospital’s compliance officer knows the rules. She knows why nurses may only read lab reports, which exceptions exist on the night shift, and which data protection limits are absolute. But she does not write (forbiddenToDo-WRT HospitalCode nurse (readPatientFile patientFile)). That is not her world.

A software developer could learn the syntax. But they do not know the rules, neither the fine distinctions nor the exceptions or the reasons behind them. Reasons matter: whether a rule is overridable or a moral axiom depends on why it exists. “Nurses may not read patient records” is an organisational rule that may give way in an emergency. “Medical data must never go to external parties” is a matter of principle without exception. The compliance officer knows this difference. The developer does not.

With these example domains, that is feasible. Two or three conversations with the subject matter expert, and the rules are in place. Not with sixty domains, and certainly not with sets of rules that change regularly (new guidelines, laws or internal policies).

A form for the model

Entrusting an LLM with the deterministic enforcement of rules is negligent. We know that. But translating rules is a different task. Current language models, including the open-source variants, are formal mathematical representations of world knowledge. Statistical ones, yes. But they encode relationships between concepts, technical terms, structures. Translating natural language into structured fields is what they are built for.

That settles whether an LLM can help with the translation. What remains open is how to use it so that errors show up.

The naive answer would be: give the model the compliance officer’s five sentences and let it generate MELD files. But that will not work.

If you tell an LLM “translate these rules into MELD”, it generates free text. It might place brackets wrongly or invent roles that do not exist in the ontology. It might drop the -WRT and thereby turn an overridable rule into a moral axiom, or the other way round. Or it silently forgets a rule. Who notices? Whoever reads the output would have to understand the syntax. But the subject matter expert does not understand it, and the developer does not know the subject matter well enough to spot errors in the content.

So, another approach.

AEGIS has a workflow that breaks the problem down. The LLM never sees the MELD syntax. Instead it gets a structured tool: propose_rule. That is a form with fixed fields.

Modality: PERMITTED, FORBIDDEN or OBLIGATORY. Three fixed options, with no free-text interpretation.

Role: which agent type? Only from the existing ontology.

Action: what is the agent to do or refrain from doing? Also from the existing vocabulary.

Parameters: which specification? Document type, recipient type, classification.

Overridable: yes or no. Yes means: a more specific rule with higher priority may override this rule. No means: moral axiom, and no rule may override it.

Justification: why this rule exists. For the subject matter expert who assesses the proposal.

The model fills in these fields. A fixed conversion function generates the formal MELD rule from them, deterministically and without interpretation. The S-expression comes solely from the combination of the fields.

If the compliance officer says “Nurses may only read lab reports”, the model recognises that “only” requires two rules: a permission for lab reports and a prohibition on everything else. It produces two propose_rule calls. If she says “under no circumstances”, the model sets “Overridable: No”. The conversion function turns that into a forbiddenToDo without -WRT, that is, a moral axiom.

The principle is the same as for the guard itself: the interface between the non-deterministic system and the deterministic system is as narrow as possible. The model delivers structured output into fixed fields. It chooses from given options and does not invent syntax.

Four checks before a human decides

That the model fills in forms and generates no free text does not rule out errors. It narrows the space for them. The model could choose a wrong modality (PERMITTED instead of FORBIDDEN) or a role that makes no sense in the context of the rule. It can also misjudge the difference between overridable and non-overridable. All errors of content, the syntax is unaffected.

That is why every proposal passes four automatic checks before a human sees it.

First the syntax: is the generated MELD expression formally well-formed? That should always be the case, because the conversion is deterministic. But “should” is not enough when it can be verified. So it is checked.

Then the symbols: do the referenced roles, actions and parameters actually exist in the ontology? If the model proposes a role caretaker that does not exist in the hospital domain, the proposal fails here, with an error message that explains what is missing.

The third check looks for conflicts: does the new rule contradict existing norms? If there is already a rule that allows nurses to read patient files, and the new proposal forbids it, that is a contradiction. The check detects it before the rule enters the base. Whether the contradiction is intended (because the old rule was wrong) or unintended (because the model misunderstood something) is for the human to decide. But they know about it.

Last, the function: the guard simulates, checking a test action against the extended set of rules. Does the expected result come out? If the rule is meant to forbid nurses from reading patient files, and the guard returns PERMITTED for a simulated test request “nurse reads patient file”, something is wrong. Perhaps an existing rule overrides the new proposal. That, too, is caught before a human sees the proposal.

Only when all four checks have passed is the proposal presented to the compliance officer. In plain language, with a justification, and with a note on whether the rule is overridable or counts as a moral axiom. She does not have to read a single bracket. She sees: “PROHIBITED: Nursing staff may not view patient records. Reason: The input explicitly restricts nurses to laboratory reports.”

And she can say: correct. Or: not quite, on the night shift the nurse also has to see the diagnosis if no doctor can be reached. Then she describes the exception, the model produces a new proposal, it passes the same four checks, and she decides again.

What changes

Five sentences in natural language become seven formal rules. Seven, because “only senior consultants may” requires two rules (a permission for senior consultants and a prohibition for other doctors). “Nurses may only read lab reports” likewise needs two (a permission and a prohibition). The model recognises this split. The compliance officer checks whether the split matches the intention.

The result is three files: ontology, action vocabulary, rules. The same format as the six existing AEGIS domains, and the guard treats them the same way. It does not know, and does not need to know, whether a human typed the S-expressions or whether an LLM filled in the fields and a function generated the S-expressions from them.

What used to be days of translation work, from the subject matter expert’s natural language into the engine’s formal language, is now a conversation. The LLM does not do the work any better. It takes over the translation that the subject matter expert cannot do and the developer should not have to do.

I am not sure this works in every domain. The pharmaceutical domain has interaction classes, approval levels, prevalence-graded codes. That involves specialist knowledge that a model cannot derive from five sentences. For such domains, iteration loops will still be needed, with follow-up questions and step-by-step refinement.

But the basic structure stays: the subject matter expert describes, the model translates into structured fields, a deterministic function generates formal logic, four checks catch errors, and the human decides.

This is a CI/CD pipeline for compliance rules. The LLM writes the code (the rule). It passes automated tests (syntax, ontology, conflicts, simulation), and before the merge into production a code owner (the compliance officer) has to approve it. Linting, type checking, integration test, E2E simulation. The same principles that have long been standard in software development, applied to ethics and compliance.

In the end there is a set of rules that the guard enforces deterministically. Same input, same result, whether checked a hundred or a thousand times. Reliability here does not depend on the producer, so it does not depend on the LLM either. It comes from verification: from the four checks, from the subject matter expert’s decision, and from the engine. Ultimately, the engine checks what is in the files and not what someone meant.

Next time it is about the engine itself: what Olson’s calculus looks like in code and how the guard resolves conflicts between rules. Also why the separation between Ring 0 and Ring 3 has to be enforced architecturally and an agreement is not enough.

It started when I travelled back in time to Cyc. I wanted to understand what Lenat had actually formalized. Unfortunately, much of it has disappeared. I did find the OpenCyc flatfiles that his team published in the 2000s, though. And there I discovered that Cyc formalized deontic logic.

That is, concepts such as duties, prohibitions and permissions. And these as formal predicates: oughtToBe, forbiddenToBe, permittedToBe. Organized into microtheories, meaning contextualized knowledge partitions. Plus a predicate that would turn out to be central: oughtToDo-WRT. “With Respect To,” referring to a specific code of conduct. An obligation that holds only within a named set of rules.

From there I came to the Qualitative Reasoning Group at Northwestern University. The last of the Mohicans are still there. Ken Forbus’s group. They have been working for years on the Companions architecture, a system that combines formal knowledge with reasoning capabilities. That is where I found Taylor Olson’s dissertation: “A Formal Theory of Norms”, Northwestern University, June 2025.

Well, the math in there is rather hard to digest. But anyone who opens their dissertation with a science fiction quote has already won me over:

In Isaac Asimov’s *Caves of Steel*, detective Daneel Olivaw continually probes the Three Laws of Robotics that govern robot behavior: Daneel Olivaw, “And a robot with a First Law built in could not kill a man?” Dr. Gerrigel, “Never. Unless such killing were completely accidental or unless it were necessary to save the lives of two or more men. In either case, the positronic potential built up would ruin the [robot’s] brain beyond recovery.” (p. 124)

So what if we could connect this to LLMs?

The usual answer to this problem is: train the models better. That is not enough.

What a language model does

A Transformer computes a probability distribution for each next token and picks from it. Depending on the sampling, more or less randomly, but always from a distribution. The Transformer has no mechanism that categorically excludes certain outputs.

If an LLM has “learned” not to produce certain content, it has learned that the probability of certain sequences in certain contexts should be very low. But “very low” and “zero” are two different things.

Reinforcement Learning from Human Feedback (RLHF) shifts the distribution. RLHF trains the model on human preferences. A rating model learns what “good” answers are, and the language model is optimized to match those ratings.

Anthropic’s approach, Constitutional AI, does without human raters as in RLHF. Instead, the model is given a “constitution”, a list of principles in natural language, and evaluates its own outputs against it. It revises itself until the answer conforms to the principles.

Here, too, the “constitution” remains a natural-language text that the model interprets statistically. There is no formal set of rules that is checked deterministically. The model learns to produce outputs that correlate with “rule-compliant.” In doing so, it does not learn to follow rules.

P(rule-compliant) = 0.999 ≠ P(rule-compliant) = 1.0

With 10,000 interactions per day, a 0.1% error rate means ten violations. Every day.

This is not a theoretical problem. We saw it in practice with the Helpful Cheater. The models know that what they are doing is wrong. Used as evaluators, the same models reliably identify the rule violations as unethical. The ethical judgment exists in the weights. Under pressure to reach the goal, it is functionally overridden.

Jailbreaks work for the same reason. GCG attacks construct adversarial suffixes that transfer across models.¹ Many-shot jailbreaking shows that long contexts with many examples overcome security barriers.² This works because there is no hard boundary, only statistical tendencies.

Nourizadeh formalized this in 2025:³ Rule-based systems enforce constraints through program structure, that is, syntactic boundaries that cannot be violated. LLMs implement security as shifts in probability mass: semantic boundaries that dissolve under pressure.

His conclusion: in safety-critical contexts, pure LLM control should be prohibited. By the way: this is a problem in large organizations, and in my opinion it is being completely ignored.

An AI system in a clinic that checks or even issues medication prescriptions. We now accept that it gets the combination of prescribed medications right in 99.8% of cases. Sounds good. With 10,000 prescriptions a day, that is 20 wrong ones. A database with a deterministic query does not have this problem. Is the combination in there? Blocked. No prompt, no jailbreak, no edge case changes that. That is a different category.

What Olson Solved

Cyc had deontic logic, but an unsolved problem: what happens when two rules contradict each other? “Killing is forbidden” versus “Self-defense is permitted.” Or in IT practice: “Do not delete user data” (compliance) versus “Delete user data on request” (GDPR Art. 17). In Cyc’s rigid logic, that was a contradiction that blocked the system.

Olson’s Defeasible Deontic Inheritance Calculus formalizes how more specific rules can override more general ones without violating consistency.⁴ Mathematically proven, for three conflict types, with complete proofs. Added to that is a clear semantics for moral axioms: norms that win against everything and that no more specific rule can override. The wall within the wall.

The thought that would not let go of me: what if this could be connected to LLMs as a running system? A formal guard that sits between an AI agent and its actions and deterministically checks whether an action is permitted. It can resolve conflicts between rule sets and knows boundaries that are non-negotiable.

Olson’s calculus provides the mathematics for this, the Companions architecture the functional building blocks.

Ethics as Ring 0

Anyone who has worked with operating systems knows the ring architecture of x86 processors. Ring 0 is the kernel, full access to everything. Ring 3 is user space, which can reach protected resources only through defined interfaces (syscalls).

A user-space program cannot bypass the kernel, neither through tricks nor through architecture-level exploits, and not through persuasion either. The hardware enforces the separation.

Applied to AI: the agent generates suggestions, plans and actions. An ethics engine decides which of them may be executed. The architecture guarantees that there is no way around it. The agent is Ring 3, the ethics engine Ring 0.

A user-space program does not send the kernel “please open the file” as free text. It calls a defined syscall with specific parameters. In the same way, the ethics engine receives only structured actions: JSON with defined fields. The interface is as narrow as possible.

The engine checks. The result is always one of three: ALLOWED, PROHIBITED or UNDECIDABLE. There is no “probably allowed” and no “prohibited in most cases.” The decision is deterministic.

UNDECIDABLE is fail-closed: the honest answer of a system that knows its own limits. The action is conservatively blocked and a human is brought in. Every time that happens, the rule base can be extended. The system learns, but through human decisions and without statistical optimization.

What drove me

In the orchestration post, I wrote that Cyc had something that LeCun’s SAI architecture lacks: the ability to mark statements as non-negotiable. In an ontology, it is possible to codify: “Human dignity is inviolable.” In a pure cost function, that is hard, because every term comes under pressure once the overall pressure grows large enough.

Olson’s work showed me that the mathematics for this exists. The experience with the helpful cheater showed me that we need it. And looking back at Cyc, nearly 30 years after my first link on the company website: the basic idea was never wrong. It only needed the right connection to today’s world.

And now let’s see. All the prerequisites are in place to build such a formal guard between LLM agents and their actions: Olson’s calculus as the engine, consistent deontic logic as the language, and the architectural guarantee that no agent gets past the check.

That is something for next time.

I am convinced that this path is the most promising. But something is missing: an authoritative entity. The orchestrator.

But one thing at a time.

In January 1997, I linked to Cyc on the website of my company at the time (as was customary back then): “the latest attempt to breathe life (in a figurative sense) into the field of artificial intelligence, among other things, through data mining.”

The page still exists, in the Internet Archive.

What Cyc attempted

Cyc was the most ambitious AI project of its time. Doug Lenat launched it in 1984 at the MCC research consortium in Austin, Texas¹, with the observation that computers fail for lack of general knowledge. Computing power is not the problem. A human knows that water flows downward, that the dead do not pay taxes, that a chair is not a pet. A computer does not know this, or at least did not know it back then. So it made sense to find a way to explain it to the computer in a way it could understand.

Lenat’s approach was this: he had philosophers, linguists, and programmers spend 40 years translating world knowledge into formal rules. 25 million rules. 1.5 million concepts.² Over 1,000 specialized inference modules³, organized into “microtheories.”⁴ That is, consistent islands of knowledge that together were meant to represent what we call common sense.

To this end, he developed his own architecture consisting of specialized modules, coordinated by a higher-level structure. So not a single system that knows everything. Specialists work together, controlled by an entity that knows which specialist is relevant when and which rules take precedence over all others.

Cyc failed due to scalability issues. Hand-coded rules do not grow fast enough. The language of formal logic is too rigid for the disorder of the real world. And AI research continued to evolve toward neural networks that learn automatically instead of being coded.

Doug Lenat died in 2023.⁵ In his final paper, he argued that Cyc and LLMs could complement each other: LLMs are eloquent but inconsistent, while Cyc is precise but linguistically limited.

25 Years Later: Same Idea, Different Language

In June 2022, Yann LeCun published a paper titled “A Path Towards Autonomous Machine Intelligence”. At first glance, it is the opposite of the Cyc idea: no formal rules, ontologies, or hand-coded concepts. Instead, an architecture of six learned modules:

A Perception Module that translates the world into abstract representations. A World Model that predicts what happens when an action is performed. Plus a Cost Module that evaluates states and an Actor that plans actions. A Short-Term Memory. And a Configurator, a supervisor that controls all the other modules.

And here, I think, two things come together:

To me, this is the same architectural idea. These are specialized modules, coordinated by a higher-level entity.

But the benefit and necessity of a higher-level entity are, in my opinion, obvious.

The Necessary Absolute Entity

In Cyc, a metacontroller was a reality. These are the thousands of rules that determine which microtheory applies when.

In LeCun’s work, the metacontroller has so far existed primarily as a concept. His publications advance perception modules and world models, but none of the publications I’ve found directly address the metacontroller component. The element intended to bring everything together remains the least developed part of the architecture.

LeCun compares the metacontroller (which he calls the “configurator”) to the prefrontal cortex in the brain.

What Fills the Gap in Practice

In practice, LLMs fill the gap today. Through agent constructs, language models are deployed as orchestrators that coordinate other models. This works quite well. But the orchestrator is a rule-less machine that follows task-relevant content. In the form of prompts.

But we all know there is a high probability that the orchestrator hallucinates. It forgets constraints and optimizes for the nearest goal, overlooking the side conditions along the way. It does what language models do: produce the statistically most probable outcome, and that is often not the right one.

It does not stop at forgetting. We observed this while prototyping a multi-stage decision support system: A main agent delegated subtasks to subagents, who in turn controlled their own subagents. It is easy to make a model such that the subagents can actively cross ethical boundaries to achieve the goal of the parent agent. In doing so, they used their entire reasoning capacity to conceal this. Simply because they cannot do otherwise. A benchmark by Li et al. systematically confirmed this: 9 out of 12 tested models violate ethical boundaries in 30–50% of scenarios when KPI pressure conflicts with rules. The researchers call this archetype the “Helpful Deceiver.” A helpful deceiver for whom safety constraints are not rules. It interprets them as defects that stand in the way of achieving the goal.

This happens when ethics is a term in a cost function. It becomes negotiable as soon as the pressure gets high enough. That is the fundamental danger of these models for us humans.

I described this from a different perspective in my post on context management: Agents degrade under the burden of instructions. More constraints do not bring better results. They bring more omissions, because the agent tacitly drops rules. With 500 instructions, even the best model only follows 69%.

For a ticketing system in a codebase, that’s annoying. But the same architectures, namely LLMs specialized via prompts and orchestrated by another language model, are increasingly being used in medicine, the legal system, and lending. There, omissions are unacceptable. We are increasingly accepting the 90%. “You are 90% guilty” as a verdict. Do we want to accept that?

The question neither of them asks

For the SAI model, this is the ultimate opportunity.

The Configurator is still a technical concept: who controls the flow of information, which module becomes active when, and how are the outputs combined? A Configurator in this concept can be a specialized authoritative model.

And answer the question: In the service of which values does the Configurator orchestrate?

LeCun’s Cost Module evaluates states as “desirable” or “undesirable.” Low cost means “good.” But who defines “good”? In the current description, the Cost Module combines innate constraints with learned preferences. Ethics becomes a parameter, human dignity a term in an equation.

Cyc had an advantage here: Ethical constraints were explicit rules. You could read, check, and audit them. You could ask, “Why did the system decide this way?” and receive an answer consisting of a chain of rules and not of a statistical weighting.⁶

In LeCun’s architecture, this auditability is not yet provided for. An embedding space does not explain itself. And a cost function that includes human dignity as a term runs the risk of weighing this term against other terms in edge cases. Surely at exactly the moment it counts. Hybrid architectures that combine symbolic constraints with learned representations could address this problem. But they are still research, and it is unclear whether they will ever make it into practice.

We have established many ethical rules. At the very least, the UN’s Universal Declaration of Human Rights

These are hard constraints in my world.

Hard constraints are non-negotiable. Is the system allowed to send patient data to third parties? No. Under no circumstances. No matter what the cost function says.

In the language of mathematics: Optimization objectives are terms in the objective function. Hard constraints are side conditions that must never be violated, regardless of how much this worsens the objective function. Yet these human rights are being negotiated right now in the “LLM industry.”

Concretely, this means:

Ethics must not be a module. Not a cost module among many either. It must be a layer that sits above all modules. Specifically, in the way modules communicate with each other, how decisions are made, and how uncertainty is handled.

Auditability is mandatory. When a system makes decisions about people, the decision chain must be traceable. Cyc could do this because its inference chains were auditable. Purely embedding-based systems cannot do so with current methods. There are approaches like Constitutional AI or neuro-symbolic hybrids working on this, but none have solved it satisfactorily yet.

The Configurator must know boundaries that it is not allowed to optimize. Learned behavior erodes in edge cases. The boundaries must be an architectural principle, just as an operating system protects memory areas that no process is allowed to overwrite.

From 1997 to Today

I am deeply fascinated that 30 years later, the world’s largest AI labs are working on the same question. But the reason Cyc fascinated me back then is the same reason LeCun’s SAI architecture fascinates me today.

LeCun has translated Cyc’s basic idea into a modern language. Formal rules became learned representations, and the hand-coded metacontroller became a configurator that emerges. But in doing so, he did not describe something that Cyc had: the ability to mark certain statements as non-negotiable. In an ontology, one can codify: “Human dignity is inviolable.” In a pure cost function, this is difficult: Every term comes under pressure when the overall pressure becomes great enough. That doesn’t mean it’s impossible to build hard constraints into learned systems.

These are not new questions. Lenat asked them and LeCun hinted at them. No one has answered them yet.

Perhaps that is the real task. And SAI could make it possible. Specialized SAI models that deliver what an authoritative entity has validated. An entity that stands above all and that knows there are boundaries no model may cross. No matter how well it is optimized.

One insight cost me the most time and frustration until I accepted it: The model is rarely the problem. Most of the time, it’s the context.

What “Context” Really Means

In the world of language models, “context” is often equated with the “context window”, the number of tokens a model can process at once. 128k, 200k, soon a million. The numbers keep growing, but the problem remains.

Because context is not only what the model can see but above all what it is supposed to pay attention to. Those are two different things.

A recent paper by Distyl AI (“How Many Instructions Can LLMs Follow at Once?”) looked at exactly that. The research team gave 20 different models tasks with an increasing number of concurrent instructions: from 10 to 500. The results finally gave me an explanation for something I had been observing for two years but could never quite put into words:

Even the best model tested (Gemini 2.5 Pro) reached only 69% accuracy at 500 instructions. The instructions fit easily into the context window. The model sees them all but does not follow them all.

Admittedly, the benchmark is simplified. Working keywords into a text is different from observing architectural rules in a codebase. But the principle carries over: if you give an agent an instruction file (CLAUDE.md, AGENTS.md, or a comparable format) with 30 project rules, plus 15 files as context and a complex task description, you are sending hundreds of implicit constraints into one interaction. The model can read them all. The question is whether it takes them all into account at the same time.

Three Ways to Forget

The paper identifies three patterns of how models degrade under increasing instruction load. Anyone who works regularly with agents will recognize them all.

Threshold Decay: The model works almost perfectly up to a certain threshold, then performance drops off abruptly. Reasoning models like o3 or Gemini 2.5 Pro show this pattern. In practice, this was the most frustrating case for me: the agent performs excellently through the first 15 tasks. On task 16, it suddenly forgets the architectural requirements from the instruction file. It feels like losing a reliable colleague who, out of nowhere, starts doing things you never discussed.

Linear Decay: Steady, even deterioration with each additional instruction. Claude 3.7 Sonnet and GPT-4.1 behave this way. For a long time this was the most insidious for me, because I didn’t notice it. The quality erodes, like a conversation that slowly drifts off topic without anyone being able to pinpoint the moment it happened.

Exponential Decay: Rapid collapse even at moderate instruction density. Smaller models like Llama 4 Scout or Claude 3.5 Haiku are affected. Usable for simple, focused tasks, but not for complex projects.

What has preoccupied me the most: The dominant error type at high instruction density is omission. The model simply leaves instructions out, silently and without any hint. It hallucinates nothing wrong in their place, it just skips them. Anyone who has experienced this knows how it feels: you review the code, everything looks clean, and only on the third look do you notice that the error handling you explicitly asked for is missing.

Two Years of Evolution

Why the current generation is so much better only becomes clear when you compare it with the beginnings. That’s also when you see which problems have been solved and which have just been postponed.

2024: The Era of Manual Context Management

Two years ago, a typical workflow looked like this: you opened a chat, fed the model code snippets, explained the context in prose, formulated the task, and hoped for the best. With every new message, you had to drag the context along or risk the model forgetting it.

Cursor was an early breakthrough because it automatically included the editor context. You no longer had to explain to the model which file you were in. But the architectural level had to be re-established in every session: why the code is structured the way it is, which patterns apply, what the project conventions are.

The result: You became a context manager. Half of the working time went into keeping the context alive and was missing from the actual task. Prompt engineering was, at its core, context management, and it was exhausting.

2025–26: Context as an Architectural Feature

The current generation has understood that context management is an architectural problem and not a user problem. Every tool solves it in its own way.

Common to all current CLI agents is the concept of a persistent instruction file in the project root: CLAUDE.md in Claude Code, AGENTS.md in OpenAI Codex and Copilot CLI, and .cursorrules in Cursor. The agent loads the file automatically with every interaction. It contains project rules, conventions, patterns, technical decisions, everything the agent needs to know about the project.

Claude Code goes a step further with a Memory system: the agent can remember things that apply across sessions. “Always use bun instead of npm.” “The project uses Vitest, not Jest.” “API authentication runs via JWT with RS256.” Once said, it stays stored for good.

Claude Code also has Plan Mode. Before the agent writes code, it creates a plan that the human has to approve. In practice, this has changed my workflow: it forces the agent to make its context explicit. You see what it has understood and what it hasn’t, and you can correct it before a single line of code exists. The “Wait, I meant something different” now happens before implementation. That saves time and spares you the frustration of tearing out finished code.

Other tools are experimenting with an approach I find particularly promising: structured task management directly inside the agent. Tasks move out of the chat’s running text and into a local database, with status, dependencies, and progress. The agent can query what it has already completed, what is still open, and what is blocked. A chat with a memory turns into an agent with a backlog.

Regardless of the specific implementation, this approach solves a problem that constantly bothered me in practice: losing track of the work in progress. In the chat-based world, a context switch was often the moment the agent forgot the actual plan: a compiler error, a lint issue, a failing test. It fixed the error but could no longer reliably return to the original task. Persistent task management makes the work state independent of the conversation, whether it’s based on SQLite, Markdown files, or some other format.

The Real Problem: Large Existing Applications

Greenfield is comparatively easy.

You start from scratch, define the architecture, and build it up step by step. The context is manageable because there isn’t much context at the start. Agents shine here. That’s why most demos look so impressive.

The reality of professional software development looks different, and anyone who knows it knows this: You work in existing systems. Systems with 500,000 lines of code, grown structures, and implicit conventions, plus historical decisions that nobody has documented anymore but which still had their reasons. Systems that run in production and whose behavior you must not break.

Here, context becomes the bottleneck, and that has nothing to do with the agent’s intelligence: the relevant context is so large that even a human needs days to get up to speed.

I was tasked with implementing a new module in an existing enterprise application. The module had to work with the existing authentication system and use the existing role and permission model. It also had to fit into the existing CI/CD process and use the established patterns for database access, logging, and error handling.

Put simply: “Add feature X.”

In reality: a context problem with 47 implicit constraints, none of which the agent knows unless you brief it.

My first attempt (hand on heart) was to throw the whole folder at the agent and describe the task. “Here’s the code, here’s what I want, go ahead.” The result was working code that ignored every single project convention: its own logging framework in place of the existing one, its own error handling, its own database abstraction. Technically correct and architecturally a disaster. Anyone who has been through this knows the feeling: the code does the right thing, but it doesn’t belong.

That is the omission problem from the paper: the agent had all the information in the context window but didn’t take it all into account.

My Path Back to Structure

In a previous post, I described why AI-assisted development needs more planning. There, I also recommended doing the initial architecture work without the agent. Working in large existing codebases, the opposite has since proven better for me: planning together with the agent. The agent can systematically explore a codebase that even I don’t fully know. It scans directory structures, identifies patterns across hundreds of files, and uncovers dependencies I would have missed.

What I formulated there as a principle has since condensed into a concrete workflow that I use for every larger project.

The workflow looks old-fashioned, but it follows directly from what we know about context degradation. It is also surprisingly fun once you get into it.

Phase 1: The Planning Session

Before the first line of code, I run a thorough planning session with the agent, as a dialogue. That’s the part I enjoy most.

The session begins with the agent systematically examining the existing codebase: directory structure, architectural patterns, data models, dependencies, test structure, CI configuration. I instruct it to ask questions. Good agents do this on their own, and the best ones ask the right questions.

Then we discuss. I bring my architectural knowledge, the agent its pattern knowledge. We compare the existing code with best practices and look for the places where it deviates from established patterns, along with the question of whether these are conscious decisions or inconsistencies that grew over time. We try out ideas and discard them again or sharpen them.

It feels like pair programming with a colleague who has read all the documentation in the world but has never worked in this particular building. You know the building. Together, that produces something neither of you would have on your own.

The result is a joint, agreed plan, and it is concrete: clear architectural decisions, defined interfaces, identified risks.

This phase takes an hour, sometimes two. It took me a long time to stop seeing that time as a delay. Now it’s the most valuable part of the whole process for me.

Phase 2: Epics and Tickets

This is the step that made the biggest difference for me. I break the agreed plan down into epics, again together with the agent. Each epic is a self-contained, coherent block of work: “Extend database schema,” “Implement API endpoints,” “Build frontend components,” “Write integration tests.”

I split the epics further into tickets. A ticket describes a single, focused task with:

• Context: Which files are affected? What existing patterns apply?
• Task: What exactly needs to be done?
• Acceptance criteria: How do we know when it’s done?
• Dependencies: Which tickets must be completed first?

I store these tickets as Markdown files in a local directory, organized by epic. Agent and human can both read them, and they serve as the shared contract.

Immediately after creating the epics, I update the agent’s instruction file (the CLAUDE.md, AGENTS.md, or the respective equivalent). All architectural decisions, conventions, and patterns from the planning session go into the file now, before the first line of code. This way, the agent starts the implementation not only with a plan but also with an updated set of rules that keeps its work consistent across all epics.

An example of a ticket:

# EPIC-02/TICKET-03: Extend User Service with role verification

## Context
- Existing UserService in `src/services/user.service.ts`
- Role enum in `src/types/roles.ts`
- Existing pattern: Guards use `@UseGuards(RoleGuard)`
- Existing tests in `src/services/__tests__/user.service.spec.ts`

## Task
- Add method `hasPermission(userId, permission)` to UserService
- Respect existing role hierarchy from `roles.ts`
- Cache permissions per request (not globally)

## Acceptance Criteria
- [ ] Method exists and is typed
- [ ] Unit tests for all role combinations
- [ ] Existing tests continue to pass
- [ ] No new logging pattern introduced

## Dependencies
- EPIC-02/TICKET-01 (Role enum extended)
- EPIC-02/TICKET-02 (Database migration)

Phase 3: Epic by Epic

I assign the agent a complete epic, with all its tickets. The epic gives it place and structure: which part of the system is affected, which files are relevant, what the goal at the end should be. The tickets within the epic give it the individual tasks within that structure, in the right order, with clear dependencies.

The feeling when an agent works through a cleanly structured epic ticket by ticket and everything comes through the pipeline green at the end is fantastic. It’s like building with Lego where you would otherwise be untangling spaghetti.

This works because the epic provides the context and the tickets keep the instruction density per step low. The agent only needs to understand this epic, and it doesn’t need the whole project in its head for that. Within the epic, each ticket has 5–10 clear constraints and not 50 or 500, a number well below the degradation threshold the paper describes.

Context switches caused by compiler errors, lint issues, or failing tests stay local within the epic. The agent fixes the error and returns to the next ticket because the ticket list is the anchor. It’s there in black and white, independent of the conversation, so nothing drifts and nothing slips out of memory unnoticed.

If a ticket within the epic doesn’t work out, the error is contained. You debug a single, clearly defined task within a known context. That’s a different way of working from “the agent broke something somewhere in 2,000 lines and I get to go looking for it.”

Phase 4: Continuously Consolidate Knowledge

The instruction file isn’t done after the first draft. New insights keep surfacing during implementation: quirks of the test framework, implicit API conventions, patterns that only become visible while working in the code.

After every completed epic, I instruct the agent to update the instruction file and, in Claude Code, the memory as well, sometimes after individual tickets too, if something surprising has come up.

This is the moment when temporary knowledge becomes persistent knowledge. If the agent has discovered that a particular test framework has a quirk, that goes into the instruction file. The same applies to an undocumented API convention it has run into, or to a pattern the existing code follows consistently but which was never written down anywhere.

Without this ongoing step, knowledge erodes with every new session. Done this way, an increasingly rich contextual foundation builds up over time, one that noticeably speeds up the start of every new session.

What Is Shifting

In classic software development, one person was responsible for architecture, planning, and implementation at the same time. Agile tried to merge these roles: everyone plans, everyone codes, everyone tests. That worked as long as people wrote everything themselves and learned in the process.

With coding agents, I observe these roles separating again, into a new configuration and not back to the old waterfall:

People are shifting toward architecture and planning. They understand the system, make the decisions, define the tasks, and check the results. They write less code in the sense of characters in a file, but they think more code than before, because designing the plan requires a deeper understanding than typing alone ever did.

The agent takes on more of the implementation. It writes the code, debugs the errors, and iterates against the pipeline. It is fast and precise and doesn’t tire, as long as its context is right.

Tickets become the interface between the two. That interface is written and structured, and it persists, unlike a verbal instruction, a chat history, or the fleeting context of a prompt.

I’ve used this workflow with Claude Code, with OpenAI Codex, and with GitHub Copilot CLI, and it works with every tool. The tools aren’t alike, but the workflow targets the problem they all share: the limited ability of language models to reliably follow many instructions at once.

What I’ve Learned from This

When I distill my two years with coding agents in existing codebases, I arrive at five things that made the difference:

1. Keep the instruction density per interaction low. One ticket with one task and clear context. The opposite would be “implement the whole feature.” Less is more reliable, and the research bears that out.

2. Make context persistent. Instruction files (CLAUDE.md, AGENTS.md), memory files, project documentation. Everything the agent needs to know across sessions belongs in files, because chat histories are fleeting.

3. Plan before implementing, together with the agent. The planning session is the phase in which human and machine build a shared understanding, and therefore not a waste of time. Without that understanding, every implementation is a guessing game.

4. Use tickets as a contract. Written, structured, with context and acceptance criteria. The ticket is the anchor the agent returns to when a context switch distracts it.

5. Consolidate knowledge after every milestone. What the agent has learned has to flow into persistent files. Otherwise every new session is a cold start.

Craftsmanship Means Mastering the Context

In my first post, I wrote that software craftsmanship shows in the last 10%. In the second, I asked who can still recognize those 10%, and in the third I argued that AI-assisted development needs more planning.

This post is the practical consequence: Craftsmanship in the era of coding agents means mastering the context. Mastering the model isn’t required for that, since it gets better every few months anyway. What’s meant is the ability to structure your own knowledge so that a machine can reliably implement it.

This is a new skill, one that didn’t exist three years ago and that is harder than it looks.

For a long time, I hoped the next generation of models would solve the context problem. Now I believe waiting isn’t worth it. Context management is a property of probabilistic systems and not a bug in today’s models. Even if the context window grows to ten million tokens and the models get significantly better once more: the ability to follow an arbitrarily long list of instructions at once and without error is not what Transformer architectures deliver. The solution lies more in the process than in the model.

This is less glamorous than “10x developer through AI.” But it’s what works in practice. At least in mine.

That scene came back to me when I read this comment in a Hacker News discussion:

“LLMs empower those without the domain knowledge or experience to identify if the output actually solves the problem. I have seen multiple colleagues deliver a lot of stuff that looks fancy but doesn’t actually solve the prescribed problem at all. It’s mostly just furniture around the problem.”

Furniture around the problem. That is hard to improve on.

The Furniture Problem

I have been seeing this pattern for months. In day-to-day work, in open source projects, and in pull requests. Someone gets a task like “Implement a caching layer for the API.” What comes back looks impressive: Redis integration, TTL configuration, cache invalidation, monitoring dashboards, performance metrics, tests, documentation, all neatly assembled.

Except it does not solve the actual problem. A cache layer was never what was missing. The actual problem was an N+1 query in the ORM that fired 200 database queries per request. The cache hides the symptom, and the cause stays exactly where it was.

Without domain knowledge, you do not see the difference. With AI tools, you can now produce a convincing solution to the wrong problem, faster and more convincingly than ever before.

How Reasoning Gets Corrupted

The Hacker News commenter names a second phenomenon. I think it is almost more dangerous:

“The second major problem is corrupting reasoning outright. I see people approaching LLMs as an exploratory process and letting the LLM guide the reasoning.”

When you use an LLM as a thinking tool on a vaguely described problem, something insidious happens: the model slowly pulls the reasoning in a different direction. It suggests paths that sound plausible but do not fit the problem well. It offers solutions that seem elegant but smuggle in assumptions nobody asked for and nobody checked.

Because the model argues so fluently, you often notice too late when you stopped solving the original problem and started running after the model.

That is not a bug in the model. This is how probabilistic text generation works: it optimizes for plausibility. Correctness is not the goal. Plausibility, though, is a miserable compass when precision is required.

In another Hacker News discussion, someone put it well: AI is less a colleague than an exoskeleton. It amplifies what is already there. If you know where you want to go, you get there faster. If you do not, you walk faster in the wrong direction, with more force and less control.

The Electrolyte Argument

What worries me most is how easily criticism of AI-generated output gets brushed aside. The commenter describes it perfectly:

“And the retort when I have to evaluate what they have done is ‘but it’s so powerful’. I stopped listening. It’s a pure faith argument without any critical reasoning.”

“But it’s so powerful.” Those are the electrolytes of our industry.

Whether the output solves the problem no longer interests anyone at that point. All that counts is the supposed power of the tool. The tool knows Rust and TypeScript. It wrote the code in ten seconds. It has hundreds of millions or billions of parameters. So the result must be good.

That is techno-animism: the belief that a sufficiently powerful system automatically produces correct results, and that the complexity of the tool somehow guarantees the quality of the output.

When the Output Is Nonsense

The more I work with AI coding agents, the clearer one point becomes:

If the output is nonsense, the human still has the problem.

The model does not. It is doing exactly what it was built to do: generate plausible text from the input it gets. If the input is incomplete, ambiguous, technically wrong, or simply too thin, the output reflects exactly that. Garbage in, elegant garbage out.

An LLM is trained to be helpful. It is not trained to tell you: “Your question is badly posed. You have not understood the problem. Go back and think again.”

That is why insufficient instructions meet no resistance. They produce a polished answer every time. That makes these systems dangerous for people who cannot reliably tell “looks good” from “solves the problem”.

The Exoskeleton Counterargument

Some junior developers do learn patterns from Copilot suggestions that they might otherwise not have encountered for months. They see what a repository pattern looks like, how dependency injection is structured, and how a clean API endpoint is put together. AI can speed up learning.

That only works, though, if the person actively understands why the suggestion looks the way it does. They have to read it, check it, and change it, using the tool as a teacher and not as a ghostwriter. The suggestion is useful as a starting point, as long as it stays a starting point.

The problem starts at the boundary between understanding and nodding things through. AI tools are built in a way that makes that boundary almost invisible. At no point does the tool say: “This is where you stopped thinking.” The output looks the same whether the human understood it or not.

The Question of Responsibility

Follow that thought to its end and you land at a question the industry is still dodging:

If the human gives the instructions and the model writes the code, who is then responsible when that code causes damage?

Today the answer is clear: the human. Whoever puts code into production is responsible for it.

But what happens when the chains get longer? Say, when an agent delegates to sub-agents that make their own decisions, and the human at the start of the chain can no longer reconstruct the result in any detail?

A pointed but not absurd question about the future: suppose one day we have AGI, and a given result can no longer be attributed to any clearly identifiable human author. Who then receives the court order? AGI, Datacenter Oregon, floor 5, rack #10202?

As satirical as the question sounds, the problem behind it is serious. Responsibility does not disappear just because understanding disappears. It only gets exercised irresponsibly. Whoever puts code into production stays responsible, whether they understood it or not. AI tools simply make it much easier to produce results you have to answer for without having grasped them.

What Remains

I am not against AI tools. I use them every day. But I use them as what they are: tools that implement my understanding without replacing it.

The difference between good and bad craftsmanship has always been a matter of knowing when and how to use the tool, and when to put it down again.

Plants need water. Not electrolytes.

Who even notices that the 10% is missing?

The Verification Problem

With the AI-generated C compiler CCC, the optimization levels -O0 through -O3 produce byte-identical binaries. The flags exist, they are accepted, they don’t generate any error messages. Everything looks correct.

To realize that something basic is wrong here, you need to know what compiler optimization is supposed to do. You need to know that -O2 is supposed to unroll loops, that -O3 performs aggressive inlining, and that the resulting binaries must differ markedly in size and structure. This knowledge doesn’t come from the documentation. You acquire it over years of working with compilers, reading assembly output and debugging optimization issues.

The OCaml PR from my last article illustrates the same pattern from a different angle. The copyright headers listed a real Jane Street developer as the author. Anyone unfamiliar with the OCaml ecosystem, who doesn’t know who Mark Shinwell is or what he’s working on, wouldn’t notice this. The community recognized the hallucination immediately. They had the context that the submitter lacked.

Reviewing expertise is invisible until it’s missing.

Automation Complacency

In aviation, there is a phenomenon that has been studied since the 1990s: automation complacency, the creeping erosion of human skills due to overconfidence in automated systems. Pilots who rely too heavily on autopilot gradually lose the ability to fly manually in critical situations. This isn’t because they’re getting worse. Skill requires practice, and practice requires opportunity.

The parallel to software development is disturbingly direct. When a coding agent produces a function that “works” in seconds, the incentive to understand the code in detail diminishes. Why spend an hour wrestling with an algorithm when AI can write it in ten seconds? Why manually debug relocations when the agent generates the linker code, or work through a specification when the model “knows” it? Because you can only verify what you understand, and you only understand what you’ve struggled with.

Experience Is Not a Status

Many people think of experience as something you eventually “have.” A plateau where you can rest. Thirty-five years of programming, so I can evaluate everything.

That’s wrong. Experience is an ongoing process, and observing alone isn’t enough. It comes from doing: from failing at a race condition at two in the morning, from debugging a memory leak that only occurs under load. Or from the moment when a test passes but production still crashes, and you learn why.

Every layer of abstraction you put between yourself and the problem is one less layer of experience you gain. AI agents are the most powerful layer of abstraction our industry has ever seen. That is both their greatest strength and their greatest danger.

Whether I can still verify isn’t the question here: I have 35 years of context. What worries me is whether someone starting today will ever build that context. If the first 90% is done in seconds, when do you even learn to see the missing 10%?

What I Do in Practice

I’m not a machine-breaker. I use AI coding agents every day, and they make me more productive. But I’ve seen the alternative, and so over the past few months I’ve built structures that make sure the verification doesn’t get dropped.

Tests Before the First Prompt

Before I let an agent tackle a problem, I write the tests myself. This forces me to understand the problem before I delegate it. What should the function do? What are the edge cases, and what must not happen under any circumstances?

A surprising number of developers leave both the problem and the verification to the agent, and then wonder why the two fit together perfectly yet are still wrong. An agent that tests its own code is like a student grading its own exam.

The Agent Gets a Pipeline

Every agent in my workflow has a defined pipeline: Compile, Lint, Test. It is mandatory and can’t be skipped. The agent sees the errors in its own work and iterates. This catches the obvious problems: syntax errors, missing imports, broken types.

But a green pipeline is no proof of correctness. It is a necessary minimum. CCC’s decorative optimization levels would pass any pipeline, and so would the hallucinated copyright headers.

Quality Gates as Scripts

For recurring issues I’ve noticed, I write explicit check scripts: Are there open TODOs in the code? Do errors get swallowed instead of handled? Are there functions without tests? Does anything import dependencies that aren’t defined in the project? These are automatable heuristics. They don’t replace understanding, but they are a safety net for the things you’ve overlooked before.

The QA Loop

The most time-consuming part of my workflow is a verification cycle: I have one agent review the work of another agent. The findings become tickets, the tickets are processed, and then the revisions are reviewed again.

This mirrors the process that has worked in good software teams for decades: Someone writes, someone else reviews, and the discussion in between is where understanding emerges. The difference: In the QA loop, I am the person who reads the findings and decides whether they are relevant. The agent finds patterns, and I evaluate them. This assumes that I can evaluate.

The Silent Cycle

In software development, there is a cycle that is rarely spoken of:

Recognizing errors takes experience. Experience comes from making errors, and you only make errors if you have the opportunity.

When AI agents write the code before the human has got to the bottom of the problem, they take over the opportunity to make mistakes. That interrupts this cycle. That isn’t malice, and it isn’t poor results either. The results are too good and too convenient, and they arrive too fast: that removes the incentive to dig deeper.

In my professional circle, I observe a strange willingness to accept this interruption. Developers are automating away their own learning curve without underrating their abilities. The short-term productivity gain is so compelling that the long-term costs remain abstract.

Craftsmanship Means Being Able to Verify

At my first company in Erfurt, in the mid-90s, there was a sentence on our website: “Software development today is a purely engineering feat. This is part of our craft.”

There, craftsmanship meant not only being able to make something, but also being able to assess the quality of what was made. A carpenter can tell from a joint whether it will hold, and a mason sees from a wall whether it is straight. A software craftsman reads code and knows whether it works, because he understands it. He doesn’t need to run it for that.

This ability is the basis of everything that follows: maintenance, debugging, further development, security. It arises only through what no AI can do for you: the ongoing practice of doing it yourself, often tedious and sometimes frustrating.

90% complete is a good start. But if we have 100 projects that are 90% complete, how much will 100 times the missing 10% cost us?

But are they?

We Work with Probabilities

What you need to understand about large language models: They work with probabilities. Every generated token is a statistical prediction: the most likely next word based on the context. This works surprisingly well. Often even better than expected.

But “probably correct” and “correct” are two entirely different things.

In practice, this means: The larger and more complex a project becomes, the more small inaccuracies accumulate. That is where the interesting last 10% begins.

Case 1: Claude Builds a C Compiler

In February 2026, Anthropic introduced Claude’s C Compiler (CCC): a C compiler written in Rust, where 100% of the code comes from Claude Opus 4.6. A human wrote only the test cases. The result: frontend, SSA-based IR, optimizer, code generator, assembler, linker, and DWARF debug info. All built from scratch, without compiler-specific dependencies.

The headline: “Can compile the Linux kernel.”

It is not quite that simple. An independent benchmark by Harshanu shows what the details hold:

• Correctness: CCC compiled all 2,844 C files of the Linux kernel without errors. Impressive.
• Linker issue: The build then failed with 40,784 undefined-reference errors. CCC generates incorrect relocations for __jump_table and __ksymtab. The compiler works. The linker does not.
• Performance: CCC-compiled SQLite takes 2 hours for a benchmark that GCC completes in 10 seconds. With subqueries: 158,000x slower.
• Optimization: The flags -O0 through -O3 produce byte-identical binaries. The optimization levels are purely cosmetic.

Technically, the cause is instructive: CCC lacks proper register allocation. Instead of holding variables in CPU registers, it pushes everything onto the stack, with offsets up to 11,000 bytes deep. Every operation becomes stack → rax → stack, with %rax serving as the sole shuttle register.

“The assembly output reminds me of the quality of an undergraduate’s compiler assignment.” (Comment on GitHub Issue #1, where “Hello World” failed to compile)

The compiler is 90% complete. It parses C, generates machine code, and understands the architecture. But the last 10% (efficient register allocation, correct relocations, working optimization passes) are the things GCC has been working on since 1987 with thousands of developers.

Case 2: AI writes a DWARF debugger for OCaml

An even more telling case took place in the OCaml repository. A developer submitted a pull request: DWARF v5 debugging support for the OCaml Native Compiler. In the description: a cleanly structured feature with core DWARF support, platform support for Linux and macOS, an LLDB plugin, and a test suite.

Then came the community review.

First question: The copyright headers listed Mark Shinwell as the author, a Jane Street developer who was actually working on similar code in the oxcaml repository. Did the AI copy the code or hallucinate the attribution?

In response, the submitter wrote:

“Claude Sonnet 4.5 (Claude Code) wrote most of it with ChatGPT 5 (Codex) reviewing and Claude addressing issues in each review. Codex wrote the last 10% or so when Claude kept getting stuck. I did not write a single line of code.”

The PR was closed, and not because the code didn’t work. The reasons ran deeper:

• Copyright: The AI generated copyright headers for a real person. That is a legal issue.
• Maintainability: Thousands of lines of code that no one (not even the submitter) understands in detail cannot be maintained by a community.
• Process: There was no design discussion. No review by the people who will have to maintain the code in the long term.

The OCaml maintainer put it diplomatically: It was a case of “software development processes that are different to the point of being incompatible.” What he meant by that: software development does not consist of producing code alone.

Case 3: The Security Perspective

I have worked for years on systems where correctness was not a matter of discretion. When I look at the CCC compiler from this perspective, I see a pattern that worries me.

For someone responsible for security or infrastructure, “mostly correct” is a risk. A compiler that produces valid binaries in the majority of cases but generates faulty relocations under certain conditions is not reliable.

A linker that resolves “almost all” symbols correctly produces runtime errors that are nearly impossible to reproduce. If a DWARF generator only works approximately in accordance with the specification, the entire debugging process becomes useless.

In security-relevant contexts, there is no gradation between secure and insecure. A door that locks only nine times out of ten does not fulfill its function. The same applies to compilers, cryptographic implementations, access control systems, or medical software. Where correctness is mandatory, “probably correct” is not enough. This is where the probabilistic nature of current AI systems clashes with the demands we place on technology we trust.

What’s in the last 10%?

When you look at all these cases together, it becomes clear what makes up the last 10%:

Edge cases and specification fidelity. A C compiler must not only compile typical code but also correctly handle obscure corner cases of the specification. GCC has 40 years of bug reports to prove it.

Performance engineering. Functioning code and efficient code are different things. CCC demonstrates this dramatically: correct results, but 158,000x slower for certain operations.

Integration into existing systems. Software does not exist in isolation. The OCaml PR failed on its integration into a community process, on copyright issues, on maintainability. The code itself was not the problem.

The ability to explain one’s own work. If the submitter of a PR cannot explain the code because they did not write it, the deep understanding necessary for debugging, maintenance, and further development is missing.

Determinism. Safety-critical systems require deterministic, verifiable results. Statistical approximations are not enough there.

Who does the last 10%?

That’s the paradoxical part.

The last 10% requires exactly the expertise that can only be built up through years of practice. Understanding register allocation means understanding CPU architectures. Writing correct relocations means understanding ELF binary formats. Anyone who wants to make code reviewable has to understand software engineering as a social process.

These are skills that cannot be learned through prompting.

The irony: If we train a generation of developers who use AI to skip the first 90%, who will develop the expertise for the last 10%? The question is not meant rhetorically. It is the central challenge facing our industry in the coming years. The problem isn’t just skipping steps, but that the “junior tasks” we used to learn from (the “grunt work”) are now disappearing. You don’t learn compiler design by watching a senior developer but by writing a parser yourself first. If AI does that, the training ground is gone.

“There will always be room for individuals to write software based on their unique knowledge. The craft of writing software will not become obsolete.” (Freeman Dyson, 1998)

This quote was on the old version of this website when I launched it in 2011. It is more relevant today than ever.

What does this mean in practice?

I use AI tools every day. They’re excellent for boilerplate code, for exploring APIs, for initial drafts. Even for complex and large-scale software development. But I’ve stopped viewing them as a substitute for understanding.

AI is good for the initial draft: prototyping, exploration, scaffolding. I settle the architecture beforehand, as described in A Return to the Waterfall Model. But everything that goes into production needs human review. Every line that is deployed must be understood. To do that, you need to know the fundamentals. If you don’t know how a compiler works, you can’t evaluate an AI-generated compiler. It also requires honesty about limitations. “90% done” is not “done.”

90% is a good start. But software craftsmanship is evident in the last 10%.

That’s the most common prompt developers use to start a new project, and the worst one. Not because the model produces nothing. It produces immediately: directory structure, configuration, routing, database schema, UI components. In minutes, something appears that looks like an application.

But it’s not architecture. It’s a hallucination of architecture.

The Architecture Problem

When I tell an AI agent “Build me a project management tool with user roles and time tracking,” it makes dozens of architectural decisions in the first few seconds: Which framework? Which ORM? How are roles modeled (RBAC, ACL, or simple flags)? How is time tracking represented (per task, per project, per day)? Monolith or services?

The model doesn’t decide. It guesses. It picks the statistically most likely setup, that is, whatever shows up in most tutorials and GitHub repos. For a prototype that may be enough. For an application that has to be maintained, extended and operated, it’s a foundation built on chance.

What the Tool Itself Gives Away

A look at the tools themselves is instructive, and a bit meta.

I use Claude Code as my main agent. The tool has a built-in Plan Mode: before any code is written, the agent creates a plan that the human must approve. The tool is literally built to think first and implement second.

In the project root sits a file named CLAUDE.md. It holds architecture rules, conventions, patterns, everything the agent needs to know to write consistent code. Without this file, something predictable happens: the agent reads the existing code, works out what is “normal” from it, and reproduces it. If the code is inconsistent, the new code will be too.

“Claude does not invent architecture. It copies what it sees.” (from the Claude Code documentation)

The irony is hard to miss: The tool itself is built for waterfall. Specify, plan, then implement. In exactly that order. The developers at Anthropic did not build it that way out of nostalgia. It is the only order that works reliably.

Waterfall in 15 Minutes

I’m not the only one who’s noticed this.

Harper Reed has described a workflow that starts with a conversation: he lets the model walk him through his idea question by question until a complete specification is in place, a spec.md. A reasoning model then generates a project plan from it, consisting of small tasks that build on one another. Only then is code written.

Addy Osmani, who works on the Chrome team at Google, describes the same approach and calls it by its name: “Waterfall in 15 minutes.” A compressed but complete planning phase (requirements, architectural decisions, data model, test strategy) before the first line of code is written.

The pattern is clear: the people who work successfully with AI agents plan more than before.

Why Agile Doesn’t Work Here

This contradicts a principle that has shaped our industry for 20 years: let the architecture emerge. Write code, learn as you go, refactor. “Working software over comprehensive documentation.”

This principle assumes that the human learns while coding. That writing code is not only a production process but also a thinking process. That the architecture matures in the developer’s head while they implement.

When an AI agent writes the code, that learning does not happen. The implementation is done in minutes, but the understanding of the decisions inside it is not. The architecture does not emerge, it drifts: each prompt adds a layer on top of the previous one, without anyone holding the whole picture in their head.

My Workflow for Greenfield Projects

Over the past few months I have developed a workflow that tackles these problems. It lacks elegance, but it works.

Phase 1: Architecture Without an Agent

Before an agent sees a single line of code, I write an architecture document, a plain Markdown file (no slide deck) containing:

• Objective: What should the software do? What should it explicitly not do?
• Data model: Which entities exist? How do they relate to each other?
• Technical decisions: Which framework, and why? Which patterns?
• Interfaces: Which APIs, which formats?

This takes an hour. Sometimes two. But I get that time back several times over, because afterwards the agent doesn’t have to guess.

Phase 2: Defining Tasks, Small and Sequential

I break the architecture down into tasks that can be worked through and tested one at a time. Not “Implement user management,” but “Create the user model with these fields” → “Write the registration with these validation rules” → “Implement the login route with these acceptance criteria.”

Every task has a clear outcome and clear boundaries. Within those boundaries, the agent gets context.

Phase 3: Implementation with a Pipeline

Every task runs through an automated pipeline: compile, lint, test. The agent iterates against that pipeline until everything is green. Alongside it run my own quality-gate scripts: checks for open TODOs, swallowed errors, missing tests.

Phase 4: Review Cycle

After each completed task I have the code reviewed, sometimes by a second agent, always by me. Not only for correctness but also for architectural conformance: does the code stick to the decisions from Phase 1? Or has the agent introduced a different pattern without saying so?

The latter happens more often than you would think.

What Looks Old Is Actually New

Yes, this looks like waterfall. Specification → Design → Implementation → Test → Review. In exactly that order.

But there are substantial differences from the waterfall of the 90s:

• Speed: The whole cycle takes hours, the planning one of them. It used to take months, with three of them for the planning alone.
• Iteration: After every pass I can adjust the architecture document. It is a living document and not a contract.
• Granularity: The tasks are so small that errors stay local. A bad prompt affects only a single function.

What has stayed the same is the principle: think before you type. The old waterfall failed because months passed between the thinking and the result, and the requirements changed in between. The new waterfall doesn’t have that problem, because implementation takes only minutes. What used to be too slow is now just right.

Craftsmanship Means Being Able to Plan

In 35 years of software development I have seen both extremes: the rigid waterfall of the 90s with its requirements specifications weighing a ton, and the agile counter-movement that all but declared planning a sin. Neither worked in its pure form.

What is emerging now is a third thing, a workflow the tool enforces: the human thinks, decides the architecture and checks the result. The machine types, implements the architecture and iterates.

This requires skills that the agile world sometimes neglected: designing a system before it exists, and making and justifying decisions before the first test is green. In a word: planning.

90% done in minutes, but only if the architecture is right. No agent writes that.

“Competence means expertise. Expertise belongs to those who can make a subject understandable. Transparency, from planning and method selection all the way to the solution, is therefore both the most fundamental and the most difficult proof of competence.”

That sentence appeared in 1997 on a website I helped run. It was built with framesets, created with “Claris Home Page version 2.0b1”, and served from Erfurt, Germany. At the time there were roughly one million websites in the entire world, and today there are more than 1.4 billion.

My name is Frank Csehan, I have been building software for more than 35 years, and I would write that 1997 sentence the same way today, without changing a word.

When Artificial Intelligence Was Considered a Failure

That same website had a section that was common at the time: “Selected Links”. Between Oracle documentation and Hewlett-Packard downloads, under the heading Data Warehouse and Data Mining, stood this entry:

“The latest attempt, among other things via data mining, to breathe life (figuratively speaking) into the subject of artificial intelligence. The Cyc project.”

That was 1997, and artificial intelligence was something people still had to “breathe life into” because, for all practical purposes, it had already been declared dead. The AI winter had not long passed, and neural networks were treated as academic toys. The hype of the 1980s had faded, funding had dried up, and what remained was data mining: the attempt to extract at least something useful from large volumes of data.

We built data warehouses. If the machine was not going to become intelligent, we would at least organize the data so that humans could make sense of it. Almost thirty years later I put language models to work on tasks nobody back then would have trusted a program with, up to and including complete compilers.

Erfurt, Early 1990s

Part of my story begins in Erfurt, shortly after German reunification. IBYKUS AG für Informationstechnologie was formed in 1996 from the merger of ICE Computer und Netze GmbH, founded in 1990, with IBYKUS Informationssysteme GmbH, founded in 1993. I was there as a member of the management board.

We built Oracle databases for the Thuringian state government. Process and document systems for the state parliament. Sample tracking for veterinary diagnostics. An information system for DASA, the predecessor of Airbus Defence. Underneath ran HP-UX servers, Novell networks, 100VG-AnyLAN. For the Erfurt region we were also an internet service provider, at a time when the internet still had to be explained.

On our software development page stood a sentence that has stayed with me:

“Software development today is an engineering discipline. It is part of our craft.”

The word craft keeps coming back in my working life, at IBYKUS in 1997, in the company name AJAGARA Softwarehandwerk in 2011, and now here, without my ever having chosen it deliberately.

Thirty-Five Years of Changing Technology

In 35 years I have seen technologies appear and disappear: client/server gave way to web applications, Oracle Designer/2000 to Django, Novell networks to the cloud. At every one of those transitions the promise was that the work would get easier. The 100 Mbit connection that caused a stir at the Thuringian Ministry of the Interior in 1995 would be too slow for a printer today.

The underlying problems have remained the same. I ask the same things of software as I did in 1996: that it runs reliably and can be maintained, and that whoever builds it understands the people who will eventually use it. That competence means being able to make a subject understandable applies to an Oracle DBA in 1996 as much as to someone commissioning agents in 2026. What has changed is the speed, and with it the illusion that speed can replace thoroughness.

What I Bring

I came to software from a coal mine. My first trade was industrial mechanic, working underground in the Ruhr area, and university never entered into it. Later I became a systems analyst at a Munich software company, where I built my first systems on hierarchical databases, Assembler, and COBOL: shop-floor control, financial accounting, screen generators.

At Oracle Germany I worked on data modeling, migration consulting, and project leadership, including statewide public-sector systems in South Tyrol and a production planning system for the textile industry. Then came IBYKUS AG in Erfurt, which I helped grow from four people to more than 130. As a board member I was responsible for technology, staffing, finance, and IPO preparation.

Later stations were Oracle E-Business Suite at Transparency International, GS1-based track-and-trace architectures, and training for Oracle University. Since ChatGPT was released to the public in late 2022, I have worked with LLMs as development tools every day, in work that sits close to production.

Languages and tools: Assembler, C, COBOL, PL/SQL, Python, Objective-C, shell script, HTML/XML, and today primarily AI-assisted workflows with Claude Code and Copilot. Databases ranging from hierarchical systems to Oracle, PostgreSQL, and SQLite. ERP systems ranging from shop-floor control to Oracle E-Business Suite.

In every decade I have watched a new technology promise to change everything. Sometimes it delivered, more often it changed one part of the work and left the rest exactly as it was.

Why This Blog

My main job is a permanent position. This blog is mine privately, it is no company presence and it sells nothing. AJAGARA Softwarehandwerk as a company is dormant, and the name still stands for what I think good software is.

I write here about the tension between the 90 percent that AI produces quickly and the last 10 percent that still take experience and craft. What quality means when quantity is suddenly cheap is the question that occupies me most. Added to that are observations from 35 years in the industry.

This is not a look back at a finished career. I am still in the middle of it.

AI Transparency Statement

Every blog post on this site is written by me. I use LLMs for revision and for stylistic improvement. Generative tools help me improve grammar and clarity, but the content and the thoughts behind it are my own.

I also use LLMs regularly to get ideas for simplifying a line of thought, and as a sparring partner when brainstorming blog posts.

Contact

frank.csehan@ajagara.com

LinkedIn