Put AI in the pipeline, not in the demo

Every company I have watched adopt AI has gone through roughly the same arc. Someone builds a chatbot. It demos well. It ships. Usage is fine for a month and then flat. Meanwhile the delivery process — the actual machine that turns requirements into running software — is exactly as slow as it was before.

The mistake is a layer mistake. AI got applied to the product surface, where it is visible, instead of to the delivery pipeline, where it compounds.

Where the time actually goes

Before adding AI anywhere, it is worth being honest about where a two-week cycle really goes. In my experience on enterprise projects, coding is rarely the bottleneck. The bottleneck is queues:

• Waiting for a requirement to be clarified
• Waiting for a review
• Waiting for a test suite that takes 40 minutes
• Waiting for someone who understands the legacy module to be free
• Re-doing work because the requirement meant something else

None of those are solved by a chatbot in the product. All of them are addressable in the pipeline.

The four places it has actually paid off

Requirements triage. Enterprise requirements arrive as prose — emails, meeting notes, a Word document with tracked changes from three stakeholders. The first engineering task is turning that into something specific. A model is genuinely good at first-pass extraction: pulling out the implied entities, listing the ambiguities, flagging the two sentences that contradict each other. It does not replace the conversation with the client. It makes the conversation shorter and better-targeted, because you arrive with a list of exactly what is unclear.

Review pre-pass. Not "AI reviews the PR and approves it" — that is the demo version, and it is bad. The useful version is narrower: run a model over the diff before a human sees it, and have it catch the class of things humans are bad at catching on a screen. Inconsistent error handling. A migration without a rollback. A new endpoint that nobody added to the auth policy. The human reviewer then spends their attention on design, which is the only part a human is uniquely good at.

Test generation for the boring half. Most test suites have a long tail of cases nobody wrote because writing them is tedious and each one is individually low-value. Boundary conditions, null handling, the fourth and fifth enum variant. Generated tests are excellent here and poor at the cases that require actually understanding the domain. Use them for the tail; write the important ones yourself.

Legacy archaeology. The single highest-leverage use I have found. When you inherit a module with no documentation and no original author, the expensive part is building a mental model. Pointing a model at it and asking targeted questions — what writes to this table, what happens if this is null, where does this config come from — collapses days of reading into hours. It will be wrong sometimes. You verify. It is still enormously faster than starting cold.

The rule that keeps it honest

Every one of those has the same shape: the model produces a draft, a human owns the result.

The moment you invert that — the model's output goes to production without a human who is accountable for it — you have not saved time. You have moved the work later, into an incident, where it costs more.

This is not caution for its own sake. It is where the economics actually are. A generated test that nobody read is worse than no test, because it produces confidence without coverage. A generated migration that nobody checked is a Saturday.

What this looks like in practice

Concretely, on the projects I work on:

1. Automate the queue, not the craft. If a step involves a human waiting for another human, that is a candidate. If a step involves a human making a judgement call, that is not.
2. Put it where the feedback is fast. AI in CI is good because CI tells you immediately when it was wrong. AI in a nightly batch job that emails a report is bad, because nobody reads the report and nothing corrects it.
3. Measure cycle time, not adoption. "80% of engineers use the tool" is not a result. "The median PR now merges in 6 hours instead of 26" is a result.
4. Keep the local option. Some codebases cannot leave the building. Running models locally with Ollama is slower and less capable, and it is often the only version of this that is legally available. Plan for it rather than discovering the constraint late.

The unglamorous conclusion

The AI work that has actually changed how fast my team ships is invisible to users. It is a review pre-pass, a requirements extractor, and a set of generated edge-case tests. None of it would demo well. All of it compounds every single sprint.

The chatbot was more fun to build. The pipeline is what shipped the software.