Point of View

The Debt Nobody Names: Why Enterprise AI Keeps Re-Deciding Nothing

Background

Cloud computing left the enterprise with two kinds of debt nobody fully budgeted for. The first was technical debt in the code itself — shortcuts taken to ship faster, quietly compounding. The second was sprawl: unused instances, orphaned storage, subscriptions nobody remembered provisioning, spend nobody had assigned to own. Both took years to become visible, and both eventually forced a reckoning, usually in the form of a cost-optimization initiative nobody wanted to run.

Enterprise AI is accumulating a third kind of debt, and it looks almost nothing like the first two. It isn't sitting in the code. It isn't sitting in unused infrastructure. It's sitting in a decision that was made once, correctly, and then never revisited.

A team scopes a proof of concept. They pick a model. They write prompts. They size a context window. They wire up a workflow, maybe a single agent, maybe a small chain of them. It works. Leadership is pleased. The POC becomes production. Everyone moves on to the next initiative.

And then, almost without exception, nobody ever goes back to re-size or re-strategize.

The Questions Nobody Reopens

Six questions worth asking about any AI decision still running exactly as it launched:

1. Model choice. Is this still the right model, or has a cheaper one become good enough since launch?
2. Prompt drift. Has the prompt kept pace with how the task itself has evolved?
3. Context window sizing. Is it sized for what the task needs today, or for what someone guessed at launch?
4. Routing. Should this task even be routed the same way, now that better options exist?
5. Quality drift. Has the bar for "good enough" output quietly moved, without anyone deciding it should?
6. Agent-vs-human boundary. Could a step that used to need a person now be handled by an agent — or does a step now run by an agent need stringent human oversight again?

None of these are technology questions. Every one is a decision question. And every one was already answered once, at launch, under time pressure, by whoever happened to be the decision maker in the room.

This Isn't a New Kind of Debt. It's an Old Problem, Recurring.

We've made the case elsewhere that the real constraint on enterprise transformation was never technology. It's decision latency: the delay between knowing something and someone actually acting on it. What's easy to miss is that decision latency doesn't only show up once, at the moment of the original decision. It recurs, indefinitely, every time the conditions underneath that decision change and nobody notices, because nobody was ever assigned to keep watching.

A model choice made in a POC isn't a one-time technical configuration. It's a decision with a shelf life nobody labeled, the same way a product or an application has a shelf life, requiring modernization as the technology or the business process underneath it evolves. The same is true of a prompt, a context window, a routing rule, or an agent boundary. They're just compressed into a far faster pace and shorter feedback loops than a typical modernization cycle ever moves at. Every one of them was a reasonable call under the information available at the time, and every one of them silently expires as the model landscape shifts, as pricing changes, as the task itself evolves, as the organization's own tolerance for automated judgment shifts. The debt accumulates exactly the way cloud sprawl did: not through a single bad decision, but through the absence of anyone whose job it is to periodically ask whether last year's good decision is still this year's good decision.

Why Nobody Reopens It

This isn't a story about laziness or poor engineering discipline. It's a coordination and ownership problem, and it tends to show up in one of three shapes.

Sometimes it's mechanical: there's minimal to no observability in place during the POC phase at all, and even where it exists, it often can't keep pace with how fast the feature itself is changing. Nobody can tell the model has drifted, or that a cheaper option now exists, because nobody built the monitoring that would surface it. The debt is invisible before it's ever a decision to make.

Sometimes it's structural. The observability might even exist, but there's no clearly assigned owner for "revisit this AI decision periodically." It fell under whoever ran the original POC; that person moved to a different initiative, and the responsibility evaporated with them rather than transferring to a role, or into production operations, where it belonged. This is a familiar pattern to anyone who has watched an application move from MVP to production and into business-as-usual — ownership of the original decisions rarely survives that handoff intact.

And sometimes it's political. Revisiting the original choice means implicitly questioning whoever made it, often a senior sponsor who championed the initiative publicly. Nobody wants to be the one who raises that question, so the safer organizational move is silence, and the debt compounds by default.

Three different root causes, three different fixes. Which is why quantifying "how stuck is this" as a single severity score has never been the useful question. The useful question is which of the three is actually at play here. A mechanical gap gets solved with better tools and instrumentation, a structural gap gets solved with clear ownership, and a political gap gets solved when leadership steps in to provide air cover and clear the path. Treat all three the same, and you'll spend a quarter building a dashboard nobody asked for to fix what was actually a trust problem.

A Recent Example

At one global manufacturing company's annual internal innovation event, the written scoring criteria now require every entry to include AI, regardless of whether AI is the right tool for the idea. That mandate is itself worth naming: it rewards visible AI use before anyone has judged whether the AI use is sound.

Two entries from this year's event show the pattern from both sides. One team built a multi-agent orchestration product meant to take a requirement, analyze it, design it, and generate code with minimal human involvement. A senior engineer on that team, since promoted to architect, was separately shown a single, narrowly-scoped prompt: one that asked a model to analyze a requirement against the actual codebase, separate fact from inference, critique the proposed design, and call for an anti-corruption layer where the design looked shallow. His own conclusion was that the orchestration product wasn't close to matching what that one prompt did. The gap wasn't compute or agent count. It was decision quality, settled before any orchestration ever started.

A second entry designed a feature to export structural engineering data to CAD format and back to PDF, merging into the company's core structural report output. It worked, and it won. It also depended on a code library archived and unmaintained since 2023, ran through a paid conversion service accessed via a trial account nobody had evaluated for cost or scale, and was written directly into a decade-old core platform that multiple ongoing initiatives are actively trying to decouple work away from, not add to. When the decoupling question was raised, leadership's answer was to leave it alone: the decision had already been made, and reopening it now would cost more organizational capital than absorbing the debt quietly would.

Mechanical: an end-of-life dependency and an unmanaged SaaS trial. Structural: no gate exists between a hackathon win and a shipped architecture decision. Political: once leadership had publicly praised the outcome, questioning it became the costlier option. Same three buckets, same failure to revisit — just moving at hackathon speed instead of POC speed.

What This Actually Costs

The obsession with AI activity is itself part of why this debt compounds so quietly: more pilots, more copilots, more agents stood up, all pointed at launching rather than revisiting. An organization chasing the appearance of AI momentum has every incentive to keep launching and very little incentive to circle back and ask whether what it already launched still deserves the model, the prompt, or the architecture it was given a year ago. Motion gets mistaken for progress, and the unexamined POC just keeps running, quietly costing more than it should, quietly delivering less than it could, with no one positioned to notice either.

The fix isn't a technology upgrade. It's the same fix decision latency always calls for: someone has to own the recurring question, on a cadence, with actual authority to act on the answer. Not a one-time model selection exercise. Not a slide reviewed once at launch and filed away. An actual, assigned, recurring decision — the same discipline you'd expect for any other asset with a shelf life, applied to the AI decisions everyone quietly assumed were permanent.

The debt was never in the code, and it was never in the model. It was in believing a decision, once made, stays made.

Where in your organization is a model, a prompt, or an agent boundary still running exactly as it was configured at launch — and who, if anyone, would actually notice if it shouldn't be?