An alert fires at 2:17 AM. The new AI-powered customer service bot is telling users they can get refunds on non-refundable airline tickets. An engineer, bleary-eyed and mainlining coffee, starts digging. She checks the Kubernetes pods. They're healthy. She looks at the API logs. Status 200 OK. The model is responding, the Python code is executing, the infrastructure is solid. There is no bug, not in the traditional sense.
After two hours, she finds it. The problem isn't in the code. It's in a single sentence, buried on page 47 of a marketing PDF that was updated three weeks ago and quietly ingested by the retrieval-augmented generation system. A copywriter, trying to simplify the language around "fare class exceptions," inadvertently created an ambiguity. The RAG system picked it up, the LLM interpreted it literally, and the company is now on the hook for thousands of dollars in fraudulent refunds.
This is the new technical debt. It’s not algorithmic; it’s semantic.
For decades, we’ve developed a sophisticated culture around managing code-based debt. We have linters, static analysis, version control, and rigorous testing suites. We can trace a failure from a user-facing error back to a specific line of code committed by a specific developer on a specific day. We have a language for this. We have a process.
That entire toolchain is useless when the bug is a matter of interpretation. Semantic debt accumulates in the squishy, human-language layers of the stack that we are now treating as core infrastructure. It lives in the master system prompt, a 2,000-word essay that is edited on the fly by a product manager to be more "helpful." It festers in the thousands of documents—Confluence pages, Word docs, Zendesk articles—that form the knowledge base for a RAG system. It is embedded in the JSONL file of hand-picked examples used to fine-tune a base model, a set of assumptions about the world that will never be revisited.
Each of these components is a dependency, but they don't show up in a dependency graph. They are invisible liabilities. Changing a single phrase in a core prompt to fix one edge case can silently break three others. Deleting an outdated document from the vector database might fix a known issue but introduce a new hallucination because the model has lost a piece of
Generated by Reportify AI — Automate your team's status reports, standups, and weekly updates. Try free →