The alert fired at 2:17 AM. Not a crash, not a 503, but a spike in parsing errors from the customer onboarding service. A junior engineer, bleary-eyed and scrolling through logs, found the culprit. The JSON object returned by the new AI-powered address verification API looked different. Yesterday, the key was streetAddress. Today, it was street_address. The system, built on the assumption that machines keep their promises, choked. The contract was broken.
Except there was no contract. Not really.
For two decades, we built complex, distributed systems on a simple, bedrock principle: the Application Programming Interface (API) is a formal agreement. A system exposes a set of endpoints, each accepting a predictable input and returning a predictable output, defined by a rigid schema. It was the digital handshake. You could build a business on a well-documented REST API. You could generate client libraries from an OpenAPI spec. It was deterministic, machine-readable, and wonderfully boring. That boredom was the foundation of reliability.
Now, we are wiring our core business logic to large language models, and we are discovering that the handshake has been replaced by a shifty-eyed nod. The new contract is a negotiation, happening in real time, with an entity that has no memory of its previous promises.
The interface for an LLM is not a structured set of commands; it’s a text box. The input is a prompt, a piece of natural language prose. The
Generated by Reportify AI — Automate your team's status reports, standups, and weekly updates. Try free →