Build versus buy gets treated as a vendor-evaluation spreadsheet exercise. It’s actually the decision that determines whether your AI system scales with the business or collapses the first time load triples.
“Should we build our own model-serving layer?” is really asking “who owns this when it breaks at 2am, and what does it cost us if we’re wrong?” Framed that way, a lot of build-vs-buy decisions answer themselves, and not always in the direction procurement expected.
The infrastructure decision that gets revisited every six months isn’t a decision, it’s a symptom. Make the call once, with the real constraints in front of you: expected scale, integration cost, and what you’re accountable for if the vendor’s roadmap doesn’t match yours. That’s a decision that holds.
Governance usually gets built the same way: quickly, right before an audit, by whoever is free that week. It shows, and boards and regulators can tell the difference between a system designed with guardrails and one that had them stapled on afterward.
A bolted-on audit trail logs what the system did after the fact. A built-in one makes it structurally impossible for the system to act outside its guardrails in the first place. The difference is invisible in a demo and very visible the first time something goes wrong in production.
The best governance reviews we’ve sat through were boring. No jargon deployed to obscure a gap, no slideware standing in for an actual control. Just a plain answer to a plain question, backed by a system built to make that answer true. That’s the bar: could you explain this control to your board and your engineers in the same sentence?
Nobody puts “stalled AI pilot” on a budget line. It shows up disguised as compute spend, a headcount request nobody can justify, and a board slide that says “in progress” for the third quarter running.
A pilot that doesn’t ship isn’t neutral, it’s actively costing you. Seats on tools nobody uses daily. Compute for a staging environment that never graduates. And the least visible cost: a competitor who started later and shipped first, because they treated production as the goal from day one instead of a phase two.
It’s rarely the model. In almost every stalled engagement we’ve inherited, the gap was the same: nobody owned the path from a working notebook to a monitored, on-call production service. The demo team and the production team were different people, with different incentives, and the handoff between them was where the project quietly died.
The pilot was never the hard part. Shipping it was.
Every RAG demo looks the same: upload a handful of clean PDFs, ask a friendly question, watch the model answer correctly. Then it goes to production, meets your actual document set, and the wheels come off.
Most teams pick a chunk size once, during the demo, and never touch it again. But a 500-token chunk that works for a policy handbook falls apart on a table-heavy financial filing. The fix isn’t a bigger model, it’s treating chunking strategy as an engineering decision with its own test suite, not a default left over from a tutorial.
Retrieval systems quietly go stale the moment nobody owns re-indexing. We’ve seen production systems answering confidently from a knowledge base that was six months out of date, because “someone will re-run the embedding job eventually” is not a pipeline.
When a RAG system gives a wrong answer, the instinct is to patch the prompt. Usually the actual fault is upstream: bad chunking, stale indices, or no source-of-truth pipeline at all. Fix the retrieval layer first. The prompt was never the problem.