When clients ask me to estimate the cost of an AI feature, they mean the token bill. That number is easy to forecast and, in my experience, rarely the thing that blows the budget. The costs that actually hurt are the ones that don't show up until you're already in production.
The costs that don't show up in the pricing calculator
- Evaluation infrastructure. Someone has to build and maintain a way to know whether the system is getting better or worse after every change. Without it, every deploy is a guess. This is real engineering time that has nothing to do with model calls.
- Prompt and context drift. The prompt that worked great in March quietly degrades as your data changes, as the underlying model gets updated by the provider, or as usage patterns shift. Nobody bills you for re-discovering this in October.
- Latency tax on the rest of the product. Adding an LLM call to a flow that used to be instant changes your UX contract. You now need loading states, streaming, timeout handling, and fallback paths that didn't exist before — real frontend and backend work.
- Vendor lock-in disguised as convenience. Building deeply against one provider's function-calling format, structured output schema, or fine-tuning pipeline feels productive until you need to switch providers for cost, latency, or an outage — and discover how much is welded to their specific API shape.
The line item that surprised me most
Human review time. Every production LLM feature I've shipped eventually needed a human-in-the-loop review step somewhere — flagged low-confidence outputs, edge cases the eval set didn't cover, escalations from an agent workflow. Nobody scopes this at the start because it feels like it should be automatable "later." Later arrives fast, and by then it's urgent, not planned.
Budgeted: API costs, hosting, one-time integration work
Actually paid for: API costs, hosting, integration work,
+ eval pipeline maintenance
+ prompt/context monitoring
+ UX rework for latency
+ ongoing human review capacity
What I budget for now
On every new AI feature at Soletechnix, I now scope four things before writing the first prompt: an evaluation approach, a monitoring plan for drift, a UX plan for latency and failure states, and a rough estimate of ongoing human review load. None of these show up on a token-pricing page. All of them show up on the invoice eventually, one way or another.
The token bill is the cost you can see. Budget for the ones you can't yet — they're the ones that actually decide whether the feature survives its first six months in production.