One post tagged with "observability"

LiteLLM isn't an infrastructure detail. It's the money bottleneck for almost everything we do with models: the Resizes AI chat, customer agents, internal pipelines. When the bill went up, the instinct was "the chat is expensive." The truth was broader: a shared proxy, no map, and an agent that turns one message into a loop of calls.

This post covers how we saw it, how we almost broke it by blindly cutting turns, and what ended up in code: per-agent and per-conversation attribution, less context, fewer tools, a real cap in the runtime, and a Grafana dashboard covering everything that goes through LiteLLM.