Skip to main content
The Grafana “LiteLLM Spend & Budgets” dashboard (uid litellm-spend). Reads LiteLLM’s spend data directly from CNPG Postgres via a least-privilege read-only role: no LiteLLM Enterprise, no Prometheus, no Langfuse. LiteLLM Spend and Budgets dashboard showing total spend, spend by model, spend by key, and remaining budget per key Spend over time, spend by model and by key, and remaining budget per key, served from CNPG Postgres.

​
Model

  • DB role grafana_ro: CNPG managed role, pg_read_all_data (SELECT-only). Defined in platform/litellm/db/cluster.yaml; password GSM litellm-grafana-ro-password → ESO secret litellm-grafana-ro (ns litellm).
  • Datasource: ESO-rendered Secret grafana-datasource-litellm (ns monitoring) labelled grafana_datasource: "1"; the Grafana datasource sidecar auto-imports it (platform/observability/values.yaml).
  • Dashboard: dashboards/litellm/litellm-spend-dashboard.json (uid litellm-spend), loaded by the grafana-litellm app in the llm-gateway profile.

​
1. Prereq: the secret (do this first)

Then sync litellm-bootstrap (creates the role + ESO) and grafana-litellm (datasource + dashboard). CNPG reconciles the grafana_ro role from the secret automatically.

​
2. View it

Make a budgeted call through LiteLLM first (see litellm.md §3) so there are spend rows to show.

​
3. Troubleshooting

Datasource missing in Grafana. The datasource sidecar must be enabled (grafana.sidecar.datasources.enabled: true, searchNamespace: ALL) and the Secret must carry the grafana_datasource: "1" label. Check it rendered:
Datasource present but “connection refused” / auth failed.
  • Confirm the role exists: kubectl -n litellm exec -it litellm-pg-1 -- psql -U postgres -c '\du grafana_ro'.
  • Confirm the ESO secret resolved: kubectl -n litellm get secret litellm-grafana-ro and kubectl -n litellm get externalsecret litellm-grafana-ro.
  • The datasource targets litellm-pg-rw (the writable primary). With a single instance the -ro service has no endpoints: do not point the datasource at it until HA adds replicas.
Empty panels. No spend logged yet (make a call), or the time range predates any traffic, or LiteLLM changed a table/column name. The SQL quotes "LiteLLM_SpendLogs" / "LiteLLM_VerificationToken" and "startTime"; re-check column names with \d "LiteLLM_SpendLogs" if a LiteLLM bump breaks a panel.

​
4. Not here

Per-request prompt/token tracing is Langfuse (deferred): its v3 footprint is the reason. LiteLLM’s own Prometheus metrics are Enterprise-gated and intentionally unused.