> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensorcost.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Budgets and alerts
> How budgets, thresholds, and burn rate actually work — and where the alerting story is honestly incomplete today.
# Budgets and alerts
Budgets are TensorCost's spend caps; alerts are what fires when one is crossed, or when a policy is violated. This page documents both, and is deliberately specific about which alert paths actually run on a schedule versus which only work when you check them by hand — that distinction matters more here than almost anywhere else in the product.
## Budgets
A budget is defined against one **dimension** — tenant, environment, team, project, provider, model, instance family, tag, or a custom key — for one **period** (week, month, quarter, or year), with a cap amount and an **enforcement mode**:
| Mode | What it does today |
| ---------- | --------------------------------------------------------------- |
| `alert` | Fires an alert when a threshold is crossed. Fully working. |
| `soft_cap` | Accepted as a valid mode; behaves identically to `alert` today. |
| `hard_cap` | Accepted as a valid mode; behaves identically to `alert` today. |
`soft_cap` and `hard_cap` do not currently gate or block anything — there is no admission-control path in the product yet that stops spend when a hard-cap budget is exceeded. All three modes alert; only `alert` is honestly named for what it does. Don't rely on `hard_cap` as an enforcement mechanism until this ships.
Budgets can be nested — a child budget's cap can't exceed its parent's, and reducing a parent's cap below what its children already sum to is rejected — so a tenant-level budget can have team-level budgets underneath it that must stay within the total.
### Thresholds — actual spend, fixed percentages
Every budget carries a list of **alert thresholds** — percentages of the cap that, once actual spend crosses them, fire an alert. The budget-creation form in the console sets these to **80% and 100%** by default. The underlying model allows any set of percentages up to 200% of the cap, configurable per budget via the API, but the console form doesn't expose that — what you create from the UI today is fixed at 80/100.
These thresholds compare **current utilization** (actual spend so far in the period, as a percentage of the cap) against a fixed number. They do **not** move as a team's normal spend grows, and they do not fire on a forecast — only on spend that has already happened.
### What the forecast fields do — and don't do
Every budget's status computation includes two forward-looking figures:
* **`days_to_overrun`** — at the current burn rate, how many days until this budget exceeds its cap.
* **`projected_overage_cents`** — the projected dollar amount over cap by period end, at the current rate.
Both are computed and shown in the console — the Budgets list and the per-budget detail page both surface "projected to exceed cap in about N days, roughly \$X over by period end" when the projection is close enough to be worth a call-out. But neither figure drives an alert. Crossing 80% or 100% of actual spend fires an alert; a budget that's on pace to blow through its cap in three days, while still under 80% today, does not.
If you want an alert that can fire *before* the cap is reached, see the pace-based `budget-burn-rate` rule described below — it exists, but it isn't creatable from the console today.
### Budget detail
Each budget has a dedicated detail page — the enforcement mode, current-period utilization, a burn-down chart (cumulative spend against a linear path to the cap), and a 12-period trend table showing how this budget performed in prior periods. Useful for a bookmark or an alert-email deep link, not just the list view.
## Alerts (Rules · Alerts)
The Alerts tab shows two live feeds: currently open alerts and alerts resolved in the last 24 hours. This is the surface a policy violation or a budget breach lands on.
### What actually fires an alert
There are two genuinely working alert paths today:
1. **Budget threshold crossings.** When a budget's actual utilization crosses one of its alert thresholds (see above), an alert is created automatically — severity `critical` at 100%, `warning` at ≥ 90%, `info` below that. This runs every time new cost data is ingested; no manual step is needed.
2. **Burn-rate rules (`budget-burn-rate`).** A pace-based rule: actual spend so far, compared against a pace-adjusted expectation for elapsed time in the period (e.g. "you're at 60% of your monthly budget, but only 40% of the month has passed"). This is the one alert type in the product that reasons about pace rather than a flat percentage, and it **is** swept on a schedule (weekly) and does fire real alerts.
### What does not run automatically — even though the form lets you create it
The "+ New rule" form on the Alerts page lets you build a simple metric/operator/threshold condition — for example, "GPU utilization below 10% for 2 hours." Creating one of these gives you a working **Test** button, which previews how many times the rule would have fired against historical data. It does not give you a live rule. There is no scheduled job anywhere in the product that evaluates a plain metric-threshold rule and creates a real alert from it — only the `budget-burn-rate` rule kind is actually swept, and that kind isn't buildable from this form at all (it's API-only today).
If you create a metric-threshold alert rule from the console today, it will never fire on its own. The only way to see whether it would have caught something is to re-run **Test** by hand. This is a real gap, not a UI bug — document it for anyone relying on these rules for live monitoring.
### Other honest gaps in the alerting model
* **Severity is fixed, not configurable.** Budget-threshold severity is a hardcoded mapping (100% → critical, ≥ 90% → warning, else info); there's no way for a tenant to redefine what counts as "high" or to subscribe only to critical alerts.
* **No owner or routing field.** Alerts and policy violations route to notification channels (Slack, PagerDuty, webhook) — destinations, not accountable people. There's no field that assigns an open alert to a person or team; "acknowledged by" records who reacted after the fact, not who was on the hook before it fired.
* **Detection runs on raw metrics, not cost-to-serve.** Rule conditions and the anomaly detector both key on raw values (GPU utilization, memory, temperature, or raw spend) — nothing in the alerting or anomaly pipeline normalizes against usage or revenue, so a spend increase that tracks legitimate growth looks identical to a real problem.
## Two related Rules tabs
**Hardware anomalies** (Rules · Hardware anomalies) is a separate surface: a statistical outlier detector over raw GPU telemetry — utilization, memory, temperature, power draw — meant to catch a genuinely broken instance, not a spend problem. It shares the same three gaps listed above (fixed severity bands, no ownership, no cost-to-serve context), and it's deliberately not folded into the Alerts feed — a spend-alert reader and a hardware-telemetry reader are different audiences, and merging them was tried and reverted for exactly that reason.
**Actions** (Rules · Actions) is the operational queue for what enforcement has done or is about to do — approve or reject pending autonomous actions (see [Policies, guardrails, and routing](/policies-guardrails-and-routing)), and audit what already ran. It's downstream of a policy firing, not an alerting surface itself.