Designing Budget Alert Systems That Don't Cry Wolf

A budget alert system that fires too often gets ignored. One that fires too late is useless. The hard part is not implementing alerts; it is tuning them so they fire at the right time with the right action. This article covers the design decisions behind Styrby's budget alert system and the tradeoffs between different approaches.

The Alert Fatigue Problem

Alert fatigue is well-documented in operations monitoring. PagerDuty reports that teams ignore 30% or more of alerts when volume is too high. The same dynamic applies to budget alerts: if your daily budget is $20 and you regularly spend $18, the 80% notification fires most days. After a week, you stop paying attention. The day you actually overspend, you miss the alert.

The fix is not to remove alerts. It is to ensure every alert carries information that requires a decision.

Graduated Actions

Styrby uses three alert tiers, each with a different action:

The key insight: each tier escalates the disruption. Notify is informational. Slow Down is a gentle nudge. Hard Stop requires explicit action. A developer who ignores the notification will notice the slowdown. One who ignores the slowdown will be stopped.

Period-Based vs. Rolling Windows

Budget alerts can reset on a fixed period (daily at midnight, weekly on Monday) or use a rolling window (last 24 hours, last 7 days). Each has tradeoffs.

Period-Based (What Styrby Uses)

Budgets reset at a fixed time. A daily budget of $25 resets at midnight in the user's configured timezone.

• Pro: Easy to understand. "I can spend $25 today" is clear.
• Pro: Aligns with billing cycles. Monthly budgets match monthly invoices.
• Con: Boundary exploitation. A user can spend $25 at 11:59 PM and $25 at 12:01 AM, effectively spending $50 in 2 minutes.

Rolling Windows

A "daily" rolling budget looks at the last 24 hours from now, not from midnight to midnight.

• Pro: No boundary exploitation. $25 in 24 hours means $25 in any 24-hour window.
• Con: Harder to reason about. "How much can I still spend today?" depends on when you spent yesterday.
• Con: More expensive to compute. Checking a rolling window requires summing all cost records in the window for every check.

Why We Chose Period-Based

The boundary exploitation problem sounds bad in theory. In practice, it rarely matters. Developers do not intentionally game their own budget alerts. The mental simplicity of "daily budget resets at midnight" outweighs the theoretical precision of rolling windows.

We also considered the implementation cost. Period-based budgets need one sum query per period: SELECT SUM(cost) WHERE date = today. Rolling windows need a range query for every check. With BRIN indexes on the timestamp column, both are fast, but period-based is simpler to cache and debug.

Threshold Tuning

How do you set the right budget? Too low and you hit alerts constantly. Too high and the alerts never fire when they should.

Our recommendation for new users:

1. Run for one week without budget limits. Just track costs.
2. Calculate your average daily spend and the standard deviation.
3. Set the daily budget at average + 2 standard deviations. This means the budget only triggers on genuinely unusual days.
4. Review monthly and adjust as usage patterns change.

Example: if your average daily spend is $12 with a standard deviation of $4, set the budget at $20. The notify alert at 80% ($16) fires on high-spend days. The hard stop at 100% ($20) fires only on unusually expensive days.

Multiple Budget Scopes

Styrby supports budgets at multiple levels:

• Global: Total spend across all agents and projects
• Per-agent: Separate limits for expensive vs. cheap models
• Per-session: Limit any single session's cost

These scopes are evaluated independently. A per-session limit of $10 can fire even if the daily budget is nowhere near its limit. This catches runaway individual sessions without affecting the overall budget.

Database Design for Budget Checking

Budget checks happen frequently (after every agent message), so they need to be fast. Styrby uses a materialized view for daily cost summaries:

-- Materialized view refreshed every 5 minutes
CREATE MATERIALIZED VIEW mv_daily_cost_summary AS
SELECT
  user_id,
  agent_type,
  project,
  date_trunc('day', recorded_at) AS day,
  SUM(total_cost_usd) AS total_cost,
  COUNT(*) AS record_count
FROM cost_records
WHERE recorded_at >= now() - interval '90 days'
GROUP BY user_id, agent_type, project, date_trunc('day', recorded_at);

-- Budget check query (fast, uses materialized view)
SELECT SUM(total_cost) FROM mv_daily_cost_summary
WHERE user_id = $1 AND day = current_date;

The tradeoff: materialized views are slightly stale (up to 5 minutes). For budget alerts, this delay is acceptable. A 5-minute lag on a daily budget means you might overshoot by 5 minutes of spending before the alert fires. At typical token costs, that is a few cents.

What We Would Do Differently

If starting over, we would add predictive alerts: "at your current rate, you will hit your daily budget by 3 PM." This is more useful than threshold alerts for preventing overspend because it gives earlier warning. We plan to add this in a future release.