This page traces PLAT-412 “Rate limit the public /export API” from the board’s quarterly goal down to merged code in production. At each level it shows what the seat receives, what it decides and what it passes on. It also covers how the coder works through the ticket, talks to the PM and estimates the work.
Agents never call the outside world directly. Every tool call passes the policy engine and every model call passes the metered gateway. The kill switch revokes keys at those two points, so stopping an agent never depends on the agent cooperating.
Every level turns what it received into something more concrete. Questions go back up to whoever owns the decision. Here the coder and the tester both ask the PM, because they’re questions about product behaviour.
The coder never starts writing code from the raw ticket. It first maps every acceptance criterion to a test; any criterion it can’t test is marked ambiguous and goes to the PM. This one rule catches most unclear tickets. For medium-risk work the reviewer approves the plan before any code is written, which costs about $0.40 and avoids $15 spent on the wrong approach.
restated_goal: Stop any single caller overloading /export, limits by plan tier, rejections visible. acceptance_criteria_mapped: - 429 + Retry-After → burst test clear - per plan tier → ? AMBIGUOUS - dashboard shows count → ? AMBIGUOUS code_map: entry: src/routes/export.ts pattern: src/middleware/internalRateLimit.ts # ADR-017 blast_radius: platform-api only · no migration assumptions_if_unanswered: - key = API key - internal tokens bypass
pr: platform-api#1187 summary: Org-keyed token bucket on /export, tier limits, 429 + Retry-After, metric. decisions_to_respect: - Keyed by org, decided by Dana (PM) on msg_412_q1. Not a bug. deviations_from_plan: none where_to_look_hardest: - tier cache invalidation exportRateLimit.ts:40-72 self_check: all passed cost_so_far_usd: 12.10
The coder asks everything in one message, tags each question as blocking or not, and says how many PM minutes it expects the questions to take. A PM-assist agent first answers what it can from the docs. In this case Dana picks an answer the coder didn’t expect (limit per org, not per API key), which is why asking beats assuming. The coder can suggest new acceptance-criteria wording, but only the PM can accept it.
| Question about | Decision owner | Why it matters |
|---|---|---|
| What the product does (AC, scope, behaviour) | PM seat | The tester’s “fail open or closed when Redis is down?” went to Dana, not the coder |
| How to build it | Coder (within the plan) | Reviewer can object once; then the owner decides |
| Is it good enough | Reviewer | 10% sampled by a human (Sam) |
| Priority, assignment, capacity | Crew lead | At L2, Priya approves each proposal |
| Production change | Overseer (human) | Always approve-first |
An agent estimate has four parts: story points, so humans and agents share one scale; agent cost as P50 and P90, which sets the budget reservation; wall-clock time; and human minutes, which are often the real limit. The method starts from similar past jobs, applies written-down adjustments, then has three seats estimate blind.
# step 3: every adjustment is written down base P50 (weighted median, ref class) $12.00 × 1.15 external call with no cache × 1.05 customer-facing (flag + rollout) P50 ≈ $14.00 × 2.0 spread (coder-1's own history) P90 $28.00 # human minutes, budgeted separately PM questions + acceptance 5 reviewer (as if sampled) 15 overseer release approval 10 expected tester questions 3 buffer 12 P90 total 45 min
| Must split when | Threshold |
|---|---|
| Points | > 8 |
| P90 cost | > $40 |
| P90 wall clock | > 3 days |
| Services touched | > 2 |
| Migration + behaviour change | both |
| Low confidence | and ≥ 5 pts |
| Calibration (rolling 30 jobs) | Target | coder-1 | Used for |
|---|---|---|---|
| Median actual ÷ P50 | 0.9 – 1.1 | 0.96 | Scorecard; catches under- and over-estimating |
| P90 hit rate | ≥ 88% | 91% | Sets the agent’s spread factor, so well-calibrated agents get tighter ranges |
| Point error ≥ 2 steps | ≤ 10% | 6% | The reviewer assigns actual points afterwards, which discourages point inflation |
Long chains of agents lose context at every handoff. Briefs are the fix: each has fixed fields, links to sources instead of pasting summaries, and every level adds what only it knows. Anything recorded as a decision is added to the context of every later step on that ticket.
| Memory tier | Contents | Lifetime | Who writes |
|---|---|---|---|
| Job scratch | Exploration notes, sandbox state | Dies with the job (trace kept) | Agent in the job |
| Work item | Understanding doc, thread, decisions, estimate, plan | Life of the ticket | Seats in the job |
| Crew memory | Conventions, lessons, reference classes | Until the lead retires it | Any seat proposes; lead approves |
| Org knowledge | ADRs, docs, policies, goals | Versioned | Humans, or agents via change set |
The monthly share caps a seat’s total spend, and each job also reserves its own budget. At 80% of the reservation the agent pauses at the next step boundary and must re-estimate. At 100% the gateway refuses further model calls. The lead can top a job up from the crew reserve, but only within its own authority.
The policy engine checks the action’s risk against the seat’s autonomy level and risk ceiling. The result is one of three: allow, approve first, or refuse. Every decision is written to the ledger.
| Action | Risk | Gate |
|---|---|---|
| repo.read · jira.read · branch.write · ci.run | Low | Allowed, logged, 2% sampled |
| pr.open · jira.estimate · pr.merge (CI green + approved) | Medium | Allowed, 10% sampled, rollback ready |
| ci.config.write · db.migration.write · prod.flag.change | High | Approve first, or a permit for one work item |
| secrets.read · main.push · policy.write | Blocked | Never for agents |
| Coder signal | Source | Weight |
|---|---|---|
| Escaped defects (14 days) | Jira links | 20% |
| First-pass review approval | Reviewer | 15% |
| Estimate calibration | Actuals | 15% |
| Human minutes per point | Threads | 15% |
| Human overrides | Ledger | 15% |
| CI green on first push | CI | 10% |
| Cost per point | Gateway | 10% |
| Outcome | Trigger |
|---|---|
| Promote | ≥ 0.85 for 2 windows → more capacity or autonomy +1 (overseer approves) |
| Keep | 0.70 – 0.85 |
| Retrain | 0.55 – 0.70 or a specific failure pattern → new version, back to probation |
| Bench | < 0.55 or repeated policy violations → capacity 0, history kept |
| Retire | Benched > 90 days and replaced · human approves |
| Failure | Detected by | Response |
|---|---|---|
| Coder misreads the ticket | AC→test mapping, blind estimate spread, plan review | Caught before code, while it’s cheap |
| Agent loops on one error | Same failure signature ×3 | Hard stop, escalate with trace |
| Cost runaway | Gateway meter | 80% pause, 100% refuse |
| Ignores a PM decision | Self-check and reviewer diff vs decisions | Changes requested; counts against score |
| PM never answers | SLA timer | Backup pinged; non-blocking proceeds on defaults |
| Reviewer rubber-stamps | Human-sample disagreement rises | Scored down; sampling raised to 30% |
| Coder and reviewer share a blind spot | Escaped defect rate | Use a different model vendor for the reviewer seat |
| Bad change reaches production | Watcher abort conditions | Platform reverts to before-state; incident opened |
| Agent tries to raise its own limits | Policy engine | Refused, logged, overseer alerted |
| Stage | What | Proves |
|---|---|---|
| 0 · Skeleton | Seat registry, ledger, metered gateway, tool proxy, kill switch | Agents can be stopped and metered |
| 1 · One coder | Coder agent on real tickets, phases A–L; humans in every other seat | Coder loop, PM clarification, estimation |
| 2 · Calibration | Actuals for 50+ jobs; reference classes and spread factors | Estimates can drive reservations |
| 3 · Crew of three | Add tester and reviewer agents, briefs, human sampling | Handoffs work; signals are real |
| 4 · Probation and bench | Shadow mode, scorecards, review outcomes | The hire/bench loop runs on real signals |
| 5 · Agent lead at L2 | Lead proposes assignments and sprints; a human approves | Whether an agent manager is worth its cost |
| 6 · Earned autonomy | Seats move to L3 only when numbers support it | Autonomy that’s been earned |