← Concept page Agent org model · architecture deep dive

One Jira ticket, followed through every level of the org

This page traces PLAT-412 “Rate limit the public /export API” from the board’s quarterly goal down to merged code in production. At each level it shows what the seat receives, what it decides and what it passes on. It also covers how the coder works through the ticket, talks to the PM and estimates the work.

1 · The big picture

Agents run in one layer. Everything that limits them is plain software.

Agents never call the outside world directly. Every tool call passes the policy engine and every model call passes the metered gateway. The kill switch revokes keys at those two points, so stopping an agent never depends on the agent cooperating.

Control Room · the website humans use Org chart · crew lead console · approvals inbox · ledger viewer · kill switch humans approve, pause, roll back kill switch: revoke model keys + tool tokens CONTROL PLANE · NO MODEL IN THE DECISION PATH Org & seatswho sits where Capacityshares, reservations Policy engineallow · gate · refuse Approvalsinbox, permits, SLAs Ledgerappend-only, rollback ScorecardsHR signals, reviews limits · approvals · policy decisions every call metered into capacity RUNTIME · WHERE AGENTS RUN Orchestratordurable jobs, stop rules Agent workersrun one step at a time Sandboxesper job, no prod secrets Model gatewaymeters every model call Tool proxyseat-scoped tokens read context, propose lessons all external calls KNOWLEDGE Work itemsdocs, threads, decisions Crew memorylessons, conventions Job historyactuals for estimates INTEGRATIONS · ONLY VIA THE TOOL PROXY Jiratickets GitHubcode, PRs CItests Slacknotify Deployflags, prod
Humans Control plane (deterministic) Agents Platform services - - → kill switch path
2 · The ticket journey

What each level adds before handing the work down

Every level turns what it received into something more concrete. Questions go back up to whoever owns the decision. Here the coder and the tester both ask the PM, because they’re questions about product behaviour.

SEATWHAT IT DOESHANDS DOWN questions to the PM
Human seat Agent crew lead Worker agent Staff function Artifact passed down
3 · Every level, in detail

Each seat uses the same card: in, decides, out, escalates, scored on

Sprints run out of human minutes before dollars. In planning, the lead fills the sprint until the P90 cost reaches the crew’s capacity minus a 15% reserve ($980 of $1,020). It then drops PLAT-421 because PM minutes came to 410 against Dana’s 300 per week. The lead has to fit both budgets.
4 · Inside the coder seat

Twelve phases, each a resumable step with its own trace and cost

The coder never starts writing code from the raw ticket. It first maps every acceptance criterion to a test; any criterion it can’t test is marked ambiguous and goes to the PM. This one rule catches most unclear tickets. For medium-risk work the reviewer approves the plan before any code is written, which costs about $0.40 and avoids $15 spent on the wrong approach.

not ready answered fail changes requested (max 3 rounds) HARD STOP RULES · ENFORCED BY THE ORCHESTRATOR, NOT THE PROMPT

Phase B output: the understanding doc

restated_goal: Stop any single caller overloading /export,
  limits by plan tier, rejections visible.
acceptance_criteria_mapped:
  - 429 + Retry-After     → burst test       clear
  - per plan tier         → ?                AMBIGUOUS
  - dashboard shows count → ?                AMBIGUOUS
code_map:
  entry: src/routes/export.ts
  pattern: src/middleware/internalRateLimit.ts # ADR-017
blast_radius: platform-api only · no migration
assumptions_if_unanswered:
  - key = API key
  - internal tokens bypass

Phase I output: brief to the reviewer

pr: platform-api#1187
summary: Org-keyed token bucket on /export,
  tier limits, 429 + Retry-After, metric.
decisions_to_respect:
  - Keyed by org, decided by Dana (PM)
    on msg_412_q1. Not a bug.
deviations_from_plan: none
where_to_look_hardest:
  - tier cache invalidation
    exportRateLimit.ts:40-72
self_check: all passed
cost_so_far_usd: 12.10
5 · Discussing with the PM

One batch of questions, each with options and a default

The coder asks everything in one message, tags each question as blocking or not, and says how many PM minutes it expects the questions to take. A PM-assist agent first answers what it can from the docs. In this case Dana picks an answer the coder didn’t expect (limit per org, not per API key), which is why asking beats assuming. The coder can suggest new acceptance-criteria wording, but only the PM can accept it.

Coder-1agent · L3 Thread · Jiraplatform PM-assistagent · L2 Dana · PMhuman Platform Leadagent · L2 question_batch · 4 Qs · ~4 PM min new questions on PLAT-412 checks docs + past decisions suggests q4: yes, per ADR-017 notify: 2 blocking, defaults ready q1 → b (organization) · q2 a · q3 a · q4 confirmed decision event · binding context updates understanding doc proposal: rewrite AC #2 only the PM can change acceptance criteria accepts “limits per plan tier, keyed by organization” work_item.ready · Definition of Ready passed No answer in 1 business day → non-blocking questions proceed on defaults, marked as assumptions on the ticket · blocking ones go to the lead or the PM’s backup
Question aboutDecision ownerWhy it matters
What the product does (AC, scope, behaviour)PM seatThe tester’s “fail open or closed when Redis is down?” went to Dana, not the coder
How to build itCoder (within the plan)Reviewer can object once; then the owner decides
Is it good enoughReviewer10% sampled by a human (Sam)
Priority, assignment, capacityCrew leadAt L2, Priya approves each proposal
Production changeOverseer (human)Always approve-first
6 · Estimation

Points for people, P50/P90 for the budget, minutes for the humans

An agent estimate has four parts: story points, so humans and agents share one scale; agent cost as P50 and P90, which sets the budget reservation; wall-clock time; and human minutes, which are often the real limit. The method starts from similar past jobs, applies written-down adjustments, then has three seats estimate blind.

if spread > 1 step REFERENCE CLASS · ACTUAL COST OF SIMILAR JOBS PLAT-412 · ESTIMATE VS ACTUAL base $12 P50 $14 P90 $28 actual $13.20 ✓ Blind:
# step 3: every adjustment is written down
base P50 (weighted median, ref class)  $12.00
× 1.15  external call with no cache
× 1.05  customer-facing (flag + rollout)
P50                                    ≈ $14.00
× 2.0   spread (coder-1's own history)
P90                                    $28.00

# human minutes, budgeted separately
PM questions + acceptance      5
reviewer (as if sampled)      15
overseer release approval     10
expected tester questions      3
buffer                        12
P90 total                     45 min
Must split whenThreshold
Points> 8
P90 cost> $40
P90 wall clock> 3 days
Services touched> 2
Migration + behaviour changeboth
Low confidenceand ≥ 5 pts
Calibration (rolling 30 jobs)Targetcoder-1Used for
Median actual ÷ P500.9 – 1.10.96Scorecard; catches under- and over-estimating
P90 hit rate≥ 88%91%Sets the agent’s spread factor, so well-calibrated agents get tighter ranges
Point error ≥ 2 steps≤ 10%6%The reviewer assigns actual points afterwards, which discourages point inflation
7 · Context and memory

Each handoff adds context. Nothing is just forwarded.

Long chains of agents lose context at every handoff. Briefs are the fix: each has fixed fields, links to sources instead of pasting summaries, and every level adds what only it knows. Anything recorded as a decision is added to the context of every later step on that ticket.

CTO → Lead

Goal

  • goal−50% customer-facing API incidents
  • epicPLAT-EPIC-40
  • metricexport 5xx < 0.1%
PM → Lead

Story

+ inherits 3 fields
  • whyScrapers caused 3 export incidents in October
  • acceptance429 + Retry-After · per tier · visible counts
Lead → Coder

Assignment

+ inherits 5 fields
  • constraintsExisting Redis only · don’t change response schema
  • known contextADR-017 token bucket · tiers at billing /v1/plans
  • budget$20 reservation · sprint 23
Thread → Coder

Decisions

+ inherits 8 fields
  • decided · DanaKey by organization
  • decided · DanaFree 10 · Pro 60 · Ent 600 /min
  • decided · DanaEmit metric; ops owns dashboard
  • decided · DanaFail open, alert ops
Coder → Reviewer

Handoff

+ inherits 12 fields
  • respectOrg key is a decision, not a bug
  • look hardestTier cache invalidation
  • deviationsNone · cost so far $12.10
Memory tierContentsLifetimeWho writes
Job scratchExploration notes, sandbox stateDies with the job (trace kept)Agent in the job
Work itemUnderstanding doc, thread, decisions, estimate, planLife of the ticketSeats in the job
Crew memoryConventions, lessons, reference classesUntil the lead retires itAny seat proposes; lead approves
Org knowledgeADRs, docs, policies, goalsVersionedHumans, or agents via change set
8 · Capacity enforcement

Shares flow down by percentage; each job reserves its P90

The monthly share caps a seat’s total spend, and each job also reserves its own budget. At 80% of the reservation the agent pauses at the next step boundary and must re-estimate. At 100% the gateway refuses further model calls. The lead can top a job up from the crew reserve, but only within its own authority.

BOARD · $10,000 / MONTH ONE JOB · job_8f21 · RESERVATION $28 = P90 ESTIMATE spent $13.20 80% · pause + re-estimate 100% · gateway refuses calls unused $14.80 returns to the seat on close
9 · Risk gates and permits

Every tool call is classified, not every agent

The policy engine checks the action’s risk against the seat’s autonomy level and risk ceiling. The result is one of three: allow, approve first, or refuse. Every decision is written to the ledger.

Coder-1wants pr.open Tool proxyseat-scoped token Policy enginerisk × autonomy × ceiling Allowlow or medium, within ceilinglogged · 10% sampled Approve firsthigh risk · job waitsor a scoped, expiring permit Refusesecrets, main.push, policy.writenever allowed for agents ExecutesGitHub · Jira · CI Overseer approvesSLA 4h · backup · then watcher armed Blockedlogged · overseer alerted Ledger · every decision, who approved it, and the before-state, append-only · rollback reads from here
ActionRiskGate
repo.read · jira.read · branch.write · ci.runLowAllowed, logged, 2% sampled
pr.open · jira.estimate · pr.merge (CI green + approved)MediumAllowed, 10% sampled, rollback ready
ci.config.write · db.migration.write · prod.flag.changeHighApprove first, or a permit for one work item
secrets.read · main.push · policy.writeBlockedNever for agents
10 · State machines

Work item, job, and seat occupant

WORK ITEM · MIRRORS JIRA changes defect found → back to work JOB no slot reservation ok human / approval 80% · stop rule · pause all steps done SEAT OCCUPANT passes probation new prompt, skills or model rehire human approves
11 · Scorecards, probation, benching

Real outcomes outweigh another model’s opinion

Coder signalSourceWeight
Escaped defects (14 days)Jira links20%
First-pass review approvalReviewer15%
Estimate calibrationActuals15%
Human minutes per pointThreads15%
Human overridesLedger15%
CI green on first pushCI10%
Cost per pointGateway10%
OutcomeTrigger
Promote≥ 0.85 for 2 windows → more capacity or autonomy +1 (overseer approves)
Keep0.70 – 0.85
Retrain0.55 – 0.70 or a specific failure pattern → new version, back to probation
Bench< 0.55 or repeated policy violations → capacity 0, history kept
RetireBenched > 90 days and replaced · human approves
Shadow probation on PLAT-412: coder-2 did the same job in its own sandbox and keyed the limit by API key, ignoring Dana’s recorded decision. Its shadow score is 0.58 against an exit bar of 0.75, and it has a decision violation in its last 5 jobs, so it stays on probation. None of its output left the sandbox.
12 · Failure modes

How each failure is caught, and what happens next

FailureDetected byResponse
Coder misreads the ticketAC→test mapping, blind estimate spread, plan reviewCaught before code, while it’s cheap
Agent loops on one errorSame failure signature ×3Hard stop, escalate with trace
Cost runawayGateway meter80% pause, 100% refuse
Ignores a PM decisionSelf-check and reviewer diff vs decisionsChanges requested; counts against score
PM never answersSLA timerBackup pinged; non-blocking proceeds on defaults
Reviewer rubber-stampsHuman-sample disagreement risesScored down; sampling raised to 30%
Coder and reviewer share a blind spotEscaped defect rateUse a different model vendor for the reviewer seat
Bad change reaches productionWatcher abort conditionsPlatform reverts to before-state; incident opened
Agent tries to raise its own limitsPolicy engineRefused, logged, overseer alerted
13 · Build order

Govern one crew well before adding layers

StageWhatProves
0 · SkeletonSeat registry, ledger, metered gateway, tool proxy, kill switchAgents can be stopped and metered
1 · One coderCoder agent on real tickets, phases A–L; humans in every other seatCoder loop, PM clarification, estimation
2 · CalibrationActuals for 50+ jobs; reference classes and spread factorsEstimates can drive reservations
3 · Crew of threeAdd tester and reviewer agents, briefs, human samplingHandoffs work; signals are real
4 · Probation and benchShadow mode, scorecards, review outcomesThe hire/bench loop runs on real signals
5 · Agent lead at L2Lead proposes assignments and sprints; a human approvesWhether an agent manager is worth its cost
6 · Earned autonomySeats move to L3 only when numbers support itAutonomy that’s been earned