The complaint that started it
It began as a budget gripe, not a security project.
“I am still running into monthly spend limits with Claude when trying to use git-steer to scan CVEs across all of my repos. This is a bottleneck that should not exist. git-steer should be using its own resources and GitHub workflows and its own app to do this. What is the issue here?!”
git-steer is my self-hosted fleet security tool — a GitHub App plus a pile of Actions workflows that’s supposed to scan ~43 repos for vulnerabilities, open fix PRs, verify them, and merge the safe ones, all on its own compute. The whole philosophy is zero footprint, zero humans in the happy path. So why was checking the status of my own fleet burning through a Claude plan?
The answer turned out to be the first of many layers, and unpeeling them took a full day. What I want to write down isn’t just the bugs — it’s the part that surprised me. The AI was tireless, precise, and relentless at the investigation. But at every fork that actually mattered, the human got there first. The forecasting, the architectural framing, the “no, it’s bigger than that” — that was me, out ahead of the machine. This is a story about that division of labor.
Layer 1: the model was the loop
The first finding was almost embarrassing in hindsight. Nothing in git-steer’s design required Claude at all — the heartbeat workflow, the escalation ladder, the fix workers all run on GitHub Actions with zero AI involvement. The expensive path was that I was driving CVE scans from a chat session, which turned the model into a per-repo control loop: scan a repo, reason about it, call the next tool, repeat, 43 times. The autonomous pipeline existed. I just wasn’t using it; I was hand-cranking the thing it was built to do automatically.
So the agent’s first proposal was sensible: stop scanning from chat, let the heartbeat do it, and give me a cheap way to ask for status — a single status.json published to GitHub Pages that I could read for pennies instead of re-scanning the fleet.
We built it. And that’s where the floor fell out.
Layer 2: the dashboard was lying
The freshly-published status.json cheerfully reported a 100% clean fleet — zero open alerts, fix rate 1.0 — while simultaneously listing four repos as hard-stopped escalations. Those two facts cannot both be true. A spotless fleet doesn’t have hard-stops.
The agent found the contradiction’s root quickly: the dashboard generator parsed my managed-repos.yaml with a naive regex that collapsed all 43 repos into a single malformed name, so the scan queried garbage and “found” nothing. The public dashboard had been showing a falsely-clean fleet for who knows how long.
Here’s the first place the human pulled ahead. The agent fixed the parser and was ready to move on. I wasn’t. Reading the report, something clicked that the step-by-step analysis hadn’t surfaced yet:
“So first and foremost, this is not just an issue with one repo — it’s all of my repos and orgs. The issue is with how git-steer manages, operates, and then automates these PR/CVE workflows.”
That reframe mattered. The AI was solving a bug. I was looking at a class. Once we corrected the parser and re-ran, the truth came out: 234 open alerts across 7 repos, eight of them critical, a real fix rate of 10.7%. The fleet wasn’t clean. It had been quietly rotting behind a green dashboard.
Layer 3: the whole control loop had stalled
With observability restored, the agent did what it’s genuinely great at — it mapped the entire autonomous architecture and surfaced a stack of compounding defects:
- Three different remediation paths that had drifted apart, one of which merged fixes with no verification gate at all.
- An npm “fix” step that ran
rm -f package-lock.json && npm audit fix— which failed silently and committed the lockfile deletion, stripping the lockfile and fixing nothing. - A verification gate that only looked at the repository root for a manifest, so every monorepo (frontend + backend, packages/*, etc.) verified nothing and got stuck forever in an unverifiable limbo.
- An escalation counter that had climbed to 9 against a threshold of 3 — silently, invisibly, with no one watching.
This is the second place I overtook the machine. As I read the agent’s gap analysis, I typed:
“So as I was reading through your report, I was thinking that this is a control plane issue. And then I read further where you confirmed it.”
I’d named it before the confirmation landed. The agent had ten discrete gaps; I had one word — control plane — that explained why each gap was inevitable: nothing owned the whole loop, so every stage could fail in a way the next stage couldn’t see. A blind observer, a destructive fixer, a verifier that couldn’t verify, an escalator that escalated into the void. The human pattern-matched the shape; the AI then filled in the rigor.
The Snyk detour: trust, but verify the verifier
Midway through, I pasted in a Snyk dashboard — “47 recently vulnerable projects” — expecting it to be the meat. The human forecast here was a skepticism the AI didn’t start with: I’d removed Snyk from this fleet months ago, and I didn’t trust what it was telling me.
Good instinct. The agent checked the four repos I pasted against OSV.dev and GitHub’s Advisory Database — the actual authoritative sources — and found that two of them had zero real vulnerabilities. Snyk was ~80% noise. Worse, the one “gap” it looked like we’d found (a FastAPI advisory Dependabot missed) turned out to be a superseded re-attribution — the vulnerability had migrated to python-multipart, which the repo had already patched. Dependabot was right to stay quiet. OSV was over-reporting. Snyk was just wrong.
The lesson the human carried in and the AI confirmed: don’t remediate off the scanner that shouts loudest. Reconcile against the source of truth first. If I’d let the AI just start “fixing” the Snyk list, we’d have burned effort on phantoms.
Bringing it together: one control plane
Once we both saw it as a control-plane problem, the fix wrote itself into an architecture decision record — ADR-007 — and a set of moves:
- One gated remediation worker (
fix → verify → disposition), replacing three drifting ones. The ungated path and a redundant CVE-triage path were retired — my call: prune hard, don’t maintain parallel mechanisms. - A monorepo-aware gate that discovers every package in the tree and builds each one, so verification actually runs on the repos the fleet is made of.
- Non-destructive dependency fixes — and a guard that physically refuses to commit a lockfile deletion.
- Verdict provenance written to an append-only ledger, so every merge-or-hold decision is auditable.
- Event-driven + reconciliation triggers, so a new critical acts in minutes instead of waiting for the daily tick.
The AI did the heavy lifting here — hundreds of lines of workflow YAML, a testable gate runner, the retirements, the security hardening, even disposing of CodeQL false-positives on its own tooling per my existing doctrine. It was fast and it was correct. At one point I just said: “100% approval to proceed, you don’t need to nag me to finish.” And it didn’t. It executed.
But notice what I was doing while it executed: forecasting. “Retire the sweep worker and the fabric path.” “It’s a fleet problem.” “This is a control plane issue.” Each of those was a steering input that changed the shape of the work, not the work itself. The agent optimized within the frame. I chose the frame.
The truth test
The last move was the one I cared about most: prove it works on real repos, not in theory.
We dispatched the rebuilt worker at DriveIQ — a polyglot monorepo that had been permanently stuck — and watched. The gate discovered three packages and built each independently:
backend(Python): PASSfrontend(npm): PASSdocker-extension/ui(npm): FAIL
Rollup verdict: NO-GO, held for human review. The fix was real and non-destructive — actual version bumps in the lockfiles, no deletions — but one package didn’t build with the bump, so the system refused to merge it broken and pinpointed exactly which package and why.
That’s the whole thing working. The old root-only gate could never have told me which package breaks. The old worker would have deleted the lockfile and called it a fix. Now: real fixes, verified per-package, with the un-mergeable residual surfaced to a human instead of silently merged or silently dropped.
Who actually did what
If you want a clean takeaway, here it is.
The AI was a phenomenal investigator and an indefatigable implementer. It read the whole codebase faster than I could, traced every dataflow, wrote the workflows, ran the tests, dismissed the false-positives, and proved the result on live infrastructure. I would not want to do any of that by hand.
But the AI moved forward — gap to gap, fix to fix. The human moved up. Every inflection point in this story was a human forecast that outran the machine’s analysis: that a single bug was actually systemic; that “systemic” meant control plane; that the loudest scanner was lying; that three mechanisms should become one. The model confirmed each of those — rigorously, with evidence — but it confirmed them after I’d called them.
That’s not a knock on the AI. It’s the shape of the collaboration that worked. The machine is the better analyst. The human, for now, is still the better forecaster — the one who feels the shape of the problem a few moves before the evidence arrives. Point the model at the right frame and it will out-execute you all day. Choosing the frame is still the job.
The fleet drains itself now. The dashboard tells the truth. And I’m not spending a Claude budget to ask my own tools how they’re doing.
— Built with git-steer and Claude Code. The control plane is ADR-007; the truth test was real.
