← all writing

I deleted the file. verify_symbol still said the function existed.

Leonard is the local-first tool I built to keep Claude Code honest against my actual code. The pre-edit hook intercepts every Edit and Write, parses the symbol references, and checks each one against a SQLite index of every function and type in the project. If Claude references NewLoginHandler and the index has no row for it, the hook blocks the edit before Claude gets to say "fixed it." The promise is the model can't fabricate API surfaces it wishes were there.

I red-teamed Leonard this weekend. The bug I found lets Claude verify a function that doesn't exist.

The flow is small. Claude says it's about to write a fix that calls LoginHandler.ServeHTTP. The pre-edit hook calls verify_symbol over MCP with the name. The MCP tool hits SQLite, finds a row, returns exists: true with the file path and line number. Claude proceeds. The fix passes go vet. The tests pass. The PR goes up.

But the file the row points to was deleted three days ago.

The audit that found this is in the same family as the audit-methodology post from a couple weeks back, and the Bosun bughunt-1 writeup I put out the same day. Same shape: I went back to a tool I'd already shipped and asked "what would a fresh red-team lens find that I missed." Twelve prior bughunt rounds on Leonard, two security reviews, every HIGH and CRITICAL closed. I expected LOWs and a few MEDIUMs. I got three HIGHs. F020 below is the one I keep coming back to because of how small the fix turns out to be against how long the bug has been shipping.

#The discriminating test

I built a sandbox project with about 1500 fixture files, indexed it, then deleted two files on purpose to watch the prune sweep work. Same operation against both:

rm fixtures/a_prune_test.go
rm fixtures/l3_scratch/will_be_deleted.go
leonard index

Same DB. Same command. Different outcomes:

verify_symbol returned exists: false for the first one (correct, file is gone). For the second one it returned exists: true with a precise file path and line number for a file that did not exist on disk anymore. I checked twice. I ran ls fixtures/l3_scratch/will_be_deleted.go and got "No such file or directory." Then I called verify_symbol and got back exists: true, file: fixtures/l3_scratch/will_be_deleted.go, line: 2.

The only thing different between the two files was alphabetical sort position.

#The root cause

The indexer runs pruneStaleFiles at the end of every leonard index. The sweep enumerates every row in the files table, stats each path on disk, and DELETEs any row whose path doesn't resolve. The enumeration uses Store.ListFiles("", ""). The contract is "list files matching the pattern and language filters; both empty means all files." Reasonable name, reasonable shape.

A previous round, security review #4, hardened ListFiles against a monorepo OOM. The fix added a SQL LIMIT MaxSymbolQueryRows at the boundary, capping at 1000 rows. The cap was correct for its caller, the MCP list_files tool, which is paginated by intent. 1000 is a reasonable default page size for a function whose job is to feed an LLM context window.

The prune sweep also calls ListFiles("", "") because it predates the hardening and the function name says "all files." The prune sweep is asking "give me everything, I'm going to stat each one." The cap silently quits at row 1000 alphabetically. Files past row 1000 are never seen by the stat loop. Their rows live in the index forever, regardless of whether the file still exists on disk.

Same function, two callers, only one of them is paginated. The cap is wrong for the unpaginated caller and right for the paginated one. The function name carries no warning. The call site carries no warning. The caller asking "all files" gets quietly fewer, and the result is a silent index-poison.

#Why eleven rounds missed it

The TestIndexAll_PrunesDeletedFiles regression test exists. It was added in bughunt-2 to close the original "prune doesn't run" finding. The test creates a fixture corpus of about 30 files, deletes one, runs the indexer, and asserts the row got removed. It passes. It has been passing every CI run since.

Every file in that fixture is at sort position under 30. The cap fires at row 1000. The cap never fires inside the test. The test was perfectly correct for the bug it was written to prevent, and perfectly blind to the bug the hardening introduced afterward. Two reviewers, including me, signed off on the hardening. We checked that the MCP tool was now safe. We did not check that the function's other callers were still getting what they thought they were getting.

What makes the bug worse to think about is that the CLAUDE.md project-setup section explicitly says Leonard handles "250 to 2000 files typical." The typical-project range straddles the cap. Every typical-sized project has been silently degrading since sec-4 shipped six minor versions ago. The symptoms only show up for files alphabetically late in the tree, which is exactly where dead code tends to accumulate: vendor/, web/static/, zz_legacy.go. The dead code keeps verifying as alive.

#The fix is at one file

internal/index/indexer.go:394:

files, err := i.Store.ListFiles("", "")

becomes a new Store.IterFiles cursor that has no LIMIT (it's an internal call, not an MCP-exposed one). The MCP list_files tool keeps its current cap. The prune sweep gets every row. About a dozen lines including the new store method, the call-site change, and the import.

The fix also closes a second finding, F003 in the rollup, at the same call site. leonard index prints leonard: indexed 1000 file(s) at the end of every run. That count comes from the same capped ListFiles call (wire_real.go:65). The operator running leonard index on a real project always sees indexed 1000, regardless of the actual count, because the count function is the capped function. Two different bugs, same site, one fix.

The regression test for F020 is fifty lines: a fixture generator that produces 1500 dummy files, an assert that the alphabetically-last file gets pruned after deletion. Future-me adding any new internal user of ListFiles runs into the same test and remembers the cap.

#What I'm chewing on

Two things I think this generalizes to.

First, when you write a regression test for a capacity bug, the fixture should be at least 50% past the cap. A 30-file test for a function that caps at 1000 is asking the cap to never fire during the test. A cap that never fires during testing can be silently wrong somewhere else and you'll never know. Whichever direction your cap protects, force the test to cross it.

Second, when you cap a function, grep every caller and tag each one as either "paginated consumer" or "give-me-all consumer." The paginated consumers are safe. The give-me-all consumers are bugs in waiting. Either split the function name (ListFilesPage, ListAllFiles), or split the function itself, but don't leave the same name doing both jobs with a cap silently changing the answer.

Bughunt-12 found 46 things on Leonard. Three were HIGH. The other two HIGHs were different shapes. One was a malformed entry in any ground-truth file (a single bad heading in stories.md) silently taking down the whole adapter at server start. The other was Claude bypassing the content filter by writing files with cat > file shell redirects, because the pre-edit hook's Bash matcher inspects the command name but not the command content. Both worth their own posts. F020 is the one I keep coming back to because the fix is so small compared to how long it's been bleeding.

The next round of Leonard bughunt starts by asking "what other capped functions in this codebase have unpaginated callers." I have a hunch about a few. I'd rather find them now than ship two more versions and let one of them bite a real project.

Audit your own caps. Honestly, a cap that's never been adversarially asked "and you're sure every caller of you wants this?" is a cap that has not been audited.

Bughunt-12's full audit lives in leonard/audits/: brief, per-lane writeups, consolidated rollup. F020 plus F003 plus the other 44 findings are on the audit/bughunt-12 branch. The fix is one commit.

What caps in your own code have you never deliberately asked to fail? When was the last time a regression test ran the fixture past the boundary the test was written for?

← all writing