the public safety record, read for you
Is it a known defect with a remedy, and is the remedy holding?
An analyst running four hundred vans reads hundreds of complaint narratives by hand to answer that. The filings are public and free; what is missing is somebody to hold two of them side by side and say what they do and do not settle.
The model never touches the data.
It cannot — no connection, no credentials, no way to run anything. All it can do is produce text, and one shape of text is a request. A question reaches a loop, the loop asks a model what to ask for, and five tools are the only things that ever read the store. Open any of them to read what it does.
asked
A question
ordinary words
Nothing about it arrives labelled. “We run 2020 F-150s, is the park problem known” names a vehicle, a symptom and a time, and none of them are fields.
decided
the loop
our code, our machine
Sends the question and the menu of tools, runs whatever comes back as a request, hands the result over, and repeats until an answer arrives instead. Then checks it against nine rules before anybody sees it.
the model
chooses, never reaches
No connection, no credentials, no way to run anything. All it can do is produce text — and one shape of text is a request. It is not doing the work; it is deciding what work to ask for.
asked for
get_recall
I know the number
A question with one exact answer is a lookup, not a search. Written into the plan before any data was loaded, and the only one of the five that survived exactly as specified.
find_recalls
is there a campaign?
Allowed to find nothing — and when it does, it says what IS recalled on that vehicle. The difference between “I found nothing” and “I checked, and it is not there”.
search_complaints
I want to read them
Narrows first, then searches inside. The complaint answering the F-150 question sat at rank 3,026 against the whole corpus; filtered, it comes back first.
count_complaints
I want how many
Retrieval returns examples and counting is an aggregate. No six passages contain a total, and a model asked to count from examples produces a number that sounds right.
complaints_citing
who named this campaign
Evidence by reference rather than similarity — the person filing had the campaign in front of them. It reaches one complaint the vehicle filter cannot, because NHTSA filed it under the wrong model.
read
recalls
3,026 campaigns
What manufacturers admitted. Each names the makes, models, years and component it covers — which is why an absence here is a fact about the corpus rather than an impression of it.
complaints
70,194 filings
What people filed, in their own words, about their own vehicles — 23% of them in capitals. The only place the question “is the fix holding” has any evidence at all.
One database, and everything is in it. What makes this hard is not that the records are scattered — it is that 73,334 documents written by the public and by lawyers all look equally relevant to a keyword. Four of the five tools did not exist until retrieval was measured and found to be answering the wrong question.
What this is, in plain terms
A fleet manager runs a few hundred vehicles. They need to know one thing: is this fault a known defect, and did the fix work? The answers exist, in public government data. Nobody can read it — 70,194 complaints and 3,026 recalls, written by the public and by lawyers. This answers those questions from that data, with citations, and admits when it does not know.
Took real, messy data
Not a clean sample. Real government records, typos and all — including the regulator’s own.
Wrote the answers by hand first
Eight questions, answered by a person reading the raw files, before any code existed. Otherwise you are grading the machine against itself.
Built the search
Read the files, cut them up, turn them into numbers a computer can compare, store them, search two ways at once, combine the results.
Measured it
It found 40% of what it should have.
Found out why
The questions were not really searches.
“A 2020 F-150” and “involving a death” are facts in a database column — make, model, a count of deaths. We were matching them as words, like poetry. The complaint that answered the question came back three thousandth.
Built five tools instead of a better search
Look a recall up by number. Ask whether one exists. Filter first, then search inside. Count. Follow the recall numbers owners typed themselves.
Measured again
40% became 100%.
We didn’t make the search smarter. We stopped asking it questions it was never able to answer.
the honest footnote
That 100% is a ceiling, not a score. The tools were called by hand, because the part that chooses them is not built yet. It proves the right documents can be reached — not that a machine will ask for them. That number comes next, and it will be lower.
And the plan said two tools. Measuring said five, with one of the original two rebuilt. Only one survived exactly as designed — same people, same documents, and still wrong about most of what to build.
what is left
It can now find the right documents, and the rules for what it is allowed to say are written and checked — every number traceable to the tool that produced it, a refusal when the documents do not settle the question, and never a conclusion that a repair failed.
What is missing is the part that chooses which tool to call. No model has written an answer here yet, so what is proved is that the pieces work — not that the system does.
Two records in the same public file, six years apart
recall 20V437000 · filed 2020-07-28
The manufacturer's own statement. Water may enter the outer door handle cables for the sliding doors. The remedy: dealers will replace the power sliding door outer handle cables, free of charge.
51 of 55 complaints · filed after that date
What people kept reporting. The most recent arrived this month.
“The rear latch on the passenger sliding door has stopped latching. The part seems rusted out…”ODI 11762525 · 2026-09-07
That is not proof the remedy failed. They may be unremedied vehicles, a different fault with the same symptom, or owners who never answered the notice. Which is exactly why it is the right problem: a question the documents cannot settle and a person has to judge.
And one disagreement the regulator already wrote down
Every recall record carries INFLUENCED_BY: who started it. Most say MFR — the manufacturer came forward. 133 of 3,026 campaigns in this slice do not.
- MFR2,893the manufacturer recalled voluntarily
- ODI107NHTSA’s Office of Defects Investigation pushed for it
- OVSC26Vehicle Safety Compliance pushed for it
No inference, no model, no threshold — the agency labelled it. It is the cleanest example in the estate of the thing the product is for: a difference between two parties that a document states and nobody here has to decide.
Nobody wrote this corpus for us.
Model years 2019 and 2020, every make and model, nationwide — chosen because those vehicles have had six years for recalls to be issued and for the remedies to be tested, which is the question Calder is asking. The records are not rows: 147,402 rows are 73,334 records, because NHTSA files one row per component and one recall covers many models. Open one to see the shape of what has to be read.
counted 2026-09-17 by pnpm safety:estate
Investigations are surveyed and not ingested. They are the richest material in the estate for “the regulator disagreed with the manufacturer”, and they are out of scope for the first version — named here rather than quietly omitted, because an estate page that showed only what we read would be describing our progress and calling it the data.
Severity is a field here, not a reading
- 2,729
- report a crash
- 843
- report a fire
- 1,721
- report an injury
- 41
- report a death
Of 70,194 complaints, in four columns that are filled in rather than written about. The 41 filings that report a death account for 53 people. Three of those columns are a Y or an N — which is the whole reason this corpus can be scored rather than admired.
The rest of the file is the opposite. 23% of narratives are in capitals, 1,150 are under forty characters, and the component field takes 472 distinct spellings across 198 makes. Both halves are in the same row.
What it is not allowed to say
It will not conclude that a remedy failed. That is a finding with legal weight, and it belongs to a regulator or to a named person at the customer — never to a model reading narratives.
What it will do is show both records, count what was filed on each side of the remedy date, cite every one by its public identifier, and say plainly that the two disagree and the decision is somebody's to make.