LLM import that created 790 duplicate records

An LLM that writes into an existing database uses the document's names, not yours, and "not found" quietly turns into "new". That is how one import created 790 duplicate cars. The fix was not a smarter model. It was teaching the pipeline the database's vocabulary, and repairing the data from decisions written down in advance.

What happened

At Arc AI we build and run a B2B vehicle data and valuation platform for a client. At its heart is a catalogue of cars: make, model, version (such as "2.0 TDI Style"), engine, power and list price. Since April 2026 an LLM pipeline has kept it up to date. Manufacturers publish their prices as PDF brochures. The pipeline reads each one, pulls out every version and looks for it in the catalogue. Found: add the new price. Not found: create a new car.

The catalogue was typed by people over many years and has its own spelling. BMW's estate is 3er Touring; Toyota's SUV is RAV 4, with a space. The brochures say 3 Series Touring and RAV4.

Four steps, each doing its job, produced the failure:

1. The extractor copied the model name from the brochure, exactly as it should.
2. The lookup found no catalogue model with that name.
3. The matcher, which picks the right car from the candidates the lookup returns, got an empty list and correctly answered "new".
4. The importer created the model and saved the name as a known answer. From then on, every import "knew" that 3 SERIES TOURING was a model.

In August 2026 one import created 1,361 cars, and 790 of them already existed. The new prices went to the copies. A later check, which compares one price list with the next, saw the originals missing and marked them as discontinued. The fixed value lists picked up junk too, such as a gearbox value that literally read :null.

Why LLM imports fail this way

Matching records that describe the same real thing is called entity resolution, and names are its weakest signal. An LLM extractor makes this harder, not easier, because it is faithful to the source. The better it reads the brochure, the more exactly it copies the brochure's vocabulary instead of yours. The risk applies to any AI integration in which a model reads documents and puts their data into a system the business already runs.

We had seen it before: in May 2026 a brochure said VOLKSWAGEN where the catalogue says VW, and the pipeline duplicated an existing ID.5. We fixed make names then, but not model names.

What we learned

"Not found" is not "new". An empty search means either the car is new or the names don't match. The pipeline always assumed the first.

A pipeline must not learn from itself. A cache of answers is useful until it stores the system's own guesses next to verified answers, with nothing to tell them apart. Then one wrong name becomes permanent.

Knowledge belongs in data, not prompts. Our prompts serve dozens of makes. A rule tuned for one brand quietly breaks another, so brand knowledge has to live somewhere it can be reviewed and tested.

In a repair, the risks aren't equal. A missed duplicate costs a tidy-up later. A wrong merge deletes a real car and rewrites its price history. When the evidence is unclear, keep both.

How we fixed the pipeline

A naming profile per make. Each of the 54 makes now has a small JSON file: how the catalogue names models, a map from brochure names to catalogue names, the order of words in a version name, real examples, and a list of open questions. We wrote BMW's by hand. LLM agents drafted the other 53 from evidence pulled out of the catalogue, and an alias was only allowed with proof. Doubts went into the open-questions list, which is never shown to a model. A plain script with no AI in it then checked every alias against real models. (Our first attempt ran fifteen agents at once, and all of them ran out of session halfway. We now run three at a time.)

The profile is not pasted into prompts whole. Each reader gets a short version within a budget: about 6,500 characters for the extractor and 2,500 for the matcher, whose prompt already carries the candidate cars.

A lookup order that trusts evidence first. Code resolves names before any LLM is asked:

def resolve_model(make, raw_name):
    for step in (reviewed_mapping, # human-reviewed only, never importer-created
                 profile_alias, # the make's naming profile
                 exact_match, # case- and accent-insensitive
                 small_llm): # last resort, profile as context
        model = step(make, raw_name)
        if model:
            return model
    return None # still unknown: search the whole make by specs before calling it "new"

Cheap, deterministic steps run first, so most answers are explainable and cost nothing.

Fixed value lists. Fuel, gearbox and body type always map to an existing value, and the importer records which default it used. A new body type is now a person's decision.

An early warning. A report per make lists every new model next to any existing car with the same specs. A spike in "new" is a naming problem until proven otherwise.

On a Fiat Doblo brochure that had created two fake models in August, a test run now matches 9 of 9 versions to existing cars.

How we repaired the data

A script that finds duplicates and merges them on the fly is tempting. But a merge deletes a row, and a guess made inside a script leaves no record of why. So this needed careful repair, not a clever script: decide first, then execute.

Every decision was written into reviewed files, make by make: this new car is that old one, these versions move under the old model, and these 43 model lines are genuinely new and stay. The tool then executed the files literally. It previewed by default and refused to run without a confirmed backup. Inside each transaction it rebuilt the plan with the rows locked and stopped if anything differed from the preview, so a change made between preview and run could not slip through.

A rehearsal on a fresh copy of production stopped three times, and each stop taught us something. One entry would have deleted a "surplus" row that turned out to be a different trim. A naming step couldn't be checked before the merges it depended on had run. And two writes landing in the same second tripped a safety check, because timestamps are stored to the second.

Two rules came out of the reviews. A different trim name means a different car, and this rule alone withdrew 72 matches we had already accepted. And identical prices prove nothing: Hyundai prices the IONIQ 5 and IONIQ 6 the same, trim for trim, and one draft nearly deleted real IONIQ 6 entries on that basis.

The repair took Team ArcAI quite some time. Afterwards we cleaned up the project so no stale and depricated tooling remained. What stays are the profiles, the fixed lists, the report and the rules.

A checklist for your own LLM imports

• Give the pipeline your database's vocabulary, as data rather than prompt text.
• Keep system-created answers apart from reviewed ones.
• Resolve names with cheap, deterministic steps first and the LLM last.
• Track how many "new" records each import creates.
• Repair from reviewed decisions: preview, back up, lock and re-check, rehearse.
• When unsure, keep both records.

Clean names pay off beyond one repair

The same names now drive our match cache that learns from every import. If an LLM pipeline writes into data you already own and duplicates are piling up, talk to Team ArcAI.