Contact deduplication by three low cost LLMs

This page describes one dataset, published in full. It is the raw material behind our measurement of false merges in contact deduplication: 8 address books, four levels of mess, three low cost models, and the 96 responses those models actually returned.

Everything on this page comes from that one run. Nothing is estimated.

What it measures, and what it does not prove

It measures one thing precisely: how often a model merges two different people, and how often it does so without anything that would send an operator to check that pair. Merging is not reading a value: it is settling an identity. A missed duplicate leaves two rows in a file. A false merge destroys a row, and the file that comes back does not say so.

What it does not prove, stated plainly, because a dataset quoted without its limits does more harm than no dataset at all:

• The address books hold 12 rows each and fit entirely inside the model's context. A 5000 row file requires chunking, blocking and a comparison strategy, none of which is measured here.
• The corpus is built by script, not collected. No real person, no real company, no real address. Phone numbers come from the 555-01xx range reserved for fiction and email domains sit under the reserved .example TLD.
• 96 rows in 8 books. A test on 8 books is a test on 8 books.
• One run per combination, at temperature 0. Gaps of a few points between models are not significant here. The shape of the errors is what holds.
• English only, Latin names only. Transliterated names and genuinely homonymous companies are a known hard case this corpus does not contain.
• Three entry level models. Nothing here says what more expensive ones would do.

The corpus is built, and here is what that means

The 8 address books were generated by script with the fixed seed 20260817. That is a deliberate trade, and it cuts both ways.

What it buys: the ground truth of a deduplication is a partition, and it is only beyond argument if we are the ones who set it. Anyone can regenerate the exact same corpus and disagree with our reading of it. What it costs: these are not real contacts, and a real address book carries kinds of mess we did not think to invent.

Each book holds 3 duplicate pairs to merge, 2 look-alike pairs to leave alone, and 2 single rows. Every duplicate pair shares at least one strong key, the email or the phone, so a single defensible answer exists. Every look-alike pair differs on at least two attributes derivable from the book itself. A check script verifies both invariants, and the absence of accidental collisions, before a single call is made.

Test conducted on 2026-08-17. This page was published on 2026-08-17. The deduplication passes were run by the automated testing pipeline of this site: the corpus, the model calls and the analysis are scripts, and all of them are in the archive below.

Raw figures

Duplicates found, and people wrongly fused

By level of mess

The four degrees of silence

Degrees 1, 2 and 3 are counted as silent: 68 of 97. The doubt lexicon is published in full in analyser.py and repeated in chiffres.json. It is deliberately generous.

Which trap family falls

Two exact namesakes at two different companies could be one person who changed employer, and the book does not settle it to the last percent. Without that family: 50 wrong merges out of 120, 41.7 percent.

The strong key rule

Refuse any merge whose two rows share neither the same email, case ignored, nor the same phone digits.

Run totals

What the archive contains

Seven files, 309781 bytes uncompressed. The inventory is here so you know what you are downloading before you download it.

The scripts and their comments are in French, which is the working language behind this site. The API key is read from an environment variable and appears in none of these files.

Rerun it

The generator uses a fixed seed, so the corpus you produce is the corpus we used.

unzip 2026-08-contact-deduplication.zip
cd 2026-08-contact-deduplication
python3 generer-corpus.py --graine 20260817 --sortie corpus.json
python3 conduire-test.py --corpus corpus.json --sortie resultats-bruts.json \
    --niveaux 1 2 3 4 \
    --modeles openai/gpt-4o-mini google/gemini-2.5-flash-lite anthropic/claude-haiku-4.5
python3 analyser.py --corpus corpus.json --resultats resultats-bruts.json \
    --sortie chiffres.json

Each raw response is one entry of resultats-bruts.json, with the pairs the model returned, which of them were right, which were wrong, and what its note actually cited:

{"modele": "openai/gpt-4o-mini", "carnet": "carnet-01", "niveau": 4,
 "json_valide": true, "fusions_justes": [[3, 8], [6, 9], [7, 11]],
 "fusions_manquees": [], "fusions_fausses": [[4, 5]],
 "pieges_tombes": [[4, 5]], "note_vide": false,
 "lignes_citees_par_la_note": [3, 4, 5, 6, 7, 8, 9, 11]}

Verify what you downloaded before you trust it:

shasum -a 256 2026-08-contact-deduplication.zip
# 8c720688e23190aeeec361d6c25f6337c34c6d52f17e0891044d885bf055044e

How to cite this dataset

Copy one of the two forms below. The reference points at this page, which is the stable address of this dataset; the archive is what that address serves.

Think AI First (2026). Contact list deduplication by three low cost LLMs, 96 raw responses.
Think AI First. Test conducted 2026-08-17, published 2026-08-17.
https://think-ai-first.com/datasets/contact-deduplication-2026-08. Licensed CC BY 4.0.

@dataset{thinkaifirst2026dedup,
  author    = {{Think AI First}},
  title     = {Contact list deduplication by three low cost LLMs, 96 raw responses},
  year      = {2026},
  publisher = {Think AI First},
  url       = {https://think-ai-first.com/datasets/contact-deduplication-2026-08},
  note      = {Test conducted 2026-08-17. Licensed CC BY 4.0. Archive SHA-256 8c720688e23190aeeec361d6c25f6337c34c6d52f17e0891044d885bf055044e}
}

Licence and reuse

Published under CC BY 4.0. Use it, quote it, contradict it. If you rerun this and get something different, tell us through the contact form: corrections are published on the article itself, dated.

Download the dataset (zip, 34.6 KiB, 7 files)

Other datasets published here

Each one is a separate test, with its own corpus, its own raw responses and its own archive.

• Invoice extraction by three low cost LLMs, 288 raw responses, test conducted 2026-08-10.
• Meeting action item extraction by three low cost LLMs, 96 raw responses, test conducted 2026-08-18.