You kick off a bulk duplicate detection job against your Account or Contact table. You want a real number: how many duplicates are actually sitting in this environment. The job runs, finishes, and hands you a result set.
It stops at 5,000.
If your table has more duplicates than that, the job doesn't tell you it gave up early. It just returns what it found up to the ceiling, and the result looks complete unless you already know to be suspicious of it.
Where the limit comes from
This isn't a bug or a misconfiguration. It's documented behavior. Every duplicate detection job, whether you launch it from Settings > Data Management > Duplicate Detection Jobs or trigger one through the API, runs on the same underlying mechanism (BulkDetectDuplicates), and Microsoft's own documentation states plainly that a duplicate detection job returns a maximum of 5,000 duplicate records.
It's also worth knowing that Dynamics 365 CE only ships default duplicate detection rules for accounts, contacts, and leads. Any other table needs a rule built and published before a bulk job will find anything on it at all.
Put those two facts together and the pattern is predictable: the tables most likely to have accumulated years of duplicates through imports, integrations, and manual entry (accounts and contacts) are also the ones where a single cleanup job is most likely to hit the 5,000 mark before it's covered the full dataset.
The workaround: batching the job yourself
The Duplicate Detection Job wizard lets you scope a run with a saved view or a Select filter instead of running it against the whole table. That's the lever most admins reach for once they realize the ceiling exists:
- Split the table into ranges, by creation date, record owner, or an alphabetical slice, small enough that each batch stays under 5,000 results.
- Run the job against one range at a time, exporting or recording the results before moving to the next.
- Track which ranges have already been scanned, and watch the edges of each range carefully so records don't fall through the gap between two batches or get counted twice.
It works. It's also entirely manual, and the bookkeeping is on you: a spreadsheet of which slice covered which date range, run on whatever cadence someone remembers to kick it off again.
Why this doesn't actually close the problem
A batched bulk job gives you a number for the day you ran it. It says nothing about the records created an hour later.
Every channel that doesn't go through the native duplicate detection dialog, Web API inserts, Power Automate flows, Dual-Write syncs, is still writing new records the whole time you're working through your batches. (We've written separately about how Web API inserts bypass duplicate detection by default, which is worth a read if integrations are a big part of how records land in your environment.) A bulk job, however carefully batched, is a snapshot. It tells you where things stood, not where they stand.
That's fine as an occasional audit. It's not a process you'd want to run your data quality on. The batching overhead scales with data volume, not down from it, so the bigger the environment gets, the more manual tracking the workaround demands, right as it becomes least practical to keep up with by hand.
What actually closes the gap
A batched bulk job only ever solves half of this. It gets you caught up once (Clean), but it does nothing about the records created the moment after the job finishes (Prevent). Run one without the other and you're not closing the backlog, you're re-opening the same batching exercise on a recurring schedule.
Plauti for Dynamics 365 CE runs both inside your own Dataverse environment, installed from Microsoft Marketplace. Bulk detection has no 5,000-record ceiling and covers standard and custom entities, not just the three Microsoft ships default rules for, and the same rules check records arriving through the API, so integrations stop adding to the pile while you're clearing it. If you're mid-cleanup and want a number that doesn't depend on how many batches you remembered to run: .