This article will explain how to setup, test and run a bulk deduplication on one or more lists in your account. If you are looking for a general introduction to duplicates, check out our article on duplicates. If you want to know exactly what happens when leads are merged, check out our article on merging.

If you have many duplicates, you should start by checking your configuration. First, you need to make sure the duplicate detection is legit, meaning that the dupes we detect are actually dupes. Go to Manage -> Duplicates and create a duplicate report. Review the leads and check if it looks sane.

Example duplicate report

If the system detect duplicates that are not duplicates, you may want to adjust how we do the detection in the first place. This is done from the list configuration page under "Dedupe".

The "Dedupe" settings page under list configuration

Once you have confirmed that all your lists has the right settings, you can move on to running a bulk deduplication job. If you change settings on a list, please allow the system some time to reset and redo the detection routines.

To start a bulk deduplication job, go to Manage -> Duplicates and review the Dedupe section on the right hand side.

The deduplication won't touch any leads that are not in state "New". This is the safest operation as new leads usually does not contain any activity that can be lost.

Value won leads over the others, meaning that won leads should always be kept intact during the deduplication. Likewise, value call backs higher than all others but won leads, and so on.

Example: If two dupes are "New" and "Won", the "Won" lead will be kept and the "New" lead will be deduped using the selected dedupe method.

Example: If two dupes are "Call back" and "New", the "Call back" lead will be kept and the "New" lead will be deduped using the selected dedupe method.

Value leads currently queued over completed leads.

Example: If two dupes are "New" and "Won", the "New" lead will be kept and the "Won" lead will be deduped using the selected dedupe method.

Example: If two dupes are "Call back" and "Lost", the "Call back" lead will be kept and the "Lost" lead will be deduped using the selected dedupe method.

Only acts on duplicates where one of the two leads sits on a blocklist. The lead on the blocklist is always the one kept; the lead on your ordinary list is the one deduped using the selected method. Duplicates that don't involve a blocklist are left alone entirely, and the blocklists themselves are never deduped.

Use this to scrub a newly imported list against people who have asked not to be contacted.

The same as above, but it only touches leads still in state "New". A lead you have already worked — won, lost, called back — is left alone even if it matches a blocklist entry.

Use this when you want to keep your existing call history intact and only scrub the leads you haven't reached yet.

Run the deduplication and collect number on how many leads would be affected by the chosen strategy on the chosen lists. A dry run does not change anything and is safe to run just to see what will happen if a real deduplication was run.

Merges "Merging" into "Keeping". The "Keeping" lead will not have any data overwritten.

The "Merging" lead will be deleted along with its activity log. WARNING - if existing activity is deleted, this also affects reports. For example leads that were won, that is deleted, will have the winning event deleted, and that win will no longer appear in the reports.

myphoner will not report these leads as duplicates anymore.

The duplicate is archived rather than deleted. Nothing is lost: the lead keeps all of its data and its activity log, it just moves out of the queue so nobody calls it. Each archived lead is marked with the category "Deduplicated", or "Blocklisted" when you used one of the blocklist strategies, and gets a note in its activity log recording who ran the deduplication — so you can always tell why a lead was archived.

This is the safest of the destructive methods — choose it over Delete when you would rather not lose the history, or when you are not yet certain the duplicates are genuine.

myphoner will not automatically merge a lead that has more than 20 duplicates. A cluster that large almost always means the detection is matching on a value that many leads share — often a placeholder, or a contact detail that belongs to the data source rather than to each individual company.

When the run meets one of these, it skips that cluster and carries on with the rest. The run still completes, and the summary lists what was skipped so you can look at those leads yourself.

If you see skipped clusters, check the "Dedupe" settings on the list named in the summary. Matching on email and phone is switched on by default, so a list where every lead carries the same shared email address will report all of those leads as duplicates of each other. Turning off the matching you don't need, and leaving only a genuinely unique field such as a company registration number, will usually clear it. Changing those settings re-runs the detection for that list, which takes a while on a large list.