Scrubbed, coded or anonymous? Decoding the data cleaning vocabulary

Direct identifiers reveal a person immediately: name, email, phone or account number. Buyers say these are removed.

Indirect identifiers are less obvious: a date of birth, a ZIP code, a job title. Each one alone points nowhere, but a handful together often does. Research from 2019 showed 15 such attributes were enough to identify 99.98% of Americans correctly.

In a spreadsheet you can drop the Name column. Written messages have no such column. NIST points out that in free-form notes, names and addresses are not labelled, harmless details can look personal, and genuine identifiers can get through.

Look at your own mail: email signatures, nicknames, a line like "the Garcia kitchen remodel on Oak Lane", a scanned driver's licence in an attachment. A scrubbing tool must catch every one.

Scrubbing involves a balance. The harder you clean, the safer people are and the less useful the data becomes. Too little cleaning endangers people; too much leaves the buyer with less worth paying for.

• Polyshares: hires an outside company to scrub personal data before licensing.
• Cognor Labs: removes or substitutes details while the data is still inside the seller's systems.
• Mercor: describes its scrubbing as automated.
• Nyne: its ads say client names are converted to codes.
• Spirit Airlines sale: Tonic.ai de-identifies the files and certifies the result to the court ahead of each transfer, the toughest standard seen so far.

1. 1Is the method deletion, or replacement with codes?
2. 2Who carries it out: the buyer, a contractor or our own staff?
3. 3Will we review a scrubbed sample before release?

Then reduce the scrubber's workload: exclude the folders richest in personal information, such as HR files and customer account records, before anything is shared.