De-identification, explained: what gets removed, what gets replaced, what stays

The value in operational records is the sequence: a request came in, someone asked a question, a decision got made, something happened next. None of that depends on knowing who the customer was. De-identification is the process of keeping the sequence and removing the people, the companies, and the details that would point back to them.

It is done before training and, in a properly structured deal, it is a contractual obligation of the buyer with a standard, a verifier, and a remedy. You approve the rules, you review a small sample built under those rules, and nothing is licensed until you are satisfied with what you see.

Every field in a record gets one of three treatments.

• Remove. The value is deleted or replaced with a placeholder such as [removed]. Used for anything that is identifying and carries no workflow value: card numbers, bank details, government ID numbers, street addresses, dates of birth, attachments with personal data.
• Replace. The value is swapped for a consistent fictional one. Jordan Lee becomes Person A everywhere Jordan Lee appears. Harbor Dental becomes Company 01 in every ticket, email, and deal. Used for names, email addresses, phone numbers, account identifiers, and company names, where the structure of the record matters and the identity does not.
• Keep. The value stays. Used for the content of the work: the problem described, the question asked, the fix applied, the status, the timestamps (often coarsened to a week or quarter), the amounts where they are not identifying.

Removal is simpler and safer; replacement preserves more value. A good set of rules uses removal for the dangerous fields and replacement for the structural ones.

A fictional support ticket, before and after. Replacements are made up.

Notice what survived: the problem, the diagnosis, the fix, the time to resolve, the satisfaction score. That is the whole training value of the ticket. Notice what did not: anything that would let a reader find Angela or Summit Payroll.

The difference between amateur and professional de-identification is consistency. If Requester A is a different fictional person in every ticket, a buyer loses the fact that the same customer raised five related tickets over two years, which is part of what they are paying for. If Requester A is the same fictional person everywhere, the structure survives.

Consistency has to hold across systems as well as within them. The customer in the CRM, the requester in the helpdesk, and the counterparty in the email thread are the same company, and they should map to the same fictional identity in all three. That is why de-identification is done against the whole licensed set with a single mapping table, and why the mapping table itself is treated as the most sensitive artifact in the deal. It stays with the seller or a named verifier and never goes to the buyer.

Free text is where consistency breaks. Names appear inside sentences, in signatures, in forwarded messages, in nicknames. Modern tooling handles most of it; the sample review is where you catch the rest. Expect to iterate on the rules once or twice before the sample is clean.

Three honest limits.

• It cannot be perfect. No process removes re-identification risk entirely, and anyone who says 100 percent is overselling. The standard to ask for is a documented one, applied by a named party, verified by sampling, with a remedy if it is missed.
• It cannot undo training. Deleting the supplied copy at the end of the term does not remove what a model learned from it. That is why field-of-use terms matter as much as deletion terms.
• It cannot fix a rights problem. De-identifying a record you were not permitted to license does not make it licensable. Privileged, regulated, and client-owned classes are excluded before de-identification starts. The rights checklist covers that.

Three arrangements are common. The buyer de-identifies on receipt under a contractual standard; this is the most frequent model in published buyer programs and works when the standard, verification, and remedy are written down. The seller de-identifies before transfer, which keeps the raw records in-house but requires tooling and effort on your side. A named third party does it under contract to both sides, which is the cleanest chain of custody and costs money that comes out of the deal.

Whichever arrangement you choose, the records move directly from you to whoever is doing the work, after signing. We never receive, hold, or process datasets. Our role is to make sure the standard is in the contract and the buyer has agreed to it before we introduce them.

Six clauses turn de-identification from a promise into an obligation.

1. The standard, by name, or the specific rules if there is no named standard that fits.
2. Who performs it and when: before training, within a stated number of days of receipt.
3. Who verifies it, how (sampling rate, method), and who sees the verification report.
4. What happens on a miss: notice, remediation, and the right to require deletion of affected records.
5. Custody of the mapping table: who holds it, who may never receive it.
6. Deletion of the supplied copy at the end of the term, with written confirmation.

If a buyer will not agree to these, that is useful information. The buyers we work with have already agreed to them before an introduction is made.