In the DSR (data subject request) work I have been doing, the machinery has a clear shape. A user exercises their right to access or deletion. The system locates every store that holds their data — the user record, the search index, the analytics export, the logs, the customer support ticket, the derived copies in the warehouse. It runs the appropriate workflow: export for access, deletion cascade for erasure. The regulators are satisfied, or close enough. The team moves on.
The assumption embedded in that workflow is that personal data lives in identifiable stores with known schemas and known owners. Run the inventory, build the pipeline, and you can account for the data. That assumption is getting harder to defend as more teams fine-tune models on user-generated content, use retrieval-augmented generation backed by user data, or embed document contents into vector stores. The data is still personal. The store it lives in has no rows to delete.
Where the gap actually is
GDPR's right to erasure under Article 17 does not explicitly address model weights. The guidance most legal teams arrive at is: delete from all identifiable stores, document your process, and proceed. For a 2018 interpretation of what "personal data in a system" meant, that guidance was reasonable. A 2025 system that has fine-tuned a model on a user's support tickets, product feedback, or document contents is in different territory.
A model trained on a user's data has that data implicitly encoded in its weights. It can reproduce patterns from that data. It can generate content in a style shaped by it. Under the right prompting conditions — membership inference attacks, training data extraction — it can reveal information about specific individuals. That is a privacy leak that your DSR workflow does not address, because the data doesn't live in a table. It lives in parameters.
What tagging actually solves
Machine unlearning — removing specific training examples from a deployed model without full retraining — is still mostly research with some promising early results. For most production teams, if a user's data is in a deployed model's training set and that user exercises their right to erasure, the practical options are: (a) retrain from a checkpoint that pre-dates their data, or (b) accept that you cannot fulfil the erasure right with respect to the model and document that limitation. Neither is a good place to be.
The intervention that is tractable right now is upstream: prevent data that carries an erasure risk from entering the training pipeline in the first place. That means tagging. Every record that enters a training pipeline should carry its consent signals as metadata, not just whether it is user data, but whether this specific user, for this specific purpose, has an active consent record that covers training use.
-- A field-level tagging schema for training-eligible records
CREATE TABLE training_record_metadata (
record_id UUID PRIMARY KEY,
source_store TEXT NOT NULL, -- e.g. 'support_tickets', 'doc_contents'
source_record_id TEXT NOT NULL, -- FK to the originating record
user_id UUID, -- NULL if non-personal
consent_purpose TEXT[], -- e.g. ARRAY['product_improvement','model_training']
consent_granted BOOLEAN NOT NULL,
consent_version TEXT, -- the policy version at time of consent
training_eligible BOOLEAN GENERATED ALWAYS AS (
user_id IS NULL OR (consent_granted AND 'model_training' = ANY(consent_purpose))
) STORED,
exportable_at TIMESTAMPTZ, -- earliest time this record can be used
dsr_flagged BOOLEAN DEFAULT FALSE,-- set to TRUE on erasure request
flagged_at TIMESTAMPTZ
);
The dsr_flagged column is the key primitive. When a user files an
erasure request, the DSR pipeline sets this flag on every record derived from
their data — in the production stores, and in the training metadata table. The
pipeline that assembles training datasets filters on training_eligible = TRUE
AND dsr_flagged = FALSE. Records that have been flagged never enter a
future training run.
The lineage problem
The harder problem is the records that already made it into training datasets before the flagging system existed. Training data is often assembled once, compressed, and stored on object storage — an S3 prefix that nobody touches until the next training run. The lineage between those training records and the consent records in the production database breaks at the export step. The file on S3 doesn't know where its rows came from.
Fixing this retroactively is painful. The practical path forward for most teams is:
- Treat training datasets as first-class DSR inventory entries. If you have a data inventory for your DSR programme — and if you don't, stop reading this and go build one — training datasets belong in it alongside your production stores. Each dataset should have an owner, a retention policy, and a process for handling deletion requests against it.
- Preserve lineage at export time. When you assemble a training dataset from a production store, write a manifest. The manifest maps training record identifiers back to source record identifiers and source store names. Store the manifest alongside the dataset. This is a one-time engineering investment that pays for itself on the first DSR that touches a trained model.
-
Build a training exclusion stage into future pipelines.
Before each training run, cross-reference the candidate dataset against the
DSR flag table. Records with
dsr_flagged = TRUEare excluded. This is not machine unlearning — you're not removing their influence from existing weights — but it prevents compounding the exposure in future runs.
def filter_training_batch(
records: list[dict],
db: Connection,
) -> list[dict]:
"""
Filter training records against DSR flags and consent state.
Should run at dataset assembly time, not at training time.
"""
source_ids = [r["source_record_id"] for r in records]
flagged = set(db.execute("""
SELECT source_record_id
FROM training_record_metadata
WHERE source_record_id = ANY(%s)
AND (dsr_flagged = TRUE OR training_eligible = FALSE)
""", [source_ids]).fetchall())
accepted = [r for r in records if r["source_record_id"] not in flagged]
rejected_count = len(records) - len(accepted)
if rejected_count:
log.info(
"training_batch_filter",
total=len(records),
rejected=rejected_count,
pct_rejected=round(rejected_count / len(records) * 100, 2),
)
return acceptedWhat this doesn't solve
It's worth being clear about the limits. Tagging and exclusion addresses future training runs. It does not address models already deployed that were trained on data you can no longer account for. For those models, the honest position is to document the limitation, understand what classes of inference attacks are applicable, and make a risk-based decision about whether retraining is warranted.
Membership inference and training data extraction attacks are not theoretical. They have demonstrated real-world effectiveness against production models, including models much larger than most enterprise fine-tunes. The fact that your model is not GPT-4 does not mean it's not vulnerable. It means the attack surface is smaller and the adversarial incentive may be lower — which is a different thing.
The minimum viable approach
For teams that don't yet have any of this infrastructure, the minimum viable approach is much simpler than the full system described above. Before the next training run:
- Identify every source of personal data in the training dataset.
- Confirm that each source has a consent record covering training use, or exclude it.
- Write a manifest that maps training records back to source records.
- Add the training dataset to your DSR data inventory.
That's not a system. It's a discipline. But it's the discipline that makes a system possible later without requiring you to reconstruct history from scratch.
The teams that are going to handle this well are the ones that start treating training data as a first-class concern in their privacy programme now, before a request forces them to. The ones that don't will build the same infrastructure under pressure, under a deadline, and with worse outcomes.