The data you thought you deleted
Not every isolation problem is a bug in the code. Sometimes it is a gap in the docs, and on a privacy feature that gap is the whole story. While checking data-deletion completeness on Langfuse, the LLM observability platform, we found that its erasure documentation implied more than the deletion actually did. The clarification is now merged into Langfuse's official docs.
Most of our research finds cross-tenant reads. This is the sibling question we ask right next to it: when a customer deletes a user, does the data actually leave every surface? That is the core of a GDPR Article 17 erasure, and it is where "delete" and "deleted everywhere" quietly diverge.
What the docs said, and what deletion did
Langfuse lets you delete a user's traces by userId, which is the path its
Managing Personal Data guidance
points to for a data-subject request. Trace deletion cascades: it removes the traces and their
related observations and scores. The reasonable reading, for anyone using this to satisfy an
erasure request, is that the user is now gone.
But personal data in an LLM stack does not only live in traces. It can also sit in other objects
a team creates, such as datasets built from real inputs. Deleting a user's
traces does not touch those. So a userId-scoped deletion can look complete while
leaving the same personal data behind one surface over, exactly the kind of residue an erasure
attestation exists to catch.
The fix: make the docs match reality
The change is small and now live in Langfuse's official documentation, on the Data Deletion page:
Trace deletion (e.g., deleting a user's traces by userId for a data deletion
request) removes the traces and their related observations and scores. It does not
remove personal data that may also be stored in other objects, such as datasets. To erase a
user's data completely, delete those objects as well, or delete the entire project.
No behavior changed; the deletion works as designed. What changed is that a reader relying on it for compliance now knows the boundary, and what else to delete to actually finish the job. Langfuse reviewed and merged it promptly.
Why this is the same work as the code findings
Verifying isolation and verifying erasure are two sides of one question: can data reach, or remain, somewhere it should not? A cross-tenant read is data reaching the wrong tenant. An incomplete erasure is data remaining after it should be gone. Both hide in the second and third place the same object is stored, the place nobody re-checked. Sometimes closing that gap is a pull request against the code. Sometimes, as here, it is making the privacy documentation state the boundary honestly, so a team does not certify an erasure that did not happen.
Check your own erasure surfaces
If you run anything that stores user content, the erasure question to ask is concrete: when you honor a deletion request, which surfaces does it actually clear, and which does it leave, logs, caches, datasets, fine-tunes, backups, search indexes? The dangerous ones are the surfaces that aren't on the deletion path but still hold the data.
Sectum AI checks exactly this at runtime: it plants canary data across a stack's surfaces and verifies what a deletion actually removes, with a signed evidence pack you can hand to an auditor. It is open source; point it at your own stack.
Explore the open source More research
Contribution: langfuse-docs PR #3064 (merged 2026-09-17), a documentation clarification on erasure completeness. This is a docs fix, not a vulnerability. Found while verifying multi-tenant isolation and data-deletion completeness with Sectum AI.