When a data engineering team decides to anonymize sensitive data before it reaches an AI pipeline, the first instinct is almost always the same: write a script, run it once, ship it. It works, right up until someone adds a new source, a new field, or a new destination, and the anonymization step has to be found, understood, and patched by whoever's on call that week.
That's the part nobody budgets for. Anonymization gets treated like a project with an end date, when the pipelines it protects don't have one. Data keeps flowing. Every new batch is a new opportunity for the same sensitive values to slip through a rule that was correct the day it was written and stale three months later.
We've watched this play out with a compliance-automation team that came to us mid-build. They were collecting evidence from source control, HR systems, and cloud tools, storing it as raw JSON and LLM-generated summaries in SQL and data storage. That data could contain PII, so it had to be handled before it landed. Their first idea was JSON-level encryption. Partway through evaluating it, they reframed the problem themselves: what they actually needed wasn't encryption, it was policy-based data scrubbing of sensitive values, inline, before the write. That's a different architecture, and it's the one we'd built.
{{banner-large="/banners"}}
What a fast internal build gets right, and what it skips
This was a genuine build-vs-buy decision, not a vendor bake-off. The team had strong data engineers who could have shipped a working scrubber in a sprint or two. They didn't, because they'd already seen what a rushed version leaves out:
Policy nuance. A blind find-and-replace treats every sensitive-looking value the same way. A policy-based approach decides what happens to a value based on what it actually is and where it's headed: driven by classification, not a regex that fires the same way on a 1099 and a marketing spreadsheet.
Referential integrity. If the same customer ID or SSN appears twenty times across a dataset, a naive scrub either breaks the relationship (twenty different placeholders) or leaves it too recognizable (the same literal value, unmasked, twenty times). It needs to resolve to the same placeholder every time: structure preserved, value gone.
Audit logging. The first version of an internal tool almost never logs what it changed and why. The tenth version, built under a compliance deadline, usually does, after someone asks for it during an audit.
None of these are hard problems individually. They're just the ones that get cut when "get something working" is the deadline, and they're exactly the ones that turn into incidents later.
{{cs-1="/banners"}}
Inline, not after the fact
The distinction that matters most isn't redaction versus encryption. It's when data scrubbing happens relative to the write.
A batch job that runs nightly against a data lake is still useful, but it leaves a window: everything written since the last run sits there unscrubbed until the next pass. An inline API closes that window entirely. Scrub API sits in the data path itself (before a write to SQL and data storage, inside a Databricks job, in front of a downstream system like a ticketing tool) and resolves each value synchronously, as part of the request. The data is never at rest anywhere in an unscrubbed state, because there's no "later" for it to wait for.
It supports three output modes depending on what the destination needs: format-preserving (a realistic replacement value that keeps the same shape: useful when downstream systems validate format), fully redacted (a placeholder, when the value itself has no further use), or roles removed (the field dropped entirely). All three carry referential integrity: the same input always resolves to the same output, so joins and relationships in the data survive the transformation.
{{cs-2="/banners"}}
What this looks like when it's real
The compliance-automation team's evaluation came down to speed and completeness. They had the engineering capacity to build something; what they didn't want to spend was the time re-discovering, one incident at a time, which layers actually mattered.
They deployed Teleskope's managed Scrub API into their own AWS account, behind two IAM roles and their existing Google SSO: no crawling required for the scope they needed. It closed cleanly as a build-vs-buy win: faster to market than the internal alternative, with the policy control, referential integrity, and audit trail a rushed build tends to leave for later.
This pattern shows up across the market right now. Security and data teams at different companies, in different industries, are independently landing on the same requirement: real-time, code-level data scrubbing built into the pipeline itself, not a manual step bolted on after the fact.
None of that is surprising once you separate the two questions teams actually have. "Is our data exposed to AI risk?" is a question most security and data teams already have an answer to. "Can we act on that before the data moves, not after?" is the one that's still mostly answered with a script somebody owns informally and nobody's tested against the next schema change.
If you're maintaining that script today, you're maintaining infrastructure, not solving the security problem it was supposed to solve.
