PII · generation
Synthetic data platform
A production pipeline that scans documents and schemas once, writes a masking blueprint, then fans out synthetic copies with human review, vaultless tokenization, and an agent tool bus.
SYSTEM MAP, PRINCIPLES, TRUST BOUNDARIES
01Principles
- Detect once, generate many. Semantic models and OCR run on a template. The output is a JSON blueprint of spans, types, and scores. Copies are a fan-out over that blueprint.
- Stateful workflows, not raw queues. Masking pauses for human-in-the-loop. Temporal owns the DAG, retries, and the pause — not a pile of consumers.
- Headless APIs. Preview, rules, and chat never mutate source files. The React client overlays boxes from the blueprint.
- Vaultless first. Prefer format-preserving encryption for deterministic synthesis. Stateful faker maps live in Redis with a NoSQL cold path so RAM does not hold the universe.
- Ephemeral plaintext. Unmasked blobs are encrypted with customer keys and auto-deleted. Custom JS synthesizers run in Wasmtime with a 50ms budget and a read-only root filesystem.
- Scale detectors to zero. GPU nodes are KEDA-scaled. Idle cost should be near zero.
02System map
Rendering diagram…
03Temporal DAG
Upload writes the source object, inserts a job row, and starts a workflow:
Extract → Detect → HITL pause → Fan-out generate → Package
The pause is a first-class signal, not a poller pretending to be a human. Rejection re-enters Detect with the previous blueprint as prior.
04Trust boundaries
| Boundary | Rule |
|---|---|
| Object store | Plaintext TTL + KMS. No long-lived unmasked blobs. |
| Workers | Generation pods readOnlyRootFilesystem. No shell. |
| Synthesizers | Wasmtime, 50ms, memory cap. Not eval in the worker. |
| Chat | Natural language maps to named MCP tools. No free-form SQL from the model. |
| Tokens | Short-lived MCP tokens carry a persona (which tool groups). Guidance is not an access control. |
05Failure modes
- Detector idle: KEDA scales to zero; the workflow waits, it does not crash.
- Blueprint rejected: HITL writes corrections; the feedback agent later weights those corrections (0.35 in the ensemble).
- Cache miss on a mapping: Redis → cold store → generate and backfill.
- Tool call ambiguity: the MCP gateway returns
needs_clarificationinstead of guessing.