Regulators do not grade on a curve. A GEV file rejected for a single malformed TCKN is the same outcome as one rejected for systemic data corruption — you missed the window, you owe an explanation, and someone in the compliance team is now drafting a letter you do not want to read. After several years of running statutory pipelines simultaneously — GEV and HAYMER for insurance, FATCA and CRS for cross-border tax reporting, and weekly Merkez Bankası submissions — I have a strong opinion on what zero-error tolerance actually means operationally.
It is not a quality goal. It is an architectural constraint that, if you take it seriously, forces decisions you would not otherwise make. Compliance frameworks describe the what. They almost never describe the how in a way that survives contact with production data.
Stop Treating Validation as a Stage
The most common architecture I see — and the one I inherited more than once — treats validation as a step near the end of the pipeline. Extract, transform, validate, submit. When a record fails validation at the end, you have three bad options: drop it, fix it manually, or delay the submission.
All three are operational failures dressed up as process.
Validation has to be a property of every stage, not a stage of its own. Concretely:
- Schema enforcement at ingestion. If your source system can emit a policy record with a null TCKN or a malformed VKN, you catch it at the boundary where the record enters your pipeline, not three hops downstream where the lineage is murky.
- Type contracts between stages. Each transformation step declares what it accepts and what it emits. If a downstream stage expects a 10-digit VKN and gets 11 digits, the pipeline fails loudly at the transition, not silently in the output file.
- Referential checks during transformation, not after. When you are mapping policy data for HAYMER, every foreign key — agent code, branch code, product code — gets resolved against the reference dataset in-flight. A broken reference is a pipeline error, not a submission error.
The shift is philosophical: errors are not things you find at the end. They are things the pipeline refuses to propagate.
Reconciliation Hooks Are Non-Negotiable
Every statutory submission has a number it must agree with. For GEV, it is the production reports from the policy admin system. For FATCA and CRS, it is the customer master and the holdings ledger. For Merkez Bankası, it is the general ledger.
If you cannot reconcile your submission to its authoritative source before you submit, you are gambling.
The operational pattern that works:
- Every pipeline emits not just the submission file but a reconciliation artifact — counts, sums, hash totals, broken down by the dimensions the regulator cares about (branch, product, period).
- A separate process, ideally owned by a different person, pulls the same numbers from the authoritative source and compares.
- The submission is gated on the reconciliation passing. No exceptions. No "we'll fix it in next month's submission."
I have seen teams skip this because the source system is slow, or because the reconciliation query is hard to write. Both are real problems. Neither is an acceptable reason to submit a file you have not verified. If the reconciliation is too expensive to run on every submission, the answer is to make it cheaper, not to skip it.
Rollback Has to Exist Before You Need It
Most teams discover they have no rollback protocol the first time they need one. By then it is too late.
For statutory pipelines, rollback means three concrete things:
- Every submission is reproducible from versioned inputs. If I want to know what the GEV file looked like on the 15th of last month and why, I can rebuild it from the exact source snapshots and the exact code that produced it. This requires snapshotting source data at submission time and tagging the code version. Not optional.
- Corrections follow a documented protocol. When you discover an error after submission — and you will — there is a clear path: identify the affected records, produce a correction file, document the delta, notify the regulator if required. The first time you do this should not be a panic.
- You can re-run a submission against a fixed dataset without re-running everything. If the issue was in a reference table, you fix the reference and re-emit. The pipeline supports this because it was designed to.
The Human Layer Most Frameworks Ignore
Compliance documents describe controls. They rarely describe the operational habits that make controls actually work. A few that I will not negotiate on:
- No submission on a Friday afternoon. Not because the system cares, but because if something fails, the people who can fix it are leaving. Schedule submissions when the bench is full.
- One person submits, another person verifies. Even if the verification is a five-minute check against the reconciliation artifact, the second pair of eyes catches things the first pair stopped seeing months ago.
- Every rejection becomes a regression test. When FATCA rejected a batch because a passport number contained a space, that exact case became a test that runs on every subsequent batch. The pipeline does not forget mistakes.
- Calendar discipline. Statutory deadlines do not move. The internal deadline for being submission-ready is at minimum 48 hours before the regulatory deadline. If you are testing the submission file at 4pm on the deadline day, you have already failed — you just do not know it yet.
Where Compliance Frameworks Fall Short
The frameworks — internal audit checklists, ISO-flavored quality manuals, the standard "data governance" deliverables — describe a world where errors are caught by review. They assume a human in the loop with time to think. In practice, a person looking at a 200,000-row submission file cannot meaningfully review it. The review is theater unless the pipeline itself is the control.
This is the part that most compliance teams resist, because it sounds like you are removing oversight. You are not. You are moving the oversight to where it can actually function — into the design of the pipeline, the contracts between stages, the reconciliation gates, the rollback protocol. The human review becomes a check on the system, not a check on the data.
Zero-error tolerance is achievable. It is not achievable by trying harder. It is achievable by designing systems that refuse to produce errors in the first place, and by accepting that this constraint will shape every architectural decision from ingestion to submission. Once you accept that, the work becomes clear. Until you accept it, you are one bad batch away from a letter you do not want to write.