Every time EGM or SEDDK announces a new reporting obligation, the same conversation plays out in the same meeting rooms. Somebody prints the technical specification. Somebody else asks how many weeks we have. A project manager builds a Gantt chart. IT scopes the extract. Reporting scopes the submission. Everyone agrees it is tight but doable.
Then the work starts, and within two weeks the project quietly stops being about the deadline and starts being about archaeology.
The visible work is not the work
Building a pipeline that produces an XML or XBRL file matching a regulator's schema is not hard. The transformation logic is deterministic. The submission channel is documented. The validation rules are published. A competent team can wire this in a sprint.
What takes months is the part nobody scoped: figuring out whether the data the regulator wants was ever captured in the first place, and if it was, whether it means what the regulator thinks it means.
Some concrete examples from the last few years in Turkish insurance and pensions:
- A new field asks for the channel through which a policy was sold. Your core system has been storing this as a free-text agent code for eleven years. Half of those codes belong to intermediaries that no longer exist. Nobody ever normalized broker vs. bancassurance vs. direct because internal reporting never needed the distinction.
- A regulator wants premium breakdowns by coverage type at a granularity your product configuration never enforced. The data is technically there, but it is spread across three product generations, each with its own coding conventions, and the mapping table lives in a retired analyst's inbox.
- A cross-border reporting requirement needs the ultimate beneficial owner's country of residence at policy inception. You have current UBO country. You do not have historical UBO country, because you overwrite the field on every KYC refresh.
In every one of these cases, the pipeline work took two weeks. The data reconstruction took four to seven months.
The gap was created years before the regulation existed
This is the part that makes regulatory onboarding structurally miserable: the deficiency was locked in a decade ago, by product managers and system architects who had no reason to preserve a dimension nobody was asking about.
Data models optimize for the questions being asked at the time they are designed. If your 2014 policy administration system did not distinguish between five subtypes of health rider, no ETL and no data lake built on top of it will retroactively produce that distinction. You can infer, you can approximate, you can back-populate from adjacent fields — but you cannot recover information that was never recorded.
And regulators do not care. They ask for the field, and "our historical system did not capture this" is not an acceptable response. So firms end up doing one of three things:
- Building elaborate inference logic that produces a defensible but fundamentally reconstructed value
- Reporting nulls and negotiating with the regulator about historical scope
- Silently misreporting because the deadline arrived and the inference engine was not ready
The third option happens more than anyone admits.
Why every firm keeps making the same mistake
Because the onboarding budget is written by whoever read the technical specification, not by whoever knows the data. The spec is fifty pages of field definitions and validation rules. It reads like a software problem. So it gets estimated as a software problem.
The person who could tell you that policy_channel has been unreliable since the 2018 core migration is a senior analyst who is not in the scoping meeting. The person who knows that product_code_v2 was never fully reconciled with product_code_v3 works two floors down and reports to a different director. The institutional memory required to correctly scope a regulatory obligation is distributed across people who are not asked, and documented in places nobody searches.
So the estimate is always wrong by the same factor. Not because the team is incompetent, but because the scoping question was framed incorrectly. It should not be "how long to build the pipeline." It should be "how confident are we that the fields the regulator is asking for exist, are consistent across time, and mean what the regulator thinks they mean."
What actually works
A few practices that shorten the pain, based on what I have seen work versus what teams keep re-learning:
- Do a data audit before you scope the project. For every field in the new specification, answer three questions: does this exist in our systems, has it been captured consistently for the historical window the regulator wants, and does our definition match theirs. This audit takes a week and saves months.
- Treat historical gaps as a separate workstream. Reconstruction, inference, and back-population are not pipeline work. They need their own owner, their own sign-off, and their own documentation trail because auditors will ask.
- Document the reconstruction logic before you submit. When the regulator comes back in eighteen months asking why a specific policy has a specific channel classification, you need to be able to explain the rule. "We inferred it from agent code prefix" is a defensible answer. "I don't remember" is not.
- Push back on retroactive scope where you can. Some regulators will accept a cutoff date for historical fields if you make the case early. None will accept it if you make the case two weeks before the deadline.
- Invest in dimensional completeness now, for regulations that do not exist yet. The next obligation is coming. The fields it asks for are already being captured — or not — in the decisions you are making this quarter about product design and system architecture.
The uncomfortable conclusion
Regulatory onboarding is not a project management problem. It is a data governance problem that becomes visible only when an external party forces you to produce something your internal reporting never demanded. Every new obligation is a stress test on decisions made years earlier by people who had no way of knowing what would eventually be asked of them.
The firms that handle this well are not the ones with the best pipeline engineers. They are the ones who treat their historical data model as a liability that accumulates interest, and who understand that every field they choose not to capture today is a reconstruction project they are scheduling for their future selves.