How we build · for technology and risk teams
The detail behind the overview.
How we judge whether a data process is good enough to rely on, and how we decide which controls a piece of AI-assisted work needs.
Principles
Five pillars of a good data process.
When we look at a data flow, we ask five questions of it.
- 01
Accurate
Can I rely on this data?
Correct, complete and reconciled, so the people and systems downstream can act on it without checking it first.
- 02
Accessible
Can every consumer get exactly what it needs?
Whether the consumer is a person or a system, it gets the right data, in the right shape, at the right grain, and nothing it should not see.
- 03
Efficient
Is it using appropriate resources, and is it on time?
Compute and storage matched to the job, people's time spent where their judgement matters, and results landing when the business needs them.
- 04
Robust
What happens when it breaks?
Redundancy where it matters, observability so failures are seen, and a clear path to fix them.
- 05
Secure
Is the data protected appropriately?
Data loss prevention controls in place, access governed, and protection proportionate to how sensitive the data is.
Old process vs AI rebuild · illustrative
- Accurate
- + Better
- Accessible
- + Better
- Efficient
- + Better
- Robust
- + Better
- Secure
- − Worse
A replacement has to match or beat the old process on all five.
A new process that is faster and cheaper but less accurate, or easier to use but harder to secure, has not delivered. Exceeding on four pillars does not make up for failing on one.
This matters more in the age of AI. It is now easy to rebuild a process quickly and show that it is better on the measure everyone is watching. The questions that get skipped are the ones that decide whether the new process is safe to rely on.
Controls
Not all code deserves the same controls.
Applying one development process to everything either slows low-risk work to a crawl or lets high-risk work through with too little scrutiny. We separate the two.
Where risk sits in a data flow
Sources
- Market data feeds
- Custodian files
- Member records
Core nodes
- Ingestion and validation (high risk)
- Security master (high risk)
- Positions and valuations (high risk)
- Member data de-identification (high risk)
End of flow
- Regulatory reporting (high risk)
- Member statements (high risk)
- Ad-hoc research (low risk)
- Team dashboards (low risk)
- One-off analysis (low risk)
High risk
Durable by design.
Processes that are nodes for key data flows, or that hold sensitive or regulated data. Other systems and decisions depend on them, so a failure travels.
- Peer review, testing and staged releases
- Formal change management and audit trail
- Lineage, monitoring and rollback plans
- AI-assisted, with every change owned and reviewed
Low risk
Disposable by design.
Work that sits at the end of a flow, such as internal ad-hoc research and one-off reports. Nothing depends on it, so speed matters more than permanence.
- Fast, self-serve builds close to the question
- Sandboxed, read-only access to de-identified data
- Light-touch review proportionate to the risk
- Built to be thrown away, or promoted if it earns it
Two paths to production
Durable path
- Change request
- AI-assisted change
- Peer review
- Automated tests
- Staged release
- Monitor
Audit trail, lineage and a rollback plan at every step.
Disposable path
- Question
- De-identified sandbox
- AI-assisted build
- Answer
- Discard
If others come to depend on it, it is promoted and re-enters through the durable path.
AI speeds up both kinds of development, but not with the same setup. Durable systems need toolchains and change management that keep every AI-assisted change reviewable and accountable. Disposable work needs safe, governed environments where people can build quickly without putting core data at risk.
We help organisations get good at both, and draw the line between them in the right place.