A missing record can become a missing public service

Picture a public programme pulling records together from hundreds of local offices. One name arrives spelled three ways. A date of birth is missing. One office’s extract turns up a week late. None of that stops the central report from showing a clean, confident total.

That clean total is the problem. Behind every unresolved record there may be a person, and the operational view can make them invisible.

At population scale, reconciliation stops being a data chore and becomes part of service design. A record can decide whether someone gets a follow-up visit, how resources are allocated or whether a person can reach a service at all. Before choosing how to treat uncertain matches, missing records and late updates, you need to understand what each one means for the people involved.

Keep the exception, not just the total

Keep accepted, unresolved and rejected records clearly apart. Explain why each unresolved record is unresolved, and keep its source reference. A record that fails a technical check shouldn’t drop out of sight just because it can’t enter the final dataset yet.

Set thresholds according to consequence. Wrongly joining two people’s records causes a different harm from leaving a duplicate unresolved. A threshold that’s fine for a broad statistical estimate may be wrong for a decision about an individual. Programme owners should make that trade-off with their operational and governance teams, not leave it to a default setting.

The United Nations’ universal safeguards for digital public infrastructure frame these systems around safety and inclusion, with attention to individual and societal risk. For the people running the data, our reading is concrete: make exclusion and correction as visible as throughput.

Give every correction an owner and a way home

A central team can spot a conflict without having the authority or the evidence to resolve it. Identify who, locally or in the programme, can investigate. Send them the relevant records and a clear reason, not an unexplained error code.

Record the decision and keep the previous state. If the centre fixes its own copy while the originating office keeps sending the same error, you’ve created recurring work. Agree how an approved correction gets back to the source owner and how later submissions are checked.

And account for local reality. A late extract might mean a connectivity problem, a staffing gap or a different working calendar. Distinguish an event that didn’t happen from one that hasn’t arrived yet. Otherwise a transmission problem gets mistaken for a service failure.

Be honest about what a total can and can’t do

A good management view shows the reporting period, source coverage, late submissions and unresolved exceptions alongside the number. Those details let leaders judge whether a total is ready for the use they have in mind. A provisional total can support planning when its limits are clear. The same total may be unfit for a decision about an individual.

The UK’s Government Data Quality Framework separates completeness, uniqueness, consistency, timeliness, validity and accuracy, and they often pull against each other. Waiting for every source improves completeness and can make the data too late for an urgent decision.

Be wary of a single composite quality score. A programme needs the failure modes that matter and their consequences, not a reassuring percentage nobody can interpret.

Let AI support the investigation, not replace the judgement

AI can help organise evidence and suggest possible matches. Accepting a consequential match, or changing how someone receives a service, still belongs in the programme’s approved process, with a person who has the authority to decide. Every suggestion should carry enough evidence for that person to disagree.

Keep operational reports separate from individual decisions. A clearer aggregate view does not, on its own, authorise a change to anyone’s service.