Insights

Who signed for the model?

Question 06 of ten: who is in charge of each model, and what happens when it is wrong?

James A Lang · · From Chapter 4 of The Right to Be Wrong

The first thing the British Army handed me in 1995 was not a rifle. It was a pen.

Before I touched a piece of kit, I signed for it. Rifle, radio, sleeping bag: each one went into a ledger next to a name, and the name was mine. When I moved on, there was a handover. Every item was counted, checked against the list and signed across to the next person. If something went missing, nobody convened a working group. They knew exactly whose door to knock on.

To a civilian this can look like bureaucracy for its own sake. I spent 22 years inside it and came to see it differently. Signing for kit is not a filing system. It is a ritual of ownership, and like most rituals its real purpose is invisible to the people performing it. It makes sure that nothing in the inventory ever belongs to nobody.

Then I left the Army and started working with boards.

Ask a board how many AI models the organisation depends on and you will usually get a pause, then an estimate. Ask who is in charge of each one and the pause gets longer. The answers, when they come, are revealing. 'The data team.' 'The vendor.' 'IT, I think.' Sometimes, with great confidence, 'the AI steering committee'.

The strange thing is that none of this seems strange to the people in the room. A board that would never approve a building without a named facilities owner will happily run a pricing model, a fraud model and, more uncomfortably, a hiring screen that in practice belong to a committee. Every function is doing its job. The data scientists own the accuracy. Legal owns the contract. Risk owns the register. Nobody owns the model's behaviour on a Tuesday afternoon, months after deployment, when it does something nobody predicted.

Soldiers have a name for this condition, and they learnt it the hard way. In Britain in a Perilous World, Major General Jonathan Shaw puts it plainly: 'a problem owned by everyone is a problem owned by no-one.' On the same page he quotes General Sir Rupert Smith, who commanded British forces in the Gulf War and the UN force in Bosnia, and who was blunter still: 'If you can't identify who is in charge, you are in trouble.' By that measure, most of the boards I work with are in more trouble than they know.

That is the first half of the question. The second half is harder: what happens when it is wrong?

Here the boardroom goes quiet in a different way. Most of the AI governance frameworks I read are detailed about approval and almost silent about loss. They describe how a model gets in. They rarely describe how it gets out: who is allowed to switch it off, how quickly, on what evidence, and what the business runs on in the meantime. Leading an AI platform for the Ministry of Defence taught me to ask that question first, not last. A system you cannot stop is not a system you control. It is one you are hoping about.

To be sure, there are good reasons boards resist this. Models are not rifles. They are built by many hands, fed by data owned by others, and often rented from vendors whose internals nobody in the building can see. Pinning one name on that tangle can feel unfair, even naive. Cynics will say it simply creates a scapegoat.

They have a point, up to a point. A named owner is not the person who built everything. It is the person who answers for it: who knows what the model is for, notices when it drifts, and has the authority to stop it. The Army never expected the soldier who signed for the radio to have manufactured it. It expected them to know where it was.

So my answer to a board is short. Every model the organisation depends on gets one accountable name, written down and properly handed over when that person moves on. Every one of those names comes with a tested route to stop the model. That means a rehearsal, done at least once before it is needed, not a paragraph in a policy. A policy that has never changed a decision is not a control. It is a ritual, and the wrong kind.

The cheapest thing the Army ever issued me was that pen. I suspect it was also the most valuable. Most boards already have the pen. What they are missing is the ledger.

The ten questions boards are asking Join the launch list