Data, automation & AI
How to choose what to automate first
A scoring method for picking automation and AI candidates that survive contact with production — and the four traits that predict failure.
Automation programmes can fail on selection rather than technology: the first candidate is chosen because it is visible, or because a tool was already licensed, rather than because it is a good fit.
What follows is a scoring pass you can run in an afternoon with the people who do the work. It applies equally to scripted automation, workflow tooling and language-model features — the selection question is the same.
Build the candidate list from observed work, not opinions
Ask each team for the tasks they would drop if they could, then watch two of them actually happen. The gap between the described process and the observed one is where the automation risk lives.
Record each candidate at task level, with a trigger and a finishing condition: “when an invoice arrives by email, extract the header fields and stage it for approval” — not “automate accounts payable”.
Score each candidate on five axes
- Volume — how often it runs. Weekly beats annually, regardless of how painful the annual one feels.
- Time per run — measured on the observed instances, including the waiting and the rework.
- Variability — how many genuine exception paths exist. Count them explicitly; this axis is easy to underestimate.
- Input quality — is the input structured and consistent, or a free-text field filled in differently by each person.
- Cost of being wrong — what happens if it silently produces a bad result, and how quickly anyone would notice.
Score one to five per axis. Volume and time per run push a candidate up; variability and cost of being wrong push it down. Anything with the maximum cost of error stays out of the first wave whatever else it scores.
Reject the four shapes that predictably fail
- The undocumented judgement call. If two experienced people reach different answers from the same input, the rule does not exist yet — write it down before automating it.
- The process nobody owns. Automation makes a process rigid; a rigid process with no owner has no path to being fixed.
- The system with no supported interface. Driving a UI that was never designed for it converts a business problem into a permanent maintenance liability.
- The task about to be replaced. Automating a workflow inside a platform you are migrating off buys a few months of savings and a rewrite.
Where language models change the scoring — and where they do not
Language models genuinely relax the input-quality constraint: messy text, inconsistent formats and unstructured documents become tractable. They do not relax the cost-of-error constraint, because the failure mode is a confident wrong answer rather than an obvious crash.
- Prefer tasks where a human already reviews the output as part of the flow — the check is free.
- Prefer extraction, classification, drafting and summarising over anything that commits an irreversible action.
- Require a verifiable output: a field that can be reconciled, a citation that can be checked, a total that must match.
- Measure quality against a labelled sample before rollout, and keep measuring it after. Accuracy drifts as inputs change.
Design the first one to be reversible
- Run it alongside the manual process and compare outputs before it replaces anything.
- Route exceptions to a person by default. An automation that handles eighty per cent cleanly and escalates the rest beats one that guesses.
- Log every decision with its inputs, so a wrong result can be explained rather than re-enacted.
- Define the switch-off condition, and confirm the manual path still works if you use it.
Measure the thing you actually promised
Hours saved is the claim; capture the baseline before go-live or the claim is unfalsifiable. Track error rate and exception rate beside it — an automation that saves time while quietly increasing rework is a net loss that hides well.
Publish the result internally either way. The credibility earned by reporting a disappointing first automation honestly is what funds the second one.
What to take away
- Define candidates at task level, with a trigger and a finishing condition.
- Score volume, time per run, variability, input quality and cost of error.
- Reject undocumented judgement calls, unowned processes, unsupported interfaces and soon-to-be-replaced systems.
- Language models relax input quality, never cost of error.
- Run the first automation alongside the manual process, and log every decision.
- Capture the baseline before go-live, and track exception rate beside hours saved.
Further reading
- AI Risk Management Framework — NIST
Related service
Data, Automation & AI
If this is the decision in front of you, start a conversation with the topic already selected — it saves explaining the background twice.
