How to Run an AI Automation Audit on a 20-Person Operations Team

Most automation projects fail before any code exists. They fail at selection: someone picks the process that annoys the loudest person rather than the process with the volume, and six weeks later the saving is real but too small to notice. An audit fixes selection. It takes about five working days for a team of twenty, most of it spent watching rather than talking, and it ends with a ranked list you can defend to whoever holds the budget.

This is the process we run before scoping AI workflow automation, and it works whether we do the build afterwards or you do. The output is deliberately boring: a spreadsheet of processes with volumes, times, exception rates and a recommendation. Boring is the point. It is very hard to argue with a number your own team produced.

What an automation audit actually is

AI workflow automation means taking a sequence of operational steps that a person currently performs and giving part of that sequence to software, where at least one step needs judgement rather than a fixed rule. The judgement part is what makes it AI rather than a script. The audit's job is to find the sequences where that split is clean, the volume is real, and the cost of a mistake is survivable.

It is not a technology assessment. You do not need to know which model you will use to run this. You need to know what happens on a Tuesday.

Day one: inventory the work, not the roles

Sit with each function for forty minutes and list what they do, in units of work rather than job titles. Not "accounts payable", but "match a supplier invoice to a purchase order", "chase an approval", "answer a supplier query about payment date". A twenty-person operations team usually produces between forty and seventy of these. Anything shorter than that means people are describing their jobs instead of their tasks, and you need to ask again with a specific day in mind.

Two questions per item, no more: roughly how often does this happen, and roughly how long does it take. Rough is fine on day one. You are looking for the order of magnitude that tells you what to measure properly.

Day two: measure the top twelve

Rank by frequency multiplied by duration, take the top twelve, and get real numbers for those. Real means counted, from a system export, a mailbox, a ticket queue or a tally sheet kept for a week. Estimates inflate. In our experience the estimate is high on duration and low on frequency, and the two errors do not cancel out.

• Volume for the last complete month, from a source you can re-export later.
• Median handling time, timed on a sample of at least ten cases.
• Straight-through rate: what fraction completes with no exception, no chase and no rework.
• Rework rate and where the rework is caught, since a defect caught three days later costs far more than the original task.
• Who does it, and whether it is the only thing they do or the thing they do between other things.

Day three: find the judgement step

For each of the twelve, walk the process at the desk of the person who does it and mark every step as one of three kinds. Deterministic: the same input always produces the same action. Judgement: the person reads something and decides. Access: the person is only there because a system has no integration.

Access steps are integration work, not AI work, and they are usually the cheapest win in the whole audit. Deterministic steps are ordinary code. Judgement steps are the AI candidates, and the useful question about each one is how much context the person needs in their head to make that call. If the answer is a document and a rule, a model can do it. If the answer is four years of knowing this particular customer, it cannot, and you should stop there.

Day four: score and reject

Score each candidate on four axes: annual hours consumed, exception rate, cost of a wrong output, and how quickly a wrong output would be noticed. The first two say whether it is worth doing. The last two say how much review and guardrail you have to build, which is the part that is always underestimated.

Then reject deliberately. A candidate should be rejected if the volume is under roughly thirty cases a month, if the process is scheduled to change anyway, if nobody owns the output, or if a wrong result would reach a customer or a regulator without passing a person first. Writing down the rejections and the reason matters as much as the shortlist, because otherwise the same idea comes back in three months with a new sponsor.

Day five: write the pilot brief

Take the top candidate and write one page. What the process is, the baseline numbers from day two, which step the model performs, what the model is not allowed to do, who reviews the output, what threshold sends a case to that reviewer, and what number has to move for the pilot to be judged a success. Add the date you will judge it on.

That last item is where discipline usually leaks. A pilot with no end date becomes a permanent experiment that nobody wants to cancel. Give it six or eight weeks and a specific figure to beat.

What the results usually look like

For a twenty-person operations team, an audit like this typically surfaces three or four candidates worth building and a longer list of small integration fixes that were never anyone's project. The integration fixes often deliver first, because they are cheap and they remove the access steps that were making the real processes look worse than they are.

It also tends to surface one uncomfortable finding: a process everyone assumed was heavy turns out to consume six hours a month, and a process nobody mentioned consumes eighty. That is the audit paying for itself. Intuition about where operational time goes is reliably wrong, and it is wrong in both directions at once.

If you would like us to run this with your team, or to review a shortlist you have already produced, our AI automation service page describes what a pilot engagement includes and how we handle the review and guardrail design that the scoring in day four points at.