Picking your first AI automation: a scoring method that survives contact with reality
Most automation projects stall because the wrong process was chosen. Here is the scoring model we use to pick a first workflow that will actually ship.
By AllFixAI engineering
Start with tasks, not ambitions
Ask a room what they would like AI to do and you will hear 'write our reports'. Ask them what they retype every day and you will get something specific: invoices into the accounting system, supplier emails into the ticket queue, CVs into a consistent format. The second conversation produces projects.
The right first candidate has a clear input, a clear output, a measurable current cost, and a forgiving failure mode. If a mistake is expensive or irreversible, it is a bad first project even if the volume is attractive.
Score every candidate on five axes
We list every candidate workflow and rate each on a simple one-to-five scale. It takes an afternoon and it prevents a quarter of wasted effort.
- Volume — how many times per week does this happen?
- Handling time — how long does one instance take a person today?
- Structure — are the inputs consistent enough to define a schema?
- Tolerance — what is the cost of a single wrong output?
- Ownership — is there one named person who wants this fixed?
Measure the baseline before you build
Without a baseline you cannot prove value, and unproven projects get cancelled. Time twenty real instances, count the errors that reach production today, and work out a cost per document. This is also how you set the quality threshold the automation has to clear.
Write the number down and show it to finance. When the pilot reports later, you will be comparing against something everyone already accepted as true.
Design the fallback first
Decide before launch what happens when the model is unsure. Usually the answer is a review queue: high confidence flows straight through, low confidence goes to a person with the original document side by side. That queue is not a failure of the automation — it is the feature that lets you roll out safely and collect labelled examples for free.
Put a ceiling on spend and a cap on irreversible actions. An agent that can email a customer without approval is a risk decision, not a technical one.
Plan for the second month
Automations drift. Documents change format, models get updated, and edge cases accumulate. Budget for evaluation from the start: a set of known-good examples, an automated scorer, and a weekly glance at the confidence distribution. Twenty minutes a week keeps a useful automation useful.
Next article