What we actually mean when we say AI automation
"We want to add AI" arrives in our inbox regularly. It is never a requirement. It is a budget line looking for a problem, and the useful first move is to find out what the problem is.
Usually there is a real one underneath. Someone is spending six hours a week re-typing figures from PDFs into a spreadsheet. Enquiries sit unanswered for two days because nobody sorts the inbox. A report takes a full day to assemble every month. These are genuine costs and they are worth removing.
What is not settled is whether a language model is the thing that removes them.
The first question: is the input structured?
This one question splits most projects correctly.
If data arrives in a consistent, predictable shape — a form, a CSV, an API response, a database — you almost certainly do not need a model. You need a script, a scheduled job and somewhere to put the result. That solution is cheaper to build, runs for a fraction of the cost, produces the same output every time, and fails loudly instead of quietly.
Models earn their place when the input is genuinely unstructured and variable: scanned documents in inconsistent layouts, free-text enquiries, handwritten forms, long documents that need summarising, anything where the rule "if it looks like X do Y" would need hundreds of exceptions.
If you can write the rule down, write the rule. A model is what you reach for when the rule cannot be written down.
The second question: what does being wrong cost?
Language models are probabilistic. They will be wrong sometimes, and they will occasionally be wrong with total confidence. That is a property of the technology, not a bug to be patched out.
So the design question is what happens when it is wrong. We sort tasks roughly into three buckets:
- Low cost of error — drafting an internal summary, suggesting tags, sorting an inbox by likely topic. Let it run. A human is reading the output anyway.
- Medium — extracting figures from invoices, drafting a customer reply. The model does the work, a person approves before it counts. This is where most useful automation sits.
- High — anything that moves money, changes a medical or legal record, or goes to a customer unreviewed. Either do not automate it, or automate only the retrieval and leave the decision with a person.
Most disappointing AI projects we have seen are a bucket-three task built as though it were bucket one.
The third question: can you tell what it did?
A system nobody can inspect is a system nobody can fix. Every automation we build logs what came in, what came out, which version of the prompt or model produced it, and whether a human changed the result afterwards.
That last field is the valuable one. The rate at which people correct the output is the only honest measure of whether the thing is working. Without it you are relying on the absence of complaints, and people stop complaining long before they stop being annoyed — they just quietly go back to doing it by hand.
What this looks like in practice
A realistic example, the kind of request we get often. An organisation receives supplier invoices as PDFs and email attachments in maybe a dozen different layouts. Someone re-keys them into an accounting system.
What we would build: extraction reads each document and pulls out supplier, date, line items and total. Everything lands in a review queue showing the extracted values next to the original document. A person confirms or corrects, and only then does it reach the accounting system. Corrections are logged.
What that gets you: the re-typing disappears, the checking remains. Accuracy stays at human level because a human is still the last step, but the work goes from typing to glancing. And after a month of correction logs you know exactly which supplier's layout is causing trouble and can fix that one case specifically.
What we would not build: the same thing without the review queue, posting straight to the accounts. It would demo better and it would be a genuinely bad idea.
The cost conversation
Model pricing is per unit of text, which makes the running cost invisible until it is not. A feature that costs a rupee per request is fine at fifty requests a day and is a problem at fifty thousand.
We estimate the per-request cost and the realistic monthly volume before building, and we tell clients the number. Sometimes that conversation ends the project, which is the correct outcome when it does. It is much better to have it in week one than in month six.
Why we tell clients not to buy things
Recommending the spreadsheet, the scheduled script or the better-designed form instead of the AI project means a smaller invoice for us. We do it anyway, for a reason that is not entirely selfless: the client who was told the truth about a small project is the one who calls back about a large one.
If you have a task you think might be automatable, we are happy to look at it and tell you which of the three buckets it is in. There is no cost to asking, and a reasonable chance the answer saves you money.