AI-assisted billing review for professional firms: what the evidence supports

Author: Adam Khalid, Founder and CEO, Intel Made Simple. Sources last checked 28 October 2026.

Billing review is a tempting first use of AI in a professional firm. It is repetitive, the data sits in one system and the cash value of a good result is easy to see. It is also a place where a wrong answer costs real money and, for solicitors, regulatory trouble. This guide sets out what two well-known studies and the Solicitors Regulation Authority (SRA) say, then turns it into a test you can run before you trust any tool with your WIP or debtor list.

What does research say about AI on routine work?

The National Bureau of Economic Research (NBER) paper Generative AI at Work (opens in a new tab) studied the staggered introduction of a generative AI conversational assistant using data from 5,179 customer support agents. Access to the tool increased productivity, measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers, but with minimal impact on experienced and highly skilled workers.

Customer support is not billing, and we do not claim the 14% transfers to your firm. The finding that matters for billing is where the gain landed: on the less experienced staff doing structured work. Billing review is structured work, and a junior member of the finance team may gain more than a finance director.

What happens when the task is harder?

Harvard Business School's Working Knowledge site summarised a study of 758 consultants at Boston Consulting Group (opens in a new tab) who did tasks with and without GPT-4. On tasks inside the AI's capability, consultants using AI finished 12 percent more work at a 25 percent faster clip, and work rated 40 percent higher in quality. AI also gave consultants rated below the median beforehand the biggest boost, 43 percent compared with 17 percent for top performers.

On a more complex task designed so that consultants would excel and AI would struggle without extensive guidance, the result reversed. Consultants using AI were 19 percentage points less likely to produce the right answer. The article breaks that down: a 24 percentage point decline for consultants using AI plus training and a 13 percentage point drop for those using AI alone. It also notes that AI may somewhat stifle the diversity of ideas.

The lesson the researchers draw is a method, not a verdict. Managers need to select applications carefully, train staff to use the tool properly and move the line as AI improves. One of the authors, Edward McFowland III, says finding which tasks fall on which side of the “jagged technological frontier” requires studying it with your people, deeply in your context.

What does the SRA say about using AI?

The SRA's Risk Outlook report on AI (opens in a new tab) is the clearest UK statement for law firms. It says that you remain responsible and accountable for the outputs from AI you are using, and that you cannot delegate accountability to an IT team or external provider. It explains hallucination as a system producing highly plausible but incorrect results, and advises firms to supervise AI systems, and staff use of them, to make sure they are working as expected and providing accurate results, with supervision systems that can cope with the increased speed of AI.

Reporting on the report, Legal Futures (opens in a new tab) records the SRA warning solicitors not to trust AI to judge its own accuracy, and the regulator's line that AI should be used to speed up and automate routine tasks, supporting rather than replacing human judgement. The SRA also notes, in the same report, that it hears from small firms using a combination of off-the-shelf and generic AI technologies.

Which billing tasks suit AI, and which do not?

Putting the studies together gives a sorting rule. This is our application of the research to billing, and the research does not test these tasks directly.

• Likely inside the frontier, good first candidates: listing matters with recorded time and no bill, banding WIP by age, matching time entries to file activity, drafting a first-pass narrative for a bill from the time entries, and summarising debtor notes into a call list.
• Likely outside the frontier, keep with a person: deciding a write-down or write-off, judging whether a client will accept a bill, interpreting a retainer or fee agreement, and advising on a dispute.

The rule of thumb: AI proposes, a person disposes, and the person can see the evidence behind each proposal.

How do you test an AI tool before you trust it?

Because the SRA says not to trust AI to judge its own accuracy, the check has to come from outside the tool. This protocol is ours and takes a few hours.

1. Pick one task from the first list, for example flagging aged WIP with no recent time entry.
2. Run the tool on a closed set of data where you already know the answer, such as last quarter's matters after a manual review.
3. Take a random sample of 50 flagged items and have a person mark each as correct or incorrect against the source records.
4. Record the share that were correct, and separately record any matters the tool missed that the manual review caught.
5. Before you run the test, have the partners set the pass mark in writing. Do not set it after you see the result.
6. Run the tool live only on a pilot set at first, with a person approving every action, and repeat the sample test monthly for the first quarter.
7. Keep a log of each sample, its result and who reviewed it. That log is your evidence of supervision if a regulator or client asks.

If the tool fails the pass mark, you have lost an afternoon and learned something cheaply. If it passes, you have a number to put in front of the partners instead of a demo.

Two details make the test honest. Sample across fee earners and matter types rather than taking the first 50 rows, because a tool that works on simple conveyancing files may fail on contentious ones. And count the misses as well as the false alarms: a tool that flags every matter will find every leak and waste everyone's week, while one that stays quiet looks accurate and finds nothing. Record both numbers side by side. The SRA says supervision has to cope with the increased speed of AI, so the review rate you set must be one a person can actually sustain.

What pass mark should a billing-review test use?

The partners set the pass mark in writing before the test, as the protocol says. The number is a judgement. What a small sample can and cannot show is arithmetic, and it is worth knowing before you set the mark.

Our arithmetic: if 45 of a sample of 50 flagged matters are correct, the raw result is 90%. A 95% Wilson confidence interval on 45 out of 50 runs from roughly 79% to 96%. So a sample of 50 that reads 90% is compatible with a tool that is right about four times in five. If an error would cost real money or a regulatory problem, either raise the sample size or set a pass mark above the level you can live with at the bottom of that range.

Test misses too. Take a closed set where a manual review has already listed every genuine leak, and count how many of those leaks the tool found. A tool that flags 40 of 50 genuine leaks has found 80% of them, whatever its precision looks like.

What belongs in the review log?

The SRA says supervision systems must cope with the speed of AI, and it expects firms to supervise both the systems and staff use of them. A log is how you show it. Each monthly sample should record:

• Date, tool name and version, and the task tested.
• Sample size, how the sample was drawn and which fee earners and matter types it covered.
• Correct, incorrect and missed counts, with the reviewer's name.
• The pass mark set beforehand and whether the tool met it.
• The decision: continue, restrict to a narrower task, or stop.

Who should own AI billing review in the firm?

The SRA's line is that accountability cannot be delegated to an IT team or external provider. In a small firm the practical owner is the partner who owns billing, with the finance lead running the monthly sample and the compliance officer reading the log each quarter. If those are one person, the log needs a second reader.

What should you ask an AI vendor before a pilot?

Before any client matter data goes into a tool, get written answers to a short list. These are our suggested questions and not a regulatory checklist. Our guide to UK GDPR and AI client data explains why each one matters.

• Where is the data processed and stored, and is any of it used to train the vendor's models?
• How long is the data retained, and can you delete it on request?
• Is the vendor a processor acting on your instructions, and is there a written contract that says so?
• What security measures protect the data, and who at the vendor can see it?
• What happens to the data, and to your logs, if you stop using the tool?

A vendor that cannot answer these in writing has told you what you need to know before the pilot starts.

What about client confidentiality and data protection?

Billing data is client data. The SRA names the threats: a staff member using online AI such as ChatGPT to answer a question on a client's case, and confidential data being revealed when it is transferred to an AI provider for training. Decide which tools are approved before staff decide for you, and read our guide to UK GDPR and AI client data before any matter-level data goes into a new system.

What does it cost, and where do you start?

The SRA observes that there are many sophisticated and expensive AI products aimed at large corporate businesses, and that small firms increasingly use off-the-shelf tools. For billing review, an off-the-shelf tool plus a clear protocol is often enough to test the idea. If you want help designing the test, our AI workflow automation service builds the export, the flags and the review log, and the AI Readiness Audit, from £1,500, tells you which tasks in your firm are worth automating at all. Start with the free Leak Scan if you want to see where the money is leaking first. Our page for UK law firms shows the sector detail, and our insight summary of the SRA report and of the Harvard and BCG study give the short versions.

AI billing review questions firms ask

Does AI improve productivity on routine work?

A National Bureau of Economic Research study of 5,179 customer support agents found a 14% average gain in issues resolved per hour, including 34% for novice and low-skilled workers, with minimal impact on experienced workers. Customer support is not billing, so the figure does not transfer directly to your firm.

What goes wrong when AI does harder tasks?

In a study of 758 BCG consultants, on a complex task designed so that AI would struggle without extensive guidance, consultants using AI were 19 percentage points less likely to produce the right answer.

Can a solicitors' firm delegate responsibility for AI outputs?

No. The SRA says firms remain responsible and accountable for the outputs from AI they use, and cannot delegate accountability to an IT team or external provider.

Can you trust AI to check its own accuracy?

Legal Futures records the SRA warning solicitors not to trust AI to judge its own accuracy. The check has to come from outside the tool, which is why we use a sample test against source records.

How large should a sample test be?

We use 50 flagged items as a minimum for a first test. A result of 45 correct out of 50 is 90%, with a 95% Wilson confidence interval of roughly 79% to 96%, so higher-stakes uses need a larger sample or a higher pass mark.