The ROI of an AI project isn't proven by counting users, prompts or minutes that appear to have been saved. It is proven by comparing a real process before and after, measuring valid outcomes, total cost, adoption, quality and risk. If the organisation has no baseline or comparable group, it should talk about a hypothesis or an early signal, not a demonstrated return.
Start with the business decision
Before choosing a model, define:
- Process and owner.
- Population and volume.
- Specific problem.
- Expected outcome.
- Business metric.
- Quality and risk limits.
- Non-AI alternative.
Example: "cut the time from receiving an invoice to validating it while keeping the error rate below the threshold." That is a better brief than "automate invoices with AI."
Baseline
Over a representative period, measure:
- Volume.
- Time per case.
- Staff and systems cost.
- Errors and rework.
- Wait times.
- Complaints.
- Variability.
- Exception cases.
The baseline must use the same definition as the pilot. If time-to-execute was measured before and review time is added afterwards, the comparison is invalid.
Total cost
| Component | Examples |
|---|---|
| Build | Analysis, development, data and integration. |
| Licensing | Models, platform and connectors. |
| Operations | Inference, storage and observability. |
| People | Review, support, training and governance. |
| Risk | Incidents, errors, privacy and security. |
| Change | Adoption, documentation and processes. |
| Exit | Migration, replacement and shutdown. |
Cost per token is not cost per task. The right unit is cost per valid outcome.
Benefits
Direct
- Hours genuinely freed up.
- Lower external spend.
- Fewer errors.
- Fewer penalties.
- More capacity without adding headcount.
Revenue
- Higher conversion.
- Lower drop-off.
- More sales handled.
- Shorter time to launch.
Risk and quality
- Earlier detection.
- Better traceability.
- Fewer incidents.
- More consistent decisions.
Avoided or probabilistic benefits are kept separate from realised savings.
Useful formulas
ROI = (accumulated net benefit − investment) / investment × 100
Payback period = initial investment / periodic net benefit
Cost per valid task = total cost / tasks accepted without material correction
These formulas are only as good as the assumptions behind them. Time horizon, volume and sensitivity should be published alongside them.
Quality as a gate
A solution can save time and still destroy value through errors. Define metrics such as:
- Accuracy per field or decision.
- False positives and negatives.
- Human corrections.
- Answers without evidence.
- Complaints.
- Outcomes by group.
- Security incidents.
Critical classes are not averaged out. An error that triggers an undue payment can carry zero tolerance.
Designing the pilot
The pilot needs a hypothesis:
If we apply AI to X, then Y will improve by Z, without exceeding limits A and B.
Wherever feasible, compare against a control group or an equivalent period. Avoid cherry-picking only the easy cases. The sample should include exceptions and imperfect inputs.
The pilot runs first in shadow mode, then with assistance, and finally with limited automation.
Adoption and the human factor
Return depends on whether people actually use the tool correctly. Measure:
- Use per eligible task.
- Drop-off.
- Learning time.
- Acceptance without review.
- Corrections.
- Cognitive load.
- Satisfaction.
- Actual redistribution of time.
"Saving 10 minutes" delivers no benefit if that time is lost to interruptions or later review.
Risk-adjusted returns
A mature estimate assigns probability and impact to failure modes. This allows for a risk-adjusted expected benefit and conservative, base and optimistic scenarios.
The NIST AI RMF calls for governing, mapping, measuring and managing risk. ROI must factor in the controls the system actually needs, not be calculated as if it operated without them.
Metrics matrix
| Layer | Metric |
|---|---|
| Business | Margin, capacity, cycle time or conversion. |
| Process | Time, queue, rework. |
| AI | Accuracy, faithfulness, abstention. |
| Human | Adoption, correction and escalation. |
| Operations | Latency, availability and cost. |
| Risk | Incidents, bias and privacy. |
Every metric needs a source, a frequency, an owner and a threshold.
Simplified example
An accounting firm processes 1,000 documents a month. The baseline shows 8 minutes per document and 4% rework. The pilot proposes assisted extraction.
Integration, licensing, review and support are all counted. The benefit isn't "8,000 minutes": review time, exceptions and time never converted into capacity are subtracted. If the error rate falls and the team absorbs more volume without overtime, there is a measurable benefit. If review or complaints increase instead, the gross saving is fictitious.
Deciding to scale
Scale only if:
- It clears the minimum quality bar.
- It improves the business metric.
- Unit cost is sustainable.
- Adoption is sufficient.
- Residual risk is approved.
- Operations and support hold up.
- An exit plan exists.
If not, redesign, limit or cancel it. Stopping a pilot that creates no value is a good decision.
90-day plan
1–30
Process, baseline, hypothesis and data.
31–60
Shadow pilot, costs and evaluation.
61–90
Assisted pilot, adoption, risk and decision.
Common mistakes
- Starting with the tool.
- Having no baseline.
- Counting theoretical time.
- Ignoring human review.
- Measuring only average accuracy.
- Excluding integration and support.
- Using a biased pilot.
- Confusing adoption with value.
- Not pricing in risk.
- Scaling because of sunk cost.
Checklist
- Process and owner.
- Representative baseline.
- Measurable hypothesis.
- Total cost.
- Realised and expected benefits kept separate.
- Quality and risk as gates.
- Comparable pilot.
- Adoption and human load.
- Sensitivity and scenarios.
- Documented decision.
Frequently asked questions
What ROI should be required?
There is no universal threshold. It depends on risk, capital, the alternative and strategy. It should exceed the opportunity cost while maintaining quality.
How should time saved be valued?
Only if it converts into capacity, lower cost, better service or higher-value work. Not every theoretical minute is a financial saving.
Can this be measured before going live?
It can be estimated through a pilot, but it should be called a projection until real operation is observed.
Sources consulted
Summum AI can design the baseline, pilot, evaluation and ROI scorecard without promising savings before they are measured.