At the end of the first quarter, leadership asks: "What has AI actually delivered?" And the engineer who ran the pilot starts flipping through notes. Content was generated faster. Agents handled some of the emails. Reports arrive earlier. But a concrete number — how many hours, how many euros, what the difference is compared to before — is missing. Not because the project failed. But because nobody measured what things looked like beforehand.
This situation is more common in practice than it should be. Most sources suggest that 75–95 % of AI projects do not achieve their planned business value. One of the key reasons is not poor technology, but the absence of a measurable framework: no baseline exists, no metrics are agreed in advance, and when evaluation time arrives, we are comparing something against nothing. This article describes how to do it properly — from defining a baseline through realistic cost items to deciding when a project genuinely makes sense.
Why the baseline is the most important step
Before any AI deployment there is a state you want to improve. That state — the baseline — is the reference value against which you will measure any improvement. Without it you have no ROI. You have a story.
A baseline should capture:
- Time — how many hours per week a specific team spends on the task in question (not estimated, but actually measured or documented from a tracker)
- Error rate — the current rate of mistakes, rework, or escalations
- Cost — labour costs for that activity, plus any external fees (outsourcing, licences)
- Latency — how long it takes to process a request from input to output
The problem is that companies typically do not have these figures ready. Time spent on emails is never logged. Error rates in document processing have never been calculated. It is therefore perfectly acceptable to collect the baseline specifically ahead of the project — even retrospectively over the previous three months. The key requirement is that it must exist before the pilot launches.
For projects where measurement is difficult (for example, an AI assistant for decision-making processes), indirect proxy metrics exist: time from query submission to final decision, number of iterations needed to approve a document, the percentage of cases where the output was directly usable without editing.
What belongs in total cost
This is where most internal business cases distort reality. The presentation includes the cost of API tokens and maybe a software licence. Everything else is glossed over or pushed to later. The true cost of an AI project involves several layers:
Development and integration
- Internal engineering time or an external vendor for development, integration into existing systems, and testing
- Prototyping and experiments that did not make it to production (these are a real cost of the project)
- Data preparation — cleaning, annotation, and structuring of input data
Operations — tokens and infrastructure
For cloud API models these are token costs — for a simple copilot they may be marginal, but for agentic solutions with dozens of calls per day they grow quickly. For on-premises deployments the cost is hardware (GPU server), electricity, and maintenance. Our article on AI agent costs in production covers this calculation in more detail.
People and processes
- Change management — time for training, team adaptation, and workflow changes
- Review and oversight — for outputs that go into production (contracts, reports, emails), someone must check them. This time is systematically underestimated in ROI calculations.
- Adjusting system prompts, monitoring quality, handling outages and regressions
Maintenance and updates
Models change, APIs are updated, and business processes evolve. A deployed AI system is not a finished product — it is a living component that needs regular tuning. A realistic estimate is 15–30 % of original development cost per year in ongoing operating costs.
The good news is that this itemisation forces more precise thinking. We have seen projects where, after breaking down all the components, a copilot for a single operator turned out to be more cost-effective than a large multi-agent system — because total costs were several times lower for a similar measurable benefit.
How to calculate benefits — quantitative and qualitative
Once you have the baseline and the costs, the other side of the equation follows: measurable benefits.
Direct savings
The easiest category to calculate:
- Time saved × hourly rate = labour saving. If an agent replaces 4 hours of manual work per day at €25/hour, the annual saving is ~€26,000.
- Reduced error rate — if AI document checking reduced the error rate from 8 % to 2 %, calculate the cost of correcting each error and multiply by the difference.
- Processing speed — shortening the time from a customer query to a response can be translated into a reduction in escalations or the preservation of business cases.
Indirect benefits
These are real, but harder to measure. They include increased team capacity (people work on higher-value tasks instead of routine ones), higher customer satisfaction, and faster decision-making. Include these benefits in the business case, but do not express them as precise figures — quantify them conservatively or label them as qualitative.
Strategic value
Some projects do not pay off purely on financial terms but carry strategic value: reducing dependence on a specific vendor, regulatory compliance (EU AI Act, GDPR), improving the data infrastructure that has value for other projects as well. These benefits are a legitimate part of the business case — just clearly label them as strategic, not financial.
Payback and ROI — how to read them realistically
Once you have the numbers, the calculation is straightforward:
- ROI (%) = (Total benefits − Total costs) / Total costs × 100
- Payback period = Total costs / Monthly benefits
Where most business cases break down is the time horizon. 84 % of CEOs realistically expect positive payback to take longer than 6 months. For complex use cases (not just a copilot, but agentic workflows, RAG over company documentation, integration with ERP), a realistic horizon is 12–24 months. Pilots with a payback "within three months" are only real for very simple automations with low implementation costs.
A few practical tips when working with the numbers:
- 1.Use conservative estimates for benefits and realistic (not optimistic) estimates for costs
- 2.Present scenarios: best case, base case, worst case — with different adoption assumptions
- 3.Separate one-off costs (development, integration) from recurring ones (tokens, maintenance, people)
- 4.Do not forget the ramp-up — the first months of deployment are typically less efficient while the team adapts
A related overview on why AI pilots fail shows that the absence of measurable goals before launch is one of the most common reasons for failure.
When a project has no business case
An honest part of any ROI framework is also the decision of when an AI deployment does not make sense. We have seen projects where this decision was not made in time — and the result was six months of spent effort with zero output.
Signals that a project lacks a sufficient business case:
- Too small a scale — if the activity takes the team fewer than 5 hours per week, deployment and maintenance costs rarely reach payback within a reasonable horizon
- No data exists — an AI system without quality input data does not produce usable outputs. If the data is not there, invest in data infrastructure first
- The process is not defined — AI automates a process, not chaos. If people cannot describe the steps of how they handle the task manually, AI will not solve it for them
- Oversight requirements are too high — if every AI output still requires equally detailed review as before the AI was deployed, the effective benefit is close to zero
Rejecting a project based on ROI analysis is not failure — it is saved time and money that can be directed to where a real case exists.
Soft benefits — how to include them in a business case
The "soft benefits" category is used in presentations to paper over a weak quantitative story. That does not mean they do not exist — they do, but they must be named differently.
Instead of "improved customer satisfaction," write: "from a pre-deployment survey: 62 % of customers rated response time as too slow; after deploying the copilot, average response time dropped from 4.2 hours to 47 minutes." That is measurable — even if it is a proxy metric rather than a direct financial figure.
Instead of "greater team efficiency," write: "before deployment the team spent an average of 12 hours per week preparing status reports; after deployment, 2.5 hours." If you cannot convert this to euros, state it as a factual capacity saving — not as a financial benefit.
Benefits that are genuinely only a feeling (better morale, a more modern company image) belong in the strategic section, with no numerical value attached.
Tracking ROI after deployment — why this is different from a business case
A business case is assembled before the project. Tracking ROI after deployment is a different discipline — and most companies do it only superficially.
A production AI system should have live monitoring metrics that are evaluated regularly:
- Volume of tasks processed (and the trend)
- Output acceptance rate without human editing
- Error or escalation rate
- Actual time spent on oversight (vs. the original assumption)
These data serve two purposes: first, they confirm or refute the assumptions from the business case; second, they show where the system is degrading — the model grows stale, inputs change, drift from the original use case grows. Observability of AI systems is a topic in its own right; the foundation is logging every input, output, and agent decision.
For companies considering multiple AI projects in parallel, tracking ROI at the portfolio level is essential: not just which project has the highest ROI, but also which projects are consuming team capacity without measurable benefit.
Frequently asked questions
How do you define a baseline when you have no historical data?
Collect data before the pilot launches — even 4–6 weeks is enough for a sufficient sample for most processes. If that is not possible, use structured interviews with team members: "How many hours per week do you spend on this task? How many cases do you handle per month?" Combine this with existing system logs (ticketing system, email metadata, ERP records). An estimate with explicitly named uncertainty is better than no baseline at all.
What is a realistic payback horizon for an AI project in an industrial company?
For a simple copilot (documentation assistant, emails, reporting): 6–12 months. For RAG over company documentation or predictive analytics: 12–18 months. For a complex agentic system integrated with ERP/SCADA: 18–36 months. The "under 6 months" shortcut only applies with low implementation costs and large scale — for example when a single agent replaces hundreds of manual operations per day.
How should fine-tuning or RAG infrastructure costs be included in ROI?
Yes — and they should be stated separately from operating costs. Fine-tuning is a one-off cost (dataset preparation, training time, evaluation), but the model needs periodic updates — plan for annual repetition. RAG infrastructure (vector database, embedding pipeline, retrieval tuning) is a fixed cost with a low variable component. Both are investments in foundations that can serve multiple use cases — spread them across the projects that use them.
What should you do if the pilot showed a benefit but production deployment does not replicate it?
This is a common scenario — most pilots run on curated inputs and controlled conditions. First question: what is the acceptance rate of real outputs (without editing)? If it has dropped significantly compared to the pilot, the problem is either a distributional shift in input data, or the pilot scope did not represent actual production variability. Solution: analyse failure cases, expand the training sample, or narrow scope to the subset of cases where the system performs reliably.
When does it make sense to run a build-vs-buy analysis before the ROI calculation?
Always. ROI depends on whether you build the system internally, buy a ready-made solution, or combine the two. Each option has a different cost profile and a different payback. Build vs. buy is covered in more detail in a dedicated article on that choice — we recommend it as a prerequisite before finalising any business case.
*If you are working on a business case for an AI project and need help setting up metrics, a baseline, or a cost model — this is exactly what we help industrial clients with before they begin development. Contact us for a free consultation.*