Sorry, but the ROI on enterprise AI is abysmal

Appen points to, among other problems, a lack of high-quality training data labeled by humans

The deployment of AI projects and associated return on investment (ROI) have declined, according to a large survey of IT decision-makers.

Appen, an AI data services company, working with The Harris Poll queried 500 IT decision-makers across various US industries for its report, The 2024 State of AI.

The results indicate that despite growing enthusiasm for AI tools for marketing, communications and manufacturing – up 17 percent from 2023 – enterprise deployments and overall ROI are down.

According to Appen, the mean percent of AI projects that get deployed has fallen from 55.5 percent in 2021 to 47.4 percent in 2024, a decline of 8 percentage points. And the mean percent of deployed AI projects that have shown significant ROI has slipped from 56.7 percent to 47.3 percent.

The reason for this, Appen argues, is lack of high-quality training data, labeled by humans.

"By incorporating expert-labeled training data and rigorous evaluation processes, enterprises can better align their models with real-world needs, enhancing accuracy and relevance," the report says. "This results in models that are not only more capable but also more likely to reach deployment and deliver meaningful ROI."

Coincidentally, Appen happens to offer a data annotation platform that relies on crowdsourced labor and AI.

Appen's survey results, which include other findings about the need for data quality that align with its business, echo other findings about the elusiveness of ROI for AI.

A Gartner report issued in May found that generative AI tools had become the most commonly deployed variety of AI in the workplace while also noting that businesses have concerns about payback.

"The primary obstacle to AI adoption, as reported by 49% of survey participants, is the difficulty in estimating and demonstrating the value of AI projects," Gartner said. "This issue surpasses other barriers such as talent shortages, technical difficulties, data-related problems, lack of business alignment and trust in AI."

Financial firm Goldman Sachs also raised the issue in June when it noted in a report [PDF] that despite estimated $1 trillion in pending capital expenditure commitments, AI hasn't demonstrated its value to businesses.

We noted at the time that individual success stories like Clearview Consulting Group can be found, but clearly the path to make AI pay isn't clear to many organizations.

Professional services firm EY sounds a bit more bullish about the AI payoff. The firm in July said about a third of senior leaders report that their organizations are overseeing broad AI initiatives. Among those organizations, "investments are delivering positive returns, especially in areas like operational efficiencies (77 percent), employee productivity (74 percent) and customer satisfaction (72 percent)." ®