Balancing the Books: Gen AI's Economic Challenges

As generative AI expands in enterprises, McKinsey highlights financial sustainability over technological aspects as key. Transitioning from experimentation to ROI-focused models, leaders face economic complexities in scaling AI agents. Operational costs, influenced by context and advanced reasoning, challenge tracking measurable business value.

As generative artificial intelligence graduates from experimentation to widespread enterprise deployment, financial considerations are taking precedence over mere technological advancements, according to a new McKinsey report.

The study highlights that the initial phase of Gen AI was characterized by an emphasis on access, experimentation, and deployment. Now, the focus shifts towards ensuring financial sustainability and return on investment as businesses scale their AI systems. With the technology maturing, the decision to scale AI agents is increasingly driven by complex economic factors rather than technical challenges.

Enterprises are refocusing from merely cutting AI costs to showcasing tangible business value, with chief financial and information officers calling for concrete evidence of returns from AI investments. The report also underscores a significant gap: many companies lack frameworks to effectively measure the business impact of AI-driven decisions.

A crucial insight from the report is that nearly 60% of operating costs for agentic AI are devoted to refining and verifying responses. It identifies six primary factors contributing to these expenditures, with the enduring context being a major cost contributor.

Agentic AI tasks utilize up to a thousand times more tokens than traditional AI tasks, rendering per-token pricing obsolete. Refining AI-generated answers is particularly costly, representing about 60% of the expense of agentic tasks.

Furthermore, costs vary due to factors such as tool usage, reasoning paths, and trials, particularly when advanced reasoning applies to simple tasks. McKinsey notes that agent orchestration, the coordination of AI agents with tools and models, significantly impacts costs. Effective task management can substantially lower expenses without compromising business results.

The report also points out the importance of information structure in operating costs, emphasizing elements such as prompt design and language. Non-English text tends to consume more tokens per meaning, thus increasing costs for some languages.