AI in the workplace: Why measuring productivity is harder than you think

Reinventing workplace productivity in the AI-assisted organisation

Can generative AI’s nascent promise move beyond early productivity gains to something more tangible for the CIO and CFO – and more impactful in the workplace?

In the rush to roll-out generative AI, organisations find themselves at the crossroads between market excitement and business value.

The promise of AI is immense, but the path towards successful implementation can be fraught with challenges which move beyond technology - to people and process.

Indeed, as businesses grapple with deploying these technologies, the key to success arguably lies not just in the tools themselves, but in how leaders manage the human element of change; such as how to placate fears over job displacement, encourage ‘safe’ experimentation, discourage reckless usage and upskill staff with different competency and confidence levels.

During this online C-Suite Exchange, hosted by HotTopics in partnership with Box as part of the Infinite Intelligence community, senior technology executives - speaking under Chatham House Rule - discussed productivity in the AI-assisted workplace, the value of training and the quest for measurable business value.

AI in the workplace: The productivity paradox explained

Generative AI and agentic AI have the power to reimagine work and staff productivity.

Industry observers argue that generative AI allows staff to focus on ‘higher-value tasks’, and that the advent of agentic will automate laborious tasks at scale - promising news for CIOs that have traditionally focused more on efficiency than productivity. 

The potential for AI to expedite personal productivity is huge; McKinsey research suggests that AI productivity tools could improve labour productivity by 0.1 percent to 0.6 percent through 2040, and that such gains could be worth as much as $4.4 trillion to the global economy.

Despite this, early adopters have discovered that measuring AI productivity is far from straightforward. It remains too early to calculate financial returns - some of which have already been called into question - leading some technology executives to explore softer, ‘fuzzy’, intangible outcomes which are unlikely to satisfy the CEO or CFO. 

A CIO from a non-profit remarked that quantifying the exact value of AI in the workplace remains somewhat elusive. 

"I couldn't tell you how much money I've saved," he admitted of the organisation’s Microsoft Copilot usage, highlighting a challenge faced by many organisations. Another technology executive, working in financial services, pointed to AI ROI being calculated against employee time sheets and research surveys sent out to teams every week.

Such ambiguity on AI’s value is not so much a deterrent but rather an invitation for CIOs and CTOs to approach AI adoption with a strategic - if cautious - intentionality. A transformation director at a FMCG firm offers a compelling framework whereby AI is not a free for all - but rather use-case driven: 

"We really need to ensure that we have very targeted investments into AI if we really want to shift the needle." 

The key, she said, is not widespread, unfocused implementation, but carefully selected, scalable use cases which can demonstrate (pre-defined) value, thereby winning the confidence of the C-suite and business and crucially, allowing for further experimentation and investment.

Related content: GenAI adoption strategies

AI productivity and workflow automation: How agentic is shaping the future of work

As most CIOs attest, the introduction of any new technology brings barriers which are more cultural than technological. This is arguably exacerbated by a set of AI technologies which have long been portrayed in popular culture as, at best, job displacement tools and, at worst - somewhat dramatically – as the final days of humanity.

There remains a pervasive fear that AI will replace human workers, a narrative that creates resistance, anxiety and inertia to the change CIOs and CTOs are trying to drive through digital transformation programmes.

Forward-looking leaders are reframing this perspective, however. As one participant said, AI is about creating "a world of hyper-personalisation" where humans can focus on more strategic, creative work. And it’s here that agentic AI – systems which can autonomously make decisions on a user’s behalf, with a human still ‘in the loop’ – is already turning the dial.

In one such example, an organisation implemented AI agents for pre-registering hospital patients for procedures like MRIs, thereby reducing the processing time from 15-30 minutes to 30 seconds. At this hospital, agentic currently handles 40% of authorisations automatically. Agentic AI bots, with human staff reviewing decisions, are also expediting contract reviews, reducing the contract review time from eight days to 56 seconds while also reducing the error rate from 23% to 1.5%

Separately, at the sales department of one attendee, AI agents have been conducting sales performance audits, reviewing conversations against over 200 parameters and freeing up staff to focus on more valuable activities.

To scale these early use cases of AI in the workplace, CIOs and CTOs must take a top-down approach to change management, addressing some of these cultural concerns head-on. Organisations must:

1. Communicate a clear vision of AI as a powerful collaboration tool
2. Provide comprehensive training and skill development
3. Create a sense of security and opportunity, not threat

Related content: The C-suite's guide to AI adoption

The CIO’s balancing act: AI innovation or governance?

With the continued pace of AI adoption comes the need for robust governance. One attendee, a Chief Data Officer at a research and development institution, emphasised the importance of guidelines over restrictive policies. 

"We haven't gone with policy because I think policy can be a little bit too draconian," she explained, emphasising the danger of IP being leaked outside the company via publicly available general purpose Large Language Models (LLMs).

“The goal is to create a framework that empowers innovation while protecting intellectual property and maintaining security.”

Getting this balance right continues to be a challenge for most technology executives, caught between enabling AI innovation and governance.

“How do you make sure that you're providing value? How do you make sure that this thing doesn't get away from you?” asked Ravi Malick, CIO at Box. “Part of that conversation is; what's the right level of governance? 

The skills revolution: The quest for AI literacy

In the face of an AI market which evolves each passing day, the skills landscape is also moving quickly. 

Organisations, for the most part, know that AI is not about replacing workers, but about elevating their capabilities and output. 

One speaker, from the insurance sector, argued for creating AI ‘champions’ within the organisation who can help identify and implement use cases across different departments - and drive wider AI literacy in the workplace.

The C-Suite should be factored into such discussions. Two of the speakers suggested that while GenAI tools were first offered to C-suite leaders - in the hope of driving buy-in and establishing the necessary use cases. However, these leaders were so ‘busy running the business’ that they weren’t able to identify the appropriate use cases.

“How do you educate the workforce? How do you drive the change? How do you prepare people for the change and the transition, to really rethink how they get work done and what their area of the business looks like two, three, five years from now?,” said Malick.

Practical generative and agentic AI implementation strategies

Drawing from the collective wisdom of leaders in attendance, here are key strategies for AI workforce adoption in the drive to improve employee efficiency and productivity:

• Start small: Begin with targeted use cases that can demonstrate clear value
• Invest in training: Develop comprehensive programmes which build AI literacy
• Create cross-functional teams: Ensure AI initiatives are not departmental siloes
• Establish clear governance: Develop guardrails (policies or guidelines, risk assessments and industry frameworks) which help guide responsible innovation
• Continuous learning: Treat AI workforce adoption as an ongoing journey of adaptation

Summary: How technology executives embrace continuous AI innovation

Industry leaders say we are entering the new era of the "agentic workforce" - where AI becomes an integrated partner to employees' work processes - making decisions on their behalf to improve productivity and efficiency.

Such change doesn’t come without challenges; technology executives need to consider the business value, the governance and skillsets as discussed above - but also how AI works within existing business workflows, the reliability of their structured and unstructured data and how it integrates with their existing tech stack. Privacy and security continue to be pressing concerns, especially with publicly-available general purpose models and the reemergence of shadow IT.

Box’s Malick offers sage advice for starting out with artificial intelligence in the workplace:

"Start small. Find the use cases. Don't try to boil the ocean. Think about how you architect the technology to take advantage of what's moving fast in the industry."

Unlock AI productivity with Box’s secure, enterprise-grade AI solutions. Transform your unstructured data into business value—safely and at scale.