AI reality check: Here's what three companies learned building wallets, homes, and games

Executives from Citi, Home Depot, and Capcom describe early work with AI agents

While AI agents have moved from experimental tools to customer-facing workers in a matter of months, the next challenge is governance and reliability once those agents touch real money, real shoppers, and real creative output.

That's according to executives from Citi, Home Depot, and Capcom, who wspoke to media during a panel at Google Cloud Next last week about how they use AI agents and manage them in real-world environments.

As part of banking giant's annual $11 billion technology spend, Citi introduced a client-facing agent called Citi Sky, which Andy Sieg, the bank's head of wealth, hyped as a new channel for customer interaction on par with the introduction of the ATM.

He said Citi Sky can answer questions and act on them. It operates in voice and video and aims to garner the kind of trust Citi clients place in human advisors.

However, on the business side, Citi Sky gives the bank an auditable digital record of every customer interaction, as well as the ability to reach out to existing customers with the hope of capturing more of their spend. While the bank manages about $1 trillion in wealth for customers, those customers have $5 trillion with other banks. Citi hopes to convert that wealth and it sees agents as the way to do that.

“For us, this is about growth,” Sieg said. “If we ask ourselves, how many additional people would we need to hire to deepen relationships and to see more of that $5 trillion come to us it would be many, many thousands. Introducing Citi Sky gives us unlimited capacity to do things we need to do to deepen relationships with clients.”

As an example, he said that Citi Sky can take market moving announcements from Washington DC and deliver a simultaneous portfolio review to all of their clients based on unfolding events.

“Before we had Citi Sky, we did the best we could getting on the phone or by email,” he said.

Asked how Citi handles governance and hallucinations and mistakes, Sieg said security and accuracy topped the project's requirements list from day one. He said Citi Sky does not run on an off-the-shelf model. The bank has layered its own safeguards, wealth-management expertise, and compliance controls over Google's stack, and treats the agent as though it were an employee.

"Trust is the currency of wealth management," Sieg said. "We're operating as if Citi Sky is an employee, subject to the same securities laws, rules and regulations as everyone else."

With a single agent interface, Citi can standardize and audit every interaction, a governance tool that Citi has not previously had in its kit. He contrasted that model with existing human to human workflows.

"Every conversation that we have today is not auditable and recorded, and can vary from advisor to advisor," Sieg said.

Home Depot executive vice president and CIO Angie Brown said that the retailer is extending its agent-driven experience from the website into stores and phone calls with the company's Magic Apron tool, which packages decades of home improvement expertise into a conversational format. It now runs alongside an in-store pilot and a Google-powered phone system that replaces the traditional interactive voice response menu.

But beneath that top layer, Brown said that Home Depot has been methodical about designing a system for the AI agent era.

"From a data perspective, we have a lot of our information in BigQuery, and then what we're doing on top of that is we have Gemini enterprise and the ADK framework in there as well to help us build these agents," she said. "We use things like (Google Enterprise for Customer Experience) at the top layer to also expose the logic unit that's happening underneath there."

Brown said that the agents deliver consistency across all of the channels the company uses to reach customers. The same framework that powers Magic Apron on the website is also used by the in-store pilot and on the phone system. So if a customer asks about deck materials online, they receive the same guidance when they call or visit a store. She said Magic Apron is also being deployed to the store's professional channel to help contractors with job pricing.

That portability, Brown said, lets the retailer hold one bar for quality regardless of the customer's entry point.

"It's about understanding what the customer is trying to accomplish, the problem that they are trying to solve, and then meeting them in the moment with these technology tools that help them get to that answer," Brown said.

She said that, as Home Depot measured agents against conventional business outcomes, it found that customers who engage with the shopping agent convert at a higher rate. She said that the new phone system resolves customer questions roughly four times faster than the previous system.

Shinichi Inoue, vice president of game development platform and AI solutions at Capcom said that the company's 3,700 employees - about 75 percent of whom work in development - now save roughly 30,000 hours per month per project by handing repetitive work to AI agents, particularly in the work that goes into testing which consumes “enormous time” on modern titles.

"Testing has to be done just as if people are looking at the games, or listening to the games, or playing the games," the Capcom executive said. “We use AI agents to do those tasks.”

AI agents now perform much of that work, freeing human creators to focus on design rather than regression passes.

Inoue said that has led to an unlocking of creative value at Capcom.

"We can say we were able to become efficient, or we were able to reduce so and so, but rather, we paid more attention to the fact that because of those factors, the creators became more focused on their creativity, and that part is the significant value to us," Inoue said. ®