Retail

AI, 3D avatars revolutionize customer service, drive-thru

AI-powered chat and 3D avatars are revolutionizing retail environments, designed to enhance customer experience while freeing frontline workers for more engaging work and reducing costs for operators in various industries and verticals.

SapientX 3D AI character Fern was developed especially to help with wayfinding. Provided by SapientX.

January 10, 2023 by Daniel Brown — Editor, Networld Media Group

As reported by Digital Signage Today, hologram use in communication and advertising has been growing, from the Proto holoportation device to testing in retail environments of 3D anamorphic AI-based assistants, designed to enhance customer experience while freeing frontline workers for more engaging work and reducing costs for operators in various industries and verticals. AI based assistants (including audio-only and 3D avatar characters) have already started changing self-service kiosks in various industries, including wayfinding. We reached out to a group of experts and innovators to learn just how soon the revolution will come, and the answer surprised us.

Sooner than you think

Pete Erickson is the founder of Modev, a consortium of industry experts and companies working to develop AI and applications across various industries and verticals, including a series of events. Erickson said the revolution is already well underway, including development in voice-only along with 3D character-based assistants.

"Well, I would say, if you go through a drive thru in 2025, most the time you're going to be talking to an assistant. If you go to a retail store in 2025, if that retail store doesn't have an assistant on most every aisle to help you there, they're way behind. Yeah, so I think we're not that far off," Erickson said.

"Most cars, every car has an assistant now. And we may see the arrival of new assistants that are going to go in homes based on large language models that are quite different from what we have today. And we'll look back one day, and we'll see (that) the original fleet of Alexa's that made it out to homes were, you know, flip phones compared to smartphones.

"And so, we're probably two to three years away from that. But that's going to happen. I mean, it's all happening... The metaverse isn't one thing, but it there's going to be metaverse-ish type things happening in so many different ways. Digital twins are going to become commonplace. You know, you can go to an open house and see a digital twin of the house and go through the whole thing and be in it and experience it."

The rise of 3D avatars

David Colleen, CEO at SapientX provided more information on how AI-based 3D avatars are transforming industries. Colleen said his firm has provided 3D characters for events, wayfinding at Edmonton Airport in Canada and customer service at a major retail chain (under NDA).

Colleen is no stranger to 3D and innovation. After studying architecture at Cornell, he enjoyed a colorful career, pioneering some of the earliest use of 3D modeling for architecture, which led to an exciting opportunity when good friend Barb Singer announced she was founding a company to do 3D plugin for Netscape, the then-dominant browser. "And together we put up the very first 3d content on the internet," Colleen said, reminiscing on deep research on AI with luminaries like Selmer Bringsjord at Rensselaer Polytechnic University.

Colleen's decades of expertise have given him a realistic stance on how close we are to sentient AI, despite a slew of Hollywood blockbusters: "So I think we're nowhere near a strong intelligence," he said. "We worked with Selmer Bringsjord at Rensselaer, who was one of the leading people in synthetic logic systems. And our work with Selmer was fascinating. But getting machines to think logically and perform logically is, is a very, very deep task. The machine learning systems that are in the news so much today — they're exciting, but you have to think of them as Myna birds. Even systems like GPT3 only repeats stuff that it hears, so to speak, on the internet. And it it really doesn't apply any intelligence to the use of what it hears."

Real world applications are diverse and growing. "One of our very first prototypes was for Lowe's," Colleen said. "So our characters, because they're really good at understanding these things, and they can memorize everything in a store, they can perform two use cases for Lowes. One is to greet people and give a friendly face to to the experience. The second is to answer any kind of question about where things are. Turns out that's super useful. Second area that we're working on is for the Edmonton airport," he added, with Edmonton's avatar answering the questions traditionally reserved for guest information desks (before the recent labor shortages impacted those desks).

When contacted by Digital Signage Today for comment on the Edmonton Airport use case, Ken Brizel, CEO at ACAMP, said the following: "I am working with SapientX on the Avatar capability and it's a fantastic opportunity for information services on signage displays. For example the Edmonton Airport is one of many opportunities for information systems. It supports navigating the airport for food, buying gifts, finding terminals, finding hotels, etc. I am also looking into hospitals, doctors offices and convention centers."

Colleen also says that the technology could be used to duplicate celebrity likenesses (under contract) to add star power to the process.

Hybrid approaches reduce errors

Part of the power of SapientX's approach is leveraging multiple strategies, instead of just relying on machine learning alone (which can be a limiting factor in bare-bones GPT3).

"Our technology is a blend of of different AI technologies," Colleen said. "Think of AI is like a Swiss army knife where each blade is a different discipline of AI."

Another piece of the puzzle is that customers simply respond more powerfully to visual characters than they do to voice only. "Well, when we when we started working on this, we intuitively knew that people would respond better to talk than to a physically, visually defined entity rather than a disembodied voice. We just knew that in our guts and we knew there were a lot of naysayers about that. But what's happened over the last decade is there's been a lot of fundamental research that has shown greatly elevated user satisfaction and trust scores, amongst many other things as much as 25% improvements in those categories, if you have a good avatar," Colleen explained, citing the old adage that about 70% of communication is nonverbal.

Balancing the strengths and weaknesses of machine learning with tools like symbolic reasoning fuels SapientX's hybrid approach. "So what works much more accurately is symbolic reasoning, which is kind of a rule based system on steroids. It uses pattern and concept recognition to augment traditional rule based systems, which are out of fashion right now. But we don't care about fashion," he added. The hybrid approach helps in situations where machine learning breaks down, like when the AI can't figure out the answer to a question based on known sources (in which case it is required to improvise a response). This also helps the AI self-police for offensive content, which is something raw machine learning systems can be poor at.

User impressions

When contacted for comments about the reception of AI characters at the Augmented World Expo, Dave Lorenzini, co-founder of the AWE and director at Draw & Code, said the following: "Highly available, infinitely knowledgeable and visually interesting virtual assistants like Zander from SapientX are a great way to inform, engage and entertain attendees. I wouldn't say virtual assistants are better than humans at this point, but the gap is closing. We used SapientX's Zander assistant, who knew every detail of our event, could answer any question in any language, was entertaining as he was informative, and never took a break. Score one for AI."

Optimism over fear

While media headlines are sometimes rife with negative coverage of AI, David Colleen finds AI (and technology in general) to be a sign of humanity's ability to improvise and solve challenges creatively.

"Okay, so first of all, this is not Skynet. The press in general likes to talk about scary AI things to feed on people's fears, to, to sell newspapers, so to speak. So it's I get that that's that's happened as long as there's been a press. But the truth is, is we're just so far away from a system that can willfully be evil and in orchestrating the demise of humankind. We're not even 1% of 1% down that road. And there's a lot of really great, talented, highly ethical people working on these things. So it's it's a steady part of our conversation as we're working on AI systems to do the right thing rather than the wrong thing."

Erickson concurred. "You know, I saw a demonstration of somebody with epilepsy walking and shaking, and then they put on an AI driven exoskeleton, and they're able to just walk super smoothly because the AI is sending the signals for you know, smooth walking..." Erickson said. "The world's got a lot of problems, and a lot of challenges, and we have to solve them, and AI is going to help us solve them. So even as we talk about in-store capabilities, the most exciting ones, and probably the most compelling use cases, haven't been developed yet."

About Daniel Brown

---

Daniel Brown is the editor of Digital Signage Today, a contributing editor for Automation & Self-Service, and an accomplished writer and multimedia content producer with extensive experience covering technology and business. His work has appeared in a range of business and technology publications, including interviews with eminent business leaders, inventors and technologists. He has written extensively on AI and the integration of technology and business strategy with empathy and the human touch. Brown is the author of two novels and a podcaster. His previous experience includes IT work at an Ivy League research institution, education and business consulting, and retail sales and management.