Soul Machines, a New Zealand-based company that uses CGI, AI and natural language processing to create lifelike digital humans that can interact with humans in real time, has raised $70 million in a Series B1 round, bringing total funding to $70 million. 135 million coming. The startup will put the money into improving its Digital Brain technology, which uses a technique called “cognitive modeling” to recreate things like the human brain’s emotional response system to build autonomous animated characters.
Funding was led by new investor SoftBank Vision Fund 2, with additional participation from Cleveland Avenue, Liberty City Ventures and Solasta Ventures. Existing investors Temasek, Salesforce Ventures and Horizons Ventures also participated in the round.
While Soul Machines envisions that its technology will be used for entertainment purposes, it primarily pursues a B2B game that creates emotionally engaging brand and customer experiences. The basic problem the startup is trying to solve is how to create personalized brand experiences in an increasingly digital world, especially when the main interaction most companies have with their customers is through apps and websites.
The answer, Soul Machines believes, is a digital workforce, available any time of the day, in any language, that mimics the human experience so well that people get an emotional response, ultimately leading to brand loyalty.
“It’s like talking to a digital seller,” Greg Cross, co-founder and chief business officer, told TechCrunch. “So, for example, you can buy skin care products in an e-commerce store and have the opportunity to talk to a digital skin care consultant as part of the experience. One of the most important things we have discovered, particularly during the COVID era, is more of our shopping and the way we experience brands happens in a digital world. Traditionally, a digital world is very transactional. Even chatbots are quite transactional – you type in a question, you get an answer. What drives us as a company is to think about how we envision that human interaction with all the digital worlds of the future?”
It’s worth noting that Soul Machines’ other co-founder Mark Sagar has won Academy Awards for his AI engineering efforts that created the characters in the “Avatar” and “King Kong” movies. Perhaps the skill behind producing such realistic digital people is why Soul Machines reported a 4.6x higher conversion rate, a 2.3% increase in customer satisfaction and that customers are two times more likely to buy after interacting with one of the the company’s products, Yumi, a digital skincare specialist for SK II, a P&G brand.
The startup has partnered with brands such as Nestle Tollhouse to create Ruth, an AI-powered cookie coach who can answer basic cookie baking questions and help customers find recipes based on what they have in their kitchen. Soul Machines has also partnered with the World Health Organization to create Florence, a virtual health worker available 24/7 to provide digital counseling services to people trying to quit smoking or learning more about COVID-19. There’s also Viola, who lives on the company’s website as an example of a digital assistant who can answer questions and interact with content, such as YouTube videos or maps, that she pulls out.
“Soul Machines’ Digital People solution has been particularly well received in the service industry where companies want to improve their online customer service experience beyond a text-based chat or audio-only call who typically have long wait times to talk to a live person. Anna Lo, investment director at SoftBank Investment Advisers, told TechCrunch. “With autonomous animation, the custom persona Soul Machines produces is also a useful tool for recruiting customers for new product and service inquiries.”
Lo also pointed to potential applications in the telehealth sector where patients prefer a live video experience. Digital people could help provide patients with a level of privacy and comfort to ask sensitive questions in a way that gives doctors the freedom to handle more hands-on medical situations, Lo said.
To consumers, many digital assistants can feel more like a gimmick than a useful tool. But with these assistants, companies can collect first-party data about their customers, which can be used to acquire and retain customers and add more value, instead of having to spend huge sums of money to buy that data from social networks. media platforms or Google AdWords. said Cross.
While Soul Machines has a clear overview of ways it can improve the future of the customer experience, it still has a long way to go to get the technology where it needs to be. The digital people (or shall we say digital) Ladies, because Soul Machines clearly subscribes to the woman-as-a-servant philosophy) currently on the market feel like visual chatbots. They only seem to be able to answer scripted questions or questions formulated in a specific way, and they have a few answers that they recycle.
For example, Viola introduces itself to the user by saying, “I’m here to help you explore the world. Ask me a Who, What, Where or Why question, and let’s see where we’re going.”
You can ask Viola what she is, what Soul Machines is, and some other random questions that can be taken from online encyclopedias like, “Where’s New Zealand?” or “What’s the Big Bang?” I asked her what cognitive modeling and deep learning were, and she said, “Sorry, I don’t know what that is.”
If the user asks a question that is not easy to answer, Viola often offers a standard deflection like any standard chatbot. Or she will respond in ways that are surprising, if certainly not intended. For example, I asked Viola, “Why do you look sad?” She responded by pulling up a YouTube video of “I’ll Stand by You” by The Pretenders. Not exactly the answer I was looking for, but at least Viola seems to interact with the content she brings out by looking at and gesturing at it. This suggests Viola is aware of the content in her digital world, Cross said.
Florence and Ruth were similarly limited to formulating questions in a way that they had been trained to understand and respond, and to answer questions that fell within the confines of their operational design domains. For her part, Florence had quite a face-imitating function. When I smiled at her, she smiled back, and it was a beautiful, genuine-looking smile that really endeared me to her.
As customers interact with one of Soul Machine’s digital people, information about their facial expressions and the way they react emotionally is collected, anonymized, and used to train the digital brain so it can interpret those responses and provide an appropriate response.
In order to measure the progress in autonomous animation, Soul Machines has written a white paper that proposes a framework made up of five-like levels – there is a level 0, which is “No Autonomy” – only a recorded animation, like a cartoon .
Levels 1 and 2 feature pre-recorded and human-written animations (think how animated characters mimicked the movements of real actors in movies like “Avatar” or “The Lord of the Rings”). Levels 3 through 5 include real-time, dynamically generated, content-aware animation. Soul Machines currently places itself at level 3, or “Guided Animation”, which it defines as a “Cognitively Trained Animation (CTA) system [that] uses algorithms to generate a series of animations without the need for explicit authoring. Authors are evolving into trainers focused solely on defining the scope of content and role. The system informs the trainers about points for improvement.”
Soul Machines is working on level 4 autonomy, or “Goals-Based Animation,” where the CTA system dynamically generates new animations to help it achieve the goals set by the teacher, Cross said. The system tries out new interactions and learns from each interaction under the guidance of a trainer. An example of this could be a virtual assistant advising clients on complex financial situations and creating new behaviors on the fly, but those behaviors would all be consistent with the branding and marketing goals provided by the company.
Or it could be a company that uses a digital version of a celebrity brand ambassador to answer questions about its products in a digital showroom. Soul Machines recently announced its intention to build a series of celebrity digital twins. Last year, the company began working with basketball player Carmelo Anthony, of the Los Angeles Lakers, to create a digital likeness of him, something it has done before with rapper Will.I.am, who was featured in a 2019 episode of “The Age of AI,” a YouTube Originals series hosted by actor Robert Downey Jr.
Anthony is already a Nike ambassador, so in theory Soul Machines could use his likeness to create an experience that may only be available to VIP customers who own a set of NFTs to unlock that experience, Cross said. That digital Anthony might also be able to speak Mandarin or another language with his own voice, which would open up brands to new audiences.
“We’re really gearing up for this next big step from today’s 2D Internet world, which I believe will still be our driveway, into the 3D world metaverse where digital humans need to be fully animated,” Cross said.
Soul Machines currently has prototypes of digital humans who can communicate with each other by emotionally responding and answering questions they have asked each other, Cross said. The co-founder thinks this will first be applied to metaverse spaces such as a digital fashion store inhabited by multiple people, some of whom are digital humans and some avatars controlled by humans.
In the future, Soul Machines envisions a world where people can create digital replicas of themselves.
“We’re well on our way to creating these hyper-realistic digital twins of ourselves at some point in the future and being able to train them just by interacting with them online,” Cross said. “And then to be able to send them to the metaverse to work for us while we’re playing golf or laying on the beach. that is a version of the world of the future.”
This article has been updated with a quote from Softbank.