Beem, an app that lets you live stream yourself in AR, raises $4 million – TechCrunch

Startups

What’s next next to Zoom calls and FaceTime? What about beaming yourself from one device to another in real time using augmented reality? That’s the premise behind a startup called Beem, today announcing its first consumer app, $4 million in seed funding, and its longer-term plan to become a communications technology for the AR glasses of the future.

Today, consumers are familiar with AR technology through interactions with Snapchat lenses and TikTok effects, through mobile games such as Pokémon Go, and by visualizing products they are considering buying, such as furniture they place in their room, or makeup trying them virtually through an AR filter. But using AR for telepresence, as Beem intends, isn’t common.

Surprisingly, the idea for the startup comes from a founder who grew up without much access to technology. Beem CEO Janosch Amstutz describes his parents as “hippies” who lived in a small Australian beach town, Byron Bay, where they raised cows and chickens, used solar energy, collected rainwater and didn’t even have a telephone – just a two-way radio that the community would share.

“We put a lot of emphasis on face-to-face communication,” he explains of the lack of modern technology in his alternate childhood.

Meanwhile, communication continued to evolve from home phones to cell phones to Skype. “But then we kind of stopped,” Amstutz says. “Beem’s premise is that there will be an inevitable next step in the way people communicate digitally, which is more credible and immersive than a video call.”

Amstutz initially rebelled against his upbringing by working in the physical goods trade in the steel industry. But he did not find the work satisfactory. Finally, he realized he wanted to do something different – and specifically, he wanted to tackle the problem of evolving modern communication.

The founder assembled a team of computer scientists and researchers to work on the concept, today including Chief of Product Damian Hickey, formerly head of product at AR pioneer Blippar; and Denis Islamov, co-founder and CTO, with a background in applied mathematics and physics.

After its official foundation in 2017, Beem (formerly HoloMe) did not fully focus on the consumer market. Instead, it developed AR projects and campaigns for brands, companies and other organizations including Vogue, Carolina Herrera, Hermes, Louis Vuitton, H&M, Forever 21, Warner Music, LADBible (in partnership with KSI and Craig David), the British army, TEDx and several others.

During this time, Beem’s technology was used to stream an artist’s music in AR, conduct a live conference at users’ homes in AR, create “virtual catwalks” and create other types of AR experiences for his clients.

This experience allowed Beem to stay afloat while testing and developing his technology, but the company was never intended to be a development studio, Amstutz says — it wanted to provide a consumer experience.

With the Beem mobile app, which launched in its latest version last month, the aim is to put this kind of live AR – or 3D-like experience, if you prefer – into the hands of consumers.

To work, users download the Beem app for iOS or Android and position themselves in the camera’s viewfinder by mounting the phone to capture their entire frame or have a friend record it. Beem takes the video, segments the human in the video away from the background using his proprietary computer vision algorithms and cloud infrastructure, processes the asset in real time and packages it for the viewer.

The recipient receives the link they open in their mobile browser, where they are directed to a microsite that is activated for each different web AR-like interaction and then rolled back again when the communication is complete. Here, Beem has access to the phone’s accelerometer, which allows the calculations on the server to skew and alter the sender’s video feed in your own space to give you the illusion of three-dimensionality.

The receiver holds up their phone and places the “live beamed” person in their own space by tapping a dot on the floor – just like you do with other AR tools.

Image Credits: beem

The end result is what appears to be a sort of hologram of the person in your room talking to you in real time. You can pinch and zoom in on the person and move them around, but you can’t view them from all sides because that’s not captured in the original video. The quality isn’t as good as, say, that cute AR “green man” trending right now on TikTok or Google Search’s AR animals, but it’s smart because it’s live.

“We don’t have to be super technical… We need the mind of the viewer to believe that what they see is a real person and that is enough,” says Amstutz.

Beem users can use the app to host “conferences” such as AR Zoom calls, where they stream the holographic version of themselves up to 25 people with two-way audio. They can also launch one-way live streams to a wider audience or send pre-recorded video messages.

Image Credits: Beem app

Beem has filed for patents on his engine’s infrastructure and specific patents on various parts of his technology, such as video segmentation, his proprietary web experience, and his method of creating 3D-like experiences without being “actually” 3D. So far, two have been awarded.

To date, the app has been adopted by a few TikTokers including Lev Cameron, Jake Joseph Everett-Rose, and Michael Fallon for content creation. In the UK, Beem also partnered with an association of preparatory schools that have trialled the technology for teacher-pupil communication, with plans to roll out to 750,000 students. On Valentine’s Day, Beem users sent more than 1,500 AR messages.

Beem now has a cadence of about 500 posts per day and has seen about 14.2 million views of “beems” in the past six weeks, the company says.

However, the reality is that the usage situation of Beem for consumers is still a bit cumbersome – you have to hang up the phone or have someone hold it, record yourself and when recording a message there was a delay of a few minutes during processing before your link can be shared.

Image Credits: Screenshot Beem App

However, Amstutz says that Beem’s ideal usage scenario won’t be the cell phone.

“The ideal circumstance for two-way telepresence is to wear augmented reality glasses and have a tracking webcam in your room,” he explains. “So you could essentially control the goggles with your voice… and then the tracking webcam knows it’s time to track and record me, send me to you, and I’ll put you in my place at the same time.” see space and vice versa.”

The founder says Beem has a prototype AR glasses experience it’s working on for a set of AR glasses developed by an undisclosed FAANG company and a major chip supplier. But ultimately, Beem wants to bring this system to every AR goggle in the future.

In addition to launching the consumer app, Beem is announcing $4 million in seed funding from 5 Lion, Ascension Ventures, Grouport Ventures, Inertia Ventures, Lior Messika and other strategic angel investors. The money will be spent on growing the team and further developing the product ready for AR glasses in consumer markets.

“Innovation in AR creates huge opportunities for a variety of different markets, and the potential for AR’s impact is huge,” said Beem investor, Lior Messika. “Beem’s technology creates the infrastructure for AR to become truly accessible and viral. The global shift to digital is moving fast and Beem is leading the way,” he added.

Leave a Reply

Your email address will not be published.