From a news perspective, the past month has been exciting. And interesting. So we rounded up the biggest AR announcements out of Facebook, Google, and Apple, and packed them into a blog post.
Facebook F8 2019
For a deeper dive into this topic, check out our complete F8 2019 recap here. The AR and eComm specific highlights are as follows:
Instagram Checkout is getting a tagging tool, which allows users to shop looks from brands and creators directly from their posts. Additionally, Instagram is expanding Checkout to Explore, where currently 20% of users spend their time, enabling a more seamless path to purchase. Not to be outdone, WhatsApp is enabling product catalogs for business owners using the app. Finally, Spark AR, Facebook’s AR creator platform is expanding to Windows. To date, more than 1 billion unique users have experienced AR on Facebook’s platforms.
Google I/O 2019
The AR announcements out of I/O this year were a long time coming, but well worth the wait. We took a deep dive into what these updates mean for retailers, which you can check out here. From a bird’s-eye-view:
Google is adding web-based 3D and AR capabilities to all ARCore compatible (newer) Android devices and Chrome on iPhone. This means that 3D and AR functionality is available to consumers on their devices outside of specialized apps across major device types. Furthermore, Google included Search in these capabilities. This makes sense as Search is what made Google the superstar it is today. Already live on many devices, users can open 3D and AR experiences by searching any topic that has a 3D model already attached to it. This is significant for brands as they implement searchable 3D and AR experiences on their own websites.
Apple WWDC 2019
Apple’s annual WWDC conference just took place, so we’ve devoted a bit more time here to explaining these updates. The main point is that everything got better and easier to use.
AR Kit 3
Apple announced support for automatic real time occlusion of people, which is a computer vision problem that has been really challenging for everyone building AR experiences. Body occlusion involves understanding where human figures begin and end so that virtual objects can account for people wandering into the frame and adjust accordingly.
This means correctly determining whether the new person is supposed to be in front of or behind the virtual object and rendering the scene based on that. Worth noting, this is just a first step. Environmental object occlusion is not yet supported, and body occlusion is far from perfect.
In addition to body occlusion, ARKit 3 is bringing real-time full body motion capture. This means that AR face filters can be extended to the rest of the body. Think handbag tracked to a body to compliment the glasses being tracked to a face. Additionally, face tracking now supports up to 3 people at a time when viewed by the front-facing cameras on iPhone X/XR/XS and iPad Pro. For the most part, forward-facing camera experiences have previously been limited to face tracking while rear-facing camera experiences offered only world tracking capabilities. With ARKit 3, both face and world tracking can be used simultaneously in either camera, meaning an object can be placed in the world next to a person along with another object being tracked to that person’s face or body. This makes for more elaborate experiences with multiple objects and tracking scenarios. Think glasses and handbag tracked to the face and body with travel bag placed on the ground, setting up ‘shop the look’ opportunities in AR similar to what we’re seeing in Instagram posts.
Additional improvements to AR Kit 3 include enhanced 3D object detection, meaning objects are better recognized in complex environments and improved machine learning to detect planes in the environment more quickly than before.
For Creators
On the developer side, Apple announced Reality Kit and Reality Composer, a two-part tool for building and deploying AR. Reality Kit is a framework that makes it easier to build AR experiences. Currently, gaming platforms, Unity and Unreal, own a lot of this territory, but these platforms were not originally built with AR in mind, which means there is room for improvement. Reality Kit is custom-built for iOS, with test and demo functionality for iphone and ipad built in. On the front end, Reality Composer, helps developers output interactive scenes with full AR support. It includes hundreds of virtual objects, with the ability to import and customize USDZ files, as well as animate and trigger responses to these virtual objects within scenes. Reality Composer runs both in XCode and as an iOS app, ensuring greater access to developers wherever they are developing.
Finally, Apple announced updates to Quick Look, its program enabling 3D and AR across built-in iOS apps such as Safari, Messages, Mail, News, and Notes. Coming in iOS 13, Quick Look will also include video recording, multiple model support, audio support, an AR coaching UI, and finally Apple Pay within the Quick Look experience. What does this mean for retailers? Shoppers will be able to record video of their product experience, and purchase directly in Quick Look via Apple Pay. Retailers can allow shoppers to view more than one product at a time in AR, and use audio to convey product details that may go unnoticed. And for those users who may be overwhelmed by the new experience that is augmented reality, retailers can rely on Apple’s own coaching UI to guide them through the experience. Currently Quick Look offers no explanation of what is to come, which is why we recommend retailers offer their own AR tutorial alongside an AR button.
What does it all mean?
To recap, AR is everywhere. Everywhere. It is available seamlessly to users across Apple and Android devices, in-app and on the web. AR development is getting easier and AR experiences are getting better, which means ubiquity. For shoppers, AR can be discovered on the web, in an app, via Search, and on social media. Retailers who haven’t begun incorporating 3D and AR into their digital merchandising strategy will soon be the minority. Scaling this strategy means aligning with technology partners who understand the implications and requirements of distribution across platforms and devices. We are quickly marching toward the day when consumers expect AR product experiences as an essential part of the purchase journey, and pioneering retailers will be the ones setting those expectations.
Not a current customer but ready to get started?
Demo the Platform Today!Already a current customer? Log in to Axis Today!
