《Adeia:2023扩展现实(XR)体验的未来白皮书(英文版)(12页).pdf》由会员分享,可在线阅读,更多相关《Adeia:2023扩展现实(XR)体验的未来白皮书(英文版)(12页).pdf(12页珍藏版)》请在三个皮匠报告上搜索。
1、The Future of XR ExperiencesWHITEPAPERadeia I The Future of XR Experiences I 2introduction Looking ahead toward the coming decade and the growth of extended reality(XR),new paradigms for user experience,new use cases and new device types will emerge.For this to happen,the industry must move beyond t
2、he limited approach of putting existing 2D experiences into virtual reality(VR)and augmented reality(AR).Often,developers leap headfirst into porting a 2D app into XR without fully thinking through the unique challenges and special considerations of the medium.Broadly speaking,XR is an umbrella term
3、 encompassing a wide range of VR and AR technologies.As XR becomes more prominent,the industry needs to focus on building XR-native applications,asking whether an application is something that can only be done in XR or can also be done in XR.XR-native applications are more useful and valuable for us
4、ers.To thrive,XR technology needs to inhabit an ecosystem of useable and secure devices,high-performance codecs,compelling applications,robust network enhancements and dynamic services.The user experience must be centered on intuitive and responsive user inputs,contextual awareness and a broader und
5、erstanding of user state and intent.Whats more,these new ecosystems must monetize in a variety of ways,so sustainable businesses can be built around them.This paper explores this market and its potential,then provides useful insight into key technology trends and areas for innovation.The following t
6、echnologies will be the focus of the discussion under the heading of XR:Virtual reality uses a head-mounted display (HMD)to immerse the user in 3D environments.VR is commonly used for gaming and training.Augmented reality overlays virtual objects onto the physical world and makes them viewable throu
7、gh various cameras,screens and even wearable devices like glasses.AR displays can be see-through(i.e.,transparent lenses)or pass-through(i.e.,front-facing cameras feeding video to a screen).adeia I The Future of XR Experiences I 3The terms VR and AR tend to be used somewhat interchangeably,but its i
8、mportant to understand that they are distinct concepts.Various companies are using other terms in their own messaging and branding efforts.In the same way the evolution of the smartphone,and Apples iPhone in particular,was a forcing function for all applications to be developed with a new device in
9、mind,XR will push application development into new territories.To make the most of the new hardware and its capabilities,this new 3D paradigm will be about more than taking existing applications and moving them into a new ecosystem.It will require new ways of thinking and new application development
10、 methods.Brand new challenges will emerge.Certain things will be familiar from the gaming world,such as server-side rendering.But until recently,the focus has been on a single 3D experience.In VR,for instance,the user has one experience at a time,such as a VR video game.In AR,the user has generally
11、been experiencing applications where they are manipulating 3D objects in space,either in an individual or collaborative way.As impressive as these experiences have been,they are still relatively narrow expressions of the potential capability of VR and AR applications.They dont approach the true tran
12、sformative potential of XR.adeia I The Future of XR Experiences I 4the emergence of XR devices and spatial computingApples recent product announcement Vision Pro,their HMD holds significant potential to catapult the XR market into its next phase of evolution.The Vision Pro as presented at its unveil
13、ing in the summer of 2023 moves the HMD from its previous position as a video gaming accessory into something more consumer-centric and beyond just gaming.This device is marketed as the first general-purpose HMD for what Apple calls“spatial computing.”Apple is only the latest tech player to enter th
14、is space.Meta,the parent company of Facebook,offers a VR headset called the Meta Quest 3.Other devices are available from well-established device makers such as HTC,Microsoft and Sony,as well as a host of startups entering this exciting new market.Many XR experiences will use pre-existing content th
15、at requires video encoding.Video codecs have evolved to be capable of handling the complexities inherent in immersive experiences.Earlier video codecs were designed to deliver 2D video presented on rectangular screens.Modern codecs such as VVC,the latest MPEG codec,also known as H.266,are much more
16、versatile,offering the ability to encode and decode anything from basic 2D video to a fully three-dimensional VR experience or other new types of video like interactive AR content.Whats more,these new codecs are more efficient.VVC,for instance,offers a greater than 40 percent improvement in coding e
17、fficiency over its predecessor(HEVC/H.265),meaning much more data can be delivered using the same bandwidth.On the other hand,many XR experiences will be interactive and require different approaches.With XR,rather than just watching video,the entire environment becomes an ambient experience.Instead
18、of merely encoding video,the devices and applications work together to render and track objects locally.For example,imagine watching The Little Mermaid.Rather than a movie limited to a 2D screen,an XR device adds another layer to the experience:the feeling of being underwater,with fish swimming arou
19、nd you.This experience is done in such a way that it doesnt distract from the film,but adds to it.Another interesting application for XR is data analytics.XR provides the ability to perform analytics from an immersive perspective,to essentially be“inside”the data.Gesture recognition enables natural
20、actions like pinching and touching to access and manipulate data,much like science-fiction films(think Minority Report).The data is no longer a flat 2D representation.codecs play a key roleadeia I The Future of XR Experiences I 5adeia I The Future of XR Experiences I 6While a spreadsheet wont look a
21、ll that different in an XR headset,reporting and data analytics tools will be able to draw from a database and create dynamic 3D,interactive visualizations.The thinking behind this is that users tend to engage more intuitively with data when they can look at it from more than just one angle.There ar
22、e countless examples in online retail where AR experiences are being made possible.Theres the virtual dressing room,where users can“try on”clothing and accessories using a selfie camera.Online furniture retailers are using AR to allow visitors to visualize what a new sofa,bookcase or end table would
23、 look like in their room,just by dragging the item into a camera view.Car manufacturers offer AR experiences to simulate sitting in the drivers seat and looking around the cars interior,to get a basic feel for the experience before taking a test drive.manufacturingFrom a manufacturing perspective,VR
24、 hardware has gotten much better in the past few years because OEMs have changed their design approach.Until recently,the VR industry was building prototypes using components originally intended for smartphones.Today,the industry has matured to the point where its making new devices from scratch usi
25、ng XR-specific components.Today,the industry has matured to the point where its making new devices from scratch using XR-specific components.adeia I The Future of XR Experiences I 7motion-to-photon latencyWhen using VR headsets,the human brain can get confused if theres too much delay between when m
26、otion is perceived and when it happens in the body.This is because theres a slight conflict between visual information and the sense of bodily awareness(also known as proprioception).For a VR headset,this process is a loop:you turn your head,the HMD detects this rotation and processes it,the new ima
27、ge is rendered,the rendered image is displayed,and your eyes detect the change.The amount of time this loop takes is referred to as motion-to-photon latency,and its the key to ensuring user comfort.When motion-to-photon latency is longer than 50 milliseconds(i.e.,about 1/20th of a second),the experi
28、ence can induce nausea.This is similar to how some people experience motion sickness on boats or airplanes:theres a noticeable difference between what the eyes see and what the body perceives.When motion-to-photon latency is less than 25 ms(i.e.,1/40th of a second),the user doesnt perceive any delay
29、.Most wired headsets today are capable of motion-to-photon latency of around 35 ms,which is in the tolerable range for most people,but could still be improved substantially.Motion-to-photon latency is a limiting factor for VR because such low latency has typically required most of the video renderin
30、g to happen on the device.Since the HMD needs to be relatively small and lightweight,and it needs to stay at a tolerably cool temperature,manufacturers are limited in how much on-board processing can happen while maintaining a latency below 50 ms.As networks improve and transmission speeds become fa
31、ster,much of the processing and rendering functionality should be offloaded to more powerful computers at the edge of the network while keeping the motion-to-photon latency below tolerable limits.As things stand today,this isnt yet a reality.Advances in wireless technology and edge computing show si
32、gnificant promise,however.One day we should get there,and when we do,there will be a cascading effect on XR technology adoption and proliferation.Devices will become smaller and lighter in weight,run cooler and become more powerful,all while requiring less energy.Even though motion-to-photon latency
33、 is dictated primarily by the hardware and firmware involved,responsibility for user comfort ultimately falls on application developers.All major hardware manufacturers in this space Apple,Meta and Microsoft,in particular have developed and published excellent,research-based user experience(UX)guide
34、lines for those designing XR experiences for these devices.The onus is on the application developer to follow these guidelines,and while many of them do,some do not.Motion sickness was never a major concern for software developers until VR devices came into existence.Now it is a real issue that deve
35、lopers must take into work enhancementsXR traffic also requires a lot of bandwidth in order to maintain latency within tolerable limits.Even Apple highlighted that if you connect your Mac to Vision Pro,their current internal networking can only handle one screen at a time.This means that improvement
36、s need to be made not just in terms of the XR devices themselves,but also in the other components of the ecosystem:computers,broadband networks(wired and wireless)and so on.As this technology proliferates,it will introduce challenges in office environments.For example,if lots of workers are using hi
37、gh-bandwidth XR devices simultaneously,its going to put a large amount of stress on enterprise networks.Companies will need to improve their internet and computing infrastructures to be robust enough to handle large volumes of XR traffic in addition to the everyday tasks they handle today.Enterprise
38、 IT departments will have to make decisions about things like where applications are running:which applications will run from the cloud and which need to be hosted on a corporate server?How will the enterprise edge computing network be structured?Each enterprise will need to answer these questions a
39、nd more,taking into consideration various concerns about bandwidth,traffic management,system resources and security.adeia I The Future of XR Experiences I 10Companies will need to improve their internet and computing infrastructures to be robust enough to handle large volumes of XR traffic in additi
40、on to the everyday tasks they handle today.adeia I The Future of XR Experiences I 11other aspects of user experienceSince we are talking about user experience,its worth pointing out that XR is going to create new paradigms.Think back to the early days of graphical user interface,the paradigm most of
41、 us use every day.You interface with the computer using a keyboard and mouse or trackpad.The point-and-click interface is intuitive and ubiquitous.In XR there will be different ways to interface with the system,such as gesture recognition,hand tracking and eye tracking.Diegetic user interfaces(UIs),
42、which fit within the context of the experience,will become more common in the future.Instead of pointing and clicking to open a calendar app,for instance,the XR experience might include a calendar on the wall,and as you look at it and reach for it,the system uses a combination of hand and eye tracki
43、ng to know that your intent is to open the calendar to schedule a meeting,or to review your appointments for the day.Diegetic UIs have been used commonly in video games to form a kind of fusion of the games mechanics and its narrative,where the function is incorporated into the world itself,and the
44、player character can interact with it.For instance,some games might use a world map and the character is portrayed as looking at a paper map they are holding in their hands.Or a characters health and status information might be incorporated into their appearance,such as visibly bleeding or limping r
45、ather than showing a health bar.Some of these experiences will need time to take shape.XR technology is still new enough that the industry doesnt have lots of user experience data to draw from to figure out what other types of diegetic interfaces may emerge.adeia I The Future of XR Experiences I 12c
46、ontextual awarenessWeve all seen videos of people crashing into walls while wearing VR headsets.When wearing an HMD,the user becomes immersed in a virtual experience,but the real world still exists.Its critical for XR developers to build in a way to make sure the user is still aware of the environme
47、nt outside the virtual space for their own safety and that of others around them.Another common issue with VR headsets is the user becomes so immersed in their experience that they become unaware of other people in the room.Apple has demonstrated an interesting solution:if someone approaches while y
48、oure wearing the Vision Pro,a representation of your eyes appears on the outside of your HMD,visible to the other person,and the Vision Pro indicates to you that someone is approaching.Since they are worn on our faces,these devices are also able to monitor our status.The same technology that allows
49、for eye tracking can easily determine how closely the user is paying attention.They can potentially monitor for motion sickness,or the degree to which a user is enjoying an experience.The hardware enables the application to be more responsive to the state of the user than ever before.adeia I The Fut
50、ure of XR Experiences I 13security and privacyThe flip side of contextual awareness is security and user privacy.Because the devices can so closely track things like our eyes,privacy and security considerations become very important.Most companies have thus far been very proactive about their inform
51、ation security posture,with several announcing that they plan to not only safeguard biometric information but also that they wont sell user data.The close(even intimate)nature of a camera-enabled device worn directly on the face has the potential to be more invasive if the data associated with it is
52、nt carefully protected.monetizationThe most prominent hardware players in this market Apple and Meta are taking different approaches to the business model.For its part,Meta is believed to lose money on every headset it sells.They are currently offering the Meta Quest 3 for$500.While we dont know wha
53、t the hardware costs to manufacture,a reconstruction of component costs tells us its costing Meta quite a bit more than that to manufacture.The assumption is they plan to make back their losses selling XR apps,advertising and related services.Apple on the other hand has announced an estimated retail
54、 price for the Vision Pro of$3500.In keeping with their typical product strategy,they intend to target a particular type of high-value customer who is interested in an elegant product and is less sensitive to price.Its possible that Apple may make more money per unit sold than the manufacturing cost
55、(or at least break even),an inverse of the strategy employed by Meta.Other companies fall somewhere in between these ends of the product pricing spectrum.It remains to be seen which of these strategies will help bring XR to the masses.Adeia is a leading R&D and intellectual property(IP)licensing com
56、pany that accelerates the adoption of innovative technologies in the media and semiconductor industries.Adeias fundamental innovations underpin technology solutions that are shaping and elevating the future of digital entertainment and electronics.Adeias IP portfolios power the connected devices that touch the lives of millions of people adeia I The Future of XR Experiences I#around the world every day as they live,work and play.For more,please visit .