《MobiDev:2023年增强现实应用程序创建技术指引(英文版)(20页).pdf》由会员分享,可在线阅读,更多相关《MobiDev:2023年增强现实应用程序创建技术指引(英文版)(20页).pdf(20页珍藏版)》请在三个皮匠报告上搜索。
1、Table of ContentsTypes of Augmented Reality AppsMARKER-BASED ARMARKERLESS ARLOCATION-BASED ARSUPERIMPOSITION ARTechnologies Used to Develop Augmented RealityMobile Augmented Reality PlatformsAR DEVELOPMENT FOR IOS DEVICES:ARKIT 5 AND ARKIT 6LOCATION ANCHORSLIDAR AND DEPTH APIRAYCASTING APIARKIT 6 UP
2、DATESROOMPLANAR DEVELOPMENT FOR ANDROID DEVICES:ARCORECLOUD ANCHORSAR FACESAR IMAGESARKit vs ARCore Features ComparisonCross-Platform Mobile AR Development in UnityWeb-based Augmented Reality TechnologiesAugmented Reality Development For AR WearablesAI Comes to the Rescue When AR is Not EnoughWhere
3、to Start Building AR ApplicationsMAKING THE MAGIC HAPPENHow MobiDev Can Help You1The development of AR projects is more difficult than it seems at first glance,especially if you are looking to create not an ordinary AR feature for fun,but aserious multifunctional product.Augmented reality has gone b
4、eyond theentertainment industry and is actively used in construction,retail,education,manufacturing,real estate,and more these days.This guide covers thedevelopment of such projects that require not just animations or graphics,butthe design of complex systems for truly innovative solutions.Types of
5、Augmented Reality AppsBefore diving into augmented reality development tools,lets recall types of ARapplications including marker-based,markerless,location-based,andsuperimposition AR solutions.MARKER-BASED ARThese applications utilize a particular marker like a QR code or other image.3Dcontent in t
6、he app is placed in the world relative to,or on top of the marker.Anolder,but interesting example of marker-based AR is the PlayStation 3Wonderbook,a gaming peripheral that allows players to view a spell book ontheir screen.The book rotates and moves when they pick up the device andmove it around.Th
7、e camera uses the patterns on the actual book as a referenceto display the AR content,a technology that is often used today on Snapchat andInstagram.We use a marker-based approach to create solutions for AR indoor navigation,for example.The user has to scan the marker to start a route with AR pointe
8、rsand reach the desired location within the building.MARKERLESS ARInstead of using set patterns or codes to trigger the content,markerless ARutilizes a camera to detect environment patterns and motion sensors to detectsurfaces and place 3D objects.This typically involves a number of differenttechnol
9、ogies working together,such as:GPS and other location toolsDigital compass2CameraAccelerometer and GyroscopeDepth sensorsThe latest devices are equipped with depth-sensing hardware(LiDAR,ToF)toimprove precision.Markerless AR is powered not only by depth sensors andother positioning data but also by
10、ML algorithms on top of this data.This allowsfor a more accurate rendering of 3D content and powers the illusion that digitalobjects are part of the real world.Markerless AR is probably the most popular type of augmented reality now.Youcan use it for numerous cases.For example,its used to build try-
11、on solutionslike IKEA Place,that allow you to try furniture that you want to buy right at yourhome or office in AR.Augmented reality measurement tools for the constructionindustry are also powered by markerless technology.LOCATION-BASED ARWhen users enter a particular location,AR applications can us
12、e that data toaccurately display virtual content.This is how location-based AR works.Insteadof simply displaying an object in relative space,developers can show objects ingeographical space for users to observe and interact with.Technology-wise,location-based AR relies on GPS,a digital compass,and a
13、naccelerometer.Moreover,there are several approaches to narrow down theposition of the device:BLE(Bluetooth low energy)beaconsVPS(Visual Positioning System)Wi-Fi RTTUWB(Ultra-WideBand)This type of AR is often used in solutions for the travel industry because it allowsyou to launch an AR experience a
14、s soon as the user reaches a point of interest.3SUPERIMPOSITION ARThis type of AR involves digitally replacing an object or superimposing a virtualobject on top of another.For example,an app that can digitally change the colorof your couch could be considered superimposition AR.This technique is use
15、fulfor cause-and-effect demonstrations.For example,a user could point a cameraat areas of their city to see what it looked like ten years ago from an archive ofGoogle Maps.Technologies Used to Develop AugmentedRealityTechnologies used in augmented reality app development can depend on anumber of fac
16、tors,like the type of hardware being used,the available power ofthe device,and what application AR is being used for.Mobile Augmented Reality PlatformsSmartphones have unique advantages compared to other AR platforms.Theyreprevalent in the market and are extremely portable.This makes AR moreaccessib
17、le to many consumers since bulky headsets and elegant smart glasseshavent quite hit the mainstream just yet.Because of this,mobile AR is a primetarget for business applications.4Although mobile AR may not be the most powerful or immersive,it certainly hasthe potential to be very profitable and is on
18、e of the most important augmentedreality trends to keep track of.Moreover,mobile AR can be a cost-effective wayfor business owners to join the metaverse trend.Try-on solutions that allow youto test cosmetics or clothes before buying,AR avatars and filters available tousers on a smartphone can help b
19、usinesses to communicate with customerseven in a virtual environment.There are three approaches toward mobile AR that businesses need to choosefrom:Native Android AR applications with ARCoreNative iOS AR applications with ARKitCross-platform appsNative augmented reality app development allows develo
20、pers to take advantageof the power of a device.In turn,a cross-platform application may not be able totake advantage of powerful native features but it can minimize the developmenttime.Creating the same app with native code on each platform will be moreexpensive,but if more power and features are re
21、quired,it may be more effectivein the long run.If the application is fairly simple,cross-platform code may beenough.Despite Android dominating the global OS market,developers on GitHub havehistorically seemed to prefer ARKit based on the number of repositories over thepast several years.5AR DEVELOPM
22、ENT FOR IOS DEVICES:ARKIT 5 AND ARKIT 6LOCATION ANCHORSLocation anchors allow developers to affix virtual objects in the real world byusing geographic coordinates.For example,location anchors could display athree-dimensional icon or text in space next to an iconic building.Since locationanchors are
23、dependent on Apple Maps data,this means that if the city is notsupported by Apple Maps,functionality for location anchors may be limited.One of the most groundbreaking elements of location anchors is how location isapproximated.GPS is simply not precise enough to provide location anchors ona users s
24、creen.To assist,developers can look around with their camera to allowthe device to scan for features on buildings around them.By using thesearchitectural features in conjunction with Apple Maps Look Around data,a userslocation can be better approximated for location anchors.This feature is extremely
25、 useful for building AR outdoor navigation solutions aswell as AR apps for tourism and retail.LIDAR AND DEPTH APIDepth API is another valuable feature of ARKit 4 which continues to play animportant role in ARKit 5 and ARKit 6.It utilizes one of the most powerfulhardware features for AR on a mobile d
26、evice,the LiDAR scanner on the iPad Proand Phone 12 Pro,12 Pro Max,13 Pro,13 Pro Max and 14 Pro smartphones.Thisenables much better analysis of scenes and allows for real-world objects toocclude virtual objects with much better accuracy.LiDAR is used not only for augmented reality projects but for o
27、ther areas wheresurface scanning is required.Lidar allows you to more accurately recognizesurfaces both vertically and horizontally,and scan complex surfaces.However,its capabilities arent limitless.For example,it cannot scan behind objects,it onlyscans in a straight line,and to scan an object from
28、all sides,you have to goaround it.LiDAR can also work incorrectly with highly reflective shiny surfaceslike glass or a mirror.6Android devices have an analog of LiDAR ToF sensor,but on a very smallnumber of devices.RAYCASTING APIApples Raycasting API enables enhanced object placement in conjunction
29、withdepth data.This allows much more accurate placement of virtual objects on avariety of surfaces while taking into account their curvature and angle.Forexample,its possible to use this to place an object on the side of a wall or alongthe curves of a sofa rather than simply flat on the floor.ARKIT
30、6 UPDATESARKit 6 brought several new features to improve AR experiences on iOS devices.These include better motion capture,improvements to camera access,andadditional locations for Location Anchors.They also included Plane Anchors,afeature that allows tracking flat surfaces like tables.This makes it
31、 easier forthose surfaces to be moved without disrupting the AR experience.ROOMPLANA relatively new,cool feature of ARKit is the RoomPlan framework.RoomPlanallows you to get a 3D plan of a room in a matter of seconds using the iPadLiDAR scanner and AI processing of the received data.The quality of t
32、he outputdepends on the quality of the sensor and the type of surface.With RoomPlan you can output room scans in USD or USDZ file formats.Theseinclude the dimensions of the room and the items contained in the room,suchas furniture.These file formats can then be imported into other software forfurthe
33、r processing and development.RoomPlan significantly accelerates andshortens the development of solutions such as AR measurement applications.As part of our ARKit research here at MobiDev,we tested using ARKit for gazetracking.This opens up a number of possibilities for iOS,such as eye-basedgestures,
34、vision-based website heatmap analytics,and preventing distracted ordrowsy driving.7This is just one of many cases from our portfolio.You can check out more of ourAR demos here.AR DEVELOPMENT FOR ANDROID DEVICES:ARCOREIn an effort to stay competitive,Google has pushed ARCore far to remain one ofthe m
35、ost versatile AR development platforms in the world.Lets explore some ofthe features that are used by developers.CLOUD ANCHORSThis tool allows users to place virtual objects in physical space that can beviewed by other users on their own devices.Google has made sure that CloudAnchors can be seen by
36、users on iOS devices as well.On the Android side,ARCore is introducing new features to match ARKitsadvances.ARCore v1.33.0 introduces new Cloud Anchors endpoints and terrainanchors.These both improve the geographic anchoring of virtual objects.Lastyear,Google introduced ARCore Geospatial API,which,s
37、imilar to ARKit LocationAnchors,utilizes data from mapping databases.In this case,Google Earth andStreet View image data is used to identify the users geographic location anddisplay virtual objects in those locations accurately.8The following MobiDev demo demonstrates how ARCore object detectionfeat
38、ures for a virtual user manual work.AR FACESARCore enables developers to work with high-quality facial renderings bygenerating a 468-point 3D model.Masks and filters can be applied after theusers face is identified.This is one of the most popular use cases for AR appfeatures.AR faces help in buildin
39、g solutions like virtual cosmetics and accessorytry-ons for eCommerce.AR IMAGESVirtual business cards and advertising posters are just a few of the applicationsthat are possible with augmented reality images.2D markers can be used toimplement these features,as well as more advanced solutions like AR
40、 indoornavigation.We have tested ARCore with indoor navigation solutions at MobiDev.Over thepast several years,applications like indoor positioning have gotten even easiersince our first experiments,making this technology more feasible for use in thereal world.Check out this video to learn more abou
41、t it.9ARKit vs ARCore Features ComparisonThese two frameworks for Android and iOS respectively,are nearly identicalwhen viewed from a features perspective.However,the real difference betweenthese two devices is hardware consistency.Apple iPhone and iPad devices arelargely more consistent when it com
42、es to the behavior and capability of theirhardware.Meanwhile,Android devices are built by a number of different manufacturerswith different specifications.That is,they have greater fragmentation,andbesides,not all models support AR.Because of this,it becomes more difficult todeliver a consistent exp
43、erience across many different Android devices.In ourexperience,customers prefer ARKit development with the prospect ofexpanding to Android in the future.10Given the limitations of Android hardware,its important to keep in mind howpowerful your AR experience will be and what devices it should be runn
44、ing on.Should it primarily be running on the latest and greatest Samsung and Pixeldevices with high-performance AR features,or should it be less intensive to runon more devices?The choice is yours,but experienced AR developers are alwaysthere to help you find the best solution.Cross-Platform Mobile
45、AR Development inUnityIf utilizing the full power of native AR frameworks on Android and iOS isntnecessary and if your goal is faster time to market,cross-platform ARdevelopment in Unity may be a good option.Unity AR Foundation is a helpfulframework for cross-platform augmented reality app developme
46、nt.It supportsPlane detection,Anchors,Light estimation,2D image tracking,3D objecttracking,body tracking,Occlusion,and more.However,there are some missing features depending on the platform youre on.For example,Unity AR Foundation has limitations with some features for ARCoreat the moment,such as 3D
47、 object tracking,meshing,2D&3D body tracking,11collaborative participants,and human segmentation.If you want your app to becross-platform,youll have to keep these missing features in mind.In ourexperience,the best practice for developing AR solutions is to create nativeapplications.This allows you t
48、o provide a high-quality user experience and avoidpotential pitfalls.Web-based Augmented Reality TechnologiesOn one hand,web AR is an extremely accessible technology because it can runon a multitude of devices without installing any additional software.On theother hand,web AR is very limited in feat
49、ures and power.Some businesses are already utilizing web AR for technologies like virtual try-onsolutions.For example,Maybelline,LOral,and other companies,have theoption for users to virtually try on cosmetic products using their front-facingcamera and web AR software.Web AR is best implemented for
50、simple tasks like facial recognition filters,changing the appearance or color of an object in a scene like the color of hair,replacing backgrounds for video conferencing,and more.However,it cannotprovide a high level of accuracy and performance for more complex projects.Itsimportant to remember thes
51、e limitations when deciding what platform yourapplication should run on.Augmented Reality Development For ARWearablesWhen we talk about wearable technology for augmented reality,we typicallyrefer to gear like Microsoft HoloLens and more portable and comfortable glasseslike Google Glass.From a softwa
52、re engineering standpoint,Microsoft Hololences development isbased on the Microsoft technology stack and Azure Cloud.Check our demo that showcases the capabilities of the Microsoft HoloLens forproduct presentation and educational purposes.12As for AR Glasses,most of the hardware is Android-based,and
53、 manufacturersprovide SDKs for engineers to create apps.There are still some substantialaspects to be considered.The first is User Experience and User Interface,as thepattern of using such software is entirely different from what we are used towith smartphones.Then,engineers must be capable of build
54、ing lightweight andoptimized apps,as energy consumption is still the main pain point for manydevices.AI Comes to the Rescue When AR is NotEnoughAugmented reality is an amazing technology,but sometimes the capabilities ofready-made AR frameworks are not enough,especially if we are talking aboutcomple
55、x innovative solutions.This is where artificial intelligence algorithmscome to the rescue.AI is used in one way or another under the hood of manyfeatures of AR tools.For example,ARKits RoomPlan applies machine learningalgorithms for object detection and scene understanding.But if ready-madefeatures
56、still cannot provide the desired result,there is always a chance that theimplementation of custom AI can change it for your project.What AI can provide for AR projects:13improved object detection,object recognition,and image trackingcollecting more data about the environment for an improved ARexperi
57、enceimproved accuracy of measuring distance and objectsFor example,standard AR libraries have limited capabilities to create a solutionlike a virtual fitting room or virtual try-on(poor identification of body parts,rendering issues,etc.).Therefore,this requires the development of a customhuman pose
58、estimation model that could detect any key points in real time if theprocessed image contains only part of the body.As MobiDev has in-house AI labs,our expertise allows us to cover such complexprojects that require a combination of AR and AI.We can use ready-madeartificial intelligence tools or crea
59、te our own models that will allow you to bringyour innovative solution to life.Where to Start Building AR ApplicationsThe development of any software product begins with the definition of businessgoals.Only when you clearly understand what results you want to achieve,can adevelopment team help to cr
60、eate a workflow for you that meets your businessneeds.Traditionally,the process of creating an AR-powered product consists of5 steps.14The process of building an augmented reality mobile app starts with thepre-contract phase.Its important for you to discuss the project requirementswith developers so
61、 that everyone is on the same page about the projectsobjectives and scope.It also allows back-and-forth feedback between developersand the business to help them clarify the idea and choose the best possibleroute to success.15Following this,choosing a technology stack is the next logical step.This is
62、essential for business and technical analysis,as it will decide what platform isthe best to use and how the project will be built.Moreover,engineers have tocome up with a clear vision of how they plan to achieve targeted results,asbringing AR to a product often has hidden pitfalls to be considered.A
63、fter that,prototypes and 3D models are created.Once this is done,morethorough development begins.This involves backend&mobile features,ARmodules,and QA.Also,there are a number of different challenges with testingAR features that should be taken into consideration.For example,it requirestesting diffe
64、rent physical objects,in different scenes,under different lighting tomake sure that your app provides a consistent user experience.Finally,the app is finalized for deployment.The deployment will involve a shiftfrom development to support.The app needs to be updated and supported tobe compatible with
65、 new SDKs and new device requirements.MAKING THE MAGIC HAPPENBusinesses with experience in AR applications have a head start on achievingtheir goals.However,businesses that dont have internal augmented realitydevelopment teams may find it challenging to achieve their vision without helpfrom experien
66、ced augmented reality development professionals.Enlisting thehelp of experienced AR developers is a great way to build your product and geta return on your investment.How MobiDev Can Help YouOur team of augmented reality app developers is no strangers to challenges.MobiDev has experience with develo
67、ping applications for a number of differentsolutions across various industries.With over 400+experts,13 years ofexperience,and more than 450 projects in our portfolio,were more thancapable of working with you to achieve your goals.Having extensive experience with innovative technologies,our augmente
68、dreality developers know how to overcome the limits of existing AR frameworks tocreate more effective solutions.Combining AR with advanced artificial16intelligence and machine learning algorithms can help you provide moreaccurate and realistic AR experiences,improve app performance and increasecustomer satisfaction.If youre ready to start a project,feel free to reach out to us so we can talk abouthow we can achieve your vision.17