augmented reality

  • Home
  • augmented reality
augmented reality
Why Microsoft and Apple are Investing in Augmented Reality
07 April 2022
Microsoft’s Multibillion-Dollar Deal with the Army

In March of 2021, the U.S. Army announced an almost $22 billion deal with Microsoft for a specialized version of their Hololens along with software and services. These augmented reality devices, based on the integrated visual augmentation system (IVAS) prototype version of the Hololens would only account for a small fraction of the cost; the rest covers services and Microsoft’s Azure computing. The headset employs augmented reality to project maps, a compass and thermal imaging over the real-world environment. It can help a soldier aim a weapon and can be used for training and rehearsal as well as actual combat. The deal is for a five-year term with an optional five additional years.

“The IVAS headset, based on HoloLens and augmented by Microsoft Azure cloud services, delivers a platform that will keep soldiers safer and make them more effective.”

~ Alex Kipman, technical fellow at Microsoft, introducer of the HoloLens in 2015
Apple’s New AR/VR Headset and Glasses

While Microsoft has secured a coveted government contract, Apple is focusing on the private sector. Apple has dabbled in AR and VR for almost two decades, but with the popularity of ARKit, their augmented reality platform for iOS, they’ve committed more resources to the technology. Apple has plans to roll out an AR headset in 2023 and the sleeker AR glasses in 2024 or 2025. The headset will contain a processor on par with that in their Mac computers and will be able to operate independently of a computer or smartphone. The headset will support both AR and VR (mixed reality) like Microsoft’s Hololens.

But despite AR being virtual reality’s less flashy cousin (It doesn’t create a virtual environment, only enhances our existing one), Apple ultimately seems more interested in AR (according to MacRumors) due to AR’s wide range of applications. The headset will run an operating system called rOS or “Reality OS,” and Apple plans an entire app store to provide immersive applications for the hardware. Apple Glasses (the potential name of the headset’s sleeker offspring) will be a slimmer version that resembles traditional eyewear. They will even be capable of integrating with prescription lenses.

What These Augmented Reality Investments Mean

Clearly, both Microsoft and Apple see augmented reality as playing a significant role in our future. AR can improve the safety and efficiency of combat, healthcare, shopping, of walking down the street to the coffee shop. AR is going to be part of our lives in the near future in a wide variety of settings, from the armed forces to your living room armchair.

AR can help make Right to Repair a reality.
Right to Repair Gains Ground; AR Technology is Here to Support It
06 April 2022
AR Technology can help make Right to Repair a reality.

Samsung has announced they will finally give consumers what they need to repair some of its Galaxy devices. Both users and third-party repair shops will have access to genuine parts and step-by-step guides. Apple is planning to launch a similar program giving users of the iPhone 12 and 13 access to repair manuals, and they say the program will eventually expand to some of its Mac computers. Major manufacturers are now supporting Right to Repair. (CNN Business)

Access to genuine Apple or Samsung parts and manuals is a boon for Right to Repair, but when we get down to the nuts and bolts of consumers actually opening up the cases of our phones and laptops, how will that go? We end-users may be too intimidated to take a screwdriver to our precious devices and their complex inner workings, even with directions. And when we do get into those parts and get stuck, we’ll need to call tech support. Companies like Apple and Samsung will have to provide that support and the experienced technicians to go with it. But maybe not as much as we think.

Where Augmented Reality Fits Into Right to Repair

Augmented reality — the digital overlay of information onto the real world — is already in the works for various devices. Imagine that technology applied to Right to Repair. Those users who will now be able to replace their own Galaxy screen or iPhone battery can see step-by-step instructions, not in a 2-D manual, but projected onto their device, reducing the likelihood of mistakes and confusion and the odds of needing to call tech support — a cost and headache saver for manufacturer and consumer.

Why We Need Right to Repair and AR

Americans dispose of 416,000 cell phones per day, and only 15-20 percent of electronic waste is recycled. Augmented reality can not only partner with Right to Repair to save us money, it can help save the planet. When, in the fall of 2020, school districts in the U.S. were short 5 million laptops due to supply chain issues, that problem could have been solved with repaired devices; plenty of laptops were manufactured to meet this need. (U.S. Pirg) Right to Repair and AR is a win-win pairing. It’s a cost savings for consumers who can now fix their own devices instead of buying new ones. It has a positive impact on the environment, and it’s good business. With the likes of Apple and Samsung getting involved in Right to Repair and several tech companies rolling out AR devices, it’s an investment in the future.

Augmented reality helps take apart a system board.
Why Right to Repair and Augmented Reality are Natural Partners
01 April 2022
Augmented reality helps take apart a system board.

Last year, President Biden signed an executive order directing the Federal Trade Commission to draw up some new rules for companies who like to limit who can repair your device. Tech manufacturers, for instance, have historically kept diagnostic and instructional materials under lock and key, available only to authorized shops. Ditto for official replacement parts. Many devices are also assembled in such a way (with glue, for instance) that makes it almost impossible to take them apart without breaking them. 
While Biden’s order didn’t change anything instantly, the Right to Repair movement is in full swing. Some state legislatures have already enacted Right to Repair laws for the auto industry. This thing is going to happen, and it’s going to create a need to disseminate information and repair instructions in a format the average consumer can understand. Augmented reality, with its digital overlay on the real world, can make end-user repairs almost seamless.

How AR and Right to Repair Help the Consumer

The battery life of your laptop has tanked. This is less than ideal for a device whose whole point is that it’s mobile and not constantly tied to a power outlet. But that’s the only thing wrong with it. You can take it somewhere and get the battery replaced. Maybe. Or you can give in and buy a new computer. Either way, it’s going to cost you a significant amount of money. 
Now imagine you can order a new battery and download instructions on how to swap it out. Instead of sifting through a PDF document, you hold your phone up to your laptop and step-by-step instructions are projected in real-time. You don’t have to know anything beforehand. The directions start with “remove this screw” and a digitally-projected arrow showing you exactly where it is. The AR instructions progress through each step of the process until your battery is replaced and your laptop is whole again. The process took minutes and cost you far less than taking it into a shop or buying a new computer. 

How the Manufacturer Benefits from AR Instruction

Typically, when a business provides instructions to consumers, it comes in the form of website or PDF text and two-dimensional diagrams. The manufacturer must provide tech support for any difficulties the repairer might run encounter. This means paying trained people to staff that help desk. 
AR instruction provides support as if an expert were in the consumer’s home giving instructions. This means needing fewer trained staff on hand to field problems. With quality AR instructions, users need less support. Brand loyalty is built when the customer has a satisfactory experience repairing their own device with AR directions the manufacturer has provided.

Right to Repair, Supported by AR, is Better for the Environment.

It’s not much of a leap to realize the environmental impact of being able to replace a battery rather than sending an entire device to the landfill, of making it easier for people to fix their own products — a tractor, a toaster, a smartphone. The key, as is often the case with new ways of doing things, is making it as simple as possible. 
AR removes barriers. It gets rid of the inefficient need to shift focus between written instructions and what you’re working on, reducing the chance of errors. It’s so user-friendly and intuitive, it disposes of the anxiety we feel about opening up complex electronics. Kaalo Design is already providing AR development services to help manufacturers quickly comply with Right to Repair standards. The technology and applications for AR instruction aren’t a futuristic concept; it’s already here. Right to repair is coming, and AR is key.

augmented reality repair
Right to Repair: How Augmented Reality Can Help
29 March 2022
Augmented reality apps can walk you through repairs.

Right to Repair refers to proposed legislation that would make it realistic for any repair shop or even an average consumer to fix an electronic device. Right now, it is technically legal to fix your own phone or laptop, but as Thorin Klosowski in The New York Times Wirecutter points out, something being legal doesn’t necessarily mean it’s possible. In recent history, many electronics manufacturers have kept the necessary tools and information to repair their products under close guard. In order for someone to replace a battery in a smartphone, for instance, they need access to diagnostics, instructions (possibly using augmented reality), parts and tools.

What We Need to Make Right to Repair Work

Firstly, products have to be designed to be repaired. You have to be able to open the outer casing without breaking it, for example. Microsoft improved the repairability of their Surface Laptop 3 simply by rearranging a few of the inner parts, proving that such a move doesn’t necessarily mean a design overhaul. In addition, as consumers, we’ll need to shift our approach to shopping for products, taking the feasibility of repair into consideration. To this end, France has developed a repairability index to help people identify how easily a product can be fixed. Next, we need the tools to make fixes practical for the average repair shop or end-user. That’s where augmented reality (AR) comes in.

How Augmented Reality Can Make Swapping Out Parts Easier

Most of us would be intimidated by the idea of opening up a laptop to replace a battery, but imagine you had an expert standing over your shoulder, patiently walking you through each step. That’s what AR technology can do. It can overlay digital information onto your real device, giving you step-by-step instructions at your own pace, without the need to shift your attention between written instructions and the device. There are a lot of advantages to this approach:

  • It’s less expensive for the end-user who doesn’t have to buy a new device.
  • It leads to greater accuracy in repairs than written or even video instructions can.
  • It’s low cost to the manufacturer, who doesn’t have to use human technicians or publish and update written manuals
  • It’s better for the environment, keeping more electronics in use and out of the landfill.

The best thing is, this technology already exists in an accessible way. You can use your smartphone to walk you through replacing a cracked screen or battery. Here’s an app, being considered by several major tech companies, that shows how AR instruction can work: KAR on the App Store; KAR on Google Play.


It’s not limited to electronics. AR repair could realistically extend the life of everything from your toaster to your vehicle, reducing costs for manufacturers and consumers and producing less waste. 

augmented reality mapping
How Augmented Reality Takes GPS Maps to the Next Level
25 March 2022

Maps have come a long way. Where we used to consult unwieldy paper maps, we now receive step-by-step directions on our smartphones, complete with traffic information. All of these iterations, however, tend to take our attention from what’s in front of us. When walking, it’s an inconvenience; when driving, it’s dangerous. There is a cognitive gap, where information is lost, between looking at directions and turning our attention back to the real-time path. Augmented reality, with directions overlaid onto the road in front of us, can remove this gap and increase efficiency and safety. (more on how our brains process GPS directions)

Augmented Reality Enhances Directions Inside

AR can make finding your way in a large warehouse, shopping mall, or train station easier. In 2021, Google Maps introduced its Live View feature for select areas and buildings. Imagine you’re in the airport and need to find your gate, the nearest coffee or an ATM. The Live View AR feature will overlay step-by-step directions onto your real-time surroundings through your phone’s screen. This same technology can be used to direct warehouse workers to find specific products in vast spaces, saving time and reducing errors. (more on AR uses in warehouse settings)

AR and Outside Walking Directions

Google actually pioneered their Live View AR feature outdoors, in 2019, before taking it to public inside spaces. It overlays large arrows onto your environment through your phone screen making it very difficult to get lost. One journalist tested it, purposely choosing the wrong direction, and found the AR function redirected him just as seamlessly, if not more so, than traditional GPS. Another crowd-sourced project, AED4EU by Lucien Engelen, puts AR walking direction to emergency use. The app can quickly direct you to the nearest AED (automatic external defibrillator) in case of emergency. Users upload AED locations and volunteers physically verify them. (more on AR applications for healthcare)

AR Driving Directions and the Heads-Up Display (HUD)

A heads-up display is just that — a display of information that allows you to keep your eyes on the road ahead as you drive. Several vehicles, such as the Audi Q4 e-tron EV, already offer versions of this AR-driven technology.  This kind of AR-driven head-sup display can “overlay the driver’s view of the real world through the windshield with helpful navigation directions, warning signs, and vehicle data.” (Dan Carney, Design News, 2021) Panasonic has also developed an AR HUD predicted to be in cars by 2024. It displays lane edges, objects in the road and other driver information. The Mercedes Benz MBUX aviation system also uses AR to project floating arrows as driving directions. (Roberto Baldwin, Car & Driver 2021)

Augmented reality, as applied to GPS directions, whether walking, biking or driving will reduce error and increase safety and efficiency. Whether you’re looking for bagels at the airport, searching for a sofa in a warehouse or driving to someone’s house in a rainstorm, AR will get you there faster and easier.

AR Training
How Augmented Reality Training Will Improve the Way We Learn
24 March 2022

Every workplace requires some sort of training — upfront and ongoing. Training with augmented reality (AR) can enhance that, helping employees learn faster, more safely and at lower costs. With AR, people can experience hands-on training without the risk or expense of a real-world work situation.

“There is truth in the concept that people learn by doing. AR provides the opportunity for hands-on learning in a structured, interactive way. Trainees can practice job activities in real-time at their own pace and without pressure from customers, managers and performance expectations.”

~ Lorne Fade, Forbes, Feb. 2021
AR and New Employee Training

According to Michael Porter, some companies are already using augmented reality to reduce the training time for new employees to nearly zero. It also means they are able to hire people who are initially less skilled. Package delivery company DHL has found this especially advantageous with their seasonal hires during peak shipping times. The real-time training and guidance they can provide with AR has made their onboarding process faster and reduced their reliance on human instructors. Other organizations, including the U.S. Navy and Boeing, have also taken advantage of the efficiency of augmented reality training. (Harvard Business Review, 2019)

AR and Ongoing Training and Support

Historically, companies that engage in product maintenance have provided field engineers with support through service manuals and over the phone. Xerox now uses AR to connect its field techs with experts, increasing first-time fix rates by 67 percent and reducing average fix times by two hours. (Harvard Business Review, 2019) In this way, AR can be used not only in training new employees but in supporting existing ones, remotely and expediently.

Augmented Reality Support for Customers

With AR’s ability to overlay step-by-step instructions on a real-world environment, it can support customers in solving their own problems. When Xerox experimented with using AR in this way, they managed to solve 76 percent more problems without on-site help, which saved on travel costs and minimized downtime. Most significantly, customer satisfaction went up by 95 percent. (Harvard Business Review, 2019) This model could apply to end-user products from laptops to vehicles to kitchen appliances, increasing efficiency in repairs for both companies and customers.

Augmented reality training has applications in retail through simulated customer service interactions, with the military and combat training, medicine and hands-on anatomical learning and even in space with a reconstructed Mars landscape. (Forbes, Feb. 2021) The possibilities in increased efficiency, lower cost and safer training experiences give augmented reality, as a learning tool, almost endless potential.

Kitchenaid Mixer AR
Why Augmented Reality is the Next Big Thing
16 March 2022

Virtual reality has gotten a lot of attention in the gaming industry, and now with Facebook rebranding as Meta, people anticipate even more applications for VR. But augmented reality, the tech that transforms data into images and overlays it onto our real world, is going to beat it to the punch. In fact, it’s already here.

“By superimposing digital information directly on real objects or environments, AR allows people to process the physical and digital simultaneously, eliminating the need to mentally bridge the two. That improves our ability to rapidly and accurately absorb information, make decisions, and execute required tasks quickly and efficiently.”

– Michael Porter, Harvard Business Review, 2019
Who’s Using AR Now and For What?

Augmented reality has all sorts of applications for driving directions, shopping, healthcare and instructions for how to replace your laptop’s battery. Industry giants like Amazon, Facebook, General Electric, Mayo Clinic and the U.S. Navy are already using AR for a major boost in quality and productivity. For a visual, imagine the animal faces of Snapchat superimposed onto real faces or Pokemon Go characters overlaid onto the real streets. Now instead, think of furniture overlaid onto a room, so you can see how it looks in your home before ordering it. Imagine information like heart rate and other vital signs projected right onto a patient during surgery.

Do You Need Smart Glasses to Use AR?

When researching the possibilities for AR tech, we often find it relies on the development of smart glasses so that consumers, employees, doctors can view information projected onto the world around them, but augmented reality doesn’t have to wait for smart glasses tech to catch up and become more affordable. AR is available right now to anyone who has a smartphone. With various apps, you can view your environment through the phone screen and see what a potential bedroom suite would look like in your space. You can do the same with walking navigation. In a large warehouse or airport, you can view real-time instructions through your phone on the walkway in front of you.

Over the next few years, the experience of augmented reality will be even more seamless through head-mounted interfaces and smart glasses. “Increasingly,” writes Michael Porter, “the constraint is not a lack of data and knowledge but how to assimilate and act on them — in other words, the interface with humans. AR is emerging as a leading solution to this challenge.”

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

How Augmented Reality Will Revolutionize Manufacturing

What Augmented Reality Can Do for Healthcare

augmented reality and healthcare
What Augmented Reality Can Do for Healthcare
11 March 2022

Augmented reality’s ability to integrate digital information with the real-world environment has the potential to enhance and improve healthcare in a variety of groundbreaking ways. Diagnosis, telehealth visits and surgeries all stand to benefit from increased accuracy and efficiency, and AR is already been integrated into healthcare in some places. AR is expected to generate $1,918.6 million in the healthcare market by 2026. (Research Dive)

Collecting Blood Samples with Augmented Reality

If you’ve ever had your blood drawn, you may have experienced the phlebotomist struggling to find a vein. The hit-and-miss of this procedure can be painful, time-consuming and stressful for both parties. At AccuVein, a medical device company, they use AR technology to convert the heat signature of veins into an image. That image is then superimposed onto the patient’s skin, making the vein easier to find the first time. (Harvard Business Review, 2019)

Augmented Reality in the Operating Room

Imagine a surgery in which information is projected directly onto the patient without physicians having to move their attention to a monitor or another team member for updates. Microsoft Hololens 2 does just this, allowing the surgeon to keep both hands on the procedure. These hands-free solutions, according to Deloitte Research, will transform the healthcare business model. Automatic external defibrillator (AED) location is an emergency application for AR outside of the operating room. Using an app on a smartphone, anyone can see locations of the nearest AEDs overlayed onto the environment.

AR in Telehealth and Diagnosis

The number of telemedicine visits increased dramatically during the pandemic, and AR can improve this experience as well. With augmented reality, patients and care providers can draw and annotate on a 3-D screen. When combined with machine learning algorithms, AR can help detect diseases such as cancer. Google’s 2020 announcement for an AR-based microscope for the Department of Defense is just such an example. Real-time camera images are processed with computer diagnostics to diagnose disease at an early stage. (mobidev.biz)

AR can help a patient understand how a drug work in their body without sifting through complicated medical texts. It could even help students learn about the human body in a highly interactive and engaging 3-D format. There are many more applications for augmented reality in healthcare; this is just the tip of the iceberg.

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

How Augmented Reality Will Revolutionize Manufacturing

AR Product Demo
How Augmented Reality Will Revolutionize Manufacturing
09 March 2022

“While reality is three-dimensional, the rich data we now have to inform our decisions and actions remains trapped on two-dimensional pages and screens.”

Michael Porter & James Heppelmann

Augmented Reality aims to change those limitations by merging our two-dimensional information with the three-dimensional world. Harvard Business Review (2019) predicts that AR will permeate every industry and a multitude of organizations in the coming years, including universities and social enterprises. And nowhere is AR’s impact on human performance more obvious than in manufacturing.

Augmented Reality and Manufacturing

AR’s overlay of digital information on a real-world environment helps technicians prevent mistakes and work more efficiently. It’s already at work in some sectors. Newport News Shipbuilding, a designer of U.S. Navy aircraft carriers, uses AR to inspect newly built ships. The digital projections highlight construction structures, not part of the finished product, that must be removed. Previously, engineers compared the finished ship against its 2-D blueprints, a process that took around 36 hours. AR has reduced that inspection time to just 90 minutes. (Harvard Business Review, 2019)

Augmented Reality Product Demos

AR’s digital augmentation can make providing a demo for a client in any industry easy and cost-effective. Bosch Rexroth uses AR enhancement to demonstrate the CytroPac hydraulic power unit. Their customers see a 3-D representation of the unit’s pump and cooling options. The AR image of the system can be manipulated and reconfigured to customers’ individual specifications in an instant. (Harvard Business Review, 2019) With AR, there is no need to create and transport expensive, bulky prototypes to the customer, and the AR representations are far superior and more dynamic than any two-dimensional schematic.

Training with Augmented Reality

When a company needs to train employees at a distance, instruction manuals and video education is the norm, but these are not interactive, nor do they adapt to individual learning styles. Assembly instructions can be cumbersome and time-consuming to follow. AR instruction is real-time and projected onto the actual environment, providing step-by-step instructions right on the equipment. Boeing has used AR training for some of its aircraft manufacturing procedures. AR instruction walked trainees through a 50-step wing assembly process and cut assembly time down by 35 percent over 2-D instructions. The number of first-time assemblers who did it correctly increased by 90 percent. (Harvard Business Review, 2019)

Augmented reality, as is apparent, boosts speed and accuracy in the manufacturing process, not to mention cost savings in product demos. Merging digital information with real-world products increases our ability to absorb information and quickly and accurately apply it to the object in front of us. Amazon, Facebook, General Electric, Mayo Clinic and the U.S. Navy are using AR to their advantage already. It’s only a matter of time before it becomes ubiquitous.

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

Explore products in the context of your own home with AR.
How Augmented Reality Will Change Shopping
03 March 2022

Augmented reality (AR), the technology that superimposes digital information on a real-world environment, will change the way we shop for everything from laptops to furniture to shoes. Several companies have rolled out their own versions of AR shopping. Alibaba’s app Taobao Buy allows shoppers to interact with a select range of products online. Amazon’s AR view feature lets us see a product in our home before purchasing it. Both Target and Ikea have made use of AR so that consumers can try out furniture in their homes ahead of ordering it. Statista predicts the AR market will reach $198 billion in 2025, and the number of AR mobile users will reach 3.5 billion this year alone. But “try before you buy” isn’t the only application for AR in retail.

Store Mapping

Large warehouse spaces and shopping malls can use AR to direct both consumers and employees in 3D real-time. Directions are given step-by-step and superimposed over the actual space’s image through a smartphone camera. Consumers can find what they need and make purchases more quickly. And with the increased use of “buy online pick up in-store,” AR store mapping can increase employees’ efficiency in finding products to pull.

Virtual Fitting Rooms

Trying out products virtually before purchase isn’t limited to blenders and sofas. Both Target and Sephora are using AR to allow customers to test products such as makeup, glasses and shoes. This application works both in-store and from the comfort of the shopper’s home. The AR experience in-store is becoming even more compelling with the evolution of smart mirror technology. The virtual fitting room market, valued at $3.5 billion in 2021, is predicted to reach $12.97 billion by 2028. (Fortune Business Insights, 2021)

Product Exploration

So a potential buyer can see how that bedroom suite will look in their specific bedroom using AR technology just by pointing a smartphone’s camera at the space. But what about specs, opening the drawers of the nightstand, or weight capacities? AR overlays information directly onto the image. You can open drawers, look inside, see information about each individual piece. Exploring the capabilities and interiors of furniture, vehicles and appliances can all be done virtually, making both product investigation and, as a business, showcasing your own wares to clients easy and accessible from anywhere.

The bottom line with AR technology and shopping? It makes it easier and less expensive. If you can try a recliner in your space before you have it shipped to you, everyone saves on shipping costs, fossil fuels and labor. If you have step-by-step directions projected into your environment, you save time by not getting lost. Shopping experiences will change dramatically with the application of augmented reality. They already are.

Sources:

Aimagazine.com, mobidev.biz, fortunebusinessinsights.com, statista.com