March 2022 – Experience Design

Right to Repair: How Augmented Reality Can Help

29 March 202229 March 2022

Augmented reality apps can walk you through repairs.

Right to Repair refers to proposed legislation that would make it realistic for any repair shop or even an average consumer to fix an electronic device. Right now, it is technically legal to fix your own phone or laptop, but as Thorin Klosowski in The New York Times Wirecutter points out, something being legal doesn’t necessarily mean it’s possible. In recent history, many electronics manufacturers have kept the necessary tools and information to repair their products under close guard. In order for someone to replace a battery in a smartphone, for instance, they need access to diagnostics, instructions (possibly using augmented reality), parts and tools.

What We Need to Make Right to Repair Work

Firstly, products have to be designed to be repaired. You have to be able to open the outer casing without breaking it, for example. Microsoft improved the repairability of their Surface Laptop 3 simply by rearranging a few of the inner parts, proving that such a move doesn’t necessarily mean a design overhaul. In addition, as consumers, we’ll need to shift our approach to shopping for products, taking the feasibility of repair into consideration. To this end, France has developed a repairability index to help people identify how easily a product can be fixed. Next, we need the tools to make fixes practical for the average repair shop or end-user. That’s where augmented reality (AR) comes in.

How Augmented Reality Can Make Swapping Out Parts Easier

Most of us would be intimidated by the idea of opening up a laptop to replace a battery, but imagine you had an expert standing over your shoulder, patiently walking you through each step. That’s what AR technology can do. It can overlay digital information onto your real device, giving you step-by-step instructions at your own pace, without the need to shift your attention between written instructions and the device. There are a lot of advantages to this approach:

It’s less expensive for the end-user who doesn’t have to buy a new device.
It leads to greater accuracy in repairs than written or even video instructions can.
It’s low cost to the manufacturer, who doesn’t have to use human technicians or publish and update written manuals
It’s better for the environment, keeping more electronics in use and out of the landfill.

The best thing is, this technology already exists in an accessible way. You can use your smartphone to walk you through replacing a cracked screen or battery. Here’s an app, being considered by several major tech companies, that shows how AR instruction can work: KAR on the App Store; KAR on Google Play.

It’s not limited to electronics. AR repair could realistically extend the life of everything from your toaster to your vehicle, reducing costs for manufacturers and consumers and producing less waste.

How Augmented Reality Takes GPS Maps to the Next Level

25 March 202225 March 2022

Maps have come a long way. Where we used to consult unwieldy paper maps, we now receive step-by-step directions on our smartphones, complete with traffic information. All of these iterations, however, tend to take our attention from what’s in front of us. When walking, it’s an inconvenience; when driving, it’s dangerous. There is a cognitive gap, where information is lost, between looking at directions and turning our attention back to the real-time path. Augmented reality, with directions overlaid onto the road in front of us, can remove this gap and increase efficiency and safety. (more on how our brains process GPS directions)

Augmented Reality Enhances Directions Inside

AR can make finding your way in a large warehouse, shopping mall, or train station easier. In 2021, Google Maps introduced its Live View feature for select areas and buildings. Imagine you’re in the airport and need to find your gate, the nearest coffee or an ATM. The Live View AR feature will overlay step-by-step directions onto your real-time surroundings through your phone’s screen. This same technology can be used to direct warehouse workers to find specific products in vast spaces, saving time and reducing errors. (more on AR uses in warehouse settings)

AR and Outside Walking Directions

Google actually pioneered their Live View AR feature outdoors, in 2019, before taking it to public inside spaces. It overlays large arrows onto your environment through your phone screen making it very difficult to get lost. One journalist tested it, purposely choosing the wrong direction, and found the AR function redirected him just as seamlessly, if not more so, than traditional GPS. Another crowd-sourced project, AED4EU by Lucien Engelen, puts AR walking direction to emergency use. The app can quickly direct you to the nearest AED (automatic external defibrillator) in case of emergency. Users upload AED locations and volunteers physically verify them. (more on AR applications for healthcare)

AR Driving Directions and the Heads-Up Display (HUD)

A heads-up display is just that — a display of information that allows you to keep your eyes on the road ahead as you drive. Several vehicles, such as the Audi Q4 e-tron EV, already offer versions of this AR-driven technology. This kind of AR-driven head-sup display can “overlay the driver’s view of the real world through the windshield with helpful navigation directions, warning signs, and vehicle data.” (Dan Carney, Design News, 2021) Panasonic has also developed an AR HUD predicted to be in cars by 2024. It displays lane edges, objects in the road and other driver information. The Mercedes Benz MBUX aviation system also uses AR to project floating arrows as driving directions. (Roberto Baldwin, Car & Driver 2021)

Augmented reality, as applied to GPS directions, whether walking, biking or driving will reduce error and increase safety and efficiency. Whether you’re looking for bagels at the airport, searching for a sofa in a warehouse or driving to someone’s house in a rainstorm, AR will get you there faster and easier.

How Augmented Reality Training Will Improve the Way We Learn

24 March 202224 March 2022

Every workplace requires some sort of training — upfront and ongoing. Training with augmented reality (AR) can enhance that, helping employees learn faster, more safely and at lower costs. With AR, people can experience hands-on training without the risk or expense of a real-world work situation.

“There is truth in the concept that people learn by doing. AR provides the opportunity for hands-on learning in a structured, interactive way. Trainees can practice job activities in real-time at their own pace and without pressure from customers, managers and performance expectations.”
~ Lorne Fade, Forbes, Feb. 2021

AR and New Employee Training

According to Michael Porter, some companies are already using augmented reality to reduce the training time for new employees to nearly zero. It also means they are able to hire people who are initially less skilled. Package delivery company DHL has found this especially advantageous with their seasonal hires during peak shipping times. The real-time training and guidance they can provide with AR has made their onboarding process faster and reduced their reliance on human instructors. Other organizations, including the U.S. Navy and Boeing, have also taken advantage of the efficiency of augmented reality training. (Harvard Business Review, 2019)

AR and Ongoing Training and Support

Historically, companies that engage in product maintenance have provided field engineers with support through service manuals and over the phone. Xerox now uses AR to connect its field techs with experts, increasing first-time fix rates by 67 percent and reducing average fix times by two hours. (Harvard Business Review, 2019) In this way, AR can be used not only in training new employees but in supporting existing ones, remotely and expediently.

Augmented Reality Support for Customers

With AR’s ability to overlay step-by-step instructions on a real-world environment, it can support customers in solving their own problems. When Xerox experimented with using AR in this way, they managed to solve 76 percent more problems without on-site help, which saved on travel costs and minimized downtime. Most significantly, customer satisfaction went up by 95 percent. (Harvard Business Review, 2019) This model could apply to end-user products from laptops to vehicles to kitchen appliances, increasing efficiency in repairs for both companies and customers.

Augmented reality training has applications in retail through simulated customer service interactions, with the military and combat training, medicine and hands-on anatomical learning and even in space with a reconstructed Mars landscape. (Forbes, Feb. 2021) The possibilities in increased efficiency, lower cost and safer training experiences give augmented reality, as a learning tool, almost endless potential.

Why Augmented Reality is the Next Big Thing

16 March 202216 March 2022

Virtual reality has gotten a lot of attention in the gaming industry, and now with Facebook rebranding as Meta, people anticipate even more applications for VR. But augmented reality, the tech that transforms data into images and overlays it onto our real world, is going to beat it to the punch. In fact, it’s already here.

“By superimposing digital information directly on real objects or environments, AR allows people to process the physical and digital simultaneously, eliminating the need to mentally bridge the two. That improves our ability to rapidly and accurately absorb information, make decisions, and execute required tasks quickly and efficiently.”
– Michael Porter, Harvard Business Review, 2019

Who’s Using AR Now and For What?

Augmented reality has all sorts of applications for driving directions, shopping, healthcare and instructions for how to replace your laptop’s battery. Industry giants like Amazon, Facebook, General Electric, Mayo Clinic and the U.S. Navy are already using AR for a major boost in quality and productivity. For a visual, imagine the animal faces of Snapchat superimposed onto real faces or Pokemon Go characters overlaid onto the real streets. Now instead, think of furniture overlaid onto a room, so you can see how it looks in your home before ordering it. Imagine information like heart rate and other vital signs projected right onto a patient during surgery.

Do You Need Smart Glasses to Use AR?

When researching the possibilities for AR tech, we often find it relies on the development of smart glasses so that consumers, employees, doctors can view information projected onto the world around them, but augmented reality doesn’t have to wait for smart glasses tech to catch up and become more affordable. AR is available right now to anyone who has a smartphone. With various apps, you can view your environment through the phone screen and see what a potential bedroom suite would look like in your space. You can do the same with walking navigation. In a large warehouse or airport, you can view real-time instructions through your phone on the walkway in front of you.

Over the next few years, the experience of augmented reality will be even more seamless through head-mounted interfaces and smart glasses. “Increasingly,” writes Michael Porter, “the constraint is not a lack of data and knowledge but how to assimilate and act on them — in other words, the interface with humans. AR is emerging as a leading solution to this challenge.”

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

How Augmented Reality Will Revolutionize Manufacturing

What Augmented Reality Can Do for Healthcare

What Augmented Reality Can Do for Healthcare

11 March 202211 March 2022

Augmented reality’s ability to integrate digital information with the real-world environment has the potential to enhance and improve healthcare in a variety of groundbreaking ways. Diagnosis, telehealth visits and surgeries all stand to benefit from increased accuracy and efficiency, and AR is already been integrated into healthcare in some places. AR is expected to generate $1,918.6 million in the healthcare market by 2026. (Research Dive)

Collecting Blood Samples with Augmented Reality

If you’ve ever had your blood drawn, you may have experienced the phlebotomist struggling to find a vein. The hit-and-miss of this procedure can be painful, time-consuming and stressful for both parties. At AccuVein, a medical device company, they use AR technology to convert the heat signature of veins into an image. That image is then superimposed onto the patient’s skin, making the vein easier to find the first time. (Harvard Business Review, 2019)

Augmented Reality in the Operating Room

Imagine a surgery in which information is projected directly onto the patient without physicians having to move their attention to a monitor or another team member for updates. Microsoft Hololens 2 does just this, allowing the surgeon to keep both hands on the procedure. These hands-free solutions, according to Deloitte Research, will transform the healthcare business model. Automatic external defibrillator (AED) location is an emergency application for AR outside of the operating room. Using an app on a smartphone, anyone can see locations of the nearest AEDs overlayed onto the environment.

AR in Telehealth and Diagnosis

The number of telemedicine visits increased dramatically during the pandemic, and AR can improve this experience as well. With augmented reality, patients and care providers can draw and annotate on a 3-D screen. When combined with machine learning algorithms, AR can help detect diseases such as cancer. Google’s 2020 announcement for an AR-based microscope for the Department of Defense is just such an example. Real-time camera images are processed with computer diagnostics to diagnose disease at an early stage. (mobidev.biz)

AR can help a patient understand how a drug work in their body without sifting through complicated medical texts. It could even help students learn about the human body in a highly interactive and engaging 3-D format. There are many more applications for augmented reality in healthcare; this is just the tip of the iceberg.

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

How Augmented Reality Will Revolutionize Manufacturing

How Augmented Reality Will Revolutionize Manufacturing

09 March 202209 March 2022

“While reality is three-dimensional, the rich data we now have to inform our decisions and actions remains trapped on two-dimensional pages and screens.”
Michael Porter & James Heppelmann

Augmented Reality aims to change those limitations by merging our two-dimensional information with the three-dimensional world. Harvard Business Review (2019) predicts that AR will permeate every industry and a multitude of organizations in the coming years, including universities and social enterprises. And nowhere is AR’s impact on human performance more obvious than in manufacturing.

Augmented Reality and Manufacturing

AR’s overlay of digital information on a real-world environment helps technicians prevent mistakes and work more efficiently. It’s already at work in some sectors. Newport News Shipbuilding, a designer of U.S. Navy aircraft carriers, uses AR to inspect newly built ships. The digital projections highlight construction structures, not part of the finished product, that must be removed. Previously, engineers compared the finished ship against its 2-D blueprints, a process that took around 36 hours. AR has reduced that inspection time to just 90 minutes. (Harvard Business Review, 2019)

Augmented Reality Product Demos

AR’s digital augmentation can make providing a demo for a client in any industry easy and cost-effective. Bosch Rexroth uses AR enhancement to demonstrate the CytroPac hydraulic power unit. Their customers see a 3-D representation of the unit’s pump and cooling options. The AR image of the system can be manipulated and reconfigured to customers’ individual specifications in an instant. (Harvard Business Review, 2019) With AR, there is no need to create and transport expensive, bulky prototypes to the customer, and the AR representations are far superior and more dynamic than any two-dimensional schematic.

Training with Augmented Reality

When a company needs to train employees at a distance, instruction manuals and video education is the norm, but these are not interactive, nor do they adapt to individual learning styles. Assembly instructions can be cumbersome and time-consuming to follow. AR instruction is real-time and projected onto the actual environment, providing step-by-step instructions right on the equipment. Boeing has used AR training for some of its aircraft manufacturing procedures. AR instruction walked trainees through a 50-step wing assembly process and cut assembly time down by 35 percent over 2-D instructions. The number of first-time assemblers who did it correctly increased by 90 percent. (Harvard Business Review, 2019)

Augmented reality, as is apparent, boosts speed and accuracy in the manufacturing process, not to mention cost savings in product demos. Merging digital information with real-world products increases our ability to absorb information and quickly and accurately apply it to the object in front of us. Amazon, Facebook, General Electric, Mayo Clinic and the U.S. Navy are using AR to their advantage already. It’s only a matter of time before it becomes ubiquitous.

More on Augmented Reality

What is Augmented Reality?

How Augmented Reality Will Change Shopping

Explore products in the context of your own home with AR.

How Augmented Reality Will Change Shopping

03 March 202203 March 2022

Augmented reality (AR), the technology that superimposes digital information on a real-world environment, will change the way we shop for everything from laptops to furniture to shoes. Several companies have rolled out their own versions of AR shopping. Alibaba’s app Taobao Buy allows shoppers to interact with a select range of products online. Amazon’s AR view feature lets us see a product in our home before purchasing it. Both Target and Ikea have made use of AR so that consumers can try out furniture in their homes ahead of ordering it. Statista predicts the AR market will reach $198 billion in 2025, and the number of AR mobile users will reach 3.5 billion this year alone. But “try before you buy” isn’t the only application for AR in retail.

Store Mapping

Large warehouse spaces and shopping malls can use AR to direct both consumers and employees in 3D real-time. Directions are given step-by-step and superimposed over the actual space’s image through a smartphone camera. Consumers can find what they need and make purchases more quickly. And with the increased use of “buy online pick up in-store,” AR store mapping can increase employees’ efficiency in finding products to pull.

Virtual Fitting Rooms

Trying out products virtually before purchase isn’t limited to blenders and sofas. Both Target and Sephora are using AR to allow customers to test products such as makeup, glasses and shoes. This application works both in-store and from the comfort of the shopper’s home. The AR experience in-store is becoming even more compelling with the evolution of smart mirror technology. The virtual fitting room market, valued at $3.5 billion in 2021, is predicted to reach $12.97 billion by 2028. (Fortune Business Insights, 2021)

Product Exploration

So a potential buyer can see how that bedroom suite will look in their specific bedroom using AR technology just by pointing a smartphone’s camera at the space. But what about specs, opening the drawers of the nightstand, or weight capacities? AR overlays information directly onto the image. You can open drawers, look inside, see information about each individual piece. Exploring the capabilities and interiors of furniture, vehicles and appliances can all be done virtually, making both product investigation and, as a business, showcasing your own wares to clients easy and accessible from anywhere.

The bottom line with AR technology and shopping? It makes it easier and less expensive. If you can try a recliner in your space before you have it shipped to you, everyone saves on shipping costs, fossil fuels and labor. If you have step-by-step directions projected into your environment, you save time by not getting lost. Shopping experiences will change dramatically with the application of augmented reality. They already are.

Sources:

Aimagazine.com, mobidev.biz, fortunebusinessinsights.com, statista.com

What is Augmented Reality?

01 March 202203 March 2022

You don’t have to imagine what this Tesla would look like in your driveway.

What is AR?

Most people are familiar with virtual reality, which generates an entirely computer-generated environment. But “augmented reality” (AR) isn’t as familiar to the general public, though many of us use it on a regular basis. AR does what it says; it augments or enhances our existing environment, overlaying digital information onto the real world. Think of the animal ears available on Snapchat or the critters that pop up on-location with the Pokemon Go app. Those are entertaining examples of AR, but consider how it could be applied to driving directions.

How Your Brain Processes AR

“The power of augmented reality grows out of the way humans process information,” according to the Harvard Business Review (2019). Most of our information processing (up to 90%) is visual. How quickly and accurately we can process that information depends on mental capacity; the demand on that capacity is called “cognitive load.” That cognitive load is greater when there is a gap between where we receive information and where we apply it.

AR Makes Driving Safer.

Take GPS directions, for example. you look at your phone screen, process the instruction and then shift your attention to apply what you just saw to the real-world road in front of you. All that shifting increases cognitive load and the likelihood you’ll make a mistake (miss a turn or cause an accident). Now imagine, through an AR heads-up display, your directions are superimposed onto the road in front of you — no more shifting attention, less cognitive load, fewer mistakes. It makes for a safer, less stressful driving experience.

What Else AR Can Do

This is only a drop in the vast ocean of AR’s potential — how it can enhance our interactions with products and increase efficiency in business. From manufacturing to training to demonstrating products for customers, the digital overlay enhancement of AR has far-reaching applications. Stay tuned for more.