Recently, telework has become widespread, and many people are adopting a non-face-to-face and non-contact style of working from a remote location. Along with this, the movement to incorporate XR (extended reality or cross reality) technology, which fuses the real world and the virtual world, into business is accelerating. This time, we will introduce how to use “Microsoft HoloLens 2” in business and points to note, based on the use case of “Microsoft HoloLens 2”, which is attracting attention as a state-of-the-art device .
Utilization of XR technology spreading with the spread of remote work
Due to the corona crisis, many industries have started semi-forced telework. Due to the development and expansion of online tools and the “accustomed” to telework, it is not uncommon for people to think that “remote work is rather more efficient”. Some companies have already started to reduce the size of their offices by making telework a basic form of work.
On the other hand, an increasing number of companies that emphasize real-world communication are returning to the office work system or adopting a “hybrid” style that combines telework and office work, depending on the situation. With the diversification of work styles, every company needs to prepare an environment that can flexibly respond to business partners and business content. What I can say without a doubt is that, as we enter the new normal era, it is an irreversible change that will not return to the previous work style or business environment.
Against this background, there is a growing movement to explore ways to remotely handle tasks that traditionally required a visit to the site. Recently, in particular, some devices use XR technologies such as VR (virtual reality), AR (augmented reality), and MR (mixed reality) to enable non-face-to-face and non-contact work from remote locations. One of them is the use of Microsoft’s latest device, HoloLens 2.
HoloLens 2 is a product in the category of so-called smart glasses and head-mounted displays. It’s a type of wearable computer that you wear on your head like glasses. It has the advantage of being able to superimpose and operate virtual image information such as holograms (3D images) on the image of the real world projected in front of you. Based on examples of wide application areas supported by DXC Technology Japan, we will organize three points that should be noted when using.
Case of the manufacturing industry where trouble was handled only by local staff
● Difficulty in dealing with troubles on business trips due to COVID-19
At production sites in the manufacturing industry, a wide variety of manufacturing equipment and devices are in operation that are used for product assembly and quality inspection. Company A’s equipment management department constantly monitors whether these manufacturing devices and equipment are operating normally, and is in charge of maintenance when necessary. Members stationed at the production site are responsible for routine inspection work. When some kind of trouble occurred, an expert with specialized knowledge and experience from another base would go on a business trip to the production site and deal with it while giving instructions and guidance alongside the on-site members.
However, due to the COVID-19 pandemic, business trips for specialists in remote areas were restricted, making it difficult to provide support in the event of trouble. Therefore, Company A decided to introduce a device that can remotely support work from a remote location. DXC Technology Japan provided a gimmick using HoloLens 2 and Dynamics 365 Remote Assist.
● Precise instructions can be given by the MR application
Microsoft’s Dynamics 365 Remote Assist is an MR application that uses HoloLens 2 to work while exchanging information in real-time. The video seen by field staff wearing HoloLens 2 can be shared, and remote experts can give instructions and guidance through Microsoft Teams on PCs and smartphones. Instructions such as “operate the lever on the right side” can be accurately conveyed by using the on-screen arrow or pencil annotation function.
As a result of introducing this mechanism, Company A is now able to deal with problems by itself, without the need for specialists to travel to the site. Dynamics 365 Remote Assist enables us to ask questions directly to team members even when there are unclear points during regular inspection work, which has dramatically improved work speed, efficiency, and quality. It also has the advantage of saving the time and money required for a specialist to travel when a problem occurs.
● Dynamics 365 Guides is also effective for training
There are also cases where it is used for training staff in charge of product assembly work on production lines. Here, you can use Dynamics 365 Guides, which can display hologram guidance overlaid on HoloLens 2 images. In the past, OJT was done while looking at the manual with a veteran, but now you can safely and effectively train without the manual blocking your hands.
Even in many cases published on the web, it seems that the manufacturing industry is roughly divided into (1) remote support and (2) training.
Things to keep in mind when introducing a device
1. The fastest way is to try it on the actual machine first.
What can be said about XR technology, in general, is that there is a big difference between understanding texts and materials and experiencing them on a real machine. No matter how strong your imagination is, the experience you get from the actual device is different from the impression you get on paper (through texts and materials). When we give explanations inside and outside the company, we try to get them to experience the actual machine as much as possible. A 10-minute experience leads to greater understanding and acceptance than pages of material.
The term “immersive experience”, often used in the context of XR, is a new term for an experience that is difficult to describe, but words alone are rarely enough. However, when you experience it, you will know exactly what the words mean. After the experience, we often hear the words, “The sense of the future is amazing.” I think it’s the impression that the future that was imagined in the world of movies and novels is being realized at hand.
You can try out relatively simple business scenarios with the standard settings level of Dynamics 365 Remote Assist and Guides. By experiencing it, you can think more specifically about your further requests (can you do this?) and expectations (what you want to do!). It tends to be a so-called “air battle” where it is difficult to understand what each other is saying about requests and expectations only on paper. When considering, we strongly recommend the experience on the actual machine.
2. Setup at the time of introduction is difficult remotely
DXC Technology Japan has worked on the introduction and construction of several remote work support mechanisms. Since the initial configuration work is difficult to perform remotely, we have recommended doing it locally. This is because it is essential to understand the size of the work area, the tools, and the work that you want to support remotely at the actual site (some equipment may not be able to take photos or videos of the site due to security issues).
Especially in Dynamics 365 Guides, it is necessary to attach a QR code marker as the origin to the “location that is easy to see without obstruction” of the device/equipment. If the placement is incorrect, you will have to start over again. Also, the accuracy of recognition is limited to a few centimeters. Even if you want to see it more accurately, there are times when the device/equipment is not recognized depending on the viewing direction. Placing QR code markers on “movable” devices/equipment should be a separate training scenario.
In this way, it is important for the customer and us to proceed with the initial setup work together at the site. Some customers laughed and said that even though remote support was introduced, this work itself could not be done remotely.
3. Consider not only the introduction but also how to continuously improve and operate
XR leaves a strong impression with a new experience upon introduction. On the other hand, we tend to end up with evaluations such as “wonderful” and “interesting.” It is also important to make calm evaluations to contribute to the improvement of operations.
It should be noted that “HoloLens 2 can not be used to remotely support any business”. For example, HoloLens 2 has a “hand tracking” function in which the sensor recognizes the hand, but of course, it cannot be recognized in the position where the hand is hidden. It should be considered (currently) that it is difficult to reproduce the sensory work of an expert, which is called the so-called “craftsmanship”. The appropriate usage that can be achieved at the setting level of HoloLens 2 and Dynamics 365 Guides is basic training, explanation of work procedures, and work based on checklists.
Decomposing work support contents into appropriate “granularity” is also important for business improvement. For example, Dynamics 365 Guides can measure and record the time workers spend on each step. One method is to aggregate and graph this data using Microsoft Power BI, an analysis tool, and break it down into units that facilitate business improvement.
HoloLens 2 becomes hot while using Dynamics 365 Remote Assist, so it is difficult to use for a long time. It is also necessary to think about how to operate the device, such as wearing it when it is necessary and putting it to sleep when it is not.
Consideration of these points is necessary to use HoloLens2 as a powerful device for remote work support and effective training.
DXC Technology provides enterprise XR services using various smart glasses, including HoloLens 2. One of them is a SaaS communication platform called DXC Remote Expert®. Remote experts can handle local requests and leave relevant documentation. We also support cooperation with core systems based on the knowledge we have accumulated through our extensive experience.
