DIOR IT

Project Goal

• How can robotics and immersive XR technologies contribute to maintenance and control?
• How can real-time immersive digital images of a complex object be created from 3D scans and photographs?
  

In the shipping, aviation and road transport sectors, there are numerous opportunities to increase efficiency through the use of robotics and virtual and augmented reality (VR/AR). The DIOR IT project - Digitalization of existing infrastructure for optimization and maintenance using robotics and immersive technologies - is investigating how mobile robotics and immersive technologies (AR/VR) can be combined to achieve this increase in efficiency. The Fraunhofer CML is analyzing the potential of these technologies in the logistics sector, where there are numerous sources of danger. Robots can enable safe inspection and thus shorten the actual maintenance work, which saves time and increases safety. In addition, (semi-)autonomous inspections offer the advantage that the operator can use his working time elsewhere or operate several robots at the same time.

The CML relies on immersive XR technologies to implement these autonomous and remote-controlled inspection and maintenance operations. To evaluate relevant technical innovations, a variety of scalable automation methods were analyzed, with dimensions, materiality, lighting and accessibility being key criteria. The aim is to identify best practices for capturing a wide range of different motifs for representation in immersive XR.

The tested methods enable the production of lifelike digital images of objects. Directly accessible data from photo series and 3D scans are used for this purpose.

Objectives

As part of the project, remote control software was developed that makes it possible to control the Spot® walking robot from the manufacturer Boston Dynamics using a VR application. A current 360° live image of the robot is transmitted wirelessly to the VR headset and displayed.

The operator can see the current position of the spot on a map at all times and can either control it manually or give it a movement order on the map so that the spot navigates autonomously to the target point. Relevant points can be marked in the VR view, which can later be displayed on site using AR glasses. In addition, rooms were digitized using photogrammetric techniques to enable a future view of the maintenance case. The visualization of this data in a 3D environment and the quality assurance of the data taken from 2D documentation were further project components.

Tasks of the CML

• Interface for remote control
• Robot code for transmitting the camera images, processing the remote control signals and transmitting the current position
  
• VR application for displaying the camera image and sending the remote control signals. The collected data is to be used for a meaningful representation of different objects.
  

In addition, 3D scans were used to optimize the capture and display of complex objects in immersive XR environments. The integration and processing of these scans represent an important advance in precise and comprehensive digital imaging.