Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

• Main Authors: Thomas Hulin, Michael Panzirsch, Harsimran Singh, Andre Coelho, Ribin Balachandran, Aaron Pereira, Bernhard M. Weber, Nicolai Bechtel, Cornelia Riecke, Bernhard Brunner, Neal Y. Lii, Julian Klodmann, Anja Hellings, Katharina Hagmann, Gabriel Quere, Adrian S. Bauer, Marek Sierotowicz, Roberto Lampariello, Jörn Vogel, Alexander Dietrich, Daniel Leidner, Christian Ott, Gerd Hirzinger, Alin Albu-Schäffer
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2021-06-01
• Series: Frontiers in Robotics and AI
• Subjects: telerobotics; model-augmented telemanipulation; shared control; shared autonomy; haptic constraints
• Online Access: https://www.frontiersin.org/articles/10.3389/frobt.2021.611251/full

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges.