Jens Kober

Teaching Robots Interactively (TERI)

Project type

ERC Starting Grant; 2019-2023

Abstract

Programming and re-programming robots is extremely time-consuming and expensive, which presents a major bottleneck for new industrial, agricultural, care, and household robot applications. My goal is to realize a scientific breakthrough in enabling robots to learn how to perform manipulation tasks from few human demonstrations, based on novel interactive machine learning techniques.

Current robot learning approaches focus either on imitation learning (mimicking the teacher’s movement) or on reinforcement learning (self-improvement by trial and error). Learning even moderately complex tasks in this way still requires infeasibly many iterations or task-specific prior knowledge that needs to be programmed in the robot. To render robot learning fast, effective, and efficient, I propose to incorporate intermittent robot-teacher interaction, which so far has been largely ignored in robot learning although it is a prominent feature in human learning. This project will deliver a completely new and better approach: robot learning will no longer rely on initial demonstrations only, but it will effectively use additional user feedback to continuously optimize the task performance. It will enable the user to directly perceive and correct undesirable behavior and to quickly guide the robot toward the target behavior. In my previous research I have made ground-breaking contributions to the existing learning paradigms and I am therefore ideally prepared to tackle the three-fold challenge of this project: developing theoretically sound techniques which are at the same time intuitive for the user and efficient for real-world applications.

The novel framework will be validated with generic real-world robotic force-interaction tasks related to handling and (dis)assembly. The potential of the newly developed teaching framework will be demonstrated with challenging bi-manual tasks and a final study evaluating how well novice human operators can teach novel tasks to a robot.

Project members

dr. Zlatan Ajanović, dr. Carlos E. Celemin Paez, dr. Marta Ferraz, ing. Giovanni Franzese, Dr.-Ing. Jens Kober, dr. Ravi Prakash, dr. Leandro de Souza Rosa,

Publications with videos

Giovanni Franzese, Ravi Prakash, and Jens Kober. Generalization of Task Parameterized Dynamical Systems using Gaussian Process Transportation. arXiv:2404.13458 [cs.RO], 2024. [bibtex] [pdf] [doi] [code] [video]

Jianyong Sun, Jens Kober, Michael Gienger, and Jihong Zhu. Learning from Few Demonstrations with Frame-Weighted Motion Generation. In Experimental Robotics: The 18th International Symposium (ISER 2023) (Marcelo H. Ang Jr, Oussama Khatib, eds.), pp. 339–350, 2024. [bibtex] [pdf] [doi] [video]

Yulei Qiu, Jihong Zhu, Cosimo Della Santina, Michael Gienger, and Jens Kober. Robotic Fabric Flattening with Wrinkle Direction Detection. In Experimental Robotics: The 18th International Symposium (ISER 2023) (Marcelo H. Ang Jr, Oussama Khatib, eds.), pp. 339–350, 2024. [bibtex] [pdf] [webpage] [doi] [video]

Armin Avaei, Linda van der Spaa, Luka Peternel, and Jens Kober. An Incremental Inverse Reinforcement Learning Approach for Motion Planning with Separated Path and Velocity Preferences. Robotics, 12(2):61, 2023. [bibtex] [pdf] [doi] [video]

Ravi Prakash and Laxmidhar Behera. Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration. IEEE Transactions on Automation Science and Engineering, :1–14, 2023. [bibtex] [doi] [video]

Giovanni Franzese, Leandro de Souza Rosa, Tim Verburg, Luka Peternel, and Jens Kober. Interactive Imitation Learning of Bimanual Movement Primitives. IEEE/ASME Transactions on Mechatronics, 29(5):4006 – 4018, 2024. [bibtex] [file] [doi] [code] [video]

Carlos E. Celemin and Jens Kober. Knowledge- and Ambiguity-Aware Robot Learning from Corrective and Evaluative Feedback. Neural Computing and Applications, 35(23):16821–16839, 2023. [bibtex] [pdf] [doi] [code] [video]

Jelle Douwe Luijkx, Zlatan Ajanović, Laura Ferranti, and Jens Kober. PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning. In NeurIPS 2022 - 5th Robot Learning Workshop: Trustworthy Robotics, 2022. [bibtex] [pdf] [url] [webpage] [video]

Mariano Ramírez Montero, Giovanni Franzese, Jeroen Zwanepol, and Jens Kober. Solving Robot Assembly Tasks by Combining Interactive Teaching and Self-Exploration. arXiv:2209.11530 [cs.RO], 2022. [bibtex] [pdf] [doi] [code] [video]

Linda van der Spaa, Giovanni Franzese, Jens Kober, and Michael Gienger. Disagreement-Aware Variable Impedance Control for Online Learning of Physical Human-Robot Cooperation Tasks. In ICRA 2022 full day workshop - Shared Autonomy in Physical Human-Robot Interaction: Adaptability and Trust, 2022. [bibtex] [pdf] [url] [code] [video]

Anna Mészáros, Giovanni Franzese, and Jens Kober. Learning to Pick at Non-Zero-Velocity From Interactive Demonstrations. IEEE Robotics and Automation Letters, 7(3):6052–6059, 2022. [bibtex] [pdf] [url] [doi] [code] [video]

Giovanni Franzese, Anna Mészáros, Luka Peternel, and Jens Kober. ILoSA: Interactive Learning of Stiffness and Attractors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7778–7785, 2021. [bibtex] [pdf] [doi] [code] [video]

Snehal Jauhri, Carlos E. Celemin, and Jens Kober. Interactive Imitation Learning in State-Space. In 2020 Conference on Robot Learning (CoRL) (Jens Kober, Fabio Ramos, Claire Tomlin, eds.), PMLR, vol. 155 of Proceedings of Machine Learning Research, pp. 682–692, 2021. [bibtex] [pdf] [html] [code] [video]

Giovanni Franzese, Carlos E. Celemin, and Jens Kober. Learning Interactively to Resolve Ambiguity in Reference Frame Selection. In 2020 Conference on Robot Learning (CoRL) (Jens Kober, Fabio Ramos, Claire Tomlin, eds.), PMLR, vol. 155 of Proceedings of Machine Learning Research, pp. 1298–1311, 2021. [bibtex] [pdf] [html] [code] [video]

Rodrigo Pérez-Dattari, Carlos E. Celemin, Giovanni Franzese, Javier Ruiz-del-Solar, and Jens Kober. Interactive Learning of Temporal Features for Control: Shaping Policies and State Representations From Human Feedback. IEEE Robotics & Automation Magazine, 27(2):46–54, 2020. [bibtex] [pdf] [doi] [code] [video]

Publications without videos

Runyu Ma, Jelle Douwe Luijkx, Zlatan Ajanović, and Jens Kober. ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models. arXiv:2403.09583 [cs.RO], 2024. [bibtex] [pdf] [webpage] [doi]

Lucas Cosier, Rares Iordan, Sicelukwanda Zwane, Giovanni Franzese, James T. Wilson, Marc P. Deisenroth, Alexander Terenin, and Yasemin Bekiroglu. A Unifying Variational Framework for Gaussian Process Motion Planning. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), 2024. [bibtex] [pdf] [code]

Zlatan Ajanović, Bakir Lačević, and Jens Kober. Value Function Learning via Prolonged Backward Heuristic Search. In ICAPS 2023 Workshop: PRL Workshop Series – Bridging the Gap Between AI Planning and Reinforcement Learning, 2023. [bibtex] [pdf] [url]

Tomás Coleman, Giovanni Franzese, and Pablo Borja. Damping Design for Robot Manipulators. In Human-Friendly Robotics 2022 (Pablo Borja, Cosimo Della Santina, Luka Peternel, Elena Torta, eds.), Springer International Publishing, Cham, pp. 74–89, 2023. [bibtex] [file] [doi]

Eveline Drijver, Rodrigo Pérez-Dattari, Jens Kober, Cosimo Della Santina, and Zlatan Ajanović. Robotic Packaging Optimization with Reinforcement Learning. In IEEE 19^th International Conference on Automation Science and Engineering (CASE), pp. 1–7, 2023. [bibtex] [pdf] [doi]

Cristian Meo, Giovanni Franzese, Corrado Pezzato, Max Spahn, and Pablo Lanillos. Adaptation Through Prediction: Multisensory Active Inference Torque Control. IEEE Transactions on Cognitive and Developmental Systems, 15(1):32–41, 2023. [bibtex] [file] [doi]

Zlatan Ajanović, Emina Aličković, Aida Branković, Sead Delalić, Eldar Kurtić, Salem Malikić, Adnan Mehonić, Hamza Merzić, Kenan Šehić, and Bahrudin Trbalić. Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals \\ BOS: Vizija Bosne i Hercegovine u Doba Umjetne Inteligencije: Svjetski Trendovi, Mogućnosti, Odabrani Primjeri i Realistični Ciljevi. In Scientific-Professional Conference "Artificial Intelligence in Bosnia and Herzegovina"- research, application and development perspectives, Federalno ministarstvo obrazovanja i nauke/znanosti : Fondacija za inovacijski i tehnološki razvoj, pp. 13–46, 2022. [bibtex] [pdf] [url]

Carlos Celemin, Rodrigo Pérez-Dattari, Eugenio Chisari, Giovanni Franzese, Leandro de Souza Rosa, Ravi Prakash, Zlatan Ajanović, Marta Ferraz, Abhinav Valada, and Jens Kober. Interactive Imitation Learning in Robotics: A Survey. Foundations and Trends® in Robotics, 10(1–2):1–197, 2022. [bibtex] [pdf] [url] [doi]

Carlos E. Celemin and Jens Kober. Uncertainties Based Queries for Interactive Policy Learning with Evaluations and Corrections. In Companion Publication of the 2021 International Conference on Multimodal Interaction, pp. 192–193, 2021. [bibtex] [pdf] [doi]

Bart Bootsma, Giovanni Franzese, and Jens Kober. Interactive Learning of Sensor Policy Fusion. In 30^th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pp. 665–670, 2021. [bibtex] [pdf] [doi]

Jan Scholten, Daan Wout, Carlos E. Celemin, and Jens Kober. Deep Reinforcement Learning with Feedback-based Exploration. In IEEE Conference on Decision and Control (CDC), pp. 803–808, 2019. [bibtex] [pdf] [doi] [code]

Daan Wout, Jan Scholten, Carlos E. Celemin, and Jens Kober. Learning Gaussian Policies from Corrective Human Feedback. arXiv:1903.05216 [cs.LG], 2019. [bibtex] [pdf] [doi] [code]