Teaching Robots Interactively (TERI)

TERI comic

Project type

ERC Starting Grant; 2019-2023

Abstract

Programming and re-programming robots is extremely time-consuming and expensive, which presents a major bottleneck for new industrial, agricultural, care, and household robot applications. My goal is to realize a scientific breakthrough in enabling robots to learn how to perform manipulation tasks from few human demonstrations, based on novel interactive machine learning techniques.

Current robot learning approaches focus either on imitation learning (mimicking the teacher’s movement) or on reinforcement learning (self-improvement by trial and error). Learning even moderately complex tasks in this way still requires infeasibly many iterations or task-specific prior knowledge that needs to be programmed in the robot. To render robot learning fast, effective, and efficient, I propose to incorporate intermittent robot-teacher interaction, which so far has been largely ignored in robot learning although it is a prominent feature in human learning. This project will deliver a completely new and better approach: robot learning will no longer rely on initial demonstrations only, but it will effectively use additional user feedback to continuously optimize the task performance. It will enable the user to directly perceive and correct undesirable behavior and to quickly guide the robot toward the target behavior. In my previous research I have made ground-breaking contributions to the existing learning paradigms and I am therefore ideally prepared to tackle the three-fold challenge of this project: developing theoretically sound techniques which are at the same time intuitive for the user and efficient for real-world applications.

The novel framework will be validated with generic real-world robotic force-interaction tasks related to handling and (dis)assembly. The potential of the newly developed teaching framework will be demonstrated with challenging bi-manual tasks and a final study evaluating how well novice human operators can teach novel tasks to a robot.

Wind Turbine Brain (WT Brain)

WT Brain logo

Project type

TKI Wind op Zee; 2017-2020

Description

Background

Wind turbines behave well in a lot of different wind conditions, in the lightest breeze and the heaviest storms. In the end, it boils down to producing energy at the lowest possible cost. That is why the system that controls the response of a wind turbine to the wind, must keep finding a balance between producing energy and the loads acting on the wind turbine. This project focusses on finding new ways to finding this balance. These new ways are becoming available due to the rapid scientific progress in the fields of (deep) machine learning and the use of, amongst other things, neural networks. By applying these methods in wind turbine control, we expect to realise a 5-10% design equivalent load reduction.

Goal/ objective

The goal of the project is to improve the wind turbine controller and its design process. This will be achieved by using machine learning (ML) to tune the controller in the design phase. The controller will contain conventional estimators and controller structures. Neural or Bayesian networks (hereafter: artificial networks (AN)) will be used to search for opportunities to further improve the conventional controller design.

Work programme
  1. ML methods and AN architectures: AN types are compared and the architecture of an AN-based controller is selected. Based on that selection, appropriate ML methods are selected.
  2. Conventional controller tuning with ML: The selected target function may be difficult to capture for the conventional controller. ML can tune the parameters based on the target function directly.
  3. AN-controller: Incorporate AN in the controller and tune it.
  4. Tuning and testing: The new controller structures are tuned for manufacturer and theoretical wind turbine models
Outcome

Situations where improvements are sought are low, normal and highly turbulent winds, starts and stops and extreme wind conditions, such as extreme operating gusts or extreme direction changes. The improvement is expected to reduce design equivalent loads on the blades and the wind turbine tower and to reduce extreme loads. These reductions can directly reduce the cost of a wind turbine. Alternatively, the load reduction on the blades can be used to employ longer blades, resulting in a higher production.

TUD project members

ir. Nikolaos Moustakis, Dr.-Ing. Jens Kober, prof.dr.ir. Jan-Willem van Wingerden

Project consortium

Energy Research Centre of the Netherlands (ECN), Lagerwey Group B.V., TU Delft, Siemens Gamesa

Learning Physical Human-Robot Cooperation Tasks

example task

Project type

Industry project (Honda Research Institute Europe GmbH, HTSM PPS-toeslag); 2017-2021

Abstract

Human-robot interaction and collaboration is of fundamental importance for any robot leaving the safety of fences on a highly-structured factory floor: service and care scenarios, medical applications, offshore, maintenance and inspection, as well as industrial assembly. In this project, we will develop new concepts and techniques for robot learning that endow robots with the capability to physically interact and collaborate with humans. In particular, we will consider tasks related to joint handling of large objects, i.e., jointly transporting and manipulating them. Examples include transporting and assembling light traverses, or changing tires on a car.

Project members

ir. Linda van der Spaa, ir. Tamas Bates, Dr.-Ing. Jens Kober, Dr.-Ing. Michael Gienger

Publications

Tamas Bates, Jens Kober, and Michael Gienger. Head-tracked off-axis perspective projection improves gaze readability of 3D virtual avatars. In SIGGRAPH Asia - Technical Briefs, 2018. [ bib ]

Michael Gienger, Dirk Ruiken, Tamas Bates, Mohamed Regaieg, Michael Meißner, Jens Kober, Philipp Seiwald, and Arne-Christoph Hildebrandt. Human-robot cooperative object manipulation with contact changes. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018. [ bib | .pdf ]

Deep Learning for Robust Robot Control (DL-foRCe)

Project type

NWO Natural Artificial Intelligence; 2015-2019

Abstract

While robots can flawlessly execute a set of commands to achieve a task, these commands are mostly encoded by hand. There is a need for effective learning methods that can deal with the uncertainty in the robot's environment, in particular when only broad goals are specified, and the learning algorithm has to learn motor commands to achieve these goals. This typically involves reinforcement learning (RL). However, current RL for robotics tasks relies on ad hoc function approximators and is typically not robust to changes in the task, environment, or robot uncertainty (compliant robot actuators, or wear and tear). The aim of this project is to integrate two emerging notions in order to make reinforcement learning for robot control more robust and efficient: dynamic feedback control policies for robust control combined with deep neural networks to learn low-dimensional parameterizations of such control policies. This approach promises a generic and robust approach to reinforcement learning for robotic control.

Project members

ir. Tim de Bruin, Dr.-Ing. Jens Kober, Prof Dr Sander Bohté, Prof Karl Tuyls, prof.dr. Robert Babuška

Publications

Lucian Buşoniu, Tim de Bruin, Domagoj Tolić, Jens Kober, and Ivana Palunko. Reinforcement Learning for Control: Performance, Stability, and Deep Approximators. Annual Reviews in Control, 2018. [ bib | .pdf ]

Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. Experience selection in deep reinforcement learning for control. Journal of Machine Learning Research, 19(9):1--56, 2018. [ bib | http ]

Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. Integrating state representation learning into deep reinforcement learning. IEEE Robotics and Automation Letters, 3(3):1394--1401, 2018. The contents of this paper were also selected by ICRA'18 Program Committee for presentation at the Conference.bib | .pdf ]

Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. Improved deep reinforcement learning for robotics through distribution-based experience retention. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016. [ bib | .pdf ]

Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. Off policy experience retention for deep actor critic learning. In Deep Reinforcement Learning Workshop, Advances in Neural Information Processing Systems (NIPS), 2016. [ bib | .pdf ]

Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. The importance of experience replay database composition in deep reinforcement learning. In Deep Reinforcement Learning Workshop, Advances in Neural Information Processing Systems (NIPS), 2015. [ bib | .pdf ]