Jens Kober

Open Deep Learning Toolkit for Robotics (OpenDR)

Project type

EU Horizon 2020 program, call H2020-ICT-2018-2020 (Information and Communication Technologies); 2019 – 2022

Abstract

The aim of OpenDR is to develop a modular, open and non-proprietary deep learning toolkit for robotics. We will provide a set of software functions, packages and utilities to help roboticists develop and test robotic applications that incorporate deep learning. OpenDR will enable linking robotics applications to software libraries such as tensorflow and the ROS operating environment. We focus on the AI and cognition core technology in order to give robotic systems the ability to interact with people and environments by means of deep-learning methods for active perception, cognition and decisions making. OpenDR will enlarge the range of robotics applications making use of deep learning, which will be demonstrated in the applications areas of healthcare, agri-food and agile production.

TUD project members

ir. Bas van der Heijden, ir. Jelle Luijkx, Dr. Osama Mazhar, Dr. Laura Ferranti, Dr.-Ing. Jens Kober, prof.dr. Robert Babuška,

Project consortium

Aristotle University of Thessaloniki, Tampere University of Technology, Aarhus University, Delft University of Technology, Albert-Ludwigs-Universität Freiburg, Cyberbotics Ltd., PAL Robotics S.L., Agro Intelligence ApS

Publications with videos

Bas van der Heijden, Laura Ferranti, Jens Kober, and Robert Babuška. Efficient Parallelized Simulation of Cyber-Physical Systems. Transactions on Machine Learning Research, 2024. Reproducibility Certification. [bibtex] [pdf] [url] [code] [video]

Jelle Douwe Luijkx, Zlatan Ajanović, Laura Ferranti, and Jens Kober. PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning. In NeurIPS 2022 - 5th Robot Learning Workshop: Trustworthy Robotics, 2022. [bibtex] [pdf] [url] [webpage] [video]

Osama Mazhar, Sofiane Ramdani, and Andrea Cherubini. A Deep Learning Framework for Recognizing Both Static and Dynamic Gestures. Sensors, 21(6), 2021. [bibtex] [pdf] [doi] [video]

Publications without videos

Runyu Ma, Jelle Douwe Luijkx, Zlatan Ajanović, and Jens Kober. ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models. arXiv:2403.09583 [cs.RO], 2024. [bibtex] [pdf] [webpage] [doi]

Bas van der Heijden, Jelle Douwe Luijkx, Laura Ferranti, Jens Kober, and Robert Babuška. Engine Agnostic Graph Environments for Robotics (EAGERx): A Graph-Based Framework for Sim2real Robot Learning. IEEE Robotics & Automation Magazine, :2–15, 2024. Early Access. [bibtex] [pdf] [webpage] [doi] [code] [video]

Alexander Keijzer, Bas van der Heijden, and Jens Kober. Prioritizing States with Action Sensitive Return in Experience Replay. In Sixteenth European Workshop on Reinforcement Learning (EWRL), 2023. [bibtex] [pdf] [url]

Nikolaos Passalis, Stefania Pedrazzi, Robert Babuška, Wolfram Burgard, Daniel Dias, Francesco Ferro, Moncef Gabbouj, Green Ole, Alexandros Iosifidis, Erdal Kayacan, Jens Kober, Olivier Michel, Nikolaos Nikolaidis, Paraskevi Nousi, Roel S. Pieters, Maria Tzelepi, Abhinav Valada, and Anastasios Tefas. OpenDR: An Open Toolkit for Enabling High Performance, Low Footprint Deep Learning for Robotics. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 12479–12484, 2022. [bibtex] [pdf] [doi]

Osama Mazhar, Robert Babuška, and Jens Kober. GEM: Glare or Gloom, I Can Still See You – End-to-End Multimodal Object Detector. IEEE Robotics and Automation Letters, 6(4):6321–6328, 2021. The contents of this paper were also selected by IROS'21 Program Committee for presentation at the Conference. [bibtex] [pdf] [doi]

Osama Mazhar and Jens Kober. Random Shadows and Highlights: A New Data Augmentation Method for Extreme Lighting Conditions. arXiv:2101.05361 [cs.CV], 2021. [bibtex] [pdf] [doi] [code]

Bas van der Heijden, Laura Ferranti, Jens Kober, and Robert Babuška. DeepKoCo: Efficient Latent Planning with an Invariant Koopman Representation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 183–189, 2021. [bibtex] [pdf] [url] [doi]