1,720,967 research outputs found
Apprentissage de politiques multi-tâches pour la robotique
No full textDeveloping versatile robots capable of performing diverse tasks has the potential to alleviate human labor in physically demanding, dangerous, and tedious activities. However, the progress of robot learning has been relatively slow compared to other domains of machine learning partially due to the lack of large-scale robotics datasets. This thesis aims to introduce novel methods for learning multi-task policies for robotics. In our first contribution, we present a novel reinforcement learning algorithm that learns goal-reaching policies by interacting with the environment. Our approach incorporates imagined subgoals to guide policy learning during training, resulting in higher sample efficiency and the ability to solve more complex temporally extended tasks. In our second contribution, we propose a method for learning policies in multi-task vision-based manipulation environments that can follow human video instructions. By utilizing an existing large dataset of labeled human videos, we achieve this without requiring annotated robot demonstrations or task-specific reward shaping.Le développement de robots généralistes capables d’accomplir une vaste gamme de tâches présente un énorme potentiel pour alléger la charge de travail humain dans des tâches physiquement exigeantes, dangereuses ou fastidieuses. Cependant, les progrès de l’apprentissage robotique ont été relativement lents par rapport à d’autres domaines de l’apprentissage automatique, en partie en raison du manque de jeux de données de grande envergure pour la robotique. Cette thèse vise à présenter de nouvelles méthodes pour l’apprentissage des politiques multi-tâches pour la robotique. Dans notre première contribution, nous présentons un nouvel algorithme d’apprentissage par renforcement qui apprend des politiques d’atteinte d’objectifs en interagissant avec l’environnement. Notre approche intègre des sous-objectifs imaginés pour guider l’apprentissage de la politique lors de l’entraînement, ce qui se traduit par une meilleure efficacité d’échantillonnage et la capacité à résoudre des tâches temporellement plus complexes. Dans notre deuxième contribution, nous proposons une méthode pour apprendre des politiques capables de suivre des instructions vidéo humaines dans des environnements de manipulation multi-tâches basés sur la vision. En utilisant un ensemble de données volumineux existant de vidéos humaines annotées, nous parvenons à cela sans avoir besoin de démonstrations robotiques annotées ni de conception de fonctions de récompenses spécifiques pour chaque tâche
Apprentissage de politiques multi-tâches pour la robotique
No full textDeveloping versatile robots capable of performing diverse tasks has the potential to alleviate human labor in physically demanding, dangerous, and tedious activities. However, the progress of robot learning has been relatively slow compared to other domains of machine learning partially due to the lack of large-scale robotics datasets. This thesis aims to introduce novel methods for learning multi-task policies for robotics. In our first contribution, we present a novel reinforcement learning algorithm that learns goal-reaching policies by interacting with the environment. Our approach incorporates imagined subgoals to guide policy learning during training, resulting in higher sample efficiency and the ability to solve more complex temporally extended tasks. In our second contribution, we propose a method for learning policies in multi-task vision-based manipulation environments that can follow human video instructions. By utilizing an existing large dataset of labeled human videos, we achieve this without requiring annotated robot demonstrations or task-specific reward shaping.Le développement de robots généralistes capables d’accomplir une vaste gamme de tâches présente un énorme potentiel pour alléger la charge de travail humain dans des tâches physiquement exigeantes, dangereuses ou fastidieuses. Cependant, les progrès de l’apprentissage robotique ont été relativement lents par rapport à d’autres domaines de l’apprentissage automatique, en partie en raison du manque de jeux de données de grande envergure pour la robotique. Cette thèse vise à présenter de nouvelles méthodes pour l’apprentissage des politiques multi-tâches pour la robotique. Dans notre première contribution, nous présentons un nouvel algorithme d’apprentissage par renforcement qui apprend des politiques d’atteinte d’objectifs en interagissant avec l’environnement. Notre approche intègre des sous-objectifs imaginés pour guider l’apprentissage de la politique lors de l’entraînement, ce qui se traduit par une meilleure efficacité d’échantillonnage et la capacité à résoudre des tâches temporellement plus complexes. Dans notre deuxième contribution, nous proposons une méthode pour apprendre des politiques capables de suivre des instructions vidéo humaines dans des environnements de manipulation multi-tâches basés sur la vision. En utilisant un ensemble de données volumineux existant de vidéos humaines annotées, nous parvenons à cela sans avoir besoin de démonstrations robotiques annotées ni de conception de fonctions de récompenses spécifiques pour chaque tâche
Learning Video-Conditioned Policies for Unseen Manipulation Tasks
Full text linkThe ability to specify robot commands by a non-expert user is critical for
building generalist agents capable of solving a large variety of tasks. One
convenient way to specify the intended robot goal is by a video of a person
demonstrating the target task. While prior work typically aims to imitate human
demonstrations performed in robot environments, here we focus on a more
realistic and challenging setup with demonstrations recorded in natural and
diverse human environments. We propose Video-conditioned Policy learning (ViP),
a data-driven approach that maps human demonstrations of previously unseen
tasks to robot manipulation skills. To this end, we learn our policy to
generate appropriate actions given current scene observations and a video of
the target task. To encourage generalization to new tasks, we avoid particular
tasks during training and learn our policy from unlabelled robot trajectories
and corresponding robot videos. Both robot and human videos in our framework
are represented by video embeddings pre-trained for human action recognition.
At test time we first translate human videos to robot videos in the common
video embedding space, and then use resulting embeddings to condition our
policies. Notably, our approach enables robot control by human demonstrations
in a zero-shot manner, i.e., without using robot trajectories paired with human
instructions during training. We validate our approach on a set of challenging
multi-task robot manipulation environments and outperform state of the art. Our
method also demonstrates excellent performance in a new challenging zero-shot
setup where no paired data is used during training.Comment: ICRA 2023. See the project webpage at
https://www.di.ens.fr/willow/research/vip
Learning Video-Conditioned Policies for Unseen Manipulation Tasks
No full textICRA 2023. See the project webpage at https://www.di.ens.fr/willow/research/vip/International audienceThe ability to specify robot commands by a non-expert user is critical for building generalist agents capable of solving a large variety of tasks. One convenient way to specify the intended robot goal is by a video of a person demonstrating the target task. While prior work typically aims to imitate human demonstrations performed in robot environments, here we focus on a more realistic and challenging setup with demonstrations recorded in natural and diverse human environments. We propose Video-conditioned Policy learning (ViP), a data-driven approach that maps human demonstrations of previously unseen tasks to robot manipulation skills. To this end, we learn our policy to generate appropriate actions given current scene observations and a video of the target task. To encourage generalization to new tasks, we avoid particular tasks during training and learn our policy from unlabelled robot trajectories and corresponding robot videos. Both robot and human videos in our framework are represented by video embeddings pre-trained for human action recognition. At test time we first translate human videos to robot videos in the common video embedding space, and then use resulting embeddings to condition our policies. Notably, our approach enables robot control by human demonstrations in a zero-shot manner, i.e., without using robot trajectories paired with human instructions during training. We validate our approach on a set of challenging multi-task robot manipulation environments and outperform state of the art. Our method also demonstrates excellent performance in a new challenging zero-shot setup where no paired data is used during training
Goal-Conditioned Reinforcement Learning with Imagined Subgoals
No full textICML 2021. See the project webpage at https://www.di.ens.fr/willow/research/ris/International audienceGoal-conditioned reinforcement learning endows an agent with a large variety of skills, but it often struggles to solve tasks that require more temporally extended reasoning. In this work, we propose to incorporate imagined subgoals into policy learning to facilitate learning of complex tasks. Imagined subgoals are predicted by a separate high-level policy, which is trained simultaneously with the policy and its critic. This high-level policy predicts intermediate states halfway to the goal using the value function as a reachability metric. We don't require the policy to reach these subgoals explicitly. Instead, we use them to define a prior policy, and incorporate this prior into a KL-constrained policy iteration scheme to speed up and regularize learning. Imagined subgoals are used during policy learning, but not during test time, where we only apply the learned policy. We evaluate our approach on complex robotic navigation and manipulation tasks and show that it outperforms existing methods by a large margin
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Reinforcement Learning from Wild Animal Videos
No full textWe propose to learn legged robot locomotion skills by watching thousands of wild animal videos from the internet, such as those featured in nature documentaries. Indeed, such videos offer a rich and diverse collection of plausible motion examples, which could inform how robots should move. To achieve this, we introduce Reinforcement Learning from Wild Animal Videos (RLWAV), a method to ground these motions into physical robots. We first train a video classifier on a large-scale animal video dataset to recognize actions from RGB clips of animals in their natural habitats. We then train a multi-skill policy to control a robot in a physics simulator, using the classification score of a third-person camera capturing videos of the robot\u27s movements as a reward for reinforcement learning. Finally, we directly transfer the learned policy to a real quadruped Solo. Remarkably, despite the extreme gap in both domain and embodiment between animals in the wild and robots, our approach enables the policy to learn diverse skills such as walking, jumping, and keeping still, without relying on reference trajectories nor skill-specific rewards.Project website: https://elliotchanesane31.github.io/RLWAV
- …
