Vision-based imitation learning in heterogeneous multi-robot systems: Varying physiology and skill

Jeff Allen, John Anderson*, Jacky Baltes

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Imitation learning enables a learner to improve its abilities by observing others. Most robotic imitation learning systems only learn from demonstrators that are similar physically and in terms of skill level. In order to employ imitation learning in a heterogeneous multi-agent environment, we must consider both differences in skill, and physical differences (physiology, size). This paper describes an approach to imitation learning from heterogeneous demonstrators, using global vision. It supports learning from physiologically different demonstrators (wheeled and legged, of various sizes), and self-adapts to demonstrators with varying levels of skill. The latter allows different parts of a task to be learned from different individuals (that is, worthwhile parts of a task can still be learned from a poorly-performing demonstrator). We assume the imitator has no initial knowledge of the observable effects of its own actions, and train a set of Hidden Markov Models to create an understanding of the imitator's own abilities. We then use a combination of tracking sequences of primitives and predicting future primitives from existing combinations of primitives, using forward models to learn abstract behaviors from demonstrations. This approach is evaluated using a group of heterogeneous robots that have been previously used in RoboCup soccer competitions.

Original languageEnglish
Pages (from-to)147-161
Number of pages15
JournalInternational Journal of Automation and Smart Technology
Issue number2
Publication statusPublished - 2012
Externally publishedYes


  • Heterogeneity
  • Hidden markov models
  • Imitation learning
  • Multi-robot systems
  • Robotic soccer

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Human-Computer Interaction
  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Artificial Intelligence


Dive into the research topics of 'Vision-based imitation learning in heterogeneous multi-robot systems: Varying physiology and skill'. Together they form a unique fingerprint.

Cite this