Reinforcement Learning and Planning for Large-Scale Systems

About the project

Many controlled systems, such as robots in open environments, traffic and energy networks, etc. are large-scale: they have many continuous variables. Such systems may also be nonlinear, stochastic, and impossible to model accurately. Optimistic planning (OP) is a recent paradigm for general nonlinear and stochastic control, which works when a model is available; reinforcement learning (RL) additionally works model-free, by learning from data. However, existing OP and RL methods cannot handle the number of variables required in large-scale systems.

Therefore, this project will develop a planning and reinforcement learning framework for large-scale system control. On the OP side, methods will be developed to deal with large-scale actions and next states. An approach that accelerates large-scale OP by integrating RL will also be designed. The methods will be validated theoretically as well as in applications, with an application focus on assistive mobile manipulators.

The project is being funded under the Young Teams program of the Romanian Authority for Scientific Research, via UEFISCDI (project number PNII-RU-TE-2012-3-0040, contract number 58/2013). It takes place at the Automation Department of the Technical University of Cluj-Napoca, Romania, for a duration of four years and a total budget of about 180 000 EUR.

Team

Lucian Busoniu, principal investigator (associated professor, PhD Eng), focuses on the fundamental research aspects of large-scale OP and RL.
Tamas Levente (lecturer, PhD Eng) focuses on the robotics application aspects of the project.
Elod Pall (researcher) focuses on continuous-action methods as well as real-time control and robotics.
Koppany Mathe (PhD student, now graduated) has worked in the project over the period July-September 2013. He has developed a method for accelerating OP in a class of optimal control problems.

Get in touch with the team members or email the PI at lucian [at] busoniu [dot] net.

See also the publications resulting from the project, as well as the international network and researcher exchanges contributing to it, and other impact the project is having. The administrative info page contains more detailed logistical information such as progress reports.