The project has resulted in the publications listed below. Use the "»" button to reveal an abstract of each publication, and "«" to hide the abstract again (requires Javascript).
Books
-
L. Busoniu, L. Tamas (editors),
Handling Uncertainty and Networked Structure in Robot Control,
Springer, Series Studies in Systems, Decision and Control. February 2016, ISBN 978-3-319-26327-4.
»
Abstract:
This book focuses on two challenges posed in robot control by the increasing adoption of robots in the everyday human environment: uncertainty and networked communication. Part I of the book describes learning control to address environmental uncertainty. Part II discusses state estimation, active sensing, and complex scenario perception to tackle sensing uncertainty. Part III completes the book with control of networked robots and multi-robot teams.
Each chapter features in-depth technical coverage and case studies highlighting the applicability of the techniques, with real robots or in simulation. Platforms include mobile ground, aerial, and underwater robots, as well as humanoid robots and robot arms.
The text gathers contributions from academic and industry experts, and offers a valuable resource for researchers or graduate students in robot control and perception. It also benefits researchers in related areas, such as computer vision, nonlinear and learning control, and multi-agent systems.
See the book's website at http://rocon.utcluj.ro/roboticsbook/ for additional information about the book and how to obtain it, as well as downloadable material.
«
Journal papers
-
L. Busoniu, A. Daniels, R. Babuska,
Online Learning for Optimistic Planning.
Engineering Applications of Artificial Intelligence,
vol. 55,
pages 60–72,
2016.
»
Abstract: Markov decision processes are a powerful framework for nonlinear, possibly stochastic optimal control. We consider two existing optimistic planning algorithms to solve them, which originate in artificial intelligence. These algorithms have provable near-optimal performance when the actions and possible stochastic next-states are discrete, but they wastefully discard the planning data after each step. We therefore introduce a method to learn online, from this data, the upper bounds that are used to guide the planning process. Five different approximators for the upper bounds are proposed, one of which is specifically adapted to planning, and the other four coming from the standard toolbox of function approximation. Our analysis characterizes the influence of the approximation error on the performance, and reveals that for small errors, learning-based planning performs better. In detailed experimental studies, learning leads to improved performance with all five representations, and a local variant of support vector machines provides a good compromise between performance and computation.
Online at ScienceDirect.
«
-
L. Busoniu, R. Postoyan, J. Daafouz,
Near-optimal Strategies for Nonlinear and Uncertain Networked Control Systems.
IEEE Transactions on Automatic Control,
vol. 61,
no. 8,
pages 2124–2139,
2016.
In press.
»
Abstract: We consider problems where a controller communicates with a general nonlinear plant via a network, and must optimize a performance index. The system is modeled in discrete time and may be affected by a class of stochastic uncertainties that can take finitely many values. Admissible inputs are constrained to belong to a finite set. Exploiting some optimistic planning algorithms from the artificial intelligence field, we propose two control strategies that take into account the communication constraints induced by the use of the network. Both strategies send in a single packet long-horizon solutions, such as sequences of inputs. Our analysis characterizes the relationship between computation, near-optimality, and transmission intervals. In particular, the first strategy imposes at each transmission a desired near-optimality, which we show is related to an imposed transmission period; for this setting, we analyze the required computation. The second strategy has a fixed computation budget, and within this constraint it adapts the next transmission instant to the last state measurement, leading to a self-triggered policy. For this case, we guarantee long transmission intervals. Examples and simulation experiments are provided throughout the paper.
Online at IEEEXplore.
«
-
K. Mathe, L. Busoniu,
Vision and Control for UAVs: A Survey of General Methods and of Inexpensive Platforms for Infrastructure Inspection.
Sensors,
vol. 15,
no. 7,
pages 14887–14916,
2015.
»
Abstract: Unmanned aerial vehicles (UAVs) have gained significant attention in recent years. Low-cost platforms using inexpensive sensor payloads have been shown to provide satisfactory flight and navigation capabilities. In this report, we survey vision and control methods that can be applied to low-cost UAVs, and we list some popular inexpensive platforms and application fields where they are useful. We also highlight the sensor suites used where this information is available. We overview, among others, feature detection and tracking, optical flow and visual servoing, low-level stabilization and high-level planning methods. We then list popular low-cost UAVs, selecting mainly quadrotors. We discuss applications, restricting our focus to the field of infrastructure inspection. Finally, as an example, we formulate two use-cases for railway inspection, a less explored application field, and illustrate the usage of the vision and control techniques reviewed by selecting appropriate ones to tackle these use-cases. To select vision methods, we run a thorough set of experimental evaluations.
Online at MDPI.
«
-
L. Busoniu, C. Morarescu,
Topology-Preserving Flocking of Nonlinear Agents Using Optimistic Planning.
Control Theory and Technology,
vol. 13,
no. 1,
pages 70–81,
2015.
»
Abstract: We consider the generalized flocking problem in multiagent systems, where the agents must drive a subset of their state variables to common values, while communication is constrained by a proximity relationship in terms of another subset of variables. We build a flocking method for general nonlinear agent dynamics, by using at each agent a near-optimal control technique from artificial intelligence called optimistic planning. By defining the rewards to be optimized in a well-chosen way, the preservation of the interconnection topology is guaranteed, under a controllability assumption. We also give a practical variant of the algorithm that does not require to know the details of this assumption, and show that it works well in experiments on nonlinear agents.
Online at CTT.
«
-
L. Busoniu, C. Morarescu,
Consensus for Black-Box Nonlinear Agents Using Optimistic Optimization.
Automatica,
vol. 50,
no. 4,
2014.
»
Abstract: An important problem in multiagent systems is consensus, which requires the agents to agree on certain controlled variables of interest. We focus on the challenge of dealing in a generic way with nonlinear agent dynamics, represented as a black box with unknown mathematical form. Our approach designs a reference behavior with a classical consensus method. The main novelty is using optimistic optimization (OO) to find controls that closely follow the reference behavior. The first advantage of OO is that it only needs to sample the black-box model of the agent, and so achieves our goal of handling unknown nonlinearities. Secondly, a tight relationship is guaranteed between computation invested and closeness to the reference behavior. Our main results exploit these properties to prove practical consensus. An analysis of representative examples builds additional insight and shows that in some nontrivial problems the optimization is easy to solve by OO. Simulations on these examples accompany the analysis.
Online at ScienceDirect.
«
Contributions to books
Conference papers
-
E. Pall, L. Tamas, L. Busoniu,
Analysis and a Home Assistance Application of Online AEMS2 Planning.
Accepted at
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-16),
Daejeon, Korea,
9–14 October
2016.
»
Abstract: We consider an online planning algorithm for partially observable Markov decision processes (POMDPs), called Anytime Error Minimization Search 2 (AEMS2). Despite the considerable success it has enjoyed in robotics and other problems, no quantitative analysis exists of the relationship between its near-optimality and the computation invested. Exploiting ideas from fully-observable MDP planning, we provide here such an analysis, in which the relationship is modulated via a measure of problem complexity called near-optimality exponent. We illustrate the exponent for some interesting POMDP structures, and examine the role of the informative heuristics used by AEMS2 in the guarantees. In the second part of the paper, we introduce a domestic assistance problem in which a robot monitors partially observable switches and turns them off if needed. AEMS2 successfully solves this task in real experiments, and also works better than several state of the art planners in simulation comparisons.
«
-
L. Busoniu, E. Pall, R. Munos,
Discounted Near-Optimal Control of General Continuous-Action Nonlinear Systems Using Optimistic Planning.
In
Proceedings IEEE American Control Conference (ACC-16),
Boston, USA,
6–8 July
2016.
»
Abstract: We propose an optimistic planning method to search for near-optimal sequences of actions in discrete-time, infinite-horizon optimal control problems with discounted rewards. The dynamics are general nonlinear, while the action (input) is scalar and compact. The method works by iteratively splitting the infinite-dimensional search space into hyperboxes. Under appropriate conditions on the dynamics and rewards, we analyze the shrinking rate of the range of possible values in each box. When coupled with a measure of problem complexity, this leads to an overall convergence rate of the algorithm to the infinite-horizon optimum, as a function of computation invested. We provide simulation results showing that the algorithm is useful in practice, and comparing it with two alternative planning methods.
«
-
K. Mathe, L. Busoniu, L. Barabas, L. Miclea, J. Braband, C. Iuga,
Vision-Based Control of a Quadrotor for an Object Inspection Scenario.
In
Proceedings 2016 International Conference on Unmanned Aircraft Systems (ICUAS-16),
pages 849–857,
Arlington, USA,
7–10 June
2016.
»
Abstract: Unmanned aerial vehicles (UAVs) have gained special attention in recent years, among others in monitoring and inspection applications. In this paper, a less explored application field is proposed, railway inspection, where UAVs can be used to perform visual inspection tasks such as semaphore, catenary, or track inspection. We focus on lightweight UAVs, which can detect many events in railways (for example missing indicators or cabling, or obstacles on the tracks). An outdoor scenario is developed where a quadrotor visually detects a railway semaphore and flies around it autonomously, recording a video of it for offline post-processing. For these tasks, we exploit object detection methods from literature, and develop a visual servoing technique. Additionally, we perform a thorough comparison of several object detection approaches before selecting a preferred method. Then, we show the performance of the presented filtering solutions when they are used in servoing, and conclude our experiments with evaluating real outdoor flight trajectories using an AR.Drone 2.0 quadrotor.
Online at IEEEXplore.
«
-
J. Xu, L. Busoniu, T. van den Boom, B. De Schutter,
Receding-Horizon Control for Max-Plus Linear Systems with Discrete Actions Using Optimistic Planning.
In
Proceedings 13th International Workshop on Discrete Event Systems (WODES-16),
pages 398–403,
Xi'an, China,
30 May – 1 June
2016.
»
Abstract: This paper addresses the infinite-horizon optimal control problem for max-plus linear systems where the considered objective function is a sum of discounted stage costs over an infinite horizon. The minimization problem of the cost function is equivalently transformed into a maximization problem of a reward function. The resulting optimal control problem is solved based on an optimistic planning algorithm. The control variables are the increments of system inputs and the action space is discretized as a finite set. Given a finite computational budget, a control sequence is returned by the optimistic planning algorithm. The first control action or a subsequence of the returned control sequence is applied to the system and then a receding-horizon scheme is adopted. The proposed optimistic planning approach allows us to limit the computational budget and also yields a characterization of the level of near-optimality of the resulting solution. The effectiveness of the approach is illustrated with a numerical example. The results show that the optimistic planning approach results in a lower tracking error compared with a finite-horizon approach when a subsequence of the returned control sequence is applied.
Online at IEEEXplore.
«
-
C. Militaru, A.-D. Mezei, L. Tamas,
Object Handling in Cluttered Indoor Environment with a Mobile Manipulator.
In
Proceedings 2016 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR-16),
Cluj-Napoca, Romania,
19–21 May
2016.
-
L. Busoniu, M.-C. Bragagnolo, J. Daafouz, C. Morarescu,
Planning Methods for the Optimal Control and Performance Certification of General Nonlinear Switched Systems.
In
Proceedings 54th IEEE Conference on Decision and Control (CDC-15),
Osaka, Japan,
15–18 December
2015.
»
Abstract: We consider two problems for discrete-time switched systems with autonomous, general nonlinear modes. The first is optimal control of the switches so as to minimize the discounted infinite-horizon sum of the costs. The second problem occurs when switches are a disturbance, and the worstcase cost under any sequence of switches is sought. We use an optimistic planning (OP) algorithm that can solve general optimal control with discrete inputs such as switches. We extend the analysis of OP to provide sequences of switches with certification (upper and lower) bounds on the optimal and worst-case costs, and to characterize the convergence rate of the gap between these bounds. Since a minimum dwell time between switches must often be ensured, we introduce a new optimistic planning variant that can handle this case, and analyze its convergence rate. Simulations for linear and nonlinear modes illustrate that the approach works in practice.
Online at IEEEXplore.
«
-
T. Wensveen, L. Busoniu, R. Babuska,
Real-Time Optimistic Planning with Action Sequences.
In
Proceedings 20th International Conference on Control Systems and Computer Science (CSCS-15),
pages 923–930,
Bucharest, Romania,
27–29 May
2015.
»
Abstract: Optimistic planning (OP) is a promising approach
for receding-horizon optimal control of general nonlinear systems.
This generality comes however at large computational costs,
which so far have prevented the application of OP to the control
of nonlinear physical systems in real-time. We therefore introduce
an extension of OP to real-time control, which applies open-loop
sequences of actions in parallel with finding the next sequence
from the predicted state at the end of the current sequence.
Exploiting OP guarantees, we provide conditions under which
the algorithm is provably feasible in real-time, and we analyze
its performance. We report successful real-time experiments for
the swingup of an inverted pendulum, as well as simulation results
for an acrobot, where the impact of model errors is studied.
Online at IEEEXplore.
«
-
R. Postoyan, L. Busoniu, D. Nesic, J.
Daafouz,
Stability of Infinite-Horizon Optimal Control with Discounted Cost.
In
Proceedings 53rd IEEE Conference on Decision and Control (CDC-14),
pages 3903–3908,
Los Angeles, USA,
15–17 December
2014.
»
Abstract: We investigate the stability of general nonlinear
discrete-time systems controlled by an optimal sequence of
inputs that minimizes an infinite-horizon discounted cost. We
first provide conditions under which a global asymptotic stability
property is ensured for the corresponding undiscounted
problem. We then show that this property is semiglobally
and practically preserved in the discounted case, where the
adjustable parameter is the discount factor. We then focus on
a scenario where the stage cost is bounded and we explain
how our framework applies to guarantee stability in this case.
Finally, we provide sufficient conditions, including boundedness
of the stage cost, under which the value function, which serves
as a Lyapunov function for the analysis, is continuous. As
already shown in the literature, the continuity of the Lyapunov
function is crucial to ensure some nominal robustness for the
closed-loop system.
Online at IEEEXplore.
«
-
K. Mathe, L. Busoniu, R. Munos, B. De Schutter,
Optimistic Planning with a Limited Number of Action Switches for Near-Optimal Nonlinear Control.
In
Proceedings 53rd IEEE Conference on Decision and Control (CDC-14),
pages 3518–3523,
Los Angeles, USA,
15–17 December
2014.
»
Abstract: We consider infinite-horizon optimal control of
nonlinear systems where the actions (inputs) are discrete.
With the goal of limiting computations, we introduce a search
algorithm for action sequences constrained to switch at most
a given number of times between different actions. The new
algorithm belongs to the optimistic planning class originating
in artificial intelligence, and is called optimistic switch-limited
planning (OSP). It inherits the generality of the OP class, so
it works for nonlinear, nonsmooth systems with nonquadratic
costs. We develop analysis showing that the switch constraint
leads to polynomial complexity in the search horizon, in
contrast to the exponential complexity of state-of-the-art OP;
and to a correspondingly faster convergence. The degree of
the polynomial varies with the problem and is a meaningful
measure for the difficulty of solving it. We study this degree
in two representative, opposite cases. In simulations we first
apply OSP to a problem where limited-switch sequences are
near-optimal, and then in a networked control setting where
the switch constraint must be satisfied in closed loop.
Online at IEEEXplore.
«
-
L. Busoniu, R. Munos, Elod Pall,
An Analysis of Optimistic, Best-First Search for Minimax Sequential Decision Making.
In
Proceedings IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL-14),
pages 1–8,
Orlando, USA,
10–13 December
2014.
»
Abstract: We consider problems in which a maximizer and a minimizer agent take actions in turn, such as games or optimal control with uncertainty modeled as an opponent. We extend the ideas of optimistic optimization to this setting, obtaining a search algorithm that has been previously considered as the best-first search variant of the B* method. We provide a novel analysis of the algorithm relying on a certain structure for the values of action sequences, under which earlier actions are more important than later ones. An asymptotic branching factor is defined as a measure of problem complexity, and it is used to characterize the relationship between computation invested and near-optimality. In particular, when action importance decreases exponentially, convergence rates are obtained. Throughout, examples illustrate analytical concepts such as the branching factor. In an empirical study, we compare the optimistic best-first algorithm with two classical game tree search methods, and apply it to a challenging HIV infection control problem.
Online at IEEEXplore.
«
-
L. Busoniu, L. Tamas,
Optimistic Planning for the Near-Optimal Control of General Nonlinear Systems with Continuous Transition Distributions.
In
Proceedings 19th IFAC World Congress (IFAC-14),
Cape Town, South Africa,
24–29 August
2014.
»
Abstract: Optimistic planning is an optimal control approach from artificial intelligence, which can be applied in receding horizon. It works for very general nonlinear dynamics and cost functions, and its analysis establishes a tight relationship between computation invested and near-optimality. However, there is no optimistic planning algorithm that searches for closed-loop solutions in stochastic problems with continuous transition distributions. Such transitions are essential in control, where they arise e.g. due to continuous disturbances. Existing algorithms only search for open-loop input sequences, which are suboptimal. We therefore propose a closed-loop algorithm that discretizes the continuous transition distribution into sigma points, and call it sigma-optimistic planning. Assuming the error introduced by sigma-point discretization is bounded, we analyze the solution returned, showing that it is near-optimal. The algorithm is evaluated in simulation experiments, where it performs better than a state-of-the-art open-loop planning technique; a certainty-equivalence approach also works well.
«
-
E. Pall, K. Mathe, L. Tamas, L. Busoniu,
Railway Track Following with the AR.Drone Using Vanishing Point Detection.
In
Proceedings 2014 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR-14),
Cluj-Napoca, Romania,
22–24 May
2014.
»
Abstract: Unmanned aerial vehicles are increasingly being used and showing their advantages in many domains. However, their application to railway systems is very little studied. In this paper, we focus on controlling an AR.Drone UAV in order to follow the railway track. The method developed relies on vision-based detection and tracking of the vanishing point of the railway tracks, overhead lines, and other related lines in the image, coupled with a controller that adjusts the yaw so as to keep the vanishing point in the center of the image. Simulation results illustrate the method is effective, and are complemented by vanishing-point tracking results on real images.
«
-
K. Mathe, L. Busoniu, L. Miclea,
Optimistic Planning with Long Sequences of Identical Actions for Near-Optimal Nonlinear Control.
In
Proceedings 2014 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR-14),
Cluj-Napoca, Romania,
22–24 May
2014.
»
Abstract: Optimistic planning for deterministic systems (OPD) is an algorithm able to find near-optimal control for very general, nonlinear systems. OPD iteratively builds near-optimal sequences of actions by always
refining the most promising sequence; this is done by adding all possible one-step actions. However, OPD has large computational costs, which might be undesirable in real life applications. This paper proposes an adaptation of OPD for a specific subclass of control problems where control actions do not change often (e.g. bang-bang, time-optimal control). The new algorithm is called Optimistic Planning with K identical actions (OKP), and it refines sequences by adding, in addition to one-step actions, also repetitions of each action up to K times. Our analysis proves that the a posteriori performance guarantees are similar to those of OPD, improving with the length of the explored sequences, though the asymptotic behaviour of OKP cannot be formally predicted a priori. Simulations illustrate that for properly chosen parameter K, in a control problem from the class considered, OKP outperforms OPD.
«
-
L. Busoniu, C. Morarescu,
Consensus for Agents with General Dynamics Using Optimistic Optimization.
In
Proceedings 2013 Conference on Decision and Control (CDC-13),
Florence, Italy,
10–13 December
2013.
»
Abstract: An important challenge in multiagent systems is consensus, in which the agents must agree on certain controlled variables of interest. So far, most consensus algorithms for agents with nonlinear dynamics exploit the specific form of the nonlinearity. Here, we propose an approach that only requires a black-box simulation model of the dynamics, and is therefore applicable to a wide class of nonlinearities. This approach works for agents communicating on a fixed, connected network. It designs a reference behavior with a classical consensus protocol, and then finds control actions that drive the nonlinear agents towards the reference states, using a recent optimistic optimization algorithm. By exploiting the guarantees of optimistic optimization, we prove that the agents achieve practical consensus. A representative example is further analyzed, and simulation results on nonlinear robotic arms are provided.
The downloadable PDF contains an extended version of the proof.
«
Workshop papers
-
L. Busoniu, A. Daniels, R. Munos, R. Babuska,
Optimistic Planning for Continuous-Action Deterministic Systems.
Presented at the 8emes Journees Francophones sur la Planification, la Decision et l'Apprentissage pour la conduite de systemes (JFPDA-13),
Lille, France,
1–2 July
2013.
»
Abstract: We consider the optimal control of systems with deterministic dynamics, continuous, possibly large-scale state spaces, and continuous, low-dimensional action spaces. We describe an online planning algorithm called SOOP, which like other algorithms in its class has no direct dependence on the state space structure. Unlike previous algorithms, SOOP explores the true solution space, consisting of infinite sequences of continuous actions, without requiring knowledge about the smoothness of the system. To this end, it borrows the principle of the simultaneous optimistic optimization method, and develops a nontrivial adaptation of this principle to the planning problem. Experiments on four problems show SOOP reliably ranks among the best algorithms, fully dominating competing methods when the problem requires both long horizons and fine discretization.
This is an extended version of our ADPRL-13 paper with the same title.
«
Master theses
-
C. Iuga,
Vision based quadrotor navigation around objects .
Technical University of Cluj-Napoca,
Romania,
2015.
»
Abstract: This project aims to perform video inspection of railroad semaphores, by approaching them using GPS based navigation and then performing vision-based object detection of the semaphores. We tested and developed solutions for GPS based navigation, and we succesfully achieved object detection using vision-based methods by integrating machine learning concepts. We trained many classifiers to be used in the specific case of semaphore detection and we ran a thorough set of experiments on these classifiers. We highlighted the most relevant parameters and corresponding values, for our highest performance classifier, which achieved a 100% detection rate with no false detections, during the simulations on the test video file.
«
-
E. Pall,
Vision-Based Quadcopter Navigation for Following Indoor Corridors and Outdoor Railways.
Automation Department,
Technical University of Cluj-Napoca,
Romania,
2014.
»
Abstract: Quadcopters are among the most agile mobile robots,
their autonomus navigation is a challenging subject. Low-cost
quadcopters are accessible for civilians, but the control of these robots
must be robust and reliable. This thesis presents two vision-based
applications for autonomous navigation in indoor and outdoor
environments. The first application is a corridor following drone,
which navigates the drone autonomously through indoor and outdoor
hallway-like scenes. The second application is a railway following
drone.
«
-
A. Bulezyuk,
Radio Remote Controlled Aircraft Vehicle development with implementation of an Android device intended for Data Measurements and Object Tracking.
Technical University of Cluj-Napoca and Erasmus University College Brussels,
Belgium,
2013.
-
K. Mathe,
Optimistic Planning with Constant Controls over Multiple Time Steps.
Automation Department,
Technical University of Cluj-Napoca,
Romania,
2013.
»
Abstract: Optimistic Planning for Deterministic systems (OPD) is an optimal control algorithm able to deal with very general dynamics and cost functions, but this generality comes at high computational costs: exponential in the planning horizon. The goal of this thesis is to adapt OPD to the specific class of problems in which long ranges of repeated actions are preferred. Minimum-time, bang-bang control is one example of this class. We design two new planning algorithms adapted to this setting. The first examines in addition to single-step actions also up to K repetitions of each action. This obtains improvements in some problems, but analysis shows the behavior may be poor in general. The second algorithm limits the number of changes in the control action, and has much more promising analytical and empirical results. It requires only polynomial computation time and reaches constrained near-optimal solutions with less computation than OPD.
«
Disclaimer: The following applies to the papers that are directly available for download as PDF files. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each copyright holder. In most cases, these works may not be reposted without the explicit permission of the copyright holder. Additionally, the following applies to IEEE material: Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE.
Additionally, the following applies to IEEE material: Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE.
|