Towards a science of autonomy

The aim of my research is to facilitate the integration of autonomous and robotic systems into the human space. This requires a science of autonomy at the intersection of control theory and cognitive science. Understanding the inferences, decisions and intents formed by human individuals and incorporating them into the autonomous decision and control process will be key in enabling the intelligent systems of the future to interact with people in an intuitive and reliable way.


The reach-avoid game

A STARMAC quadrotor

A STARMAC quadrotor during a flight test in the Robotics Lab at UC Berkeley.

Fundamental to the study and design of autonomous and semi-autonomous systems is the field of optimal control and dynamic game theory. In particular, the problem of simultaneously ensuring safety and performance (found in robot motion planning, air traffic management and unmanned vehicle control) can be posed as a reach-avoid dynamic game: we seek to take the system to some desired conditions without falling into any forbidden configurations, under the action of a malicious agent that tries to frustrate our goals. This malicious agent represents the worst-case turn of events given all our uncertainty about the real-world system: model inaccuracies, exogenous disturbances and possibly the actions of other agents in the environment.

These problems lend themselves to the extremely powerful dynamic programming approach, which in continuous time and space takes the form of the Hamilton-Jacobi-Bellman and Hamilton-Jacobi-Isaacs equations, and their study leads to reachability analysis and viability theory. However, in the general case where the system dynamics, targets and constraints depend on time, solving the reach-avoid game is a theoretically challenging and computationally expensive problem. Much of my recent work has been focusing on extending the Hamilton-Jacobi formulation to solve the reach-avoid game in the time-varying case without incurring impractical computational explosion.


Multi-agent systems and the scalability problem

An important challenge in the field of robotics and control is finding viable ways of providing safety and performance guarantees in complex multi-agent systems, like robot teams or traffic networks. Unfortunately, state-of-the-art solution methods for Hamilton-Jacobi equations tend to scale exponentially with the dimensionality of the continuous system, and therefore multi-agent problems (whose dimension is proportional to the number of agents) pose a daunting task for reachability analysis.

Thanks to our recent ability to solve time-varying reach-avoid games, we can now devise methods to effectively analyze large multi-agent problems. The core principle is to start by solving a small problem with, say, one or two players, and then incrementally add new players to each computed solution; new players assume that the trajectories of previous agents have been fixed, inducing moving targets or obstacles in the new game. This allows finding a solution to the large problem by sequentially building up on problems of tractable size. The great appeal of this technique is that, while its solution would generally (although not in all relevant cases!) be suboptimal, the performance and safety guarantees obtained from it would be fully applicable. This way, we can smartly combine these smaller problems to achieve slightly conservative but fully correct solutions to problems that used to be completely intractable. One important practical use of this work is the autonomous planning and execution of guaranteed conflict-free trajectories for large numbers of unmanned aircraft systems in shared airspace, which will become a crucial need in the coming years.


Safe learning-based control

Learning in the safe set

Trajectory in position-velocity space of a quadrotor learning to track a vertical reference while remaining inside its safe set.

With the goal of safe robot-human interaction in mind, I am working in safe learning-based control, which combines the outstanding results of online machine learning with reliable safety guarantees based on reachability analysis. This technique makes it possible to extend the use of reinforcement learning and other learning-based algorithms to safety-critical systems (i.e. where we can’t afford to have our learning robot “try the wrong thing and crash”). This includes contexts like collaborative human-robot manipulation, robotic surgery, unmanned flight and autonomous driving.


Multi-world control

Another area that I am starting to explore is hierarchical control and multi-world reasoning for complex autonomous systems. The design of integrative control schemes that will achieve high-level goals on some discrete abstract representation layer (e.g. drive from San Francisco to Los Angeles) while satisfying constraints in the continuous physical world (say, don’t crash into anyone) is an open research problem that requires delving into the fundamental properties of hybrid dynamical systems and brims with exciting questions and opportunities.


Let humans join the game

I am beginning to investigate principled ways of applying the knowledge and insights from psychology, cognitive science and neurobiology to the study and design of multi-agent systems comprising both humans and autonomous agents. These systems will become increasingly ubiquitous and complex over the next few decades, and understanding how they work will be one of the keys to technological and social progress in the 21st Century.

Robotics in the human space