Reinforcement learning for flight control

The problem

How can aircraft safely operate at the limits of their flight envelope?

Fixed-wing UAVs offer significant advantages in range and efficiency, but operating them in complex or confined environments presents a major challenge. Tasks such as landing in restricted spaces or performing aggressive manoeuvres require aircraft to operate in highly nonlinear flight regimes, where conventional control techniques can struggle to deliver the required performance.

One example is a perched landing, inspired by the way birds rapidly decelerate before landing. These manoeuvres require precise control while exploiting the full flight envelope of the aircraft. Designing controllers capable of reliably performing such manoeuvres remains a significant aerospace engineering challenge.

The increasing availability of machine learning techniques provides new opportunities to address these problems. However, reinforcement learning systems typically rely on large amounts of training data generated in simulation, and controllers that perform well in a virtual environment do not always transfer successfully to real aircraft. This difference between simulated and real-world performance is known as the "reality gap".

Our solution

Using artificial intelligence to develop advanced flight controllers

Professor Tom Richardson and colleagues in the Bristol Flight Lab are applying reinforcement learning to develop flight control systems for fixed-wing UAVs operating in challenging flight conditions. Rather than being programmed with predefined responses, reinforcement learning agents learn control strategies through repeated interaction with a simulated environment.

A major focus of the research has been the development of controllers capable of performing bird-inspired perched landings. Through simulation and flight testing, the team has demonstrated that reinforcement learning can discover effective control strategies for these highly nonlinear manoeuvres while maintaining applicability to real-world aircraft operations.

To address the reality gap, researchers have investigated techniques including domain randomisation, improved state representations and optimisation of neural network architectures. By exposing reinforcement learning agents to a wide range of atmospheric disturbances during training, the team demonstrated significant improvements in controller robustness and real-world landing performance.

Building on this work, the research expanded beyond perched landings to aggressive high-angle-of-attack manoeuvres for aircraft operating in confined environments. By exploiting highly nonlinear aerodynamic behaviour near the limits of the flight envelope, reinforcement learning was used to generate agile manoeuvres that would be difficult to design using traditional control approaches alone.

Results and outcomes

The research has advanced understanding of how artificial intelligence can be applied to aerospace control systems, including:

• demonstrating reinforcement learning-based control for bird-inspired perched landings in fixed-wing aircraft
• developing techniques to improve the transfer of learned controllers from simulation to real-world flight testing
• improving controller robustness by incorporating atmospheric disturbances during the training process
• enabling aggressive high-angle-of-attack manoeuvres that exploit nonlinear aerodynamic behaviour
• demonstrating how fixed-wing UAVs can combine efficient cruise flight with highly agile terminal manoeuvres
• providing evidence that reinforcement learning can complement established model-based control methods across a wide range of flight conditions.

The work highlights the growing potential for machine learning to enhance aircraft autonomy and expand the operational capabilities of future aerial vehicles.

Looking ahead

As autonomous aircraft become increasingly important across aerospace applications, future control systems will need to operate safely and reliably in complex and uncertain environments. Reinforcement learning offers a promising route to achieving this goal, enabling aircraft to exploit the full extent of their flight envelope while maintaining robust performance. Ongoing research is exploring how artificial intelligence can be combined with established flight-control methods to support the next generation of autonomous air vehicles.