The Promise of Synthetic Data and Simulation to Accelerate Autonomous Flight
Synthetic data and simulation open up a world of opportunity for engineers in the aviation industry, as it promotes a safe and sustainable way to train and test machine learning (ML) algorithms used to develop artificial intelligence (AI) models for autonomous systems. But why now? What makes synthetic data generation so crucial to our aspiration for more autonomous flight?
Synthetic Data
Aviation is experiencing an ML and AI boom, and data is essential. Data is integral to the training and testing of ML algorithms that can be used to develop AI models, for example. Until recently, much of this data used has been gathered from real-world sources, like in-flight images, but synthetic data is becoming increasingly valuable.
Synthetic data mimics real-world information. The self-driving car industry utilized synthetic data early on to fill gaps where real-world data collection would be difficult or impossible – like extreme or dangerous scenarios. Similarly, synthetic data is being used in aviation, because it is both less costly than gathering real-world data and it can mimic scenarios in which real-world data would be too difficult or too dangerous to collect.
Autonomous Flight
In aviation, synthetic data is widely being used to develop algorithms that can ultimately enable AI functions like autonomous flight. Automated functions, in some form, have been present in aircraft since the first part of the 20th century. Adopting automated technologies has not only led to increasingly sophisticated aircraft, but it has increased operational safety and efficiency. While early autopilot systems were relatively simple, they enabled aircraft to fly straight and level on a course without a pilot’s constant attention and aided pilots by reducing their workload and limiting pilot fatigue.
Automation has since evolved from these early applications and resulted in functions like the computer-enabled fly-by-wire system that many aircraft have been equipped with since the 1980s. Fly-by-wire not only saved space and weight, but further advanced safety and operational efficiency by refining pilot inputs and preventing aircraft from operating beyond performance limits.
Synthetic data likewise contributes to this evolution, further improving flight safety and efficiency. While real-world data is often best, it can be expensive, and it takes time to gather and label. With today’s resources, however, researchers can create synthetic datasets that mimic an array of conditions and scenarios, even unusual corner cases. These datasets can simulate obstacles on the apron or in the sky, variable weather conditions, and light from different angles of the sun. But they could also simulate conditions in which it might be too dangerous for a test pilot to fly, or extraordinary circumstances like a flock of flamingos suddenly obstructing a runway.
A window into Acubed’s Simulation Team
At Acubed, engineers are validating the use of synthetic data as part of their work towards developing more autonomous flight. They apply computer vision technology – interpreting images and 3D signals to perform tasks – to aircraft systems, improving the safety and efficiency of the next generation of aircraft. Using an aircraft equipped with cameras and sensors, Acubed is visiting airports throughout the United States, gathering real-world flight data to train its computer vision system. (Leveraging this technology, Airbus has achieved breakthroughs in both the ATTOL and DragonFly projects, autonomously taxiing, taking off, and landing a commercial aircraft, and demonstrating autonomous emergency operations.)
Utilizing a commercial flight simulator customized with proprietary software and a simulation framework based on Unreal Engine – a leading real-time 3D rendering tool widely used in the video games and film, as well as other fields – the simulation team is creating detailed digital twins and supplementing collected real-world data with synthetic data to more effectively train and test ML algorithms. (A study by Airbus and OneView, for example, showed that synthetic data improved the accuracy of ML algorithms in detecting and identifying different aircraft by 20%.)
By applying different synthetic datasets to real-world data, the team can adjust the simulator to train decision-making capabilities in a range of photorealistic conditions, like different cloud formations or light conditions depending on the sun’s position. Using these tools to create a detailed 3D environment, the team can add incremental complexity to train and test their vision-based flight system in a variety of degraded conditions, including nighttime, low visibility, and non-nominal aircraft attitudes.
Flight Ready
Synthetic data not only enables researchers to train and test ML algorithms in a variety of scenarios, but to do so in conditions that might otherwise be unsafe or prohibited. Extreme weather or aircraft operating beyond performance limits could not be explored without simulation. By simulating these corner cases using synthetic data, researchers can respond to rising air travel demands and safely accelerate the timeline needed to train and test autonomy models, ultimately validating them for certification and commercial use.