The Short and the Long of It: Aviation Development at Software Speed
In the aviation industry, development cycles can often be measured in years. A new plane design? Seven to nine years. Certification? Three to five years. With safety as the industry’s number one priority, those timelines are logical and welcome.
And yet at Acubed we’re pursuing new innovations at breakneck speeds, speeds more commonly associated with our software-oriented neighbors here in Silicon Valley. In the software industry, development cycles are broken down in sprints that are commonly measured in days or weeks.
Acubed’s Wayfinder team is uniquely combining these two approaches: we are iterating on our software on a weekly cadence, while running whole flight campaigns over the course of several months. We began flying our Beechcraft Baron 58 in July 2020 and concluded our first flight test milestone at the end of 2020.
When we took to the skies, our objectives for this initial flight test campaign were to:
- Demonstrate the feasibility of Wayfinder’s autonomy architecture
- Build and successfully operate a functional hardware and software system
- Develop a suitable test platform for Wayfinder’s algorithm and data-driven process development
Our next frontier? Data volume. Today, the data sets required to build and inform aviation-grade autonomy systems simply don’t exist (the automotive industry has so far been the focus of much of the data acquisition efforts). This means we are creating an asset that is wholly new. Succeeding in the quest for massive aviation data volume requires many flights. That’s why in just half a year, we flew three to four times more than comparative programs would fly in three years.
What does that mean in terms of real numbers? Over the course of the campaign so far, we’ve collected millions of high-quality, real images organized into hundreds of flight phases in and out of available Bay Area airports. These images are being used to validate simulation-generated imagery and improve machine learning algorithms.
For machine learning experts, it’s interesting to think about what shorter development timelines mean in terms of seeing one’s work in action. In other, more traditional machine learning roles, people work towards an outcome such as a model that might perform only slightly better on a given data set. In the case of Wayfinder, each time we fly, the system’s computer is loaded with information that has been captured and analyzed from the previous flight. The practical results of all data from previous flights are seen right then and there. At the end of each flight, our team knows it’s not blindly acquiring more data (no pun intended), but that the information will have an immediate and lasting impact on our future efforts.
This immediacy certainly kept our team engaged during the pandemic. Energy positively snowballed as we got the aircraft modified and flying and deployed new models on the plane itself, which kept the work real for our engineers.
Our first flight test campaign allowed us to demonstrate the feasibility of using real-time imagery to feed machine learning developed algorithms to drive CAT-III level aircraft precision approach guidance. We also developed a data pipeline to efficiently transfer imagery from the airplane to a data center and complete a data-driven development process cycle.
This effort resulted in two important outcomes: meeting stringent accuracy requirements (derived from CAT-III level aircraft precision approach guidance) >90% of the time for the data in the validation set, and a slightly lesser value of the time for the test data set. The validation set was only tested once and was not accessible to the machine learning team during development. While the system as designed and tested did not quite yet meet our accuracy target requirements, it constitutes a solid baseline for further development.
To that end, we have kicked off a new flight test campaign this year and are constantly improving as we go. Progressing step by step, we welcome the challenge as we develop the ground-breaking autonomous systems that will allow our industry to continue to grow.