Beyond the Image: How Context Drives Smarter AI at Acubed
Many of Acubed’s innovations revolve around one powerful idea: computer vision, which is teaching computers how to see. Human vision isn’t just about seeing what’s around them, it’s the ability to almost instantly interpret what is seen. How do we teach this to a computer? In order to get the full picture, we need context.
To understand, let’s start with a few basic examples. Looking at the images below, can you determine which ones include a real child?
Just by glancing, it’s obvious that the last two images feature a real child, while the first is a mural. But why are these so easy to categorize? How do we know that it is a child in a robot suit and not a robot? How do we know instantly that it is a mural on a building, and not an actual child? Much of the answer is in the context.
The first image meets all the usual expectations for a child, so the rest of the image - the context - isn’t required in order to be confident in accurate identification. For the second image, the background (a typical autumn scene), the costume-like appearance of the robot and the playful pose all indicate that this is indeed a child who just happens to be wearing a robot costume, likely for Halloween. Lastly, for the mural, the proportions of the child’s body relative to surrounding features like the different sky background of the mural compared to sky behind the buildings confirm it’s not a real child. For a human, this kind of context is understood instinctively; we take it in and draw conclusions without even thinking about it.
When training a computer to see, we don’t usually focus on the context. The tendency is to focus on key features that are critical in identifying an object (or person). Features like eyes, nose, arms, legs, etc. This can result in missed detections (seeing the robot, not the child), or false positive detections (seeing a child, not a mural). Knowledge about the image context is critical for explaining failure modes, guiding development and understanding edge cases.
Why does this matter?
Acubed follows a data-driven approach, every design and development decision is informed by what we learn from the data itself. This process is iterative: understanding the data we have helps us identify what data we still need. Then we complete the cycle by collecting, curating, training new models and evaluating the results in order to further refine the system. Whether we’re building systems for autonomous taxiing, Foreign Object Detection (FOD) detection or factory oversight, context helps us identify edge cases, explain failures and improve performance. By training our models on real-world data with rich environmental detail, we’re able to develop smarter, more reliable solutions that reflect how the world actually works.
Now, let’s dive deeper and how this relates to the capabilities our work impacts such as vision-based landing. Aircraft operate in a wide variety of conditions, think day and night. To ensure reliable performance we must not only capture that data, but understand and characterize those conditions with metadata to be confident that the system behaves as it should in all environments. The contextual metadata attached to each image helps explain why data collected by Acubed’s Flight Lab, even at the same airport and runway, can produce drastically different results. Analyzing this contextual metadata across our data lake informs future collection flights to ensure we collect data where we need it most, whether that’s filling gaps in test data or supplementing training data where a performance boost is needed. By balancing datasets with image context, we reduce bias and ensure our models perform reliably across seasons, times of day, geographic regions and other variables.
Take the below images for example, both represent the same airport, same runway and the same time of day, but during different times of the year. If all of our data was collected during the summer, there would be no way to judge performance during the winter months. Expanding data collection across seasons allows us to fill in context gaps in our data lake, helping to shed light on less obvious sources of bias that could affect model performance.
As understanding of the problem space deepens, additional context is continually added to the data. New techniques may provide fresh perspectives, while ongoing analysis can reveal unexpected dependencies. This growing contextual insight strengthens comprehension of model performance and supports the development of safe, reliable new capabilities.
If you are interested in joining our team, check out our job postings here.