Relevant Data Beats Big Data

AIRPORT0030925_480It was day after Christmas 2015. I was in an Iberian Airlines flight circling over London Heathrow airport, notorious for being one of the most congested airports in the world. As I saw the flight tracker on the screen it kept making those nice long ovals around the airport waiting for its turn land.

It is by no means an isolated event and likely you have experienced the same several times. We all can relate to the uncomfortable feeling, shuffling in seats hoping we won’t miss the connecting flight to Denver or Chicago. We sit there thinking about the times we had to dash through the terminal, catch a bus, a train and another long dash to Terminal 5.

The problem?

Strong headwinds are the biggest single cause of delay minutes at Heathrow, more than the notorious London fog.

Following the standard air traffic metric of distance of separation between planes, as they approach the runway the planes are separated during their final approach by a distance of 3-7 miles. As physics tells us, with strong headwinds it takes longer for planes to traverse same distance as it does without headwinds. So the throughput, or number of planes landed per hour, drops 30% from a normal day. All these minutes added to cascading delays, planes circling airports, flight cancellations, planes waiting on runways and lost productivity.

The solution? Big Data?
There is considerable amount of data generated just at the airport when planes are about to land. Each plane sends location information. There are sensors on planes that continuously collect wind and weather information. The control tower has all the planes mapped out, collecting continuous data on motion, location, and  relative position. The control tower has its own wind and weather sensors, High Definition cameras and Pan-Tilt-Zoom cameras, surveillance and meteorological sensors, microphones, signal light guns and other devices.

This data has Volume, Variety and Velocity and by that definition is Big Data. Could that help avoid flight delays and allay my fears of missing connecting flights? After all isn’t value in all this data waiting to be unlocked?

Well Heathrow Airport did solve the delay problem. Their data analysis and solution dramatically cut down delays and brought back throughput almost to the standard levels without the wind. Data helped. Analysis helped. But not the way you think and definitely not Big Data as we know it or a 1000 compute node cluster.

Their analysis started with looking at the current metric of separation which is based on distance. Since the time of traversal of this distance becomes a variable with headwinds, air-traffic controllers switched from measuring miles to measuring minutes. That is minutes of separation between planes at their relative velocities.


Once you set the minutes, software computes the correct distance of separation and conveys to planes. This results in same time between landings with headwinds as normal days and cut down 50% of delays.


Well did the big data not help at all? When the metric was changed from miles to minutes the unaddressed question was, “Was it safe for planes to follow closer than they did before?”. The NATS, the air traffic management company, did have access to five years of data on wake vortices, the turbulence that forms behind a plane. With the specific question in hand they looked at the data on dissipation of vortices in strong winds.

NATS has studied over 158,000 flights using state of the art equipment to accurately measure the behavior of aircraft wake vortices in strong headwinds. The results show that they dissipate more quickly in windy conditions, therefore allowing aircraft to be closer together on final approach.This specific data on dissipation answered the question on safety.

So they started with a specific question (hypothesis if you will) and looked at right sized relevant data to get answers. This is not big data by a long shot – they did not store everything, went dredging in it hoping to find something interesting.

What they found at Heathrow airport can now be rolled out across the world resulting in many million miles and fuel wasted and wasted productivity from cancellations. They did unlock value but not from petabytes of data.

They did this because of relevant data. Not big data.