Missing Data?


#1

Hi,

I think I’ve stumbled over an issue with the data being returned by GetLastTrack and GetHistoricalTrack for what is an historical flight.

I’m calling FlightInfoEx for ident NH202 as follows
flightxml.flightaware.com/json/F … 0&offset=0
As expected this returns me the information for 10 flights.

I then make a call to GetLastTrack for the same ident
flightxml.flightaware.com/json/F … dent=NH202
This returns me 745 rows of data. As an aside it would be great if this method also returned the faFlightID :smiley: . As it is at the time of writing the data being returned appears to be for faFlightID ANA202-1337664533-airline-0311.

To verify this I made the following call using the GetHistoricalTrack to return the historical tracking data for the same faFlightID
flightxml.flightaware.com/json/F … rline-0311
This only returns me 62 rows of data. The timestamp on the first 62 rows matches the first 62 rows of data returned by GetLastTrack so I’m fairly certain this is the same flight. I would however expect to have recieved the same 745 rows of data using this method.

Am I correct in assuming this is a bug or am I using the wrong methods?


#2

FlightXML’s GetHistoricalTrack intentionally returns only actual data points, which means that approximated positions that have been synthesized based on extrapolations of movement are not included. This design choice was originally made in order to ensure that people making use of our FlightXML data for research or analysis purposes are not mislead by our synthetically generated data, since it is of lower quality than the other points that represent true measurements. In some cases the synthetic data only represents interpolations between real points for the purposes of smoothing the line drawing on our website.

Meanwhile, GetLastTrack does not do this same exclusion so it is returning the synthetic/projected positions mixed in with the data. This behavior was preserved from FlightXML1 because it was the legacy behavior when it was originally implemented then.

In the future it is possible that we might offer an argument or setting that would allow the synthetic data points to also be included in the results of GetHistoricalTrack if the user explicitly requests them.


#3

Hi,

The nature of the missing data in my particular case seems to be more of a truncation of the GetLastTrack data than a subset of it.

For example - using this call for a London to Tokyo flight (the first ‘arrived’ flight at the time of writing)…
flightxml.flightaware.com/json/F … rline-0147

…I get 57 results - the last of which I’ve included below
{“timestamp”:1338146574,“latitude”:53.042701721199996712,“longitude”:1.8498200178099999391,“groundspeed”:475,“altitude”:320,“altitudeStatus”:"",“updateType”:“TA”,“altitudeChange”:""}

I would expect this to be in the vicinity of Tokyo but it is actually still very close to the UK. If I’ve understood your reply correctly this would imply there are no actual data points beyond the waypoint I included above.
Also (at the tiem of writing) those 57 results match the first 57 results returned by GetLastTrack rather than a subset of 57 actual data points distributed evenly across the flight path.

BOb


#4

If you look at the website’s presentation of that positional data at flightaware.com/live/flight/ANA2 … /EGLL/RJAA you will see that the green line that represents the actual flight data only extends a few minutes from Heathrow and is also truncated once it leaves our coverage area.

FlightAware receives flight information in varying levels of quality, which does not always include positional information–for example, sometimes airlines might simply send us confirmation of departure or arrival times without revealing the latitude/longitude of the airplane.


#5

Thank you for clarifying that.

The reason I was using GetHistoricalTrack is that we were planning on using the last waypoint data for a given flight so we could plot these on a map. In addition we were going to overlay the waypoints from the current airborne flight (if there is one at the time the request is made) on top of this. - ie to show the “last route” or “last vs current”

In fact this is very similar to the tracking I can see on your website - for example using your tracking of ident VIR20 using your website tracking - flightaware.com/live/flight/VIR20 I can see you’re plotting a solid green line and a dotted blue line representing the flight route. I’ve tried using various methods to retrieve enough data to reproduce an equivalent of that map (for the same ident) but I just can’t seem to get any data back. For example

GetLastTrack should return the active flight or last flight


http://flightxml.flightaware.com/json/FlightXML2/GetLastTrack?ident=VIR20

returns
{"error":"no data available"}


So I grabbed and stored the faFlightIds from FlightInfoEx - flightxml.flightaware.com/json/F … 0&offset=0
Then using only those returned faFlightIDs where the flight has an actualArrivalTime

  • all calls to GetHistoricalTrack
    return no data - eg.

http://flightxml.flightaware.com/json/FlightXML2/GetHistoricalTrack?faFlightID=VIR20-1338096548-airline-0017

returns
{"error":"no data available"}


  • all calls to DecodeFlightRoute
    return no data - eg.

http://flightxml.flightaware.com/json/FlightXML2/DecodeFlightRoute?faFlightID=VIR20-1337837314-airline-0017

returns
{"error":"unable to decode"}

What methods would you recommend I use to provide me with enough data to produce an equivalent of your tracking map?


#6

There is a certain period of time after the last flight where GetLastTrack may not return any useful data because that flight has already left the active memory cache used by that function. This is an unavoidable implementation detail that happens to get exposed by the legacy implementation of this function (which was originally from FlightXML1).

DecodeFlightRoute currently has a very strict implementation and returns a failure if it is unable to parse or identify any of the waypoints in the planned route. This means it will generally fail with a lot of international flights that are even partially outside of the US airspace. I do agree this is suboptimal, so I will investigate the effort on our side of making DecodeFlightRoute be more lenient in its parsing and just return the points that it is able to successful identify.

DecodeFlightRoute + GetLastTrack + GetHistoricalTrack is probably the best that you can do, with the caveats that you’ve already identified… (DecodeFlightRoute will frequently fail on exotic routes, and GetHistoricalTrack only returns actual positions and only for past flights.)


#7

Another option is to use InFlightInfo to request the current flight for an ident and use the “waypoints” element to draw the planned route (dashed blue line), and the “latitude” and “longitude” for the current position. Then call GetLastTrack for the actual route (solid green line). Both of those functions are only available for the currently active flight, and not past flights nor for a specific faFlightID.