iframe header
  • Join FlightAware (Why Join?)
  • Login
  • US Flag 
06:16AM EDT

All times are UTC - 6 hours [ DST ]




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Missing Data?
PostPosted: Fri May 25, 2012 9:23 am 
Offline

Joined: Fri May 25, 2012 2:18 am
Posts: 3
Hi,

I think I've stumbled over an issue with the data being returned by GetLastTrack and GetHistoricalTrack for what is an historical flight.

I'm calling FlightInfoEx for ident NH202 as follows
http://flightxml.flightaware.com/json/F ... 0&offset=0
As expected this returns me the information for 10 flights.

I then make a call to GetLastTrack for the same ident
http://flightxml.flightaware.com/json/F ... dent=NH202
This returns me 745 rows of data. As an aside it would be great if this method also returned the faFlightID :D . As it is at the time of writing the data being returned appears to be for faFlightID ANA202-1337664533-airline-0311.

To verify this I made the following call using the GetHistoricalTrack to return the historical tracking data for the same faFlightID
http://flightxml.flightaware.com/json/F ... rline-0311
This only returns me 62 rows of data. The timestamp on the first 62 rows matches the first 62 rows of data returned by GetLastTrack so I'm fairly certain this is the same flight. I would however expect to have recieved the same 745 rows of data using this method.

Am I correct in assuming this is a bug or am I using the wrong methods?


Top
 Profile  
 
 Post subject: Re: Missing Data?
PostPosted: Fri May 25, 2012 10:38 am 
Offline
FlightAware Staff

Joined: Wed Jul 27, 2005 3:00 am
Posts: 234
Location: FlightAware WHQ
FlightXML’s GetHistoricalTrack intentionally returns only actual data points, which means that approximated positions that have been synthesized based on extrapolations of movement are not included. This design choice was originally made in order to ensure that people making use of our FlightXML data for research or analysis purposes are not mislead by our synthetically generated data, since it is of lower quality than the other points that represent true measurements. In some cases the synthetic data only represents interpolations between real points for the purposes of smoothing the line drawing on our website.

Meanwhile, GetLastTrack does not do this same exclusion so it is returning the synthetic/projected positions mixed in with the data. This behavior was preserved from FlightXML1 because it was the legacy behavior when it was originally implemented then.

In the future it is possible that we might offer an argument or setting that would allow the synthetic data points to also be included in the results of GetHistoricalTrack if the user explicitly requests them.


Top
 Profile  
 
 Post subject: Re: Missing Data?
PostPosted: Mon May 28, 2012 3:30 am 
Offline

Joined: Fri May 25, 2012 2:18 am
Posts: 3
Hi,

The nature of the missing data in my particular case seems to be more of a truncation of the GetLastTrack data than a subset of it.

For example - using this call for a London to Tokyo flight (the first 'arrived' flight at the time of writing)...
http://flightxml.flightaware.com/json/F ... rline-0147

...I get 57 results - the last of which I've included below
{"timestamp":1338146574,"latitude":53.042701721199996712,"longitude":1.8498200178099999391,"groundspeed":475,"altitude":320,"altitudeStatus":"","updateType":"TA","altitudeChange":""}

I would expect this to be in the vicinity of Tokyo but it is actually still very close to the UK. If I've understood your reply correctly this would imply there are no actual data points beyond the waypoint I included above.
Also (at the tiem of writing) those 57 results match the first 57 results returned by GetLastTrack rather than a subset of 57 actual data points distributed evenly across the flight path.

BOb


Top
 Profile  
 
 Post subject: Re: Missing Data?
PostPosted: Mon May 28, 2012 6:17 pm 
Offline
FlightAware Staff

Joined: Wed Jul 27, 2005 3:00 am
Posts: 234
Location: FlightAware WHQ
bobpardoe wrote:
Hi,

The nature of the missing data in my particular case seems to be more of a truncation of the GetLastTrack data than a subset of it.

For example - using this call for a London to Tokyo flight (the first 'arrived' flight at the time of writing)...
http://flightxml.flightaware.com/json/F ... rline-0147

...I get 57 results - the last of which I've included below
{"timestamp":1338146574,"latitude":53.042701721199996712,"longitude":1.8498200178099999391,"groundspeed":475,"altitude":320,"altitudeStatus":"","updateType":"TA","altitudeChange":""}

I would expect this to be in the vicinity of Tokyo but it is actually still very close to the UK. If I've understood your reply correctly this would imply there are no actual data points beyond the waypoint I included above.
Also (at the tiem of writing) those 57 results match the first 57 results returned by GetLastTrack rather than a subset of 57 actual data points distributed evenly across the flight path.

BOb


If you look at the website's presentation of that positional data at http://flightaware.com/live/flight/ANA2 ... A/tracklog you will see that once it leaves our European coverage area, the position is entirely estimated. This is consistent with the "truncation" that you are seeing. If you zoom in on the flight map for http://flightaware.com/live/flight/ANA2 ... /EGLL/RJAA you will see that the green line that represents the actual flight data only extends a few minutes from Heathrow and is also truncated once it leaves our coverage area.

FlightAware receives flight information in varying levels of quality, which does not always include positional information--for example, sometimes airlines might simply send us confirmation of departure or arrival times without revealing the latitude/longitude of the airplane.


Top
 Profile  
 
 Post subject: Re: Missing Data?
PostPosted: Tue May 29, 2012 9:02 am 
Offline

Joined: Fri May 25, 2012 2:18 am
Posts: 3
Thank you for clarifying that.

The reason I was using GetHistoricalTrack is that we were planning on using the last waypoint data for a given flight so we could plot these on a map. In addition we were going to overlay the waypoints from the current airborne flight (if there is one at the time the request is made) on top of this. - ie to show the "last route" or "last vs current"

In fact this is very similar to the tracking I can see on your website - for example using your tracking of ident VIR20 using your website tracking - http://flightaware.com/live/flight/VIR20 I can see you're plotting a solid green line and a dotted blue line representing the flight route. I've tried using various methods to retrieve enough data to reproduce an equivalent of that map (for the same ident) but I just can't seem to get any data back. For example

GetLastTrack should return the active flight or last flight
Code:
http://flightxml.flightaware.com/json/FlightXML2/GetLastTrack?ident=VIR20

returns
{"error":"no data available"}

So I grabbed and stored the faFlightIds from FlightInfoEx - http://flightxml.flightaware.com/json/F ... 0&offset=0
Then using only those returned faFlightIDs where the flight has an actualArrivalTime...

  • all calls to GetHistoricalTrack return no data - eg.
    Code:
    http://flightxml.flightaware.com/json/FlightXML2/GetHistoricalTrack?faFlightID=VIR20-1338096548-airline-0017

    returns
    {"error":"no data available"}
  • all calls to DecodeFlightRoute return no data - eg.
    Code:
    http://flightxml.flightaware.com/json/FlightXML2/DecodeFlightRoute?faFlightID=VIR20-1337837314-airline-0017

    returns
    {"error":"unable to decode"}

What methods would you recommend I use to provide me with enough data to produce an equivalent of your tracking map?


Top
 Profile  
 
 Post subject: Re: Missing Data?
PostPosted: Tue May 29, 2012 2:18 pm 
Offline
FlightAware Staff

Joined: Wed Jul 27, 2005 3:00 am
Posts: 234
Location: FlightAware WHQ
bobpardoe wrote:
GetLastTrack should return the active flight or last flight
Code:
http://flightxml.flightaware.com/json/FlightXML2/GetLastTrack?ident=VIR20

returns
{"error":"no data available"}



There is a certain period of time after the last flight where GetLastTrack may not return any useful data because that flight has already left the active memory cache used by that function. This is an unavoidable implementation detail that happens to get exposed by the legacy implementation of this function (which was originally from FlightXML1).

bobpardoe wrote:
So I grabbed and stored the faFlightIds from FlightInfoEx - http://flightxml.flightaware.com/json/F ... 0&offset=0
Then using only those returned faFlightIDs where the flight has an actualArrivalTime...

  • all calls to GetHistoricalTrack return no data - eg.
    Code:
    http://flightxml.flightaware.com/json/FlightXML2/GetHistoricalTrack?faFlightID=VIR20-1338096548-airline-0017

    returns
    {"error":"no data available"}




That flight is the May 29 flight for VIR20, which has not even departed yet. Maybe your actualArrivalTime check is not correct. There will not be any historical (or current) track available for it yet. http://flightaware.com/live/flight/VIR2 ... /KSFO/EGLL

bobpardoe wrote:
  • all calls to DecodeFlightRoute return no data - eg.
    Code:
    http://flightxml.flightaware.com/json/FlightXML2/DecodeFlightRoute?faFlightID=VIR20-1337837314-airline-0017

    returns
    {"error":"unable to decode"}

  • What methods would you recommend I use to provide me with enough data to produce an equivalent of your tracking map?


    DecodeFlightRoute currently has a very strict implementation and returns a failure if it is unable to parse or identify any of the waypoints in the planned route. This means it will generally fail with a lot of international flights that are even partially outside of the US airspace. I do agree this is suboptimal, so I will investigate the effort on our side of making DecodeFlightRoute be more lenient in its parsing and just return the points that it is able to successful identify.

    DecodeFlightRoute + GetLastTrack + GetHistoricalTrack is probably the best that you can do, with the caveats that you've already identified... (DecodeFlightRoute will frequently fail on exotic routes, and GetHistoricalTrack only returns actual positions and only for past flights.)


    Top
     Profile  
     
     Post subject: Re: Missing Data?
    PostPosted: Tue May 29, 2012 5:33 pm 
    Offline
    FlightAware Staff

    Joined: Wed Jul 27, 2005 3:00 am
    Posts: 234
    Location: FlightAware WHQ
    Another option is to use InFlightInfo to request the current flight for an ident and use the "waypoints" element to draw the planned route (dashed blue line), and the "latitude" and "longitude" for the current position. Then call GetLastTrack for the actual route (solid green line). Both of those functions are only available for the currently active flight, and not past flights nor for a specific faFlightID.


    Top
     Profile  
     
    Display posts from previous:  Sort by  
    Post new topic Reply to topic  [ 7 posts ] 

    All times are UTC - 6 hours [ DST ]


    Who is online

    Users browsing this forum: No registered users and 0 guests


    You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum

    Search for:
    Jump to:  
    cron
    iframe sidebar


     

    Airport Tracker/Info


    -or-