My server monitors aircraft within a LATLON box using SEARCH api once every five minutes. For each aircraft returned by the search, it stores the time and the faFlightID in a database. It then waits 5 hours (arbitrary delay chosen by me) and issues GetHistoricalTrack API for each of the faFlightIDs. Mainly, this works and returns the full track of the flight which by that time has landed some hours ago.
But for some flights, GetHistoricalTrack returns a SOAP error “no data available”. In that case, my server repeats the GetHistoricalTrack every five minutes until it gets a proper answer (or I intervene to stop it). Right now it is getting “no data available” for four flights as follows:-
[28-Jul-2015 19:15:02 UTC] fagrab: SOAP error getting data for fid=1125 flightcode='EXS605' zwhen=2015-07-28 06:49:44 faID='EXS605-1437891300-schedule-0020:0'"no data available"
[28-Jul-2015 19:15:03 UTC] fagrab: SOAP error getting data for fid=1129 flightcode='EXS685' zwhen=2015-07-28 07:34:41 faID='EXS685-1437893100-schedule-0282:0'"no data available"
[28-Jul-2015 19:15:03 UTC] fagrab: SOAP error getting data for fid=1144 flightcode='BEE861' zwhen=2015-07-28 09:49:51 faID='BEE861-1437890400-schedule-0188:0'"no data available"
[28-Jul-2015 19:15:03 UTC] fagrab: SOAP error getting data for fid=1153 flightcode='EXS170' zwhen=2015-07-28 11:54:00 faID='EXS170-1437903000-schedule-0077'"no data available"
[28-Jul-2015 19:20:01 UTC] fagrab: SOAP error getting data for fid=1125 flightcode='EXS605' zwhen=2015-07-28 06:49:44 faID='EXS605-1437891300-schedule-0020:0'"no data available"
[28-Jul-2015 19:20:01 UTC] fagrab: SOAP error getting data for fid=1129 flightcode='EXS685' zwhen=2015-07-28 07:34:41 faID='EXS685-1437893100-schedule-0282:0'"no data available"
[28-Jul-2015 19:20:02 UTC] fagrab: SOAP error getting data for fid=1144 flightcode='BEE861' zwhen=2015-07-28 09:49:51 faID='BEE861-1437890400-schedule-0188:0'"no data available"
[28-Jul-2015 19:20:03 UTC] fagrab: SOAP error getting data for fid=1153 flightcode='EXS170' zwhen=2015-07-28 11:54:00 faID='EXS170-1437903000-schedule-0077'"no data available"
The zwhen field in my server’s error message is the time the flight was returned by the SEARCH.
Meanwhile, if I use the FA web site to find those flights and then click on “track log and graph” it shows good ADS-B tracks for EXS605 and EXS685. For EXS170 all the points are “FA estimated” so a SOAP error on that one may be sensible. For BEE861 all the points are “FlightAware UK” with no ADS-B shown although the track looks good. All four flights have landed some hours ago.
I am wondering why this is happening, especially for the two where the live FA web site shows good ADS-B tracks.
Bump. Would appreciate an answer from FlightAware on this. Below are some more that failed recently. Could it be that any track which has at least one point marked “estimated” gives the “no data available” error, even though it has lots of valid ADS-B points and the FA web site will display its track? (e.g. this one
flightaware.com/live/flight/TOM4 … N/tracklog )
"2015-07-28 06:49:44", "EXS605-1437891300-schedule-0020:0"
"2015-07-28 07:34:41", "EXS685-1437893100-schedule-0282:0"
"2015-07-28 09:49:51", "BEE861-1437890400-schedule-0188:0"
"2015-07-28 11:54:00", "EXS170-1437903000-schedule-0077"
"2015-07-28 17:57:50", "BEE805-1437931500-schedule-0014:2"
"2015-07-29 06:19:55", "RYR3513-1438128163-fa-0096:0"
"2015-07-29 06:49:32", "RYR8132-1437977400-schedule-0124:0"
"2015-07-29 06:59:37", "RYR8162-1437978300-schedule-0146:0"
"2015-07-29 09:49:33", "TOM48-1438134846-ed-0000:0"
"2015-07-29 11:09:50", "BAW2203-1437993000-schedule-0335:0"
"2015-07-29 19:49:00", "EXS170-1438017300-schedule-0169"
"2015-07-30 05:58:00", "BEE1301-1438061700-schedule-0093:6"
"2015-07-30 06:54:54", "RYR8132-1438063800-schedule-0044:0"
"2015-07-30 06:59:45", "EZY3083-1438063800-schedule-0042:0"
"2015-07-30 09:04:00", "BEE802-1438072200-schedule-0161:2"
"2015-07-30 09:23:51", "TOM92-1438219381-ed-0008:0"
"2015-07-30 11:54:00", "EXS170-1438075500-schedule-0180"
Not all tracks can be reliably retrieved through FlightXML currently. Our website does not directly use FlightXML for accessing its data, so the results displayed there do not always match what is possible to access via FlightXML.
Thanks. In past two days 8 of 175 flights monitored by my server have not been accessible via GetHistoricalTrack, so it is not too serious a problem. These are ordinary scheduled airline flights, so no security issue arises. I hope that in due course it will be possible to improve the API such that the Get HistoricalFlight API works - after the plane has landed - for all flights returned by Search API (possibly excluding those where the track is entirely “FA estimated”).
I’ve just analysed the 13-day period from 14th to 26th September. During this time my server issued 2794 GetHistoricalTrack APIs, and 563 of them returned the error “No data available”. That is a failure rate of 20%.
A list of the faFlightIDs that failed is in the third column of this CSV http://www.brisk.org.uk/pics/GetHistoricalTrack_fails.csv
My server issues these requests 24 hours after it is told (by SEARCH API) of an airborne flight within a specified lat/lon box. It issues the request three times, at one hour intervals, before recording a failure.
I really do consider that this rate of failure of GetHistoricalTrack is much too high.
Can you give any advice as to how it could be reduced?
Does that list represent flights that always failed on all attempts, or just failed on at least one of the times you made the request?
Also keep in mind that a “no data available” return doesn’t necessary mean that an error has occurred. It can also mean that there are simply no actual lat/lon data positions available for that flight (ie: the flight is in a coverage area where we do not receive positions, the flight never occurred, or we only received departure/arrival times and no positions).
However, in a number of those failures do appear to be with a mismatched faFlightID “fork” that cannot be retrieved. FlightXML usually tries to hide the complexity of the “forks” but there’s something unintended going on. As a temporary workaround, you might try calling GetHistoricalTrack using “ident@departureTime” as the argument (with the departureTime being the integer epoch timestamp). I think that should allow a large number of those failed requests to succeed, but we’re investigating making lookups using the slightly-wrong faFlightID fork work automatically for you.
Thanks for quick response. I only put a flight into my table of failures after all three attempts have failed.
Re. your second para, I only know about each of these flights because a SEARCH API, conducted 24-hours earlier, over a lat/lon box covering part of southern UK, returned the flight. Thus it automatically excludes flights which never took off, and very unliklely no ADS-B reports in view of position of box:
$params = array("query" => '-latlong "50.53 -2.15 51.15 -1.0"', '-inAir' => 1, "howMany" => 15, "offset" => 0);
$result = $client->Search($params);
Re. your third para, I’m not sure what is meant by a “fork”. All I know is that the faFlightID is given me by SEARCH results. I can easily change the program to use ident@departureTime, but will take a week or so to see results as I don’t currently store that info.
Update: I changed my server and it’s just now doing getHistoricalTrack on flights for which I have stored a departureTime. The rate of failure seems to be about the same:-
[03-Oct-2015 11:00:02 UTC] Error flightcode='BEE1410' zwhen=2015-10-02 06:04:53 faID='BEE1410-1443590826-airline-0299:0' depTime=1443765779 param="BEE1410@1443765779" "flight not found"
[03-Oct-2015 11:00:04 UTC] Error flightcode='EXS831' zwhen=2015-10-02 06:24:00 faID='EXS831-1443591600-schedule-0000' depTime=1443765660 param="EXS831@1443765660" "no data available"
[03-Oct-2015 11:00:04 UTC] Error flightcode='BCY1616' zwhen=2015-10-02 06:34:00 faID='BCY1616-1443590400-schedule-0000' depTime=1443765480 param="BCY1616@1443765480" "no data available"
[03-Oct-2015 11:00:05 UTC] Error flightcode='BEE760' zwhen=2015-10-02 06:47:00 faID='BEE760-1443591900-schedule-0001:6' depTime=1443767364 param="BEE760@1443767364" "flight not found"
[03-Oct-2015 11:00:05 UTC] Error flightcode='RYR9142' zwhen=2015-10-02 06:49:24 faID='RYR9142-1443593400-schedule-0000:0' depTime=1443767646 param="RYR9142@1443767646" "flight not found"
[03-Oct-2015 11:00:05 UTC] Error flightcode='MON894' zwhen=2015-10-02 06:54:50 faID='MON894-1443592800-schedule-0002:0' depTime=1443767931 param="MON894@1443767931" "flight not found"
[03-Oct-2015 11:00:05 UTC] Error flightcode='RYR3172' zwhen=2015-10-02 06:58:56 faID='RYR3172-1443594600-schedule-0002:0' depTime=1443767820 param="RYR3172@1443767820" "flight not found"
[03-Oct-2015 11:00:05 UTC] Error flightcode='RYR8162' zwhen=2015-10-02 07:19:47 faID='RYR8162-1443594300-schedule-0000:0' depTime=1443769334 param="RYR8162@1443769334" "flight not found"
[03-Oct-2015 11:00:06 UTC] Error flightcode='BOS6' zwhen=2015-10-02 08:24:41 faID='BOS6-1443732301-139-0' depTime=1443752580 param="BOS6@1443752580" "flight not found"
[03-Oct-2015 11:00:06 UTC] Error flightcode='BCY1617' zwhen=2015-10-02 08:59:00 faID='BCY1617-1443600000-schedule-0000' depTime=1443773700 param="BCY1617@1443773700" "no data available"
[03-Oct-2015 11:01:00 UTC] Error flightcode='BEE3042' zwhen=2015-10-02 09:08:00 faID='BEE3042-1443590826-airline-0149:2' depTime=1443776740 param="BEE3042@1443776740" "no data available"
[03-Oct-2015 11:01:37 UTC] Error flightcode='NAX7016' zwhen=2015-10-02 09:19:54 faID='NAX7016-1443582000-schedule-0000:2' depTime=1443757200 param="NAX7016@1443757200" "flight not found"
[03-Oct-2015 11:01:51 UTC] Error flightcode='TAP354' zwhen=2015-10-02 09:24:00 faID='TAP354-1443597600-schedule-0000' depTime=1443770880 param="TAP354@1443770880" "no data available"
[03-Oct-2015 11:02:05 UTC] Error flightcode='BEE7001' zwhen=2015-10-02 09:21:42 faID='BEE7001-1443604500-schedule-0001:2' depTime=1443777636 param="BEE7001@1443777636" "flight not found"
[03-Oct-2015 11:03:36 UTC] Error flightcode='BEE3025' zwhen=2015-10-02 09:56:06 faID='BEE3025-1443590826-airline-0393:0' depTime=1443779574 param="BEE3025@1443779574" "no data available"
[03-Oct-2015 11:04:22 UTC] Error flightcode='BEE502' zwhen=2015-10-02 09:58:54 faID='BEE502-1443590826-airline-0387:10' depTime=1443779003 param="BEE502@1443779003" "no data available"
[03-Oct-2015 12:00:02 UTC] Error flightcode='BOS6' zwhen=2015-10-02 08:24:41 faID='BOS6-1443732301-139-0' depTime=1443752580 param="BOS6@1443752580" "flight not found"
[03-Oct-2015 12:00:02 UTC] Error flightcode='BCY1617' zwhen=2015-10-02 08:59:00 faID='BCY1617-1443600000-schedule-0000' depTime=1443773700 param="BCY1617@1443773700" "no data available"
[03-Oct-2015 12:00:03 UTC] Error flightcode='NAX7016' zwhen=2015-10-02 09:19:54 faID='NAX7016-1443582000-schedule-0000:2' depTime=1443757200 param="NAX7016@1443757200" "flight not found"
[03-Oct-2015 12:00:03 UTC] Error flightcode='BEE7001' zwhen=2015-10-02 09:21:42 faID='BEE7001-1443604500-schedule-0001:2' depTime=1443777636 param="BEE7001@1443777636" "flight not found"
[03-Oct-2015 12:00:05 UTC] Error flightcode='TAP354' zwhen=2015-10-02 09:24:00 faID='TAP354-1443597600-schedule-0000' depTime=1443770880 param="TAP354@1443770880" "no data available"
[03-Oct-2015 12:00:06 UTC] Error flightcode='BEE3025' zwhen=2015-10-02 09:56:06 faID='BEE3025-1443590826-airline-0393:0' depTime=1443779574 param="BEE3025@1443779574" "no data available"
[03-Oct-2015 12:00:26 UTC] Error flightcode='KLM745' zwhen=2015-10-02 11:09:33 faID='KLM745-1443576900-schedule-0000:0' depTime=1443756840 param="KLM745@1443756840" "flight not found"
[03-Oct-2015 12:02:00 UTC] Error flightcode='SWR401' zwhen=2015-10-02 11:24:45 faID='SWR401-1443608100-schedule-0001:0' depTime=1443782559 param="SWR401@1443782559" "flight not found"
[03-Oct-2015 12:04:25 UTC] Error flightcode='BCY1816' zwhen=2015-10-02 11:49:00 faID='BCY1816-1443610500-schedule-0001' depTime=1443784500 param="BCY1816@1443784500" "no data available"
Some get “no data avaiable”; others get “flight not found”. In the above the left hand time is the time of the GetHistoricalTrack, and the zwhen is the timestamp of the flight previously returned by Search.
“Param” in the error messages above is what I am passing as the faFlightID to GetHistoricalTrack. Do I have the syntax of the request correct?
Update: I just analysed October 1st to 12th: 2398 GetHistoricalTracks, 460 failed either with “no data available” or “flight not found”. That’s a 19% failure rate. This is using flightcode@depTime as suggested, instead of faFlightID. So the failure rate is about the same as before, but with flightcode@depTime I get some “flight not found” errors whereas with faFlightID I always got “no data available”.
This problem in ongoing. And also affecting others:
http://discussions.flightaware.com/flightxml/getlasttrack-no-data-available-t36193.html
I would appreciate some indication of where FA is on this. Is this failure rate considered acceptable, or is there work ongoing that is trying to solve this problem? In my case all the flights reaqlly were in the air 24 hours earlier, and are all over southern England, so ADS-B coverage is good.
This is something we intend to improve, however I can’t provide you an estimate of when we’ll be able to do so. The affected flights have sensitive positional data associated with them, currently preventing us from automatically returning them properly through FlightXML.
Could you kindly clarify what you meant by "sensitive positional data "?
In my case, most of the flights that did not return track data are from mainstream locations in the US, arriving at a major airport as well.
Thank you.
Me also, Quietsky. All scheduled regular airline flights. Must be an awful lot of VVIPs travelling on these if “sensitive positional data” refers to security considerations.
We receive additional data from NATSUK and Eurocontrol (among others), which have specific data re-distribution limitations. There is a bug in FlightXML that prevents it from automatically fetching the data corresponding to the correctly declassified version of the flight when certain state changes occur during the course of the flight. The final security validation check causes the request to be rejected, which is why no data gets returned in those cases.
Hi,
I’m also seeing “no data available” when fetching any flight using FlightInfo (ex. EZY4545). Is this also because of security restrictions?
Thanks,
Carlos.
No, that is unrelated. EasyJet has poor coverage because they operate out of our primary coverage areas.
