Inconsistent lifeguard flight reporting?

[pclifton]

I’m working on a pet project to collect lots of data on medevac/lifeguard flights. I have a scraper script that hits SearchBirdseyeInFlight each hour, queries all lifeguard flights, and stores that information in a database. Recently it’s come to my attention that there are some Lifeguard flights that have not made it into my database. This comes courtesy of my partner/friend who is a air ambulance pilot, and noticed that some of his company’s flights were not in my database.

I’m generating a log file of my API queries, and looking back through it, I’ve had no failures or gaps in script coverage at the times the flights in question were missed. As far as I can tell, my script is running without a hitch and getting good results on each cycle, but these flights are missing for whatever reason.

Example: flightaware.com/live/flight/N81A … /KLAX/KPDX shows as MEDEVAC on the details page, but it was not found/stored by my scraper script.

Bear in mind that I don’t have any “live” proof that this isn’t the fault of my script. That is, I haven’t looked up a current Lifeguard flight on Flightaware, then done an API query and found that flight not listed in the results. The flight mentioned above is the most recent example of this I have, and it’s way too old to show up in SearchBirdseyeInFlight results.

But I was wondering if there are known issues with coverage of these flights via the API. My gut feeling would be that the data served up by the API would match the data on the Flightaware website, but perhaps it might be something specific to the API logic for SearchBirdseyeInFlight?

I’m just trying to get the most complete data set possible here, and trying to scope out every possibility for this discrepancy. I appreciate any help.

[bovineone]

I assume you’re doing a query of “{true lifeguard}”. There’s another way to search for lifeguard flights “{= prefix G}” and in some cases that may return a partially non-overlapping set of results. We’re going to be consolidating our data in the next few days so that either of these queries produce the same results, but as a temporary workaround you can execute both queries separately.

[pclifton]

Thanks…I just figured out (literally moments ago) that I do have a problem with my script. I was getting incomplete data sets from my API requests because of the limits on result set sizes. It just never occurred to me that I was hitting that; I assumed that the default limit was 15, but it seems that I’ve been getting back a maximum of 11 results per query.

In any case, I’ve used SetMaximumResultSize to bump up my limits, and now I’m getting a LOT more data. I expect this should take care of my problem. If I run across any inconsistencies, I’ll reply back here, but at this point it seems safe to say that the API is fine, it’s just this dumb developer causing problems.