the last days i wrote some scripts that make use of the data my raspberry pi harvests via dump1090. what confuses me now is that the unique hex counts per utc-day differ from what on my fa-stats page is shown. so i tried to gather this information with another different method - but again the numbers showed nearly the same (1 aircraft less) but very different result from flightaware.
example saturday 9/9/2017:
fa-stats: 3532 aircrafts
my pi: 3619 aircrafts
my first idea was - maybe there are some ‘one hit wonders’ means aircrafts where i got just one message and flightaware says these are just errors. so i looked into this - but no. there are two hex numbers with less than 100 messages - all others got from 100-16000 hits. while further investigating i found 11 starting with a ~ what means they are anonymized and because of that probably duplicates of real ones. another is hex 000000 with more than 70000 hits that probably is rubbish too.
but in the end i have 12 faulty hex numbers from 3619 => 3607 aircrafts remaining and that is still 75 more than my flightaware stats page shows!
did i overlook something? are my counts wrong or flightawares?
you find my first test-method here if you want run the same test for your site or have a look into for logic and/or code errors: Daily Log/E-Mail Aircraft Report directly on your RaspberryPi
p.s. the second test generated 14,500,000 database entries and 1.6 GB data on a single utc day for my site. must be big fun for FA to handle this multiplied with 10,000 sites
p.p.s. i’m aware that of course the fa-sytem over the day lags a little - but yesterday it began similar …