Major drop in aircraft seen

Is there something weird going on? I looked at my page about an hour ago and had almost seen 2000 aircraft today. Just a few minutes ago I looked again, and the total aircraft seen has plunged down to under 500. Any idea what could cause that?

flightaware.com/adsb/stats/user/djmd808

Same thing on mine. Aircraft and positions across the board. Down to like 10-15% of what it was a couple hours ago.

Definitely something odd happened.

Indeed

just had a look ,like I just started the day though it’s near 16pm … guess some work Under the hood

Glad it’s not just me. I noticed my pi was down for like 10 minutes earlier, so I started freaking out. Haha. Probably a database issue.

same thing with my stats page major drop

Phew, I thought I’d broken something :astonished:

This was due to expanding the capacity of the stats processing systems. We are planning to backfill the missing stats for today, tomorrow.

Thanks for the update!

I can confirm the same thing happening to all the sites here in Bulgaria.
Now I am seeing that this thing is global.
Thanks for the explanation.

Its amusing, my page right now is a strange mix of english and what looks like german. :laughing:

Seems the missing stats are lost in space forever :laughing:

I’m still seeing a fairly blank spot in my graph as well from yesterday!

Apologies for the glitch the other day. We understand what happened but unfortunately it is not possible for us to go back and fill-in that gap. Things are correct going forward.

The glitch is related to some changes we’re making to continue to scale up as we have more and more sites. We are also in the process of deploying more hardware to continue to support hourly stats collection for all sites.

But… but… my ranking!! J/K… keep up the good work!

So the issues are fixed? I’m asking because I have two receivers. One is showing the hatched no-data square in the weekly graph. The other shows data there. If I look at local data, it seems normal.

IOW, hourly received reports shows blank for Saturday UTC for my station that normally has the most data. The other one looks normal (has data for Saturday UTC), as of this writing.

Edit: It looks like it’s been back-filled now.

Time to bring out the pitchforks! :stuck_out_tongue:

The changes helped with processing speed but didn’t entirely solve the problem so stats will still be somewhat behind at peak times (and then catch up after the peak passes, as you saw). The next step is to throw more hardware at it.

Mine shows the ‘bars’ for the last three hours as well now. I’m glad the data is being received at least.

Tough problem to solve. I suppose demand on servers increases with the square of the number of sites or something like that. Adding more servers, though, will also increase traffic among the servers, right? Anyway, it seems like performance has dropped since the change. Are configurations still being tweaked?

Edit: fixed spelling error

We made some software configuration tweaks. We’re planning to provision additional hardware today which will give us many months of additional growth capacity.