need help with mysterious feeder dropouts

Hi,
I’ve had my pi chugging away with stock dump1090 and piaware for several weeks (two months actually) with NO issues… then, all of a sudden, I get feeder dropout when I look at the “My ADS-B” page - the status is “Live” and “Live” but there is no data being recorded. I’ve rebooted, reinstalled piaware and made sure that the pi is feeding valid data to PlanePotter and other clients.

Can anyone offer any suggestions to debug this issue?
Thank you
Sunil

I’ve been seeing the same thing recently. There have been ongoing intermittent issues with the FA servers not keeping up with statistics. It seems the stats eventually “catch up”, so I wouldn’t fret over it too much.

Thad - N7ILK

ok, thanks. Only issue is that my stellar stats record is being tarnished :angry: :frowning: I just broke the 500 top feeder barrier but now I’m on my way down again :frowning:

I’ve seen at least three of these and my stats have not caught up. There are gaps of up to 10 hours when no contact was being made with the server, the dara wasn’t even getting through. Log shows dump1090-mutability is collecting reports but is unable to connect with FA server. Eventually it disconnects and starts all over, the messages are lost.

10/28/2015 11:40:29 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:42:02 connecting to FlightAware piaware.flightaware.com/1200
10/28/2015 11:42:03 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:43:12 connecting to FlightAware piaware.flightaware.com/1200
10/28/2015 11:43:13 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:44:02 5045728 msgs recv’d from dump1090-muta (3379 in last 5m); 4897444 msgs sent to FlightAware
10/28/2015 11:44:36 connecting to FlightAware 70.42.6.197/1200
10/28/2015 11:44:36 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:46:07 connecting to FlightAware 70.42.6.198/1200
10/28/2015 11:46:08 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:47:26 connecting to FlightAware piaware.flightaware.com/1200
10/28/2015 11:47:26 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:49:02 5049076 msgs recv’d from dump1090-muta (3348 in last 5m); 4897444 msgs sent to FlightAware
10/28/2015 11:49:10 connecting to FlightAware piaware.flightaware.com/1200
10/28/2015 11:49:10 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:50:17 connecting to FlightAware 70.42.6.197/1200
10/28/2015 11:50:17 error during tls handshake: handshake failed: connection reset by peer, will try again soon…
10/28/2015 11:51:23 connecting to FlightAware 70.42.6.198/1200

This coincides with a wider FA outage. Do you have an example that wasn’t this morning?

The logs are only available for the last two days and yesterday was OK.

There was another outage around the 21st when messages were lost. For some reason not everybody was affected, some were live throughout, others like myself suffered the disconnection. Maybe we are not all connecting to the same server?

Yeah, the 21st was another FA-side outage where I got a plan B running to salvage some of the traffic but it was only able to handle about 25% of the piawares. It’s been a bad week…

If it does happen again let me know.

to my site this happens intermitting constantly over the last 8 days. stats and mlat have massive problems - not only hiccups …

https://dl.dropboxusercontent.com/u/39745369/error_01.jpg

https://dl.dropboxusercontent.com/u/39745369/error_02.jpg

p.s. sadly no data was recovered as dbaker said it were

With feeds stopping at 2100z tonight it appears that this problem is still ongoing.

Peter

yep - your Hourly Collection Graphs look exactly like mine - and my stats over the last ten days decreased about 10-20%
i think this will go on for a while until they get it fixed as there were zero improvements over last days …

see ads-b-flight-tracking-f21/stats-are-behind-t36218.html

It’s not just that they are behind, some days they never catch up. This doesn’t seem to be affecting everyone the same though.

yes i lost aircrafts/positions too - about 10-20% over the last 10 days. all planeplotter sites seem to be not affected or at least only minimal affected …

Can you provide a specific date/time when this has happened to you?

As far as I am aware, the stats reflect all the data that FA has (up to 2-3 hours behind realtime). There have been a couple of system-wide outages over the last 10 days which would have dropped some data on the floor (see @flightawareops); this are reflected in the stats that you see.

If you think there is something else going on, then I am going to need a specific example to look at.

10/21 about 40% planes/positions lost
10/25 until today about 5-15% planes/positions lost and over the day extreme lagging

obj - the stats thing sucks as this is the unique selling proposition of flightaware - but don’t spend a single hour on bringing them back to me!
in some days/weeks when fa managed the problems anything will be fine again - and i don’t care about stats more than 14 days ago.

but what i’m really interested in is the question about mlat percentages here → http://discussions.flightaware.com/viewtopic.php?p=183443#p183443 and my two posts below …

thanks again

That was the outage on the 21st mentioned above and in the ops link above.

10/25 until today about 5-15% planes/positions lost

That’s too vague to be useful. Can you point out a specific hour where stats are showing fewer position reports than you expect?
There were outages on the 26th and 28th that will affect your total position count for those days.

and over the day extreme lagging

OK, now you’re just not bothering to read what I wrote. Stats are 2-3 hours behind realtime; they catch up.

no - sadly i did not make write ups. just saw it over the day and when comparing to others sites with my feed. but i guess the lower stats are results of intermitting mlat failure. but again it’s not worth to spend your time just for bringing me back old stats.

sorry - but i do not really understand what this sentence means?

No, mlat positions do not count towards current stats at all, so it doesn’t matter what mlat is doing. (separate mlat stats are coming soon, in theory)