Delay in offline notification?

I received the “This email is a friendly notification that your PiAware ‘xxxxxxx’ (xx:xx.xx.xx.xx) is not currently sending live flight data to FlightAware.” The notification time is set to 6 hours. At the time of the email the system had been dead for 16 hours. This is the second time in a week or so the unit got pi-faced. Both times the notification was 16 hours or more. Is it just me? Is there something in my configuration that results in this, something in the settings on FA, or an implementation of the $IF_JOEL_SWITCH?
Why the pi dies is a separate issue and solely on my side.

I think it’s something in FA - not settings.
I’m also running the PlaneFinder client - they do notify you more reliably (alas also at 6 hours)

Let me add a “me too” to this thread.

Do not disturb the beast.

My RP has been running flawlessly since February in its outdoor housing. I recently opened the unit to retrieve and copy the SD card so as to install the latest manual upgrades with the confidence that a backup was at hand. With the cloned SD card, the system started hanging up a couple of times a day. Thinking it was the replacement SD card not being as tolerant of the Florida heat, I put the original SD card back in.

The system was responding to pings last evening at 7 or so. This morning I find that the unit had not reported for 10 hours meaning that it hung up only minutes after I had ensured that it was alive. There was no email from Flightaware of this problem even though my setting is for a 6 hour warning.

If anyone has a nice monitoring script that can be run on another Pi, please share it with us. I would bet most folks have multiple units running various tasks around the house. I have one running OpenVPN and and another running Icecast2 and DarkIce. Either one has extra cycles to share for monitoring purposes.

I think the answer would be to set up public/private key authentication between the Pi’s for ssh
linuxproblem.org/art_9.html

Then send remote commands to the pi vis ssh
malcontentcomics.com/systemsboy/ … a-ssh.html

to check the remote processes dump1090, piaware, etc
ps -fC dump1090-mutability

do a couple of experiments to find the exit status from the ssh if the remote machine failed to respond (… has crashed completely)

if it did respond, did it give the requested info (is the program running)

This is the first I’ve heard of people missing outage notices.

How are you determining the device is down? Inability to connect to web interface / ssh? Did you check your FA stats page? It’s possible it was still connected to FA and not hung completely.

Body of message:This email is a friendly notification that your PiAware ‘dogberry21’ (b8:27:eb:d7:35:7b) is not currently sending live flight data to FlightAware. Your receiver last connected to FlightAware on 04-Jul-2015 07:00PM. FA stats page listed last activity 16 hours before. Pi running headless, not responding to ping, ssh. Script to restart on loss of wifi did not seem to trigger (user problem). Am just reporting the delay of demise notification.
thanx

In my case, my stats page Data Feed and Feeder Check-in fields were at 10 hours. (Site 5089 - KTPA). I could not ping or SSH to the Pi.

I checked my account for the validity of my email address which was OK. I have since changed it to different one.

It has now foobarred on two consecutive days. The first day I caught it after just a couple of hours.

Was this in the last 24h? We had the outage notifications turned off due to an unrelated issue, but that’s very rare.

That would explain it. Thanks for looking into it.

Yes, between 7 pm last evening (my last check of good status), and 5:55 EST this morning.

Just my luck, my first outage over 10 minutes in 6 months and the notification function is off line.

I use a crude but effective monitoring script running on another Linux VM in my “datacenter.” It pings the RPi every 5 minutes, and also samples the FATSV output port for a few seconds to make sure messages are being received. If not, it sends an e-mail alert.

I dumped it up on github in case anyone is interested. github.com/ProHill/ADS-B-Stuff/ … /piping.sh

Andy

Andy, You DA MAN. Thanks for this.

TD