If it were solely for the FlightAware, it would be 
Just another item to add to the tool box of tricks for managing sites that are often unattended for hours or days at a time.
I had to reset the router again this morning before leaving for work.
I think the ultimate solution since whatever underlying problem is unlikely to be properly diagnosed, debugged, and a fix implemented in the forseeable future, is simply a hardware watchdog circuit. Router or something else stops responding? Cycle the power, leaving it off for a couple of minutes.
I never needed to power off the device and wait several minutes, so my Monit instance is doing what is expected.
If you have to reset the router for getting the connect established again, then you have a problem with the router which Flightaware cannot fix
I had exactly that with my old Netgear router - From time to time it would just drop the internet connection and fail to reconnect. I had a dedicated Pi Zero with a relay in the power cable to the router. If the Pi stopped being able to connect to the internet for five minutes, it dropped the power to the router, waited thirty seconds and then put the power back on.
Exactly what I’m dealing with, combined with intermittent situations where the PiAware has frozen up (like it did from 4PM until I got home to reboot it at 8 PM) but the router was still up.
So I’m using 2 independent power channels. First script pings the PiAware. Second script does exactly what yours does with the exact same response. If the router is being reset it sets a semaphore to tell the first script to expect the PiAware will not respond for a while. Only when the internet is visible and the PiAware is non-responsive does it drop power to the PiAware for about a minute or so the restore it.
One thing I plan to do either tonight or tomorrow night is clone the micro SD card, as I have had other Pi applications (such as an APRS gateway running DireWolf) exhibit intermittent lockups if the SD is failing. It seems to write and over-write a huge number of .json files per hour which might eventually reach write endurance limits on the micro SD.
I stumbled on an underlying cause (I suspect two) of my intermittent issues. It was a marginal connection on the custom interface board (not a true “hat” as it has no EEPROM or special configuration) going into intermittent undervolt due to not delivering enough current to the Pi during peak demand.
The second was most likely thermal. I changed out the interface board from another project, and added a cooling fan (it is a Pi 3B+), and with the changing seasons, the overheating events stopped by mid September. The only other outage since was the USB dongle flaking out.
I am adopting a Python script from another unrelated project that uses stuff from the gipozero library as well as time, os, and sys to grab cpu.temperature, time / date, and the output of a DS18B20 temperature probe to monitor ambient temperature, and combine all that into a collection of daily log files of up to 1440 samples per day.
This gives a much more fine grained detailed set of data points as to exactly the magnitude of the temperature excursions and determine what level of mitigation (e.g. build a small air conditioned space in the shop building for certain Pi based instrumentation) may be needed.
and an update.
It looks like the power / thermal issues were the cause. The feeder has had continuous up-time since April 2022, all the way through this past summer’s heat.
2 Likes