How do I troubleshoot this? Setup is a Pi3B 1090/978 feeder (10.2 SD card) and occasionally drops UAT. A reboot sometimes fixes it, though I’ve needed to un/replug the 978 orange FA dongle for a few seconds to get UAT back online the last four times.
Plenty of low/slow back country flyers and medical evac copters fly near the house, so drops are apparently not due to lack of UAT traffic.
I look forward to replies to your post.
I have separate dedicated 978 and 1090 receivers on the small Colorado airport where I live. (5V4)
I started with RPi Zero’s, but have upgraded both to RPi 4B’s.
In spite of my best efforts, the 978 will need to be power booted every 1-3 months, (with the RPi Zero, it was way more often) when I receive a shutdown notice. Plenty of 978 traffic here as well.
Jay
Hi all,
Not an answer, but a tip for fixing the issue that does not require a reboot or unplugging and plugging in the dump978-fa RTL device.
I have run both dump1090-fa and dump978-fa for many years. The dump1090-fa is always rock solid reliable, and easily goes many months between an occasional reboot done for other reasons. The dump978-fa for me usually fails about every 4 months or so.
I’m not certain, but I have always suspected that dump978-fa is less reliable because it relies on SoapySDR. Dump1090-fa uses the standard librtlsdr libraries by default. Dump1090-fa does support SoapySDR, but it does not use SoapySDR by default with RTL dongles.
When it fails, the dump978-fa service has restarted multiple times, and then just gives up. The easiest way to resolve it is to simply unplug and plug in the device. Then a dump978-fa restart works perfectly.
My Pi is NOT remote or hard to get to, but I still thought it would be a good learning experience to troubleshoot this and look for a “remote” fix.
The solution to manually unplugging was a Pi/Linux program called “usbreset”. That allows you remotely to reset the device connection. Dump978-fa will then reconnect.
pi@raspberrypi:~ $ whereis usbreset
usbreset: /usr/bin/usbreset
pi@raspberrypi:~ $ usbreset
Usage:
usbreset PPPP:VVVV - reset by product and vendor id
usbreset BBB/DDD - reset by bus and device number
usbreset "Product" - reset by product name
Devices:
Number 001/005 ID 0bda:2838 RTL2838UHIDIR
Number 001/004 ID 0bda:2838 RTL2838UHIDIR
Number 001/003 ID 0424:ec00
Number 001/002 ID 0424:9514
pi@raspberrypi:~ $ rtl_test
Found 2 device(s):
0: Realtek, RTL2838UHIDIR, SN: 301
1: Realtek, RTL2838UHIDIR, SN: 411
Yes, I could use other usb commands to verify which of my RTL devices (serial=401 is 978 UAT) is which, but it is quicker and easier to simply reset both 1090/978.
These two commands reset both devices.
pi@raspberrypi:~ $ usbreset 001/005
Resetting RTL2838UHIDIR ... ok
pi@raspberrypi:~ $ usbreset 001/004
Resetting RTL2838UHIDIR ... ok
Then I simply restart both 1090 and 978 services as below. 1090 would probably restart automatically after the reset. Becaue 978 had hard failed, it probably needs the restart command to kick start it.
pi@raspberrypi:~ $ sudo systemctl restart dump1090-fa.service
pi@raspberrypi:~ $ sudo systemctl restart dump978-fa.service
==========
As part of troubleshooting, I also discovered and started using an excellent Pi/Linux program called “monit”.
monit
Using monit, that monitors the dump978-fa program which should always be running. The monit program checks that process, along with a number of other things I have configured once every 2 minutes. If the process is not running, it e-mails me a descriptive alert.
After setting up monit, it took me about 4 months to have my next dump978-fa failure. I had the e-mail alert on the failed process within 3 minutes. I then remotely over ssh reset the usb devices and was back in operation. My Pi is here locally, but this fix would have worked anywhere with ssh access.
Since starting with monit, I have only had 2 dump978-fa failures. Each was approximately 4 months apart. I also use monit for many other things.
So, while not a solution, the combination of monit for monitoring and usbreset works very well for me. Monit actually has ways to not only detect the failure, but could also easily run a script that would automatically run usbreset and restart the failed process. That would be fully automatic with just an e-mailed notification, but I haven’t bothered to automate it yet.
Months ago I mentioned usbreset on another group and two people reported it helped them with remote systems. One had remote telescope photography software that would occasionally lock up. Usbreset fixed the problem without having to go out to the telescope dome. Another had remote sites related to bird tracking with multiple usb hubs and usb decoders. Usbreset solved his problems also and did not require a site visit.
Regards,
-Dan
Thanks, Dan, this information is very helpful. Tried the usbreset commands locally and they work on both dongles. 978/1090 services didn’t come back online so I had to manually restart each.
I have a cron job set up to reboot the Pi3 every Sunday morning and will add a script to run those commands on occasion.
Hi,
I have never had to rely on weekly reboots here to keep things reliable.
You could always do something like a daily, middle of the night, cron script that would reset both devices and restart both services. That would just be the 4 commands. That would take the system offline for probably less than 10 seconds once per day. If your 978 died, it would be recovered that night.
Since mine goes about 4 months between lockups, I haven’t bothered to automate it. For me, if my system was remote but not remotely controlled, then I would probably do daily restarts or possibly use something like monit to monitor the process, and then trigger usbreset and service restarts on a failure.
Do you have remote ssh access to your system at the airport?
Yes, if you find that a Pi reboot doesn’t always reset the 978 device, you could always add a cron script that starts on every reboot. Just make it wait 60 seconds or so after reboot, then usbreset both devices, then restart both services.
Regards,
-Dan
Yep. I realized later that the other reply was the one at the airport, not yours.
Regards,
-Dan