FlightAware Discussions

Dump1090-fa crashing / surging?

FlightAware ADS-B Flight Tracking
#1

Brief situation overview:

I have built a RPi4 running TwisterOS, Wine & Box86. I am using the Wiedehopf dump1090-fa installation and directly uploading to several tracker sites, including FR24 and Plane Finder. But, I am running the full Windows version of PlanePlotter through Wine. Eventually, I got everything working correctly. I then bought a second RPi4 and cloned the first SD card. I now have the second running in my loft, aerial on the roof, uploading as stated. The first is in my office and is more for testing / experimenting.

Both are connected to the same external aerial. I use a Uputronics Filtered Preamplifier, then an ADSB signal splitter that was recommended, then one goes into a blue FA dongle and then quality a USB cable to one RPi4, with the other going into a orange FA dongle with FA light blue SMA filter and then a quality USB cable to the other RPi4.

The RPi4 “downstairs” in my office crashes intermittently. Or, rather, it seems to surge and output (at least to PlanePlotter) stops. So far, I have observed that when this occurs the CPU % that dump1090-fa is using increases from about 12% to about 66%.
The unit in my loft has never had this occur.

Checks that I have carried out so far:

  1. The RPi4 in my office first operated for two weeks in my loft without any failure.
  2. I have swapped the two USB cables over at the dongles and the problem remains with the RPi4 in my office. (When this failure does not occur, it is interesting that the orange FA dongle picks up slightly more traffic than the blue one, constant with whichever RPi4 is being feed from the orange dongle).
  3. Obviously, the feed to my office uses a much longer USB cable (albeit of high quality). But, when connected to my Windows laptop and using the Windows version of dump1090, no failures are occurring.

It is my belief that something is causing the CPU % surge, which then remains at about 66%, even though dump1090-fa does not seem to be functioning at that stage.

Any help gratefully appreciated,
Ian

image
image1920×1080 464 KB

#2

I’d say the power supply doesn’t supply as much power?
Sounds like the SDR is not working properly due to power issues from a long USB cable combined with maybe slightly lower voltage on one of the RPi4 you have.

#3

Thank you. I will try swapping the two power supplies (that was one that i did not think of). Also, I will but an official RPi one, so that I have a spare.

#4

Check the dump1090-fa logs when you are having trouble:

journalctl -e -u dump1090-fa
(q to exit the pager)

#5

Thanks. I have swapped the the power supplies between the two RPi’s. I have also double checked the longer USB cable again in a Windows PC (where it is fine). Nothing changes. Ditto with connecting the USB to either the blue or black RPi connections. Sometimes dump1090-fa simply stops and is then not there under “Top”. But, more often, it goes to 66% or so CPU and does not put out useable data. This is the case now, output below.

image
image1920×1080 725 KB

Appreciate any further help please,
Ian

#6

image
image1920×1080 446 KB

#7

“cb transfer status: 1” is a generic USB transfer error. Could potentially be hardware problems in the dongle itself, hardware problems with the USB port, or a kernel-level bug.

You could try the piaware sdcard image to eliminate any software-side problems. If you still see misbehaviour it’s almost certainly hardware.

You could also try swapping the two Pi 4s in case it’s a problem with the physical board.

Another possibility is to look at the USB bus traffic with e.g. usbmon / wireshark which may give better visibility of exactly what’s failing, but that is a fairly “cooked” capture and if there are USB protocol-level errors going on you probably won’t see much beyond a generic -EPROTO error. (And hardware USB analyzers that let you see the bus-level traffic are not cheap…)

#8

That’s not a good test as the conditions aren’t the same.
Avoiding USB extensions is usually the way to go.

#9

Still crashing and - accepting that the long USB is probably the fault - but that seems more often when traffic is high. Overnight crash now, with low traffic, but here is the readsb log:

Feb 12 07:45:53 RaspberryPi4 readsb[547]: Beast TCP output: Send Error: Broken pipe: 192.168.1.224 port 53398 (fd 24, SendQ 278, RecvQ 0)
Feb 12 07:45:53 RaspberryPi4 readsb[547]: Beast TCP output: Send Error: Broken pipe: 192.168.1.225 port 44498 (fd 22, SendQ 278, RecvQ 0)
Feb 12 07:45:53 RaspberryPi4 readsb[547]: Beast TCP output: Send Error: Broken pipe: 192.168.1.224 port 53392 (fd 21, SendQ 278, RecvQ 0)
Feb 12 07:45:53 RaspberryPi4 readsb[547]: High load: modesNetPeriodicWork() elapsed1/2/3/interval 0/193/33/11416346 ms, suppressing for 5 seconds!
Feb 12 07:45:56 RaspberryPi4 readsb[547]: Fri Feb 12 07:45:56 2021 GMT  No data received from the SDR for a long time, it may have wedged, exiting!
Feb 12 07:45:56 RaspberryPi4 readsb[547]: Fri Feb 12 07:45:56 2021 GMT  Waiting for receive thread termination
Feb 12 07:45:57 RaspberryPi4 readsb[547]: Reattaching kernel driver failed!
Feb 12 07:45:57 RaspberryPi4 readsb[547]: Fri Feb 12 07:45:57 2021 GMT  Normal exit.
Feb 12 07:45:57 RaspberryPi4 systemd[1]: readsb.service: Succeeded.
Feb 12 09:14:25 RaspberryPi4 systemd[1]: Started readsb ADS-B receiver.
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: Fri Feb 12 09:14:25 2021 GMT  readsb starting up.
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: readsb version: wiedehopf git: eec4f27 (restrict proxy_string debug output to netIngest mode, Sat Jan 23 09:23:35 2021 0100)
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: struct sizes: 1600, 24, 64, 108
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: json_reliable: 2
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: Using lat:   51.2968, lon:   -0.0925
Feb 12 09:14:25 RaspberryPi4 readsb[15093]: rtlsdr: using device #0: Generic RTL2832U (Realtek, RTL2832U, SN 00001000)
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Found Rafael Micro R820T tuner
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: rtlsdr: tuner gain set to 48.0 dB
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Raw TCP output: Listening on port 30002
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Beast TCP output: Listening on port 30005
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: SBS TCP output: Listening on port 30003
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Beast TCP input: Listening on port 30004
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Beast TCP input: Listening on port 30104
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: startup complete after 0.663 seconds.
Feb 12 09:14:26 RaspberryPi4 readsb[15093]: Allocating 16 zero-copy buffers

Thanks,
Ian

#10

So that shows something i don’t like at all, it was hanging for 2.5 hours before exiting.

I’ve changed around my readsb code a bit more in that regard … i’m hopeful it will no longer hang.
I’d recommend you run the install script again to get the changed code.

I would be very surprised if the crashes are related to traffic as the SDR has no clue about the number of aircraft … the software gets the raw sample data and always at the same rate.

I don’t suppose you have a powered USB hub you could run where the SDR is?
That way the SDR is powered more directly.

#11

Or if not avoidable use one with good quality. There are some working perfect and others (even shorter) impact the reception by 30