PiAware RPi keeps having data outages since Sept

RPi feeder since 2019. Same hardware has been working strong since start with no issues. Since Sept. 2023, site has had 11 data outages. Data outages last up to 2 days long. I’ve tried reformatting CF card and reinstalling PiAware, but still have outages (and lost the streak for the receiver :frowning: ). I checked RPi heatsink, and nothing else has changed.
What should I check next?

PiAware Version: 8.2 
Dump1090-fa Version: dump1090-fa 8.2 
Dump978-fa Version: dump978-fa 8.2 
CPU Temperature: 46.7 C 
CPU Load: 0% 
System Uptime: 1 day, 12 hours 57 minutes 

Power supplies are a common issue. Power outages can damage them or the RPI itself.
Is the device connected using wifi or ethernet?
Have you tried a different RPI, Power Supply or SD Card?

1 Like

Hi @jonhawkes2030 ,
Interesting. Device is connected via wifi, 6 ft from AP. I’ll try another PSU. If it continues, I’ll try formatting a new SD card with a different RPI and Power Supply.
Thanks for the suggestions!

There is a command that you can run in the terminal that will display if you have had any recent power problems, but I can’t remember what it is. I will hunt around and see if I can find it.

edit - found it. Run this command from the command line. If you get no output and it just kicks you back to the prompt, your power supply may be ok.

sudo dmesg --ctime | grep voltage

Note that if you are not using one of the official Raspberry Pi power supplies (using a phone charger or some such), then your power should be immediately suspect any time something weird goes on. Those devices don’t put out sufficient amperage at 5 volts. They are also not designed for a 100% duty cycle (powering an always-on device, vs charging a battery occasionally). They may work, they may even work for a relatively long time. But they are not the right tool for the job and you can’t fault them when they fail.

this is a undervoltage one i use not sure if it would help

sudo dmesg --ctime | grep voltage
bash -c “$(wget -nv -O - https://raw.githubusercontent.com/wiedehopf/adsb-scripts/master/throttled.sh)”

It’s a PSU bought from a RPI seller, but not an office PR foundation PSU.
Nothing with

sudo dmesg --ctime | grep voltage

But saw the following with

sudo dmesg --ctime

hundreds of:

[Mon Nov 20 07:53:35 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121

At boot. Then:

[Mon Nov 20 07:53:35 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 4
[Mon Nov 20 07:53:35 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 07:53:35 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 07:53:35 2023] usb 1-1.3: USB disconnect, device number 32
[Mon Nov 20 07:53:36 2023] usb 1-1.3: new high-speed USB device number 33 using dwc_otg
[Mon Nov 20 07:53:36 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 07:53:36 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 07:53:36 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 07:53:36 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 07:53:36 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 07:57:09 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 3
[Mon Nov 20 07:57:09 2023] usb 1-1.3: USB disconnect, device number 33
[Mon Nov 20 07:57:09 2023] usb 1-1.3: new high-speed USB device number 34 using dwc_otg
[Mon Nov 20 07:57:10 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 07:57:10 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 07:57:10 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 07:57:10 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 07:57:10 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 07:57:55 2023] usb 1-1.3: USB disconnect, device number 34
[Mon Nov 20 07:57:55 2023] usb 1-1.3: new high-speed USB device number 35 using dwc_otg
[Mon Nov 20 07:57:55 2023] usb 1-1.3: unable to read config index 0 descriptor/all
[Mon Nov 20 07:57:55 2023] usb 1-1.3: can't read configurations, error -71
[Mon Nov 20 07:57:56 2023] usb 1-1.3: new high-speed USB device number 36 using dwc_otg
[Mon Nov 20 07:57:56 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 07:57:56 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 07:57:56 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 07:57:56 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 07:57:56 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 07:59:35 2023] usb 1-1.3: USB disconnect, device number 36
[Mon Nov 20 07:59:36 2023] usb 1-1.3: new high-speed USB device number 37 using dwc_otg
[Mon Nov 20 07:59:36 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 07:59:36 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 07:59:36 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 07:59:36 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 07:59:36 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:00:32 2023] usb 1-1.3: USB disconnect, device number 37
[Mon Nov 20 08:00:32 2023] usb 1-1.3: new high-speed USB device number 38 using dwc_otg
[Mon Nov 20 08:00:32 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:00:32 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:00:32 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 08:00:32 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 08:00:32 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:01:22 2023] usb 1-1.3: USB disconnect, device number 38
[Mon Nov 20 08:01:22 2023] usb 1-1.3: new high-speed USB device number 39 using dwc_otg
[Mon Nov 20 08:01:22 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:01:22 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:01:22 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 08:01:22 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 08:01:22 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:01:55 2023] usb 1-1.3: USB disconnect, device number 39
[Mon Nov 20 08:01:56 2023] usb 1-1.3: new high-speed USB device number 40 using dwc_otg
[Mon Nov 20 08:01:56 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:01:56 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:01:56 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 08:01:56 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 08:01:56 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:03:44 2023] usb 1-1.3: USB disconnect, device number 40
[Mon Nov 20 08:03:44 2023] usb 1-1.3: new high-speed USB device number 41 using dwc_otg
[Mon Nov 20 08:03:44 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:03:44 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:03:44 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 08:03:44 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 08:03:44 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:05:24 2023] usb 1-1.3: USB disconnect, device number 41
[Mon Nov 20 08:05:24 2023] usb 1-1.3: new high-speed USB device number 42 using dwc_otg
[Mon Nov 20 08:05:24 2023] usb 1-1.3: string descriptor 0 read error: -71
[Mon Nov 20 08:05:24 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:05:24 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:05:24 2023] usb 1-1.3: can't set config #1, error -71
[Mon Nov 20 08:05:24 2023] usb 1-1.3: USB disconnect, device number 42
[Mon Nov 20 08:05:24 2023] usb 1-1.3: new high-speed USB device number 43 using dwc_otg
[Mon Nov 20 08:05:25 2023] usb 1-1.3: New USB device found, idVendor=0bda, idProduct=2832, bcdDevice= 1.00
[Mon Nov 20 08:05:25 2023] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Nov 20 08:05:25 2023] usb 1-1.3: Product: RTL2832U
[Mon Nov 20 08:05:25 2023] usb 1-1.3: Manufacturer: Realtek
[Mon Nov 20 08:05:25 2023] usb 1-1.3: SerialNumber: 00001000
[Mon Nov 20 08:05:55 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 7
[Mon Nov 20 08:05:55 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 2
[Mon Nov 20 08:05:55 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 1
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 7
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 7
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 7
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 2
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 3
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 4
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121
[Mon Nov 20 08:10:46 2023] usb 1-1.3: usbfs: usb_submit_urb returned -121

And then finally

[Mon Nov 20 08:16:43 2023] usb 1-1.3: USB disconnect, device number 59
[Mon Nov 20 08:16:43 2023] usb 1-1.3: new high-speed USB device number 60 using dwc_otg
[Mon Nov 20 08:16:44 2023] usb 1-1.3: device descriptor read/64, error -71
[Mon Nov 20 08:16:44 2023] usb 1-1.3: device descriptor read/64, error -71
[Mon Nov 20 08:16:44 2023] usb 1-1.3: new high-speed USB device number 61 using dwc_otg
[Mon Nov 20 08:16:45 2023] usb 1-1.3: device descriptor read/64, error -71
[Mon Nov 20 08:16:46 2023] usb 1-1.3: device descriptor read/64, error -71
[Mon Nov 20 08:16:46 2023] usb 1-1-port3: attempt power cycle
[Mon Nov 20 08:16:46 2023] usb 1-1.3: new high-speed USB device number 62 using dwc_otg
[Mon Nov 20 08:16:57 2023] usb 1-1.3: device not accepting address 62, error -110
[Mon Nov 20 08:16:57 2023] usb 1-1.3: new high-speed USB device number 63 using dwc_otg
[Mon Nov 20 08:17:08 2023] usb 1-1.3: device not accepting address 63, error -110
[Mon Nov 20 08:17:08 2023] usb 1-1-port3: unable to enumerate USB device

All of this at boot.

 bash -c “$(wget -nv -O - https://raw.githubusercontent.com/wiedehopf/adsb-scripts/master/throttled.sh)”
2023-11-21 20:19:07 URL:https://raw.githubusercontent.com/wiedehopf/adsb-scripts/master/throttled.sh [1421/1421] -> "-" [1]
if: line 1: “#!/bin/bash: No such file or directory

The only USB device is the PiAware USB-antenna dongle.

Hardware	: BCM2835
Revision	: a02082
Model		: Raspberry Pi 3 Model B Rev 1.2

Ethernet is over USB, but I couldn’t find that wifi is over USB.

You aren’t using a USB extension cable, or a (powered or unpowered) USB hub, are you? Anything connected to the GPIO pins?

Nope. Only thing connected is the PiAware Pro Stick Plus.

I’m definitely not the expert here, but those error messages seem to suggest that your USB ports can’t decide what mode to run in. That dwc_otg error googles out to having something to do with whether the USB port runs in device mode or host mode.

Are you running the 64 bit version of Raspberry Pi OS? I found some forum posts from people who seemed to have the same error and “fixed it” by reimaging down to the 32 bit version.

I’ve been running the same OS image on the same hardware with no issues for 4 years until 2 months ago. It’s curious. I’ll check that out as an option along with swapping out the PSU and other HW.

Yeah, those errors indicate USB communication problems; in rough order of likelihood, that can mean:

  • power supply problems
  • physical problems with the USB connection (less likely if there’s no cable involved)
  • a failing dongle or a failing Pi

Maybe unlikely but had one day the issue that my Airspy stick lost connection several times over an hour and then returned working.

After i squeezed the connector a bit and plugged it in again it was sitting thighter and it was working again as expected.

My experience with USB SDRs is that a short cable greatly improves reliability, and that plugging the dongle straight into the port works okay but can be prone to the bulk of the dongle skewing the connection. The Nooelec SMART I have here is incredibly prone to this, such that even a small sideways movement when plugged in crashes it. A short hand-off cable, like below, stabilises it.

usb_extension

1 Like

True - I was thinking more along the lines of fewer things to fail, but you’re right about the direct connection being marginal because of the leverage of the dongle on the port.

3 Likes

My experience is exactly the same. Loose fit of dongle coupled with leverage of dongle on USB port makes it particularly prone to malfunction.

Another disadvantage of direct plugin becomes obvious when someone tries to plugin two dongles directly into Pi (1090 & 978 or 1090 & 162 or 1090 & AirBand).

2 Likes

I also found my pi ran several degrees cooler by placing a short high quality USB cord between pi and dongle

Yes, thermal isolation! Exact same findings here. Very evident on 1090 graphics.

I replaced the PSU with a new Canakit / USA PSU, and still having the same issue. I’ll try a USB cable extension, or maybe another USB port on the RPI.