MLAT Clock Unstable after moving to USB boot

FlightAware ADS-B Flight Tracking
1

I copied and migrated my SD card piaware setup to a USB flash drive, removing the SD card. Now I’m getting MLAT time issues, despite being basically rock solid for months (with this SD > USB the only change).

My local network uses a local NTP server, with the router redirecting port 123 requests to that local server. It’s worked fine up until now with this set up. But just for fun, I disabled the firewall rules so that clients can actually hit the NTP servers they request from (rather than the masqueraded response that comes from the local NTP). Still getting clock unstable.

While I still have some troubleshooting to perform, I wonder: has anyone else that is booting from a USB drive on their pi suffered clock unstable MLAT issues?

ETA: Looked at the PoE stats for the pi (as it runs on PoE) on my switch. It’s pulling 11-12.4w, on a PoE HAT that provides… 12.5w of power supply. I may be running into power/voltage issues in bumping against the limit of the PoE supply. I don’t have power draw records from when I ran on the SD card boot. Might have to throw the SD back in and re-check.

1 Like
2

mlat timing is sensitive to anything else on the USB bus (the 2832 drops data very easily if the bus is not perfect, and it’s that dropped data that causes mlat timing issues). So it may just be that the USB drive is causing too much contention there. How much I/O is actually going on?

3

3 SDRs- the blue FA pro stick+ for 1090, red AirNav for UAT, and a nooelec NESDR running for AIS.

4th USB port is the USB3.1 boot drive.

There’s nothing otherwise running on this pi beyond piaware/ais-catcher/graphs1090.

4

That is a lot to put on one USB2.0 bus. When one device is USB2.0, all four ports run at USB 2.0 speeds.
run lsusb -t or lsusb -v to see this.

2 Likes
5

Better remove USB boot drive, and use only microSD card.

1 Like
6

I thought about that. Well, wondered whether it would show what port each thing was plugged into so I didn’t have to climb up and see which SDR is plugged into the second USB3 port.

I see the usb storage key running on USB3.0 speeds, and the SDRs at USB2.0, unless I’m misinterpretting what this says:

pi@pi4b2:~ $ lsusb -tvv
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    /sys/bus/usb/devices/usb3  /dev/bus/usb/003/001
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
    /sys/bus/usb/devices/usb2  /dev/bus/usb/002/001
    |__ Port 2: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
        ID 090c:1000 Silicon Motion, Inc. - Taiwan (formerly Feiya Technology Corp.) Flash Drive
        /sys/bus/usb/devices/2-2  /dev/bus/usb/002/002
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    /sys/bus/usb/devices/usb1  /dev/bus/usb/001/001
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        ID 2109:3431 VIA Labs, Inc. Hub
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/002
        |__ Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=usbfs, 480M
            ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
            /sys/bus/usb/devices/1-1.1  /dev/bus/usb/001/003
        |__ Port 1: Dev 3, If 1, Class=Vendor Specific Class, Driver=, 480M
            ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
            /sys/bus/usb/devices/1-1.1  /dev/bus/usb/001/003
        |__ Port 3: Dev 4, If 0, Class=Vendor Specific Class, Driver=usbfs, 480M
            ID 0bda:2832 Realtek Semiconductor Corp. RTL2832U DVB-T
            /sys/bus/usb/devices/1-1.3  /dev/bus/usb/001/004
        |__ Port 4: Dev 5, If 0, Class=Vendor Specific Class, Driver=usbfs, 480M
            ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
            /sys/bus/usb/devices/1-1.4  /dev/bus/usb/001/005
        |__ Port 4: Dev 5, If 1, Class=Vendor Specific Class, Driver=, 480M
            ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
            /sys/bus/usb/devices/1-1.4  /dev/bus/usb/001/005

I hadn’t previously read up on the actual hardware of the pi 4b, but looking at it (VL805 USB controller), all 4 ports are usb2, but the two usb3 ports go through at usb3 speeds if a usb3 device is plugged into them (which again, depends on if I’m reading the block diagram right):

image
image840×1000 52.7 KB

All said, I need to troubleshoot both with a powered usb hub, or rolling back to the SD card. Neither was really desired since I want to stick this in an outdoor enclosure on a much taller mast and access would be more limited. Idea being that the SD card would be a point of failure before a flash drive would.

1 Like
7

So for troubleshooting purposes, I tried connecting all of the SDR dongles to an externally-powered USB hub (and the flash drive still directly connected). Let it run this way for a while to see the log entries several times for MLAT server status, thinking PoE power supply might be the issue. Still clock unstable.

Rolled back to the SD card, been solid ever since. Lost a day of data in graphs1090, but that’s no biggie.

8

I think you overcrowded the USB bus by the I/O of the SSD.
3 SDR dongles generate already a lot of I/O so this might be too much for the USB bus on the Pi.
It might work with less then 3 SDR’s but that would defeat the purpose I guess.
The SD card doesn’t use the USB bus so that reduced the traffic on it.

9

@pp6000v2

Instead of using a USB flash boot, using a good quality and large size microSD card seems an easy and practical solution under your circumstances.

10

@pp6000v2

RasPiKey: Plug And Play EMMC Module For Raspberry Pi | The Pi Hut)

 

Screenshot_20230804_090001_Samsung Internet
Screenshot_20230804_090001_Samsung Internet1080×1302 239 KB

11

I may change up the SDR setup. The AIS-catcher service is processing 16GB+/hr. My antenna is homebrew, and running on a ~50ft length of RG6 feed cable. Once I move everything outside I can try with a much shorter cable and see if I actually pick anything up. I know the port terminal is close enough that I should be receiving the cruise ships and tankers, but as-is, I get only a few “other” messages from the nav buoy beacons, and never positions. May change to a 137MHz weather antenna, and since that isn’t continuous use, I might be able to get away with the USB flash drive. We’ll see.

I like that eMMC-to-SD. Not super cheap or sleek, but would get the job done in an outdoor enclosure.

12

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.