Anomaly report regarding MLAT

FlightAware ADS-B Flight Tracking
1

Hi,

since today, the following anomaly keeps popping up here:

This feeder is not being used for multilateration because its timing information appears to be unreliable. This can be caused by the site location being incorrect, or because your Pi is running out of free CPU.

Actually, MLAT was working quite well since beginning here. Position is set perfectely and average CPU consumption on my Pi2 was 14% during the last 24 hours, so I am wondering what is going on here.

Is there some server-side problem, or do I have a problem somewhere? Any idea?

Regards,

Kabuse

2

Neither of my feeders are showing any MLAT positions today. Statistics on my side look fine, but the stats on my Flightaware page seem to be missing for today.

Cheers!
LitterBug

3

This is coming up with me as well. Running a Pi 2 and CPU usage hasn’t been more than 25%.

edit: This has only appeared since yesterday. All has been well otherwise in the two weeks I have had the Pi 2 up and running with the PiAware image.

4

Looks like the problem started around 02:00 EST

LitterBug

5

Both of your feeders haven’t checked in in the last 12 hours or so and don’t seem to be currently connected.

6

Looks like you are dropping USB data, your receiver clock periodically runs slow by 10-20us which is a symptom of some USB data getting dropped. It’s enough jitter that the server will kick you out for a while. (nb: by receiver clock I mean a clock that is driven by counting samples received from the dongle, it’s not related to the real-time system clock; if it runs unexpectedly slow compared to other receivers after compensating for the inherent frequency differences between dongles, then it means that some samples were dropped and not counted)

I’ve never tracked down the exact cause of this, it seems to vary from Pi to Pi and dongle to dongle. CPU load definitely makes it worse, but if you’re on a Pi 2 it’s not that. Maybe check your USB connections / cable?

Note that the anomaly reports have not changed the server behaviour, they’re just giving you visibility of the server decisions that were going on anyway.

7

I’ve been seeing this as well.

I’m using the latest PI, a new USB hub and the cables are new.

If there are any commands I can run to assist in gathering information, I’d be happy help.

8

If you’re running dump1090-mutability I’d be interested if turning off 2.4MHz / --oversample improves things.
One theory is that running the dongle at 2.4MHz is right on the edge of what it can do and some individual dongles don’t quite work 100% at that speed.

9

Hmm, strange. :confused: Especially, that it worked up to now and just came up today. Connections should be fine, I think.

I am also using an USB-Hub, with one more device connected to it, besides the RTL-SDR. Could this have any negative effects?

I will have a look, if this will continue tomorrow and if so, I will reboot my Pi and will keep watching.

Btw, at the moment no problem with it. MLAT planes show up.

10

Do you really need to connect the dongle via the USB hub?

I don’t use one and have never had a timing issue.

11

I think, with the Pi2, electrically, there is no need for an (powered) USB-Hub.

But, since I am also using a WIFI-dongle with an external antenna, it is mechanically more straightened, since the four USB-ports at the Pi(2)B can get a bit congested.

12

I remember reading that adding


"max_usb_current=1"

to the config.txt on the Pi 2 models sets the available current over USB to 1.2A (default is 600mA). I don’t use it, as I’m using a powered hub and haven’t seen an anomaly report. (Where are these reports seen, BTW, maybe I’m getting them but I don’t know where to look?)

I am running the latest git versions of all the applications and I get a wide variety of receivers


08/05/2015 11:57:24 mlat(4069): Server status:   not synchronized with any nearby receivers (check the configured receiver position)
08/05/2015 12:12:24 mlat(4069): Server status:   ok; synchronized with 14 nearby receivers
08/05/2015 13:57:26 mlat(4069): Server status:   ok; synchronized with 29 nearby receivers
08/05/2015 14:12:27 mlat(4069): Server status:   ok; synchronized with 27 nearby receivers
08/05/2015 14:27:27 mlat(4069): Server status:   ok; synchronized with 1 nearby receivers
08/05/2015 14:42:28 mlat(4069): Server status:   ok; synchronized with 26 nearby receivers
08/05/2015 14:57:28 mlat(4069): Server status:   ok; synchronized with 7 nearby receivers
08/05/2015 15:12:28 mlat(4069): Server status:   ok; synchronized with 22 nearby receivers
08/05/2015 15:27:28 mlat(4069): Server status:   ok; synchronized with 25 nearby receivers

Edit: http://raspberrypi.stackexchange.com/questions/27708/is-setting-max-usb-current-1-to-give-more-power-to-usb-devices-a-bad-idea

13

On the stats page. They’re driven from the same data that fa-mlat-client (not in a release yet, but it looks like you have github HEAD) uses for its server status so the piaware logs are probably more useful overall.

The variation in number of synchronized receivers may be normal if you have patchy use of ADS-B. I get pretty much constant ~40 receiver sync here, but the majority of the traffic is ADS-B - not true in the US.

14

Sounds like that is the cause:


08/05/2015 12:42:25 mlat(4069): Aircraft: 0 of 5 Mode S, 0 of 0 ADS-B used
08/05/2015 12:57:25 mlat(4069): Aircraft: 1 of 4 Mode S, 0 of 1 ADS-B used
08/05/2015 13:12:25 mlat(4069): Aircraft: 6 of 6 Mode S, 0 of 1 ADS-B used
08/05/2015 13:27:25 mlat(4069): Aircraft: 4 of 6 Mode S, 4 of 4 ADS-B used
08/05/2015 13:42:26 mlat(4069): Aircraft: 8 of 10 Mode S, 4 of 4 ADS-B used
08/05/2015 13:57:26 mlat(4069): Aircraft: 12 of 15 Mode S, 5 of 6 ADS-B used
08/05/2015 14:12:27 mlat(4069): Aircraft: 9 of 13 Mode S, 8 of 9 ADS-B used
08/05/2015 14:27:27 mlat(4069): Aircraft: 10 of 11 Mode S, 2 of 2 ADS-B used
08/05/2015 14:42:28 mlat(4069): Aircraft: 10 of 12 Mode S, 4 of 4 ADS-B used
08/05/2015 14:57:28 mlat(4069): Aircraft: 12 of 12 Mode S, 3 of 5 ADS-B used
08/05/2015 15:12:28 mlat(4069): Aircraft: 8 of 10 Mode S, 6 of 8 ADS-B used
15

Bounced both and they are back online. One is latest stock image, other is mutabilitry with latest piaware. Piaware process was running on both. Not sure why they weren’t talking. It’s like there was a network glitch and they didn’t recover. Both are pi2. Web interface was showing fine, and stats were collecting on the mutabilitry Pi.

Guess I’ll restart piaware process if it happens again…

Cheers,
LitterBug

16

Tried it. About 20 minutes after setting OVERSAMPLE=“no” and restarting dump1090-mutability the issue reappeared. I did verify that mlat results were showing locally after turning off 2.4MHz.

17

Oh well, so much for that theory. You could maybe try swapping out the dongle if you have a spare.

(Of my two test systems here, one shows the problem occasionally, the other doesn’t; it seems to be specific to the dongle)

18

Same issue here on a Pi2. I am using a USB extension cable…

19

Maybe you do need the hub for some of the devices, but if there is a possibility of timing issues with the dongle through the hub, try connecting it directly - keep dongle + pi + dump1090 setup as simple as possible. Just put the other stuff via the hub.

USB extender is a long USB cable with a one port hub on the end isn’t it?

20

I had no problems running a B+ or a Pi2 with the SDR dongle plugged directly in as long as I was not using a wireless network dongle and/or my wireless keyboard dongle at the same time. I usually run wired networking with putty so USB power is not an issue. I recently had lockups on my test rig and finally figured out that it wasn’t my Power supply, but rather I had forgotten to unplug my wireless keyboard dongle. Pi was getting hot and locking up with the extra USB power load…

Your results may vary depending on dongles…

Cheers!
LitterBug

Was using keyboard to diagnose why DHCP was not working on my network with the latest PiAware image… Found it to be related to the Raspian change which uses the DUID instead of the MAC address for DHCP reservation <\edit>