Mlat keeps stopping

I updated to 3.7.1. The 1090 on the Airspy (which also serves as a bias tee) is connected to the Odroid N2. I also consolidated the 978 orange FA dongle to the N2.

Now, MLAT for 1090 works as long as I don’t have DUMP978-FA running. When I stop the process, MLAT works as expected. When I start DUMP978-FA, MLAT starts dropping the synchronized servers. After about 15 minutes the status changes to the dreaded “not synchronized with any nearby receivers”.

I am using the Odroid 12V 2A power supply (which cost me all of $5.50 --a steal). Might this be the problem? Anything else maybe? I attached the Airspy USB utilization that shows when 978 is running and when it is not. Is there a way to check the power supply hypothesis?

If you’re going all the way down to “not synchronized” then that means the airspy is dropping a lot of data (so much so that mlat can’t find pairs of ADS-B position messages with even roughly similar timings to do synchronization - effectively dropping >100ppm will do this). I’d suspect you’re hitting USB limits.

I would have expected the N2 to handle an rtl-sdr dongle in addition to the airspy.
(USB bus wise)

But that may not be the case.
If you have a powered USB hub you can try that to rule out power problems on the USB ports with that.
(Using it for the 978 dongle should suffice)

I’m not sure how the USB chip or chips of the N2 are layed out.
You can try using a different USB port for the 978 dongle.
If the ports are somehow paired by 2 then using another port might solve the problems.

A better measure than MLAT synchronization is checking the airspy_adsb log for lost samples:

sudo journalctl -e -u airspy_adsb

Thanks for all the feedback.

journalctl -e -u airspy_adsb

Does not contain any notes about dropped samples.
I also changed the USB port for the orange dongle. Alas, no luck.

I will try the powered USB hub. Amazon to the rescue. Will report back.

That is really strange.
What does it say? Anything?

No matter the CPU load alone basically means something is not working right.
The powered hub should reveal if it’s an data or power bottleneck.

This is since the reboot last night (I thought maybe a reboot would solve the issue)

– Reboot –
May 24 18:48:40 odroid systemd[1]: Started Airspy ADS-B receiver.
May 24 18:48:41 odroid airspy_adsb[2110]: Listening for beast clients on port 29999
May 24 18:48:41 odroid airspy_adsb[2110]: Listening for asavr clients on port 47806
May 24 18:48:41 odroid airspy_adsb[2110]: Acquired Airspy device with serial XXXXXXX
May 24 18:48:41 odroid airspy_adsb[2110]: Decoding started at 20 MSPS
May 24 18:48:53 odroid airspy_adsb[2110]: Push client connected to localhost:30104 (beast)
root@odroid:~#

That is the end of the log-file.
I confirmed this morning again that starting dump978-fa and skyview978 causes the outage.I suppose I can try starting just dump978-fa alone without skyview. Let me try that.

It’s not skyview :wink:
Doesn’t do anything basically besides using very little CPU.

confirmed :slight_smile:

I added the powered USB hub for the 978 dongle (AirSpy directly into the N2): no change. The moment I start dump978-fa the MLAT for 1090 stops. On the FA My ADS-B page, the MLAT is green. But it won’t synchronize:

May 26 18:10:02 odroid piaware[25646]: mlat-client(25841): Receiver status: connected
May 26 18:10:02 odroid piaware[25646]: mlat-client(25841): Server status: not synchronized with any nearby receivers
May 26 18:10:02 odroid piaware[25646]: mlat-client(25841): Receiver: 383.2 msg/s received 153.0 msg/s processed (40%)
May 26 18:10:02 odroid piaware[25646]: mlat-client(25841): Server: 0.0 kB/s from server 0.0kB/s TCP to server 1.8kB/s UDP to server
May 26 18:10:02 odroid piaware[25646]: mlat-client(25841): Aircraft: 36 of 47 Mode S, 49 of 53 ADS-B used

The only MLAT calculation I participate in is FA. No other MLAT is processed. I have tried restarting piaware, dump1090-fa, the Airspy software. To no avail. I get between 150 and 250 MLAT aircraft reported per day, and less than 100,000 positions (~5%). So, not a ton.

My understanding is that dump978 does not use MLAT at all. I have not explicitly turned it off in the settings (and I do not know how to do this). Does the Airspy use have something to do with it? The 978 antenna is attached to the orange FA dongle.

Just for fun, I tried to switch on the MLAT for FR24. Worked like a charm.

I am lacking ideas.

In Piaware SD card image 3.7.1, by default dump978-fa is disabled.
For details how to turn dump978 ON/OFF, see this thread:

Piaware SD card image 3.7.1 Quickstart Guide

Thanks. I am able to start and stop it. The version I use is compiled from scratch to ARM64 (from your notes here). It seems to do something, though no aircraft in the last hour.

1 Like

For information of others using a compiled add-on package, the dump978-fa can be turned ON/OFF by making:
For turning ON:
ENABLED=yes

For turning OFF:
ENABLED=no

in the file /etc/default/dump978-fa

Yeah that’s not really an indication. It displays that it’s working even if it’s really not synchronized.

Airspy on 20 MHz uses a lot of bandwidth.

Apparently the USB on the board can’t handle the additional throughput.
If you switch to 12 MHz it will most likely work.

I’m actually very surprised that the Odroid N2 has such a bottle-neck.

Can you show the configuration?
cat /etc/default/dump978-fa

and check the dump978-fa version?
dump978-fa --help

You are losing regular messages as well.
Even if airspy is not showing the loss, if MLAT isn’t working properly you are losing messages.
It’s visible in your graphs as well.

1 Like

Sure. This is the 978 config. I added the gain but otherwise ‘as is’

root@odroid:~# cat /etc/default/dump978-fa

ENABLED=yes
RECEIVER_OPTIONS=“–sdr driver=rtlsdr,serial=00000978 --format CS8 --sdr-gain 38.6”
DECODER_OPTIONS=“”
NET_OPTIONS=“–raw-port 30978 --json-port 30979”

Yeah that’s fine.
You can also check if you have 3.7.1 with
dump978-fa --help

It has some performance improvements.

I see the Odroid N2 has a USB 2 OTG connector.
If that USB port is supplied by another USB controller it could be solution.
(Could just as well be on the same USB controller though)

You would need an OTG adapter though.

Maybe it’s easiest to continue running a separate RPi for dump978.

1 Like

Confirmed that it is 3.7.1

root@odroid:~# dump978-fa --help
dump978-fa 3.7.1

Also, I can confirm that switching to frequency 12 in the Airspy setup, “SAMPLE_RATE = 12” seems to have solved the it. The MLAT works, and the UAT 978 appears to be running (inclement weather today so no planes seen since I switched it on at around 5pm). For fun, the Airspy CPU utilization plot included. Much lower CPU usage with sample rate 12.

I am going to let 1090 and 978 run on the N2. If it continues to sputter, I will put 978 on the RPi.

Thank you for all the suggestions.

You could try 20 MHz and -p for bit packing.
May free up enough USB bandwidth for airspy_adsb to play nice with dump978.

But if 12 MHz works just as well for you that’s great as well.

Good suggestion. I tried this and this plays nice with DUMP978-FA as well. The airspy is running with these settings:

/usr/local/bin/airspy_adsb -v -f 1 -b -p -l 29999:beast -l 47806:asavr -c localhost:30104:beast -g 17 -m 20

3 Likes

After a full day of running, I can report that the Airspy lost samples. Switched to 12 Mhz and no bit packing.

Seems the Odroid N2 gets pushed with dump978 and Airspy running for dump1090

Too bad the N2 is limited in USB bandwidth.

If the main thing for you is to have the skyview978 on odroid and also combine the piaware feed, that is both possible with running only dump978 on another device.
Just in case you want a combined feed but use 20 MHz sample rate.