Multilateration (MLAT) now available on PiAware!

Something obviously changed. Did you let FA auto-upgrade piaware or not?

Check the current operating settings, Is there any connection between fa-mlat and dump on 30104, or not?

Just repeating “I didn’t change anything” isn’t going to help. There is nothing wrong with mlat-results in the new version, but, again

  1. The syntax for setting the mlat-results destination has changed
  2. The actual mlat messages has changed, they are now coded in a different message.

/M

Well, if it’s nothing immediately obvious, then you need to provide enough information to actually diagnose the problem: your piaware and dump1090 versions, your actual configuration (output of “piaware-config”), contents of /var/log/piaware.log. Without that info, I’d just be guessing.

Thanks for taking a mental interrupt to think about this at all; I understand that since I did make a change to the configuration to add an MLAT output port I’m likely the only one experiencing this problem.

I have “Auto-update PiAware software” enabled so in theory should be running the latest and greatest. According to ‘My ADS-B’ I have “PiAware (SD Card) 3.1.0”. I’m not actually going to be able to RemoteDesktop into the Pi until this coming weekend to gather more version and other details/logs.

I was planning to just flatten and rebuild the Pi with a new PiAware image, verify everything works, and then use the new command to add an addition MLAT output port. However if you guys are interested for your own reasons in understanding why my config change is interoperating with the new bits this way, I’m more than happy to provide whatever details/logs are needed to debug.

Let me know, and thanks again for the help.

David

If you can provide the things I asked for above when you have a chance then I’m happy to take a look.

Trying my best to trouble-shoot / understand why I don’t see any MLAT being recorded.
I am running piaware & dump1090-fa from package builds I made on a debian VM
using a blue ProStick Plus. It appears to be working for ADBS as FA is recording those
records but still no MLAT received (or processed ).

Is there anything which reports there is not enough overlap or receiving bad data?
If understand the log below, it is sending UDP packets.
The lat and long set appear to be set correctly.

Note: I am aware the antenea has limitted view right now.

Here are recent logs.




Jan 29 01:17:15 debian piaware[2174]: piaware version 3.3.0 is running, process ID 2174
Jan 29 01:17:15 debian piaware[2174]: your system info is: Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 GNU/Linux
Jan 29 01:17:16 debian piaware[2174]: Connecting to FlightAware adept server at piaware.flightaware.com/1200
Jan 29 01:17:16 debian piaware[2174]: Connection with adept server at piaware.flightaware.com/1200 established
Jan 29 01:17:17 debian piaware[2174]: TLS handshake with adept server at piaware.flightaware.com/1200 completed
Jan 29 01:17:17 debian piaware[2174]: FlightAware server certificate validated
Jan 29 01:17:17 debian piaware[2174]: encrypted session established with FlightAware
Jan 29 01:17:17 debian piaware[2174]: adept reported location: 37.xxxx, -122.xxxx, 720ft AMSL
Jan 29 01:17:17 debian piaware[2174]: Receiver location changed, restarting dump1090
Jan 29 01:17:17 debian piaware[2174]: attempting to restart dump1090..
Jan 29 01:17:17 debian piaware[2174]: attempting to restart dump1090-fa using 'systemctl --no-block try-restart dump1090-fa.service < /dev/null'...
Jan 29 01:17:17 debian piaware[2174]: dump1090 restart appears to have been successful
Jan 29 01:17:17 debian piaware[2174]: logged in to FlightAware as user mdebusschere
Jan 29 01:17:17 debian piaware[2174]: my feeder ID is 740599b3-d0a7-41a3-7aff-xxxx
Jan 29 01:17:17 debian piaware[2174]: site statistics URL: https://flightaware.com/adsb/stats/user/mdebusschere#stats-37304
Jan 29 01:17:17 debian piaware[2174]: multilateration data requested
Jan 29 01:17:17 debian piaware[2174]: no ADS-B data program is serving on port 30005, not starting multilateration client yet
Jan 29 01:17:18 debian piaware[2174]: no ADS-B data program seen listening on port 30005 for 3 seconds, next check in 60s
Jan 29 01:17:49 debian piaware[2174]: 0 msgs recv'd from dump1090; 0 msgs sent to FlightAware
Jan 29 01:18:17 debian piaware[2174]: Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --input-type dump1090 --r
esults beast,connect,localhost:30104 --results beast,listen,30105 --results ext_basestation,listen,30106 --udp-transport 70.42.6.191:8433:2614718051
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): fa-mlat-client 0.2.8 starting up
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): Using UDP transport to 70.42.6.191 port 8433
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): Listening for Beast-format results connection on port 30105
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): Listening for Extended Basestation-format results connection on port 30106
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): Input connected to localhost:30005
Jan 29 01:18:17 debian piaware[2174]: mlat-client(2255): Input format changed to BEAST, 12MHz clock


Jan 29 11:22:49 debian piaware[2174]: 18496 msgs recv'd from dump1090-fa (528 in last 5m); 18496 msgs sent to FlightAware
Jan 29 11:27:49 debian piaware[2174]: 19009 msgs recv'd from dump1090-fa (513 in last 5m); 19009 msgs sent to FlightAware
Jan 29 11:32:49 debian piaware[2174]: 19387 msgs recv'd from dump1090-fa (378 in last 5m); 19387 msgs sent to FlightAware
Jan 29 11:33:31 debian piaware[2174]: mlat-client(2255): Receiver status: connected
Jan 29 11:33:31 debian piaware[2174]: mlat-client(2255): Server status:   connected
Jan 29 11:33:31 debian piaware[2174]: mlat-client(2255): Receiver:  113.0 msg/s received       29.1 msg/s processed (26%)
Jan 29 11:33:31 debian piaware[2174]: mlat-client(2255): Server:      0.0 kB/s from server    0.0kB/s TCP to server     0.4kB/s UDP to server
Jan 29 11:33:31 debian piaware[2174]: mlat-client(2255): Aircraft: 6 of 14 Mode S, 5 of 5 ADS-B used
Jan 29 11:37:49 debian piaware[2174]: 19705 msgs recv'd from dump1090-fa (318 in last 5m); 19705 msgs sent to FlightAware



Try it on bare metal. You are getting no synchronization at all, not even attempts at it, which usually means that your mlat timing is very badly wrong. Last time I saw this was with a VM that would drop around 10% of USB traffic.

Unfortunately, I don’t have bare metal available. The VM and host CPU is running at around 1%-4%, i7 CPU (Virtual Box) windows host.
Is there anyway to determine if this is USB traffic issue and/or anything else which can impact MLAT time?
It doesn’t make sense the USB traffic can be dropped without any record of this happening.

I just restarted everything and noticed this message in log
Jan 29 19:38:06 debian piaware[455]: mlat-client(835): Server status: not synchronized with any nearby receivers

So, is there a way to determine if the issue is related to bad / missing data on my side or not enough nearby receivers to make it work?
The status message “not synchronized with any …” is some what ambiguous.

Where you are, there are tons of receivers. Unless you’ve put your antenna inside a metal shed or something it should be synchronizing :wink:
It does not tell you that the timing data is bad because it is so far wrong that it doesn’t even look like wrong timing any more, it looks like you’re receiving different messages entirely (if that makes sense…)

I know that “silently drops lots of USB traffic” doesn’t make sense, but that’s what I saw previously with a VM which had the same symptoms as yours.
I’d suggest picking up a Pi and trying piaware on that if you don’t have convenient bare metal.

Follow-up: to test outside of possible VM/USB issue I loaded the windows version of dump1090 and
made sure that works. Then I configured the piaware running in the VM to use the windows instance.
It is now showing sync and recording MLAT. So I don’t have any info to blame the VM-USB at this point,
maybe it’s the linux RTL driver / VM clocking, I don’t have time to debug that issue any more.

As much as I don’t want to plug another device in, I will probaly resort to getting a Pi3…
Hopefully, I can power it from POE - it will at least let me put it close to the antenna later on.

Thanks for the input.

I started receiving the ‘This feeder is not being used for multilateration because its timing information appears to be unreliable…’ message when I switched booting from a micro SD card to a USB SSD drive (connected to a UUGEAR powered USB hub). I am using the standard FA image running on a Pi3 with a FA Prostick connected to the powered USB hub.
What exactly is causing this behavior is difficult to say, CPU use is max. 20% (nothing else running on the Pi, NTP is running correctly). Since I’ll move the entire setup outside I feel more comfortable having the SSD in the equation compared to a micro SD card.
I shall try another USB to SATAIII cable and potentially also a smaller and less power hungry SSD to see if that changes anything (although the total power consumption - 300mA Prostick, 200mA Pi, 4mA USB Hub and 1000mA SSD, 1504mA in total is well within spec, even for the PoE standard).

Update:
Booting from a USB connected device requires a PI firmware.
Going through the syslog again, I noticed there were frequent dumps being produced and this was also apparent when I started monitoring the PI’s green led which was active ever few minutes. I did two things:

  1. Disabled wifi (I do not need it) → No go
  2. Add dtoverlay=mmc in /boot/config.txt → the dumps disappeared and and so did the activation of the PI’s green led

This seems to have worked since the ‘This feeder is not being used for multilateration because its timing information appears to be unreliable…’ message has not shown since.
Now happily running on what seems to be a stable setup booting from SSD.

Hi Obj,

I finally got the requested info. I uploaded /var/log/piaware.log to: https://1drv.ms/u/s!AlUorSoxzp3ujSUQhsOxCR8N-yFs

The other data is inline here:

pi@piaware ~ $ sudo piaware-config
allow-auto-updates yes # value set at /boot/piaware-config.txt:5
allow-manual-updates yes # value set at /boot/piaware-config.txt:4
image-type piaware # value set at /boot/piaware-config.txt:6
mlat-results-format “beast,connect,localhost:30104 basestation,listen,12345” # value set at /boot/piaware-config.txt:7

pi@piaware /usr/bin $ piaware -v
3.1.0

pi@piaware /usr/bin $ dump1090 --help

| dump1090 ModeS Receiver Ver : flightaware-1.2-3 |

Thanks!

David

That all looks correct and I don’t see any connection errors in the piaware log.

However your dump1090 is somewhat old. I suspect that it is too old to understand DF18 TIS-B positions, which is now how mlat results are formatted. Can you try upgrading to the latest dump1090-fa?

I wasn’t aware I had to manually update components when I had auto-update enabled. That’s fine though. Is there a link you can point me too to make sure I do it correctly?

Also, Seattle had a snow storm (I’m on a rural Island near Vashon) and I’m on generators, but my Pi and WiFi repeater are at the top of a tower with a UPS that won’t last for much longer, so I might not get to it right away.

David

The mismatch is partly my fault - we pushed an autoupdate of piaware, but for non-sdcard installs or for installs that were originally on 2.1, that only does piaware and not dump1090-fa as well. I’ll look at pushing a corresponding autoupdate for dump1090-fa (it was low priority since it shouldn’t have affected anything, but I missed the mlat result thing)

In the meantime you can push an update to dump1090 via the “send command to device” option on your ADS-B stats page

Sorry this is progressing at the rate of a 17th century letter exchange between Boston and San Francisco…

I tried this and it didn’t seem to work:


[2017-02-11 13:36 PST] manual update (user-initiated via their flightaware control page) requested by adept server
[2017-02-11 13:36 PST] performing manual update, action: dump1090
[2017-02-11 13:36 PST] *** running command '/usr/lib/piaware/helpers/run-apt-get update' and logging output
[2017-02-11 13:36 PST] run-apt-get(24535): /etc/apt/trusted.gpg.d/flightaware-archive-keyring.gpg does not exist; installing it
[2017-02-11 13:36 PST] run-apt-get(24535): /etc/apt/sources.list.d/piaware.list does not exist; creating it
[2017-02-11 13:37 PST] run-apt-get(24535): Err http://archive.raspberrypi.org wheezy Release.gpg
[2017-02-11 13:37 PST] run-apt-get(24535): Something wicked happened resolving 'archive.raspberrypi.org:http' (-5 - No address associated with hostname)
[2017-02-11 13:37 PST] run-apt-get(24535): Err http://raspberrypi.collabora.com wheezy Release.gpg
[2017-02-11 13:37 PST] run-apt-get(24535): Something wicked happened resolving 'raspberrypi.collabora.com:http' (-5 - No address associated with hostname)
[2017-02-11 13:37 PST] run-apt-get(24535): Err http://flightaware.com wheezy Release.gpg
[2017-02-11 13:37 PST] run-apt-get(24535): Something wicked happened resolving 'flightaware.com:http' (-5 - No address associated with hostname)
[2017-02-11 13:37 PST] 32949 msgs recv'd from dump1090 (836 in last 5m); 32807 msgs sent to FlightAware
[2017-02-11 13:37 PST] run-apt-get(24535): Ign http://archive.raspberrypi.org wheezy Release
[2017-02-11 13:37 PST] run-apt-get(24535): Ign http://raspberrypi.collabora.com wheezy Release
[2017-02-11 13:37 PST] run-apt-get(24535): Err http://flightaware.com wheezy Release.gpg
[2017-02-11 13:37 PST] run-apt-get(24535): Something wicked happened resolving 'flightaware.com:http' (-5 - No address associated with hostname)
[2017-02-11 13:37 PST] run-apt-get(24535): Ign http://flightaware.com wheezy Release
[2017-02-11 13:38 PST] run-apt-get(24535): Ign http://flightaware.com wheezy Release

and indeed looking on the PI itself, the 1090 version is still at 1.2-3.

If there a way to manually update these bits?

Thanks,

David

That means your DNS setup on the Pi is broken. The upgrade process depends on having functional DNS to be able to fetch the packages.