fa-mlat-client 0.2.4 restarting every 6 minutes or so?

FlightAware ADS-B Flight Tracking
1

Morning all,

Looking at the logs in fA reports, I see fa-mlat-client looks to be restarting every 6 minutes or so?

Any known issue currently?

I’m running mutability, so should fA’s MLAT even be running, or does Mutability replace only dump1090?



Note: this log pane refreshes itself automatically every 15 seconds.  Refresh log contents →
[2016-04-20 08:05 BST] 46019 msgs recv'd from dump1090-mutab (419 in last 5m); 46019 msgs sent to FlightAware
[2016-04-20 08:05 BST] mlat(732): Receiver status: connected
[2016-04-20 08:05 BST] mlat(732): Server status: synchronized with 59 nearby receivers
[2016-04-20 08:05 BST] mlat(732): Receiver: 363.5 msg/s received 6.5kB/s from receiver
[2016-04-20 08:05 BST] mlat(732): Server: 0.9 kB/s from server 0.0kB/s TCP to server 1.9kB/s UDP to server
[2016-04-20 08:05 BST] mlat(732): Results: 391.5 positions/minute
[2016-04-20 08:05 BST] mlat(732): Aircraft: 11 of 16 Mode S, 10 of 13 ADS-B used
[2016-04-20 08:10 BST] 46378 msgs recv'd from dump1090-mutab (359 in last 5m); 46378 msgs sent to FlightAware
[2016-04-20 08:15 BST] 46773 msgs recv'd from dump1090-mutab (395 in last 5m); 46773 msgs sent to FlightAware
[2016-04-20 08:20 BST] 47247 msgs recv'd from dump1090-mutab (474 in last 5m); 47247 msgs sent to FlightAware
[2016-04-20 08:20 BST] mlat(732): Receiver status: connected
[2016-04-20 08:20 BST] mlat(732): Server status: synchronized with 52 nearby receivers
[2016-04-20 08:20 BST] mlat(732): Receiver: 253.4 msg/s received 4.6kB/s from receiver
[2016-04-20 08:20 BST] mlat(732): Server: 0.5 kB/s from server 0.0kB/s TCP to server 1.1kB/s UDP to server
[2016-04-20 08:20 BST] mlat(732): Results: 220.0 positions/minute
[2016-04-20 08:20 BST] mlat(732): Aircraft: 6 of 14 Mode S, 17 of 20 ADS-B used
[2016-04-20 08:25 BST] 47821 msgs recv'd from dump1090-mutab (574 in last 5m); 47821 msgs sent to FlightAware
[2016-04-20 08:30 BST] 48390 msgs recv'd from dump1090-mutab (569 in last 5m); 48390 msgs sent to FlightAware
[2016-04-20 08:35 BST] 48865 msgs recv'd from dump1090-mutab (475 in last 5m); 48865 msgs sent to FlightAware
[2016-04-20 08:35 BST] mlat(732): Receiver status: connected
[2016-04-20 08:35 BST] mlat(732): Server status: synchronized with 49 nearby receivers
[2016-04-20 08:35 BST] mlat(732): Receiver: 273.5 msg/s received 5.0kB/s from receiver
[2016-04-20 08:35 BST] mlat(732): Server: 0.5 kB/s from server 0.0kB/s TCP to server 1.2kB/s UDP to server
[2016-04-20 08:35 BST] mlat(732): Results: 205.5 positions/minute
[2016-04-20 08:35 BST] mlat(732): Aircraft: 9 of 17 Mode S, 13 of 14 ADS-B used
[2016-04-20 08:40 BST] 49267 msgs recv'd from dump1090-mutab (402 in last 5m); 49267 msgs sent to FlightAware
[2016-04-20 08:45 BST] 49661 msgs recv'd from dump1090-mutab (394 in last 5m); 49661 msgs sent to FlightAware
[2016-04-20 08:50 BST] 50104 msgs recv'd from dump1090-mutab (443 in last 5m); 50104 msgs sent to FlightAware
[2016-04-20 08:50 BST] mlat(732): Receiver status: connected
[2016-04-20 08:50 BST] mlat(732): Server status: synchronized with 58 nearby receivers
[2016-04-20 08:50 BST] mlat(732): Receiver: 198.6 msg/s received 3.6kB/s from receiver
[2016-04-20 08:50 BST] mlat(732): Server: 0.6 kB/s from server 0.0kB/s TCP to server 1.1kB/s UDP to server
[2016-04-20 08:50 BST] mlat(732): Results: 258.4 positions/minute
[2016-04-20 08:50 BST] mlat(732): Aircraft: 17 of 20 Mode S, 15 of 16 ADS-B used
[2016-04-20 08:55 BST] 50542 msgs recv'd from dump1090-mutab (438 in last 5m); 50542 msgs sent to FlightAware
[2016-04-20 09:00 BST] 51011 msgs recv'd from dump1090-mutab (469 in last 5m); 51011 msgs sent to FlightAware
[2016-04-20 09:05 BST] 51553 msgs recv'd from dump1090-mutab (542 in last 5m); 51553 msgs sent to FlightAware
[2016-04-20 09:05 BST] mlat(732): Receiver status: connected
[2016-04-20 09:05 BST] mlat(732): Server status: synchronized with 56 nearby receivers
[2016-04-20 09:05 BST] mlat(732): Receiver: 311.1 msg/s received 5.7kB/s from receiver
[2016-04-20 09:05 BST] mlat(732): Server: 0.6 kB/s from server 0.0kB/s TCP to server 1.2kB/s UDP to server
[2016-04-20 09:05 BST] mlat(732): Results: 287.9 positions/minute
[2016-04-20 09:05 BST] mlat(732): Aircraft: 17 of 24 Mode S, 12 of 17 ADS-B used
[2016-04-20 09:10 BST] 52113 msgs recv'd from dump1090-mutab (560 in last 5m); 52113 msgs sent to FlightAware
[2016-04-20 09:15 BST] 52720 msgs recv'd from dump1090-mutab (607 in last 5m); 52720 msgs sent to FlightAware
[2016-04-20 09:20 BST] 53361 msgs recv'd from dump1090-mutab (641 in last 5m); 53361 msgs sent to FlightAware
[2016-04-20 09:20 BST] mlat(732): Receiver status: connected
[2016-04-20 09:20 BST] mlat(732): Server status: synchronized with 56 nearby receivers
[2016-04-20 09:20 BST] mlat(732): Receiver: 435.8 msg/s received 7.9kB/s from receiver
[2016-04-20 09:20 BST] mlat(732): Server: 1.1 kB/s from server 0.0kB/s TCP to server 2.3kB/s UDP to server
[2016-04-20 09:20 BST] mlat(732): Results: 503.2 positions/minute
[2016-04-20 09:20 BST] mlat(732): Aircraft: 17 of 29 Mode S, 21 of 24 ADS-B used
[2016-04-20 09:25 BST] 54101 msgs recv'd from dump1090-mutab (740 in last 5m); 54101 msgs sent to FlightAware
[2016-04-20 09:30 BST] 54893 msgs recv'd from dump1090-mutab (792 in last 5m); 54893 msgs sent to FlightAware
[2016-04-20 09:35 BST] 55601 msgs recv'd from dump1090-mutab (708 in last 5m); 55601 msgs sent to FlightAware
[2016-04-20 09:35 BST] mlat(732): Receiver status: connected
[2016-04-20 09:35 BST] mlat(732): Server status: synchronized with 56 nearby receivers
[2016-04-20 09:35 BST] mlat(732): Receiver: 436.1 msg/s received 7.9kB/s from receiver
[2016-04-20 09:35 BST] mlat(732): Server: 1.1 kB/s from server 0.0kB/s TCP to server 2.0kB/s UDP to server
[2016-04-20 09:35 BST] mlat(732): Results: 481.3 positions/minute
[2016-04-20 09:35 BST] mlat(732): Aircraft: 21 of 30 Mode S, 19 of 22 ADS-B used
[2016-04-20 09:40 BST] 56160 msgs recv'd from dump1090-mutab (559 in last 5m); 56160 msgs sent to FlightAware
[2016-04-20 09:45 BST] 56745 msgs recv'd from dump1090-mutab (585 in last 5m); 56745 msgs sent to FlightAware
[2016-04-20 09:50 BST] 57240 msgs recv'd from dump1090-mutab (495 in last 5m); 57240 msgs sent to FlightAware
[2016-04-20 09:50 BST] mlat(732): Receiver status: connected
[2016-04-20 09:50 BST] mlat(732): Server status: synchronized with 47 nearby receivers
[2016-04-20 09:50 BST] mlat(732): Receiver: 395.8 msg/s received 7.1kB/s from receiver
[2016-04-20 09:50 BST] mlat(732): Server: 1.0 kB/s from server 0.0kB/s TCP to server 1.9kB/s UDP to server
[2016-04-20 09:50 BST] mlat(732): Results: 440.0 positions/minute
[2016-04-20 09:50 BST] mlat(732): Aircraft: 11 of 19 Mode S, 11 of 21 ADS-B used
[2016-04-20 09:58 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 09:58 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 09:58 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:02 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:02 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:02 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:02 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:02 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:02 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.191:9292:2848012015
[2016-04-20 10:02 BST] mlat(28779): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:02 BST] mlat(28779): Using UDP transport to 70.42.6.191:9292
[2016-04-20 10:02 BST] mlat(28779): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:02 BST] mlat(28779): Input connected to localhost:30005
[2016-04-20 10:02 BST] mlat(28779): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:07 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:07 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:07 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:07 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:07 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:07 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.197:9205:846292348
[2016-04-20 10:07 BST] mlat(29087): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:07 BST] mlat(29087): Using UDP transport to 70.42.6.197:9205
[2016-04-20 10:07 BST] mlat(29087): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:07 BST] mlat(29087): Input connected to localhost:30005
[2016-04-20 10:07 BST] mlat(29087): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:17 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:17 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:17 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:17 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:17 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:17 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.191:9318:734070912
[2016-04-20 10:17 BST] mlat(29543): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:17 BST] mlat(29543): Using UDP transport to 70.42.6.191:9318
[2016-04-20 10:17 BST] mlat(29543): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:17 BST] mlat(29543): Input connected to localhost:30005
[2016-04-20 10:17 BST] mlat(29543): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:24 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:24 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:24 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:24 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:24 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:24 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.197:9242:3746017439
[2016-04-20 10:24 BST] mlat(29819): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:24 BST] mlat(29819): Using UDP transport to 70.42.6.197:9242
[2016-04-20 10:24 BST] mlat(29819): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:24 BST] mlat(29819): Input connected to localhost:30005
[2016-04-20 10:24 BST] mlat(29819): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:25 BST] 60869 msgs recv'd from dump1090-mutab (462 in last 5m); 59784 msgs sent to FlightAware
[2016-04-20 10:30 BST] 61390 msgs recv'd from dump1090-mutab (521 in last 5m); 60305 msgs sent to FlightAware
[2016-04-20 10:40 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:40 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:40 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:40 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:40 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:40 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.191:9354:2864249941
[2016-04-20 10:40 BST] mlat(30572): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:40 BST] mlat(30572): Using UDP transport to 70.42.6.191:9354
[2016-04-20 10:40 BST] mlat(30572): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:40 BST] mlat(30572): Input connected to localhost:30005
[2016-04-20 10:40 BST] mlat(30572): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:40 BST] 62412 msgs recv'd from dump1090-mutab (490 in last 5m); 60891 msgs sent to FlightAware
[2016-04-20 10:47 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:47 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:47 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:47 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:47 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:47 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.197:9303:109164904
[2016-04-20 10:47 BST] mlat(30965): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:47 BST] mlat(30965): Using UDP transport to 70.42.6.197:9303
[2016-04-20 10:47 BST] mlat(30965): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:47 BST] mlat(30965): Input connected to localhost:30005
[2016-04-20 10:47 BST] mlat(30965): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:54 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:54 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 10:54 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:54 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 10:54 BST] multilateration data requested, enabling mlat client
[2016-04-20 10:54 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.198:9026:1163298978
[2016-04-20 10:54 BST] mlat(31396): fa-mlat-client 0.2.4 starting up
[2016-04-20 10:54 BST] mlat(31396): Using UDP transport to 70.42.6.198:9026
[2016-04-20 10:54 BST] mlat(31396): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 10:54 BST] mlat(31396): Input connected to localhost:30005
[2016-04-20 10:54 BST] mlat(31396): Beast-format results connection with localhost:30104: connection established
[2016-04-20 10:55 BST] 63823 msgs recv'd from dump1090-mutab (475 in last 5m); 61951 msgs sent to FlightAware
[2016-04-20 11:01 BST] autoUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 11:01 BST] manualUpdate is not configured in /etc/piaware or by piaware-config
[2016-04-20 11:01 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 11:01 BST] multilateration support enabled (use piaware-config to disable)
[2016-04-20 11:01 BST] multilateration data requested, enabling mlat client
[2016-04-20 11:01 BST] Starting multilateration client: /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --results beast,connect,localhost:30104 --results ext_basestation,listen,30106 --udp-transport 70.42.6.191:9393:1955270402
[2016-04-20 11:01 BST] mlat(31711): fa-mlat-client 0.2.4 starting up
[2016-04-20 11:01 BST] mlat(31711): Using UDP transport to 70.42.6.191:9393
[2016-04-20 11:01 BST] mlat(31711): Listening for Extended Basestation-format results connection on port 30106
[2016-04-20 11:01 BST] mlat(31711): Input connected to localhost:30005
[2016-04-20 11:01 BST] mlat(31711): Beast-format results connection with localhost:30104: connection established
2

FWIW, I did have a couple of “x% of MLAT reports sent were not received by FlightAware servers” yesterday, shown in the fA reports screen?

3

Note that the logs seen on the ADS-B stats page don’t show any messages logged while piaware is disconnected (for obvious reasons).

Since piaware logs when starting fa-mlat-client (or when it unexpectedly exits) and I don’t see those messages I would guess that you are periodically losing your connection to FA entirely (or perhaps the whole Pi is rebooting).

Check the local logs.

(edit: looking on the server side, yes, you are disconnecting regularly; I can’t tell why exactly from the server side)
(edit2: your current TCP connection with FA does not look healthy, I think you have network problems)

4

Thanks for the feedback @Obj - I’ll check the lan tonight…

5

@Obj, I’ve not looked at the local logs before, how do I get to / read those?

6

They are in /tmp/piaware.out