Multilateration (MLAT) now available on PiAware!

[JonAdams]

@JonAdams

@dstreufe

I have a Pi2 and do not have that problem. If you run the “top” command, what is it showing as chewing up the CPU?

top 30 or so processes.

Will also point out that right now i’m not getting the anomaly, only occasionally, so the next time I’m getting the notice I will try the “TOP” command again.;

cheers and 73 - Jon N7UV

[dstreufe]

@JonAdams

@JonAdams

@dstreufe

I have a Pi2 and do not have that problem. If you run the “top” command, what is it showing as chewing up the CPU?

top 30 or so processes.

Will also point out that right now i’m not getting the anomaly, only occasionally, so the next time I’m getting the notice I will try the “TOP” command again.;

cheers and 73 - Jon N7UV

Does it say something like “might be due to high CPU utilization”? I get the same thing from time to time, but my CPU utilization is never high. I think that’s a red herring…

[fusionimage]

the message rate and mlat quality is dependant on the cpu load.
i have dump1090-mutability, piaware, fr24feed and modesmixer2 running on one rpi and get an average msg rate of 250/s. if i ONLY run dump1090, i get over 900/s

[dstreufe]

@fusionimage

the message rate and mlat quality is dependant on the cpu load.
i have dump1090-mutability, piaware, fr24feed and modesmixer2 running on one rpi and get an average msg rate of 250/s. if i ONLY run dump1090, i get over 900/s

Is that on a Pi 2?

[lk2855]

I am finding that using the Pi B+ I am running close to 100% at all of my sites. I have dump1090 mutability (1.15 dev), piaware, FR24feed and PFClient running. Before MLAT the utilization was reasonable (80% or less). Now, I am finding the sites becoming increasingly unstable, likely due to CPU overload. I frequently can’t connect via SSH or use the dump1090 web page although it seems to be sending data to FA. I can use the local console but since most of these run “headless” that’s not an option.

dump1090-mutability appear to be using about 50-55% of the CPU (two processes) and the FA mlat client another 5%.

The one site (the busiest) that I have upgraded to Pi 2 is fine. It is not running FR24Feed as they don’t like the MLAT data being inserted and I use Virtual Radar Server to aggregate all on my sites.

I am also running collectd and rrdtool to gather statistics. I’m not sure how much overhead these have but I do see the CPU utilization peak when the CRON job runs.

I’m open to any suggestions on how to lower the CPU load on these sites. I’d rather not replace all of the Pi B+ with Pi 2. I am going to start with a fresh SD card on my test site tonight and see how that goes.

-Larry

[fusionimage]

@dstreufe

Is that on a Pi 2?

nope, rpi 1 with 512mb ram

with only dump1090 the cpu load is around 80% where i get ~900msgs/s

[PeterHR]

@lk2855

I am finding that using the Pi B+ I am running close to 100% at all of my sites. I have dump1090 mutability (1.15 dev), piaware, FR24feed and PFClient running. Before MLAT the utilization was reasonable (80% or less). Now, I am finding the sites becoming increasingly unstable, likely due to CPU overload. I frequently can’t connect via SSH or use the dump1090 web page although it seems to be sending data to FA. I can use the local console but since most of these run “headless” that’s not an option.

dump1090-mutability appear to be using about 50-55% of the CPU (two processes) and the FA mlat client another 5%.

The one site (the busiest) that I have upgraded to Pi 2 is fine. It is not running FR24Feed as they don’t like the MLAT data being inserted and I use Virtual Radar Server to aggregate all on my sites.

I am also running collectd and rrdtool to gather statistics. I’m not sure how much overhead these have but I do see the CPU utilization peak when the CRON job runs.

I’m open to any suggestions on how to lower the CPU load on these sites. I’d rather not replace all of the Pi B+ with Pi 2. I am going to start with a fresh SD card on my test site tonight and see how that goes.

-Larry

My PiB (before upgrade to Pi2) I ran with a medium overclock (set by sudo raspi-config) - this made a big difference to the utilisation.

IF you are running dump1090-mutability, are you running v1.15dev - This has:

• a switch that by default does not pass the MLAT data back out to other feeders
• the ability to use lightttpd as the webserver rather than the code built into dump1090 (http ip-address:8080 - using built in, ip-address/dump1090 - using lightttpd)

My upgrade to Pi2 was brought on by the additional compute resource needed for the MLAT client.

[lk2855]

@PeterHR

@lk2855

I am finding that using the Pi B+ I am running close to 100% at all of my sites. I have dump1090 mutability (1.15 dev), piaware, FR24feed and PFClient running. Before MLAT the utilization was reasonable (80% or less). Now, I am finding the sites becoming increasingly unstable, likely due to CPU overload. I frequently can’t connect via SSH or use the dump1090 web page although it seems to be sending data to FA. I can use the local console but since most of these run “headless” that’s not an option.

dump1090-mutability appear to be using about 50-55% of the CPU (two processes) and the FA mlat client another 5%.

The one site (the busiest) that I have upgraded to Pi 2 is fine. It is not running FR24Feed as they don’t like the MLAT data being inserted and I use Virtual Radar Server to aggregate all on my sites.

I am also running collectd and rrdtool to gather statistics. I’m not sure how much overhead these have but I do see the CPU utilization peak when the CRON job runs.

I’m open to any suggestions on how to lower the CPU load on these sites. I’d rather not replace all of the Pi B+ with Pi 2. I am going to start with a fresh SD card on my test site tonight and see how that goes.

-Larry

My PiB (before upgrade to Pi2) I ran with a medium overclock (set by sudo raspi-config) - this made a big difference to the utilisation.

IF you are running dump1090-mutability, are you running v1.15dev - This has:

• a switch that by default does not pass the MLAT data back out to other feeders
• the ability to use lightttpd as the webserver rather than the code built into dump1090 (http ip-address:8080 - using built in, ip-address/dump1090 - using lightttpd)

My upgrade to Pi2 was brought on by the additional compute resource needed for the MLAT client.

I am running 1.15dev. I have it sending the MLAT data out so I can see it in VRS. I know there is a way of sending the standard data out the default port and the combined MLAT/ADB-B out an alternate port. I just haven’t figured out all of the command line arguements to make it happen. I am using lighttpd as the webserver.

I currently have 5 active sites and would rather not have to buy new Pi 2’s for each of the locations. More likely I’ll kill off FR24Feed, PlaneFinder and the reports before buying new hardware.

[PeterHR]

@lk2855

…snip…
I am running 1.15dev. I have it sending the MLAT data out so I can see it in VRS. I know there is a way of sending the standard data out the default port and the combined MLAT/ADB-B out an alternate port. I just haven’t figured out all of the command line arguements to make it happen. I am using lighttpd as the webserver.

I currently have 5 active sites and would rather not have to buy new Pi 2’s for each of the locations. More likely I’ll kill off FR24Feed, PlaneFinder and the reports before buying new hardware.

The data aggregation sites apart from FR24 have released feeders to ignore the MLAT data

I think it may come down to a choice of feed FR24 (block the MLAT data appearing at dump1090 output) or use VRS (let that data through).

I’d choose the second - since FR24 have had the opportunity fix their client, and haven’t chosen to do it (Pi feeders are not important enough, they are only really interested in the FR24 Radarcapes they control)

[JonAdams]

@dstreufe

@JonAdams

@JonAdams

top 30 or so processes.

Will also point out that right now i’m not getting the anomaly, only occasionally, so the next time I’m getting the notice I will try the “TOP” command again.;

cheers and 73 - Jon N7UV

Does it say something like “might be due to high CPU utilization”? I get the same thing from time to time, but my CPU utilization is never high. I think that’s a red herring…

The anomaly is reported as:
“Anomaly report for PiAware feeder with a MAC address of b8:27:eb:76:2c:6f:
CPU load of 92% is fairly high. You might consider running fewer programs on your Pi, or upgrading your Pi to a faster version such as the Pi 2.”

Since I don’t monitor it closely, I can’t tell how often the Pi2 is in this state. Here’s a brief dump of the last few minutes on the log:

“2015-08-28 07:26 MST] 1515102 msgs recv’d from dump1090 (1971 in last 5m); 1514379 msgs sent to FlightAware
[2015-08-28 07:31 MST] 1517107 msgs recv’d from dump1090 (2005 in last 5m); 1516384 msgs sent to FlightAware
[2015-08-28 07:36 MST] 1519190 msgs recv’d from dump1090 (2083 in last 5m); 1518467 msgs sent to FlightAware
[2015-08-28 07:40 MST] mlat(7660): Receiver connection: ready
[2015-08-28 07:40 MST] mlat(7660): Server connection: ready
[2015-08-28 07:40 MST] mlat(7660): Receiver: 316.0 msg/s received 5.5kB/s from receiver
[2015-08-28 07:40 MST] mlat(7660): Server: 0.8 kB/s from server 0.0kB/s TCP to server 2.8kB/s UDP to server
[2015-08-28 07:40 MST] mlat(7660): Results: 361.2 positions/minute
[2015-08-28 07:40 MST] mlat(7660): Aircraft: 142 known, 96 requested by server
[2015-08-28 07:41 MST] 1521190 msgs recv’d from dump1090 (2000 in last 5m); 1520467 msgs sent to FlightAware
[2015-08-28 07:46 MST] 1523138 msgs recv’d from dump1090 (1948 in last 5m); 1522415 msgs sent to FlightAware”

Any ideas?

Cheers and 73 - Jon N7UV

[PeterHR]

@JonAdams

…snip…

The anomaly is reported as:
“Anomaly report for PiAware feeder with a MAC address of b8:27:eb:76:2c:6f:
CPU load of 92% is fairly high. You might consider running fewer programs on your Pi, or upgrading your Pi to a faster version such as the Pi 2.”

Since I don’t monitor it closely, I can’t tell how often the Pi2 is in this state. Here’s a brief dump of the last few minutes on the log:

“2015-08-28 07:26 MST] 1515102 msgs recv’d from dump1090 (1971 in last 5m); 1514379 msgs sent to FlightAware
[2015-08-28 07:31 MST] 1517107 msgs recv’d from dump1090 (2005 in last 5m); 1516384 msgs sent to FlightAware
[2015-08-28 07:36 MST] 1519190 msgs recv’d from dump1090 (2083 in last 5m); 1518467 msgs sent to FlightAware
[2015-08-28 07:40 MST] mlat(7660): Receiver connection: ready
[2015-08-28 07:40 MST] mlat(7660): Server connection: ready
[2015-08-28 07:40 MST] mlat(7660): Receiver: 316.0 msg/s received 5.5kB/s from receiver
[2015-08-28 07:40 MST] mlat(7660): Server: 0.8 kB/s from server 0.0kB/s TCP to server 2.8kB/s UDP to server
[2015-08-28 07:40 MST] mlat(7660): Results: 361.2 positions/minute
[2015-08-28 07:40 MST] mlat(7660): Aircraft: 142 known, 96 requested by server
[2015-08-28 07:41 MST] 1521190 msgs recv’d from dump1090 (2000 in last 5m); 1520467 msgs sent to FlightAware
[2015-08-28 07:46 MST] 1523138 msgs recv’d from dump1090 (1948 in last 5m); 1522415 msgs sent to FlightAware”

Any ideas?

Cheers and 73 - Jon N7UV

If it’s not overclocked - I’d check how warm the chip is (command “vcgencmd measure_temp” ), possible increase ventilation and overclock it to medium (sudo raspi-config)

[sjacket99]

I been running Wheezy with Dump1090-Mut V1.15. I been running around 37-40% CPU usage on my RPi2 just for Dump1090-Mut. Today i loaded up another SD card with just piaware image 2.1-2. Now I am running around 26-30% for Dump1090. Has anyone else notice this?

[N456TS]

@sjacket99

I been running Wheezy with Dump1090-Mut V1.15. I been running around 37-40% CPU usage on my RPi2 just for Dump1090-Mut. Today i loaded up another SD card with just piaware image 2.1-2. Now I am running around 26-30% for Dump1090. Has anyone else notice this?

It’s different programs with different configurations. One doing more then the other.

[Fill]

for the site admins : now that mlat has been cooking well the past few months, i’m wondering what’s next in terms of the hobbyist/professional flight tracking scene. have we reached the “top of the mountain” so to speak, with mlat?

[dbaker]

@fill

for the site admins : now that mlat has been cooking well the past few months, i’m wondering what’s next in terms of the hobbyist/professional flight tracking scene. have we reached the “top of the mountain” so to speak, with mlat?

The first step was to get it working, then to expose it to PiAware directly, and now we’re waiting on some back-end performance improvements before we implement it on the web site. This will also let us improve position frequency on both MLAT and ADSB.

[PeterHR]

@JonAdams

@dstreufe

I have a Pi2 and do not have that problem. If you run the “top” command, what is it showing as chewing up the CPU?

top 30 or so processes.

 1924 root      20   0 15908 6972 1728 R  45.9  1.6   1175:36 dump1090
 2158 root      20   0 11220 9536 5208 S  32.7  2.1 630:41.60 fa-mlat-client
 2096 root      20   0 17288 7828 4508 S   4.5  1.8  61:22.43 piaware
 2156 root      20   0  2652 2124 1860 S   1.3  0.5  17:53.15 faup1090
 4522 pi        20   0  4672 2488 2080 R   1.0  0.6   0:01.62 top
    3 root      20   0     0    0    0 S   0.6  0.0  10:14.53 ksoftirqd/0
    7 root      20   0     0    0    0 S   0.6  0.0   9:21.07 rcu_preempt
 4507 pi        20   0  9264 3584 2996 S   0.3  0.8   0:00.19 sshd
    1 root      20   0  2148 1328 1224 S   0.0  0.3   0:08.95 init
    2 root      20   0     0    0    0 S   0.0  0.0   0:00.00 kthreadd
    5 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 kworker/0:0H
    8 root      20   0     0    0    0 S   0.0  0.0   0:00.00 rcu_sched
    9 root      20   0     0    0    0 S   0.0  0.0   0:00.00 rcu_bh
   10 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 khelper
   11 root      20   0     0    0    0 S   0.0  0.0   0:00.01 kdevtmpfs
   12 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 netns
   13 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 perf
   14 root      20   0     0    0    0 S   0.0  0.0   0:00.15 khungtaskd
   15 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 writeback
   16 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 crypto
   17 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 bioset
   18 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 kblockd
   19 root      20   0     0    0    0 S   0.0  0.0   2:48.24 kworker/0:1
   20 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 rpciod
   21 root      20   0     0    0    0 S   0.0  0.0   0:00.00 kswapd0
   22 root      20   0     0    0    0 S   0.0  0.0   0:00.00 fsnotify_mark
   23 root       0 -20     0    0    0 S   0.0  0.0   0:00.00 nfsiod

Any ideas would be appreciated…

Cheers and 73 - Jon N7UV

Mi Pi2 is handing about 50% more data than yours, this is what TOP looks like for me…
top - 07:40:02 up 16 days, 17:22, 1 user, load average: 0.42, 0.41, 0.42

Tasks:  88 total,   1 running,  87 sleeping,   0 stopped,   0 zombie
%Cpu(s): 10.0 us,  2.1 sy,  0.0 ni, 87.2 id,  0.0 wa,  0.0 hi,  0.7 si,  0.0 st
KiB Mem:    949408 total,   228024 used,   721384 free,    82728 buffers
KiB Swap:   102396 total,        0 used,   102396 free,    74908 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 1979 dump1090  15  -5 27400  15m 1884 S  30.1  1.7   6112:28 dump1090-mutabi
23916 root      20   0 11224 9528 5240 S  11.6  1.0 491:32.43 fa-mlat-client
 7477 root      20   0 50816 7356 2172 S   2.6  0.8 523:12.85 pfclient
 2162 root      20   0 17820 8252 4536 S   2.0  0.9 360:38.64 piaware
 2447 root      20   0  2656 1588 1420 S   1.0  0.2 209:27.20 faup1090
30714 pi        20   0  4676 2416 2072 R   0.7  0.3   0:00.09 top
    3 root      20   0     0    0    0 S   0.3  0.0  22:12.37 ksoftirqd/0
    7 root      20   0     0    0    0 S   0.3  0.0  65:50.88 rcu_preempt

Are you getting dump1090 to do more work by having ‘aggressive’ turned on or is is spending more time with error correction

maybe post the last block ot /var/log/dump1090-mutability.log … lets see if there are clues there.

[later edit]
hmm, you’re not running dump1090-mutability, don’t know where the log file will be.

[obj]

I’ve noticed that US-based feeders use quite a bit more CPU in fa-mlat-client for the same sort of message rate, because there’s a much higher proportion of non-ADS-B traffic compared to Europe.

[Fill]

2020 so close yet so far…

[joelwiley]

@fill

2020 so close yet so far…

In that timeframe, Stalin revamped the Russian Steel industry.

[bidgee]

I’ll be switching MLAT back on, need to test out the load on the Pi (since install a logger for the ADSB message rate, aircraft seen, networking ect but still need to develop a way of showing I’m feeding on the local Pi page) but does FA’s MLAT work with Dump1090 mutability?