Dump1090 mutability 1.14 or 1.15dev performance?


#1

Past weekend I reinstalled my RaspberryPi ADS-B receiver.

After installing dump1090 mutability 1.14 and some testing I upgraded to dump1090 mutability 1.15dev
I had the impression that the number of aircraft shown in the web interface of dump1090 was a little bit less than running the 1.14 version.
A real side-by-side comparison is very difficult due to the lack of a complete 2nd and comparable setup.

What is your experience with the performance of the 2 versions?


dump1090 mutability - best configuration
FlightFeeder vs USB FlightAware Dongle
#2

i never testet 1.14 vs 1.15 - just other flavors of dump1090 vs 1.15 - where mutability 1.15 performed best.
did you set ALL soft switches e.g. gain etc. corresponding while testing …
did you look how other sites nearby performed over the days when you were testing …


#3

Yes, they where the same. I used the parameter “agc” so maybe the behavior of this is different in the 2 versions.

did you look how other sites nearby performed over the days when you were testing …

No, I did not compare this to other sites but is a good idea. I have 2 MicroSD card with the same setup with the different dump1090-muta versions.

What I did:

  • Installed 1.14. Looked at the amount of airplanes in the web interface for a couple of minutes and was seeing approx. 120 aircraft
  • Installed 1.15dev (takes a couple of minutes) and was seeing 80-90 aircraft and observed this for a couple of minutes
  • I reverted to 1.14 (using a cloned MicroSD with the install before of the setup before upgrading to 1.15dev) and was back with approx. 120 aircraft

Of course I can not exclude coincidence but the time frame in which this was all
done was in about 30 minutes.


#4

while not 100% shure i’d think agc aka -10 is in both versions the same - simply the highest possible gain.
but there are lots of other switches (e.g. --enable-agc --max-range --fix --phase-enhance --aggressive --oversample …) that could have influence to the number of aircrafts you see. did you set all the same way?

as i have some raspis laying around here from my testings some weeks ago - maybe i have the time over the weekend and do a side by side testing 1.14 vs 1.15 …

tom

p.s. the differences you saw (90 vs 120 planes) is really huge - try to shorten the time frame from 30 minutes down to 120 seconds - within 30 minutes my site often has differences about 30% …


#5

That is also what bugs me. From 120 to 90 and the back to 120 (is this due to the version or coincidence in airtraffic).
I will do some testing with two similar Raspberries where the only difference will be the Muta version.
The only “lag” I would have then is shutting down one raspberry, plugging the SDR receiver into the second one and rebooting.
I will spend some time on this on the weekend.


#6

why do you need two raspis therefore?
-> just take two sd-cards (one with 1.14 and another with 1.15)
-> boot from first sd-card and look how many planes you see
-> shutdown and change sd-card
-> boot from other sd-card and look how many

this way you can easily see traffic difference within 120 seconds


#7

It is a luxury “problem” :wink:.
The Raspi running my ADS-B setup is in a closed case and it is a bit tricky to open it and to remove the card.
I have a second simular Raspi for experimenting. Both boards are PI 2.


#8

ah - ok - understand :slight_smile: one of my raspis has an expensive case too - but it is possible to change sd-card without opening the whole thing http://discussions.flightaware.com/ads-b-flight-tracking-f21/world-best-raspberry-pi-case-arrived-today-t36010.html


#9

If 2 Ras Pis are used for comparison, it is not only setting which have to be identical, the hardware should also be identical:
–Antenna
–Antenna location
–Amplifier & filter if any (can have somewhat different performance even if of same make & model)
–Coax type, length and routing
–F to MCX pigtail
–DVB-T Dongle (can have somewhat different performance even if of same make & model)
–Ras Pi board

To overcome the differences of hardware, it is best to use the technique suggested by TomMuc - use only only one set of hardware, and swap the microSD card.


#10

I did some testing this evening.

Used setup

  • Raspberry PI 2 (two units)
  • MicroSD cloned, after cloning one MicroSD upgraded to Dump1090-Muta 1.15dev
  • Raspbian Jessie 4.1.12-v7+ #825 SMP PREEMPT Fri Nov 6 18:36:38 GMT 2015
  • SDR Nooelec NESDR Mini 2+ 0.5PPM TCXO RTL-SDR Receiver
  • Jetvision Diaspon MCX roof mount antenna

Procedure

Series 1
Step 1:

  • Reboot raspberry 1 with version 1.14
  • After reboot and response in dump1090 webpage wait 60 seconds
  • Take screenshot
  • Shutdown raspberry 1
  • Unplug power and SDR

Step 2:

  • Plugin SDR and power
  • Reboot raspberry 2 with version 1.15
  • After reboot and response in dump1090 webpage wait 60 seconds
  • Take screenshot
  • Shutdown raspberry 2

Series 2
Same as Series 1 but with order reversed(so first raspberry 2, then raspberry 1).

Time between screenshots: 132 seconds for series 1 and 145 for series 2 (including the 60 seconds stabilization).



                   Series 1             Series 2
                   1.14     1.15dev     1.14     1.15dev
Aircraft total     95       102         104      99
With positions     83       66          80       68
Messages/sec       886.4    833.9       903.7    900.3

The main difference I see is between the aircraft with positions.
I will repeat the test this weekend a couple of more.


#11

agree - the number of planes with positions is really mysterious :slight_smile:
but again i’d encourage you using the same raspi for testing and only switch the sd-card - just to exclude one more possible point of error …

p.s. what’s about ‘but there are lots of other switches (e.g. --enable-agc --max-range --fix --phase-enhance --aggressive --oversample …) that could have influence’ my question from above?


#12

The configuration of the dump1090 versions is 100% the same (checked and double checked) using sudo dpkg-reconfigure dump1090-mutability.
If I recall correctly there is somewhere a config file which contains more parameters. I this is changed with the installed version it could explain something.

What I also will try next, is checking with a file as input created with RTL_SDR, With this I should be able to excluded another factor.


#13

hmmmm - as i said before i’ll try to reproduce your findings with one of my raspis over the weekend. if (over here) there are planes at all while lufthansa strike continues :slight_smile:


#14

There is a strong possibility the difference is caused by hardware, and not software. I strongly recommend using SAME (not similar or identical, but SAME) hardware, and only changing software by swapping microSD card.


#15

If the “analog” part of the hardware used would be different, this can play a factor. But that is not the case since the same (not similar, the same) SDR and antenna is used.

The logic of the same hardware leading to different results vs. different software versions leading to different results is unlikely.
If the board/processor and other digital components would play a role, you would see differences in any software used and not only in dump1090.


#16

Are you using the 2MHz or the 2.4MHz demodulator? I don’t usually use the 2MHz one, maybe something broke there.

In the 2.4MHz path, it hasn’t really changed from 1.14, just some refactoring, so I’d be surprised if there was any real difference if you’re running with the same options.


#17

To identify if the problem is with hardware or software, interchange the microSD cards of two RPi and results will indicate where the problem lies.


#18

I am only using the 2.4Mhz path. I will try today with some more testing and with a file as input. What I already mentioned, it can be all coincidence because of fluctuation in traffic and still to little testing.


#19

This morning I have created a binary file with my “production” setup.


rtl_sdr -f 1090000000 -s 2400000 -g 0 output.bin

I transferred this the output.bin file to my test raspberry (same hardware!) and processed the file with


dump1090-mutability --ifile /home/pi/output.bin --oversample --modeac --phase-enhance --stats

Below are the results


Version 1.14                                                               Version 1.15dev
Statistics: Sat Nov 14 11:35:41 2015 CET - Sat Nov 14 11:36:36 2015 CET    Statistics: Sat Nov 14 11:43:52 2015 CET - Sat Nov 14 11:44:43 2015 CET
Local receiver:                                                            Local receiver:
 1086 sample blocks processed                                               142344192 samples processed
 0 sample blocks dropped                                                    0 samples dropped
 0 Mode A/C messages received                                               0 Mode A/C messages received
 1039866 Mode-S message preambles received                                  1007766 Mode-S message preambles received
 631734 with bad message format or invalid CRC                              611270 with bad message format or invalid CRC
 352651 with unrecognized ICAO address                                      341104 with unrecognized ICAO address
 55481 accepted with correct CRC                                            55392 accepted with correct CRC
 -10.6 dBFS mean signal power                                               -10.2 dBFS mean signal power
 -1.0 dBFS peak signal power                                                -1.1 dBFS peak signal power
 1376 messages with signal power above -3dBFS                               1367 messages with signal power above -3dBFS
55481 total usable messages                                                55392 total usable messages
0 surface position messages received                                       0 surface position messages received
2806 airborne position messages received                                   2801 airborne position messages received
2044 global CPR attempts with valid positions                              1869 global CPR attempts with valid positions
175 global CPR attempts with bad data                                      248 global CPR attempts with bad data
 0 global CPR attempts that failed the range check                          0 global CPR attempts that failed the range check
  175 global CPR attempts that failed the speed check                        248 global CPR attempts that failed the speed check
0 global CPR attempts with insufficient data                               0 global CPR attempts with insufficient data
69 local CPR attempts with valid positions                                 43 local CPR attempts with valid positions
  69 aircraft-relative positions                                             43 aircraft-relative positions
  0 receiver-relative positions                                              0 receiver-relative positions
518 local CPR attempts that did not produce useful positions               641 local CPR attempts that did not produce useful positions
  0 local CPR attempts that failed the range check                           0 local CPR attempts that failed the range check
  13 local CPR attempts that failed the speed check                          21 local CPR attempts that failed the speed check
0 CPR messages that look like transponder failures filtered                0 CPR messages that look like transponder failures filtered
                                                                           18322 non-ES altitude messages from ES-equipped aircraft ignored
128 unique aircraft tracks                                                 129 unique aircraft tracks
0 aircraft tracks where only one message was seen                          0 aircraft tracks where only one message was seen
CPU load: 71.9%                                                            CPU load: 75.8%
  38426 ms for demodulation                                                  34720 ms for demodulation
  1055 ms for reading from USB                                               3682 ms for reading from USB
  18 ms for network input and background tasks                               27 ms for network input and background tasks
Sat Nov 14 11:36:36 2015 CET  Normal exit.                                 Sat Nov 14 11:44:43 2015 CET  Normal exit.

There are clear differences. I do not understand every line and impact on performance.
In the 1.14 stats there is no occurrence of the line "xyz non-ES altitude messages from ES-equipped aircraft ignored"
Based on the lines “69 aircraft-relative positions” (1.14) and “43 aircraft-relative positions” (1.15dev), a difference in performance could be understood.

Is there somebody on the forum who can give an interpretation of the results?


#20

The demodulator-level number is this one:



55481 total usable messages                                                55392 total usable messages


This is a count of valid messages that the demodulator found. I would have expected them to be identical in a perfect world but obviously something has drifted a little. That said, there is only a 0.1% difference here, which is down in the noise.

1.15 actually saw one extra aircraft in total.

The other main difference is in the CPR stats; these counters cover the interpretation of valid messages from the demodulator as aircraft positions. This area of the code has changed significantly in 1.15 to try to throw out a lot more bad data e.g. improbable aircraft movement, filtering out temporary drops in NUCp which tend to happen when an aircraft loses confidence in its GPS position briefly. So I guess that’s where your differences are coming from - 1.14 may be generating more positions, but some of them are probably suspect data.

You may want to retest with receiver location / receiver range provided as that affects how the CPR logic works.

Note that piaware takes the raw messages and does its own interpretation (in faup1090) which is based on the 1.15 code anyway, so all that really matters for piaware feeding purposes is the raw messages from the demodulator. The position decoding stuff only affects your local use of the data.