Why do you use dump1090-mutability?

FWIW, --net-verbatim is forced on when aggressive mode is enabled so that downstream consumers (like piaware!) have a chance to notice and discard that untrustworthy data (it means that the original, uncorrected, message goes to the network - the client can then decide how it wants to correct or discard the message). It’s really unfortunate if people are actually going and bypassing that - then we get junk but have no reliable way of knowing it’s junk.

3 Likes

This is what they claim:

“… it’s better because it compared the dongle with the receiver with one antenna and the dna receives it much further. Especially with the 1.14 version of the dump and enabled aggressive mode, phase-enhance, and oversample.”

The discussion at Russian forum about dump1090 ver 1.14 was triggered when someone complained of poor mlat stats for his recently installed Radarcape compared to mlat stats of his Flightfeeder.

He later clarified that he is NOT using dump1090 as decoder. His clarification and dump1090 configuration is given below:

.
.

Nearby are two stations - ffa1978 & nejron
ffa1978:

6.10.2019
Positions Reported Total: 126,062

nejron

Positions Reported Total: 57 315

That’s a fairly meaningless comparison if you’re trying to compare the demodulator, since they’re in different locations with different antennas and different receiver hardware (one is a beast-gps)

1 Like

I conducted an experiment after reading this topic.
Uninstall 1090-fa, then install v1.14.
POSITIONS REPORTED 8,314 (FA), 10,288 (v1.14).
This experiment was short lived… After installation rbfeeder on PiAware the system crashed and now offline.

1 Like

I have some signal generation work in (slow) progress that should allow for a good comparison without relying on unpredictable live data. It’s some time away but I’ll include the 1.14 demodulator in my tests when it’s ready. I’d be really surprised if the 2MHz (1.14) demodulator is better, though - I did extensive live comparisons between the 2.0 and 2.4MHz demodulators when I was writing the 2.4MHz version and the 2.4MHz version was consistently better.

3 Likes

Perhaps different mobile communication standards have different interference with the desired signal. Or other regional features.

I have some preliminary empirical results and they match what I expected - the 2.4MHz demodulator is much better than the 2MHz demodulator.

This is from synthesizing ADS-B messages on a LimeSDR mini at various signal strengths and looping the signal back to a Prostick running whichever version of dump1090, then measuring the proportion of messages at each gain decoded by the receiver. (different message contents are used for each signal level, so the infrastructure can match up received messages with transmitted signal levels)

v1.14 never gets above about 70%, while dump1090-fa v3.8.0 reaches 100% over about a 25dB range. The overall dynamic range is similar if you’re generous with what you consider an acceptable reception rate.

Both versions were run with --fix --gain 49 and ran on the same receiver hardware.

7 Likes

Russian forum’s response to @obj’s above comments:
(now they host it in domain .in instead of .ru)

https://forum.adsb.in/f9/airnav-radarbox-comstation-ads-b-mlat-priemnik-1854/index19.html#post48198

1 Like

Are you also able to perform the same comparison using multiple synthesized sources at various strengths instead of a sequential loopback? Also curious how v1.15 stacks up in that scenario.

Sequential loopback isn’t very telling unless you are also introducing random noise into the signal to better replicate true signals - at least i would’t think since everything should fly right through to the final phases of demodulation. That said, better than nothing and perhaps I’m missing something still.

I don’t understand what you’re asking; how would using multiple sources (I assume you mean multiple transmitting SDRs?) change anything? What do you mean by “sequential loopback”? The test message sequence is repeated a large number of times to try to eliminate any e.g. phase sensitivity; I could certainly randomize the order of message transmission but given that there’s already 1ms gaps between messages I’d be very surprised if that changed anything.

I do have a mode which introduces AWGN (in addition to whatever you’re getting naturally through the transmit/receive chain) but that’s more for working out the SNR needed to decode, not absolute sensitivity.

I don’t have a good setup for testing fruit yet, which is maybe what you’re asking about.

I can’t reproduce this either. In my testing, 1.14 regular < 1.14 --oversample << 1.15 --oversample < 3.8.0. 1.14 with --oversample has worse sensitivity but better overall reception compared with 1.14 without --oversample. See below for results. All were run at gain 49 with --fix.

(I am really curious about the “step” at about -42dBFS, it’s a feature I’ve seen consistently in all my dump1090 testing so far, but it’s not visible when testing with a Beast and it’s still present under all sorts of gain settings so it does seem like a demodulator thing not a transmit-side problem)

2 Likes

Yes, that’s more or less what I was asking about. By sequential, I was referring to your message pipeline (orderly looping of messages back to the Prostick with 1ms gaps from the LimeSDR as you stated). Multiple feeds/loopbacks would introduce message overlap with differential SNR’s which would be closer to a real-world test since we don’t see orderly messages unless there is only a single transmitter within range without other interference.

Again, I could be completely misunderstanding your setup and testing method here, so hopefully this makes more sense of what I was trying to get at. :slight_smile:

Yeah, currently the testing deliberately avoids any sort of message overlap; testing fruit (overlapping messages) is definitely planned, but I need to work out what a sensible thing to generate/measure there is. Just randomly smushing a bunch of messages together won’t be great…

2 Likes

He is not ready to budge…

REPLY 1: (Click here to see original reply in Russian)
not so much worse than 1.15 compared to 1.14 with oversamble. I didn’t check on simulated signals, but in the real world. I just trash 1.15 even with changes to the code and enable agressive mode. I’ll compile 3.8 and see for fun. It entered there at 3.8 analogue of agressive mode. Well, I’ll cut out the default verbatim option, although I tried the same 2-bit correction earlier (in addition to agreesive) - it doesn’t give anything (which is understandable because agressive is 2 bit crc correction). It did not change demod2400 itself 3 years, so it makes no sense to compile

REPLY 2: (Click here to see original reply in Russian)
compiled, removed the verbatim restriction … At the level of 1.14 it is accepted + -. What it has in the graphs that it makes such a difference, I don’t know. Probably in the spherical vacuum, and so, in real life it’s no better than 3.8. Besides the source code I had to edit .Upd. I checked the number of messages in the planeplotter - we expect the result, + - 2-3 messages are the difference. But version 3.8 doesn’t rubbish with the same removed verbatim restrictions in 1.14 - everything is fine. So I stay on the right 1.14

.

Thanks for relaying. Everything I can find (both real-world and controlled tests) say that 3.8.0’s demodulator is much better than 1.14’s; without some hard data from him there’s nothing more I can do here. I don’t know what he’s saying about verbatim mode.

@obj
You have done enough to prove your point, but he is not ready to budge. I wont try to convince him any more.

Thanks a lot for all your input & help.

I’d expect that he wants 2 bit corrected data output for his VRS.
Because VRS won’t be able to deal with verbatim data as it can’t 2 bit correct itself.
(not sure if it even can correct 1 bit)

Did 1.14 already only forward the 2nd message for each ICAO?
I suppose if he wants all the crap messages, that would be a “feature” as well that you immediately send a message and not wait for a 2nd even if it’s very likely bogus.

I can understand that he doesn’t want to deal with added restrictions placed on the user by dump1090-fa. It has artificial restrictions to combat “bad data”, that makes it annoying to deal with for people who want every last maybe even bogus message.
I’d not be surprised that if it may be that he’s seeing some bogus messages with 1.14 which then tells him that it’s superior.
(For testing he compiled after turning off that verbatim restriction in the source)

While he refers to tests, i don’t see any data or graphs provided.
I could also imagine he uses some statistics that somehow benefit 1.14. Which could mean that the statistics are just different while the actual demodulation is better with 3.8.

Anyhow that’s all speculation.
I don’t get the impression he’s that interested in proving anything.
Simply fine with using the old version.

1 Like

Well, that would be the thing to fix…

1 Like