Working on native AirSpy support for dump1090-fa

[gtj0]

@prog, @obj, yeah thanks! Now I have to find the code to implement it.

[SoNic67]

Github has some implementations. Search “Finite Impulse Response filter”.
Some are written in C… and could be adapted:

sclark003/FIR-Lowpass-Filter-C/blob/master/Lowpass filter FIR code C

[gtj0]

Thanks @SoNic67. I also have Matlab but I have to make sure my license covers generating C source code for open-source projects.

[gtj0]

@obj Thoughts on using octave or gnuradio functions in dump1090-fa?
Oh…or liquid-dsp.

[JRG1956]

Can I ask a few questions, of anyone who cares to respond, to help those of us “following along at home”?

My direct experience of digital signal processing is 40 years old was in the completely different frequency domain of seismic data processing, so I will not be able to offer much here apart from code testing. My questions around the problem(s) being solved and the likely quantum of benefits:

1. First off, it’s important to recognise that there don’t need to be any benefits apart from the pleasure the project gives to those involved.

2. Also, replacing airspy_adsb with an end to end open source solution would clearly yields benefits in terms of transparency, risks around long-term support and potential for others to contribute to ongoing improvement.

3. Acknowledging the above, how much of an improvement in message detection rates can be expected from native airplay support in dump1090-fa relative to feeding via airspy_adsb?

4. Where is that improvement likely to come from?

5. Will you need an R2 to see a benefit, or will those of us with Airspy Minis also be able to play?

6. Finally, it seems that one motivation is the philosophy that, generally, higher sample rates are better, and that native support will facilitate this and result in more successful decodes. Is that a misreading on my part? If not, isn’t USB2 bandwidth going to put an upper bound on what can be achieved with the current generation of hardware?

I look forward to following progress.

[obj]

@gtj0

@obj Thoughts on using octave or gnuradio functions in dump1090-fa?
Oh…or liquid-dsp.

octave is probably way too heavyweight.

libvolk (gnuradio) is a possibility, though it has two main problems: GPL licensing (not an issue for dump1090), and a lack of kernels that address the specific problems that dump1090 wants dsp stuff for.

liquid-dsp is a possibility. When I looked at it previously it didn’t seem to have much SIMD support.

[caius]

@JRG1956

• Acknowledging the above, how much of an improvement in message detection rates can be expected from native airplay support in dump1090-fa relative to feeding via airspy_adsb?
• Where is that improvement likely to come from?

I think that getting dump1090 somewhere in the ballpark of the current decoder would be quite an achievement. airspy_adsb is really quite well optimised now, and I don’t know that there is much improvement to be had in terms of message rates. After all, the decoder can only extract data that is there - if nearly all messages that are possible to be discriminated by the hardware are already being decoded, then by definition there is only a small performance gain possible.

There’s more than one way to measure performance however, and it’s possible a different DSP approach could yield a decoder that doesn’t require as much in the way of CPU resources which might allow better performance on lower powered hardware.

I think if improved performance is the goal then I suspect that will be difficult to achieve, but as you say there are other motivations that make it a worthwhile exercise.

[gtj0]

Great questions!

@JRG1956

• First off, it’s important to recognise that there don’t need to be any benefits apart from the pleasure the project gives to those involved.

Quite true! Well, as long as the effort doesn’t cause any negative effects. @obj’s comment about octave is a case in point. I wouldn’t want to cause bloat for my own pleasure.

@JRG1956

Also, replacing airspy_adsb with an end to end open source solution would clearly yields benefits in terms of transparency, risks around long-term support and potential for others to contribute to ongoing improvement.

That was part of my motivation along with lowering the barrier for users with AirSpy devices to get a working setup.

@JRG1956

Acknowledging the above, how much of an improvement in message detection rates can be expected from native airplay support in dump1090-fa relative to feeding via airspy_adsb?

To be honest, I’m not sure. The folks involved with airspy_adsb certainly know their stuff but having the code available in dump1090-fa means that even more people can contribute their expertise. Even if the dump1090-fa implementation is on par with airspy_adsb, the benefits mentioned above still apply.

@JRG1956

Where is that improvement likely to come from?

Unknown at this time.

@JRG1956

Will you need an R2 to see a benefit, or will those of us with Airspy Minis also be able to play?

The Mini can still do 10MS/s and has the same 12 bit ADC so it should see the same benefits.

@JRG1956

Finally, it seems that one motivation is the philosophy that, generally, higher sample rates are better, and that native support will facilitate this and result in more successful decodes. Is that a misreading on my part? If not, isn’t USB2 bandwidth going to put an upper bound on what can be achieved with the current generation of hardware?

It is and it isn’t. You can’t get IQ format data 20 or 24 MS/s on a USB2 bus but you CAN get “real” format data at those rates. With packing turned on it results in about 283 Mb/s on the USB bus. I’m still thinking about whether to reconstruct the IQ format or use the real data directly.

[gtj0]

@caius

I think if improved performance is the goal then I suspect that will be difficult to achieve, but as you say there are other motivations that make it a worthwhile exercise.

Exactly. Improved performance could be a side benefit if other smart folks can get involved but was never the primary goal.

[obj]

@gtj0

You can’t get IQ format data 20 or 24 MS/s on a USB2 bus but you CAN get “real” format data at those rates. With packing turned on it results in about 283 Mb/s on the USB bus. I’m still thinking about whether to reconstruct the IQ format or use the real data directly.

AIUI, the airspy host libs will already give you that IQ format, no reconstruction needed: 10MSPS complex is 20MSPS real at the hardware level; 12MSPS complex is 24MSPS real at the hardware level. The reason for halving the sampling rate is that you don’t “need” the extra sample rate to represent the available bandwdith (e.g. 20MSPS real can represent 10MHz of bandwidth; you only need 10MSPS complex to represent 10MHz of bandwidth)

You could derive e.g. 20MSPS IQ from 20MSPS real if you wanted, but there’s probably not a huge benefit to it. You won’t magically get a better shaped waveform (that is limited by the bandwidth available at the ADC) but you’d be able to select a pulse sampling point with finer resolution. Not useful if you’re just decimating it down to 2.4 though.

Re bandwidth the interesting bandwidths for ADS-B are probably related to the odd harmonics of the modulation rate (since it looks something like a square wave) - i.e. you’ll probably see most improvement in pulse shape when you can capture +/-1MHz (>2MHz bandwidth), +/-3Mhz (>6MHz bandwidth), +/-5MHz (>10MHz bandwidth), etc.

[gtj0]

It’ll give you the IQ format but because it’s 2 16 bit values per sample, you can’t sample at 20 or 24 MS/s because even with packing it exceeds the USB2 bandwidth. So we can get real INT16 at 20 or 24 but I was contemplating whether we should convert it to IQ format internally because that’s that the rest of dump1090 expects or just make changes so we can use the real format directly. I haven’t had a chance to look at the converters or demod_2400 in depth yet.

[obj]

@gtj0

So we can get real INT16 at 20 or 24 but I was contemplating whether we should convert it to IQ format internally

My point is that “get real int16 at 20MSPS and convert it to IQ format” is exactly what the airspy host lib is already doing when you ask for 10MSPS, IQ format.

If you wanted 20MSPS IQ with full bandwidth from a single channel ADC, you’d need to sample at 40MHz. (And the R820T2 IF filters don’t go that wide anyway…)

[gtj0]

I see the code in libairspy that does the iqconvert but that doesn’t match with what I see as far as usb transfers go…

airspy_rx -r tempiq -f 1090 -t 3 -a 12000000 -g 21 -p 0 -n 25600000
is Int16 REAL and runs the USB bus at about 189 Mb/s.

airspy_rx -r tempiq -f 1090 -t 2 -a 12000000 -g 21 -p 0 -n 25600000
is Int16IQ and runs the USB bus at about 344 Mb/s.

[obj]

That seems fine to me?

If you ask for 12MSPS real, the hardware produces 12e6 real samples/sec and you get that directly. This is 6MHz of RF bandwidth.

If you ask for 12MSPS complex, the hardware produces 24e6 real samples/sec, and the host lib converts that to the 12e6 IQ samples/sec that you asked for. This is 12MHz of RF bandwidth.

Twice as much RF bandwidth needs twice as much USB bandwidth.

Here is where the doubling of the hardware sample rate when you request an IQ sample format happens.

[gtj0]

Ah I get it now. I think. You don’t need complex to construct IQ though do you? The original real sample becomes I, and Q is I shifted by 90 degrees which you can do with Hilbert.

In any case, if we really don’t need IQ then it’s a moot point.

[obj]

Not quite. You can use a Hilbert transform to generate an analytic signal from a real signal, but that’s not what is happening here.

What’s happening here is the host lib is doing the digital equivalent of what you’d do in the analog world to generate an I/Q signal - mixing the real signal with two oscillators with a 90 degree phase difference. This both generates the I/Q signals and also does the final frequency shift, downconverting from the IF to baseband.

The IF is selected to be at Fs/4 which means the oscillators can be very simple - they’re just the sequence (1, 0, -1, 0, …).

Then you run that through a low-pass filter, drop every second sample, and you’re done. There is a lot of scope for optimization in the implementation of those filters because there are lots of constant zeros in both the input and the filter itself (if you select it carefully - halfband filters again) and so a lot of the FIR work just disappears.

dump1090 doesn’t need IQ [yet!] but if you’re not using the IQ-based envelope detector then you need to do envelope detection some other way. You’ll need to know exactly how the tuner is set up (LO / IF frequencies etc) if you’re working with the real signal.

[gtj0]

@obj

What’s happening here is the host lib is doing the digital equivalent of what you’d do in the analog world to generate an I/Q signal - mixing the real signal with two oscillators with a 90 degree phase difference.

Does that require the sample from the SDR to be complex though?

I’m also a little confused about what @prog said earlier…

Working on native AirSpy support for dump1090-fa

The IQ output works for all the native sample rates in both 16bit and float formats. As a general remark, decimating to 2.4 MSPS IQ will only reduce the probability of good decodes (reduced timing resolution.) Also, a proper decimation that preserves the available dynamic range that comes out of the radio requires good FIR filters. This makes everything harder on the cheap/old generation SBCs. This means you have to demodulate, filter and decode at the input sample rate without losing any data…

If libairspy is requesting complex data to convert to IQ then it can’t possible work at 20 and 24 because of the USB saturation and in fact, libairspy does throw a airspy_set_samplerate() failed: AIRSPY_ERROR_LIBUSB (-1000) error.

[obj]

@gtj0

Does that require the sample from the SDR to be complex though?

No. The (digital) mixing with two oscillators is what does the transformation from a stream of real samples to a stream of complex samples. One mixer produces the in-phase / real part of the complex sample; the other mixer produces the quadrature / imaginary part of the complex sample.

In the airspy hardware there is only a single channel - the ADC measures a single signal, and the R820T2 outputs a single signal. The data received from the airspy is always this ADC measurement of a single channel. When you ask libairspy for complex samples, you are asking it to convert that single channel of data into a complex-sample form; this is a mathematical transform and does not change what the hardware does.

(SDRs that generate the in-phase and quadrature parts of the signal in hardware do exist - these involve a zero-IF tuner stage that outputs two signals, and two ADC channels to digitize those. The airspy does not do this)

@gtj0

If libairspy is requesting complex data to convert to IQ then it can’t possible work at 20 and 24

It can’t operate at 20/24 complex, correct. But I think you misunderstand what’s going on.

If you ask for complex samples, libairspy tells the hardware to provide single-channel ADC data at twice the sampling rate you asked for (see my github link earlier). This is because if you’re asking for, say, 10MSPS complex, you expect that signal to have 10MHz of bandwidth. To capture 10MHz of bandwidth from a real signal, the hardware needs to sample at 20MSPS.

As the airspy hardware is limited to at most 20/24MSPS (single channel ADC) sampling, that means the highest IQ sample rate you can ask for is 10/12MSPS.

[gtj0]

@obj

If you ask for complex samples, libairspy tells the hardware to provide single-channel ADC data at twice the sampling rate you asked for (see my github link earlier). This is because if you’re asking for, say, 10MSPS complex, you expect that signal to have 10MHz of bandwidth. To capture 10MHz of bandwidth from a real signal, the hardware needs to sample at 20MSPS.

AhHa! Yeah I missed that part. It’s still real, just double the rate. I’m still not clear why @prog said IQ was supported at all rates though.

[wiedehopf]

@gtj0

IQ was supported at all rates though.

Have you checked the native rates in the specifications?
Just a hint, 20 and 24 MSPS are not native rates.

It’s obvious, otherwise progs statement wouldn’t make sense