More Messages with Beamforming

FlightAware ADS-B Flight Tracking
1

km4kfl/rsadsbma: A proof of concept for ADS-B decoding using two antennas and beamforming.

I noticed you have support for the BladeSDR. So, I thought, why not utilize the card to its full potential. The idea is to get more messages per second by finding messages that normally would have been missed by a single antenna. The cost is more CPU and core usage.

The idea is very simple. You take two coherent antenna sample streams, such as from the BladeSDR, and then you rotate one of the streams in phase, scale its amplitude or not, and then combine it with the first stream. This creates different antenna patterns and in general can sweep the beam created across space. This means some packets/messages/frame end up with an SNR that is high enough to be decoded.

The problem is it consumes much more CPU and it pretty much requires threads which means it’s going to work best on a system with multiple cores. I have a beefy computer. You’re milage may vary which is part of the reason I wanted to share it.

The program is very minimal. It is only meant to give you guys something to test and see if there is any who think it is worth the extra CPU.

First, you clone the repository. Then make sure you have Rustup installed which you can find at rustup.rs - The Rust toolchain installer. Also make sure Python 3.x is installed. Next, you go into the cloned directory and type python bladesdr.py --serial 9da --freq-offset 0 but use the serial for your BladeRF card and if you know it’s frequency inaccuracy you can type it in but zero should work fine. Now, if that succeeds the small Python frontend is running that will shovel raw samples out over TCP.

Next, you type something like cargo run --release – --thread-count 16 --cycle-count 20 and this runs the programs with 16 threads and 20 cycles per thread. The number of cycles is per thread, and it is the number of times the program randomly guesses a phase adjustment and amplitude. You need the number of threads to be no more than the number of cores in the machine. You can also try doubling the number if you have hyper-threading but I have not tested that. You may have to increase or decrease the cycles if you see a message that says TOO SLOW because what is happening is your losing samples because you’re not processing the buffers in time.

The idea is to find not the absolute maximum of cycles but a comfortable number that still gives some headroom without triggering the TOO SLOW message. That means your maximizing CPU usage to a comfortable level. This means more power consumption, heat, and all the other bad things about running your CPU hard. I’m not sure but you might get good results with a lower amount of CPU usage and the returns on using more CPU are diminishing. This I can not say which is why I only wrote a minimal program for testing.

As the program runs it prints elapsed and buffer time. You want elapsed to be lower than buffer time. It also prints statistics that look like this:

=========   ANTENNA A   =============
 DF11: 1.3 24
 DF17: 0.1 2
 DF18: 0.0 0
OTHER: 1049.0 18997
ALL  : 1050.5 19023
=========   ANTENNA B   =============
 DF11: 0.7 13
 DF17: 0.1 1
 DF18: 0.0 0
OTHER: 1049.8 19010
ALL  : 1050.5 19024
========= BOTH ANTENNAS =============
 DF11: 2.9 53
 DF17: 1.0 19
 DF18: 0.2 3
OTHER: 21244.6 384718
ALL  : 21248.8 384793

What this says is the rate per second and total count of each message type. It does that for DF11, DF17, and DF18. Anything else is other. The ALL column is for all messages even ones that could not decode - I might have to check that.

The interesting part is that it does the statistics for each of the antennas individually and then it does it for using both antennas. You should capture more messages using both antennas and it should be worth the extra CPU and power you are now consuming.

The antennas that you use should be the same, but you could use different ones. If you use directional antennas, you can try pointing them in the same direction or different. It shouldn’t hurt the program. You also want the same gear on each antenna and for it to be the same as possible. For example, if you use an LNA on one antenna then you want it on the other. If you use a filter on one antenna then you want to use the same brand and type of filter on the other. You can try using different brands of filters but your milage may vary and that goes for all gear. You essentially want the two antennas to be as identical as possible.

The idea here being used is the same as with beamforming. We are just simply searching around for packets in both time and space.

Feel free to fork the project, copy it, and do whatever you like. You can call it your own or send changes in the form of pull requests. I am not sure if I will work on it anymore. It might be that it isn’t worth it. This point here was to get the idea out and see if it’s worth it.

2

Interesting! Some random comments:

  • I think you probably want to dedupe more aggressively than the current “index of starting sample”. Probably you can decode the same message at various slightly different offsets.
  • what sort of antenna spacing do you use?
  • have you tried more evenly spaced beams, rather than randomly picking theta?
  • I wonder if taking receiver location + decoded ADS-B aircraft position + strongest phase / beam would let you map out the effective real-world directions of each chosen phase offset
1 Like
3

Can you be a bit more specific how the stuff works?
But very interesting. Let me add some rules that i believe in:

  • Multiple SDRs with antennas having their gain mostly disjoint and demodulated separately will outperform any multiple SDR setup with → mixed up → 1 demodulator (for ADS-B)
  • Getting beyond the bottleneck of the usual unidirectional aerial antenna solution will require to demodulate several antenna signals in parallel, and these antennas should be directional and cover not too much overlap sectors.
4

AIUI the short version is:

You have antennas that are physically separated. They’ll receive the same signal with a phase difference due to the separation. The phase difference varies depending on the direction of the transmitter (and the exact frequency, antenna geometry, etc, but let’s assume those are fixed)

For a particular direction, if you back out the phase difference and add the two signals, you get coherent reinforcement for signals from that direction, but incoherent averaging as you move away from the specific direction. So the SNR in that direction should improve.

And since you’re doing this in software, you can just try lots of different directions <=> lots of phase differences, on the same received data, and see what sticks.

5

Yep, i am totally with you. This is what stream1090 will do in the very far future. Take some Df-17 message with position (Not because of the position, just because that is a volatile payload) from two or more SDRs with antenas that share certain area. Sync it up. And you can then do fancy things in software. Important is that you get two SDRs lined up. ADS-B will be good for that.

6

I think you probably want to dedupe more aggressively than the current “index of starting sample”. Probably you can decode the same message at various slightly different offsets.

It should work fine since I’m sampling at 2mhz which puts each bit on exactly two samples. But it is possible the actual bit is sampled too early or too late. I’ve seen the phase correction code but I don’t think it applies to this issue since I’m not doing it. Meaning it’s either there or it isn’t.

what sort of antenna spacing do you use?

I just throw the antennas up and place them where they look good. They are spaced about one inch apart and it’s two directional/sector antennas. I checked how big a quarter wavelength is around that frequency, and I made sure they within that distance or closer.

However, if you really wanted to maximize and IF it would is a question then you might be more careful. I have not got around to testing that yet and seeing if it makes much of a difference. I would suspect that moving them too far apart would be bad. Say near or over half a wavelength because any more and the phase would wrap around.

have you tried more evenly spaced beams, rather than randomly picking theta?

Yes, I have tried spacing the beams out evenly. I didn’t prove it by doing hard tests but the random approach just felt like a good way to do it. As to which method is better, I do not know.

I wonder if taking receiver location + decoded ADS-B aircraft position + strongest phase / beam would let you map out the effective real-world directions of each chosen phase offset

You know I want to do this. But when I took transmissions from the same `address` or from what should be the same transmitter the `theta` jumped around and did not have a definable pattern to it. I was basing it on the highest SNR.

You also have a mirror image problem when using a uniform linear array, so you have 180 degrees of ambiguity if I remember right.

I’m just guessing the theta jumps, in terms of picking the one with the highest SNR, due to the fact that sometimes the atmosphere or things in the environment can create echoes and the various constructive and deconstructive effects of that happening at each antenna.

If you used other methods of tracking the theta the SNR might jump around but it should in theory track it. But then you also have issues of how fast the plane is flying and how close which changes its tangential velocity.

I’m not enough of an expert on beamforming to really say with any definity.

A good example of the theta and how it jumps around and changes based on highest SNR is this track of a transmitter.

[-1.45, 0.14, 2.08, 1.6, -0.58, 2.64, -0.41, 1.51, 1.34, 3.06, -2.99, 0.84, 1.07, 0.28, -0.05, -2.59, 0.46, 2.46, 2.81, 1.05, -2.01, 0.78, 2.7, 0.99, 1.07, -0.74, -3.07, 0.07, 0.29, -1.78, -1.06, -1.09, -0.51, -0.22, 1.36, 1.52, -1.21, -2.24, -1.33, -1.63, -0.74, 1.16, 0.99, 0.88, 1.67, 1.51, -0.93, -2.93, -1.66, 0.16, -1.04, -1.64, 2.0, 0.32, 2.89, 2.05, -1.81, 1.91, 2.66, 3.08, -2.87, 0.31, -1.68, -1.23, -2.63, -0.77, 0.63, -0.1, 1.63, -3.04, -1.67, 3.08, 0.16, -2.34, -3.04, 0.43, 2.71, 0.04, -2.45, -0.49, 3.1, 0.31, -1.96, -1.23, 0.51, -1.31, -2.49, 0.29, -2.28, 1.02, 1.91, -3.06, 1.78, -2.03, 1.2, 2.77, -1.25, 2.58, 0.09, -0.22, -0.87, -1.46, -1.49, -0.29, -2.61, 2.82, -1.5, -0.04, 1.16, -0.05, 2.99, -2.52, 1.11, 2.25, 2.86, 2.99, -2.63, 0.77, -2.59]

There are some really good resources out there on beamformers and I can’t remember if some of them were or were not applicable to tracking a signal. I feel like I remember something about nulling out interference, but I don’t think it was moving but you know for a fact they do this. I just don’t know how. I wish someone would explain it to me.

Multiple SDRs with antennas having their gain mostly disjoint and demodulated separately will outperform any multiple SDR setup with → mixed up → 1 demodulator (for ADS-B)

I need some people who see more traffic to test this out and see.

Getting beyond the bottleneck of the usual unidirectional aerial antenna solution will require to demodulate several antenna signals in parallel, and these antennas should be directional and cover not too much overlap sectors

Have you considered each sector antenna being beamformed? You can divide the area into sectors with directional antennas, but each directional antenna could be a composite of many antennas. Those composites could still work as a beamformer. You would get higher gain doing it.

Just look at a Starlink user terminal dish. It is composed of I can’t even remember how many antennas. So, it can still make an impact. The problem is, I would think, that processing it all in software without the help of hardware would be expensive because it would require a lot of general-purpose CPU instead of something more specific and integrated.

7

Intuitively I’d think you want them at about lambda/2 (~14cm) spacing, as with closer spacings, some phase differences are never possible (and so you have a smaller range of useful theta values) – ignoring multipath at least.

I wonder if the randomization of the magnitude correction isn’t helping here. Omni antennas and no magnitude change might be something to try?

8

Yeah, about 95% of half lambda that way it doesn’t get too close to the edge of the range.

Yes, no magnitude change is something to try.

The theta does appear to be much more stable now. It looks like something that could be tracked fairly easily. Unfortunately, though, it doesn’t appear to be much use in actually tracking the aircraft’s direction because it’s got too much noise and it sways too much over the range. But it does have a pattern now that the magnitudes are uniform in the array. It’s still affected too much by the environment or the antenna pattern, I think.

I will have to get an omni antenna for the frequency and try it out. I also will have to spread my antennas out closer to half lambda. I’ll have to make a new mount for them later.

image
image573Ă—445 21.4 KB

9

I varied the magnitude because the literature talks a lot about varying the magnitudes. Yet my research into it today brought up the topic of sidelobe levels. Those little lobes that form adjacent to the main beam. However, I found a binomial two element array is a special case and has uniform amplitude. I will have to research it more because I seem to dig more packets up out of the noise allowing the amplitude to vary.

Perhaps, you can shed some light on this or point me in the right direction. I may already be going there. So, I am not sure if it is best to hold the amplitudes.

10

I’m rapidly getting out of my depth here, but playing around with some simulations, scaling the amplitude doesn’t seem to be hugely useful, it just kinda smears out the pattern. compare:

image
image728Ă—812 69.3 KB

image
image728Ă—812 66.6 KB

(antennas spaced along the -90 .. +90 axis, 0 degrees is “broadside”)

but take with large grain of salt, I hacked that together in 30 mins and I don’t entirely trust it.
source here if you want to play with it: beamforming graph stuff · GitHub

11

Thank for this gift. I will have to play with it to understand it better. It’s been a while since I did this kind of math so will take me a while to get back into it.

Btw, I have a cable for the BladeSDR that lets me connect two of them with the same clock. I just need to wire them up on the exposed headers and I can trigger the streams at the same instant. I say that in case you have one and you wanted to experiment with four antennas.

12

I looked at your gist and it was very helpful. However, I struggled when I reached:

mag = np.sqrt( (1 + amplitude * np.cos(phase_difference))**2 + (amplitude*np.sin(phase_difference))**2 )

Not because you are wrong but rather, I couldn’t get my head to wrap around the math right there. I’m going to give it more time and really try to understand it, but you might be able to help me.

However, you did spur me into saying okay I can’t understand it, but can I write my own version using your version as insight? So, I came up with a simpler version that I think is correct. I used complex numbers.

The np.exp(1j * theta) simply creates a complex number at that specific theta and you can feed it a list of angles also just like you expect from Numpy.

The interesting part is that the pattern shape did not change at all when I modified the amplitude of the second element or the first. It really seemed to serve to simply switch one of the antennas off. That might explain why changing the amplitude works because sometimes an omni-directional pattern picks up what the beam pattern missed. I also think that when the amplitude goes toward zero the theta matters less so it makes it jump around since I’m generating it randomly.

I’m so glad you got me to review the actual composite pattern! I thought it was doing more and it wasn’t.

13

Yes, and things will move into that direction. So we should keep some contact. I wrote stream1090 exactly for your beam approach. I have code here on my computer that does the beam.

14

Yeah that’s not the clearest, it was very much hacked together! I was mostly thinking about it geometrically, not as complex numbers, at the time..

It’s equivalent to something like this:

a = 1 + 0j
b = a * amplitude * np.exp(1j * phase_difference)
mag = np.abs(a + b)

(because rotating 1+0j by an angle d gives you cos(d)+jsin(d) - Euler’s formula etc - and the squares/sqrt are just the abs operation)

which is basically identical to what you came up with.

(note that my phase_difference is a bit different to your phase_difference)

16

I also finally realized today that it would be possible to have two different messages at the same sample index because one of the two could be decoded using a different pattern / beam. I will have to contemplate that. As of now, one of them with the lowest SNR would be thrown away.

I’m glad you mentioned that. I wasn’t thinking about it right.

I hate calling it a beam because it isn’t a beam at all with two antennas. It is actually a cone shape. I think with all uniform linear arrays it is cone shaped. It’s a cone that can swing from side to side. That’s what gives it the ambiguity of 180-degrees. The cone extends around the line between the two antennas.

17

I’ve currently got it tracking aircraft with a beamformed antenna pattern. It’s simple right now. It just keeps a running average of the `theta` and allocates one of the cycles/pipes to perform that beamforming operation while any free ones run as random. If it doesn’t get any messages from the aircraft after 10 seconds, it unassigns the cycle/pipe and it goes back to searching randomly.

I put all this work on a new branch called extended.

https://github.com/km4kfl/rsadsbma/tree/extended

I will have to do some testing to see how well it works. But I wanted to post it in case anyone had a BladeSDR and wanted to follow along or do their own tests. The framework is in a good state for anyone who wanted to do some advanced operations.

18

I built a new mount and got my antennas at about half lambda but I was thinking you know I’m going to have to upgrade my whole setup to really get some good results. For example, beamforming with two antennas is only going to net a 3dB which is a nice jump. But I can likely get a lot more out of my antenna setup. So, I’ve got to get an outside antenna setup above the roof a bit, and I’m also using some wideband antennas, so I’ve likely got interference hitting my LNAs really hard.

The house itself might be messing with the theta since my antennas are indoors. I’ve also noticed I have spotty coverage. I’m guessing that’s got to do with being indoors too. I might see 4 to 10 planes but I’m not getting a steady stream from any but one at a time and that’s with beamforming trying to lock onto the planes.

So far, the results are you end up paying $600 for at least 3dB more. I’ve got four of these cards so no big deal for me they are already bought. I’m just going to have to upgrade the rest of my frontend before I can really do any tests.

19

So, for the $600 are you referring to bladeRF 2.0 micro xA4 ($540) and another $60 for a power supply and second antenna?

20

Yep, and that’s only for 3dB more.

21

How do you mean that 3 dB – i.e. a 2 to 1 increse in number of planes tracked?