Here is more for you. Python script that crawls through the message stream.
import sys
import rs1090
planes = {}
msg_per_plane = {}
numMsgs = {}
for line in sys.stdin:
try:
if (not line.startswith('@')):
continue
curr_time = int(line[1:13], 16)
line = line[13:].strip(';\n')
res = rs1090.decode(line)
if (res == None):
binary_message = format(int(line, 16), f'0{len(line)*4}b')
print("ERROR " + binary_message[:5] + " " + line[2:8] + " " + binary_message[33:38])
continue
# remove this for stats only
print(res)
df = res['df']
if ((df == '17')):
p = res['icao24']
if not (p in msg_per_plane):
msg_per_plane[p] = 0
msg_per_plane[p] += 1
if not (df in numMsgs):
numMsgs[df] = 0
numMsgs[df] += 1
except:
print("error: " + line)
totalCount = 0
for df, cnt in numMsgs.items():
totalCount += cnt
print("DF " + df + " " + str(cnt))
for icao, cnt in msg_per_plane.items():
print(icao + "\t" + str(cnt))
It uses rs1090 which you can install via pip. It simply counts the DF-17 per plane and makes stats about the other messages. rs1090 offers way more. You can adjust as you like. It also shows you how to convert a message into a binary string so you can have a closer look. Although rs1090 can do most of these things.
You can record sample datasets by piping them into a file. For testing i am using the raw output of rtl_sdr and not the messages. I can then say:
Thereby dumping the raw iq samples into stream1090 and its output is then run through the python script. Should work with airspy_rx and corresponding stream1090 settings the same way.
So here is a longer reply regarding these bogus messages and planes. Stream1090 is a demodulator which i personally use to feed readsb which then serves as the decoder. I have to draw at some point a line in the sand and say: This is something that has to be taken care of by the decoder. I think that is understandable. I have implemented some logic that prevents forwarding crappy messages. However, this already requires me to implement some decoder logic.
Here is now the catch. Stream1090 is very aggressive in how it is demodulating things aka framing messages. For airspy hardware it could in theory with a bit of luck frame two messages from different planes at nearly the same time. And yes, i have such an example. The difference here is that messages are being framed in parallel whith a slight phase shift. Running an internal speed of 24 MHz means 24 shift registers in parallel are trying to make sense of 0s and 1s at a speed of 1MHz each. At this speed, the chance that a combination of 0s and 1s makes sense and the CRC sum is also correct gets quite high. Keep in mind that if that code would be only doing 99.99% correct, it would be an absolute diseaster. 0.01% failure is equal to 2400 msg/s of crap.
On top of this, there is CRC error correction. Compared to other software, the current state seems aggressive, but i have used versions in the past that were not public that applied a way more aggressive strategy. Will this create bogus messages? Yes. It will however also create a huge amount of usable messages.
Edit: my numbers were wrong. I was thinking about 2.4Mhz not 24MHz. So 0.01% = 2.4k crap/s
Thanks for the python script - I will have a play with it when I have time.
Re the bogus planes/messages - I think your current settings are fine.
If I’ve understood correctly, the bogus message rate is independent of the actual number of real messages being received because they are being generated out the background noise. As I have low real message rates, the proportion of bogus messages is higher, but not the absolute number being generated.
A very easy way to check if an icao address is valid and in range is going for some site that uses tar1090. Something like this: https://globe.adsbexchange.com/?icao=40749b just replace the icao address.
just a quick update what is going on currently. After a short break, I got my hands on an airspy which makes it a lot easier for me to implement things for the airspy side of things. Currently i am testing some new code that takes the raw output of airspy_rx as input instead of the default INT16_IQ.
the raw mode (-r flag) update is in the repo together with some other changes. Let me now if you experience some zombie stuff, because i made some changes to the cache timeouts.
The raw mode should greatly reduce airspy_rx cpu usage. There are two shell scripts that outline the differences.
It is me again. Regarding the raw input. So i talked a bit with @wiedehopf who provided me with some extra tool for some benchmarking. Thank you for that. I came to a preliminary conclusion after some thinking without doing a week long of testing. For this it is important to know some technical details how things work.
airspy_rx can provide different output formats. Float, uint16, int16, raw etc at different speeds. In the end everything is an IQ-pair. That is two values always form a pair. The default is int16 IQ. So if you say: Sample at 6Msps, then you will get 6 Mio pairs per second. If you say: 12Msps raw pls, then airspy_rx will give you first an I, the next value will be the Q. So basically 6Msps IQ-pairs. Raw mode does not magically increase the sampling speed.
What is not directly obvious to a user of airspy_rx is that for some formats (including the default int16 IQ) some magic is applied. Not really magic, but a band pass filter is applied and some DC removal takes place.
However, raw is really a passthrough and therefore you will get some good reduction in CPU time. No magic is taking place and therefore no band-pass filter.
So what? The thing is the following: I have a setup with very little interference. The LNA is directly hooked up to the antenna which is a high gain narrow bandwidth directional. From there on i have 30cm (1ft) of nearly osciloscope grade coax to the airspy. It could be that for me this band-pass filter does not matter. It may however matter for you. So you have to do some testing, i will later push a tiny update for the non-raw version. You have to figure out:
Plenty of interference and you really need this band-pass filter. Then use for now the non-raw version.
Few interference and you are too lazy as me to install a cooler on your RaspberryPi 5 and want it sitting there at 1.7GHz, then use the raw mode.
@JRG1956@mgrone
To continue the discussion on airspy from the other thread, take a look at the following paper: How RTL-SDR dongles work
I think the Airspy approach is closest to what is in Figure (b) shown below:
Notice that there is one A/D and that the quadrature mix is performed digitally (where the mixes become multiplies and the lowpass filters are FIR filters). This is a much better than an analog approach shown in Figure (a). The analog implementation has inherent imbalances in the IQ arms. This produces images at the negative frequency of the desired signal. Also the digital implementation has simplifications when the if freq = fs/4 where fs is the sample rate as I mentioned in an earlier post. Definitely want to go digital (DSP).
