Stream1090

FlightAware ADS-B Flight Tracking
539

I’ve noticed that the compiled filter version seems to be using more CPU than the version reading the custom filter with the -f switch.

Screenshot 2026-02-08 at 17.17.14
Screenshot 2026-02-08 at 17.17.141634×448 211 KB

I was expecting it to be more efficient - have I missed something?

The 30 minute block around 12:00 was an attempt to run the new version on a small file. It seemed to run very slowly - it was a 30 minute sample and had only processed about a third of the messages when I stopped it after it had been running for longer than that - not sure if that is related or if the new version needs a different syntax to read from a sample file. (I didn’t know about the verbose switch at that stage.)

jrg@pi52:~ $ cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -d /home/jrg/projects/stream1090/configs/airspy.ini -q >> /dev/null
[Stream1090] build 260207
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 
^C-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 181972 |   51.5% |   50.4% |   28.7% |   88.48 | 
|   Comm-B |  60736 |   17.2% |   35.1% |         |   29.53 | 
|     ACAS |  64089 |   18.1% |   25.5% |         |   31.16 | 
|     Surv |   3309 |    0.9% |   31.8% |         |   1.609 | 
|    DF-11 |  43152 |   12.2% |   53.5% |   39.9% |   20.98 | 
-------------------------------------------------------------
|  112-bit | 246376 |   69.7% |   47.1% |   22.6% |   119.8 | 
|   56-bit | 106882 |   30.3% |   40.1% |   20.8% |   51.97 | 
-------------------------------------------------------------
|    Total | 353258 |    100% |   45.2% |   22.1% |   171.8 | 
-------------------------------------------------------------
(Max. msgs/s 171.8)
Messages Total 353258
1 Like
540

@mgrone for your resampling, are you using Python scipy.signal ? For example going from 2.4 MHz to 8 MHz, would be a 10/3 factor. That is interpolate up by 10 then filter and then decimate by 3. If so, how many taps are in that filter? Is this being done in Python in real-time? Thanks.

541

Ok, so let’s disect the problem. There are three issues: easy and dificult ones. Let’s start with the easy one:

This is not correct. With

you will just run stream1090 using the native airspy device. It does not care about the stdin at all and will just happily demodulate the live input of your airspy to all eternity.

You need to remove the -d stuff. As in the examples when you use -h, sample rates and optional filter flags are enough.

cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -q -v > /dev/null

First problem solved.

There is something strange with the cpu usage. So i did a test here and for me with 6Msps, -q is faster (raspberry pi 5). What system are you running this on? Is the clock rate of that system fixed?

1 Like
542

The short answer: For sure not.

The long answer: Here i am running at 6 Msps and upsampling is set to 12 Msps. That takes on a raspberry pi 5 running at 1.8 GHz roughly 33% cpu load (does not include airspy handling). Or in other words stream1090 uses 600 Mhz. At an input sampling speed of 6 Mio IQ pairs / second, it needs in average 100 clock cycles for one pair.

This includes:

  • dc removal & running it through the FIR,
  • magnitude computation,
  • upsampling to 12 Mhz,
  • doing CRC magic,
  • message recognition

Don’t try that with python.

Regarding the upsampling, here is the loop that upsamples from 2.4 to 8 MHz.

Depending on the ratio (which is here 3 : 10), you have a basic block. 3 input mags are in a 1-to-1 correspondence with 10 output blocks. In order to interpolate, you consider one more input element. So here 4 and not 3 input magnitudes. An output element is being determined by the weighted sum of consecutive input elements. You can see from the ascii diagram that the output samples are equally shifted in time. These functions are a bit tricky to figure out.

Important to understand here is that in the subsequent step you will compare two upsampled magnitudes that are 4 indices apart. So you compare out[0] with out[4]. Why four? Because 8 MHz/4 = 2Mhz (Manchester encoding).

So you will get 8 bit streams from comparing out[0] .. out[3] with out[4] … out[7] and out[4] … out[7] with out[8] … out[11]. This sounds a bit strange, but keep in mind that you do not know where the correct Manchester encoding starts. Think of it like this: you are hooked up to the plane and get a perfect 2MHz signal. You still have to decide if a message starts at an even or odd index.

543

Ok, I think that’s called polyphase filtering. It’s doing interpolation and decimation in one fell swoop, so to speak. And you’re doing it in compiled C++ not Python.

But another more basic question, why does upsampling help?That is, you’re at 2.4 MHz, you can’t really add anything by going to 8 MHz, can you?

1 Like
544

The core problem is that for Manchester encoding you need something at 2 MHz. In a sample stream of 2.4 you will have a problem.

So if you go down the traditional road, you will look in your samples for something that somehow looks like the preamble. “somehow looks like” because you are most likely not in phase with the planes signal and you are dealing with samples @ 2.4 Mhz and not 2 Mhz anyways.

What you then have to do is to get into phase as much as possible. This will require some interpolation. Readsb does it here for rtl-based dongles if i am not mistaken:

Or in other words: You are not only checking if the message starts at sample i or i+1. You are also checking things inbetween. This requires interpolation.

Stream1090 does this differently. It is not waiting for a preamble, it creates the magnitude samples by “pre-interpolating”. Up to this point you can then do some more basic preamble detection on samples out[i], out[i+4], out[i+8] … (for the 2.4 → 8 MHz). But I am not doing that, because if i find a candidate, i will have to check the resulting message which is what i am doing now anyways. Just for every possible out[i].

545

Thanks for clarifying the usage with sample files.

I’m running stream1090 on a 2GB pi5 at a fixed 2400MHz clock rate.

I will swap back to the non-complied filter to check if it’s repeatable.

546

I think I may have an explanation for what I am seeing.

I confirmed that running latest build with the -f switch was showing a lower processor load. I then decided to compare the results of running the same sample file with -q and -f:

jrg@pi52:~ $ cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -q -v >> st1090_20260209.msg
[Stream1090] build 260207
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 
[DCRemoval] alpha: 0.005
[FlipSigns] enabled
[IQLowPass] tap count: 15 symmetric: 1
[IQLowPass] taps: {0.000566716, 0.020335, 0.00571821, -0.0545961, 0.0605642, 0.167134, 0.162813, 0.27493, 0.162813, 0.167134, 0.0605642, -0.0545961, 0.00571821, 0.020335, 0.000566716}
[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 508229 |   53.6% |   45.2% |   23.1% |   282.3 | 
|   Comm-B | 147329 |   15.5% |   33.5% |         |   81.85 | 
|     ACAS | 120289 |   12.7% |   23.5% |         |   66.83 | 
|     Surv |  40741 |    4.3% |   32.1% |         |   22.63 | 
|    DF-11 | 132469 |     14% |   48.8% |   30.8% |   73.59 | 
-------------------------------------------------------------
|  112-bit | 666334 |   70.2% |   42.8% |   18.4% |   370.2 | 
|   56-bit | 282723 |   29.8% |   38.7% |   17.3% |   157.1 | 
-------------------------------------------------------------
|    Total | 949057 |    100% |   41.6% |   18.1% |   527.3 | 
-------------------------------------------------------------
(Max. msgs/s 527.3)
Messages Total 949057
[Stream1090] Finished.

jrg@pi52:~ $ cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -f /home/jrg/projects/stream1090/optfilter_3 -v >> st1090_20260209_2.msg
[Stream1090] build 260207
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 

[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 466098 |   53.2% |   47.3% |   22.5% |   258.9 | 
|   Comm-B | 138906 |   15.9% |   36.8% |         |   77.17 | 
|     ACAS | 116272 |   13.3% |   25.9% |         |    64.6 | 
|     Surv |  33702 |    3.9% |     36% |         |   18.72 | 
|    DF-11 | 120385 |   13.8% |   50.3% |   29.9% |   66.88 | 
-------------------------------------------------------------
|  112-bit | 615635 |   70.3% |     45% |   17.8% |     342 | 
|   56-bit | 259728 |   29.7% |   40.5% |   16.6% |   144.3 | 
-------------------------------------------------------------
|    Total | 875363 |    100% |   43.8% |   17.4% |   486.3 | 
-------------------------------------------------------------
(Max. msgs/s 486.3)
Messages Total 875363
[Stream1090] Finished.

The -f total messages result is much lower than -q and they should be very close. For comparison here is the result of running the same sample through the old build with the same (or very close) filter. (That run used a script to measure run times).

jrg@MBPro2021 stream1090 % ./run_stream1090.sh
start time 12:05:42 26-01-26
cmd = cat /Users/jrg/Desktop/samples48G_4.raw | ./build/stream1090_10M -f custom_filters/260122_2_OptFilter.txt  -u 24 > samples48G_4raw_2601122opt-1.msg
[Stream1090] build 260123
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 
[Stream1090] Loaded 15 taps from custom_filters/260122_2_OptFilter.txt
-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 508215 |   53.6% |   45.3% |   23.1% |   282.3 | 
|   Comm-B | 147236 |   15.5% |   33.4% |         |    81.8 | 
|     ACAS | 120218 |   12.7% |   23.5% |         |   66.79 | 
|     Surv |  40764 |    4.3% |     32% |         |   22.65 | 
|    DF-11 | 132512 |     14% |   48.8% |   30.7% |   73.62 | 
-------------------------------------------------------------
|  112-bit | 666214 |   70.2% |   42.8% |   18.4% |   370.1 | 
|   56-bit | 282731 |   29.8% |   38.7% |   17.2% |   157.1 | 
-------------------------------------------------------------
|    Total | 948945 |    100% |   41.6% |   18.1% |   527.2 | 
-------------------------------------------------------------
(Max. msgs/s 527.2)
Messages Total 948945
finish time 12:11:57 26-01-26

Looking at that it actually confirms what I was thinking - the -f switch isn’t picking up the custom filter on the new build, as the line equivalent to the one below is missing.

[Stream1090] Loaded 15 taps from custom_filters/260122_2_OptFilter.txt

I think I have the path right in the new build?

jrg@pi52:~/projects/stream1090 $ cat optfilter_3
# ================================================================
# Instance: /Users/jrg/Desktop/samples48G.raw
# Time: 2026-01-22 18:35:17
# FS: 10.0 MHz → FS_UP: 24.0 MHz
# Number of taps: 15
# Best score: 743532
# Best total: 472628
# Best params: [0.317668312687987, 0.42272982005696474, -0.9059743282476853, 0.44462302527647163, 0.2011254668190785]
# Current bounds:
# [-0.360464, 0.639536]
# [0.217618, 1.217618]
# [-1.841086, -0.841086]
# [-0.017670, 0.982330]
# [-0.733769, 0.266231]
# Best taps:
0.0005667160730808973
0.020335007458925247
0.005718213971704245
-0.05459609255194664
0.06056421995162964
0.1671341508626938
0.1628127247095108
0.2749301493167877
0.1628127247095108
0.1671341508626938
0.06056421995162964
-0.05459609255194664
0.005718213971704245
0.020335007458925247

[Edit]

So the “better” performance with the -f switch in the new build isn’t using the custom filter and is using less CPU but getting fewer messages. The true comparison is with the old build with the custom filter. (The gap from midnight was a power outage.)

Screenshot 2026-02-09 at 09.10.48
Screenshot 2026-02-09 at 09.10.481644×456 161 KB

1 Like
547

Can you pull and rebuild?

Edit: Thank you for the feedback. I messed up loading the taps from file. I had to move the implementation, rewrote it and forgot to add the comment part.
So when reading

The function fails and the IQ filter pipeline is disabled. The result: Way less messages and way less CPU Time.

Edit2: Latest version will measure the time when using -v

1 Like
548

@JRG1956 regarding slight deviations (a few messages less in total) when you run prerecorded samples through the new version of stream1090. This is a bug in the old version that i fixed. Stream1090 does not really stream. It processes batches of data. The size of the batch is determined by input sampling rate and upsampling rate. Let’s say this batch size is B. The value of B is by the way the InputBufferSize you get when starting up.

When you cat a file into stdin of stream1090, it will consume the input at multiples of B. That works well until you hit the end of the file. Most likely the sample count of the file is not a multiple of B.

So what may happen that there is a tiny bit of file left at the end while we need B many samples for a round. It will try to read B samples, but just gets a few. This reading basically transfers data from stdin to a buffer of size B which is reused in every round. So in the very last read this buffer will contain only very few new samples, but plenty of the previous read. These may contain complete messages which have been found and will be found again.

It does not matter for your big samples. Just in case you are wondering why there is slight deviation compared to the previous version. Now, i am checking for this case and overwrite the old values with 0.

1 Like
549

Thanks for the fix, and adding the run time to the output.

For my sample, the results show that the compiled filter uses less processor load to obtain perhaps a few more messages.

Reading filter from file (not sure about all the trailing zeroes on the last tap):

jrg@pi52:~ $ cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -f /home/jrg/projects/stream1090/optfilter_3 -v >> st1090_20260209_3.msg
[Stream1090] build 260208
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 
[DCRemoval] alpha: 0.005
[FlipSigns] enabled
[IQLowPassDynamic] tap count: 15 symmetric: 1
[IQLowPassDynamic] taps: {0.000566716, 0.020335, 0.00571821, -0.0545961, 0.0605642, 0.167134, 0.162813, 0.27493, 0.162813, 0.167134, 0.0605642, -0.0545961, 0.00571821, 0.020335, 0.0005667160000000000000000000000000000000000000000000000000}
[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 508214 |   53.6% |   45.3% |   23.1% |   282.3 | 
|   Comm-B | 147236 |   15.5% |   33.4% |         |    81.8 | 
|     ACAS | 120218 |   12.7% |   23.5% |         |   66.79 | 
|     Surv |  40764 |    4.3% |     32% |         |   22.65 | 
|    DF-11 | 132512 |     14% |   48.8% |   30.7% |   73.62 | 
-------------------------------------------------------------
|  112-bit | 666213 |   70.2% |   42.8% |   18.4% |   370.1 | 
|   56-bit | 282731 |   29.8% |   38.7% |   17.2% |   157.1 | 
-------------------------------------------------------------
|    Total | 948944 |    100% |   41.6% |   18.1% |   527.2 | 
-------------------------------------------------------------
(Max. msgs/s 527.2)
Messages Total 948944
[Stream1090] Finished. (849.07s)

With the compiled filter (re-run with the latest build -same total messages):

jrg@pi52:~ $ cat samples48G_4.raw | /usr/bin/stream1090 -s 10 -u 24 -q -v >> st1090_20260209_4.msg
[Stream1090] build 260208
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples 
[Stream1090] Size of sample buffer: 36864 samples 
[DCRemoval] alpha: 0.005
[FlipSigns] enabled
[IQLowPass] tap count: 15 symmetric: 1
[IQLowPass] taps: {0.000566716, 0.020335, 0.00571821, -0.0545961, 0.0605642, 0.167134, 0.162813, 0.27493, 0.162813, 0.167134, 0.0605642, -0.0545961, 0.00571821, 0.020335, 0.000566716}
[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 508229 |   53.6% |   45.2% |   23.1% |   282.3 | 
|   Comm-B | 147329 |   15.5% |   33.5% |         |   81.85 | 
|     ACAS | 120289 |   12.7% |   23.5% |         |   66.83 | 
|     Surv |  40741 |    4.3% |   32.1% |         |   22.63 | 
|    DF-11 | 132469 |     14% |   48.8% |   30.8% |   73.59 | 
-------------------------------------------------------------
|  112-bit | 666334 |   70.2% |   42.8% |   18.4% |   370.2 | 
|   56-bit | 282723 |   29.8% |   38.7% |   17.3% |   157.1 | 
-------------------------------------------------------------
|    Total | 949057 |    100% |   41.6% |   18.1% |   527.3 | 
-------------------------------------------------------------
(Max. msgs/s 527.3)
Messages Total 949057
[Stream1090] Finished. (845.744s)

Is the difference in total messages real or related to the buffer flushing you mentioned?

The CPU difference is apparent in the two sample runs:

Screenshot 2026-02-09 at 12.24.42
Screenshot 2026-02-09 at 12.24.421638×450 143 KB

550

No, this took some time to debug. But the two low pass filters were not behaving the same way. I adjusted the implementation of the LPF-from-file to match exactly the same. This was not the case before. It is fixed now and the update is in the repo. Thank you for pointing this out.

Edit: When you run compile time taps and the same in taps from file and save the messages.

diff builtin_msgs.txt file_msgs.txt

should return nothing, that is, the two message streams should be exactly the same.

1 Like
551

While doing some Readme writing, i am also paying some attention to the long forgotten child: RTL-SDR. Airspy has fancy IQ filtering and things, while RTL-SDR has, yea, upsampling to 8 Msps only. In the repo you can now find:

  • RTL-SDR based devices have now their own IQ pipeline
  • custom_filters contains temporarily a file 2_4_TO_8_0_experimental.txt with a low-pass filter for 2.4 → 8 MHz

The taps of the filter are not final. I will wait a few more hours to see if the optimizer comes up with something better.

If you are eager to try, load the taps into stream1090 using

./build/stream1090 -s 2.4 -d ./configs/rtlsdr.ini -f custom_filters/2_4_TO_8_0_experimental.txt

if you are using native device support. Otherwise if you are doing it the rtl_sdr way. Just add the above -f option to your existing command.

So i am getting very good results with the Low-pass and the additional cpu load is worth it. I am talking stuff like this:

-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 130292 |   16.5% |   21.3% |   35.9% |   181.2 | 
|   Comm-B | 179076 |   22.6% |      0% |         |     249 | 
|     ACAS |  46193 |    5.8% |      0% |         |   64.23 | 
|     Surv |  90988 |   11.5% |      0% |         |   126.5 | 
|    DF-11 | 345441 |   43.6% |   24.3% |     15% |   480.3 | 
-------------------------------------------------------------
|  112-bit | 313794 |   39.6% |   10.1% |     17% |   436.3 | 
|   56-bit | 478196 |   60.4% |   18.8% |   11.7% |   664.9 | 
-------------------------------------------------------------
|    Total | 791990 |    100% |   15.6% |   13.7% |    1101 | 
-------------------------------------------------------------
(Max. msgs/s 1101)
Messages Total 791990

With Low-pass:

-------------------------------------------------------------
|     Type |  #Msgs |  %Total |    Dups |   Fixed |   Msg/s | 
-------------------------------------------------------------
|    ADS-B | 147351 |   16.4% |     22% |   34.5% |   204.9 | 
|   Comm-B | 208963 |   23.3% |      0% |         |   290.5 | 
|     ACAS |  53844 |      6% |      0% |         |   74.87 | 
|     Surv | 103373 |   11.5% |      0% |         |   143.7 | 
|    DF-11 | 382850 |   42.7% |   25.7% |   14.9% |   532.3 | 
-------------------------------------------------------------
|  112-bit | 361586 |   40.3% |   10.3% |   16.1% |   502.8 | 
|   56-bit | 534795 |   59.7% |   19.9% |   11.5% |   743.6 | 
-------------------------------------------------------------
|    Total | 896381 |    100% |   16.3% |   13.3% |    1246 | 
-------------------------------------------------------------
(Max. msgs/s 1246)
Messages Total 896381

These are not “a few messages more”. Let me know how it goes, if you try it.

552

There is now a good default IQ filter (-q option) for RTL-SDR dongles.

553

@JRG1956 When you are running the optimizer, what is the bottleneck? Harddrive or computation?

554

I assumed it was computation, but didn’t actually check.

I ran the optimisation on my 2021 vintage MacBook which has the M1 Pro chip. I think it was taking about six minutes per iteration.

(Edit)

Probably not relevant, but I ended up reverting to using the Homebrew GCC compiler, the work you did to get the code to compile using the Xcode compiler did compile, but the resulting app ran much more slowly

1 Like
555

@mgrone Regarding Stream1090 Readme instructions, in the “Integrating stream1090 into the stack” section, your stats output continues to run. You used to have a compile option to turn off the stats output. Can that still be used?

Also, you show how to setup readsb in that section but not dump1090-fa. The easiest way to setup dump1090-fa is to edit the file /etc/default/dump1090-fa. Change:
RECEIVER=rtlsdr to RECEIVER=none
and NET_RAW_INPUT_PORTS= to
NET_RAW_INPUT_PORTS=30001

Need to stop and restart dump1090-fa for change to take effect of course if it’s already running.

556

Yes that is still there but not documented yet. Nevermind documented now.
it is

cmake ../ --fresh -DENABLE_STATS=OFF && make

Thanks for the short dump1090-fa info

557

Thanks. It would be nice if turning on or off stats would be a run time option, rather than a compile time option. Also, are you going to add RSSI computation? – It’s easy to ask for things!

558

Yes that is why i removed the compile time option from the documentation.
To have a proper runtime variant i will first need a proper Logger. And i have to come up with something that is actually fast.

Regarding RSSI:
I may add that in the far future. Why?

  • The hot loop does not care nor is clear how RSSI affects stream1090. magnitude[t] > magnitude[t + 1/2MHz] is the only thing that matters.
  • RSSI will cost CPU time. I have to keep the last 224 magnitudes for each of the up to 24 bitstreams. There is no way around this.

I had a version with RSSI before and decided to not add that feature for now, because the purpose of it was unclear to me.

What you will get: Live gain/other options control.