That latest release compiled successfully - thanks!
1 Like
Tested it out and there is a difference to contend with. I’m going to toy with it a bit and see what I can do with the script, but I definately don’t have the expertise to tease out the finer nuances later. Captured a new 15 minute sample (and a quick run against the python script) and seeing a small difference with the custom filter so far. More CPU cycles with the script are in order!
root@filter:~/stream1090/filter_utils# cat /root/new15MinSamples.raw | /root/stream1090/build/stream1090_10M -f /root/stream1090/custom_filters/cnuwer001.txt -u 24 > /dev/null
[Stream1090] build 260123
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples
[Stream1090] Size of sample buffer: 36864 samples
[Stream1090] Loaded 15 taps from /root/stream1090/custom_filters/cnuwer001.txt
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 505364 | 26.9% | 57.5% | 43.4% | 561.5 |
| Comm-B | 1803 | 0.1% | 20.8% | | 2.003 |
| ACAS | 537687 | 28.6% | 28.5% | | 597.4 |
| Surv | 97306 | 5.2% | 35.6% | | 108.1 |
| DF-11 | 736900 | 39.2% | 62% | 34.6% | 818.8 |
-------------------------------------------------------------
| 112-bit | 517439 | 27.5% | 57.2% | 42.8% | 574.9 |
| 56-bit | 1361621 | 72.5% | 51.8% | 23.8% | 1513 |
-------------------------------------------------------------
| Total | 1879060 | 100% | 53.4% | 29.5% | 2088 |
-------------------------------------------------------------
(Max. msgs/s 2088)
Messages Total 1879060
root@filter:~/stream1090/filter_utils# cat /root/stream1090/custom_filters/cnuwer001.txt
# Time: 2026-01-23 09:27:25
# Instance: /root/new15MinSamples.raw
# FS: 10.0 MHz → FS_UP: 24.0 MHz
# Number of taps: 15
# Global best score: 2394598.0
# Global best total: 1879060
# Global best params: [-0.3851539186431228, 1.9235891937470784, -1.3781896832776364, 0.213815603809472, -0.1165845669454588]
# Current bounds:
# [-0.456251, -0.081929]
# [1.380962, 1.977075]
# [-1.408204, -1.284651]
# [0.188905, 0.504430]
# [-0.243507, -0.091657]
# Global best taps:
-0.007245163898915052
0.06197800859808922
0.09318719059228897
-0.031246760860085487
-0.09799887239933014
0.0959581509232521
0.2735322117805481
0.2236703485250473
0.2735322117805481
0.0959581509232521
-0.09799887239933014
-0.031246760860085487
0.09318719059228897
0.06197800859808922
-0.007245163898915052
1 Like
Begin long post:
So here is a very small update. I have been busy with non-stream1090 things. However, i am exploring native support for airspy and librtlsdr. Yes, sounds awesome, but it is not. Let me explain the problem.
Currently, stream1090 uses stdin via pipe as input. For basically everything: files via cat, rtl_sdr, airspy_rx. Something like this:
<source that writes to stdout> | <stream1090 that reads from stdin>
At first glance this looks like a bit of a silly solution. However, this pipe “|” does also deal with some things under the hood. Everything here is batched based, which means that stream1090 reads a certain amount of samples at once, processes them and then reads the next batch. Similarly, rtl_sdr, airspy_rx (and also probably cat do not write single samples). They write them in batches.
So the first problem here is that the batch size is not the same. You may have seen something like this:
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples
[Stream1090] Size of sample buffer: 36864 samples
In this particular case, the input buffer size of 15360 describes the batch size stream1090 will read in one go. This 15360 is derived from multiple things. Like the input output ratio of 5:12. If you read at a batch size that is not a multiple of 5 like 15358, the world gets very difficult when you have to interpolate 12 new samples out of 5 existing samples.
airspy_rx and rtl_sdr do not care about such things. USB will say: Here, data! Deal with it. They will write it to stdout, stream1090 will read it then.This means that you have something like this:
<source> -> Buffer -> <stream1090 instance>
If you now want to add native input support, you will not get around this pattern. In an asynchronous way (async is not necessary for reading from files) you will have to do the following two things simultanously:
- Source has data ready! Put it into the buffer as fast as possible at a batch size that is not related to how stream1090’s core processes things.
- Stream1090 will pull data from this buffer with a batch size as the processing stage desires.
You may say now: ye, but others do that also. That is correct and this is usually done with a ring buffer (data structure). For the non-programming people: Think of it as an anolog clock. The long pointer points to the position where it is writing the data coming from the source, the shorter pointer points to where stream1090 reads from. You always want the small pointer trailing behind the long one. This has to be synchronized.
Now the crux: Apparently the pipe “|” manages to do that and “|” is around for a very long time. It has no problem with piping around “Hello world” messages, while also moving around some Terrabytes of data. So when moving functionality away from “|” and into stream1090, the competitor is “|”.
For files you can get a tiny bit of improvement by removing the need for a buffer. Read, process, read, process. Just sequential style. This will not work with live data from hardware.
End of long post. If someone has great insights, let me know.
1 Like
I’ve produced plots for the standard Low Pass filter, the 20260115 build filter that was giving me the best results prior to the Filter Optimisation work and my “double optimised” filter. The last first optimised the 20260115 filter, then ran a second pass optimisation on that.
Green is the standard Low Pass, Orange is the 20260115 build and Blue is the Double Optimised filter. Blue certainly looks different and has a narrower passband.
I compared stream1090’s results (-u 24) on three new 30 minute samples, and looked at whether further optimisation yielded any benefit.
These new samples are from different days, and have higher message rates than the original sample. The last of the three is from a different feeder - my main feeder which has a much better view of the sky, and typically double the message rate of my test rigs.
My conclusion is that, in my case there is little to be gained from pursuing any further filter optimisation.
For reference here is a comparison against airspy_adsb for each of the samples.
2 Likes
Not quite sure the problem you’re running into. If you want to keep a blocking file-like model for reading from the input, where control stays within your demodulator, just implement the read operations in terms of consuming from your (ring) buffer – blocking that thread if needed to wait for more data – and fill the buffer asychronously in a separate thread that’s responsible for talking to libairspy or librtlsdr? If your demodulator requires batches of a particular size, you can assemble those on demand on the read side, without worrying too much about the batch size that the fill side is using.
yes, there’s thread-safety and thread-wakeup fun to be had, but the C++ abstractions of mutexes / condition variables / etc aren’t terrible ..
dump1090 does a similar sort of thing where the SDR work (and magnitude conversion) is happening in a separate thread which pushes completed buffers to the main thread for consumption, though the control flow of the main thread is still arranged as pushing a buffer at a time to the demodulator, rather than the demodulator running “forever” with blocking reads. (Not that the dump1090 design is exactly the best, but it works)
1 Like
So i went for a slightly different approach. The ring buffer is divided in blocks. The block size corresponds to what stream1090 wants as batch size (this is known at compile time). It is however not copying it. I am processing the ringbuffer data directly from the buffer into the magnitude output buffer. The writers side can write variable size of data. However, it has to communicate if new full blocks are ready.
For initial tests and debugging, i went for reading from a file instead of airspy or rtlsdr input. This was ok for testing (and it is now faster than to “cat” the file). This scenario cannot be translated to airspy or rtlsdr input. Reading from disk is faster than processing the data. So the ring buffer writer is chasing the reader.
I will do some tests with airspy/rtlsdr next. There the reader will chase the writer.
Another wip update. This time a short one:
So native airspy prototype is now running for a few hours on the RP. Everything looks fine. However, an update to the repo will take some time. I will use this opportunity to refactor various things in stream1090.
There should be a generic InputDevice interface which will hide stuff like libairspy, librtlsdr, filereader, and stdin under the hood. Also with the possibility later to implement a network input device. I will also remove some features or let’s say not extend them. For example, for airspy native there is not going to be any switch for the input format. It wll basically be: filter on, filter off, if filter on load from file.
In order to avoid more CLI mess, i am leaning towards a config file. Are there people with an opinion on this?
I think a config file would be a great idea, and something most feeders who don’t rely on custom image files are familiar with editing.
One option could be to follow airspy_adsb conventions - config file location and name:
/etc/default/stream1090
executable name and location:
/usr/local/bin/stream1090
The airspy_adsb service file is here:
Ah wait! We are talking different things here. I am talking about a config for the device/pipeline
[device]
id=0
type=airspy
id=someserial
[params]
sample_rate=12000000
lna_gain=15
mixer_gain=15
vga_gain=15
filter=1
filter_file="./taps.txt"
[device]
id=1
type=rtlsdr
id=someserial
[params]
sample_rate=2400000
gain=49.6
filter=1
filter_file="./taps_rtl.txt"
``Sorry, I misunderstood again.
I was thinking it could be a one-stop config that would include the -u setting as well, so you could just launch say “launch stream1090” and no need for _10M, etc.
1 Like
This is a long term plan. I will need the option to create instances of multiple different devices. All running with their parameters. So for your particular case it would be stream1090 -<some letter> <config file>
1 Like
The framework should be able to deal with a multiple dongle/antenna setup including syncing the dongles. That was one of the reasons why i started with stream1090 in the first place.
1 Like
Yes that makes sense.
I was thinking that the device and executable are implicit in the config file - if using an airspy, the rtl-sdr settings would be commented out, and the sample_rate=12000000 line tells you that you need to load stream1090_6M
This will be a surprise at some point 
There was something missing: “at the same time, running in parallel”
Edit: Two loop antennas, perpendicular to each other, with two SDR dongles each sync them.
1 Like
Current plan: If you want native libairspy/librtlsdr support you will need to install it (i will write something about that). Cmake will check what is available. It will silently skip parts if the libs are not there in their dev version. I hope this ok for everyone.
2 Likes
Do you want me to publish the minimal JS run map that i use for debugging?
Edit: It hooks up to readsb, is stateless, you get a ping animation every time a DF-17 message arrives.
I currently just look at tar1090 to see if stream1090 is running.
Will this give any extra information?
Thanks again for the filter utiities @mgrone! I’ve been having a fun time toying around the last few days with the diff_evolve_fir Python script, and have been able to pull some pretty incredible results.
I’m going to continue to execute tests with my sample files, but wanted to share what I’ve been able to achieve so far with my sample 1 hour dataset, captured after LNA replacement. Still lots of ideas to toy around with in my mind, but this is an excellent jumping off point. 
Build 260123/No Custom Filter:
root@eyeinthesky:~/stream1090/filter_utils# cat /root/1hoursample.raw | /root/stream1090/build/stream1090_10M -q -u 24 > /dev/null
[Stream1090] build 260123
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples
[Stream1090] Size of sample buffer: 36864 samples
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 2016958 | 25.6% | 60.3% | 42.5% | -2902 |
| Comm-B | 7939 | 0.1% | 25.3% | | -11.42 |
| ACAS | 2372716 | 30.1% | 34.3% | | -3414 |
| Surv | 382973 | 4.9% | 42.7% | | -551.1 |
| DF-11 | 3095125 | 39.3% | 64.7% | 34.6% | -4454 |
-------------------------------------------------------------
| 112-bit | 2087442 | 26.5% | 59.9% | 41.5% | -3004 |
| 56-bit | 5788269 | 73.5% | 55.3% | 23.4% | -8329 |
-------------------------------------------------------------
| Total | 7875711 | 100% | 56.6% | 28.6% | -1.133e+04 |
-------------------------------------------------------------
(Max. msgs/s 0)
Messages Total 7875711
Build 260123/Custom Filter values derived from diff_evolve_fir:
root@eyeinthesky:~/stream1090# cat /root/1hoursample.raw | /root/stream1090/build/stream1090_10M -f /root/stream1090/custom_filters/30JAN26taps.txt -u 24 > /dev/null
[Stream1090] build 260123
[Stream1090] Input sampling speed: 10 MHz
[Stream1090] Output sampling speed: 24 MHz
[Stream1090] Input to output ratio: 5:12
[Stream1090] Number of streams: 24
[Stream1090] Size of input buffer: 15360 samples
[Stream1090] Size of sample buffer: 36864 samples
[Stream1090] Loaded 15 taps from /root/stream1090/custom_filters/30JAN26taps.txt
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 2568668 | 25.3% | 39.9% | 37.8% | -3696 |
| Comm-B | 8162 | 0.1% | 14.1% | | -11.74 |
| ACAS | 3151340 | 31% | 33% | | -4534 |
| Surv | 541327 | 5.3% | 24.5% | | -778.9 |
| DF-11 | 3900707 | 38.4% | 43.3% | 39% | -5613 |
-------------------------------------------------------------
| 112-bit | 2652254 | 26.1% | 39.5% | 36.8% | -3816 |
| 56-bit | 7517950 | 73.9% | 38.4% | 22% | -1.082e+04 |
-------------------------------------------------------------
| Total | 10170204 | 100% | 38.6% | 25.9% | -1.463e+04 |
-------------------------------------------------------------
(Max. msgs/s 0)
Messages Total 10170204
It took ~500 script iterations/~14 hours to find the improved tap values, but obiviously well worth the time spent. I readily admit, I stopped the script because I was anxious to try the taps out. Saturday has never been the busiest day in the air above me, and I’m hitting the time of day when traffic falls of for me, but I’ve never been close to 3000 msgs/sec, but I hit it today. Okay, 2,999 in reality. 
Simple Excel Comparision:
Graphs1090 Charts:
I’m going to let this run for a while and get some additional graph data, but things look fantastic thus far. Again, thank you mgrone. Really nice results and I’d strongly suggest everyone give the script a shot. Heading back to heating my office by hammering these samples with the script again.
2 Likes
Was the improvement gradual or did the optimum filter just pop up at random?
If you still have the data, it would interesting to see what the results looked like at the 100th, 200th 300th and 400th iterations.
1 Like
Question to the rtl_sdr people:
Would you be so nice to check via rtl_test -s 2560000 how viable a 2.56 MHz sample rate is? Viable = Not continuously dropping samples? I am getting something like this:
Found 1 device(s):
0: Nooelec, NESDR SMArt v5, SN: 00000001
Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Supported gain values (29): 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
[R82XX] PLL not locked!
Sampling at 2560000 S/s.
Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.
Reading samples in async mode...
Allocating 15 zero-copy buffers
lost at least 92 bytes
But 92 bytes are only lost in the beginning. Later on, things seem to be fine. For other sample rates > 2.4 MHz, i am loosing samples all the time. Something like this:
Sampling at 3200000 S/s.
Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.
Reading samples in async mode...
Allocating 15 zero-copy buffers
lost at least 688 bytes
lost at least 188 bytes
lost at least 188 bytes
lost at least 68 bytes
lost at least 68 bytes
lost at least 272 bytes
lost at least 376 bytes
lost at least 256 bytes
lost at least 188 bytes
lost at least 68 bytes
lost at least 68 bytes
lost at least 376 bytes
lost at least 136 bytes
lost at least 512 bytes
lost at least 188 bytes
lost at least 68 bytes
lost at least 68 bytes
lost at least 444 bytes
Type of SDR would be nice to know. Thank you.