Thank you very much!
It is not really your tax declaration. Transfer rates are fine for me.
tax declaration ? It is tax season in the US, but spell checker must have changed something.
Anyway, I think it took about 40 min to upload. How fast was the download?
This was with respect to your data privacy. For example uploading your range plot data is already borderline depending on if you moved your location from your real location or not.
I had full download speed.
Well, there’s nothing in that file about location.
I guess you could look at the plane locations and conclude I’m in Southern CA, but that’s about it.
Thank you for your sample set. Now i have another one in my library of “Did not expect that”.
One (minor) thing I noticed about that sample: when I run it through just stream1090, the number of DF19 messages is 14. When I also run it through plane_msg_stats.py script, it says number of DF19 messages is 0. All the other stats agree.
Yep, these things happen and that is the difference between a decoder and a demodulator. I cannot tell you what the french & belgium rs1090 people do with these messages. @JRG1956 observed the same.
Edit: The script will simply ignore any errors in decoding. You can have a look and uncomment the print error message.
You can send other forum members a direct message by clicking on their name and then the Message option.
Paste the link into the message and only they will see it.
That is what I did to send some file links. The downside is that if you include any comments in the message that might be of interest to others, those people won’t see them. I forgot that, and mentioned the DF19 anomaly that you also saw.
2 Likes
So here is quick status update. I rewrote some sampler code in the backend. Custom samplers are now possible. The standard upsampler can be customized for certain sample rate ratios (and it is to maintain the current behaviour).
However, there is now also a generic function which can deal with very odd ratios. This will have some consequences. One of those odd ratios is for example:
[Stream1090] Input sampling speed: 2.56 MHz
[Stream1090] Output sampling speed: 12 MHz
[Stream1090] Input to output ratio: 16:75
[Stream1090] Number of streams: 12
[Stream1090] Size of input buffer: 24576 samples
[Stream1090] Size of sample buffer: 115200 samples
And yes, once i have done enough testing of the new code, i will push it and rtl-sdr will have the option to upsample to 12 MHz instead of only 8 MHz.
2 Likes
A preliminary version is in the repo. These are the available configs:
Supported sample rate combinations:
2.4 → 8 (uint8 IQ)
2.4 → 12 (uint8 IQ)
2.56 → 8 (uint8 IQ)
2.56 → 12 (uint8 IQ)
6 → 6 (uint16 IQ)
6 → 12 (uint16 IQ)
6 → 24 (uint16 IQ)
10 → 10 (uint16 IQ)
10 → 24 (uint16 IQ)
People NOT affected from this change are those using these configs:
2.4 → 8 (uint8 IQ)
6 → 6 (uint16 IQ)
10 → 10 (uint16 IQ)
10 → 24 (uint16 IQ)
Edit: There is one change that affects every config. That is the batch size which is processed in one go. Please let me know if you are experiencing an abnormal performance behaviour.
Regarding the DF stats you send me:
I hope you dont mind if i am doing this here in the thread instead of doing it via PM. My fault. I was short-cutting a bit with the objective function in the filter optimizer when counting extended squitter messages.
I will fix that while fixing the optimizer. So what was the issue? Your crazzzzzy amount of ADS-B messages comes from GA. These have limited transponder capabilities. I was very lazy and adjusted the score by messages starting with 8d. That is the high tier: DF-17 (5 bits) + 3 bits (i can tell you the color of the socks of the pilot). It will be fixed once i finalize the changes to the optimizer.
Sorry for the inconvenience.
1 Like
2.56 → 12, Messages Total 603863
2.56 → 8, Messages Total 594596
pi@raspberrypi:~/stream1090 $ sudo cat /home/pi/samples.raw | ./build/stream1090 -s 2.56 -u 12 -q -v > /dev/null
[Stream1090] build 260222
[Stream1090] Input sampling speed: 2.56 MHz
[Stream1090] Output sampling speed: 12 MHz
[Stream1090] Input to output ratio: 16:75
[Stream1090] Number of streams: 12
[Stream1090] Size of input buffer: 8256 samples
[Stream1090] Size of sample buffer: 38700 samples
[IQLowPass] tap count: 15 symmetric: 1
[IQLowPass] taps: (-0.00245953, -0.00200476, 0.0238445, 0.0310966, 0.0531385, 0.096997, 0.0569147, 0.484946, 0.0569147, 0.096997, 0.0531385, 0.0310966, 0.0238445, -0.00200476, -0.00245953)
[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 266858 | 44.2% | 27.9% | 34.5% | 444.5 |
| Comm-B | 1279 | 0.2% | 0% | | 2.131 |
| ACAS | 135285 | 22.4% | 0% | | 225.4 |
| Surv | 53764 | 8.9% | 0% | | 89.56 |
| DF-11 | 146677 | 24.3% | 35.7% | 38.1% | 244.3 |
-------------------------------------------------------------
| 112-bit | 271321 | 44.9% | 27.5% | 34.1% | 452 |
| 56-bit | 332542 | 55.1% | 19.7% | 21% | 553.9 |
-------------------------------------------------------------
| Total | 603863 | 100% | 23.4% | 27.2% | 1006 |
-------------------------------------------------------------
(Max. msgs/s 1006)
Messages Total 603863
DF 0 : 132101
DF 4 : 52769
DF 5 : 995
DF 11 : 146677
DF 16 : 3184
DF 17 : 258183
DF 18 : 8658
DF 19 : 17
DF 20 : 1220
DF 21 : 59
600320850 iterations @1MHz
[Stream1090] Finished. (273.856s)
pi@raspberrypi:~/stream1090 $ sudo cat /home/pi/samples.raw | ./build/stream1090 -s 2.56 -u 8 -q -v > /dev/null
[Stream1090] build 260222
[Stream1090] Input sampling speed: 2.56 MHz
[Stream1090] Output sampling speed: 8 MHz
[Stream1090] Input to output ratio: 8:25
[Stream1090] Number of streams: 8
[Stream1090] Size of input buffer: 8256 samples
[Stream1090] Size of sample buffer: 25800 samples
[IQLowPass] tap count: 15 symmetric: 1
[IQLowPass] taps: (-0.00245953, -0.00200476, 0.0238445, 0.0310966, 0.0531385, 0.096997, 0.0569147, 0.484946, 0.0569147, 0.096997, 0.0531385, 0.0310966, 0.0238445, -0.00200476, -0.00245953)
[Stream1090] Sync Stdin Mode
[Stream1090] Reading from stdin
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 262995 | 44.2% | 20.2% | 27.7% | 438.1 |
| Comm-B | 1255 | 0.2% | 0% | | 2.091 |
| ACAS | 132862 | 22.3% | 0% | | 221.3 |
| Surv | 53126 | 8.9% | 0% | | 88.5 |
| DF-11 | 144358 | 24.3% | 26.9% | 37.3% | 240.5 |
-------------------------------------------------------------
| 112-bit | 267355 | 45% | 19.9% | 27.3% | 445.4 |
| 56-bit | 327241 | 55% | 13.9% | 19.4% | 545.1 |
-------------------------------------------------------------
| Total | 594596 | 100% | 16.7% | 23.1% | 990.5 |
-------------------------------------------------------------
(Max. msgs/s 990.5)
Messages Total 594596
DF 0 : 129757
DF 4 : 52091
DF 5 : 1035
DF 11 : 144358
DF 16 : 3105
DF 17 : 254495
DF 18 : 8487
DF 19 : 13
DF 20 : 1191
DF 21 : 64
600320850 iterations @1MHz
[Stream1090] Finished. (208.798s)
1 Like
Good! i think we getting to a point where the cow is dry now for the US people.
Edit: It is in the end about the message composition. You have some similar ratio to my samples when it comes to short vs long messages. The rest however is completely different:
-------------------------------------------------------------
| Type | #Msgs | %Total | Dups | Fixed | Msg/s |
-------------------------------------------------------------
| ADS-B | 87409 | 17.2% | 32.4% | 44.9% | 145.9 |
| Comm-B | 126550 | 24.9% | 0.1% | | 211.2 |
| ACAS | 39388 | 7.8% | 0.1% | | 65.73 |
| Surv | 64549 | 12.7% | 0.1% | | 107.7 |
| DF-11 | 189463 | 37.3% | 36.5% | 19.4% | 316.2 |
-------------------------------------------------------------
| 112-bit | 216348 | 42.6% | 16.2% | 22.5% | 361 |
| 56-bit | 291011 | 57.4% | 27.3% | 14.5% | 485.6 |
-------------------------------------------------------------
| Total | 507359 | 100% | 22.9% | 17.6% | 846.7 |
-------------------------------------------------------------
No problem, this is where the discussion belongs.
1 Like
@jimMerk2 Btw: you can try to crank up the gain by a bit. It might be a bad idea. It might work. Stream1090 likes to dig around in the noise.
I set the gain to be 20 dB. The reason I chose that value is, when dump1090-fa is the demodulator, it adjusts the gain to about 20 dB during the day (using adaptive gain). It adjust it to about 33-dB at night. I set a lower limit on the minimum gain to 20-dB, it could adjust even lower without that limit. This is on a system that has a Nooelec filtered LNA with about 30-dB gain.
Screen-shot of graphs1090 for dump1090-fa demod gain adjustment:
Are you looking at the sample data with a Hex editor to see the I,Q values?
1 Like
No i haven’t had a look yet. But usually you can feed stream1090 a bit more.
A quick status update. I did some smaller changes yesterday for some of the recognition logic. Nothing dramatic (i hope). This is a more or less temporary change.
The plan now is that i will stop working on this sampling stuff and focus on the recognition code. I will do some drastic changes to the very back of the backend.
There is a certain lack of infrastructure, because i haven’t touched that part much for many months now. Stuff that is somewhat required:
- An additional table that keeps more details about the planes. Currently there is ICAOCache.hpp. This is some very fast table. But it cannot hold much data. Since it covers the complete 16-bit range.
- A secondary plane table with more details should be the foundation for a better duplicate removal system. Duplication removal in this case is not easy since i need this also for some more things i want to try in the future including multi-device support.
- Here and there people were asking about RSSI values (like @jimMerk2). These will come at some point (sooner than later). But not because people asked for this feature, more because i need some feedback of the signal that led to a message in the stream1090 context.
- This will mean that i have to keep a history of the stream for a bit. I have an idea how to do that in a simple, yet efficient way.
- This needs to be future proof, and should scale with the option to reprocess a stream after removing a message from it.
Regarding the RSSI: People like that and i get it. However, be aware that it may happen that the rules you are used to, no longer apply. I will add some more numbers to the stats screen of stream1090 once i got this going.
3 Likes
Regarding the RSSI, you might look at as “bells and whistles”, but it is useful for setting gain to the optimum value. Without it, you’re kind of operating in the dark.
Ok i should probably explain that by example: Assume for the sake of the example that you end up with 8 bit of magnitude. Stream1090 is comparison-based so this
#
# #
# #
# #
# #
# #
# #
# #
|...|
is equivalent to
#
|...|
And that is all that matters. I tried to explain that here and there. The comparison-based model is a super set of the high and low model. You want to avoid:
# #
# #
# #
# #
# #
# #
# #
# #
|...|
while at the same time you want to get this going:
#
|...|
Why? Because by design, the code will dig through that noise anyways.
Edit: I used the word noise here which is already wrong.