Stream1090

[JRG1956]

@mgrone

stream1090 gets very close and depending on your setup may take over

I had been thinking about the validity of methodology for making comparisons, and agree my results may reflect the environment of my test location - relatively low traffic densities. Also the overall 2 percent difference in performance of my two test branches may be more significant in “edge” reception situations.

Yesterday I thought I would look at the total messages received by each setup looking at the track of a single aircraft from soon after it came into view to when the signal was lost. The setups are as described earlier - I updated to the latest build of stream1090, still running IQ with -u 24.

The data was gathered simply by taking screen shots of side by side instances of tar1090. The instance on the left is running stream1090 and they on the right is airspy_adsb. These are from the start, middle and end of the observations:

15:57h

16:17h

16:32h - aircraft out of range on both instances

The results are summarised in the table below.

What they show is that the discrepancy is happening at the start and end of the track (when the received signal is weaker but I didn’t have that column switched on for airspy_adsb). In the middle of the track both airspy_adsb and stream1090 get virtually same message rates - probably both receiving all the transmitted messages.

The second table is for another aircraft that was captured in the screenshots and flying a similar track. For those interested, here is the track for VOZ1547 on the stream1090 instance. JST680 can be seen following.

I know drawing conclusions from looking at just two aircraft is dangerous, but thought I would share this anyway. I think that there may be additional factors influencing observed performance, rather than just the initial messages being discarded in the time it takes stream1090 to confirm that an aircraft is real.

I’m not seeing any real difference with the new build of stream1090 - just a jump in tracks with a single message in graphs 1090 - this doesn’t matter and Flightaware is still happily accepting the feed.

[jimMerk2]

Ok, the --save to a png file worked. I can magnify the plot in preview on the Mac. I’ll have to play around with it. But, as you say there are better ways of doing this. Thanks for the update

[jimMerk2]

So vis_util.py works fine for single messages on the RPi. However when I tried two concatenated messages the RPi ran out of memory. This is on an RPi 4B 2 GB. It seems the processing load wasn’t too much, just ran out of memory. Does this have to do with the total time span of the two messages?
That is, the difference between the two MLAT times of the two messages?
Thanks

[mgrone]

I added console output for window size and message timestamps. Should look something like this:

[vis_util] Message @0000028eb0de02a185107aff4f; with timestamp 3575349.667 microseconds
[vis_util] Message @0000028ebb745d4845eb30d2b4; with timestamp 3575575.5 microseconds
[vis_util] Window size: 329.0 microseconds
[vis_util] Loading 3299 IQ samples
[vis_util] Loading 6597 magnitude samples

Keep in mind that if your messages are too far apart, the window will get big and the script will load all samples in between. The matplot lib does not like that.

[jimMerk2]

Ok. I see what my problem is. The setup I’m using for this sees very low traffic. It’s a poor indoor antenna that only sees about 20% of regular traffic here. So the messages can be quite far apart in time. I’ll have to improve the setup.

That being said, I don’t follow the numbers above. Wouldn’t the number of magnitude samples be half the number of IQ samples? Also, I don’t quite get the timestamp numbers you have. For example the first MLAT timestamp is 0x28eb0de or 42905822 decimal. Dividing by 12 MHz gives 3.57548517 seconds or 3575485.17 microseconds not 3575349.667 microseconds. Is there some other factor in there?

[mgrone]

Ah yea maybe some details i should explain:

@jimMerk2

That being said, I don’t follow the numbers above. Wouldn’t the number of magnitude samples be half the number of IQ samples?

This is just some example that i used for plotting. It plots two different sample sets. The original interleaved IQ samples (i0, q0), (i1, q1) and internally the magnitude is computed for each pair. The second set is obtained by computing magnitudes based on the pairs (i0, q0), (q0, i1), (i1, q1), (q1, i2), … Hence the double amount of samples.

@jimMerk2

Also, I don’t quite get the timestamp numbers you have. For example the first MLAT timestamp is 0x28eb0de or 42905822 decimal.

Good observation. So first thing to understand is that for MLAT the offset, where you start counting 12 MHz clock ticks does not matter. Important is that the relativ offset of the timestamp with respect to the start of the message is constant. So if messages start at t_i, it is fine to timestamp with t_i + c as long as c is the same constant for all messages.

Stream1090 internally uses something like a 128-bit shift register for each stream. A sequence of bits (56 or 112) is only checked when the first bit has reached the end of the shift register (that is, it will be shifted out with the next tick).

                    128 bit shift register
                     <--------------------
---------------------------------------------------------
| 112 bit message body                       |          |   <- new bits
---------------------------------------------------------
^ start                                                 ^ timestamp

If the message is fine, it will be timestamped based on the current amount of samples that have been received. This of course is then already 128 bits too late and does not correspond to the start of the message. While this is fine for MLAT, it is not fine for drawing purposes. Hence, the script has to adjust the timestamp (there is additional adjustment for the preamble which is “8-bit long”, so -128 - 8 = -136).

The above explanation is for a simple passthrough setting. With upsampling things get a bit more difficult.

[obj]

FWIW dump1090 reports the timestamp as at the 56th data bit, for historical reasons I don’t really remember:

demod_2400.c

Since that’s a constant offset it doesn’t really matter for mlat purposes with a free-running timestamp like in this case. It does become more relevant for GPS-synchronized timestamps (but those aren’t using that code path at all anyway)

[jimMerk2]

One more question. How is the 48-bit MLAT counter implemented? I guess it must be in software, but how accurate is that?

[mgrone]

Quite accurate. So what happens is that stream1090 does upsampling. So instead of running on your 10 Msps, it may upsample to 24 Msps which is a ratio of 5:12. Read 5 samples, interpolate 12 new out of these 5. The result are 24 of these shift registers clocked at 1 Mhz each, which are phase shifted. There is a 24 Mhz counter which is increment every time a new bit is shifted in and is then used for the 12 MHz timestamp. In this case it is easy: (current sample count / 2). Other more strange ratios with 12 MHz are not that straightforward. Important is that you are not fooling around to much with floating point numbers. The implementation for various sample speeds can be found here.

include/DemodCore.hpp

The number in < ... > denotes the number of streams which corresponds to the frequency in MHz of the current sample counter (m_currTime)

[jimMerk2]

So ultimately it’s based on the sample rate of the ADC in the Airspy hardware, or the sample rate of the ADC in the RTL-SDR hardware? But those samples have gone through the USB interface which must have a lot of jitter. Or is that jitter a small amount compared to 12 MHz?

Come to think of it, I guess all of the processing in the RPi is jittering. But it doesn’t matter.

[mgrone]

Wrong thinking. All you care about is the steady clock of the SDR. You count samples received and need a setup that does not loose samples. It does not matter when the samples arrive. All that matters is that you count them. It took me some time and @wiedehopf some nerves to explain me.

The beauty of MLAT lies in the idea to use the sample rate as clock. T_0 is when the first sample is being sent. Do not underestimate how long this sample will take until it arrives and is being processed. All that matters is that when sampling at 10 Msps for example, a sample is 1/10 of a microsecond. It is not neccessary to process a sample right away. All you need to know is the offset of a sample. i-th sample at 10 Msps is at i/10 microseconds. This is the most accurate clock you will get unless you are using additional hardware.

Just in case you are wondering why does that work? I mean T_0 is really something random. It is when the driver starts telling the hardware to start. And that you cannot even figure out on your own system. The server where you send the messages actually will figure this out. If two receiver stations receive an ADS-B message with a plane position, the server will do the math. It knows the position of the plane and the stations. Time of flight of the signal based on the distance receiver ↔ plane and their local T_0’s (edit based on the timestamp of the message). So the server can compute the difference between the two T_0’s of the stations.

This can be done for multiple stations, and when a plane does not do ADS-B with positions, this information is then used to compute the difference in time of arrival of a message which in turn can be used to give a good estimate about the position.

[jimMerk2]

@mgrone

It does not matter when the samples arrive. All that matters is that you count them.

Yeah, I can see that. And that’s why I was asking earlier about dropped samples. A dropped sample would definitely screw things up. We have all probably run rtl_test on the RTL-SDR dongle where you can set a sample rate. If you set a sample rate of 2.4 MHz or less it runs reliably without dropping samples. If you set a sample rate even a little over 2.4 MHz it drops a lot of samples. At least that’s what I have seen. I guess we can assume it never drops samples at 2.4 MHz. That’s good because that’s the sample rate that dump1090-fa uses.

[mgrone]

No it never does drop samples there at 2.4 Msps. I have my own “rtl_sdr” which allows higher sample rates and it is a disaster. Not only dropped samples, but also the ADC results are somehow becoming bad (could be because of the dropped samples, idk)

[obj]

@jimMerk2

That’s good because that’s the sample rate that dump1090-fa uses.

Not an accident - I picked 2.4MHz exactly because it was the highest reliable rate that had a sensible ratio relative to the ADS-B bit rate.

@mgrone

No it never does drop samples there at 2.4 Msps.

It unfortunately still does, sometimes. Poor USB power or bad USB connections make this worse. The RTL2832 has a tiny sample buffer so any USB data transfer errors probably result in dropped samples because the buffer runs out of space before a retry happens.

But normally it’s infrequent enough (e.g. maybe a few times a day) that mlat works well enough. When it does happen, the mlat server notices and restarts the synchronization process.

[wiedehopf]

@obj

But normally it’s infrequent enough (e.g. maybe a few times a day) that mlat works well enough.

On good platforms this doesn’t happen in my experience.
The “few times a day” you recall might be related to the pi3 doing that?

[jimMerk2]

Yeah, I could believe that RTL-SDR dongle occasionally drops samples even at 2.4 MHz – If it drops samples at 2.4 MHz + a little bit. When you run rtl_test you probably don’t run it for hours. You run it for a few minutes and see that it’s not dropping samples and conclude yeah that’s running ok.

Somewhat related is when we were using rtLpower to look for interference in the 800 MHz to 1200 MHz using the RTL-SDR dongle. I remember the sample rate was 2.8 MHz I guess a lot of samples were being dropped. But it probably did’t matter too much since rtl_power is doing FFTs. Basically measuring power in frequency bins.

[abcd567]

@obj

But normally it’s infrequent enough (e.g. maybe a few times a day)

Well when I run Debian or Ubuntu etc through Oracle Virtual Machine on my Windoows 11 PC, the dongle dropout is quiet frequent, and status often shows “receiver NOT synchronized”.

Of course this has nothing to do with the Dongle, or USB port of my Windows PC, or dump1090-fa. It is purely due to “dongle pass-through from Windows to Vertual Machine” issue.

[JRG1956]

Apologies if I should have been able to work this out.

Could you explain your message naming convention for output when piping a recorded sample onto stream1090?

I’m trying to compare the same message as received by the two branches of my setup with vis_util.py. Here is a snippet of the terminal output from both branches pasted into Excel - I produced the sample files using a bash script that triggers the sampling at the nearly the same time on both Pi4 and the Pi5 (44 millisecond difference in this case)

I could see that the message identifiers were the same apart from the first 11 hex characters and assumed they were the same message.

I got these results for the two highlighted pairs.

jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample106-3.raw --format iq_int16 --fs 10000000 --message "@0000000dcd958d7c6a589908859a28041552a7c0;"   
[vis_util] Message @0000000dcd958d7c6a589908859a28041552a7c0; with timestamp 75247.583 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1601 IQ samples
jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample107-3.raw --format iq_int16 --fs 10000000 --message "@00000004f9d58d7c6a589908859a28041552a7c0;"
[vis_util] Message @00000004f9d58d7c6a589908859a28041552a7c0; with timestamp 27039.583 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1601 IQ samples

And

jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample106-3.raw --format iq_int16 --fs 10000000 --message "@0000001bb18fa8001a8c9d500030a8000041313c;"
[vis_util] Message @0000001bb18fa8001a8c9d500030a8000041313c; with timestamp 151108.417 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1601 IQ samples
jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample107-3.raw --format iq_int16 --fs 10000000 --message "@00000012ddd0a8001a8c9d500030a8000041313c;"
[vis_util] Message @00000012ddd0a8001a8c9d500030a8000041313c; with timestamp 102900.5 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1600 IQ samples

So they are the same messages. The graphs confirm what I saw with both branches running airspy_adsb - the Pi4/Airspy mini branch has about 3dB higher signal strength. It also captured 25% more messages than the Pi5/R2.

I’m not sure that being able to compare setups at the level of single messages is useful, but thank you for making it possible for someone with low skill levels to be able to do so. Let me know if there is anything I can do with my test setup to help your stream1090 development.

I tried using your multisample view, but it doesn’t produce anything useful because it assumes the sample files have a common timebase which mine don’t unless I can synchronise starting capture down to the microsecond level.

Finally, there may be a problem with your message selection logic. When I inadvertently used a message identifier from one of the sample files with the other sample file, it plotted a graph even though the identifier didn’t exist in the that file.

In this example I ask for a message ID from the sample107-3.raw file (second highlighted above) while designating the sample106-3 file

jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample106-3.raw --format iq_int16 --fs 10000000 --message "@00000012ddd0a8001a8c9d500030a8000041313c;"
[vis_util] Message @00000012ddd0a8001a8c9d500030a8000041313c; with timestamp 102900.5 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1600 IQ samples

It somehow comes up with the timestamp and bit boundaries from the message in sample107-3, but the signal doesn’t correlate.

[obj]

@wiedehopf

On good platforms this doesn’t happen in my experience.
The “few times a day” you recall might be related to the pi3 doing that?

It’s mostly just what I see in aggregate on the server side, long-term (> days) sync stability is rare, I haven’t dug into if it’s specific hardware combinations.

[mgrone]

So it took me some time to understand what you are doing here. Let me start in reverse order:

@JRG1956

Finally, there may be a problem with your message selection logic. When I inadvertently used a message identifier from one of the sample files with the other sample file, it plotted a graph even though the identifier didn’t exist in the that file.

There is a misunderstanding what the script does. The logic is not wrong since there is no logic. You provide a sample stream and a message. The script assumes that the message is part of the same time interval. All it does is to match the timestamp of the message and the sample rate to a common time domain (microseconds starting with the first sample). It will not relate the message to the actual signal. You can provide it even with a fake message, it does not care.

Now let’s have a look at the message examples you provided. We are talking here about the flying doctors (in a beautiful Super King Air 350). Look at the later two messages:

@JRG1956

And

jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample106-3.raw --format iq_int16 --fs 10000000 --message "@0000001bb18fa8001a8c9d500030a8000041313c;"
[vis_util] Message @0000001bb18fa8001a8c9d500030a8000041313c; with timestamp 151108.417 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1601 IQ samples
jrg@MBPro2021 vis_util % python3 vis_util.py --file /Users/jrg/Desktop/sample107-3.raw --format iq_int16 --fs 10000000 --message "@00000012ddd0a8001a8c9d500030a8000041313c;"
[vis_util] Message @00000012ddd0a8001a8c9d500030a8000041313c; with timestamp 102900.5 microseconds
[vis_util] Window size: 160.0 microseconds
[vis_util] Loading 1600 IQ samples

you will get

{'df': '21', 'squawk': '7212', 'bds40': {'bds': '40', 'selected_mcp': 15000, 'barometric_setting': 1013.2}, 'icao24': '7c6a58'}
{'df': '21', 'squawk': '7212', 'bds40': {'bds': '40', 'selected_mcp': 15000, 'barometric_setting': 1013.2}, 'icao24': '7c6a58'}

These are two Comm-B messages. You will not be able to tell the difference here. The chance that the data in this type of messages changes within a short time span is very very low.

Regarding the first two pair of messages, you will get two ADS-B messages with more variable content.

{'df': '17', 'icao24': '7c6a58', 'bds': '09', 'NACv': 1, 'groundspeed': 246.34934544260514, 'track': 147.60015982608067, 'vrate_src': 'barometric', 'vertical_rate': 0, 'geo_minus_baro': 500}
{'df': '17', 'icao24': '7c6a58', 'bds': '09', 'NACv': 1, 'groundspeed': 246.34934544260514, 'track': 147.60015982608067, 'vrate_src': 'barometric', 'vertical_rate': 0, 'geo_minus_baro': 500}

However, you need to look here at the time domain. 75247.583 vs. 27039.583 microseconds which is a difference of less than 50ms. This could be the same message. Could also be that these are two different messages containing the same data. However, your previous comment

@JRG1956

time on both Pi4 and the Pi5 (44 millisecond difference in this case)

suggests that it is the same message.