Finding optimum gain visually

[chrislfa]

Gainmap

I’ve written a script which scans the airwaves at different gains and produces a heatmap image for each gain. These can be used to visually find the optimum gain; that is the gain which gives the greatest 1090MHz signal without introducing excess in-band noise or boosting nearby unwanted signals. This is akin to performing a digital camera ISO test and visually finding the optimum setting for a given scene which gives the best overall result without introducing needless noise.

The advantage with this approach over collecting numbers is that the results can be gathered quickly and interpreted intuitively, like fine-tuning a radio while listening to it. There’s no need to analyse positions vs aircraft since, by definition, the optimum levels already maximise signal for given noise. The disadvanatge with this approach is that there is some play around the optimum level for a given setup, and further testing a couple of levels up or down over a period of days or weeks could be useful.

My results

There are 29 images produced for each test and I’ve selected a few from each run which show what I’m looking at.

Pro Stick Plus with FlightAware external antenna indoors

Same again but including some nearby GSM 950-60 signals (UK) to see how they are affected.

You can see in the first test that maximum gain does not introduce significant noise around 1090MHz. In the second test at maximum gain you can see there is a slight ‘glow’ of noise around 1090MHz but nothing that is masking signals. The GSM signals are a touch brighter at maximum gain.

Conclusion

I would be inclined to leave the installation set to AGC since I’m not seeing noticeable in-band noise and the GSM signals are not impinging. If I wanted to test a bit deeper I could try reducing gain to something like 38.6, where the signal looks about the same, there is a touch less surrounding glow and the GSM signals are slightly weaker. It may give a slight improvement.

Pro Stick Plus with FlightAware external antenna indoors and Uputronics 1090Mhz SAW filtered bandpass preamp

Same again but looking at those nearby GSM signals.

You can see in the first test that increasing the gain increases the signal until it gets to around 20.7, and then in-band noise starts to increase significantly. How are those GSM signals affected? You can see in the second test that the filtering in the preamp does a great job of killing them so they are pretty much gone at that lower gain, and only just visible at the maximum gain.

Conclusion

Reducing the gain to around 20.7 should give maximum signal with minimal in-band noise, and the filtering is nicely taking care of nearby signals so I don’t need to worry about those at all.

Overall conclusion

With the standard Pro Stick Plus and external antenna on this setup I’d probably leave the gain set to AGC. Even if it goes to max there’s no excess noise to worry about. With the filtered preamp I could leave it set to AGC as long as AGC can intelligently make the same determination and reduce the gain to eliminate the noise. It’s not clear to me if it will do that or not. Therefore I’m inclined to manually set it to 20.7 and see how the stats pan out over a couple of weeks.

These visual results are consistent with statistical results I’ve seen mentioned by a couple of other people, where a high value was good without a preamp and a mid-range value was more suitable with a preamp.

It should be noted that after I installed the preamp, positions, aircraft and range all increased with AGC still set. If AGC is acting intelligently then this may be because it has detected the noise and lowered the gain to somewhere around that optimum value. But if AGC just sets the gain to max then that means I got an improvement despite the noise, and setting the gain lower should improve things even more.

Instructions

• Follow the instructions in the Do I Need A Filter? thread to install rtl-sdr and python-imaging

• Perform a scan to get an idea of what’s around you

• Edit heatmap.py with vi or nano, scroll to the end and use a # to comment out the lines which overlay text on the images and save the file

  #    shadow_text(draw, margin, img.size[1] - 45, 'Duration: %i:%02i' % (hours, minutes), font)`
  #    shadow_text(draw, margin, img.size[1] - 35, 'Range: %.2fMHz - %.2fMHz' % (min_freq/1e6, (max_freq+pixel_bandwidth)/1e6), font)`
  #    shadow_text(draw, margin, img.size[1] - 25, 'Pixel: %.2fHz x %is' % (pixel_bandwidth, int(round(pixel_height))), font)`
  #    shadow_text(draw, margin,  img.size[1] - 15, 'Started: {0}'.format(start), font)`
  

• Use vi or nano to create a script called gainmap with the following contents

  #!/bin/bash
  
  for g in -10 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
  do
    echo "Toggling dump1090-fa service on and off..."
    sudo systemctl start dump1090-fa
    sleep 5
    sudo systemctl stop dump1090-fa
    sleep 5
    echo "*** TESTING GAIN $g ***"
    sudo rtl_power -f 1060M:1120M:100k -i 5 -c 50% -e 5m -g $g -F 9 > scan$g.csv
    ./heatmap.py scan$g.csv scan$g.png
  done
  

• Make it executable with chmod +x gainmap

• Run it with ./gainmap and wait 2.5 hours for it to complete. You will end up with 29 png image files, each with a filename of the gain that was used, eg scan25.4.png. Transfer them from your RPi to your computer using scp as detailed in the previously mentioned thread. On the Mac you can copy them to your current directory using (substitute xxxxx with your RPI’s IP address):

  scp pi@xxxxx:scan*.png .
  

• Inspect the image files and find the gain where you are getting the best signal with the least noise.

Notes

• Plan the scanning for when you have a reasonably constant level of signals. You can see this on your stats page. Eg for me it starts getting busy from 6am and then is constant from around 9am to 8pm.
• If needed edit gainmap and change the gain levels to match those for your dongle. The ones included here are for the Pro Stick Plus. If in doubt you can just use integers since dump1090 will always choose the nearest value for your receiver anyway.
• If needed edit the range in the rtl_power command to include any nearby areas of interest you identified from your initial heatmap scan. The range used here is 1060Mhz to 1120MHz, ie 1090MHz +/- 30MHz. This covers the area which picks up in-band noise from the preamp.
• Remove all previous scans and images with rm scan*.*
• In the script the starting and stopping of dump1090-fa gets around a problem where sometimes rtl_power cannot access the receiver. Quick and dirty but it works.
• If you’re using dump1090-mutability or something else then edit the file accordingly
• You may find that some gain values produce images which exhibit moiré patterns. Since these seem to occur at very low gains you can ignore them in your assessment.

Please feel free to experiment and post your results.

Regards,
Chris

[kg4pid]

I like your idea and will be trying it soon as I have just gotten a test system running. Being a ham radio operator I’m used to using my ears when making changes. Receiving a signal is all about the signal to noise ratio, not the absolute level.

[roadfun]

I’ve started this up and will report result once I get a chance to look at it. I’ll compare it to the results I got from the other approach (message counts etc).

[chrislfa]

Thanks @roadfun it’ll be interesting to see how they compare.

[roadfun]

I ran your script and placed the resulting images into a PDF document on Dropbox. The final page has a view of the mobile phone signals in my area. The full chart of that is here. I am running the blue FA stick attached to an FA antenna.

I also upload a PDF with the results of 3 runs of the gain script that collected messages etc.

I have been running with my gain set to 42.1 based on the previous scan of message count. It wasn’t a clear winner but seemed like a reasonable choice.I am not sure how to interpret your gain charts and would like your opinion when you look at those - what seems to be the best choice?

Note: I used to run with a Pepsi can antenna before deciding to try the FA antenna. The FA antenna has yet to surpass or even match my Pepsi can hence I’ve been tinkering around.

[chrislfa]

Hi @roadfun, thanks for those results, very comprehensive. I should point out I’m no expert in interpreting the heatmap results but I’ll give it a go, and that’s one reason it’s interesting to see results from others, especially when there are other tests to compare with, as you’ve provided.

I can see that a higher gain will introduce more noise, like cranking up the ISO on a photograph. I can see that the heatmap will also show increased noise in the form of an in-band glow and that visually one can compare how the signal lines look against that glow. The goal is to get the most detail/signal without needlessly ramping up the noise.

The problem is that while I can look at the heatmaps and say to myself “ah, that looks like the one with the best contrast” it’s not clear that the decoder ‘sees’ the same result. It may well be that it can see signal despite what we would consider visually to be a lot of noise. If I set the gain to the best scan visually, perhaps I am needlessly dragging it down.

The good news is that if the best gain is found then we can look at those heatmaps and say “ah, so that’s what it looks like visually” and now we know for next time and it should be much easier. As a methodology this visual approach does appear to give a decent enough starting point from which to iterate.

I’ve been doing some more testing with my own setup which has the Uputronics filtered preamp. This does a great job of killing nearby signals and introduces a lot of glow on the heatmap at high gains. Logically it seems like I don’t want to be running it up there. Until recently I’d been using AGC but it’s not clear to me how that actually works moment to moment.

I found a really interesting post from @obj who is staff who said that AGC works best with steady signals. In the context of ADS-B decoding this implies that tuner AGC ultimately gives you a result which gives you a decent result in preference to max performance. Even setting a fixed gain is a compromise between increasing performance much of the time, perhaps at the expense of performance at certain times of the day. But it’s certainly true that setting tuner gain can be used to avoid needless in-band noise from one’s own setup, and it’s not clear how AGC copes with that – does it overall do a good job of mitigating it or is it confused by it.

I set my gain to 20.7 based on that heatmap image looking like the one with the best contrast. My positions and aircraft dropped from normal AGC. I then tried it on 49.6 but reduced to 48 based on obj’s post. My positions and aircraft have gone to nearly a high yesterday. I know I’m well into the glow on the heatmap at this level. My assumption is that if I drop it a touch things could get better still as I reduce the noise, but clearly dropping it to 20.7 is too much. So for today I’m upping it to 38.6, which on the heatmap has a fair bit of glow but is a step down from what’s at 48.

Looking at your scans you’ve been running at 42 and I see little difference between the noise at 42 and the noise at 49.6. The glow you have at 49.6 is less than the glow I upped my gain into yesterday and dump1090 got more info from it. So I would be inclined to change your gain to 48 (avoiding 49.6 as per obj’s comments) and see if dump1090 can extract a bit more info from the tuner at that level.

Those signals you have down at 880MHz and 930MHz, could they impinge on your tuner’s performance? I don’t know but I don’t think upping your gain from 42 to 48 will dispoportionally increase them such that they could somehow become any more or less of a problem.

Perhaps once you’ve played with the gain you can try buying the Uputronics 1090MHz filtered preamp. You can see how it kills out of band signals nearby in my initial post. Perhaps it will have a much greater benefit for you with those signals nearby, as well as boost your in-band signals. Then you’ll be in the same position as me, seeing how much I need to reduce the gain to eliminate extra in-band noise but not needlessly going lower. The good news is that merely adding the preamp increased everything including range and aircraft, so even on AGC it ends up in a pretty good configuration.

Your numerical gain tests appear to give the best aircraft decoding at that highest gain, with higher numbers for the components at lower gains. Perhaps this too suggests that a gain around 48 will give you a slight boost over what you have now.

I’ll continue tweaking my gain here over the next week or two and once I’ve found what seems to be the optimum result I’ll post more info with the relevant heatmap scan to see what that visually looks like for future reference. It should give an idea of what kind of noise we can see on the heatmap knowing that the decoder can cut through it.

Not an exact science but I remember from tuning in to skip from other countries in the 1980s or finding the right balance of gain over tuning there’s an element of chasing a moving target to this.

[jlb56]

Hmmm tried your script but came up with the following error:

Using device 0: Generic RTL2832U
Found Rafael Micro R820T tuner
Tuner gain set to 0.00 dB.
Exact sample rate is: 2790696.209609 Hz
[R82XX] PLL not locked!

Now I am using a Prostick Plus, into a PI3 B+. I got the same error when I tried ABCD3456 filter check… ??

[evangelyul]

it indicates a problem with the dongle. I posted elsewhere same problem with my prostickplus, till I replaced its usb connector. (in the end was not even recognized by the system as usb device)

[roadfun]

Thanks for the commentary. I decided to raise my gain to 48 with the FA antenna and I’ll let that run for a few days or a week. Then I’ll likely switch back to the Pepsi antenna and run it with AGC for awhile to compare the results.

Since installing the FA antenna my station has fallen pretty significantly in the rankings and there hasn’t been anything like extended bad weather that would seem to cause that other than the antenna change. I’ve considered the Uputronics but I’m not sure I want to spend more money on the setup.

[Dxista]

Try installing the graphs included with the ADS-B Receiver Project. I find them extremely useful, and you don’t need to wait hours or days to get a picture, literally, of your receiver’s performance.

Try to keep the noise at -20 dbfs maximum. Average signal between -5 and -10 dbfs. Limit as much as possible signals stronger than about -1.5 dbfs.

In my case, 33.8 and 36.4 seem to be the best gain settings to meet the above conditions.

I have an FA antenna, RTL-SDR Blog LNA/filter combo (27 dB), and an RTL-SDR Blog dongle.

When I was ‘flying blind’, no graphs, I was using a gain setting of 20.7. Go higher than 36.4, or lower than 20.7 and performance, in my case, suffers.

[chrislfa]

@jlb56

[R82XX] PLL not locked!

Check the gainmap script and make sure that the syntax of the rtl_power command is correct, in particular the 1060M:1120M:100k part. That error means that it can’t deal with a frequency you’ve asked it to, which could imply a typo in the syntax.

[jlb56]

That was it!! It’s a prostick plus. Tried shifting it to 1060 : 1100, no good, then made it 1090 : 1090 workded! So what is the frequency spread of a prostick plus???

[jlb56]

Or sort of working … now showing this:

Traceback (most recent call last):
File “./heatmap.py”, line 615, in
summarize_pass(args)
File “./heatmap.py”, line 320, in summarize_pass
args.pixel_bandwidth = step
UnboundLocalError: local variable ‘step’ referenced before assignment
Toggling dump1090-mutability service on and off…
*** TESTING GAIN 36.4 ***
./gainmap: line 3: 542 Floating point exceptionsudo rtl_power -f 1090M:1090M:100k -i 5 -c 50% -e 5m -g $g -F 9 > scan$g.csv
loading
x: 0, y: 0, z: (0.000000, -100.000000)
Traceback (most recent call last):
File “./heatmap.py”, line 615, in
summarize_pass(args)
File “./heatmap.py”, line 320, in summarize_pass
args.pixel_bandwidth = step
UnboundLocalError: local variable ‘step’ referenced before assignment
Toggling dump1090-mutability service on and off…

[Dxista]

I have no idea where the frequency can be changed, but in the case of the ProStick Plus, I would think it must be 1090, or it may/will fall outside of the built-in filter range.

[jlb56]

@jlb56

UnboundLocalError: local variable ‘step’ referenced before assignment

Thats what I was thinking re the freq, but what is the error referring to?? In the the script, is 1090M one of the frequecncy boundaries?? If so, would you need to alter the freq step??

[jlb56]

so by making it 1090 1090 it works but comes up with that error, due to it trying to do something it cant, like dividing something by zero, not sure what figures I need to try.

[jlb56]

I see from ABCD filter post that “[R82XX] PLL not locked!” would be a standard response, I will let it run and see what happens, but it seems not to cycle???

Jeff

[jlb56]

Well ABCD567 filter script DID work. I have tried this script, as published now, after the success with the first filter script and it appears to be working. Will have to wait now for it to finish and for me to wake up again as heading to bedtime over here in OZ, will advise how it went in about 7 hours

[jlb56]

Well don’t have to wait 7 hours to see how this is going, great so far. All ready it lets me make a better choice on gains visually than looking at the standard gain test result we use. A quick snap shot:

[jlb56]

All finished, successfully done. GREAT scriipt: