Thoughts on optimizing gain

I’d recommend just running the script and saving the graph.
Then change something and run the script again a week later or even a week and a day later, so the day it ran some time on the old and new setting is excluded.

Here are the scripts I used to get the comparison plot. I used a separate gnuplot script to draw the graph because I couldn’t get gnuplot to play nicely inside the bash script the way wiedehopf did it. There are probably lots of ways this could be improved, so if someone feels inclined, feel free.

Anyway, there are two scripts to run. The first called adsbbaseline.sh should be run after the receiver setup has been unchanged for a week, since it pulls a week of data and saves it for future reference. Ideally run this before you start making changes to your receiver.

The second script will pull the most recent complete day of data, and compare it to a reference file that you specify - this would be run on the day after your receiver has been up for at least a day.

So, example usage would be:

./adsbbaseline.sh <reference>

eg.

./adsbbaseline.sh 38.6

This will give an output file called for example : 2019-07-02-38.6-baseline

Once you make changes to your receiver and want to compare it against the previous setup, you can run:

./adsbcompare.sh <reference filename>

eg

./adsbcompare.sh 2019-07-02-38.6-baseline

This will produce a graph and save it in the location http://your.pi.ip/dump1090-fa/data/compare.png

adsbbaseline.sh:

#!/bin/bash

# Fetch data from rrds

date=$(date -I --date=yesterday)
echo "Creating baseline file:"$date-$1"-baseline ending $date"


rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-local_accepted.rrd AVERAGE -s end-7days -e midnight today -r 3m -a > /tmp/messages_l-base
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-remote_accepted.rrd AVERAGE -s end-7days -e midnight today -r 3m -a > /tmp/messages_r-base
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_aircraft-recent.rrd AVERAGE -s end-7days -e midnight today -r 3m -a > /tmp/aircraft-base


# Remove headers and extraneous :


sed -i -e 's/://' -e 's/\,/\./g' /tmp/messages_l-base
sed -i -e 's/://' -e 's/\,/\./g' /tmp/messages_r-base
sed -i -e 's/://' -e 's/\,/\./g' /tmp/aircraft-base

sed -i -e '1d;2d' /tmp/messages_l-base
sed -i -e '1d;2d' /tmp/messages_r-base
sed -i -e '1d;2d' /tmp/aircraft-base


# Combine both files to create space separated data file for use by gnuplot


join -o 1.1 1.2 2.2 /tmp/aircraft-base /tmp/messages_l-base > /tmp/tmp
join -o 1.2 1.3 2.2 /tmp/tmp /tmp/messages_r-base > ${date}-$1-baseline

adsbcompare.sh

 #!/bin/bash

# Fetch yesterdays data from rrds

rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-local_accepted.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/messages_l
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-remote_accepted.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/messages_r
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_aircraft-recent.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/aircraft

# Remove headers and extraneous :

sed -i -e 's/://' -e 's/\,/\./g' /tmp/messages_l
sed -i -e 's/://' -e 's/\,/\./g' /tmp/messages_r
sed -i -e 's/://' -e 's/\,/\./g' /tmp/aircraft

sed -i -e '1d;2d' /tmp/messages_l
sed -i -e '1d;2d' /tmp/messages_r
sed -i -e '1d;2d' /tmp/aircraft

# Combine files to create data usabel by gnuplot

# Combine both files to create space separated data file for use by gnuplot

join -o 1.1 1.2 2.2 /tmp/aircraft /tmp/messages_l > /tmp/tmp
join -o 1.2 1.3 2.2 /tmp/tmp /tmp/messages_r > /tmp/$(date -I --date=yesterday)-scatter

gnuplot -c adsbcompare.plot $1


sudo cp /tmp/compare.png /run/dump1090-fa/compare.png

and you also need this gnuplot definition file:

adsbcompare.plot

date = system("date -I --date=yesterday")
base = ARG1
set terminal pngcairo enhanced size 1280,1024
set output '/tmp/compare.png'
set title 'Comparison of '.date.' against '.base
set xlabel 'Aircraft'
set ylabel 'Messages rate /s'
set grid xtics ytics
f(x) = c*x/sqrt(d+x**2) + a*x**2 +b*x
g(x) = j*x/sqrt(k+x**2) + h*x**2 +i*x
c = 4000
d = 7000
a = 0.05
b = -20
h = 0.05
i = -20
j = 4000
k = 7000
fit f(x) '/var/cache/fetch/'.date.'-scatter' via a,b,c,d
fit g(x) base using 1:($2+$3) via h,i,j,k
plot    base using 1:($2+$3) with points linecolor rgb "grey" notitle, g(x) title base, \
        '/var/cache/fetch/'.date.'-scatter' with points linecolor rgb "blue" notitle, f(x) title "Current"

Assuming all goes well, you should end up with a graph such as:

5 Likes

Hey All! Im pretty close to Multiple airports, I have 4 Major airports, (EWR, JFK, LGA, PHL) all within 100 NM, the first 3 within probably 50nm.
I just used the ‘strong signal’ to test my gain setting (set at -10 right now, but found 10-15 seems just as good… but less ‘range’)
this is the output: Percentage of strong messages: 27.467
WHEW!
Im assuming this is probably far too high to really get the good signals of the planes near by, but i Do get pretty good results (over 100 planes pretty much all day).
I have a flightaware pro stick, 50M of coax cable, and the flightaware antenna on my roof.

Think i should lower my gain??
Lowered to 15db and it got 19%,
Dropped to 10db
Percentage of strong messages: 14.814 - ran it a second time, i got 16%
Still too high? Seems like i have far fewer aircraft than my AGC setting.

Do you have aircraft flying directly over you all the time? If so, you might find that you get a higher proportion of strong signals, but that seems excessively high.

With pro stick -10 is probably too high, especially with an external antenna. I would turn it down several steps and see what you get.

I have 4 major airports near me with a Heathrow approach stack right above my house and I have my gain at about 38, though with a pre amp and loft antenna.

I dropped it down to 15 and 10 as above, but I just ran a 5db and let it run for about 10 minutes. STILL getting 32.8% strong.29%20PM
Thats the flight path most planes take into EWR. Its not DIRECTLY over head, but pretty close and we Do have a number of smaller planes in and out of Morristown Muni which is even closer.

Cross-modulation by strong Interference, possibly cell towers.

Carryout the scan as described here:

Do I Need A Filter?

Went all the way to 0db and let it run for half hour.
27% strong signals.
Ordered another filter (ADS-B LNA High Performance Triple Filter Low NF Amplifier by RTL-SDR Blog https://www.amazon.com/dp/B078TBG4H8/ref=cm_sw_r_cp_api_i_ClShDbM1DH935)
Maybe that will get me more range. I am across the street from a cell tower to. Guess the Fa filter in the pro stick just isn’t enough for me

I use cavity filters. Way too may cell towers in Brooklyn.

You most likely didn’t actually change the gain.

A gain of 0 is 55 less than -10 approximately.
Running a gain of 0 with your setup would give you 0% >-3dB, i would pretty much bet on it.

According to your stats page you are using the installed version, not the piaware sd-card version.
That means you need to set the gain in /etc/default/dump1090-fa, not with piaware-config.
See case 2 of this post to see how to do it:
For Beginners - How to Set / Change Gain

-10 is equal to around 55 in most cases.
Next lower gain value is 49.6
You could try a gain of 40 and then 35 probably.

The LNA is very nice but you should try and put it before the 50 m of coax.
You sure you don’t mean 50 ft?

Anyway putting the LNA in front of the coax run or wherever the cable comes into the house is probably connected to getting new cables, so at the end of the run will have to do for now.

Have you ordered a bias-t for the LNA already to supply it with power?
Something like this: https://www.ebay.com/itm/172460255780

Without power the LNA doesn’t work and will just block all signals.

Thanks! I cancelled the order for now, as if Im running -10 i need to find out what I can get if my signal is too strong before spending $30 on the device and another 10-15 on a power supply. I saw the 5V input (from the other page) and it looked like it was powered with USB micro - im sure theres an all in one that will do that, so if I need, ill look for that.
But back to square one.
heres the output from
sudo systemctl status dump1090-fa -l
dump1090-fa.service - dump1090 ADS-B receiver (FlightAware customization)
Loaded: loaded (/lib/systemd/system/dump1090-fa.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2019-07-04 22:59:35 EDT; 11h ago
Docs: https://flightaware.com/adsb/piaware/
Main PID: 7305 (dump1090-fa)
CGroup: /system.slice/dump1090-fa.service
└─7305 /usr/bin/dump1090-fa --device-index 0 --gain -10 --ppm 0 --net-bo-port 30005 --max-range 360 --net --ne

Jul 04 22:59:35 raspberrypi systemd[1]: Started dump1090 ADS-B receiver (FlightAware customization).
Jul 04 22:59:35 raspberrypi dump1090-fa[7305]: Thu Jul 4 22:59:35 2019 EDT dump1090-fa 3.7.1 starting up.
Jul 04 22:59:36 raspberrypi dump1090-fa[7305]: rtlsdr: using device #0: Generic RTL2832U (Realtek, RTL2832U, SN 00001000)
Jul 04 22:59:36 raspberrypi dump1090-fa[7305]: Found Rafael Micro R820T tuner
Jul 04 22:59:36 raspberrypi dump1090-fa[7305]: rtlsdr: enabling tuner AGC

Gain set to -10 and AGC enabled so…
I was editing the PiAware.conf as I saw on the FA page, as the /boot/piaware-config.txt wasnt there, so it said to edit the /etc/Piaware.conf
oh well. Ill try some more and let you all know the results.

You can feed the bias-t with the 3.3 V or 5 V from the Raspberry Pi.
(from the GPIO pins)
You’ll still need some jumper wires to connect to the pin header unless you feed comfortable soldering to the back side :stuck_out_tongue_winking_eye:

https://www.amazon.com/GenBasic-Solderless-Dupont-Compatible-Breadboard-Prototyping/dp/B01L5ULRUA
Might be cheaper in smaller quantity on ebay.

But yeah try to figure out how to change the gain first. (edit /etc/default/dump1090-fa)
I linked you a post where the different methods needed depending on the software in use are explained.

Whoa!! Finally got the gain down and Yea, i think this is much better: pi@raspberrypi:~ sudo systemctl restart dump1090-fapi@raspberrypi:~ sudo systemctl status dump1090-fa -l
● dump1090-fa.service - dump1090 ADS-B receiver (FlightAware customization)
Loaded: loaded (/lib/systemd/system/dump1090-fa.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2019-07-05 18:47:13 EDT; 2s ago
Docs: https://flightaware.com/adsb/piaware/
Main PID: 1089 (dump1090-fa)
CGroup: /system.slice/dump1090-fa.service
└─1089 /usr/bin/dump1090-fa --device-index 0 --gain 40 --ppm 0 --net-bo-port 30005 --max-range 360 --net --net

Jul 05 18:47:13 raspberrypi systemd[1]: Started dump1090 ADS-B receiver (FlightAware customization).
Jul 05 18:47:13 raspberrypi dump1090-fa[1089]: Fri Jul 5 18:47:13 2019 EDT dump1090-fa 3.7.1 starting up.
Jul 05 18:47:13 raspberrypi dump1090-fa[1089]: rtlsdr: using device #0: Generic RTL2832U (Realtek, RTL2832U, SN 00001000)
Jul 05 18:47:14 raspberrypi dump1090-fa[1089]: Found Rafael Micro R820T tuner
Jul 05 18:47:14 raspberrypi dump1090-fa[1089]: rtlsdr: tuner gain set to 40.2 dB

pi@raspberrypi:~ awk "(cat /run/dump1090*/stats.json| grep total | sed ‘s/.accepted":[([0-9]).strong_signals":([0-9]).*/BEGIN {printf “\nPercentage of strong messages: %.3f \n” , \2 * 100 / \1}/’)"

Percentage of strong messages: 1.444

I can probably even raise my gain a little… but from what i can see, im getting a lot more planes within the 10nm range around the house now.

1 Like

I have updated the script to generate a performance graph - it now includes colouring points by range, and some box plots to show data distribution. The data displayed is over the previous week.

Wiedehopf has updated the github file, so the command to generate it remains unchanged:

sudo bash -c "$(wget -q -O - https://raw.githubusercontent.com/wiedehopf/adsb-wiki/master/caius_scatter_plot.sh)"
4 Likes

Pretty. I have no interpretation but it’s pretty.

.

Tried on my Pi, but got this error

pi@raspberrypi:~ $ sudo bash -c "$(wget -q -O - https://raw.githubusercontent.com/wiedehopf/adsb-wiki/master/caius_scatter_plot.sh)"
Processing RRD data
Generating plot
bash: line 39: gnuplot: command not found
Moving plot
cp: cannot stat '/tmp/range.png': No such file or directory

EDIT:
Installing gnuplot
Taking long time

Do this first

sudo apt install gnuplot-nox --no-install-recommends

Sorry, too late, already done sudo apt install gnuplot

Ran bash script, and got following output. Where is the plot (image) located?

sudo apt install gnuplot 

sudo bash -c "$(wget -q -O - https://raw.githubusercontent.com/wiedehopf/adsb-wiki/master/caius_scatter_plot.sh)".... .... ....
1230 1.3447854189e+04  -1.20e+07  1.23e-03   -3.606918e-01   8.626150e+01  -6.107151e+03   6.141043e+03
1231 1.3447854189e+04  -1.49e+00  1.23e-04   -3.606918e-01   8.626150e+01  -6.107151e+03   6.141043e+03
1232 1.3447854189e+04  -2.71e-01  1.23e-05   -3.606918e-01   8.626150e+01  -6.107151e+03   6.141043e+03
iter      chisq       delta/lim  lambda   a             b             c             d

After 1232 iterations the fit converged.
final sum of squares of residuals : 13447.9
rel. change during last iteration : -2.70525e-15

degrees of freedom    (FIT_NDF)                        : 72
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 13.6666
variance of residuals (reduced chisquare) = WSSR/ndf   : 186.776

Final set of parameters            Asymptotic Standard Error
=======================            ==========================
a               = -0.360692        +/- 0.2496       (69.2%)
b               = 86.2615          +/- 53.44        (61.95%)
c               = -6107.15         +/- 3707         (60.7%)
d               = 6141.04          +/- 807.7        (13.15%)

correlation matrix of the fit parameters:
                a      b      c      d
a               1.000
b              -0.996  1.000
c               0.983 -0.995  1.000
d               0.515 -0.437  0.347  1.000
Moving plot

http://yourlocapiipaddress/dump1090-fa/data/graph.png

1 Like

Plot 1/2 - With PCB Antenna (Indoor, near window) + FA Filter (sky blue) + ProStick (orange)

After 1232 iterations the fit converged.

Plot 2/2 - With Flightaware Antenna (Indoor, near window) + FA Filter (sky blue) + ProStickPlus (navy blue)

After 9 iterations the fit converged.

Ah, you appear to have uncovered a problem with the label positioning. I’ll have to have a look and see if I can get gnuplot to use relative positions rather than absolute.

Edit - this should be possible - I just have to figure out the correct format and update it.