FlightAware Discussions

Thoughts on optimizing gain

$(date -I --date=yesterday)

I’ll check what -4 days for example would be.

$(date -I --date='-4 day')

If you want to translate -4 and -7 as arguments to dates:

$(date -I --date="$1 day")

So $1 is the first argument, you’d have to use -4 or another negative number.
This of course simplifies the input.

1 Like

Thanks, I’ll have a play.

I did it slightly differently in the end - cron calls a script which pulls the data from rrd then processes it and saves it in a single file that gnuplot can use:

#!/bin/bash

# Fetch daily data from rrds

rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-local_accepted.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /run/messages
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_aircraft-recent.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /run/aircraft

# Remove headers and extraneous :

sed -i 's/://' /run/messages
sed -i 's/://' /run/aircraft

sed -i -e '1d;2d' /run/messages
sed -i -e '1d;2d' /run/aircraft

# Combine both files to create space separated data file for use by gnuplot

join -o 1.2 2.2 /run/aircraft /run/messages > /var/cache/fetch/$(date -I --date=yesterday)-scatter

This is called a few minutes after midnight and names the file for the previous complete 24 hour day. I set it to use a resolution of 3 minutes, which gives a smaller number of points to plot.

Currently I have a gnuplot script that just generates a graph for the last complete day:

date = system("date -I --date=yesterday")
set terminal pngcairo enhanced size 1280,1024
set output './'.date.'-adsb.png'
set title 'Message rate compared to aircraft seen'
set xlabel 'Aircraft'
set ylabel 'Messages rate /s'
set grid xtics ytics
f(x) = a*x**2 + b*x + c
c = 0
fit f(x) '/var/cache/fetch/'.date.'-scatter' via a,b
plot '/var/cache/fetch/'.date.'-scatter' notitle, f(x) notitle

Running “gnuplot adsb.plot” generates the .png file:

Once I’ve collected a bit more data I’ll add some comparisons to it. I’m not sure a quadratic regression is ideal for this, but it seems to fit the data quite well for an RTL dongle in a busy area. It might not fit as well for an airspy.

2 Likes

On installing gnuplot:

Use the package gnuplot-nox if you want to save yourself from all the wayland and X11 (graphical UI) packages.

For a minimal installation do this:

sudo apt install gnuplot-nox --no-install-recommends

Note that you might not have all features though not installing the recommended packages.
Cuts down the number of packages installed though.

1 Like

I don’t see as many aircraft, also needed to modify the script to get the remote instead of local messages (airspy).

Data is probably a bit more scattered than usual, was messing with the system quite a bit yesterday.

Alternative function for the fit.

f(x) = a*x**2 + b*x + c + d*log(x)
c = 0
fit f(x) '/var/cache/fetch/'.date.'-scatter' via a,b,d

2 Likes

You may have more than one effect going on here.

  1. SSRs have a time budget for interrogations; above a certain number of aircraft, this budget will limit the interrogation rate per aircraft (& therefore response rate)

  2. IIRC TCAS dials back transmit power when it sees a congested environment, so again fewer replies per aircraft as the effective interrogation range is lower.

It might be interesting to plot only ADS-B messages, since the transmit rate of those messages should be mostly unaffected by the environment. I don’t think that’s a statistic that’s currently collected though.

1 Like

I am within range of Heathrow, so it’s very busy here and I have a lot of nearby traffic. There is a noticeable difference in message rate in the early morning as trans-Atlantic flights start to arrive before the local airports start operating, and later in the day.

The message rate is noticeably higher in this early period than it is later on in the day, but with a similar number of aircraft visible. This is quite likely due to the factors you mentioned - in the morning, received aircraft are mostly at high altitude and spread out across the country, whereas later the majority are within 50 miles and at lower altitudes.

There is definitely an impact on range with increased message traffic though - I see a marked decrease in the maximum during peak times compared with when it’s quieter.

It’s interesting to see wiedehopf’s airspy is much more consistent with fewer aircraft, but has the same increased distribution at higher traffic levels, probably due to the effects you describe.

AMS - AirSpy mini

1 minute interval

Can you try pasting this on the command line?

#!/bin/bash

# Fetch daily data from rrds

rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-local_accepted.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/messages_l
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_messages-remote_accepted.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/messages_r
rrdtool fetch /var/lib/collectd/rrd/localhost/dump1090-localhost/dump1090_aircraft-recent.rrd AVERAGE -e midnight today -s end-1day -r 3m -a > /tmp/aircraft

# Remove headers and extraneous :

sed -i 's/://' /tmp/messages_l
sed -i 's/://' /tmp/messages_r
sed -i 's/://' /tmp/aircraft

sed -i -e '1d;2d' /tmp/messages_l
sed -i -e '1d;2d' /tmp/messages_r
sed -i -e '1d;2d' /tmp/aircraft

# Combine both files to create space separated data file for use by gnuplot

join -o 1.1 1.2 2.2 /tmp/aircraft /tmp/messages_l > /tmp/tmp
join -o 1.2 1.3 2.2 /tmp/tmp /tmp/messages_r > /tmp/$(date -I --date=yesterday)-scatter

cd /tmp

gnuplot /dev/stdin <<"EOF"
date = system("date -I --date=yesterday")
set terminal pngcairo enhanced size 1280,1024
set output '/tmp/'.date.'-adsb.png'
set datafile separator " "
set title 'Message rate compared to aircraft seen'
set xlabel 'Aircraft'
set ylabel 'Messages rate /s'
set grid xtics ytics
f(x) = c*x/sqrt(d+x**2) + a*x**2 +b*x
c=4000
d=7000
a=0.05
b=-20
fit f(x) '/tmp/'.date.'-scatter' using 1:($2+$3) via a,b,c,d
plot '/tmp/'.date.'-scatter' using 1:($2+$3), f(x) notitle
EOF

I’ve adapted the fit function a little because your data doesn’t really fit the downward trend at the end.

The png will be located in /tmp

Edit: replaced code with better fit function.

Edit2: made the code actually work -.-

1 Like

Hmm interesting, that didn’t change the fit much at all.

Curious.

Edit: incorporated the code changes in the code two posts above

1 Like

That looks like a more reasonable fit.

I’ll work on adding remote and local messages.
That way one script will work regardless of receiver.
Edit:

Whoever wants to plot this graph, first install gnuplot:

sudo apt install gnuplot-nox --no-install-recommends

Then just use this command:

sudo bash -c "$(wget -q -O - https://raw.githubusercontent.com/wiedehopf/adsb-wiki/master/caius_scatter_plot.sh)"

The graph will be available via web, replace pi with the ip-address of your RPi:
http://pi/dump1090-fa/data/graph.png

2 Likes

pls advise when you have added remote and local messages options (for airspy) and i will install

No install available and not really planned.

The post above yours will work regardless of receiver.

Edit: Forgot to mention the script only runs once to create one graph.

I just tried. No airport close to me and using a fa pro stick.

I just changed the script to 1 week, makes for more consistent data and remove day to day variations somewhat:

You probably made a mistake by saying “whoever wants to plot this graph…” – because somebody like me comes along, somebody who hasn’t followed every procedure in the thread prior to this.

I installed gnuplot-nox using your command. It initially failed because it couldn’t find something, and of course apt-get update (or whatever the command was) fixed that. I was then able to install gnuplot-nox.

Then I tried to run your script and it said rrdtool wasn’t installed. so I did “sudo apt-get install rrdtool”. That completed fine.

Then I ran the command (the one with wget in it) and I get:

ERROR: unknown option ‘-a’
ERROR: unknown option ‘-a’
ERROR: unknown option ‘-a’
Read 0 points
No data to fit
“/dev/stdin”, line 14:

Any ideas?

Mike

It requires my graphs
Graphs for dump1090 -- my version with install script

Also works with adsb-receiver project graphs, they are similar.

Okay here for one week

1 Like