Signal Strength Heatmap

[caius]

I have written a script that captures data from dump1090-fa and plots a heatmap showing signal strength for each aircraft position. This example is plotted from about 15 hours of data.

The script samples currently visible aircraft from the aircraft.json file and records signal strength and position to a file. Gnuplot is then used to plot the data.

The plot is fairly naive, in that it ignores proper map projections so bearings won’t be quite correct, however it should give a reasonable representation of what is going on. My antenna is in the loft space, so you can clearly see radio shadows caused by the brick walls and roof structure.

The script is available here:

https://raw.githubusercontent.com/caiusseverus/adsbcompare/master/heatmap.sh

Note that you should set your receiver latitude and longitude at the top of the script in order for its position to be marked on the plot.

Usage is simple - there are two options when running the command. The first sets the duration of data gathering in minutes, and the second sets the sample interval in seconds.

Example:

./heatmap.sh 1440 10

This will collect data for 24 hours, every 10 seconds. Obviously the shorter the interval, the more data points you get but the more storage it will use.

The example above was sampled every 3 seconds for 15 hours or so, giving a total of about 2 million data points. Large numbers aren’t a problem for gnuplot, it just takes a little while to crunch through it. The data file was about 50MB.

If you want to run the script for longer periods and not worry about having to stay logged in for it to complete, you can do this:

nohup ./heatmap.sh 1440 10 &

which will start the script running in the background and leave it running if you log out.

The collected data is saved in a file in the current directory called heatmap-<date>
and the plot is also saved with the current date, but also copied to the dump1090 folder so it can be seen/saved from your browser at:
http://raspberrypi/dump1090-fa/data/heatmap.png

You need to have gnuplot, and jq (think it is by default on raspbian) installed for this to work.

Scripts related to this are kept in this repository: GitHub - caiusseverus/adsbcompare: Tools to plot and compare data gathered with graphs1090

Edit:

Thanks to his modification, if you are running timelapse1090 by @wiedehopf you can use the data already collected by that to produce plots without waiting to accumulate more data:

bash heatmap.sh -1

There is now a version that produces plots based on range and bearing, and also produces additional azimuth/elevation and range/altitude plots:

These can be produced with the script:

https://raw.githubusercontent.com/caiusseverus/adsbcompare/master/polar.sh

Usage and options are exactly the same as for heatmap.sh, however you need to have gawk and gnuplot 5.2 installed. Raspbian stretch users will need to do:

sudo apt install gawk
sudo apt install liblua5.3
curl -sO http://ftp.debian.org/debian/pool/main/g/gnuplot/gnuplot-data_5.2.6+dfsg1-3_all.deb
curl -sO http://ftp.debian.org/debian/pool/main/g/gnuplot/gnuplot-nox_5.2.6+dfsg1-3_armhf.deb
sudo dpkg -i ./gnuplot-data_5.2.6+dfsg1-3_all.deb
sudo dpkg -i ./gnuplot-nox_5.2.6+dfsg1-3_armhf.deb

Note that this involves doing a lot of calculations and so it takes some time to produce output ~15 minutes or so on a Pi 3 for a full 24 hours of data.

[wiedehopf]

@caius

The script samples currently visible aircraft from the aircraft.json file and records signal strength and position to a file.

How about restricting the planes to a certain altitude range?
(alt_baro is the relevant json entry for each aircraft)

Very nicely done!

[caius]

@wiedehopf

How about restricting the planes to a certain altitude range?
(alt_baro is the relevant json entry for each aircraft)

That’s a good idea. I’ll probably do it by capturing all the data and filtering it later. I also want to switch the plot to use polar coordinates, since it makes adding things like range rings easier.

[bramjacobse]

1 hour capture 10 sec interval

[modea]

@caius

heatmap.png

Awesome - Thank you for making this! Ran it for about an hour just to see what it did

[Dav5049915]

This tool is gonna help me figure out what trees need to uhhhh fall down in the next storm

can’t wait to try it when I get home!

[caius]

@modea

Awesome - Thank you for making this! Ran it for about an hour just to see what it did

At the moment the RSSI scale is fixed. Since most of your signals seems to be pretty strong (you are using an Airspy?) you might want to change it to get a better spread of colours. Look for this line:

set cbrange [-35:0]

and change the left number to something close to what your weakest signals are.

At some point I’ll change the script to pick a range automatically.

[modea]

in this instance I’m using FA Blue stick + GPIOlabs pre-amp. I’ll dial it in now and let it run again.

[Dav5049915]

1hr plot at 10sec with an airspy + LNA

This is AWESOME. Gonna run it for 24hrs and see what I get for dead zones.

[obj]

Another thing that can be interesting is to do the same thing but with azimuth/elevation axes.

Here’s a really old example:

[idoleo2001]

Is there any way to load “aircraft.json” get from this script (Timelapse1090) inside Heatmap script?

Thanks a lot!

[wiedehopf]

@idoleo2001

Is there any way to load “aircraft.json” get from this script (Timelapse1090)

Actually there is

You can use this modified version of the script:

https://raw.githubusercontent.com/wiedehopf/adsbcompare/use-timelapse-data/heatmap.sh

Invoke it like this:

bash heatmap.sh -1

[bramjacobse]

Heatmap

[triggers]

Lovely pieces of artwork to put on your wall:laughing:

Thanks Caius

[caius]

@wiedehopf

You can use this modified version of the script:

Nice. I’ve updated the original script.

[wiedehopf]

set terminal pngcairo enhanced size 2400,2000

set style fill transparent solid 0.3 noborder
set style circle radius 0.006

plot 'heatmap' u ($1):($2):($3) with circles lc palette

0 ft to 5000 ft:

5001 ft to 15000 ft:

15001 ft to 25000 ft:

25001 ft to 35000 ft:

35001 ft to 45000 ft:

[caius]

Where are you selecting the altitude - Are you filtering the file before, or doing it in gnuplot? I have a version that does it in gnuplot. I’ll post when I get home.

[wiedehopf]

@caius

I have a version that does it in gnuplot.

That’s probably better, i filtered with jq, but it’s slow to run that multiple times
Also didn’t fix the plot range so it stays consistent.

(I copied the data over to my laptop, it runs a little faster here, also i don’t want airspy_adsb to start dropping samples, which it does with heavy memory usage)

[abcd567]

5-Minutes Run

nohup ./heatmap.sh 5 10 &

[caius]

@wiedehopf

set style fill transparent solid 0.3 noborder 
set style circle radius 0.006

This increases memory usage by gnuplot a considerable amount - when filtering altitude in gnuplot the pi ran out of memory and rebooted. It runs fine on the same data with dots.

I compared filtering in gnuplot and awk, and it’s a bit faster with awk and more convenient, since gnuplot’s conditional plotting is a bit harder to get to do what you want.

I’ve updated the script on github - set the high and low altitude cut off at the top.

The low altitude one has some spurious data points in it which are a pain to remove. It will be easier to set range to remove those once it’s switched to polar plotting.