Signal Strength Heatmap

[Coooogz]

Installed this on a new device yesterday and had to trouble shoot a few issues that I had.
Here is my updated howto… Please shout if I have advices incorrect and/or something doesn’t work for you.

Step 1:
If not already running, install timelapse1090. More info: GitHub - wiedehopf/timelapse1090: timelapse web interface for dump1090-fa (using the map interface from dump1090-fa as base)

sudo bash -c "$(wget -q -O - https://raw.githubusercontent.com/wiedehopf/timelapse1090/master/install.sh)"

(not sure this is needed but when I ran the polar.sh script I had a few issues with it not picking up data from timelapse. EDIT: I had to start timelapse but no other issues now.

sudo systemctl start timelapse1090
sudo systemctl status timelapse1090

(One of the errors was “folder not found” either when starting timelapse1090 or further down when running polar.sh, I just created that folder, “/run/timelapse1090”) EDIT: On a clean run on a new device I did not get this error.

Step2:
Download the script:

wget https://raw.githubusercontent.com/caiusseverus/adsbcompare/master/polar.sh

Step 3:
Make sure you have a “heywhatsthat” ID to hand for you location.
Then edit polar.sh before you run it or polar.conf if you have run the script once already.

Step 4:
Install the required software:

sudo apt install gawk
sudo apt install liblua5.3
sudo apt install gnuplot-data
sudo apt install gnuplot-nox
sudo apt install jq

Step 5:
Now run:

bash polar.sh -1

Once complete, they should be at /skyaware/plots/

(timelapse1090 by default collects data per day and then resets. So it is best to run this script say 15 minutes before midnight each night, to display max amount of positions.

For the monthplot I did:

Step 1:

sudo apt install python3-seaborn

Step 2:

wget https://raw.githubusercontent.com/caiusseverus/adsbcompare/master/monthplot.py

Step 3:
Give permission to write to /usr/share/skyaware/html/plots/ (google this one)

Step 4:

python3 monthplot.py

Once complete, they should be at /skyaware/plots/heatmaps.html

Add to crontab to run automagically.
Step 1:

crontab -e

Step 2:
(add the lines at the end and of course test it works first with a more suitable time)

44 23 * * * bash /home/pi/polar.sh -1
05 00 * * * python3 monthplot.py

[wiedehopf]

@Coooogz

(timelapse1090 by default collects data per day and then resets. So it is best to run this script say 15 minutes before midnight each night, to display max amount of positions.

It doesn’t reset, it’s rolling over the data file so always has the last 24h in /run/timelapse1090 if it’s been running at least that long.

[wiedehopf]

@Coooogz

(not sure this is needed but when I ran the polar.sh script I had a few issues with it not picking up data from timelapse.

@Coooogz

(One of the errors was “folder not found” either when starting timelapse1090 or further down when running polar.sh, I just created that folder, “/run/timelapse1090”)

Really if the folder is not there, there can’t be any data collected by timelapse1090.
If you have issues with timelapse1090 not running … check the journal for that and debug it:

sudo journalctl -e -u timelapse1090 | tail -n50

Feel free to put it on pastebin if there are errors and you can’t figure it out.

[Coooogz]

Thanks, I’ve just helped someone install this and did not get this issue again. I think it was the fact that I had to start timelapse after the install and perhaps I had not started it yet with the previous install.

[jepolch]

@Coooogz

somehow the quotations dont work when pasted from here

Wrong quotation marks. “ and ” should be " and ".

[Coooogz]

Thanks for the comments btw, I’ve edited some of my post above so the howto should be more accurate now?

I set this up yesterday for someone, and they are getting some “black lines” on their graphs… Some sort of interference I assume? Am I correct or what else could it be?
Aerial is outside and clear view to the sky in the direction of this phenomena. Si it isn’t loft joists etc.

I believe they do NOT have a filter in place.

I guess next step is to get them to look at “Do i need a filter” thread?

[wiedehopf]

@Coooogz

I set this up yesterday for someone, and they are getting some “black lines” on their graphs… Some sort of interference I assume? Am I correct or what else could it be?

It’s “stuff” in the way.
Looks like trees, don’t you think

[Coooogz]

@wiedehopf

It’s “stuff” in the way.
Looks like trees, don’t you think

I don’t know the location and have not seen a photo of the horizon but looking google streetview I cant see any obvious obstructions nearby or in the distance. The “lines” of bad signal seem toop regular to be a tree in my opinion. I’ve marked a blob in green that might be a tree? then the lines with red lines… and the bad signals in the yellow circle are way off in the distance and red because they are far away (200nm+). The yellow lines are flight paths say 150nm away. (hard to picture this all in 3d)

See below for the actual aerial mounting. The “brick” chimney is actually fiberglass. I am not sure what’s inside, but the interference lines are coming from the right of this picture… So the chimney should not be a direct factor, however is there a chance the signal is bouncing back off the chimney and creating something like a moiré pattern?

How much issue will the slight slant in the aerial be causing?

[caius]

I think that kind of banding is possibly a result of diffraction. I’ve seen it appear on the elevation plots on a few different receivers where there isn’t anything obviously in the way.

@keithma gets it in his elevation plot in a direction that is directly out to sea:

[Coooogz]

Ah yes … diffraction was the word I was looking for… I googled and found “Moire affect” … and stuck with that … I see keithma’s are vertical banding? would these horizontal ones be caused by the chimney bouncing the signals back possibly?

[caius]

It could be caused by all sorts of things. Could be physical obstacles or maybe something like atmospheric lensing.

[keithma]

MTG has a pretty criss-cross pattern

[Coooogz]

Ah wow that’s interesting… is that from the signals bouncing off the sea?

[wiedehopf]

@caius

Could be physical obstacles or maybe something like atmospheric lensing.

Multipath can cause issues.
So maybe reflections?

[caius]

Something new to play with for those using airspy receivers:

This script generates some stats and plots from graphs1090 data.

This graph plots range against total aircraft, categorised by gain.

Signal to Noise Ratio against total aircraft. There are traces for minimum, 5%, 25%, 50%, 75%, 95% and maximum values, as recorded on the graphs1090 plot.

The same data, but categorised by gain using the same colours as on the graphs1090 plot.

The range of signal to noise ratio for each gain (ie maximum - minimum values).

The preceding graphs are intended to help identify whether the gain setting is appropriate. Too low a gain could compromise maximum range, and too high can reduce SNR. In my example data you can see that there is very little variation in maximum range between the gains selected by the autogain algorithm.

This graph shows the minimum noise values, which approximate to the noise floor of the receiver:

The values here should normally increase in approximately linearly with each gain step. The data shown here is atypical due to the changes in rssi measurement in the recent decoder updates.

This graph shows the average number of DF17 messages received for each ads-b aircraft, plotted against the total number of aircraft (including mode-s only/mlat aircraft).

As traffic increases, more messages are garbled leading to fewer decoded per aircraft.

This graph shows the relationship between the total number of messages and the preamble filter:

It’s plotted with messages on the x-axis since most people are using the CPU target option, which means the preamble filter will react to traffic levels. The three distinct groupings in this example are due to decoder changes.

Finally, a summary of how the reception rate of the various message types react to overall traffic:

How to install and use:

The script is written in python and uses seaborn and pandas, so you need those installed. It also assumes you are using graphs1090.

sudo apt install python3-pip libatlas-base-dev
sudo pip3 install --upgrade pip

then install pandas and seaborn:

pip3 install pandas seaborn

download the script:

wget https://raw.githubusercontent.com/caiusseverus/adsbcompare/master/airspystats.py

Create the output directory:

sudo mkdir /usr/local/share/tar1090/html/stats/
sudo chmod o+rw /usr/local/share/tar1090/html/stats/

The tar1090 web directory is used by default as it’s a convenient place that’s already available via the web server. If you want it elsewhere, just edit the outdir variable near the top of the script.

To run the script:
python3 airspystats.py

Once finished, you should be able to see the output by going to the url on your pi.
For example:

http://adsbpi/tar1090/stats/

A note on interpreting these graphs:

You need to be a bit careful drawing conclusions from these graphs unless you have considered what it is they are telling you. Most of them are categorised by gain. Since most people are now using the auto gain option, that makes gain a dependent variable. It changes with conditions, so you could for example see something like a lower SNR range for a particular gain setting, as is apparent in the examples above for gain 18. At first glance, it might appear that means that 18 is performing worse than the others, but that’s not necessarily the case.

Looking at when that gain is used shows that it is predominantly where there is minimal traffic. That makes the data much more noisy, and therefore less indicative of performance. To use these graphs more diagnostically would require controlling the gain so that it does not change dependent on traffic.

Anyway, maybe it’s useful to someone. It can at least make it easier to see how things change rather than having to extrapolate from a time series graph.

[Coooogz]

I get the following when running the script after following the steps as above… any idea?

$ python3 airspystats.py
Output directory exists
Output directory writable
Getting data…
Processing…
Traceback (most recent call last):
File “airspystats.py”, line 193, in
sns.set_theme(style=“darkgrid”)
AttributeError: module ‘seaborn’ has no attribute ‘set_theme’

Edit: Looks like it relates to an older version of Seaborn being installed. 0.9.0 in my case. Let me look at upgrading.

[caius]

Yeah was going to say, it’s probably an old version of seaborn. Did you install it previously using apt? If so it’s probably worth removing it and installing using pip instead.

Edit - It probably is that:

pi@adsbpi:~ $ apt policy python3-seaborn
python3-seaborn:
  Installed: (none)
  Candidate: 0.9.0-1
  Version table:
     0.9.0-1 500
        500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

Raspbian has ancient packages for a lot of things.

Probably best to do the same with numpy as well.

[Coooogz]

install via apt… Indeed I did…

pip3 install seaborn==0.11.2 sorted things for me…

[xsprinterx]

@caius

sudo chmod o+rw /usr/local/share/tar1090/html/stats/

Hi, followed the instructions, but get this when trying to run.
pi@Flightp1:~ $ python3 airspystats.py
Traceback (most recent call last):
File “airspystats.py”, line 2, in
import pandas as pd
File “/home/pi/.local/lib/python3.7/site-packages/pandas/init.py”, line 17, in
“Unable to import required dependencies:\n” + “\n”.join(missing_dependencies)
ImportError: Unable to import required dependencies:
numpy:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

• The Python version is: Python3.7 from “/usr/bin/python3”
• The NumPy version is: “1.21.2”

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory

[bramjacobse]