It seems to me that these two awk commands are creating the heatmap_low and heatmap_high files in in $TMPDIR instead of $wdir.
Plot is then later expecting them in wdir.
awk -v low="$low" -F "," '$4 <= low' $wdir/heatmap > $TMPDIR/heatmap_low
awk -v high="$high" -F "," '$4 >= high' $wdir/heatmap > $TMPDIR/heatmap_high
Will test again ā¦
And working
Well spotted - the error wasnāt occurring for me because Iām running it on a system with more ram.
OK Iāve uploaded a new version of the script that should fix the empty upintheair file problem and correct the working directory location. Let me know if thereās any more problems.
Iāve also added a check that a HWT id has been entered before the script will run. Should stop people having to wait for all the data processing stuff before they realise.
I happen to live too far north for upintheair to work (they cover up to 60 degrees, I live at almost 61 degrees north), is there an easy modification that can be made to the script to make it work?
Iām in Spain until Monday so I canāt really look at it until I get back. You can remove the check at the beginning, and the plots from the gnuplot section as those will cause the script to fail. The rest should still work, but you might get some error messages. Once Iām home Iāll modify it to work around the problem.
Great! Iāll try that.
Have a nice time in spain 
Some help with this error:
pi@piaware:~ $ sudo ./polar.sh 1 10
HWT OK
Using disk : /home/pi
60
Gathering data every 10 seconds until Sun Aug 25 12:50:46 EDT 2019
Number of data points collected: 756
Calculating Range, Azimuth and Elevation data:
awk: not an option: -i
Filtering altitudes
Processing heywhatsthat.com data:
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
awk: not an option: -i
awk: not an option: -i
awk: not an option: -i
set theta clockwise top
^
ā/dev/stdinā, line 26: unrecognized option - see āhelp setā.
rm: cannot remove ā/home/pi/heatmapā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_lowā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_highā: No such file or directory
cp: cannot stat āpolarheatmap_low-2019-08-25.pngā: No such file or directory
cp: cannot stat āpolarheatmap_high-2019-08-25.pngā: No such file or directory
cp: cannot stat āelevation-2019-08-25.pngā: No such file or directory
cp: cannot stat āaltgraph-2019-08-25.pngā: No such file or directory
cp: cannot stat āclosealt-2019-08-25.pngā: No such file or directory
Graphs available at :
http://10.11.11.28/dump1090-fa/data/plots.html
Graphs rendered in 62 seconds
There are 2 problems here, both probably caused because you have a different or earlier OS:
The first is that you need gawk and probably have mawk or some other version. This is easily fixable - do:
sudo apt install gawk
The second is because you need gnuplot version 5.2 to get the set theta option. This is the default version in Buster, but Stretch only has 5.0. Someone who previously had this issue fixed by manually installing these packages:
gnuplot-data
gnuplot-nox
libblua5.3
Doing this should resolve it:
sudo apt install liblua5.3
curl -sO http://ftp.debian.org/debian/pool/main/g/gnuplot/gnuplot-data_5.2.6+dfsg1-3_all.deb
curl -sO http://ftp.debian.org/debian/pool/main/g/gnuplot/gnuplot-nox_5.2.6+dfsg1-3_armhf.deb
sudo dpkg -i ./gnuplot-data_5.2.6+dfsg1-3_all.deb
sudo dpkg -i ./gnuplot-nox_5.2.6+dfsg1-3_armhf.deb
Upgrading to Buster will also fix it, however if you are running the piaware image, itās probably best not to do that until they release and update image officially.
Have the packages upgraded and installed.
Have this now.
pi@piaware:~ $ sudo ./polar.sh 1 10
HWT OK
Using disk : /home/pi
60
Gathering data every 10 seconds until Sun Aug 25 13:35:49 EDT 2019
Number of data points collected: 877
Calculating Range, Azimuth and Elevation data:
Filtering altitudes
Processing heywhatsthat.com data:
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
jq: error (at upintheair.json:1): Cannot iterate over null (null)
ā/dev/stdinā line 33: Non-numeric string found where a numeric expression was expected
rm: cannot remove ā/home/pi/heatmapā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_lowā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_highā: No such file or directory
cp: cannot stat āpolarheatmap_low-2019-08-25.pngā: No such file or directory
cp: cannot stat āpolarheatmap_high-2019-08-25.pngā: No such file or directory
cp: cannot stat āelevation-2019-08-25.pngā: No such file or directory
cp: cannot stat āaltgraph-2019-08-25.pngā: No such file or directory
cp: cannot stat āclosealt-2019-08-25.pngā: No such file or directory
Graphs available at :
http://raspberrypi/dump1090-fa/data/plots.html
Graphs rendered in 62 seconds
I do have a Pi4 arriving on Tuesday.
Currently running a 3B+ with piaware image on Stretch.
On the Pi4 I plan on installing Buster Lite, piaware, dump1090-fa.
And the Airspy files for my Airspy mini.
Does the upintheair.json file have sensible data in it? You could try deleting it and letting it download again - the script should remove an empty one if it exists - also double check that the heywhatsthat id is correct. The error implies that the file doesnāt contain any data.
Made a new panorama and downloaded it.
13k file size.
There is data in the file.
And the terrain rings display on tar1090 map display.
This seems too small to me - it should be about 270k. Are you copying the upintheair.json file to the script directory manually? If so, you shouldnāt do that - add the ID of the panorama to the field at the top of the script and let the script download it itself. It gets more data than the one you would normally use to display on the map.
Iāve uploaded a new version that fixes an unrelated bug, so you should update before you test again, and delete any existing upintheair.json prior to running.
Much better.
I did not realize the script would download the panorama info.
I was using this line to download the file:
sudo wget -O upintheair.json āhttp://www.heywhatsthat.com/api/upintheair.json?id=XXXXXXXX&refraction=0.25&alts=ā
( I did replace the "X"s with my ID )
Now just a couple of errors.
Mainly files that cannot be removed and a ā/dev/stdinā line error.
pi@piaware:~ $ sudo ./polar.sh 1 10
HWT OK
Using disk : /home/pi
60
Gathering data every 10 seconds until Sun Aug 25 15:22:43 EDT 2019
Number of data points collected: 905
Calculating Range, Azimuth and Elevation data:
Filtering altitudes
Processing heywhatsthat.com data:
Retrieving terrain profiles from heywhatsthat.com:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 258k 0 258k 0 0 59662 0 --:ā:-- 0:00:04 --:ā:-- 59680
Generating all altitudes heatmapā¦
Generating high altitude heatmapā¦
Generating low altitude heatmapā¦
Generating close range heatmap
Generating elevation heatmapā¦
ā/dev/stdinā line 100: warning: Cannot find or open file ā/tmp/tmp.Kne2uci6D4/50ā
Generating Range/Altitude plotā¦
Generating Close Range altitude plot
rm: cannot remove ā/home/pi/heatmapā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_lowā: No such file or directory
rm: cannot remove ā/home/pi/heatmap_highā: No such file or directory
Graphs available at :
http://piaware/dump1090-fa/data/plots.html
Graphs rendered in 75 seconds
That looks more promising, and those errors should be fixed in the version I just uploaded - give it a few minutes then download the script again.
New script.
Are these errors anything to worry about ?
Generating low altitude heatmapā¦
ā/dev/stdinā line 65: warning: Skipping data file with no valid points
ā/dev/stdinā line 65: warning: resetting rrange
Generating close range heatmap
ā/dev/stdinā line 76: warning: Skipping data file with no valid points
Generating elevation heatmapā¦
ā/dev/stdinā line 101: warning: Skipping data file with no valid points
If you only gathered data for a short period, itās possible that there is no valid data for those graphs available - those lines use a data file filtered to contain only aircraft below the low altitude limit set at the top of the script (default is 5000 feet). If you saw no aircraft below that altitude in the period you gathered data for, then the file will be empty and gnuplot will complain about it.
The other plots should have some output on them. If you just want to confirm the correct functioning of the script, you can temporarily raise the low altitude above the lowest altitude aircraft you are currently receiving.
Edit - the last error is complaining about the heywhatsthat data used to generate the horizon - check that you have your receiver altitude set correctly at the top of the script (should be in meters above sea level) and try deleting the existing upintheair.json to force it to re-download.