Oh that’s your link
https://jqplay.org/s/NUiW-uwOgp
this one is with 978 json
Thanks. Just running the script to test it then I’ll upload - I’ve added the map to it as well.
OK new version is uploaded with the filter fix for skyaware.
I’ve added plots with a map overlay - it should work for coastlines globally. There aren’t any country borders available in the map data unfortunately. I will probably separate out the plotting for these so that I can make them switchable for those who are not near any coast to save a bit of processing time.
The script will download the map data and process it on first running, and thereafter use the cached data in the file world_10m.txt. Deleting this will cause the script to get the data again on the next run - it’s static data however so is unlikely to change.
Example plots:
3 Likes
My script updated before it ran at 23:30 last night and I have a world_10m.txt file but I don’t see the map overlay (here). Does it need to be enabled?
1 Like
Lol you have the map overlay ! Scroll down ;+)
1 Like
Doh! I was expecting it on all the plots, I didn’t scroll all the way to the bottom.
1 Like
Post deleted : problem solved ;+)
Love your colours! Care to share?
1 Like
My tinkering. I set Altitude limits to
#Set altitude limits
low=25000
high=25000
Set color range to
set cbrange [0:10000]
Everything between 10000 and 25000 is blue…
Lower then 10000 it uses the color range.
set palette rgb 10,13,33
Thanks, I’ll have a tinker too 
I’ve finally got round to tidying up and finishing the scripts that produce the timelapse video that I posted some time ago. There are three versions - one produces a top down map, one produces an azimuth/elevation view, and one combines both:
The scripts can be found here:
These have similar dependencies to the polar script, but also require ffmpeg to create the output:
sudo apt install ffmpeg
should work on raspbian/ubuntu.
These scripts also only work with data from timelapse1090, so you need that installed for it to work.
These scripts are quite time consuming to run, since they generate a lot of png files that are combined into the final video. It is highly recommended to run them on a remote system - it takes about 2 hours on an i5 machine to run the single plot scripts, and quite a lot longer to run the dual plot one.
Those times are with timelapse1090 set to record every 3 seconds, which is a lot shorter than the default 10 seconds - expect it to be quicker with the default setting at the expense of smoothness of the animation.
I have tested it on a pi 4 2Gb, and it ran OK but is slow:
real 523m2.676s
user 459m34.518s
sys 78m11.485s
I haven’t tested it on a pi 3, and I don’t know whether it will have sufficient RAM to do it - the frame generation is actually not that demanding, but ffmpeg uses quite a bit more memory. You could always generate the frames and copy them to a PC to combine into video with some other tool.
The ffmpeg settings are fairly basic for the sake of ease of processing - it’s possible to do more fancy filtering and blending, but the default looks OK.
2 Likes
It is hard to test, frankly, with sparse data.
There is still something not entirely right. UAT are not using MLAT, but are all ADSB. So, there is no need to set MLAT to yes. However, that leads to a problem in the jq. For mlat == "no" the selection
.aircraft | .[] | select(.lat != null) | select (.lon !=null) | select(.rssi != -49.5) | select( (has("tisb") | not) or (.tisb |
contains(["lat"]) | not) ) | select(any(.mlat[] ; .) | not)
Is not working in the absence of MLAT data. So, I figured, I set mlat=no and introduce a new variable nse (978. Get it?). This then has the same values as MLAT = yes as in the first if:
for i in $datadir/history_*.json; do
echo -n "."
if [[ $nse == "yes" ]]; then
sed -e '$d' $i | jq -r '.aircraft | .[] | select(.lat != null) | select (.lon !=null) | select(.rssi != -49.5) | select( (has("tisb") | not) or (.tisb | contains(["lat"]) | not) ) | [.lon,.lat,.rssi,.alt_baro] | @csv' >> $wdir/heatmap
elif [[ $mlat == "yes" ]]; then
sed -e '$d' $i | jq -r '.aircraft | .[] | select(.lat != null) | select (.lon !=null) | select(.rssi != -49.5) | select( (has("tisb") | not) or (.tisb | contains(["lat"]) | not) ) | [.lon,.lat,.rssi,.alt_baro] | @csv' >> $wdir/heatmap
elif [[ $mlat == "no" ]]; then
sed -e '$d' $i | jq -r '.aircraft | .[] | select(.lat != null) | select (.lon !=null) | select(.rssi != -49.5) | select( (has("tisb") | not) or (.tisb | contains(["lat"]) | not) ) | select(any(.mlat[] ; .) | not) | [.lon,.lat,.rssi,.alt_baro] | @csv' >> $wdir/heatmap
elif [[ $mlat == "mlat" ]]; then
sed -e '$d' $i | jq -r '.aircraft | .[] | select(.lat != null) | select (.lon !=null) | select(.rssi != -49.5) | select( (has("tisb") | not) or (.tisb | contains(["lat"]) | not) ) | select(any(.mlat[] ; .)) | [.lon,.lat,.rssi,.alt_baro] | @csv' >> $wdir/heatmap
fi
Similar at other spots in the script.
At the end, I can’t get it to work. Possibly due to a lack of data. Try again over the weekend:
pi@piaware:~/polar $ sudo ./polar.sh 1 5
Using config file
HWT OK
ADS-B aircraft only will be plotted
Using disk : /home/pi/polar
Gathering data every 5 seconds until Mon 23 Sep 19:54:27 PDT 2019
Number of data points collected: 382
Calculating Range, Azimuth and Elevation data:
Filtering altitudes
Processing heywhatsthat.com data:
"/dev/stdin" line 15: warning: Cannot find or open file "/tmp/tmp.61tKoI3pku/heatmap"
"/dev/stdin" line 15: warning: Can't read data file
"/dev/stdin" line 16: undefined variable: STATS_mean
I tested the filter on a json from your system and it appears to be producing results - I think your problem lies elsewhere:
See this:
Using disk : /home/pi/polar
doesn’t match this:
"/dev/stdin" line 15: warning: Cannot find or open file "/tmp/tmp.61tKoI3pku/heatmap"
Which means that gnuplot is looking in the wrong place to find the data to plot. The script will use /tmp to process data only if there is more the 1Gb of RAM in the system. If there’s less, it should be using the disk in the directory the script is in. It looks like for some reason in your case it is trying to pull data from /tmp when it should be pulling it from /home/pi/polar.
I just checked the script on a system with less than 1Gb RAM and it did work OK - are you sure you haven’t inadvertently modified anything? The section that does the check is the if block at line 87 in the current version:
mem=$(free -m|awk '/^Mem:/{print $2}')
if [ "$mem" -gt "1000" ]; then
wdir=$TMPDIR
echo "Using tmpfs : $wdir"
else
wdir=$PWD
echo "Using disk : $wdir"
fi
and with that, I just now checked polar.conf again. It was missing lat=. 
I filled it in, and immediately it produced results. Yesterday I played an hour and a half to get it do something.
Thank you for pointing this out. 
.
The reason it was missing it was because I had copied it from the Odroid that runs 1090. So, the file polar.conf existed. It was just missing a line
1 Like
I have an error message with my new instalation (Pi4 and 3.7.2 version) :
Using config file
HWT OK
ADS-B and MLAT aircraft will be plotted
Using tmpfs : /tmp/tmp.3gCM6eF2NM
Using local archive:
Unpacking compressed data:
..
Retrieving recent history:
...............................................................................................................................................................
Number of data points collected: 121629
Calculating Range, Azimuth and Elevation data:
Filtering altitudes
Processing heywhatsthat.com data:
jq: error (at upintheair.json:1): Cannot iterate over null (null)
My file upintheair.json is the same like my old instalation, who was working well.
Any idea ?
Thanks.
Check a few things - that all the position fields including receiver height are filled in properly and match those used in the heywhatsthat ID - if the heights don’t match it can cause problems. I have it on the list to make the upintheair.json override the user set position which removes a potential source of problems.
Did you copy the upintheair.json file over? If so, just delete it and let the script download a new one.
1 Like
Ok i have deletd it, and with the new one downloaded by the script, all is good now !
Thanks 