Heatmap & range/altitude view for dump1090-mutability v1.15

I found an 2 days old email in my spam box:

Dino M notifications@github.com
my location is N50.022, W114.433,
I entered 50.0222, -114,422… it doesn’t like the negative sign it seems.

Thanks,
Dino

Now I know what the issue is. Not only the negative sign, but also the use of a comma instead of a dot.

I have fixed the scripts. They will now convert comma’s to dots. I hope this also helps :wink:

Thanks Dino!

Hi Ted!

Found out that in dump1090.socket30003.heatmap.pl, the output file has the Lat and long reversed for each data point. It appears it should be Lat,Lon. See line 246 I believe.

I found this in the github version and made a pull request.

After the correction my heat map came up perfectly.

Thanks again.

Thanks DinoM for fixing this! Your pull request is merged!

I hope you like it :wink:

Ted

Ted -

Logging the output of dump1090.socket30003.pl and seeing the following:


Using 'mile' the distance and 'feet' for the altitude.
Trying to connect to peer host '127.0.0.1'...
The antenna latitude & longitude are: '42.7997645477918','-74.3447649332153'
The data directory/file is: /var/log/dump1090.socket30003.pl-127_0_0_1-150904.tx
t
The log  directory/file is: /var/log/dump1090.socket30003.pl-127_0_0_1-150904.lo
g
'dump1090.socket30003.pl' will be started!
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 930781.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 930782.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 930783.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 930784.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 930785.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 976928.
Use of uninitialized value $hex_ident in pattern match (m//) at ./dump1090.socke
t30003.pl line 496, <GEN0> line 976929.


Jim

Hi Jim, That doesn’t look good :wink:

In this part of the dump1090.socket30003.pl script (near line 496) it is listing to port 30003, reading raw ADB-S data. It should read lines with these field:


"message_type","transmission_type","session_id","aircraft_id","hex_ident","flight_id","generated_date","generated_time","logged_date","logged_time","callsign","altitude","ground_speed","track","lat","lon","vertical_rate","squawk","alert","emergency","spi","is_on_ground"

The data is seperated using commas. The script is looking for the column with the hex_ident value (the 5th column). But it is not there. I am not sure why. Haven’t seen this before.

You are trying to connect to a dump1090 on the local host. May be the dump1090.socket30003.pl script listing to a dump1090 version which is not running? Or a different version then I use? I have tested this with dump1090-mutability and dump1090-piaware.

Can you check whether the dump1090.socket30003.pl is connected to port 30003, please?


pi@ted1090-3 ~ $ netstat | grep 30003
tcp     2322      0 localhost:41821         localhost:30003         ESTABLISHED
tcp        0   2322 localhost:30003         localhost:41821         ESTABLISHED


Are they Established?

Could you please show me the output of this command:


pi@ted1090-3 ~ $ ps -ef | grep dump
dump1090  1863     1 28 03:00 ?        04:03:15 /usr/bin/dump1090-mutability --net --ppm 0 --oversample --fix --phase-enhance --lat 52.085624 --lon 5.0890591 --max-range 400 --net-http-port 0 --net-ri-port 30001 --net-ro-port 30002 --net-bi-port 30004 --net-bo-port 30005 --net-sbs-port 30003 --net-fatsv-port 10001 --net-heartbeat 60 --net-ro-size 500 --net-ro-interval 1 --net-buffer 2 --stats-every 3600 --write-json /run/dump1090-mutability --write-json-every 1 --json-location-accuracy 1 --quiet
root      2360  2359  0 03:03 ?        00:00:00 /bin/sh -c sudo /home/pi/src/dump.socket-0.3.pl > /tmp/sock.log
root      2361  2360  0 03:03 ?        00:00:00 sudo /home/pi/src/dump.socket-0.3.pl
root      2362  2361 51 03:03 ?        07:18:14 /usr/bin/perl -w /home/pi/src/dump.socket-0.3.pl
pi       17239 16710  0 17:19 pts/0    00:00:00 grep --color=auto dump


May be your dump1090 does not run with the** “–net-sbs-port 30003”** option? I though that was default?

I will try to fix the error message that you got in a new version and let the script exit with a clear error message.

Ted


pi@piaware ~ $ netstat|grep 30003
tcp        0      0 localhost:43465         localhost:30003         ESTABLISHED
tcp        0      0 localhost:30003         localhost:43465         ESTABLISHED
tcp        0      0 localhost:43435         localhost:30003         ESTABLISHED
tcp        0      0 localhost:30003         localhost:43435         ESTABLISHED



pi@piaware ~ $ ps -ef|grep dump
dump1090  2393     1 23 00:00 ?        03:44:14 /usr/bin/dump1090-mutability --net --ppm -51 --oversample --fix --phase-enhance --lat 42.7997645477918 --lon -74.3447649332153 --max-range 300 --net-http-port 0 --net-ri-port 30001 --net-ro-port 30002 --net-bi-port 30004 --net-bo-port 30005 --net-sbs-port 30003 --net-fatsv-port 10001 --net-heartbeat 60 --net-ro-size 500 --net-ro-interval 1 --net-buffer 2 --stats-every 3600 --write-json /run/dump1090-mutability --write-json-every 1 --json-location-accuracy 1 --quiet --forward-mlat


Yup… --net-sbs-port 30003 is set

Here’s a telnet to 30003


pi@piaware ~ $ telnet localhost 30003
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
MSG,3,111,11111,AA44AD,111111,2015/09/04,15:55:25.820,2015/09/04,15:55:25.828,,22250,,,42.65373,-73.84005,,,,,,0
MSG,7,111,11111,AA44AD,111111,2015/09/04,15:55:25.831,2015/09/04,15:55:25.880,,22250,,,,,,,,,,0
MSG,8,111,11111,A752B3,111111,2015/09/04,15:55:25.835,2015/09/04,15:55:25.880,,,,,,,,,,,,0
MSG,8,111,11111,A752B3,111111,2015/09/04,15:55:25.845,2015/09/04,15:55:25.881,,,,,,,,,,,,0


Jim

@Jim, That looks (all three) okay. What system are you using? Can you share your OS version please? I will try to reproduce your issue. Ted


pi@piaware ~ $ uname -a
Linux piaware 3.18.11+ #781 PREEMPT Tue Apr 21 18:02:18 BST 2015 armv6l GNU/Linux


pi B

Jim

Hi Jim,

Thanks for the info. I have tried to reproduce your issue. I have download the PiAware on Raspbian Linux 2.1-2 ZIP version and installed it. Then I cloned my github repo https://github.com/tedsluis/dump1090.socket30003 to this Raspberry and started the dump1090.socket30003.pl script. It worked okay:


pi@piaware ~/git/dump1090.socket30003 $ ./dump1090.socket30003.pl 
Using 'kilometer' the distance and 'meter' for the altitude.
Trying to connect to peer host '127.0.0.1'...
The antenna latitude & longitude are: '52.085624','5.0890591'
The data directory/file is: /tmp/dump1090.socket30003.pl-127_0_0_1-150905.txt
The log  directory/file is: /tmp/dump1090.socket30003.pl-127_0_0_1-150905.log
Starting 'dump1090.socket30003.pl'....
'dump1090.socket30003.pl' (2796) is started!
Using pidfile /tmp/dump1090.socket30003.pl-127_0_0_1.pid.
^Cpositions:176
You pressed CTRL-C. Do you want to exit? (y/n)
y

Exit: The script was interrupted by CTRL-C!
pi@piaware ~/git/dump1090.socket30003 $ ls -l /tmp
total 28
-rw-r--r-- 1 pi   pi    6921 Sep  5 20:07 dump1090.socket30003.pl-127_0_0_1-150905.log
-rw-r--r-- 1 pi   pi   11415 Sep  5 20:07 dump1090.socket30003.pl-127_0_0_1-150905.txt
-rw-r--r-- 1 root root  5573 Sep  5 20:06 piaware.out
pi@piaware ~/git/dump1090.socket30003 $ cat /tmp/dump1090.socket30003.pl-127_0_0_1-150905.txt
3985A4,563,52.32397,4.96974,2015/09/05,20:06:54.680,-25.62,27.72
4CA217,11263,51.65781,5.48990,2015/09/05,20:06:54.781,138.04,54.95
40560B,10966,51.51517,7.12597,2015/09/05,20:06:54.781,106.18,153.74
471F81,10966,51.72815,4.45099,2015/09/05,20:06:54.782,-120.27,59.12
461F9C,11803,52.69206,3.42742,2015/09/05,20:06:54.782,-69.06,131.37
484F01,1812,52.42232,4.83414,2015/09/05,20:06:54.783,-35.94,41.26
etc....

Then I stopped the Piaware fadump1090.sh and installed the dump1090-mutability v1.15. And again dump1090.socket30003.pl it worked okay.

So, I really don’t know what the problem is on your raspberry.

For my it was the first time I have tried the Piaware Linux version. It is not much different from the raspbian version I use on my other raspberries. I have 4 other raspberries running Piaware. 2 using the flightaware fadump1090 version and two running the dump1090-mutability version (one with v1.14 and one with v1.15). I also run dump1090 versions for testing on x86 hardware on fedora, debian, centos and even on debian docker images. I have not (yet) experienced the issue you have. I am wondering what it can be.

I will try to publish a new version tomorrow. I will add something to debug the issue. If you have any suggestion, please let me know.

Ted

Ted -

Thanks for doing such a great job of support this. Too often folks post stuff and when someone, like myself, who hasn’t done the sysadmin end of things on linux in 20 some-odd years, tries to use it and runs into problems, there is little support.

It’s up to you if you want to try and debug it not. To be honest, I’m waiting for a more “integrated” version of your heatmap code that will work inside script.js. At this point, I have limited expertise in taking the output of the perl script and putting it into the script file to display the heatmap. I actually haven’t been able to get it to work at this point.

I will continue to monitor your progress and participate when I can.

Jim

Hi Jim,

Any help is welcome, but of course never obligated;-)

Jim, That is no problem. I can understand that.

In essence, just follow the steps from the first post of this topic. One step may be diffecult: adding the thousands of location lines (the blue ones below here) in to the script.js.
-------------
position: google.maps.ControlPosition.TOP_LEFT,
style: google.maps.MapTypeControlStyle.DROPDOWN_MENU
}
};

var heatmapData =

new google.maps.LatLng(51.21172, 3.69534),
new google.maps.LatLng(51.67090, 4.84928),
new google.maps.LatLng(49.35645, 0.22695),
new google.maps.LatLng(52.26494, 4.51668),
new google.maps.LatLng(53.20353, 6.17335),
new google.maps.LatLng(52.09680, 4.34227),
new google.maps.LatLng(52.25060, 3.85604),
new google.maps.LatLng(52.52435, 4.87610),
new google.maps.LatLng(52.44127, 5.24475),
etc…

];

GoogleMap = new google.maps.Map(document.getElementById(“map_canvas”), mapOptions);
------------

Here is one way to do that is by editting the** heatmap.csv** file:

step0: Add the red lines as described in the first post of this topic.

step1:


pi@piaware ~/git $  cat /usr/share/dump1090-mutability/html/script.js

step2: select and copy the first part (up to and including "var heatmapData = ") of the script.js in to the copy-buffer.

step3:


pi@piaware ~/git $ nano /tmp/heatmap.csv

step4: past the first part of the script.js from the copy-buffer at the start of /tmp/heatmap.csv file. Save and exit the file.

step5: select and copy the last part (everything after "var heatmapData = ") of the script.js in to the copy-buffer.

step6: past the last part of the script.js from the copy-buffer at the end of /tmp/heatmap.csv file. Save and exit the file.

step7:


pi@piaware ~/git $ sudo mv /usr/share/dump1090-mutability/html/script.js   /usr/share/dump1090-mutability/html/script.js.org

step8:


pi@piaware ~/git $ sudo cp /tmp/heatmap.csv   /usr/share/dump1090-mutability/html/script.js

note: It works more easy when you open two terminals. One terminal where you can copy the parts from the script.js and one terminal where you can past in to heatmap.csv.

step9: refresh your browser two times, to display the heatmap.

See nothing? May be you heatmap is to big? May be you made a typo or you forgot to edit the gmap.html?

I hope this helps.

Jim, If you want you can try my new release: https://github.com/tedsluis/dump1090.socket30003.git
Just update your local repository:


pi@piaware $ ~/git/dump1090.socket30003 $ git pull

It doesn’t fix your issue, but with the “-debug” option we are able to see what messages the script reads from the 30003 port. It should be something like this:


pi@piaware ~/git/dump1090.socket30003 $ ./dump1090.socket30003.pl -peer 192.168.11.44 -debug
Using the unit 'kilometer' for the distance and 'meter' for the altitude.
Trying to connect to peer host '192.168.11.44'...
The antenna latitude & longitude are: '52.085624','5.0890591'
Connected to '192.168.11.44', port 30003 (tcp).
The data directory/file is: /tmp/dump1090.socket30003.pl-192_168_11_44-150906.txt
The log  directory/file is: /tmp/dump1090.socket30003.pl-192_168_11_44-150906.log
Starting 'dump1090.socket30003.pl'....
'dump1090.socket30003.pl' (27279) is started!
Using pidfile /tmp/dump1090.socket30003.pl-192_168_11_44.pid.
'messagecount=1,message='MSG,8,111,11111,3C5442,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,,,,,,,,,,,0
'messagecount=2,message='MSG,5,111,11111,468893,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,37000,,,,,,,0,,0,0
'messagecount=3,message='MSG,7,111,11111,4BA94A,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,36000,,,,,,,,,,0
'messagecount=4,message='MSG,5,111,11111,4CA357,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,39000,,,,,,,0,,0,0
'messagecount=5,message='MSG,7,111,11111,3440CD,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,14900,,,,,,,,,,0
'messagecount=6,message='MSG,6,111,11111,4CA27B,111111,2015/09/06,09:56:59.689,2015/09/06,09:56:59.689,,,,,,,,2720,0,0,0,0
'messagecount=7,message='MSG,8,111,11111,4BA94A,111111,2015/09/06,09:56:59.690,2015/09/06,09:56:59.690,,,,,,,,,,,,0
'messagecount=8,message='MSG,6,111,11111,3C49ED,111111,2015/09/06,09:56:59.690,2015/09/06,09:56:59.690,,,,,,,,5313,0,0,0,0
'messagecount=9,message='MSG,5,111,11111,40622D,111111,2015/09/06,09:56:59.690,2015/09/06,09:56:59.690,,29025,,,,,,,0,,0,0

But I expect that you get different messages or may be empty messages.

Without the “-debug” option the script may give some new messages when something is wrong with the socket. But I am not sure what it will do in your case.

One more question: Have you tried to run the dump1090.socket30003.pl script as root:


pi@piaware ~/git/dump1090.socket30003 $ sudo ./dump1090.socket30003.pl

If you still want to try this, then share with me the output. If you do not want to test anymore, than that is fine with me :wink:

Ted

Hi,

I have released a new version: https://github.com/tedsluis/dump1090.socket30003

The biggest improvement is the automatically way of handling different unit types (like feet, meter, kilometer, mile and nauticalmile):

dump1090.socket30003.pl now writes a column header in the output file. The header columns ‘altitude’ and ‘distance’ will have a unit type added between parentheses.

This way the others scripts (dump1090.socket30003.heatmap.pl and dump1090.socket30003.radar.pl) automatically detect the correct unit type.

The scripts are downwards compatible with the old versions without headers. For the old versions it will still uses the default unit types or the ones that you specify.

You can even restart dump1090.socket30003.pl with other unit types. The script will then write a new header in to the file and from that point the new unit types will be apply.

Here is an example of the dump1090.socket30003.pl outputfile: /tmp/dump1090.socket30003.pl-192_168_11_34-150906.txt


hex_ident,altitude(meter),latitude,longitude,date,time,angle,distance(kilometer)
4CA766,11575,51.67790,2.85407,2015/09/05,08:30:23.010,-100.67,159.95
45C261,10966,51.82130,5.17868,2015/09/05,08:30:23.041,161.99,30.02
424050,10357,52.33214,4.21715,2015/09/05,08:30:23.050,-73.52,65.42
401240,10973,52.27798,3.94598,2015/09/05,08:30:23.079,-79.98,80.81
3950D1,6998,51.75334,4.62910,2015/09/05,08:30:23.091,-126.97,48.57
4841A6,1523,52.43298,5.31036,2015/09/05,08:30:23.092,31.39,41.45
342105,7447,51.35345,4.22089,2015/09/05,08:30:23.120,-131.28,101.01
hex_ident,altitude(feet),latitude,longitude,date,time,angle,distance(meter)
484443,12125,52.24008,3.99765,2015/09/05,12:54:14.926,-81.54,76395
48415E,4175,52.31666,5.17440,2015/09/05,12:54:14.932,19.48,26338
300092,3550,52.22533,4.70748,2015/09/05,12:54:14.933,-69.07,30312
3C6DD4,25975,50.77332,5.39941,2015/09/05,12:54:14.934,167.2,147491
4CA854,38000,51.80789,5.20393,2015/09/05,12:54:14.977,158.36,31868
484C5A,16375,51.80800,4.67743,2015/09/05,12:54:14.980,-125.1,41818

I have also added options to manipulate the heatmap and raderview. The help text is improved (read it on Github). More messages while data is processed. A debug option for the 30003 socket connection.

I hope you like it.

Get the scripts using:


pi@piaware ~/git $ git clone https://github.com/tedsluis/dump1090.socket30003.git

Or update them using


pi@piaware ~/git/dump1090.socket30003 $ git pull

To create a heatmap using the output of the dump1090.socket30003.heatmap.pl script, read the first post of these two posts:
http://discussions.flightaware.com/post180859.html#p180859
http://discussions.flightaware.com/post180927.html#p180927

To create a radar map using the dump1090.socket30003.radar.pl script read this post:
http://discussions.flightaware.com/post180772.html#p180772

In the coming weeks I will work on the integration of the heatmap and the radarview into dump1090.

Any feedback is welcome.

Ted
http://flightaware.com/adsb/stats/user/tedsluis

Ted -

Thanks again for the great support. I updated and will start my logging today. I need to revisit the beginning of the thread to figure out what is missing from gmap.html. I think that was the problem. Also cutting and pasting thousands of lines into script.js take quite a while.

I’ll let you know what happens.

Jim

Hi Ted,

thank you for the tool. I installed the scripts following your instructions and it seems to work fine.
One question though - if I use the radar plot script I get hardly any positions exported and the scripts says "Number of positions processed 19204 and positions within range processed 203. It looks like it’s only exporting / plotting the 203. Whats the discrepancy between all positions and positions within range please?

Thanks again for the tool!
Johann

Hi Johann,

You are referring to a messages of the dump1090.socket30003.radar.pl script.

Most likely your maximum altitude to to low. The default is 15000. If you use feet instead of meter this is not very high :wink: and positions higher then 15000 (feet of meter) will be skipped.

You can change the default “$default_max_altitudel” in the upper section of the script:


[pi@ted1090-3 $ dump1090.socket30003]# nano dump1090.socket30003.radar.pl

Change the red number to 45000 in the code below:

#!/usr/bin/perl -w

Ted Sluis 2015-09-06

Filename : dump1090.socket30003.radar.pl

#===============================================================================

Default setting:

my $default_max_altitude = 15000; # specified in the output unit (use 45000 for feet and 15000 for meter)
my $default_min_altitude = “0”; # specified in the output unit
my $default_number_of_directions = 1440; #
my $default_number_of_altitudezones = 15;
my $default_datadirectory = “/tmp”;
my $defaultdistanceunit = “kilometer,kilometer”; # specify input & output unit! kilometer, nauticalmile, mile or meter
my $defaultaltitudeunit = “meter,meter”; # specify input & output unit! meter or feet
my ($antenna_latitude,$antenna_longitude) = (52.085624,5.0890591); # Home location, default (Utrecht, The Netherlands)

Save + exit

Or you could launch the script with the parameter “-max 45000

You also can check the altitude unit and distance unit of your position data. Does the dump1090.socket30003.radar.pl script use the same units? If you uses an older version of my dump1090.socket30003.pl script, you may need to specify the correct units using the -altitudeunit -distanceunit parameters. When you run the dump1090.socket30003.radar.pl script it will tell you what units it uses for altitude and distance:


[root@msi dump1090.socket30003]# ./dump1090.socket30003.radar.pl -altitudeunit feet,meter --distanceunit meter,meter
The altitude will be converted from 'feet' to 'meter'.
The distance will be converted from 'meter' to 'meter.
The maxium altitude is 15000 meter.
The minimal altitude is 0 meter.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 15.
The latitude/longitude location of the antenna is: 52.085624,5.0890591.
An altitude zone is 1000 meter.
The following files fit with the filemask 'dump*.txt':
  /tmp/dump.socket-127_0_0_1-150915.txt
processing '/tmp/dump.socket-127_0_0_1-150915.txt':
  -default units:altitude=feet,distance=meter, position 1-5281930. processed.

An other reason can be the location. You have to specify the location of your own antenna location.
You can do that in the upper section of the script by changing the blue numbers or you cloud launch the script using the " -lat " and “-lon ” parameters.
Locations that are more then 600 km away from the antenna location will be skipped.

I hope this solves your issue. If not, please let me know what parameters you uses and show me some lines of position data.

Ted

Thanks for coming back to me Ted.
I deleted the old files in the tmp folder and restarted the script after changing the coordinates. I had converted my -4 longitude to 356 as mentioned somewhere at the beginning of the thread.
It works with the -4 now.

Number of files read: 1
Number of position processed: 1831 and positions within range processed: 1831
datafile=/tmp/radar.csv

Many thanks!
Johann

I have add KML output to the dump1090.socket30003.radar.pl script. And the “/tmp/radar.kml” output file can be displayed into the dump1090-mutability gui.

To display the “/tmp/radar.kml” file into the dump1090-mutability gui it must be published on a public web server (the google maps cloud API must have access to it). If you dont have a public web server you can use a public folder on dropbox or google drive. I have uploaded my radar.kml file to dropbox: https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/radar.kml
Be sure you give it access to everyone.

One other thing you must do in order to display the radar.kml file is adding some code to the script.js:


pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo nano script.js

Add the red lines below. The blue lines were already added to display the heatmap (see the first post of this topic). The blue lines are not needed for the radarview.

    GoogleMap = new google.maps.Map(document.getElementById("map_canvas"), mapOptions);

    //Define OSM map type pointing at the OpenStreetMap tile server
    GoogleMap.mapTypes.set("OSM", new google.maps.ImageMapType({
            getTileUrl: function(coord, zoom) {
                    return "http://tile.openstreetmap.org/" + zoom + "/" + coord.x + "/" + coord.y + ".png";
            },
            tileSize: new google.maps.Size(256, 256),
            name: "OpenStreetMap",
            maxZoom: 18
    }));

    GoogleMap.mapTypes.set("dark_map", styledMap);

    var heatmap = new google.maps.visualization.HeatmapLayer({
            data: heatmapData
    });
    heatmap.setMap(GoogleMap);

    UserKML = new google.maps.KmlLayer({
            url: "https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/radar.kml",
            map: GoogleMap,
            preserveViewport: true
    });

    // Listeners for newly created Map
    google.maps.event.addListener(GoogleMap, 'center_changed', function() {
            localStorage'CenterLat'] = GoogleMap.getCenter().lat();
            localStorage'CenterLon'] = GoogleMap.getCenter().lng();
            if (

Save and exit the script.js file.

Be sure you fill in your own URL to your radar.kml file in the code above. Of course you can just test this with my URL.

It is also possible to upload your **radar.kml **file to a web site like http://www.gpsvisualizer.com to test it.

The new release of the dump1090.socket30003.radar.pl script with KML support is available at: https://github.com/tedsluis/dump1090.socket30003.git

Here you can see the radar view and heatmap together in the dump1090-mutability GUI:

https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/Screenshot%20from%202015-09-17%2018-12-12-ANIMATION.gif

In the next week I will work on a legend where you can select/deselect the individual altitude zones of the radar view.

Ted

If have a breakthrough on loading a heatmap data from a csv file. From a movable window in the dump1090 GUI I am able to load a csv file and throttle the radius, opacity and intensity of the heatmap. I works great :wink:

I have made a 4 minute screencast and publish it on youtube:https://www.youtube.com/watch?v=Byu3zm_5Ons

https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/Screenshot%20from%202015-09-19%2009%3A32%3A26.png

I will publish the code later this weekend.

Ted

I have integrated the Heatmap (csv import) and radar range view (KML format) into a fork of the current dump1090-mutability dev v15.

Here are the features:

  • A panel for loading the heatmap data from a csv file.
  • The panel has the ability to adjust the intensity, opacity and radius of the heatmap using sliders.
  • A legend for planes colors (bases on altitude and mlat status)
  • A legend for range rings.
  • Display a KML layar to view radar range (every altitude zone has a different range).
  • The panel and the legends are draggable and can by placed on every post on the screen.
  • You can toggle the Heatmap, the Range view, the plane colors and the range rings from the sidebar on the right side.

My modified version of dump1090-mutability can be found here: https://github.com/tedsluis/dump1090
If you don’t want (or known how to) compile it from source you can also replace and add a couple of files. Down here I explain (only one way) how to do that:

First move the changed files to a backup directory as following:


pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mkdir backup.20150920
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mv planeObject.js backup.20150920
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mv config.js backup.20150920
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mv style.css backup.20150920
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mv gmap.html backup.20150920
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo mv script.js backup.20150920

Then download the new files as following:


pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/planeObject.js
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/legend.js
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/config.js
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/style.css
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/gmap.html
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/script.js
pi@ted1090-1 /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/papaparse.min.js

UPDATE 21Sept23:08: I made a couple of improvements to the repo. I remove the file jquery-2.1.1.min.js, because it is no longer needed.

Now the modifications to the dump1090-mutability are ready!

Note: edit the config.js file with your own settings. There are some new lines!

To display your own heatmap and radar range view, you must have a heatmapdata.csv and a radar.kml file. Both can be made using my other scripts: https://github.com/tedsluis/dump1090.socket30003

Download the files on you system (for example a raspberry pi) using git:


pi@piaware $ ~/git/dump1090.socket30003 $ git clone https://github.com/tedsluis/dump1090.socket30003.git

Read the help text.
Run the dump1090.socket30003.pl for at least a couple of days to gather flight data.
note: Let the dump1090.socket30003.pl script run from crontab to be sure that it won’t stop. Edit crontab like this:


pi@ted1090-3 ~ $ sudo crontab -e

And add the line of code:


03 * * * * sudo /home/pi/git/dump1090.socket30003/dump1090.socket30003.pl > /tmp/sock.log

This way crontab triggers the script every hour. Be sure you use the correct path!

Files with flight data are stored by default in /tmp. Every day a new txt (and log) file. Note: Files in /tmp will be removed after a reboot! (be you can change this setting or use an other output directory.

Ones you have some flight data you can run the dump1090.socket30003.heatmap.pl script to get the heatmapdata.csv and run the dump1090.socket30003.radar.pl script to get the radar.kml file.

If you don’t want to wait until you have enough flight data you can test the dump1090 heatmap and radar range view using my files:
heatmapdata.csv: https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/heatmapdata.csv
radar.kml: https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/radar.kml

Heatmap data file:
Option 1: Download the heatmapdata.csv file to the system where you run the web browser to view dump1090. To display the heatmap, Go to the dump1090 GUI (/dump1090/gmap.html) and click on the “Choose file” button in the heatmap panel and select the heatmapdata.csv.
Option 2: Download the heatmapdata.csv file and save it in /usr/share/dump1090-mutability/html.

Radar view KML file:
This file must be storage on a public accessible webserver on the internet. This is necessary so the Google cloud api can access it. If you don’t have a web server you can use a public dropbox folder of google doc folder to store the radar.kml file. Don’t forget to configure the URL to the file in the /usr/share/dump1090-mutability/html/config.js file.

If you use my files for demo be sure you move the map to the Netherlands (in Europe).

I have made a 3 minute YouTube video that shows how it works: https://www.youtube.com/watch?v=Qz4XSFRjLTI

https://dl.dropboxusercontent.com/u/17865731/dump1090-20150916/Screenshot%20from%202015-09-20%2020-35-45-ANIMATION.gif

Now the development is not yet finished. I want to improve a couple of things. But this is may the first version that anyone with a bit of Linux knowledge can try. Any feedback is welcome.

Ted

Ted -

I’m trying to get it installed but running in to a hiccup. I have a heatmapdata.csv and heatmapcode.csv generated after the latest run but no new heatmap.csv. What am I missing?

I also didn’t have a number of the files that you had listed (legend.js and jquery-2.1.1.min.js) I didn’t have in my install, either.

Jim