This aims to be a guide on how to get a heatmap and range/altitude view in dump1090-mutability (v1.15 dev). Down here I have put together all the important stuff, like:
- Working features.
- Known issues.
- Future wishes.
- Change log.
- Overview: How does it work.
- Screenshots, video and live view (try dump1090-mutability with heatmap and rangeview yourself)
- Installation and upgrade steps dump1090-mutability v1.15 dev.
- Installation and upgrade steps socket30003 scripts.
- Configuration.
- Resources.
Thanks
First of all I want to thank Oliver Jowett -also known as Obj- for his great work on dump1090-mutability. Without his skills and dedication, we could not enjoy his amazing dump1090 version!
What is working:
Available features in the socket30003 scripts (https://github.com/tedsluis/dump1090.socket30003)
- Collecting ADB-S data, saved in csv format (latitude, longitude, altitude, distance ,etc)
- Support for collecting ADB-S data from multiple dump1090 instances simultaneously on one host.
- Possibility to convert the altitude unit type to meter or feet and the distance unit type to meter, feet, kilometer, mile or nautical mile.
- Possiblilty to specify the data directory, antenna location, etcā¦
Available features in my dump1090-mutability fork (https://github.com/tedsluis/dump1090):
- Display a csv heatmap file.
- Adjust the opacity, intensity and radius of the heatmap from a movable panel.
- Load a heatmap from the dump1090 web directory or from the heatmap panel in the browser.
- Display a KML range/altitude file.
- Display distance range rings around the antenna.
- Provide movable legends for the altitude colors and range rings.
- Toggle plane colors between Altitude colors and adb-s/mlat position colors.
- Toggle the heatmap, the range/altitude view and the range rings on and off (including their panel and legends).
- And of course all the functionality that comes along with the original dump1090-mutability v1.15 dev by Obj.
Known issues:
- Not used the correct colors in range/altitude KML file. (This is probably caused by incorrect hsl to rbg color conversion.)
Future wishes:
- Toggle on and off individual range/altitude lines.
- One common config file for the dump1090.socket3003 scripts. (the config setting are now stored within the scripts).
- File-system utilization monitoring to prevent that the script output writes the file-system full.
Live view
You can try my dump1090 fork with heatmap and rangeview in the Google cloud: **http://130.211.110.109/dump1090/gmap.html **(This dump1090 runs on a 60-day free trail that is available until September 6 2016, more info at https://cloud.google.com/free-trial/)
Screen shots & video
I have made a 3 minute YouTube video that shows how it works: https://www.youtube.com/watch?v=Qz4XSFRjLTI
https://github.com/tedsluis/dump1090/raw/master/img/2015-09-20-20-35-45-ANIMATION.gif
Change log https://github.com/tedsluis/dump1090.socket30003:
(commit log on Github: https://github.com/tedsluis/dump1090.socket30003/commits/master)
2017-01-10: Major update
- Fix tcp 30003 connection drop issue. Implemented new socket methode.
- Handles messages using the common log file routine.
- Save settings in config file.
- Added ā-debugā and āverboseā logging options.
- Changed section radarview to rangeview.
- Bugfixes.
- Updated helptext
2015-12-17 - Improved logging for large amount of incomplete messages.
- Fixed error 'Argument isnāt numeric in division (/) in line 644.
2015-09-27 - Fixed issue in heatmap.pl and rangview.pl on finding the input files.
2015-09-24: - Renamed dump1090.socket3003 scripts to socket30003.pl, heatmap.pl and rangeview.pl.
- Renamed radar.kml to rangview.kml
- Renamed socket30003.pl output files to dump1090-<ip_address/hostname>-YYMMDD.txt
- Update help textās.
- note: more detail at: http://discussions.flightaware.com/post181471.html#p181471
2015-09-20 : - Set default max altitude in meter and in feet.
- First Altitude zone includes everything below and the last altitude zone includes everything above.
2015-09-19: - Added heatmap output in svc format.
2015-09-17: - Added range/altitude view output in KML format.
- Changed altitude colors to hsl format.
2015-09-06: - Added support for position data with column headers and unit types for altitude and distance.
- Improved progress messages.
2015-09-03: - Fixed flipped over latitude and longitude (credits for DimoM).
2015-09-02: - Fixed handling negative latitude and longitude.
- Support for meter, feet, mile, nautical mile and kilometer.
2015-08-30: - Initial version.
Change log https://github.com/tedsluis/dump1090:
(commit log on GitHub: https://github.com/tedsluis/dump1090/commits/master)
2017-01-10:
- Bugfixes.
2016-02-20: - Merged upstream commits from github.com/mutability/dump1090
- Added screenshots & video to the README.md
2015-09-24: - Renamed radar.kml to rangeview.kml in config.js.
2015-09-21: - Added support for toggling legends.
- Added support for loading a heatmap from the dump1090 web directory.
- Added support for toggling plane colors between Altitude and mlat/non-mlat colors.
- Replace a local jquery-2.1.1.min.js for a downloadable version.
2015-09-20: - Added moveable altitude legend and range legend. Added toggle buttons for heatmap, range, plane colors and rings.
- Added config items for Heatmap, altitude legend and range rings legend.
- Added buttons for toggling the heatmap, range/altitude view, colors, rings in the side bar (credits for Toverfield!).
- Altitude and range legend added (credits for Toverfield!).
- Added KML URL to config.js.
- Added KML layer support (credits for Starksk!).
- Added heatmap support and movable heatmap panel.
An overview: How does it work:
A heatmap is based on flight positions gathered by dump1090. If there are planes in a particular area, this area become a heat point on the heatmap. Many planes in an area will give that particular heat point more āweightā . More āweightā means a more intense color of that heat point on the map. The best way is the show this in an picture:
By default square area (of -/+ 5 degree lon/lat) around the antenna is divided in a matrix of 1000 x 1000 heatmap point. Of course these parameters are adjustable.
Same as a heatmap, the range/altitude view is based on gathered flight positions. We want to see how far our antenna reaches for a particular altitude. Therefor there are altitude zones defined. By default an altitude is 500 meter or 1500 feet. And by default there are 24 altitude zones defined. The main goal is to determine a maximum antenna range for every altitude zone in every direction. To achieve this the distance from the antenna to every captured flight position must be calculated. And also the direction of the flight positions compared to our antenna must be determined. Therefor there are direction zones defined, by default 1400 directions. Based on the direction a flight position belongs to a particular direction zone. Now it is possible to determine which flight position, in a particular altitude zone and direction zone, is the one that is most far away. Again, the best way is to show this with a picture:
Flight position data can be collect from dump1090. Any dump1090 (the Piaware version and the Mutability version) share ADB-S data with flight position information on port 30003. The script socket30003.pl collects the ADB-S data and saves it in csv format.
Later, after capturing flight position data for a couple of days, it is possible to create a heatmap and a range/altitude view based of the flight position data saved in the csv files. The heatmap.pl script creates a heatmapdata.csv file. And the rangeview.pl script creates a rangeview.kml file. They are saved in /tmp by default.
A modified fork of dump1090-mutability is able to load the heatmapdata.csv from /usr/share/dump1090-mutability/html directory or from the dump1090 GUI. And the rangeview.kml can be displayed while it is loaded from a public accessible web server (if you donāt have that you can just use cloud storage like Github or Google drive). This is necessary because Google Maps API needs to have access, because it will render the KML file in the cloud.
So the whole solution is two different applications which can be used together or separately. You can use the socket30003.pl, heatmap.pl and rangview.pl scripts to create a heatmap and rangeview KML file. You can use the modified dump1090-mutability to display these heatmap file and range/altitude view file. Or you can display the heatmap and range/altitude view in an other tool. And you even can create a heatmap and range/altitude view in an other way and display them using modified dump1090-mutability. I will show you what other tools you can use to display the heatmap and range/altitude view, later.
Steps to take.
What do you need to do to get a heatmap and a range/altitude view:
- Install my socket30003 Perl scripts (socket30003.pl, heatmap.pl and rangeview.pl) from GitHub.
- Install my fork of dump1090-mutability from GitHub.
- Run socket30003.pl for a couple of days to collect flight position data.
- After a couple of days run heatmap.pl and rangeview.pl
- Make the output files heatmapdata.csv and rangeview.kml available to dump1090-mutability.
These steps are described below.
Install or update dump1090-mutability v1.15 dev.
Dump1090-mutability v1.15 is still under development by Obj. The only way to get it is to build it yourself from sources. Down here is explained how you can do that.
Ones you have build and install dump1090-mutability you can find the web server files in: ā/usr/share/dump1090-mutability/htmlā.
Here are the files located that I have changed and added in order to get the heatmap and range/altitude view.
When you want to commit an update you can either rebuild and reinstall dump1090 or you can only replace the updated files.
So you have three options:
- Build & Install
- Build & update
- Replace updated files (apply a small update).
Note: This installation instructions are valid for Raspbian (the default image for Raspberry) as well as Piaware (the Flightaware version for Raspberry).
**Do you still have the dump1090 version running, supplied by Flightaware? **
In order to get the dump1090-mutability, the dump1090 version supplied by Flightaware must be stopped. But donāt warry, you Piaware will run just fine together with dump1090-mutability.
First check whether fadump1090 is running using:
pi@piaware ~ $ sudo /etc/init.d/fadump1090.sh status
Then stop it using:
pi@piaware ~ $ sudo /etc/init.d/fadump1090.sh stop
pi@piaware ~ $ sudo /etc/init.d/fadump1090.sh status
Then remove the execute permissions from the service:
pi@piaware ~ $ sudo chmod 644 /etc/init.d/fadump1090.sh
Now it wonāt launch again after a reboot 
Build & Install dump1090-mutability from source:
Is it the first time that you build dump1090 from source on this system? Then follow these steps:
Note: This instruction follows exacly the same methode as provided in the github.com/mutability/dump1090/ ā¦ /README.md by Obj. I just show all the steps and give some extra comments.
At this moment on Piaware (not on Raspbian images) you need to remove some librtlsdr packages that are unknown to the package system, see Obj notes: post180956.html#p180956
pi@piaware ~ $ sudo rm /usr/lib/*sdr*
Install some requiered rtlsdr packages provided by Obj:
Here you can find the latest versions: https://github.com/mutability/librtlsdr/releases
pi@piaware ~ $ cd /tmp
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/librtlsdr0_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/librtlsdr-dev_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/rtl-sdr_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i librtlsdr0_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i librtlsdr-dev_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i rtl-sdr_0.5.4.git-1_armhf.deb
Install some more requiered packages:
pi@piaware /tmp $ cd
pi@piaware ~ $ sudo apt-get install librtlsdr-dev libusb-1.0-0-dev pkg-config debhelper
Update your system (optional, but recommanded):
pi@piaware ~ $ sudo apt-get update
pi@piaware ~ $ sudo apt-get upgrade
If you havenāt done this yet: Install the lighttpd web server. This is recommanded by Obj, because the internal dump1090 web server is disabled.
pi@piaware ~ $ sudo apt-get install lighttpd
If you havenāt done this yet: Install the git, the Github command line tool:
pi@piaware ~ $ sudo apt-get install git
pi@piaware ~ $ mkdir git
Clone my dump1090-mutibility fork from github and build it.
Note: If you clone https://github.com/mutability/dump1090.git in stead of https://github.com/tedsluis/dump1090.git you would build the original dump1090-mutability by obj.
pi@piaware ~ $ cd git
pi@piaware ~/git $ git clone https://github.com/tedsluis/dump1090.git
pi@piaware ~/git $ cd dump1090
pi@piaware ~/git/dump1090 $ dpkg-buildpackage -b
pi@piaware ~/git/dump1090 $ cd ..
pi@piaware ~/git $ ls -l dump*.deb
-rw-r--r-- 1 pi pi 340016 Sep 21 22:21 dump1090-mutability_1.15~dev_armhf.deb
Install the dump1090-mutibility package.
Note: You can copy this package to other raspberries and install it.
pi@piaware ~/git $ sudo dpkg -i dump1090-mutability_1.15~dev_armhf.deb
(Reading database ... 84072 files and directories currently installed.)
Preparing to replace dump1090-mutability 1.14 (using dump1090-mutability_1.15~dev_armhf.deb) ...
Unpacking replacement dump1090-mutability ...
Setting up dump1090-mutability (1.15~dev) ...
Make the map view available at http://<ip_address_your_raspberry>/dump1090/ using:
pi@piaware ~/git $ sudo lighty-enable-mod dump1090
pi@piaware ~/git $ sudo service lighttpd force-reload
This uses a configuration file installed by the package at /etc/lighttpd/conf-available/89-dump1090.conf.
Optional (but recommanded):
pi@piaware ~/git $ sudo reboot
Build & update dump1090-mutability from source:
Have you ever build dump1090 from source on this system before? Then follow these steps:
Go to your git directory and pull my dump1090-mutibility fork from github and build it.
pi@piaware ~ $ cd git/dump1090
pi@piaware ~/git/dump1090 $ git pull https://github.com/tedsluis/dump1090.git
pi@piaware ~/git/dump1090 $ dpkg-buildpackage -b
pi@piaware ~/git/dump1090 $ cd ..
pi@piaware ~/git $ ls -l dump*.deb
-rw-r--r-- 1 pi pi 340016 Sep 21 22:21 dump1090-mutability_1.15~dev_armhf.deb
Install the dump1090-mutibility package.
Note: You can copy this package to other raspberries and install it.
pi@piaware ~/git $ sudo dpkg -i dump1090-mutability_1.15~dev_armhf.deb
(Reading database ... 84072 files and directories currently installed.)
Preparing to replace dump1090-mutability 1.14 (using dump1090-mutability_1.15~dev_armhf.deb) ...
Unpacking replacement dump1090-mutability ...
Setting up dump1090-mutability (1.15~dev) ...
The installation script will find previous versions of files in /usr/share/dump1090-mutability/html. You need to choose which version you want. I advise you to choose for āinstall the package maintainerās versionā. The installation script will save backup copies from the previous version, so you never lose something.
Configuration file `/usr/share/dump1090-mutability/html/config.js'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
View your previous config.js.dpkg-old and edit your new config.js:
pi@piaware ~/git $ cd /usr/share/dump1090-mutability/html
pi@piaware /usr/share/dump1090-mutability/html $ ls -l config*
-rw-r--r-- 1 root root 5369 Sep 21 22:46 config.js
-rw-r--r-- 1 root root 1963 Aug 11 21:20 config.js.dpkg-old
pi@piaware /usr/share/dump1090-mutability/html $ cat config.js.dpkg-old
pi@piaware /usr/share/dump1090-mutability/html $ nano config.js
Replace updated files.
In a case that you want to apply some small changes in the ā/usr/share/dump1090-mutability/htmlā directory you can choose not to rebuild and reinstall dump1090 from source again (which takes a lot of time), but just move the old files to a backup folder and download the updated files from Github.
Create a backup directory:
pi@piaware ~ $ cd /usr/share/dump1090-mutability/html
pi@piaware /usr/share/dump1090-mutability/html $ sudo mkdir backup.20150923
Move the old files to the backup direcory, for example:
pi@piaware /usr/share/dump1090-mutability/html $ sudo mv config.js backup.20150923
pi@piaware /usr/share/dump1090-mutability/html $ sudo mv script.js backup.20150923
Download the updated files from Github:
pi@piaware /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/config.js
pi@piaware /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/script.js
note: The files config.js and script.js are only an example. Check github.com/tedsluis/dump1090/tr ā¦ ublic_html to see which files you need to update! Click on the āRawā button and copy the correct URLās.
note: If you are not sure about which files need to be updated, I sugest that you rebuild the dump1090-mutability from source (see the instruction above: āBuild & Install dump1090-mutability from sourceā).
Configuration dump1090-mutability
You can find the config file for dump1090-mutability in: ā/etc/default/dump1090-mutabilityā
With the comments inside it explains itself.
You can edit it using:
pi@piaware ~ $ sudo nano /etc/default/dump1090-mutability
You can also configure dump1090-mutability using a very practical wizard:
pi@piaware ~ $ sudo dpkg-reconfigure dump1090-mutability
After you finish the wirzard the config file is updated.
Note: dump1090-mutability is by default only reachable from the host it self. So, before you can watch GUI in a browser of an other host (using http://<ip_address_raspberry>/dump1090) you must change the NET_BIND_ADDRESS=ā127.0.0.1ā option in the config file like this:
# Bind ports on a particular address. If unset, binds to all interfaces.
# This defaults to binding to localhost. If you need to allow remote
# connections, change this.
#NET_BIND_ADDRESS="127.0.0.1"
NET_BIND_ADDRESS=""
After you have changed the /etc/default/dump1090-mutability config file you should restart the service dump1090:
pi@piaware ~ $ sudo service dump1090-mutability restart
ok ] Restarting dump1090-mutability daemon: dump1090-mutability.
The dump1090-mutability web GUI has it own config file: /usr/share/dump1090-mutability/html/config.js
Again this config file is pretty much self explaining.
Be sure you edit these settings to set your own location, unit type, etcā¦:
pi@piaware ~ $ sudo nano /usr/share/dump1090-mutability/html/config.js
Depending on the Metric you will see that altitude legend in feet or meter:
And the distance rings legend is kilometer or nautical mile:
From the config.js file you can also choose to display or not the file selector in the heatmap panel:
And you can configure the default setting of the [heatmap], [range], [colors] and [rings] buttons with in the config.js file:
By toggle the buttons you will change these config settings.
log file dump1090-mutability
Check the log file for any error messages:
pi@piaware ~ $ sudo nano /var/log/dump1090-mutability.log
Install or update dump1090.socket3003 scripts
This covers only three Perl scripts:
socket30003.pl
heatmap.pl
rangeview.pl
I have no āmakeā file or install script for this. You can just clone the files from Github.
I asume that you have installed āgitā, because you already needed it to build dump1090-mutability.
Install dump1090.socket3003 scripts
To install the dump1090.socket3003 Perl scripts, go to your git directory and clone my repo:
pi@piaware ~ $ cd ~/git
pi@piaware ~/git $ git clone https://github.com/tedsluis/dump1090.socket30003.git
pi@piaware ~/git $ ls -l dump1090.socket30003
-rwxr-xr-x 1 pi pi 14943 Sep 23 19:30 heatmap.pl
-rwxr-xr-x 1 pi pi 27290 Sep 23 19:30 socket30003.pl
-rwxr-xr-x 1 pi pi 25339 Sep 23 19:30 rangeview.pl
-rw-r--r-- 1 pi pi 15739 Sep 23 18:44 README.md
It is best not to use the files in this directory, because you probably going to changes your default settings inside the scripts. Copy the scripts to another directory from were you will run them. This git directory will be just used to get new updates in the future.
Create a directory where you run them:
pi@piaware ~/git $ cd
pi@piaware ~ $ mkdir sock30003
pi@piaware ~ $ cp git/dump1090.socket30003/*.pl sock30003/
Now remember: Your soscket30003 script launch directory is:** /home/pi/socket30003**
Update dump1090.socket3003 scripts
Use this step if you have already the dump1090.socket3003 scripts and you want to update them.
Before you update the dump1090.socket3003 Perl scripts, you may want to make a backup.
This way you can save your default settings you have made in the scripts.
pi@piaware ~ $ mkdir sock30003/backup
pi@piaware ~ $ cp sock30003/*.pl sock30003/backup/
Move to your git directory and pull the remote changes:
pi@piaware ~ $ cd git/dump1090.socket3003
pi@piaware ~/git/dump1090.socket3003 $ git pull https://github.com/tedsluis/dump1090.socket30003.git
Copy the new files to the directory where you run them:
pi@piaware ~/git/dump1090.socket3003 $ cd
pi@piaware ~ $ cp git/dump1090.socket30003/*.pl sock30003/
Configuration dump1090.socket3003 scripts
Each script has a few configuration settings in the upper section of the script.
Most configuration settings can also be passed on the script as a startup parameter.
Most important for all three scripts is to set your antenna location (lat/lon). And if you run these scripts on default raspberry (raspbian or piaware) then it works without any other options!
Change default settings
It is best to set you own settings (like your location, preferred unit types, preferred data directory) as default inside the scripts. Edit the scripts using:
pi@piaware ~ $ cd sock30003
pi@piaware ~/sock30003 $ nano socket30003.pl
pi@piaware ~/sock30003 $ nano heatmap.pl
pi@piaware ~/sock30003 $ nano rangeview.pl
Use startup parameters for configuration
Use the ā-helpā parameter to view the help text and the meaning of the startup parameters:
pi@piaware ~/sock30003 $ ./socket30003.pl -help
pi@piaware ~/sock30003 $ ./heatmap.pl -help
pi@piaware ~/sock30003 $ ./rangeview.pl -help
Running socket30003.pl and collecting flight position data
On a raspberry with dump1090 (what ever version) socket30003.pl automaticly connects with port 30003.
pi@ted1090-3 ~/sock30003 $ ./socket30003.pl &
Using the unit 'kilometer' for the distance and 'meter' for the altitude.
Trying to connect to peer host '127.0.0.1'...
The antenna latitude & longitude are: '52.085624','5.0890591'
Connected to '127.0.0.1', port 30003 (tcp).
The data directory/file is: /tmp/dump1090-127_0_0_1-150924.txt
The log directory/file is: /tmp/dump1090-127_0_0_1-150924.log
Starting 'socket30003.pl'....
'socket30003.pl' (4302) is started!
Using pidfile /tmp/dump1090-127_0_0_1.pid.
pi@piaware ~/sock30003 $
Note: the ā &ā at the end of the line launches the command in the background. This way you can use your terminal screen, while socket30003.pl is running as a background process. But the output of the script will still be displayed on the screen. This can be very annoying because the scripts frequently displays the number of positions processed. To avoid this you can launch the script with the option ā-nopositionsā.
If you want to collect ADS-B data from more then one raspberry, just launch socket30003.pl again as following (note that the ā-peer ā belongs to that other host):
pi@ted1090-3 ~/sock30003 $ ./socket30003.pl -peer 192.168.11.8 -nopositions &
Using the unit 'kilometer' for the distance and 'meter' for the altitude.
Trying to connect to peer host '192.168.11.8'...
The antenna latitude & longitude are: '52.085624','5.0890591'
Connected to '192.168.11.8', port 30003 (tcp).
The data directory/file is: /tmp/dump1090-192_168_11_8-150924.txt
The log directory/file is: /tmp/dump1090-192_168_11_8-150924.log
Starting 'socket30003.pl'....
'socket30003.pl' (6729) is started!
Using pidfile /tmp/dump1090-192_168_11_8.pid.
pi@piaware ~/sock30003 $
Run socket30003.pl from crontab
You can run socket30003.pl from the command line, but since this flight data collection have has to continue for several days, it is best to run it as a crontab job:
pi@piaware ~/sock30003 $ sudo crontab -e
And then add the line(s):
0 * * * * /home/pi/sock30003/socket30003.pl # connects to localhost.
0 * * * * /home/pi/sock30003/socket30003.pl -peer <ip-addres or hostname> # If you have more dump1090 running on another host.
This way it will check every hour if it is still running.
Watch out for a reboot!
One important note on the /tmp directory. In Raspbian and Piaware files /tmp will be lost after a reboot! There are two reasons why files will be lost after a reboot. In Piaware (unlike the Raspbian 3.18.11 version I use) /tmp is mounted on a small RAM disk. They other reason is that TMPTIME=0 by default. Be sure that you changes those settings before you start collecting data or store your flight position data files elsewhere! You can change the TMPTIME setting for /tmp using:
pi@piaware ~ $ sudo vi /etc/default/rcS
Set TMPTIME to 14 (or as much as you like) so that files in /tmp only will deleted if their modification time is more than 14 days ago
TMPTIME=14
And change the mount setting for /tmp using:
pi@piaware ~ $ sudo vi /etc/default/tmpfs
Set RAMTMP=no and reboot.
Log file socket30003.pl
In the /tmp directory you find a log file named ādump-<ip_address/hostname>-YYMMDD.logā . Every day a new one.
Inside you can see (besides errors) details about the number op planes and positions spotted:
pi@ted1090-3 ~/sock30003 $ tail -n 24 /tmp/dump1090-127_0_0_1-150924.log
current number of flights=216,epoch=2015-09-24/09:56:00,message_count=25937349,position_count=1567983,flight_count=2694.
removed:3C4B2F,first seen=2015-09-24/09:46:52,last seen=2015-09-24/09:53:47,message_count=529,position_count=3.
removed:49D025,first seen=2015-09-24/09:49:57,last seen=2015-09-24/09:53:14,message_count=120,position_count=21.
removed:491257,first seen=2015-09-24/09:43:29,last seen=2015-09-24/09:53:53,message_count=1037,position_count=42.
removed:4D2027,first seen=2015-09-24/09:53:31,last seen=2015-09-24/09:53:31,message_count=2,position_count=0.
removed:4BAA62,first seen=2015-09-24/09:48:06,last seen=2015-09-24/09:53:43,message_count=261,position_count=3.
removed:400BB2,first seen=2015-09-24/09:15:31,last seen=2015-09-24/09:53:54,message_count=25470,position_count=678.
removed:4009F4,first seen=2015-09-24/09:20:32,last seen=2015-09-24/09:53:20,message_count=13356,position_count=715.
removed:4B1789,first seen=2015-09-24/09:42:36,last seen=2015-09-24/09:53:25,message_count=636,position_count=19.
removed:4CA352,first seen=2015-09-24/09:19:46,last seen=2015-09-24/09:53:44,message_count=25330,position_count=1397.
current number of flights=214,epoch=2015-09-24/09:57:00,message_count=26017698,position_count=1572046,flight_count=2701.
removed:484F91,first seen=2015-09-24/09:49:00,last seen=2015-09-24/09:54:09,message_count=66,position_count=0.
removed:4CA9C4,first seen=2015-09-24/09:24:10,last seen=2015-09-24/09:54:52,message_count=25531,position_count=1395.
removed:44CDC3,first seen=2015-09-24/09:50:40,last seen=2015-09-24/09:54:55,message_count=71,position_count=0.
removed:484966,first seen=2015-09-24/09:34:00,last seen=2015-09-24/09:54:33,message_count=11782,position_count=746.
removed:ABBF84,first seen=2015-09-24/09:54:16,last seen=2015-09-24/09:54:16,message_count=4,position_count=0.
removed:4CA75C,first seen=2015-09-24/09:24:32,last seen=2015-09-24/09:54:41,message_count=8249,position_count=618.
removed:511065,first seen=2015-09-24/09:32:09,last seen=2015-09-24/09:54:13,message_count=7235,position_count=384.
current number of flights=214,epoch=2015-09-24/09:58:00,message_count=26096760,position_count=1576166,flight_count=2708.
removed:406673,first seen=2015-09-24/09:29:50,last seen=2015-09-24/09:55:44,message_count=10541,position_count=571.
removed:4840D5,first seen=2015-09-24/09:24:08,last seen=2015-09-24/09:55:00,message_count=12929,position_count=993.
removed:3C4DCE,first seen=2015-09-24/09:18:44,last seen=2015-09-24/09:55:01,message_count=30233,position_count=672.
removed:43E88A,first seen=2015-09-24/09:33:09,last seen=2015-09-24/09:55:28,message_count=7220,position_count=341.
removed:3C6449,first seen=2015-09-24/09:53:27,last seen=2015-09-24/09:55:35,message_count=6,position_count=0.
Every minute an update on how much planes are currently spotted and the number of messages, positions and flights counted that day.
Between that you can see which planes were spotted, how long they were spotted and how many messages and positions were received from that plane.
socket30003.pl output
By default the output of socket30003.pl will be written in /tmp:
pi@ted1090-3 ~ $ ls -ltr /tmp/dump*.txt
-rw-r--r-- 1 pi pi 685219803 Sep 20 23:59 /tmp/dump1090-127_0_0_1-150920.txt
-rw-r--r-- 1 pi pi 439486146 Sep 21 23:59 /tmp/dump1090-127_0_0_1-150921.txt
-rw-r--r-- 1 pi pi 448420915 Sep 22 23:59 /tmp/dump1090-127_0_0_1-150922.txt
-rw-r--r-- 1 pi pi 439160328 Sep 23 23:59 /tmp/dump1090-127_0_0_1-150923.txt
-rw-r--r-- 1 pi pi 42040864 Sep 24 19:46 /tmp/dump1090-127_0_0_1-150924.txt
-rw-r--r-- 1 pi pi 766153 Sep 24 19:46 /tmp/dump1090-192_168_11_8-150924.txt
The comma separated values have a column headers. The Altitude and distance columns have their unit type between parentheses. This way the other scripts (heatmap.pl and rangeview.pl) knows what unit type is used. Each time the socket30003.pl script is restarted (may be with different unit types) it writes a new column header in to the file, so heatmap.pl and rangeview.pl always use the correct unit types:
pi@ted1090-3 ~/sock30003 $ head /tmp/dump1090-127_0_0_1-150924.txt
hex_ident,altitude(meter),latitude,longitude,date,time,angle,distance(kilometer)
780DA3,11270,52.46353,6.58246,2015/09/24,17:17:19.356,75.13,109.94
40665F,11575,51.47197,1.76338,2015/09/24,17:17:19.358,-100.74,238.71
4068F5,11270,52.28342,3.84286,2015/09/24,17:17:19.358,-80.53,87.75
40659A,5559,51.66609,6.28648,2015/09/24,17:17:19.360,110.01,94.5
400944,8529,51.41739,2.21397,2015/09/24,17:17:19.360,-103.49,211.37
484F7F,4241,51.47928,4.55658,2015/09/24,17:17:19.360,-139.87,76.72
89641E,12192,51.54172,6.77641,2015/09/24,17:17:19.361,108.5,130.8
4B0F59,7227,51.38741,7.11357,2015/09/24,17:17:19.361,109.68,159.55
4008DE,10966,50.97148,5.06633,2015/09/24,17:17:19.361,-178.87,123.89
Many positions per second. It can be as many as 5 million positions a day:
pi@ted1090-3 ~/sock30003 $ wc -l /tmp/dump1090-127_0_0_1-150922.txt
5077255 /tmp/dump1090-127_0_0_1-150922.txt
This is of course depending on your location and antenna.
Creating a heatmap
If you have collected enough data you can create a heatmap:
pi@ted1090-3 ~/sock30003 $ ./heatmap.pl
There will be no more then '100000' positions in the output file.
The maximum position weight on the heatmap will be not more then '1000'.
The resolution op the heatmap will be 1000x1000.
The antenna latitude & longitude are: '52.085','5.089'.
The heatmap will cover the area of 5 degree around the antenna, which is between latitude 47 - 57 and longitude 0 - 10.
The following files fit with the filemask '*dump1090-*':
/tmp/dump1090-192_168_11_8-150924.txt
/tmp/dump1090-127_0_0_1-150924.txt
Processing file '/tmp/dump1090-192_168_11_8-150924.txt':
-header units:altitude=meter,distance=kilometer, position 1- 11435 processed.
-header units:altitude=meter,distance=kilometer, position 11436- 11628 processed.
-header units:altitude=meter,distance=kilometer, position 11629-42744 processed. 15 positions were out side the specified area.
Processing file '/tmp/dump1090-127_0_0_1-150924.txt':
-header units:altitude=meter,distance=kilometer, position 1- 2476 processed.
-header units:altitude=meter,distance=kilometer, position 2477-1052595 processed. 162 positions were out side the specified area.
Number of sorted positions: 162410
The highest weight is '00001080' and the lowest weight is '00000003'.
Since the highest weight is more the the max weight '1000' the weight of all points will be multiplied with a factor 0.925925925925926.
Output file with java script code: /tmp/heatmapcode.csv
{location: new google.maps.LatLng(52.265, 4.699), weight: 1001},
{location: new google.maps.LatLng(52.275, 4.709), weight: 883},
{location: new google.maps.LatLng(52.275, 4.719), weight: 815},
{location: new google.maps.LatLng(52.255, 4.679), weight: 606},
{location: new google.maps.LatLng(52.245, 4.679), weight: 531},
100000 rows with heatmap position data processed!
{location: new google.maps.LatLng(50.525, 5.299), weight: 6},
{location: new google.maps.LatLng(50.525, 5.019), weight: 6},
{location: new google.maps.LatLng(50.525, 4.999), weight: 6},
{location: new google.maps.LatLng(50.525, 4.989), weight: 6},
{location: new google.maps.LatLng(50.525, 4.949), weight: 6},
Output file in csv format: /tmp/heatmapdata.csv
"weight";"lat";"lon"
"1001";"52.265";"4.699"
"883";"52.275";"4.709"
"815";"52.275";"4.719"
"606";"52.255";"4.679"
100000 rows with heatmap position data processed!
"6";"50.525";"5.299"
"6";"50.525";"5.019"
"6";"50.525";"4.999"
"6";"50.525";"4.989"
"6";"50.525";"4.949"
The first 5 and the last 5 lines of the output files are shown.
There are all kinds of options you can try to adjust the heatmap.
Now copy /tmp/heatmapdata.csv to the /usr/share/dump1090-mutability/html directory to display it in the dump1090-mutability GUI
pi@ted1090-3 ~/sock30003 $ sudo cp /tmp/heatmapdata.csv /usr/share/dump1090-mutability/html
And reload your browser!
You can also copy the heatmapdata.csv file to the host where you run your browser and select the file from the heatmap panel in the dump1090-mutability GUI.
May be you need to toggle the [heatmap] button on the sidebar to get the heatmap visible.
Be sure you have enough flight positions data collected. Here are two example, where you can see the difference:
81.525.440 million flight positions (more then 2 weeks of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/heatmapdata-example1.csv
If you zoom in on this you can see many details.
3.500.000 million flight positions (less then 1 day of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/heatmapdata-example2.csv
Could be much better. If you zoom in on this you get not more detailsā¦
Creating a range/altitude view
For a good range/altitude view you need a lot of data:
pi@ted1090-3 ~/sock30003 $ ./rangeview.pl
The altitude will be converted from 'meter' to 'meter'.
The distance will be converted from 'kilometer' to 'kilometer.
The maxium altitude is 12000 meter.
The minimal altitude is 0 meter.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 24.
The latitude/longitude location of the antenna is: 52.085624,5.0890591.
An altitude zone is 521 meter.
The following files fit with the filemask '*dump1090-*':
/tmp/dump1090-192_168_11_8-150924.txt
/tmp/dump1090-127_0_0_1-150924.txt
processing '/tmp/dump1090-192_168_11_8-150924.txt':
-header units:altitude=meter,distance=kilometer, position 1- 11435 processed.
-header units:altitude=meter,distance=kilometer, position 11436- 11628 processed.
-header units:altitude=meter,distance=kilometer, position 11629-42744. processed.
processing '/tmp/dump1090-127_0_0_1-150924.txt':
-header units:altitude=meter,distance=kilometer, position 1- 2476 processed.
-header units:altitude=meter,distance=kilometer, position 2477-1090039. processed.
Number of files read: 2
Number of position processed: 1132780 and positions within range processed: 1132780
datafile= /tmp/rangeview.csv
kmlfile= /tmp/rangeview.kml
1,Altitude zone: 0- 520,Directions: 573/ 1440,Positions processed: 10969,Positions processed per direction: min: 2,max: 290,avg: 7,real avg: 19
2,Altitude zone: 521- 1041,Directions: 731/ 1440,Positions processed: 34222,Positions processed per direction: min: 1,max: 384,avg: 23,real avg: 46
3,Altitude zone: 1042- 1562,Directions: 903/ 1440,Positions processed: 26485,Positions processed per direction: min: 1,max: 460,avg: 18,real avg: 29
4,Altitude zone: 1563- 2083,Directions: 1167/ 1440,Positions processed: 24151,Positions processed per direction: min: 8,max: 398,avg: 16,real avg: 20
5,Altitude zone: 2084- 2604,Directions: 1178/ 1440,Positions processed: 27824,Positions processed per direction: min: 13,max: 544,avg: 19,real avg: 23
6,Altitude zone: 2605- 3125,Directions: 1132/ 1440,Positions processed: 23868,Positions processed per direction: min: 4,max: 290,avg: 16,real avg: 21
7,Altitude zone: 3126- 3646,Directions: 1267/ 1440,Positions processed: 25941,Positions processed per direction: min: 10,max: 203,avg: 18,real avg: 20
8,Altitude zone: 3647- 4167,Directions: 1333/ 1440,Positions processed: 22101,Positions processed per direction: min: 5,max: 133,avg: 15,real avg: 16
9,Altitude zone: 4168- 4688,Directions: 1274/ 1440,Positions processed: 21142,Positions processed per direction: min: 12,max: 124,avg: 14,real avg: 16
10,Altitude zone: 4689- 5209,Directions: 1262/ 1440,Positions processed: 23113,Positions processed per direction: min: 8,max: 106,avg: 16,real avg: 18
11,Altitude zone: 5210- 5730,Directions: 1239/ 1440,Positions processed: 21932,Positions processed per direction: min: 24,max: 184,avg: 15,real avg: 17
12,Altitude zone: 5731- 6251,Directions: 1279/ 1440,Positions processed: 25954,Positions processed per direction: min: 17,max: 249,avg: 18,real avg: 20
13,Altitude zone: 6252- 6772,Directions: 1383/ 1440,Positions processed: 30274,Positions processed per direction: min: 17,max: 515,avg: 21,real avg: 21
14,Altitude zone: 6773- 7293,Directions: 1345/ 1440,Positions processed: 30132,Positions processed per direction: min: 23,max: 341,avg: 20,real avg: 22
15,Altitude zone: 7294- 7814,Directions: 1312/ 1440,Positions processed: 30875,Positions processed per direction: min: 13,max: 466,avg: 21,real avg: 23
16,Altitude zone: 7815- 8335,Directions: 1428/ 1440,Positions processed: 31769,Positions processed per direction: min: 11,max: 492,avg: 22,real avg: 22
17,Altitude zone: 8336- 8856,Directions: 1412/ 1440,Positions processed: 36847,Positions processed per direction: min: 9,max: 304,avg: 25,real avg: 26
18,Altitude zone: 8857- 9377,Directions: 1291/ 1440,Positions processed: 37239,Positions processed per direction: min: 20,max: 269,avg: 25,real avg: 28
19,Altitude zone: 9378- 9898,Directions: 1438/ 1440,Positions processed: 47712,Positions processed per direction: min: 44,max: 450,avg: 33,real avg: 33
20,Altitude zone: 9899- 10419,Directions: 1440/ 1440,Positions processed: 87625,Positions processed per direction: min: 80,max: 548,avg: 60,real avg: 60
21,Altitude zone: 10420- 10940,Directions: 1440/ 1440,Positions processed: 86764,Positions processed per direction: min: 49,max: 608,avg: 60,real avg: 60
22,Altitude zone: 10941- 11461,Directions: 1440/ 1440,Positions processed: 245968,Positions processed per direction: min: 170,max: 929,avg: 170,real avg: 170
23,Altitude zone: 11462- 11982,Directions: 1440/ 1440,Positions processed: 137923,Positions processed per direction: min: 101,max: 581,avg: 95,real avg: 95
24,Altitude zone: 11983- 12503,Directions: 1440/ 1440,Positions processed: 41950,Positions processed per direction: min: 32,max: 169,avg: 29,real avg: 29
In the progress report above it gets clear that there are not enough positions. For some āaltitude zoneā and some ādirection zonesā there are no positions spotted! My advise: collect more data
Now copy the /tmp/rangeview.kml file to a web server. Use a public folder on Dropbox or Google Drive if you donāt have a public accessible web server available.
Fill the correct path in the /usr/share/dump1090-mutability/html/config.js:
// KML to show:
UserMap = "https://github.com/tedsluis/dump1090/raw/master/exampledata/radar.kml"; // your KML URL (publicly available domain)
And reload your browser!
May be you need to toggle the [range] button on the sidebar to get the range/altitude view visible.
Here are two example, where you can see the difference:
81.525.440 million flight positions (more then 2 weeks of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/rangeview-example1.kml
Looks quite good.
3.500.000 million flight positions (less then 1 day of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/rangeview-example1.kml
Looking poorly. Too little dataā¦
To be continued in the next daysā¦
Any feedback is welcome!
Ted
http://flightaware.com/adsb/stats/user/tedsluis#stats-6731
Resources:
On dump1090-mutability:
- https://github.com/mutability/dump1090
- https://github.com/tedsluis/dump1090
-
http://discussions.flightaware.com/ads-b-flight-tracking-f21/what-is-easiest-way-to-install-dump1090-mutability-t35431.html
On ADB-S 30003 data: - http://www.atiner.gr/journals/technology/2014-1-X-Y-Klauza.pdf
-
http://www.virtualradarserver.co.uk/Documentation/Formats/CompressedVrs.aspx
On Google Maps Heatmaps: -
https://developers.google.com/maps/documentation/javascript/heatmaplayer
On parsing csv file in javascript: -
http://papaparse.com
On KML files: - https://developers.google.com/kml/documentation
- http://www.kmlvalidator.com
-
http://www.opengeospatial.org/standards/kml
On hosted libraries: -
https://developers.google.com/speed/libraries
HSL colors: - http://www.w3schools.com/cssref/css_colors_legal.asp
- http://stackoverflow.com/questions/2353211/hsl-to-rgb-color-conversion
- http://stackoverflow.com/questions/19996703/how-to-create-color-gradation-for-html-shape-based-on-scale-of-values-using-perl
