Heatmap & range/altitude view for dump1090-mutability v1.15

This aims to be a guide on how to get a heatmap and range/altitude view in dump1090-mutability (v1.15 dev). Down here I have put together all the important stuff, like:

  • Working features.
  • Known issues.
  • Future wishes.
  • Change log.
  • Overview: How does it work.
  • Screenshots, video and live view (try dump1090-mutability with heatmap and rangeview yourself)
  • Installation and upgrade steps dump1090-mutability v1.15 dev.
  • Installation and upgrade steps socket30003 scripts.
  • Configuration.
  • Resources.

Thanks
First of all I want to thank Oliver Jowett -also known as Obj- for his great work on dump1090-mutability. Without his skills and dedication, we could not enjoy his amazing dump1090 version!

What is working:
Available features in the socket30003 scripts (https://github.com/tedsluis/dump1090.socket30003)

  • Collecting ADB-S data, saved in csv format (latitude, longitude, altitude, distance ,etc)
  • Support for collecting ADB-S data from multiple dump1090 instances simultaneously on one host.
  • Possibility to convert the altitude unit type to meter or feet and the distance unit type to meter, feet, kilometer, mile or nautical mile.
  • Possiblilty to specify the data directory, antenna location, etcā€¦

Available features in my dump1090-mutability fork (https://github.com/tedsluis/dump1090):

  • Display a csv heatmap file.
  • Adjust the opacity, intensity and radius of the heatmap from a movable panel.
  • Load a heatmap from the dump1090 web directory or from the heatmap panel in the browser.
  • Display a KML range/altitude file.
  • Display distance range rings around the antenna.
  • Provide movable legends for the altitude colors and range rings.
  • Toggle plane colors between Altitude colors and adb-s/mlat position colors.
  • Toggle the heatmap, the range/altitude view and the range rings on and off (including their panel and legends).
  • And of course all the functionality that comes along with the original dump1090-mutability v1.15 dev by Obj.

Known issues:

  • Not used the correct colors in range/altitude KML file. (This is probably caused by incorrect hsl to rbg color conversion.)

Future wishes:

  • Toggle on and off individual range/altitude lines.
  • One common config file for the dump1090.socket3003 scripts. (the config setting are now stored within the scripts).
  • File-system utilization monitoring to prevent that the script output writes the file-system full.

Live view
You can try my dump1090 fork with heatmap and rangeview in the Google cloud: **http://130.211.110.109/dump1090/gmap.html **(This dump1090 runs on a 60-day free trail that is available until September 6 2016, more info at https://cloud.google.com/free-trial/)

Screen shots & video
I have made a 3 minute YouTube video that shows how it works: https://www.youtube.com/watch?v=Qz4XSFRjLTI

https://github.com/tedsluis/dump1090/raw/master/img/2015-09-20-20-35-45-ANIMATION.gif

Change log https://github.com/tedsluis/dump1090.socket30003:
(commit log on Github: https://github.com/tedsluis/dump1090.socket30003/commits/master)
2017-01-10: Major update

  • Fix tcp 30003 connection drop issue. Implemented new socket methode.
  • Handles messages using the common log file routine.
  • Save settings in config file.
  • Added ā€˜-debugā€™ and ā€˜verboseā€™ logging options.
  • Changed section radarview to rangeview.
  • Bugfixes.
  • Updated helptext
    2015-12-17
  • Improved logging for large amount of incomplete messages.
  • Fixed error 'Argument isnā€™t numeric in division (/) in line 644.
    2015-09-27
  • Fixed issue in heatmap.pl and rangview.pl on finding the input files.
    2015-09-24:
  • Renamed dump1090.socket3003 scripts to socket30003.pl, heatmap.pl and rangeview.pl.
  • Renamed radar.kml to rangview.kml
  • Renamed socket30003.pl output files to dump1090-<ip_address/hostname>-YYMMDD.txt
  • Update help textā€™s.
  • note: more detail at: http://discussions.flightaware.com/post181471.html#p181471
    2015-09-20 :
  • Set default max altitude in meter and in feet.
  • First Altitude zone includes everything below and the last altitude zone includes everything above.
    2015-09-19:
  • Added heatmap output in svc format.
    2015-09-17:
  • Added range/altitude view output in KML format.
  • Changed altitude colors to hsl format.
    2015-09-06:
  • Added support for position data with column headers and unit types for altitude and distance.
  • Improved progress messages.
    2015-09-03:
  • Fixed flipped over latitude and longitude (credits for DimoM).
    2015-09-02:
  • Fixed handling negative latitude and longitude.
  • Support for meter, feet, mile, nautical mile and kilometer.
    2015-08-30:
  • Initial version.

Change log https://github.com/tedsluis/dump1090:
(commit log on GitHub: https://github.com/tedsluis/dump1090/commits/master)
2017-01-10:

  • Bugfixes.
    2016-02-20:
  • Merged upstream commits from github.com/mutability/dump1090
  • Added screenshots & video to the README.md
    2015-09-24:
  • Renamed radar.kml to rangeview.kml in config.js.
    2015-09-21:
  • Added support for toggling legends.
  • Added support for loading a heatmap from the dump1090 web directory.
  • Added support for toggling plane colors between Altitude and mlat/non-mlat colors.
  • Replace a local jquery-2.1.1.min.js for a downloadable version.
    2015-09-20:
  • Added moveable altitude legend and range legend. Added toggle buttons for heatmap, range, plane colors and rings.
  • Added config items for Heatmap, altitude legend and range rings legend.
  • Added buttons for toggling the heatmap, range/altitude view, colors, rings in the side bar (credits for Toverfield!).
  • Altitude and range legend added (credits for Toverfield!).
  • Added KML URL to config.js.
  • Added KML layer support (credits for Starksk!).
  • Added heatmap support and movable heatmap panel.

An overview: How does it work:
A heatmap is based on flight positions gathered by dump1090. If there are planes in a particular area, this area become a heat point on the heatmap. Many planes in an area will give that particular heat point more ā€˜weightā€™ . More ā€˜weightā€™ means a more intense color of that heat point on the map. The best way is the show this in an picture:

By default square area (of -/+ 5 degree lon/lat) around the antenna is divided in a matrix of 1000 x 1000 heatmap point. Of course these parameters are adjustable.

Same as a heatmap, the range/altitude view is based on gathered flight positions. We want to see how far our antenna reaches for a particular altitude. Therefor there are altitude zones defined. By default an altitude is 500 meter or 1500 feet. And by default there are 24 altitude zones defined. The main goal is to determine a maximum antenna range for every altitude zone in every direction. To achieve this the distance from the antenna to every captured flight position must be calculated. And also the direction of the flight positions compared to our antenna must be determined. Therefor there are direction zones defined, by default 1400 directions. Based on the direction a flight position belongs to a particular direction zone. Now it is possible to determine which flight position, in a particular altitude zone and direction zone, is the one that is most far away. Again, the best way is to show this with a picture:

Flight position data can be collect from dump1090. Any dump1090 (the Piaware version and the Mutability version) share ADB-S data with flight position information on port 30003. The script socket30003.pl collects the ADB-S data and saves it in csv format.

Later, after capturing flight position data for a couple of days, it is possible to create a heatmap and a range/altitude view based of the flight position data saved in the csv files. The heatmap.pl script creates a heatmapdata.csv file. And the rangeview.pl script creates a rangeview.kml file. They are saved in /tmp by default.

A modified fork of dump1090-mutability is able to load the heatmapdata.csv from /usr/share/dump1090-mutability/html directory or from the dump1090 GUI. And the rangeview.kml can be displayed while it is loaded from a public accessible web server (if you donā€™t have that you can just use cloud storage like Github or Google drive). This is necessary because Google Maps API needs to have access, because it will render the KML file in the cloud.

So the whole solution is two different applications which can be used together or separately. You can use the socket30003.pl, heatmap.pl and rangview.pl scripts to create a heatmap and rangeview KML file. You can use the modified dump1090-mutability to display these heatmap file and range/altitude view file. Or you can display the heatmap and range/altitude view in an other tool. And you even can create a heatmap and range/altitude view in an other way and display them using modified dump1090-mutability. I will show you what other tools you can use to display the heatmap and range/altitude view, later.

Steps to take.
What do you need to do to get a heatmap and a range/altitude view:

  • Install my socket30003 Perl scripts (socket30003.pl, heatmap.pl and rangeview.pl) from GitHub.
  • Install my fork of dump1090-mutability from GitHub.
  • Run socket30003.pl for a couple of days to collect flight position data.
  • After a couple of days run heatmap.pl and rangeview.pl
  • Make the output files heatmapdata.csv and rangeview.kml available to dump1090-mutability.

These steps are described below.

Install or update dump1090-mutability v1.15 dev.
Dump1090-mutability v1.15 is still under development by Obj. The only way to get it is to build it yourself from sources. Down here is explained how you can do that.

Ones you have build and install dump1090-mutability you can find the web server files in: ā€œ/usr/share/dump1090-mutability/htmlā€.
Here are the files located that I have changed and added in order to get the heatmap and range/altitude view.

When you want to commit an update you can either rebuild and reinstall dump1090 or you can only replace the updated files.

So you have three options:

  • Build & Install
  • Build & update
  • Replace updated files (apply a small update).

Note: This installation instructions are valid for Raspbian (the default image for Raspberry) as well as Piaware (the Flightaware version for Raspberry).

**Do you still have the dump1090 version running, supplied by Flightaware? **
In order to get the dump1090-mutability, the dump1090 version supplied by Flightaware must be stopped. But donā€™t warry, you Piaware will run just fine together with dump1090-mutability.

First check whether fadump1090 is running using:


pi@piaware ~ $ sudo  /etc/init.d/fadump1090.sh status

Then stop it using:


pi@piaware ~ $ sudo /etc/init.d/fadump1090.sh stop
pi@piaware ~ $ sudo /etc/init.d/fadump1090.sh status

Then remove the execute permissions from the service:


pi@piaware ~ $ sudo chmod 644 /etc/init.d/fadump1090.sh

Now it wonā€™t launch again after a reboot :wink:

Build & Install dump1090-mutability from source:
Is it the first time that you build dump1090 from source on this system? Then follow these steps:

Note: This instruction follows exacly the same methode as provided in the github.com/mutability/dump1090/ ā€¦ /README.md by Obj. I just show all the steps and give some extra comments.

At this moment on Piaware (not on Raspbian images) you need to remove some librtlsdr packages that are unknown to the package system, see Obj notes: post180956.html#p180956


pi@piaware ~ $ sudo rm  /usr/lib/*sdr* 

Install some requiered rtlsdr packages provided by Obj:
Here you can find the latest versions: https://github.com/mutability/librtlsdr/releases


pi@piaware ~ $ cd /tmp
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/librtlsdr0_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/librtlsdr-dev_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ wget https://github.com/mutability/librtlsdr/releases/download/v0.5.4_git-1/rtl-sdr_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i librtlsdr0_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i librtlsdr-dev_0.5.4.git-1_armhf.deb
pi@piaware /tmp $ sudo dpkg -i rtl-sdr_0.5.4.git-1_armhf.deb

Install some more requiered packages:


pi@piaware /tmp $ cd
pi@piaware ~ $ sudo apt-get install librtlsdr-dev libusb-1.0-0-dev pkg-config debhelper

Update your system (optional, but recommanded):


pi@piaware ~ $ sudo apt-get update
pi@piaware ~ $ sudo apt-get upgrade

If you havenā€™t done this yet: Install the lighttpd web server. This is recommanded by Obj, because the internal dump1090 web server is disabled.


pi@piaware ~ $ sudo apt-get install lighttpd       

If you havenā€™t done this yet: Install the git, the Github command line tool:


pi@piaware ~ $ sudo apt-get install git
pi@piaware ~ $ mkdir git

Clone my dump1090-mutibility fork from github and build it.
Note: If you clone https://github.com/mutability/dump1090.git in stead of https://github.com/tedsluis/dump1090.git you would build the original dump1090-mutability by obj.


pi@piaware ~ $ cd git
pi@piaware ~/git $ git clone https://github.com/tedsluis/dump1090.git
pi@piaware ~/git $ cd dump1090
pi@piaware ~/git/dump1090 $ dpkg-buildpackage -b
pi@piaware ~/git/dump1090 $ cd ..
pi@piaware ~/git $ ls -l dump*.deb
-rw-r--r-- 1 pi   pi   340016 Sep 21 22:21 dump1090-mutability_1.15~dev_armhf.deb

Install the dump1090-mutibility package.
Note: You can copy this package to other raspberries and install it.


pi@piaware ~/git $ sudo dpkg -i dump1090-mutability_1.15~dev_armhf.deb
(Reading database ... 84072 files and directories currently installed.)
Preparing to replace dump1090-mutability 1.14 (using dump1090-mutability_1.15~dev_armhf.deb) ...
Unpacking replacement dump1090-mutability ...
Setting up dump1090-mutability (1.15~dev) ...

Make the map view available at http://<ip_address_your_raspberry>/dump1090/ using:


pi@piaware ~/git $ sudo lighty-enable-mod dump1090
pi@piaware ~/git $ sudo service lighttpd force-reload

This uses a configuration file installed by the package at /etc/lighttpd/conf-available/89-dump1090.conf.

Optional (but recommanded):


pi@piaware ~/git $ sudo reboot

Build & update dump1090-mutability from source:
Have you ever build dump1090 from source on this system before? Then follow these steps:

Go to your git directory and pull my dump1090-mutibility fork from github and build it.


pi@piaware ~ $ cd git/dump1090
pi@piaware ~/git/dump1090 $ git pull https://github.com/tedsluis/dump1090.git
pi@piaware ~/git/dump1090 $ dpkg-buildpackage -b
pi@piaware ~/git/dump1090 $ cd ..
pi@piaware ~/git $ ls -l dump*.deb
-rw-r--r-- 1 pi   pi   340016 Sep 21 22:21 dump1090-mutability_1.15~dev_armhf.deb

Install the dump1090-mutibility package.
Note: You can copy this package to other raspberries and install it.


pi@piaware ~/git $ sudo dpkg -i dump1090-mutability_1.15~dev_armhf.deb
(Reading database ... 84072 files and directories currently installed.)
Preparing to replace dump1090-mutability 1.14 (using dump1090-mutability_1.15~dev_armhf.deb) ...
Unpacking replacement dump1090-mutability ...
Setting up dump1090-mutability (1.15~dev) ...

The installation script will find previous versions of files in /usr/share/dump1090-mutability/html. You need to choose which version you want. I advise you to choose for ā€œinstall the package maintainerā€™s versionā€. The installation script will save backup copies from the previous version, so you never lose something.


Configuration file `/usr/share/dump1090-mutability/html/config.js'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.

View your previous config.js.dpkg-old and edit your new config.js:


pi@piaware ~/git $ cd /usr/share/dump1090-mutability/html
pi@piaware /usr/share/dump1090-mutability/html $ ls -l config*
-rw-r--r-- 1 root root 5369 Sep 21 22:46 config.js
-rw-r--r-- 1 root root 1963 Aug 11 21:20 config.js.dpkg-old
pi@piaware /usr/share/dump1090-mutability/html $ cat config.js.dpkg-old
pi@piaware /usr/share/dump1090-mutability/html $ nano config.js

Replace updated files.
In a case that you want to apply some small changes in the ā€œ/usr/share/dump1090-mutability/htmlā€ directory you can choose not to rebuild and reinstall dump1090 from source again (which takes a lot of time), but just move the old files to a backup folder and download the updated files from Github.

Create a backup directory:


pi@piaware ~ $ cd /usr/share/dump1090-mutability/html
pi@piaware /usr/share/dump1090-mutability/html $ sudo mkdir backup.20150923

Move the old files to the backup direcory, for example:


pi@piaware /usr/share/dump1090-mutability/html $ sudo mv config.js backup.20150923
pi@piaware /usr/share/dump1090-mutability/html $ sudo mv script.js backup.20150923

Download the updated files from Github:


pi@piaware /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/config.js
pi@piaware /usr/share/dump1090-mutability/html $ sudo wget https://raw.githubusercontent.com/tedsluis/dump1090/master/public_html/script.js

note: The files config.js and script.js are only an example. Check github.com/tedsluis/dump1090/tr ā€¦ ublic_html to see which files you need to update! Click on the ā€œRawā€ button and copy the correct URLā€™s.

note: If you are not sure about which files need to be updated, I sugest that you rebuild the dump1090-mutability from source (see the instruction above: ā€œBuild & Install dump1090-mutability from sourceā€).

Configuration dump1090-mutability
You can find the config file for dump1090-mutability in: ā€œ/etc/default/dump1090-mutabilityā€
With the comments inside it explains itself.
You can edit it using:


pi@piaware ~ $ sudo nano /etc/default/dump1090-mutability

You can also configure dump1090-mutability using a very practical wizard:


pi@piaware ~ $ sudo dpkg-reconfigure dump1090-mutability

After you finish the wirzard the config file is updated.

Note: dump1090-mutability is by default only reachable from the host it self. So, before you can watch GUI in a browser of an other host (using http://<ip_address_raspberry>/dump1090) you must change the NET_BIND_ADDRESS=ā€œ127.0.0.1ā€ option in the config file like this:


# Bind ports on a particular address. If unset, binds to all interfaces.
# This defaults to binding to localhost. If you need to allow remote
# connections, change this.
#NET_BIND_ADDRESS="127.0.0.1"
NET_BIND_ADDRESS=""

After you have changed the /etc/default/dump1090-mutability config file you should restart the service dump1090:


pi@piaware ~ $ sudo service dump1090-mutability restart
 ok ] Restarting dump1090-mutability daemon: dump1090-mutability.

The dump1090-mutability web GUI has it own config file: /usr/share/dump1090-mutability/html/config.js
Again this config file is pretty much self explaining.
Be sure you edit these settings to set your own location, unit type, etcā€¦:


pi@piaware ~ $ sudo nano /usr/share/dump1090-mutability/html/config.js

Depending on the Metric you will see that altitude legend in feet or meter:

And the distance rings legend is kilometer or nautical mile:

From the config.js file you can also choose to display or not the file selector in the heatmap panel:

And you can configure the default setting of the [heatmap], [range], [colors] and [rings] buttons with in the config.js file:

By toggle the buttons you will change these config settings.

log file dump1090-mutability
Check the log file for any error messages:


pi@piaware ~ $ sudo nano /var/log/dump1090-mutability.log

Install or update dump1090.socket3003 scripts
This covers only three Perl scripts:
socket30003.pl
heatmap.pl
rangeview.pl

I have no ā€œmakeā€ file or install script for this. You can just clone the files from Github.
I asume that you have installed ā€œgitā€, because you already needed it to build dump1090-mutability.

Install dump1090.socket3003 scripts
To install the dump1090.socket3003 Perl scripts, go to your git directory and clone my repo:


pi@piaware ~ $ cd ~/git
pi@piaware ~/git $ git clone https://github.com/tedsluis/dump1090.socket30003.git
pi@piaware ~/git $ ls -l dump1090.socket30003
-rwxr-xr-x 1 pi pi 14943 Sep 23 19:30 heatmap.pl
-rwxr-xr-x 1 pi pi 27290 Sep 23 19:30 socket30003.pl
-rwxr-xr-x 1 pi pi 25339 Sep 23 19:30 rangeview.pl
-rw-r--r-- 1 pi pi 15739 Sep 23 18:44 README.md

It is best not to use the files in this directory, because you probably going to changes your default settings inside the scripts. Copy the scripts to another directory from were you will run them. This git directory will be just used to get new updates in the future.

Create a directory where you run them:


pi@piaware ~/git $ cd 
pi@piaware ~ $ mkdir sock30003
pi@piaware ~ $ cp git/dump1090.socket30003/*.pl  sock30003/

Now remember: Your soscket30003 script launch directory is:** /home/pi/socket30003**

Update dump1090.socket3003 scripts
Use this step if you have already the dump1090.socket3003 scripts and you want to update them.
Before you update the dump1090.socket3003 Perl scripts, you may want to make a backup.
This way you can save your default settings you have made in the scripts.


pi@piaware ~ $ mkdir sock30003/backup
pi@piaware ~ $ cp sock30003/*.pl sock30003/backup/

Move to your git directory and pull the remote changes:


pi@piaware ~ $ cd git/dump1090.socket3003
pi@piaware ~/git/dump1090.socket3003 $  git pull https://github.com/tedsluis/dump1090.socket30003.git

Copy the new files to the directory where you run them:


pi@piaware ~/git/dump1090.socket3003 $ cd
pi@piaware ~ $ cp git/dump1090.socket30003/*.pl  sock30003/

Configuration dump1090.socket3003 scripts
Each script has a few configuration settings in the upper section of the script.
Most configuration settings can also be passed on the script as a startup parameter.

Most important for all three scripts is to set your antenna location (lat/lon). And if you run these scripts on default raspberry (raspbian or piaware) then it works without any other options!

Change default settings
It is best to set you own settings (like your location, preferred unit types, preferred data directory) as default inside the scripts. Edit the scripts using:


pi@piaware ~ $ cd sock30003
pi@piaware ~/sock30003 $ nano socket30003.pl
pi@piaware ~/sock30003 $ nano heatmap.pl
pi@piaware ~/sock30003 $ nano rangeview.pl

Use startup parameters for configuration
Use the ā€œ-helpā€ parameter to view the help text and the meaning of the startup parameters:


pi@piaware ~/sock30003 $ ./socket30003.pl -help
pi@piaware ~/sock30003 $ ./heatmap.pl -help
pi@piaware ~/sock30003 $ ./rangeview.pl -help

Running socket30003.pl and collecting flight position data
On a raspberry with dump1090 (what ever version) socket30003.pl automaticly connects with port 30003.


pi@ted1090-3 ~/sock30003 $ ./socket30003.pl  &
Using the unit 'kilometer' for the distance and 'meter' for the altitude.
Trying to connect to peer host '127.0.0.1'...
The antenna latitude & longitude are: '52.085624','5.0890591'
Connected to '127.0.0.1', port 30003 (tcp).
The data directory/file is: /tmp/dump1090-127_0_0_1-150924.txt
The log  directory/file is: /tmp/dump1090-127_0_0_1-150924.log
Starting 'socket30003.pl'....
'socket30003.pl' (4302) is started!
Using pidfile /tmp/dump1090-127_0_0_1.pid.
pi@piaware ~/sock30003 $

Note: the ā€™ &ā€™ at the end of the line launches the command in the background. This way you can use your terminal screen, while socket30003.pl is running as a background process. But the output of the script will still be displayed on the screen. This can be very annoying because the scripts frequently displays the number of positions processed. To avoid this you can launch the script with the option ā€œ-nopositionsā€.

If you want to collect ADS-B data from more then one raspberry, just launch socket30003.pl again as following (note that the ā€œ-peer ā€ belongs to that other host):


pi@ted1090-3 ~/sock30003 $ ./socket30003.pl -peer 192.168.11.8 -nopositions &
Using the unit 'kilometer' for the distance and 'meter' for the altitude.
Trying to connect to peer host '192.168.11.8'...
The antenna latitude & longitude are: '52.085624','5.0890591'
Connected to '192.168.11.8', port 30003 (tcp).
The data directory/file is: /tmp/dump1090-192_168_11_8-150924.txt
The log  directory/file is: /tmp/dump1090-192_168_11_8-150924.log
Starting 'socket30003.pl'....
'socket30003.pl' (6729) is started!
Using pidfile /tmp/dump1090-192_168_11_8.pid.
pi@piaware ~/sock30003 $

Run socket30003.pl from crontab
You can run socket30003.pl from the command line, but since this flight data collection have has to continue for several days, it is best to run it as a crontab job:


pi@piaware ~/sock30003 $ sudo crontab -e

And then add the line(s):


0 * * * *  /home/pi/sock30003/socket30003.pl       # connects to localhost.
0 * * * *  /home/pi/sock30003/socket30003.pl -peer <ip-addres or hostname>  # If you have more dump1090 running on another host.


This way it will check every hour if it is still running.

Watch out for a reboot!
One important note on the /tmp directory. In Raspbian and Piaware files /tmp will be lost after a reboot! There are two reasons why files will be lost after a reboot. In Piaware (unlike the Raspbian 3.18.11 version I use) /tmp is mounted on a small RAM disk. They other reason is that TMPTIME=0 by default. Be sure that you changes those settings before you start collecting data or store your flight position data files elsewhere! You can change the TMPTIME setting for /tmp using:


pi@piaware ~ $ sudo vi /etc/default/rcS

Set TMPTIME to 14 (or as much as you like) so that files in /tmp only will deleted if their modification time is more than 14 days ago


TMPTIME=14

And change the mount setting for /tmp using:


pi@piaware ~ $ sudo vi /etc/default/tmpfs

Set RAMTMP=no and reboot.

Log file socket30003.pl
In the /tmp directory you find a log file named ā€œdump-<ip_address/hostname>-YYMMDD.logā€ . Every day a new one.
Inside you can see (besides errors) details about the number op planes and positions spotted:


pi@ted1090-3 ~/sock30003 $ tail -n 24 /tmp/dump1090-127_0_0_1-150924.log
current number of flights=216,epoch=2015-09-24/09:56:00,message_count=25937349,position_count=1567983,flight_count=2694.
removed:3C4B2F,first seen=2015-09-24/09:46:52,last seen=2015-09-24/09:53:47,message_count=529,position_count=3.
removed:49D025,first seen=2015-09-24/09:49:57,last seen=2015-09-24/09:53:14,message_count=120,position_count=21.
removed:491257,first seen=2015-09-24/09:43:29,last seen=2015-09-24/09:53:53,message_count=1037,position_count=42.
removed:4D2027,first seen=2015-09-24/09:53:31,last seen=2015-09-24/09:53:31,message_count=2,position_count=0.
removed:4BAA62,first seen=2015-09-24/09:48:06,last seen=2015-09-24/09:53:43,message_count=261,position_count=3.
removed:400BB2,first seen=2015-09-24/09:15:31,last seen=2015-09-24/09:53:54,message_count=25470,position_count=678.
removed:4009F4,first seen=2015-09-24/09:20:32,last seen=2015-09-24/09:53:20,message_count=13356,position_count=715.
removed:4B1789,first seen=2015-09-24/09:42:36,last seen=2015-09-24/09:53:25,message_count=636,position_count=19.
removed:4CA352,first seen=2015-09-24/09:19:46,last seen=2015-09-24/09:53:44,message_count=25330,position_count=1397.
current number of flights=214,epoch=2015-09-24/09:57:00,message_count=26017698,position_count=1572046,flight_count=2701.
removed:484F91,first seen=2015-09-24/09:49:00,last seen=2015-09-24/09:54:09,message_count=66,position_count=0.
removed:4CA9C4,first seen=2015-09-24/09:24:10,last seen=2015-09-24/09:54:52,message_count=25531,position_count=1395.
removed:44CDC3,first seen=2015-09-24/09:50:40,last seen=2015-09-24/09:54:55,message_count=71,position_count=0.
removed:484966,first seen=2015-09-24/09:34:00,last seen=2015-09-24/09:54:33,message_count=11782,position_count=746.
removed:ABBF84,first seen=2015-09-24/09:54:16,last seen=2015-09-24/09:54:16,message_count=4,position_count=0.
removed:4CA75C,first seen=2015-09-24/09:24:32,last seen=2015-09-24/09:54:41,message_count=8249,position_count=618.
removed:511065,first seen=2015-09-24/09:32:09,last seen=2015-09-24/09:54:13,message_count=7235,position_count=384.
current number of flights=214,epoch=2015-09-24/09:58:00,message_count=26096760,position_count=1576166,flight_count=2708.
removed:406673,first seen=2015-09-24/09:29:50,last seen=2015-09-24/09:55:44,message_count=10541,position_count=571.
removed:4840D5,first seen=2015-09-24/09:24:08,last seen=2015-09-24/09:55:00,message_count=12929,position_count=993.
removed:3C4DCE,first seen=2015-09-24/09:18:44,last seen=2015-09-24/09:55:01,message_count=30233,position_count=672.
removed:43E88A,first seen=2015-09-24/09:33:09,last seen=2015-09-24/09:55:28,message_count=7220,position_count=341.
removed:3C6449,first seen=2015-09-24/09:53:27,last seen=2015-09-24/09:55:35,message_count=6,position_count=0.

Every minute an update on how much planes are currently spotted and the number of messages, positions and flights counted that day.
Between that you can see which planes were spotted, how long they were spotted and how many messages and positions were received from that plane.

socket30003.pl output
By default the output of socket30003.pl will be written in /tmp:


pi@ted1090-3 ~ $ ls -ltr /tmp/dump*.txt 
-rw-r--r-- 1 pi   pi    685219803 Sep 20 23:59 /tmp/dump1090-127_0_0_1-150920.txt
-rw-r--r-- 1 pi   pi    439486146 Sep 21 23:59 /tmp/dump1090-127_0_0_1-150921.txt
-rw-r--r-- 1 pi   pi    448420915 Sep 22 23:59 /tmp/dump1090-127_0_0_1-150922.txt
-rw-r--r-- 1 pi   pi    439160328 Sep 23 23:59 /tmp/dump1090-127_0_0_1-150923.txt
-rw-r--r-- 1 pi   pi     42040864 Sep 24 19:46 /tmp/dump1090-127_0_0_1-150924.txt
-rw-r--r-- 1 pi   pi       766153 Sep 24 19:46 /tmp/dump1090-192_168_11_8-150924.txt

The comma separated values have a column headers. The Altitude and distance columns have their unit type between parentheses. This way the other scripts (heatmap.pl and rangeview.pl) knows what unit type is used. Each time the socket30003.pl script is restarted (may be with different unit types) it writes a new column header in to the file, so heatmap.pl and rangeview.pl always use the correct unit types:


pi@ted1090-3 ~/sock30003 $ head /tmp/dump1090-127_0_0_1-150924.txt
hex_ident,altitude(meter),latitude,longitude,date,time,angle,distance(kilometer)
780DA3,11270,52.46353,6.58246,2015/09/24,17:17:19.356,75.13,109.94
40665F,11575,51.47197,1.76338,2015/09/24,17:17:19.358,-100.74,238.71
4068F5,11270,52.28342,3.84286,2015/09/24,17:17:19.358,-80.53,87.75
40659A,5559,51.66609,6.28648,2015/09/24,17:17:19.360,110.01,94.5
400944,8529,51.41739,2.21397,2015/09/24,17:17:19.360,-103.49,211.37
484F7F,4241,51.47928,4.55658,2015/09/24,17:17:19.360,-139.87,76.72
89641E,12192,51.54172,6.77641,2015/09/24,17:17:19.361,108.5,130.8
4B0F59,7227,51.38741,7.11357,2015/09/24,17:17:19.361,109.68,159.55
4008DE,10966,50.97148,5.06633,2015/09/24,17:17:19.361,-178.87,123.89

Many positions per second. It can be as many as 5 million positions a day:


pi@ted1090-3 ~/sock30003 $ wc -l /tmp/dump1090-127_0_0_1-150922.txt
5077255 /tmp/dump1090-127_0_0_1-150922.txt

This is of course depending on your location and antenna.

Creating a heatmap
If you have collected enough data you can create a heatmap:


pi@ted1090-3 ~/sock30003 $ ./heatmap.pl 
There will be no more then '100000' positions in the output file.
The maximum position weight on the heatmap will be not more then '1000'.
The resolution op the heatmap will be 1000x1000.
The antenna latitude & longitude are: '52.085','5.089'.
The heatmap will cover the area of 5 degree around the antenna, which is between latitude 47 - 57 and longitude 0 - 10.
The following files fit with the filemask '*dump1090-*':
  /tmp/dump1090-192_168_11_8-150924.txt
  /tmp/dump1090-127_0_0_1-150924.txt
Processing file '/tmp/dump1090-192_168_11_8-150924.txt':
  -header units:altitude=meter,distance=kilometer, position 1- 11435 processed.
  -header units:altitude=meter,distance=kilometer, position 11436- 11628 processed.
  -header units:altitude=meter,distance=kilometer, position 11629-42744 processed. 15 positions were out side the specified area.
Processing file '/tmp/dump1090-127_0_0_1-150924.txt':
  -header units:altitude=meter,distance=kilometer, position 1- 2476 processed.
  -header units:altitude=meter,distance=kilometer, position 2477-1052595 processed. 162 positions were out side the specified area.
Number of sorted positions: 162410
The highest weight is '00001080' and the lowest weight is '00000003'.
Since the highest weight is more the the max weight '1000' the weight of all points will be multiplied with a factor 0.925925925925926.

Output file with java script code: /tmp/heatmapcode.csv
{location: new google.maps.LatLng(52.265, 4.699), weight: 1001},
{location: new google.maps.LatLng(52.275, 4.709), weight: 883},
{location: new google.maps.LatLng(52.275, 4.719), weight: 815},
{location: new google.maps.LatLng(52.255, 4.679), weight: 606},
{location: new google.maps.LatLng(52.245, 4.679), weight: 531},

100000 rows with heatmap position data processed!

{location: new google.maps.LatLng(50.525, 5.299), weight: 6},
{location: new google.maps.LatLng(50.525, 5.019), weight: 6},
{location: new google.maps.LatLng(50.525, 4.999), weight: 6},
{location: new google.maps.LatLng(50.525, 4.989), weight: 6},
{location: new google.maps.LatLng(50.525, 4.949), weight: 6},

Output file in csv format: /tmp/heatmapdata.csv
"weight";"lat";"lon"
"1001";"52.265";"4.699"
"883";"52.275";"4.709"
"815";"52.275";"4.719"
"606";"52.255";"4.679"

100000 rows with heatmap position data processed!

"6";"50.525";"5.299"
"6";"50.525";"5.019"
"6";"50.525";"4.999"
"6";"50.525";"4.989"
"6";"50.525";"4.949"

The first 5 and the last 5 lines of the output files are shown.
There are all kinds of options you can try to adjust the heatmap.

Now copy /tmp/heatmapdata.csv to the /usr/share/dump1090-mutability/html directory to display it in the dump1090-mutability GUI


pi@ted1090-3 ~/sock30003 $ sudo cp /tmp/heatmapdata.csv /usr/share/dump1090-mutability/html 

And reload your browser!
You can also copy the heatmapdata.csv file to the host where you run your browser and select the file from the heatmap panel in the dump1090-mutability GUI.
May be you need to toggle the [heatmap] button on the sidebar to get the heatmap visible.

Be sure you have enough flight positions data collected. Here are two example, where you can see the difference:
81.525.440 million flight positions (more then 2 weeks of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/heatmapdata-example1.csv

If you zoom in on this you can see many details.

3.500.000 million flight positions (less then 1 day of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/heatmapdata-example2.csv

Could be much better. If you zoom in on this you get not more detailsā€¦

Creating a range/altitude view
For a good range/altitude view you need a lot of data:


pi@ted1090-3 ~/sock30003 $ ./rangeview.pl 
The altitude will be converted from 'meter' to 'meter'.
The distance will be converted from 'kilometer' to 'kilometer.
The maxium altitude is 12000 meter.
The minimal altitude is 0 meter.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 24.
The latitude/longitude location of the antenna is: 52.085624,5.0890591.
An altitude zone is 521 meter.
The following files fit with the filemask '*dump1090-*':
  /tmp/dump1090-192_168_11_8-150924.txt
  /tmp/dump1090-127_0_0_1-150924.txt
processing '/tmp/dump1090-192_168_11_8-150924.txt':
  -header units:altitude=meter,distance=kilometer, position 1- 11435 processed.
  -header units:altitude=meter,distance=kilometer, position 11436- 11628 processed.
  -header units:altitude=meter,distance=kilometer, position 11629-42744. processed.
processing '/tmp/dump1090-127_0_0_1-150924.txt':
  -header units:altitude=meter,distance=kilometer, position 1- 2476 processed.
  -header units:altitude=meter,distance=kilometer, position 2477-1090039. processed.

Number of files read: 2
Number of position processed: 1132780 and positions within range processed: 1132780
datafile= /tmp/rangeview.csv
kmlfile= /tmp/rangeview.kml
 1,Altitude zone:     0-   520,Directions:  573/ 1440,Positions processed:     10969,Positions processed per direction: min:     2,max:   290,avg:     7,real avg:    19
 2,Altitude zone:   521-  1041,Directions:  731/ 1440,Positions processed:     34222,Positions processed per direction: min:     1,max:   384,avg:    23,real avg:    46
 3,Altitude zone:  1042-  1562,Directions:  903/ 1440,Positions processed:     26485,Positions processed per direction: min:     1,max:   460,avg:    18,real avg:    29
 4,Altitude zone:  1563-  2083,Directions: 1167/ 1440,Positions processed:     24151,Positions processed per direction: min:     8,max:   398,avg:    16,real avg:    20
 5,Altitude zone:  2084-  2604,Directions: 1178/ 1440,Positions processed:     27824,Positions processed per direction: min:    13,max:   544,avg:    19,real avg:    23
 6,Altitude zone:  2605-  3125,Directions: 1132/ 1440,Positions processed:     23868,Positions processed per direction: min:     4,max:   290,avg:    16,real avg:    21
 7,Altitude zone:  3126-  3646,Directions: 1267/ 1440,Positions processed:     25941,Positions processed per direction: min:    10,max:   203,avg:    18,real avg:    20
 8,Altitude zone:  3647-  4167,Directions: 1333/ 1440,Positions processed:     22101,Positions processed per direction: min:     5,max:   133,avg:    15,real avg:    16
 9,Altitude zone:  4168-  4688,Directions: 1274/ 1440,Positions processed:     21142,Positions processed per direction: min:    12,max:   124,avg:    14,real avg:    16
 10,Altitude zone:  4689-  5209,Directions: 1262/ 1440,Positions processed:     23113,Positions processed per direction: min:     8,max:   106,avg:    16,real avg:    18
 11,Altitude zone:  5210-  5730,Directions: 1239/ 1440,Positions processed:     21932,Positions processed per direction: min:    24,max:   184,avg:    15,real avg:    17
 12,Altitude zone:  5731-  6251,Directions: 1279/ 1440,Positions processed:     25954,Positions processed per direction: min:    17,max:   249,avg:    18,real avg:    20
 13,Altitude zone:  6252-  6772,Directions: 1383/ 1440,Positions processed:     30274,Positions processed per direction: min:    17,max:   515,avg:    21,real avg:    21
 14,Altitude zone:  6773-  7293,Directions: 1345/ 1440,Positions processed:     30132,Positions processed per direction: min:    23,max:   341,avg:    20,real avg:    22
 15,Altitude zone:  7294-  7814,Directions: 1312/ 1440,Positions processed:     30875,Positions processed per direction: min:    13,max:   466,avg:    21,real avg:    23
 16,Altitude zone:  7815-  8335,Directions: 1428/ 1440,Positions processed:     31769,Positions processed per direction: min:    11,max:   492,avg:    22,real avg:    22
 17,Altitude zone:  8336-  8856,Directions: 1412/ 1440,Positions processed:     36847,Positions processed per direction: min:     9,max:   304,avg:    25,real avg:    26
 18,Altitude zone:  8857-  9377,Directions: 1291/ 1440,Positions processed:     37239,Positions processed per direction: min:    20,max:   269,avg:    25,real avg:    28
 19,Altitude zone:  9378-  9898,Directions: 1438/ 1440,Positions processed:     47712,Positions processed per direction: min:    44,max:   450,avg:    33,real avg:    33
 20,Altitude zone:  9899- 10419,Directions: 1440/ 1440,Positions processed:     87625,Positions processed per direction: min:    80,max:   548,avg:    60,real avg:    60
 21,Altitude zone: 10420- 10940,Directions: 1440/ 1440,Positions processed:     86764,Positions processed per direction: min:    49,max:   608,avg:    60,real avg:    60
 22,Altitude zone: 10941- 11461,Directions: 1440/ 1440,Positions processed:    245968,Positions processed per direction: min:   170,max:   929,avg:   170,real avg:   170
 23,Altitude zone: 11462- 11982,Directions: 1440/ 1440,Positions processed:    137923,Positions processed per direction: min:   101,max:   581,avg:    95,real avg:    95
 24,Altitude zone: 11983- 12503,Directions: 1440/ 1440,Positions processed:     41950,Positions processed per direction: min:    32,max:   169,avg:    29,real avg:    29

In the progress report above it gets clear that there are not enough positions. For some ā€œaltitude zoneā€ and some ā€œdirection zonesā€ there are no positions spotted! My advise: collect more data :wink:

Now copy the /tmp/rangeview.kml file to a web server. Use a public folder on Dropbox or Google Drive if you donā€™t have a public accessible web server available.
Fill the correct path in the /usr/share/dump1090-mutability/html/config.js:


// KML to show: 
UserMap     = "https://github.com/tedsluis/dump1090/raw/master/exampledata/radar.kml";  // your KML URL (publicly available domain)

And reload your browser!
May be you need to toggle the [range] button on the sidebar to get the range/altitude view visible.

Here are two example, where you can see the difference:
81.525.440 million flight positions (more then 2 weeks of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/rangeview-example1.kml

Looks quite good.

3.500.000 million flight positions (less then 1 day of data collection): https://raw.githubusercontent.com/tedsluis/dump1090/master/exampledata/rangeview-example1.kml

Looking poorly. Too little dataā€¦

To be continued in the next daysā€¦

Any feedback is welcome!

Ted

http://flightaware.com/adsb/stats/user/tedsluis#stats-6731

Resources:
On dump1090-mutability:

2 Likes

post156902.html#p156902

=========== 20150924: carried over from the first post to make space for documentation ===========
Hi,

I am trying to make a heatmap of flight paterns into dump1090-mutability. It looks interesting, but this is a performance intensive task for a raspberry.

https://dl.dropboxusercontent.com/u/17865731/dump1090-mutability.20150817/heatmap.gif

It gives a impressions which paths planes fly and how fare my antenna reaches. (This was made on a raspberry 1B)

I followed the example at https://developers.google.com/maps/documentation/javascript/heatmaplayer

I added the red and blue code below here to script.js:


    // Define the Google Map
    var mapOptions = {
            center: new google.maps.LatLng(CenterLat, CenterLon),
            zoom: ZoomLvl,
            mapTypeId: google.maps.MapTypeId.ROADMAP,
            mapTypeControl: true,
            streetViewControl: false,
            mapTypeControlOptions: {
                    mapTypeIds: mapTypeIds,
                    position: google.maps.ControlPosition.TOP_LEFT,
                    style: google.maps.MapTypeControlStyle.DROPDOWN_MENU
            }
    };

var heatmapData =

new google.maps.LatLng(51.21172, 3.69534),
new google.maps.LatLng(51.67090, 4.84928),
new google.maps.LatLng(49.35645, 0.22695),
new google.maps.LatLng(52.26494, 4.51668),
new google.maps.LatLng(53.20353, 6.17335),
new google.maps.LatLng(52.09680, 4.34227),
new google.maps.LatLng(52.25060, 3.85604),
new google.maps.LatLng(52.52435, 4.87610),
new google.maps.LatLng(52.44127, 5.24475)

];

    GoogleMap = new google.maps.Map(document.getElementById("map_canvas"), mapOptions);

    //Define OSM map type pointing at the OpenStreetMap tile server
    GoogleMap.mapTypes.set("OSM", new google.maps.ImageMapType({
            getTileUrl: function(coord, zoom) {
                    return "http://tile.openstreetmap.org/" + zoom + "/" + coord.x + "/" + coord.y + ".png";
            },
            tileSize: new google.maps.Size(256, 256),
            name: "OpenStreetMap",
            maxZoom: 18
    }));

    GoogleMap.mapTypes.set("dark_map", styledMap);

var heatmap = new google.maps.visualization.HeatmapLayer({
data: heatmapData
});
heatmap.setMap(GoogleMap);

    // Listeners for newly created Map

The blue lines are lat/lon flight positions. I was able to add 360000 lines with lat/lon positions which I had gathered in a view hours. Refreshing my browser took a view minutes!

I also added the red line below to the **gmap.html **script:


            <script type="text/javascript" src="coolclock/coolclock.js"></script>
            <script type="text/javascript" src="coolclock/moreskins.js"></script>
            <script type="text/javascript" src="https://maps.googleapis.com/maps/api/js?libraries=visualization&sensor=true_or_false"></script>

    <title>DUMP1090</title>
    </head>

I wrote my own Perl script to gather lat/lon flight positions from port 30003 and save them in csv format. But I would like to know how others do that?

I was not able to add more lines with lat/lon positions, because the browser crashes all the time. So this way of creating a heatmap is not a good solution.

Google maps also offers an other way of creating heatmaps: Fusion Tables heatmaps This way you can store the positions in the cloud (using a restfull api). According the documentation this works with only a little impact on performance. I believe it is experimental.

I was wondering if anyone tried this?

Ted

Thanks for the link, but my question is: has anyone tried the Fusion Tables heatmaps (not the normal Heatmaps).

Fusion Tables heatmaps are more complicated, because you have to store the lat/lon positions a Fusion Table in the Google cloud. And I read: ā€œFusion Tables API v1.0 restricts the maximum response size to 10 MB. We encourage you to migrate to the Google Fusion Tables API v2.0, which provides media download for large payloads.ā€

But seeing the Fusion Tables heatmap examples it looks very promising. They load and zoom quickly!

I have created a google fusion table to store position data for a heatmap: https://www.google.com/fusiontables

Google map fusion heatmap has a limit of 1000 positions. that is not very much. So I divided the map into 400 x 400 squares. I counted the number of flight positions per square. I used position flight data from the last day with 9718522 positions. Then I uploaded the lat/lon positions of the 1000 squares with the most flight positions in this format:


weight,location
00007259,52.27 4.7
00006843,52.27 4.69
00005363,52.25 4.67
00004895,52.26 4.68
00004819,52.46 4.75
00004331,52.39 4.74
00004253,52.37 4.74
00004203,52.45 4.75
00003921,52.44 4.75
00003801,52.24 4.67
00003677,52.47 4.75
00003644,52.19 4.8
00003632,52.26 4.78
00003536,52.38 4.74
00003509,52.23 4.68
.
thousand records!
.
00000399,52.48 4.45
00000399,52.46 4.78
00000399,52.23 5.1
00000399,52.12 5.53
00000398,52.51 4.73
00000398,52.48 4.73
00000398,52.27 4.67
00000398,52.23 5.44
00000398,52.2 4.81
00000398,52.12 4.74
00000398,51.87 4.07
00000397,52.41 5.12
00000397,52.17 5.42
etc....

There are two fields per row separated by a comma: weight and location

The column weight contains the number of flight positions I have registered in that square. This field determines the intensity of the spot on the heatmap.
As you can see, the first record (or square) represents 7259 positions and the second record represents 6843 positions. The last one represents only 397 positions.

The column location contains the latitude and longitude separated separated by a space. They will come together in the fusion table a the column location.
https://dl.dropboxusercontent.com/u/17865731/dump1090-mutability.20150818/heatmap.with.weight.png

From the heatmap it is very clear where most planes fly :wink:

The fusion tool in google drive lets you create a table, add and modify columns, import csv data and preview it on a map like above.

Loading the data goes very quick and zooming in and out goes smooth. Very hopeful :wink:
I just wish that heatmaps would work with more then a 1000 positions.

The next step is to add this heatmap layer to the dump1090 mapā€¦

Hi, I implemented your first rendition without any problems with 20k points (a few hours of logging) ā€¦ loads fine on my devicesā€¦
and Iā€™ve played a bit with the fusion heatmapsā€¦ the 1k record limit is frustrating.

Curious to know how you did the data manipulation into the sqaures you mention to reduce the record size.

I can grep the lat/long points from the log file and then work with the data in excel but Iā€™m not clear on how to get points into the area squares.

  • Dino

Made the changes in this fork. How long does it take to start seeing results?

Yes, 1k limit. But I hope it is possible to use a fusion table with the google maps api together with weight point. Is there anyone who knows how to do that?

Well, I wrote a script in Perl (dump1090.socket30003.pl) that collects the position data from dump1090 using port 30003 and saves that data in .csv files on my rapsberry. This is the format of the output:


header: hex_ident,altitude(feet),latitude,longitude,date,time,direction,distance(meter)
data:
4010DB,35000,52.31159,5.65019,2015/08/30,17:48:55.481,67.1801766700598,45756.6730179409
400A60,37000,50.78618,4.92173,2015/08/30,17:48:55.483,-172.94938714767,144949.287631512
478533,37000,51.55232,3.72444,2015/08/30,17:48:55.524,-112.113114036685,110962.328386504
4841DB,4425,52.13664,4.44099,2015/08/30,17:48:55.527,-85.2749293488927,44615.3540134401
400B44,33000,51.99193,4.97032,2015/08/30,17:48:55.540,-129.457164021526,13209.2852899247
400E14,38000,52.11044,5.48210,2015/08/30,17:48:55.547,86.2130260390998,26988.0996886073
478538,37000,52.77310,5.54024,2015/08/30,17:48:55.583,32.1346191471548,82332.7806446534
4CA9B4,26900,51.54399,5.28992,2015/08/30,17:48:55.603,160.416360285161,61786.3690908337
478535,37000,51.88899,5.57732,2015/08/30,17:48:55.631,112.766312936825,39947.5126386791
4249BD,3725,52.19934,4.85542,2015/08/30,17:48:55.647,-63.0644110287155,20348.0966975414

I collect about 6 million positions a day.

note: the column distance shows the distance to my house. The column direction shows the direction of the plane seen from my house.

Later on (after a couple of days fight position collecting) I run an other Perl script (dump1090.socket30003.heatmap.pl) that calculates a heatmap and writes it in this format to a text file:


{location: new google.maps.LatLng(51.025, 3.1), weight: 706},
{location: new google.maps.LatLng(50.925, 4.4), weight: 706},
{location: new google.maps.LatLng(50.837, 4.775), weight: 706},
{location: new google.maps.LatLng(50.75, 4.612), weight: 706},
{location: new google.maps.LatLng(50.7, 4.562), weight: 706},

A third script (dump1090.socket30003.radar.pl) makes points for a radar map with different altitude zones.

The dump1090.socket30003.pl script is the most mature of the three. The dump1090.socket30003.heatmap.pl and the dump1090.socket30003.radar.pl just produce a output file. I have not yet implemented the heatmap and radar into dump1090. I hope to do that in the coming weeks.

You can find my scripts at https://github.com/tedsluis/dump1090.socket30003. They run on raspberry raspbian and on centos/fedora as well. I am not sure whether you need to install extra perl packages. But if you get any error message about that, you can use google to find out how to install the extra packages :wink: or ask me.

Here an example of the radar map:
https://dl.dropboxusercontent.com/u/17865731/dump1090.20150830/radar.png
I just imported the radar.csv file into http://www.gpsvisualizer.com/
Still a lot to improve as you can see :wink:

Ted

Trying it now. Nicely written!

The only quirk I found so far is that the script does accept my -114. Longitude value. I have to remove the minus sign. Will this affect the distance calculations?

Well, 114 east and 114 west are completely different placesā€¦

Regarding the test you created in github, Iā€™ve installed it and I"m not seeing anything. Pardon my ignorance, but thereā€™s no ā€œheatmapdata matrixā€ is the lines of code you added. Is that elsewhere in the branch?

Pretty cool, well done.

I only ran the script for about 15 minutes and got this.

http://i300.photobucket.com/albums/nn30/adsbjunkmail/Screen%20Shot%202015-09-01%20at%201.30.51%20PM_zpsnaxxrncv.jpg

For anyone using the same Longitude format as me: -XX.YYY, just add 360 to it. In my case (-77.61 + 360) gives you 288.39, so plug that value in instead of your -XX.YY value.

(edit) And in dump1090.socket30003.heatmap.pl if you are using -XX.YYY use that value and not the converted value shown above.

Iā€™ll ran the scripts over night and see how it goes.

Made the changes above. Also changed the default input file name do dump1090.socket30003 from dump.socket30003. Not getting anything into the output .csv file. Not getting an error so I donā€™t think thereā€™s a ā€œmissing perl packageā€ but I could be wrong. Also seems none of the switches (help, source directory, filemask) work for me either. Maybe that does indicate I need some missing package.

Ideas?

Jim

Hmm, that is a bug. It does not accept negative values. I have fixed it in a new version. I also added support for feet, mile, nautical mile, meter and kilometer. You can find the new version at: https://github.com/tedsluis/dump1090.socket30003

Here is the help text of the new version:


root@msi dump1090.socket30003]# ./dump1090.socket30003.pl -help

This dump1090.socket30003.pl script can retrieve flight data (lat, lon and alt) from a dump1090 host using port
30003 and calcutates the distance and angle between the antenna and the plane. It will store these 
values in a file in csv format (seperated by commas).


This script can run several times simultaneously on one host retrieving data from multiple dump1090
instances. Each instance can use the same directories, but they all have their own data, log and 
pid files.

The script can be lauched as a background process. It can be stopped by using the -stop parameter
or by removing the pid file. When it not running as a background process, it can also be stopped 
by pressing CTRL-C. The script will write the current data and log entries to the filesystem 
before exiting...

Syntax: dump1090.socket30003.pl

Optional parameters:
	-peer <peer host>                  A dump1090 hostname or IP address. 
                                      (De default is the localhost, 127.0.0.1)
	-restart                           Restart the script.
	-stop                              Stop a running script.
	-status                            Display status.
	-data <data directory>             The data files are stored in /tmp by default.
	-log  <log directory>              The log file is stored in /tmp by default.
	-pid  <pid directory>              The pid file is stored in /tmp by default.
	-msgmargin <max message margin>    The max message margin is 10ms by default.
	-lon <lonitude>	                 Location of your antenna.
	-lat <latitude>
	-distancematric <matric>           Type of matric: kilometer, nauticalmile, mile or meter
	                                   Default distance matric is kilometer.
	-altitudematric <matric>           Type of matric: meter or feet.
	                                   Default altitude matric is meter.
        -nopositions                  Does not display the number of position when running
                                      interactive (launched from commandline).
	-help                              This help page.

Notes: 
        - To launch it as a background process, add '&' or run it from crontab:
        0 * * * * <path>/dump1090.socket30003.pl
        (This command checks if it ran every hour and relauch it if nessesary.)
        - The default values can be changed within the script (in the most upper section).


Examples:
	dump1090.socket30003.pl 
	dump1090.socket30003.pl -log /var/log -data /home/pi -pid /var/run -restart &
	dump1090.socket30003.pl -peer 192.168.1.10 -stop

Pay attention: to stop an instance: Don't forget to specify the same peer host.


You are absolutely right! :wink:

These scripts are only the start of a project to create a heatmap and a radar view. It is not yet finished :wink:
The dump1090.socket30003.pl script creates .txt files with fight positisions in the /tmp directory. One file per dump1090 host (if you got more then one dump1090) and every day a new file.
The dump1090.socket30003.heatmap.pl script creates a heatmap.csv file in the /tmp directory. Run this script after you have collected at least a day of flight position data.
To see a heatmap you should add content of the heatmap.csv file to the /usr/share/dump1090-mutability/html/script.js as I showed in my first post of this topic.
Be sure that you add the content of the file between these two red lines of the script.js

var heatmapData =

];

You can add at least 100.000 lines with position data. In my case (raspberry 1B) I could add a maximum of 240.000 lines. In such a case loading a heatmap takes more then 20 seconds.

If you want to let the dump1090.socket30003.pl script run for several days you can add it to crontab like this:
$ sudo crontab -e
Add this line
0 * * * * /dump1090.socket30003.pl
save+exit

This way you donā€™t have to stay logged in and it will check every hour if it is still running and if it is not it will be restarted.

Well, the good news is: you donā€™t miss any Perl packages, because otherwise you should get a very explicit error message. Letā€™s find out what is going on:
Does **dump1090.socket30003.pl **creates a .txt file in /tmp? Does it have content (flight positions) like this:


4492E5,7950,52.05187,5.35066,2015/09/01,22:33:26.778,97.65,18.27
780A97,10044,52.65175,5.92636,2015/09/01,22:33:26.785,54.74,84.81
45AA8D,7577,52.45250,2.17592,2015/09/01,22:33:26.791,-82.37,202.35
451E8C,10958,51.21126,3.83029,2015/09/01,22:33:26.792,-125.85,130.35
406AE3,11278,50.75885,4.42831,2015/09/01,22:33:26.794,-154.43,154.47
4B868D,11179,50.89230,4.15176,2015/09/01,22:33:26.847,-142.95,147.7
73806C,11575,50.91930,4.46655,2015/09/01,22:33:26.895,-152.85,136.65

Does it creates a log file in /tmp? Can you find any clues in there?
What options do you use to launch the scripts?
Is dump1090 running on the same host or on another host? Can you ping that host?

Ted

Ted -

All this is running on my Pi. I can get dump1090.socket20003.radar.pl to output data:


type,new_track,name,color,trackpoint,altitudezone,destination,hex_ident,Altitude,latitude,longitude,date,time,angle,distance
T,1,Altitude zone 1: 06000- 9000,yellow,1, 6000, -69,ABE147,7925,42.90994,-73.95357,2015/09/01,21:25:12.133,-69.8644341904679,34158.799205038
T,1,Altitude zone 2: 09000-12000,red,2, 9000, -69,ABE147,9325,42.94179,-73.99724,2015/09/01,21:25:45.923,-69.8493969357005,32425.0393125145
T,1,Altitude zone 3: 12000-15000,green,3, 12000, -70,A05269,14900,42.39308,-73.83780,2015/09/01,21:15:45.594,-70.0415316492588,61372.7943655646
T,0,Altitude zone 3: 12000-15000,green,4, 12000, -69,A39657,13925,42.46244,-74.01464,2015/09/01,21:30:27.097,-69.9995780852293,46217.8554079803
T,1,Altitude zone 4: 15000-18000,violet,5, 15000, -70,A05269,15350,42.40378,-73.91372,2015/09/01,21:15:06.382,-70.0295811330878,56419.2566688118
T,0,Altitude zone 4: 15000-18000,violet,6, 15000, -69,A39657,16450,42.75371,-73.76096,2015/09/01,21:27:06.042,-69.9357308097219,47920.8201674048
T,1,Altitude zone 5: 18000-21000,orange,7, 18000, -70,C06E93,19775,42.14989,-74.38287,2015/09/01,21:20:55.181,-70.0572852666456,72326.8159584803
T,0,Altitude zone 5: 18000-21000,orange,8, 18000, -69,4B1931,20975,42.73249,-73.80507,2015/09/01,21:14:38.241,-69.9374924577469,44684.557426663
T,1,Altitude zone 6: 21000-24000,cyan,9, 21000, -70,A0144E,23975,42.25200,-73.69429,2015/09/01,21:19:10.072,-70.1024735059328,80935.4515151291
T,0,Altitude zone 6: 21000-24000,cyan,10, 21000, -69,C06365,21850,42.70302,-73.37398,2015/09/01,21:31:14.079,-69.9953578999947,79987.9958606837
T,1,Altitude zone 7: 24000-27000,magenta,11, 24000, -70,A9BB62,26050,41.97528,-73.81783,2015/09/01,21:16:17.520,-70.1764560914165,101374.053086272
T,0,Altitude zone 7: 24000-27000,magenta,12, 24000, -69,AB654F,26000,43.13679,-75.77009,2015/09/01,21:14:00.784,-69.5858830650342,121868.210588071
T,1,Altitude zone 8: 27000-30000,blue,13, 27000, -70,A17498,29300,41.84569,-73.95191,2015/09/01,21:40:00.682,-70.2025426781128,110890.072194658
T,0,Altitude zone 8: 27000-30000,blue,14, 27000, -69,AA91F5,28000,43.61488,-75.95254,2015/09/01,21:37:51.879,-69.4130558725805,158716.330513758
T,1,Altitude zone 9: 30000-33000,yellow,15, 30000, -70,A17498,30200,41.84830,-73.95078,2015/09/01,21:39:57.501,-70.2018405114795,110639.442309154
T,0,Altitude zone 9: 30000-33000,yellow,16, 30000, -69,ABBB24,32325,43.85882,-76.08124,2015/09/01,21:39:55.522,-69.3207844235505,183274.783023969
T,1,Altitude zone 10: 33000-36000,red,17, 33000, -70,A17498,33225,41.86079,-73.94379,2015/09/01,21:39:49.891,-70.1986584644353,109482.921493679
T,0,Altitude zone 10: 33000-36000,red,18, 33000, -69,ABBB24,34975,42.88724,-76.43048,2015/09/01,21:31:47.853,-69.5894884946739,170312.994531162
T,1,Altitude zone 11: 36000-39000,green,19, 36000, -70,C04654,38000,41.65792,-73.83157,2015/09/01,21:48:16.332,-70.2757424741929,133806.606215052
T,0,Altitude zone 11: 36000-39000,green,20, 36000, -69,C07BBE,37000,44.28189,-77.21540,2015/09/01,21:46:59.870,-69.0553319604987,284032.620795718
T,0,Altitude zone 11: 36000-39000,green,21, 36000, -68,C05BC6,36975,44.63685,-77.38716,2015/09/01,21:37:15.447,-68.9225778515211,318551.103958638
T,1,Altitude zone 12: 39000-42000,violet,22, 39000, -70,A17498,39600,41.92030,-73.92282,2015/09/01,21:39:16.361,-70.1821104745903,103750.153000616
T,0,Altitude zone 12: 39000-42000,violet,23, 39000, -69,A8AC35,40000,44.03292,-76.43287,2015/09/01,21:33:03.165,-69.22498973853,217339.938452484
T,1,Altitude zone 13: 42000-45000,orange,24, 42000, -70,A17498,43675,42.04848,-73.86321,2015/09/01,21:38:03.186,-70.1480996382528,92413.2417056505
T,0,Altitude zone 13: 42000-45000,orange,25, 42000, -69,A78656,43000,43.17371,-76.15467,2015/09/01,21:14:29.169,-69.5300579056682,152966.797220536
T,1,Altitude zone 14: 45000-48000,cyan,26, 45000, -69,A4C328,45000,43.08121,-76.23492,2015/09/01,21:43:04.293,-69.5502560776418,157000.73246139


but running .heatmap.pl produces nothing. Is that just a ā€œdata/amount of timeā€ issue?

Jim

Cloned the repository so it should now be easier to stay up to date.

Applied the fix and the results make a lot more sense now! Thanks!

Iā€™ll let it run for a day or two and try the next step.

Cheers!

Hi Jim, Well, you have collected flight position data using dump1090.socket30003.pl, and you were able to create a rader.csv using dump1090.socket30003.radar.pl. So fare so good!
You have probably not set the position of your antenna in the script? That is my mistake. The help text was not clear about that and if you didnā€™t look into the code, you missed it :wink:

I have fixed that issue. I made (hopefully) a clear help text that says that you can change the default settings in the upper section of the script. Or you can specify your antenna location with the options ā€˜-latā€™ and ā€˜-lonā€™.

I have also added some more features, like setting the resolution of the heatmap and the maximum number of rows with positions in the heatmap file. You can also determine the size of the heatmap area using the ā€˜-degreesā€™ option:


[tedsluis@msi dump1090.socket30003]# ./dump1090.socket30003.heatmap.pl -help

This dump1090.socket30003.heatmap.pl script can create heatmap data.
At this moment it only creates a file with java script code, which
must be add to the script.js manualy in order to get a heatmap layer.
Please read this post for more info:
http://discussions.flightaware.com/ads-b-flight-tracking-f21/heatmap-for-dump1090-mutability-t35844.html

Syntax: dump1090.socket30003.heatmap.pl

Optional parameters:
	-data <data directory>          The data files are stored in /tmp by default.
	-filemask <mask>                Specify a filemask. The default filemask is 'dump.socket*.txt'.
	-lon <lonitude>                 Location of your antenna.
	-lat <latitude>
	-maxpositions <max positions>   Default is 100000 positions.
	-resolution <number>            Number of horizontal and vertical positions in output file.
	                                Default is 1000, which means 1000x1000 positions.
	-degrees <number>               To determine boundaries of area around the antenna.
	                                (lat-degree -- lat+degree) x (lon-degree -- lon+degree)
	                                De default is 3 degree.
	-help				This help page.

note: 
	The default values can be changed within the script (in the most upper section).


Examples:
	dump1090.socket30003.heatmap.pl 
	dump1090.socket30003.heatmap.pl -data /home/pi
	dump1090.socket30003.heatmap.pl -lat 52.1 -lon 4.1 -maxposition 50000


If have also added more messages to the script, so you can see what it is doing.


[tedsluis@msi dump1090.socket30003]# ./dump1090.socket30003.heatmap.pl -filemask 150721 -maxp 80000 -resol 1000 -degree 3
The resolution op the heatmap will be 1000x1000.
There will be no more then '80000' positions in the output file.
The antenna latitude & longitude are: '52.085624','5.0890591'.
The heatmap will cover the area of 3 degree around the antenna, which is between latitude 49.085 - 55.085 and longitude 2.089 - 8.089.
The following files fit with the filemask '*150721*':
  /tmp/dump.socket-127_0_0_1-150721.txt
Processing file '/tmp/dump.socket-127_0_0_1-150721.txt', '5182947' positions processed. 351361 positions were out side the specified area.
Number of sorted positions: 519931
Output file: /tmp/heatmap.csv
{location: new google.maps.LatLng(4.7034578313253, 52.2717469879518), weight: 00001988},
{location: new google.maps.LatLng(4.73960240963855, 52.374156626506), weight: 00001950},
{location: new google.maps.LatLng(4.7456265060241, 52.440421686747), weight: 00001858},
{location: new google.maps.LatLng(4.69140963855422, 52.2717469879518), weight: 00001806},
{location: new google.maps.LatLng(4.7456265060241, 52.3922289156626), weight: 00001794},

80000 rows with heatmap position data processed!

{location: new google.maps.LatLng(2.95646987951807, 51.295843373494), weight: 00000016},
{location: new google.maps.LatLng(2.94442168674699, 51.295843373494), weight: 00000016},
{location: new google.maps.LatLng(4.84803614457831, 51.2898192771084), weight: 00000016},
{location: new google.maps.LatLng(4.82996385542169, 51.2898192771084), weight: 00000016},
{location: new google.maps.LatLng(4.26369879518072, 51.2898192771084), weight: 00000016},
[tedsluis@msi dump1090.socket30003]#

At the end it shows the first and last 5 lines of the heatmap data.

You can find the new version at: https://github.com/tedsluis/dump1090.socket30003

I hope this works out for you!

Ted

I have an better example what you can do with the rader view script (dump1090.socket30003.radar.pl) output:

https://dl.dropboxusercontent.com/u/17865731/dump1090.20150830/Screenshot%20from%202015-09-02%2013-45-07-ANIMATION.gif

I used data collected by dump1090.socket30003.pl from one week (more then 42 million positions):


[tedsluis@msi dump1090.socket30003]# ./dump1090.socket30003.radar.pl -filemask 1-1507 -zones 27 -max 40500 -direct 1440
The maxium altitude is 40500 feet.
The minimal altitude is 0 feet.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 27.
An altitude zone is 1500 feet.
2300697
Number of files read: 9
Number of position processed: 42248538
datafile=/tmp/graph1.csv
  1,Altitude zone:     0-  1499,Directions:  359/ 1440,Positions processed: 519209,Positions processed per direction: min:    43,max: 29388,avg:   360,real avg:  1446
  2,Altitude zone:  1500-  2999,Directions:  359/ 1440,Positions processed: 1217322,Positions processed per direction: min:   142,max: 50120,avg:   845,real avg:  3390
  3,Altitude zone:  3000-  4499,Directions:  359/ 1440,Positions processed: 1039856,Positions processed per direction: min:    90,max: 41499,avg:   722,real avg:  2896
  4,Altitude zone:  4500-  5999,Directions:  359/ 1440,Positions processed: 902909,Positions processed per direction: min:   455,max: 38839,avg:   627,real avg:  2515
  5,Altitude zone:  6000-  7499,Directions:  359/ 1440,Positions processed: 985745,Positions processed per direction: min:  1543,max: 36063,avg:   684,real avg:  2745
  6,Altitude zone:  7500-  8999,Directions:  359/ 1440,Positions processed: 784766,Positions processed per direction: min:  1559,max: 26326,avg:   544,real avg:  2185
  7,Altitude zone:  9000- 10499,Directions:  359/ 1440,Positions processed: 847882,Positions processed per direction: min:  1422,max: 26529,avg:   588,real avg:  2361
  8,Altitude zone: 10500- 11999,Directions:  359/ 1440,Positions processed: 868736,Positions processed per direction: min:  1148,max: 19581,avg:   603,real avg:  2419
  9,Altitude zone: 12000- 13499,Directions:  359/ 1440,Positions processed: 681262,Positions processed per direction: min:   884,max: 14927,avg:   473,real avg:  1897
 10,Altitude zone: 13500- 14999,Directions:  359/ 1440,Positions processed: 684817,Positions processed per direction: min:   985,max: 12231,avg:   475,real avg:  1907
 11,Altitude zone: 15000- 16499,Directions:  359/ 1440,Positions processed: 689155,Positions processed per direction: min:  1402,max:  9531,avg:   478,real avg:  1919
 12,Altitude zone: 16500- 17999,Directions:  359/ 1440,Positions processed: 693201,Positions processed per direction: min:  1628,max: 11248,avg:   481,real avg:  1930
 13,Altitude zone: 18000- 19499,Directions:  359/ 1440,Positions processed: 752918,Positions processed per direction: min:  1541,max: 14625,avg:   522,real avg:  2097
 14,Altitude zone: 19500- 20999,Directions:  359/ 1440,Positions processed: 743833,Positions processed per direction: min:  1521,max: 22872,avg:   516,real avg:  2071
 15,Altitude zone: 21000- 22499,Directions:  359/ 1440,Positions processed: 824185,Positions processed per direction: min:  1522,max: 27312,avg:   572,real avg:  2295
 16,Altitude zone: 22500- 23999,Directions:  359/ 1440,Positions processed: 1006715,Positions processed per direction: min:  2006,max: 21138,avg:   699,real avg:  2804
 17,Altitude zone: 24000- 25499,Directions:  359/ 1440,Positions processed: 922156,Positions processed per direction: min:  1784,max: 28958,avg:   640,real avg:  2568
 18,Altitude zone: 25500- 26999,Directions:  359/ 1440,Positions processed: 873845,Positions processed per direction: min:  1078,max: 28981,avg:   606,real avg:  2434
 19,Altitude zone: 27000- 28499,Directions:  359/ 1440,Positions processed: 1028813,Positions processed per direction: min:  1406,max: 33711,avg:   714,real avg:  2865
 20,Altitude zone: 28500- 29999,Directions:  359/ 1440,Positions processed: 1022780,Positions processed per direction: min:  1676,max: 23241,avg:   710,real avg:  2848
 21,Altitude zone: 30000- 31499,Directions:  359/ 1440,Positions processed: 1348787,Positions processed per direction: min:  3623,max: 24247,avg:   936,real avg:  3757
 22,Altitude zone: 31500- 32999,Directions:  359/ 1440,Positions processed: 1740087,Positions processed per direction: min:  5486,max: 20297,avg:  1208,real avg:  4847
 23,Altitude zone: 33000- 34499,Directions:  359/ 1440,Positions processed: 3243962,Positions processed per direction: min:  8472,max: 30838,avg:  2252,real avg:  9036
 24,Altitude zone: 34500- 35999,Directions:  359/ 1440,Positions processed: 3806583,Positions processed per direction: min: 11193,max: 31342,avg:  2643,real avg: 10603
 25,Altitude zone: 36000- 37499,Directions:  359/ 1440,Positions processed: 8283956,Positions processed per direction: min: 20695,max: 66475,avg:  5752,real avg: 23075
 26,Altitude zone: 37500- 38999,Directions:  359/ 1440,Positions processed: 4604435,Positions processed per direction: min: 11185,max: 36281,avg:  3197,real avg: 12825
 27,Altitude zone: 39000- 40499,Directions:  359/ 1440,Positions processed: 2130623,Positions processed per direction: min:  5246,max: 15614,avg:  1479,real avg:  5934
[tedsluis@msi dump1090.socket30003]# 

I used http://www.gpsvisualizer.com/map_input to display the radar view, but my goal is to get this radar view into dump1090 one day :wink:
I used these settings:
https://dl.dropboxusercontent.com/u/17865731/dump1090.20150830/example.png

Just import the radar.csv file from the /tmp directory, add make some changes in the ā€œgeneral map parametersā€ and ā€œtrack optionsā€.

The dump1090.socket30003.radar.pl script works only with altitude in feet and distance in meters (and that is not the default output of the dump1090.socket30003.pl script), so be sure that you collect your data with dump1090.socket30003.pl with the following options: -distancematric meter -altitudematric feet

I will also make this script suitable for miles, nautical miles and kilometers in the coming week, so you have more flexibility.

Ted

Ted -

Thanks for the help! I downloaded the new source. Didnā€™t know how to update the stuff git clone gave me before so I just downloaded to a new directory. Still seems like it doesnā€™t like a negative longitude. Iā€™m at -74. Had to add 360 to it for the script to work. I thought I had data this am to play with but I have my pi reboot at midnight. That, of course, cleared /tmp and I lost everything. Itā€™s running now and Iā€™m going let it run the rest of the day and Iā€™ll report back.

Jim

Hi Jim, You cloned it again from github and it still rejects a negative lon/lat? That is strange? I have tested negative lat/lon values in two ways: 1) by changing the default lat/lon in the script and 2) by using the -lat and -lon parameters. I found no issues, but I may have mist something.
I just cloned it again from github to another host (using ā€œgit clone github.com/tedsluis/dump1090.socket30003.gitā€) and this is the output:


pi@ted1090-1 ~/git $ git clone https://github.com/tedsluis/dump1090.socket30003.git
Cloning into 'dump1090.socket30003'...
remote: Counting objects: 46, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 46 (delta 23), reused 39 (delta 16), pack-reused 0
Unpacking objects: 100% (46/46), done.
pi@ted1090-1 ~/git $ ls -l dump1090.socket30003
total 72
-rwxr-xr-x 1 pi pi  9050 Sep  2 16:19 dump1090.socket30003.heatmap.pl
-rwxr-xr-x 1 pi pi 25522 Sep  2 16:19 dump1090.socket30003.pl
-rwxr-xr-x 1 pi pi 13465 Sep  2 16:19 dump1090.socket30003.radar.pl
-rw-r--r-- 1 pi pi  9297 Sep  2 16:19 README.md
pi@ted1090-1 ~/git $ 

[pi@ted1090-1 dump1090.socket30003]# ./dump1090.socket30003.pl -lat -74.2 -lon -3.4
Using 'kilometer' the distance and 'meter' for the altitude.
Tring to connect to peer host '127.0.0.1'...
The antenna latitude & longitude are: '-74.2','-3.4'
The data directory/file is: /tmp/dump1090.socket30003.pl-127_0_0_1-150902.txt
The log  directory/file is: /tmp/dump1090.socket30003.pl-127_0_0_1-150902.log
Starting 'dump1090.socket30003.pl'....
'dump1090.socket30003.pl' (3253) is started!
Using pidfile /tmp/dump1090.socket30003.pl-127_0_0_1.pid.
positions:1253


I like to figure out what your issue is. Can you please show me your input and output please? May be you are using lat/lon ā€œ47,2ā€ (a comma) instead ā€œ47.2ā€ (a dot)?

You lost your flight positions data in /tmp after a reboot? What a pity? If you want to save files in your /tmp after a reboot you can change it using this setting:


sudo vim /etc/default/rcS
TMPTIME=14

This way files remain for 14 days.

Ted

Ted -

My apologies. .radar.pl didnā€™t like the negative longitude.



pi@piaware ~/dump1090.socket30003 $ sudo ./dump1090.socket30003.radar.pl -lat 42.7997645477918 -lon -74.3447649332153 -filemask *.txt
The maxium altitude is 48000 feet.
The minimal altitude is 0 feet.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 16.
The specified longitude '-74.3447649332153' is invalid!
pi@piaware ~/dump1090.socket30003 $ sudo ./dump1090.socket30003.radar.pl -lat 42.7997645477918 -lon 285.655235 -filemask *.txt
The maxium altitude is 48000 feet.
The minimal altitude is 0 feet.
The number of compass directions (pie slices) is 1440.
The number of altitude zones is 16.
An altitude zone is 3000 feet.
10058
Number of files read: 1
Number of position processed: 10059


Thanks for the change to /tmp

Jim