System Monitoring

[Akissack]

It seems, it as as straightforward as that Mods back in and up and running.

Now to explore some of these graphs

Al
PS Thanks - dschaper

@Akissack

I have just successfully re-installed dump1090-mutability and piaware to take advantage of these rrd graphs. I sacrificed a lot of modifications that I made in dump1090/html and will now reapply them.

However I note github.com/mutability/dump1090/ … ublic_html has different files to what I have installed. Can I just copy these in (some cover off some of my modifications, such as maps)

Allan

[Akissack]

@paulmerkx

Hi all,
I added the mlat results to the Aircraft seen graph like this:

Todays mlat outage is nicely reflected in the drop of aircraft with positions
/paul

I tried this but get a message “ERROR: opening ‘/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_mlat-recent.rrd’: No such file or directory”
I have mlat working as far as I am aware (I see military flights with posiitons)

Any idea why the file isnt being generated? Is there an edit missed?

Al

[Furt1veOne]

Thanks for the info Paulmerkx. Works great and nice to see the mlat info on the graphs now.

@paulmerkx

Hi all,
I added the mlat results to the Aircraft seen graph like this:

Projects by Keptenkurk: Image

To have this use the latest dump1090.py and dump1090.db from here:
https://raw.githubusercontent.com/mutability/dump1090-tools/master/collectd/dump1090.db
https://raw.githubusercontent.com/mutability/dump1090-tools/master/collectd/dump1090.py
and the latest dump1090mutability (1.15~dev). And mlat must be running ofc.

Change the aircraft graph part of make-graphs.sh by adding the two lines which have mlat in it…

aircraft_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 Aircraft Seen / Tracked" \
  --vertical-label "Aircraft" \
  --lower-limit 0 \
  --units-exponent 0 \
  "TEXTALIGN:center" \
  "DEF:all=$2/dump1090_aircraft-recent.rrd:total:AVERAGE" \
  "DEF:pos=$2/dump1090_aircraft-recent.rrd:positions:AVERAGE" \
  "DEF:mlat=$2/dump1090_mlat-recent.rrd:value:AVERAGE" \
  "CDEF:noloc=all,pos,-" \
  "VDEF:avgac=all,AVERAGE" \
  "VDEF:maxac=all,MAXIMUM" \
  "AREA:all#00FF00:Aircraft Seen / Tracked,   " \
  "GPRINT:avgac:Average\:%3.0lf     " \
  "GPRINT:maxac:Maximum\:%3.0lf             " \
  "LINE1:pos#0000FF:w/ Positions" \
  "LINE1:noloc#FF0000:w/o Positions" \
  "LINE1:mlat#000000:mlat"
}

Todays mlat outage is nicely reflected in the drop of aircraft with positions
/paul

[Furt1veOne]

@Akissack

@paulmerkx

Hi all,
I added the mlat results to the Aircraft seen graph like this:

Todays mlat outage is nicely reflected in the drop of aircraft with positions
/paul

I tried this but get a message “ERROR: opening ‘/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_mlat-recent.rrd’: No such file or directory”
I have mlat working as far as I am aware (I see military flights with posiitons)

Any idea why the file isnt being generated? Is there an edit missed?

Al

Change you dump1090.db and .py from the raw links posted earlier. I had that come up till I changed these

[TomMuc]

i got the same errors about missing directories/files as many others here - but didn’t find their solution ?

===== Finished webUI installation, creating initial graphs! =====
7069 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-24h.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
7078 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-7d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
7087 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-30d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
7096 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-365d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
===== Finished creating inital graphs! =====

p.s. found the answer myself - but just if others have the same errors. i had to install lighttpd server first:

sudo apt-get install lighttpd && sudo lighty-enable-mod dump1090

[joelwiley]

Loaded collectd onto the Rpi 2 yesterday, a piece of cake
loaded collectd onto the Rpi B+ today, a piece of something else.
Collectd is running, lighttpd is running dump1090-mutabilty 1.15
Before digging in and bashing on it just wondering, it should run on a B+?
Searching first for the One Dee Ten Tee errors.

[dschaper]

@joelwiley

Loaded collectd onto the Rpi 2 yesterday, a piece of cake
loaded collectd onto the Rpi B+ today, a piece of something else.
Collectd is running, lighttpd is running dump1090-mutabilty 1.15
Before digging in and bashing on it just wondering, it should run on a B+?
Searching first for the One Dee Ten Tee errors.

What OSes are they running? Both Raspbian Jessie or something else?

[joelwiley]

@dschaper

lsb_release -a results (edited)
What OSes are they running? Both Raspbian Jessie or something else?

Rpi B+ Distributor ID: Debian Description: Debian GNU/Linux 7.8 (wheezy) Release: 7.8 Codename: wheezy
RPi 2 Distributor ID: Debian Description: Debian GNU/Linux 7.8 (wheezy) Release: 7.8 Codename: wheezy

[mibzzer15]

@jprochazka

@mibzzer15

Dude…dude…can you please share that awesome page? Thanks!

Here you go.
Might need to modify image paths and such however.
It’s a simple bootstrap layout. Any external linking items such as JavaScript and CSS are pulled from CDNs so no need to add any other files.

This one actually differs a little bit from the one I posted. It is a version from installation I am working on which utilizes a cleaner configuration of collectd.

Seriously thank you! It looks really good. I’ve got it %50 working right now. I had to change the “localhost” option to “rpi” to get the regular dump1090 graphs to come up. I think I need some help getting the Message rate (trailing) graph and signal graph working along with all the system graphs. Do you think you can help me figure that out? I figure it might be something I need to add into the make-graphs.sh file or something, I’m really not sure…

[TomMuc]

since 2 days i run these graphs and have to state they are great - thanks to jepolch posting this here and obj and xforce30164 for their coding!

what i’m wondering is - does one of you all have found a good solution against spiking?
with spiking i talk about e.g. when rebooting the pi (or through other situations) on a graph where normally are e.g. 150 aircrafts seen and in high traffic hours up to 200 - the y-axis has e.g. a max value of 300. after ‘spiking’ it of course jumps to e.g. 3,000 max - and as side-effect the graphs are hard to read from this moment on.

possible ways for workarounds could be hardcoding the max y-scale or better solution building y-scale only along data over longer periods.
does one of you already have a good solution?

tom

[clipsnotebook]

So we have two groups of people here - those who have the graphs running and the other - have nots. The have nots seem to be mostly stuck with the directory error pointed out by TomMuc earlier and I for one have not been able to make it work even with lighthttpd running (I installed lighthttpd as I installed dump1090-mutability)

For the benefit of the have nots please may I request someone to assist us to transition from the have nots to the “have” group.

Many thanks.

[gmasem]

I was getting those file not found messages as well. Two things I had to do, if I remember correctly. First I don’t think I had rrdtool installed. Second, for some reason, there the file path is pointing at locations within /var/lib/collectd/rrd/localhost on my installation some (only some…) of the files were actually located under /var/lib/collectd/rrd/ and I had to modify one of the scripts (make-graphs.sh, if I recall correctly, to generate the images based on the different location.

[jepolch]

@gmasem

I was getting those file not found messages as well. Two things I had to do, if I remember correctly. First I don’t think I had rrdtool installed. Second, for some reason, there the file path is pointing at locations within /var/lib/collectd/rrd/localhost on my installation some (only some…) of the files were actually located under /var/lib/collectd/rrd/ and I had to modify one of the scripts (make-graphs.sh, if I recall correctly, to generate the images based on the different location.

I know this won’t make you feel any better, but the script gives me the same errors now as you are seeing. When I set up a new Pi recently I ran the script and got the exact errors as TomMuc. Fortunately, I back up my Pi’s every night, so I could take the backed up version of the data and graphics to the new Pi’s directory structure. For what it’s worth, I did install collectd and rrdtool before I ran the script. All I can suggest is that people visit the PlaneFinder thread and post the question there and hope the author of the script replies. I know he’s a college student and has been busy.

forum.planefinder.net/threads/we … ation.256/

Edit: I meant to reply to clipsnotebook.

[TomMuc]

hope it helps others here in the forum a little - these were the steps i finally installed the graph-thing on my raspi (raspbian wheezy, dump1090-mutability 1.15, piaware 2.13). while installing i got some errors here and there - but anything works fine:

it is always a good idea to update the system first:

sudo apt-get update
sudo apt-get upgrade

if already activated - deactivate dump1090 builtin webserver (set to ‘0’ instead of ‘8080’) in dump1090-mutability v1.15~dev config utility

sudo dpkg-reconfigure dump1090-mutability

install needed webserver software if not already installed:

sudo apt-get install lighttpd && sudo lighty-enable-mod dump1090
sudo shutdown -r now

install stats and graph software:

sudo apt-get install rrdtool && sudo apt-get install collectd
sudo shutdown -r now
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"

the following command has to be run twice if there is not already a crontab for sudo:

cd dump-tools
sudo ./install.sh "crontab"
sudo shutdown -r now

=> if anything went the right way that’s it and you should see the first graphs via webinterface:
ip-address-of-your-pi/collectd

for later altering some graphs if desired these two files are the most important:

sudo nano /var/www/collectd/index.html
sudo nano /home/pi/dump-tools/collectd/make-graphs.sh

here you have to make changes too if you want additional graphs auto-updated:

sudo crontab -e

e.g.
*/5 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 1h >/dev/null
*/10 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 6h >/dev/null
2,12,22,32,42,52 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 24h >/dev/null
4,24,44 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 7d >/dev/null
6 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 30d >/dev/null
8 */12 * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 365d >/dev/null

with commands like this you can force to rebuild graphs instantly:

sudo ~/dump-tools/collectd/make-graphs.sh 1h
sudo ~/dump-tools/collectd/make-graphs.sh 7d

with commands like this you can reset the databases to get fresh graphs (be cautious all old data will be destroyed)
warning: sudo when using ‘rm’ in combination with wildcards ‘*’ it could delete much more than you intended to - up to a broken system!

sudo rm /var/lib/collectd/rrd/localhost/dump1090-rpi/*.rrd
sudo rm /var/lib/collectd/rrd/localhost/table-rpi/*.rrd

[clipsnotebook]

TomMuc,
I followed your instructions word for word and the end result is the same: just the temperature graph is shown.

I still end up the same errors:

===== Finished webUI installation, creating initial graphs! =====
2544 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
2553 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
2562 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
2571 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory

The one thing I notices was that the script is referencing /var/lib/collectd/rrd/localhost/dump1090-rpi/ but the folder dump1090-rpi does not exist and that may be causing the issue. However, I am yet to figure out how to correct this. The path I think the path is referenced from the /home/pi/dump-tools/collectd/make-graphs.sh script. Since I run dump1090-mutability, would that be the reason?

[TomMuc]

do you have these directories and if so - whats inside?

/var/lib/collectd/rrd
/var/lib/collectd/rrd/localhost
/var/lib/collectd/rrd/localhost/dump1090-rpi
/var/lib/collectd/rrd/localhost/table-rpi

[clipsnotebook]

Here’s what I have:

/var/lib/collectd/rrd:
localhost

/var/lib/collectd/rrd/localhost:
table-rpi

/var/lib/collectd/rrd/localhost/dump1090-rpi:
the folder dump1090-rpi does not exist

/var/lib/collectd/rrd/localhost/table-rpi:
gauge-cpu_temp.rrd

[TomMuc]

my first try would be simply to create ‘dump1090-rpi’ within localhost and then running the install-script again.
is your os raspbian-wheezy?

[abcd567]

@TomMuc

my first try would be simply to create ‘dump1090-rpi’ within localhost and then running the install-script again.
is your os raspbian-wheezy?

I feel lucky. I have installed “collected” 4 times on Wheezy using web_install.sh without any problems.

The first & second installations were on my RPi BPlus/Wheezy and RPi Model2/Wheezy.
The third & fourth installations were again on above two RPi, after I installed new microSD cards with fresh copies of Wheezy.

[clipsnotebook]

Yup tried creating the directory but the errors were still the same

I am running:
$ lsb_release -da
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 7.8 (wheezy)
Release: 7.8
Codename: wheezy