System Monitoring


#1

Note: For anyone just starting out to set up monitoring scripts, I recommend you bypass this thread and go straight to ads-b-flight-tracking-f21/ads-b-feeder-project-setup-scripts-t36532.html There you’ll find a better solution, in my opinion. Thanks to JProchazka for his excellent ADS-B Feeder project!

There is a lot of chatter here about tweaking dump1090-mutability. If you’re not monitoring your system with graphs, you’re really only getting a pinhole view of what’s going on. PlaneFinder member xforce30164 has made it relatively easy to add rrdtool graphs (based on the original work of @obj) with his web install script. Here’s a link to his blog: blog.ayadex.com/2015/04/18/dump1 … on-script/

Here’s a screenshot of my 6-hour log view. You can easily see where I made changes to the gain settings in dump by looking at the ADS-B Signal graph.


Is it worth it to mount FA antenna on roof?
One hour outage and recovery of Dump1090
Monitoring dump1090
My recent RPi2 SD card rebuild helper list..
#2

I’ve taken it one step further by making up a little “recipe” I refer to when I reinstall Wheezy (although I’m sure it will work on a Piaware SD card install).

Install graphing stuff:


sudo apt-get install rrdtool && sudo apt-get install collectd 
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"
cd ~/pi/dump-tools
sudo ./install.sh "crontab"



#3

Thanks!


#4

After doing


sudo ./web_install.sh "dump-tools"

I get:


===== Finished webUI installation, creating initial graphs! =====
13145 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
13154 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
13163 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
13172 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-recent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all.rrd': No such file or directory
577x259
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd': No such file or directory
===== Finished creating inital graphs! =====
First part of installation is finished. please check http://<rpi-ip>/collectd and see if you see any graphs.
If that is the case please run sudo ./install.sh "crontab" from the /home/pi/dump-tools diretory to enable automatic graph creation.

I’m missing /var/lib/collectd/rrd/localhost/dump1090-rpi
How does that get installed?


#5

I have exactly the same problem. The only graph I am getting is the core temperature graph which is actually working!


#6

I just took a look at the Wiki (linked in my first post), which gives a link to the PlaneFinder thread where xforce30164 provides support for his scripts.

There’s a later version of his script. I don’t know why the version listed above doesn’t work on your Pi - is it a Piaware SD install? Or are you running Piaware-mutability? Both my Pi’s are running Raspbian Wheezy with stock Piaware installed using the .deb file. Here’s a link to the thread on PlaneFinder and also the commands to get the newer script and update your install. I hope this fixes it.

forum.planefinder.net/threads/we … ation.256/


sudo wget http://raspi.ayadex.com/volatile/web_install.0.0.2.sh
sudo cp web_install.0.0.2.sh web_install.sh
sudo rm web_install.0.0.2.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools" "remove-old"


#7

You may have to restart collectd after running the web install script, because the script replaces /etc/collectd/collectd.conf.


sudo /etc/init.d/collectd restart


#8

I’m sorry, but this didn’t work for me. I’m running a Pi2 with dump1090-mutability and the lighttpd webserver. Don’t know what’s wrong, it said it completed correctly, but there is no graphs and the “crontab” commands doesn’t work either.

…Tom


#9

The crontab install may fail the first time if root has no existing crontab. Should work if you run it again - make sure to ‘sudo’. I installed on a Pi2 running dump1090-mutability using the “recipe” in my second post. If you really want to get it working, please read the Wiki and the PlaneFinder thread. It works, or I wouldn’t have had graphs to show above. :slight_smile:


#10

Retried the crontab command and all is good. Graphs wouldn’t display until lighttpd was restarted. All good now, thank you.

…Tom


#11

I don’t know why I didn’t have the dump1090-rpi folder but I created it manually, reinstalled everything from scratch and rebooted.

Another problem I found was that I had to delete “dump1090-tools-old” folder manually or the reinstall would fail, even with the “remove-old” option.

The graphs are finally working. Now I just have to figure out how to read them. :laughing:


#12

Can I ask how you managed to get your range graph displaying in nautical miles?

Mine is showing kilometres.


#13

Your Collectd graph page shows a [hour] & [6 hour] selection at the top of the page. Will those options appear after 6 hours of data is collected? And you have some different graphs then the default install. Are there options or a config file to Collectd?


#14

Glad people are getting it working, add these lines to /var/www/collectd/index.html above the line that reads " day ]" to get one hour and six hour graphs


 <a href="#" onclick="switchView('1h')"> hour ]</a>
     <a href="#" onclick="switchView('6h')"> 6 hour ]</a>


The one hour graph is especially useful when making changes to your gain settings. You will see results in 10 minutes or less.

To get (almost) immediate results, run


cd /home/pi/dump-tools/collectd
sudo make-graphs.sh 1h
sudo make-graphs.sh 6h

After you add the lines to index.html, add the following lines to root’s crontab (use ‘sudo crontab -e’)


*/5 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 1h >/dev/null
*/10 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 6h >/dev/null


To change from Metric to Imperial values, add these lines in /home/pi/dump-tools/collectd/make-graphs.sh


imperial_range_graph(){
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 max range" \
  --vertical-label "Miles" \
  --lower-limit 0 \
  --units-exponent 0 \
  "DEF:rangem=$2/dump1090_range-max_range.rrd:value:MAX" \
  "CDEF:rangekm=rangem,1000,/" \
  "CDEF:rangeml=rangekm,0.621371192,*" \
  "LINE1:rangeml#0000FF:max range"
}

imperial_temp_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "Core Temperature" \
  --vertical-label "Degrees Fahrenheit" \
  --lower-limit 32 \
  --upper-limit 212 \
  --rigid \
  --units-exponent 1 \
  "DEF:traw=$2/gauge-cpu_temp.rrd:value:MAX" \
  "CDEF:tta=traw,1000,/" \
  "CDEF:ttb=tta,32,+" \
  "CDEF:ttc=ttb,1.8,*" \
  "AREA:ttc#ffcc00"
}



Commend out the metric_temp_graph and metric_range_graph lines near the bottom of the script (add a # at the beginning of the line) and add these lines


 imperial_temp_graph /var/www/collectd/table-$2-core_temp-$4.png /var/lib/collectd/rrd/$1/table-$2 "$3" "$4" "$5"
  imperial_range_graph /var/www/collectd/dump1090-$2-range-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"


A “commented out” line will look like this


#  metric_range_graph /var/www/collectd/dump1090-$2-range-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"


I hope I answered all the questions and got the lines exactly right. I did a lot of copying and pasting. Let me know if these changes don’t work for you.


#15

I think I know why it may not be working for people using the stock Piaware SD card image. I remember a comment by @caius that the version of rrdtool in Piaware is older than the one in Piaware - I think that’s correct. I urge everyone who has it working or is trying to get it working to read the PlaneFinder thread I mentioned above.

Also, if you want to get the trailing rate graph that you see in my screenshot above, you need to add a lot of stuff to your make-graphs.sh. It might be better to just get a copy of mine here: dropbox.com/s/u7bt1qvuqp6dn … hs.sh?dl=0

In fact, grab a copy of my /var/www/collectd/index.html here to save yourself some typing: dropbox.com/s/a466njyj0q17u … .html?dl=0

And just to make sure we’re all on the same page, use my /etc/collectd/collectd.conf: dropbox.com/s/yozibpk8l2k6a … .conf?dl=0


#16

Credit for the trailing rate graph goes to @caius. :smiley:


#17

Great job, very concise. And I was able to make all your suggested changes successfully. One additional question I would have is to how to get the RPi ADS-B Signal graph to be displayed?

Thank you, Tom


#18

That should have been included in the /var/www/collectd/index.html that was installed by the web install script and it’s a function of /home/pi/dump-tools/collectd/make-graphs.sh. In index.html you should have this line:


<img id="signal" width="577" height="273" src="dump1090-rpi-signal-24h.png">

And in make-graphs.sh you should have this block of code:


signal_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 signal" \
  --vertical-label "dBFS" \
  --upper-limit 0    \
  --lower-limit -50  \
  --rigid            \
  --units-exponent 0 \
  "DEF:signal=$2/dump1090_dbfs-signal.rrd:value:AVERAGE" \
  "DEF:peak=$2/dump1090_dbfs-peak_signal.rrd:value:AVERAGE" \
  "CDEF:us=signal,UN,-100,signal,IF" \
  "AREA:-100#00FF00:mean signal power" \
  "AREA:us#FFFFFF" \
  "LINE1:peak#0000FF:peak signal power"
}


make-graphs.sh produces the image. Look in /var/www/collectd for a file name like this “dump1090-rpi-signal-24h.png”


#19

I think you need to be running dump1090 mutability with oversampling enabled to get the signal strength graph. It provides different stats to the standard decoder.

The version of rrdtool (which creates the graphs) that a piaware image will install is likely the same as raspbian, as that is what it’s based on. I use arch Linux which is a rolling release so generally has much more up to date packages. The raspbian version should work fine.

The problem with the older version of rrdtool was that certain features that I used when I made the trailing graphs aren’t in it. These related to the way gaps in the data are handled. It’s not a major problem, just that the minimum/maximum area will look a bit different when there are gaps. For continuous data, there won’t be any difference.


#20

Thanks again, I downloaded your files from your drop-box and all is good.
…Tom