Merging old graphs1090 data

turbodogfl · June 5, 2021, 3:06pm

Have done something and graphs1090 only shows data since May 27. There is a directory named /var/lib/collectd/rrd/2021-05-27. Is it possible to merge the rrd files with the ones in localhost? Have tried copying the folder to /tmp and running Backup and restore GitHub - wiedehopf/graphs1090: Graphs for readsb / dump1090-fa / dump1090 (based on dump1090-tools by mutability) but have been unsuccessful. Any suggestions or should I give up and enjoy the data since 05-27?

pi@c16494:/var/lib/collectd/rrd $ ls -l
total 15088
drwxr-xr-x  9 root root     4096 Jun  4 16:14 2021-05-27
drwxr-xr-x 10 root root     4096 Jun  2 11:16 2021-06-02
drwxr-xr-x 10 root root     4096 Jun  4 15:59 2021-06-04
drwxr-xr-x 10 root root     4096 May 30 16:42 graphs1090-writeback-backup1
drwxr-xr-x 10 root root     4096 May 30 16:42 graphs1090-writeback-backup2
drwxr-xr-x 10 root root     4096 May 30 16:42 localhost
drwxr-xr-x  8 root root     4096 Nov 18  2019 rme_rra_backup
pi@c16494:/var/lib/collectd/rrd $ ls -l 2021-05-27
total 15088
drwxr-xr-x 2 root root     4096 Jun  2 11:23 aggregation-cpu-average
drwxr-xr-x 2 root root     4096 Jun  2 11:23 df-root
drwxr-xr-x 2 root root     4096 Jun  2 11:23 dump1090-localhost
drwxr-xr-x 2 root root     4096 Jun  2 11:23 interface-eth0
drwxr-xr-x 2 root root     4096 Jun  2 11:23 interface-wlan0
-rw-r--r-- 1 root root 15421440 Jun  4 16:16 rrd.tar
drwxr-xr-x 2 root root     4096 Jun  2 11:23 system_stats
drwxr-xr-x 2 root root     4096 Jun  2 11:23 table-localhost

The last experiment tried to tar 2021-05-27 and copy to /tmp then restore

pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ sudo rm rrd.tar
pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ sudo tar cf rrd.tar .
tar: ./rrd.tar: file is the archive; not dumped
pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ sudo tar cf rrd.tar *
tar: rrd.tar: file is the archive; not dumped
pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ ls -l
total 15088
drwxr-xr-x 2 root root     4096 Jun  2 11:23 aggregation-cpu-average
drwxr-xr-x 2 root root     4096 Jun  2 11:23 df-root
drwxr-xr-x 2 root root     4096 Jun  2 11:23 dump1090-localhost
drwxr-xr-x 2 root root     4096 Jun  2 11:23 interface-eth0
drwxr-xr-x 2 root root     4096 Jun  2 11:23 interface-wlan0
-rw-r--r-- 1 root root 15421440 Jun  4 16:16 rrd.tar
drwxr-xr-x 2 root root     4096 Jun  2 11:23 system_stats
drwxr-xr-x 2 root root     4096 Jun  2 11:23 table-localhost
pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ cp rrd.tar /tmp
pi@c16494:/var/lib/collectd/rrd/2021-05-27 $ sudo /usr/share/graphs1090/rrd-integrate-old.sh /tmp
Warning: The unit file, source configuration file or drop-ins of collectd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-heatmap_and_state.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-background.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-peak_signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_aircraft-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-remove_stale.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_range-minimum.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-demod.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-positions.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_tracks-single_message.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-noise.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-local_accepted.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_range-max_range.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-remote_accepted_0.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_range-quart1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-median.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-aircraft_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_mlat-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-quart1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-min_signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-trace_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-strong_signals.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-globe_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_cpu-reader.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_range-median.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-local_accepted_0.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-quart3.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_dbfs-NaN.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_range-quart3.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_tracks-all.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-remote_accepted_1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_gps-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_tisb-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-local_accepted_1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./dump1090-localhost/dump1090_messages-remote_accepted.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-wlan0/if_errors.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-wlan0/if_packets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-wlan0/if_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-wlan0/if_dropped.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-eth0/if_errors.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-eth0/if_packets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-eth0/if_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./interface-eth0/if_dropped.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-idle.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-softirq.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-steal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-interrupt.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-nice.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-system.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-wait.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./aggregation-cpu-average/cpu-user.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./table-localhost/gauge-cpu_temp.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./system_stats/memory-used.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./system_stats/memory-buffers.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./system_stats/memory-cached.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./system_stats/memory-free.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./disk-mmcblk0/disk_ops.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./disk-mmcblk0/disk_time.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./disk-mmcblk0/disk_merged.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./disk-mmcblk0/disk_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./disk-mmcblk0/disk_io_time.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./df-root/df_complex-reserved.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./df-root/df_complex-used.rrd.gz: No such file or directory
ERROR: error checking for source RRD /tmp/./df-root/df_complex-free.rrd.gz: No such file or directory
Warning: The unit file, source configuration file or drop-ins of collectd.service changed on disk. Run 'systemctl daemon-reload' to reload units.



All done!

wiedehopf · June 5, 2021, 3:30pm

You could replace the current dataset with the data from back then.
Then you only have a week or two as a gap.

As a first step run the graphs1090 install so you’re on the current version.
Do a backup as it is in the instructions (just in case … )

Then:

sudo systemctl stop collectd
sudo cp -afT /var/lib/collectd/rrd/2021-05-27 /var/lib/collectd/rrd/localhost
sudo sytemctl restart collectd
sudo systemctl restart graphs1090

Wait a minute and you should be showing graphs that have a gap for the last 14 days or whatever.

Merging is possible but i’d have to look it up and it’s laborious … i’d probably have to write scripting or you’d have no chance of completing the process (no offence).
And i’m not gonna write scripting, sorry

turbodogfl · June 5, 2021, 3:42pm

Will try using the old data. Looked like https://github.com/wiedehopf/graphs1090/blob/master/rrd-integrate-old.sh was supposed to merge data. Yes, there are many rrd files to merge.

wiedehopf · June 5, 2021, 3:49pm

Seems like it.

I mean … do backups of both data … look at the file syntax and try it.

sudo /usr/share/graphs1090/rrd-integrate-old.sh /var/lib/collectd/rrd/2021-05-27

Looks like this would be the syntax at a glance.

turbodogfl · June 5, 2021, 5:08pm

Thanks for your suggestions
after another try and puzzling result at bottom used your 1st suggestion and used the data in

sudo systemctl stop collectd
sudo cp -afT /var/lib/collectd/rrd/2021-05-27 /var/lib/collectd/rrd/localhost
sudo systemctl restart collectd
sudo systemctl restart graphs1090

Nice to see the old data back to Jan 2020 have tar files of the recent data
Am puzzled why it was looking for .gz files

pi@c16494:/var/lib/collectd/rrd $ sudo systemctl stop collectd
pi@c16494:/var/lib/collectd/rrd $ sudo /usr/share/graphs1090/rrd-integrate-old.sh /var/lib/collectd/rrd/2021-05-27
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-heatmap_and_state.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-background.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-peak_signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_aircraft-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-remove_stale.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_range-minimum.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-demod.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-positions.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_tracks-single_message.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-noise.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-local_accepted.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_range-max_range.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-remote_accepted_0.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_range-quart1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-median.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-aircraft_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_mlat-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-quart1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-min_signal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-trace_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-strong_signals.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-globe_json.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_cpu-reader.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_range-median.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-local_accepted_0.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-quart3.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_dbfs-NaN.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_range-quart3.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_tracks-all.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-remote_accepted_1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_gps-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_tisb-recent.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-local_accepted_1.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./dump1090-localhost/dump1090_messages-remote_accepted.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-wlan0/if_errors.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-wlan0/if_packets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-wlan0/if_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-wlan0/if_dropped.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-eth0/if_errors.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-eth0/if_packets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-eth0/if_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./interface-eth0/if_dropped.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-idle.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-softirq.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-steal.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-interrupt.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-nice.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-system.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-wait.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./aggregation-cpu-average/cpu-user.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./table-localhost/gauge-cpu_temp.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./system_stats/memory-used.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./system_stats/memory-buffers.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./system_stats/memory-cached.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./system_stats/memory-free.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./disk-mmcblk0/disk_ops.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./disk-mmcblk0/disk_time.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./disk-mmcblk0/disk_merged.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./disk-mmcblk0/disk_octets.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./disk-mmcblk0/disk_io_time.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./df-root/df_complex-reserved.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./df-root/df_complex-used.rrd.gz: No such file or directory
ERROR: error checking for source RRD /var/lib/collectd/rrd/2021-05-27/./df-root/df_complex-free.rrd.gz: No such file or directory



All done!
'''

wiedehopf · June 5, 2021, 7:06pm

Heh yeah … forgot that.
You’d have to run the script to gunzip the files.

I suppose you can still do that.

sudo systemctl stop collectd

Move the data you for the last 10 day or so back to /var/lib/collectd/rrd/localhost

sudo /usr/share/graphs1090/gunzip.sh /var/lib/collectd/rrd/localhost
sudo /usr/share/graphs1090/rrd-integrate-old.sh /var/lib/collectd/rrd/2021-05-27
sudo systemctl restart collectd
sudo systemctl restart graphs1090

Like this …
It uses the /var/lib/collectd/rrd/localhost as a template … and graphs1090 now only saves to disk gzipped once a day to prevent early sd-card fatigue.
(only writes 10 MB a day compared to up to 50 MB per hour before)
Anyhow for the integration script to work the files need to be uncompressed .

You backed up the last 10 days as i mentioned?

turbodogfl · June 6, 2021, 2:07pm

found through rrdtool dump the missing data still exists in /var/lib/collectd/rrd/2021-06-04/dump1090-localhost

pi@c16494:/var/lib/collectd/rrd/2021-06-04/dump1090-localhost $rrdtool  dump dump1090_messages-local_accepted.rrd

                        <!-- 2021-05-27 00:26:00 EDT / 1622089560 --> <row><v>NaN</v></row>
                        <!-- 2021-05-27 09:19:00 EDT / 1622121540 --> <row><v>NaN</v></row>
                        <!-- 2021-05-27 18:12:00 EDT / 1622153520 --> <row><v>5.433213889e+02</v></row>
                        <!-- 2021-05-28 03:05:00 EDT / 1622185500 --> <row><v>4.385116667e+02</v></row>
                        <!-- 2021-05-28 11:58:00 EDT / 1622217480 --> <row><v>5.057183333e+02</v></row>
                        <!-- 2021-05-28 20:51:00 EDT / 1622249460 --> <row><v>5.909233333e+02</v></row>
                        <!-- 2021-05-29 05:44:00 EDT / 1622281440 --> <row><v>2.734708333e+02</v></row>
                        <!-- 2021-05-29 14:37:00 EDT / 1622313420 --> <row><v>5.123483333e+02</v></row>
                        <!-- 2021-05-29 23:30:00 EDT / 1622345400 --> <row><v>3.963619444e+02</v></row>
                        <!-- 2021-05-30 08:23:00 EDT / 1622377380 --> <row><v>2.076683333e+02</v></row>
                        <!-- 2021-05-30 17:16:00 EDT / 1622409360 --> <row><v>4.416511111e+02</v></row>
                        <!-- 2021-05-31 02:09:00 EDT / 1622441340 --> <row><v>3.935927778e+02</v></row>
                        <!-- 2021-05-31 11:02:00 EDT / 1622473320 --> <row><v>4.394933333e+02</v></row>
                        <!-- 2021-05-31 19:55:00 EDT / 1622505300 --> <row><v>5.605627778e+02</v></row>
                        <!-- 2021-06-01 04:48:00 EDT / 1622537280 --> <row><v>3.674300000e+02</v></row>
                        <!-- 2021-06-01 13:41:00 EDT / 1622569260 --> <row><v>4.029538889e+02</v></row>
                        <!-- 2021-06-01 22:34:00 EDT / 1622601240 --> <row><v>3.955527778e+02</v></row>
                        <!-- 2021-06-02 07:27:00 EDT / 1622633220 --> <row><v>2.208783333e+02</v></row>
                        <!-- 2021-06-02 16:20:00 EDT / 1622665200 --> <row><v>4.480505556e+02</v></row>
                        <!-- 2021-06-03 01:13:00 EDT / 1622697180 --> <row><v>3.877019444e+02</v></row>
                        <!-- 2021-06-03 10:06:00 EDT / 1622729160 --> <row><v>4.318919444e+02</v></row>
                        <!-- 2021-06-03 18:59:00 EDT / 1622761140 --> <row><v>4.838016667e+02</v></row>
                        <!-- 2021-06-04 03:52:00 EDT / 1622793120 --> <row><v>3.056583333e+02</v></row>
                        <!-- 2021-06-04 12:45:00 EDT / 1622825100 --> <row><v>5.066516667e+02</v></row>
                </database>
        </rra>
</rrd>

how can this be merged?

turbodogfl · June 6, 2021, 6:56pm

pi@c16494:/var/lib/collectd/rrd $ ls -l
total 142540
drwxr-xr-x 9 root root 4096 Jun 4 16:14 2021-05-27
drwxr-xr-x 10 root root 4096 Jun 2 11:16 2021-06-02
drwxr-xr-x 10 root root 4096 Jun 4 15:59 2021-06-04
drwxr-xr-x 10 root root 4096 Jun 5 16:24 2021-06-05
drwxr-xr-x 10 root root 4096 Jun 6 09:12 2021-06-06
drwxr-xr-x 10 root root 4096 Jun 4 16:14 graphs1090-writeback-backup1
drwxr-xr-x 10 root root 4096 Jun 4 16:14 graphs1090-writeback-backup2
drwxr-xr-x 10 root root 4096 Jun 4 16:14 localhost
drwxr-xr-x 8 root root 4096 Nov 18 2019 rme_rra_backup
-rw-r–r-- 1 root root 34703360 Jun 6 14:25 rrd0605.tar
-rw-r–r-- 1 root root 30238720 Jun 5 12:22 rrdl.tar
-rw-r–r-- 1 root root 30842880 Jun 5 12:23 rrdo.tar
-rw-r–r-- 1 root root 34703360 Jun 6 14:39 rrd.tar
-rw-r–r-- 1 root root 15421440 Jun 2 11:26 tar0602.tar
pi@c16494:/var/lib/collectd/rrd $

One file rrd.tar should be /var/lib/collectd/rrd/
have made backup tar files for directories in /var/lib/collectd/rrd

pi@c16494:/var/lib/collectd/rrd $ ls -l
total 142540
drwxr-xr-x 9 root root 4096 Jun 4 16:14 2021-05-27
drwxr-xr-x 10 root root 4096 Jun 2 11:16 2021-06-02
drwxr-xr-x 10 root root 4096 Jun 4 15:59 2021-06-04
drwxr-xr-x 10 root root 4096 Jun 5 16:24 2021-06-05
drwxr-xr-x 10 root root 4096 Jun 6 09:12 2021-06-06
drwxr-xr-x 10 root root 4096 Jun 4 16:14 graphs1090-writeback-backup1
drwxr-xr-x 10 root root 4096 Jun 4 16:14 graphs1090-writeback-backup2
drwxr-xr-x 10 root root 4096 Jun 4 16:14 localhost
drwxr-xr-x 8 root root 4096 Nov 18 2019 rme_rra_backup
-rw-r–r-- 1 root root 34703360 Jun 6 14:25 rrd0605.tar
-rw-r–r-- 1 root root 30238720 Jun 5 12:22 rrdl.tar
-rw-r–r-- 1 root root 30842880 Jun 5 12:23 rrdo.tar
-rw-r–r-- 1 root root 34703360 Jun 6 14:39 rrd.tar
-rw-r–r-- 1 root root 15421440 Jun 2 11:26 tar0602.tar
pi@c16494:/var/lib/collectd/rrd $

![Screenshot 2021-06-06 145559|690x213](upload://mPQY67Dhc18kqfjFemuvr9A5dVT.jpeg)

wiedehopf · June 6, 2021, 9:17pm

sudo systemctl stop collectd
sudo cp -afT  /var/lib/collectd/rrd/2021-06-04/ /var/lib/collectd/rrd/localhost
sudo /usr/share/graphs1090/gunzip.sh /var/lib/collectd/rrd/localhost
sudo /usr/share/graphs1090/gunzip.sh /var/lib/collectd/rrd/2021-05-27
sudo /usr/share/graphs1090/rrd-integrate-old.sh /var/lib/collectd/rrd/2021-05-27
sudo systemctl restart collectd
sudo systemctl restart graphs1090

Try this.
If it doesn’t work … i’ll call it done.

turbodogfl · June 6, 2021, 9:38pm

Thanks for the magic solution!

system · June 6, 2022, 9:39pm

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Is there a way to transfer graphs1090 history to a new install? ADS-B Flight Tracking	6	720	December 26, 2019
Graphs for dump1090 -- my version with install script ADS-B Flight Tracking	1325	51815	December 22, 2023
Restoring graphs1090 error on different architecture ADS-B Flight Tracking	4	347	March 11, 2022
Graphs1090 and Dump1090-fa running on different Raspberry Pi ADS-B Flight Tracking	14	1622	January 5, 2023
Clear graphs1090 Data ADS-B Flight Tracking	4	501	March 17, 2022

Merging old graphs1090 data

Related Topics