Cloning Booted RPi's microSD card on Windows PC through Network

It’s incredibly slow on a Pi3B but I don’t know if that’s due to network speed to the NAS or just the speed of the Pi itself. It took an hour and a half to backup a 16Gb card.

A nice touch is being able to disable and enable services so I’ve told it to stop vnstat while it’s doing the backup so it doesn’t mess with my daily network statistics.

/edit - Boy did it thrash the Pi as well…

That’s in my loft and has no additional cooling - I wouldn’t normally run it during the day so the ambient temperature will be lower overnight by at least 10°C. It didn’t throttle at all though so that’s good.

It’ll be interesting to see how well it runs on the main Pi up my mast overnight.

Sure, i got the point and have to admit you’re right

3 Likes

ADDITIONAL STUFF

(1) Location of executable script after installation is completed

/usr/local/bin/raspiBackup.sh

(2) Location of cron file

/etc/cron.d/raspiBackup  

(3) To Start raspiBackup with Systemd Instead of Cron

https://github.com/framps/raspiBackup/tree/master/installation/systemd

(4) Use raspiBackup without installation to create a backup immediately

https://www.linux-tips-and-tricks.de/en/quickstart-rbk/#a8

  1. Download raspiBackup
    curl -sSLO https://www.linux-tips-and-tricks.de/raspiBackup.sh

  2. Run backup script by following command. Pass the backup partition as last parameter when invoking raspiBackup.
    sudo bash ./raspiBackup.sh -a : -o : -z -m detailed /home/pi/Windows-Share

  3. If no dd backup should be created pass the backup type tar or rsync with option -t , e.g. sudo bash ./raspiBackup.sh -t tar or sudo bash ./raspiBackup.sh -t rsync

  4. Help for all other options of raspiBackup is displayed with bash ./raspiBackup.sh -h

(5) HELP MENU

pi@raspberrypi:~ $ sudo raspiBackup.sh -h 

raspiBackup.sh 0.6.5.1, 2020-06-08/15:28:45 - a056338
Usage: raspiBackup.sh [option]* {backupDirectory}
 -General options-
-b {dd block size} (default: 1M)
-D "{additional dd parameters}" (default: no)
-e {email address} (default: no)
-E "{additional email call parameters}" (default: no)
-f {config filename}
-g Display progress bar
-G {message language} (EN or DE) (default: EN)
-h display this help text
-l {log level} ( Debug | Off) (default: Debug)
-m {message level} ( Detailed | Minimal) (default: Minimal)
-M {backup description}
-n notification if there is a newer scriptversion available for download (default: yes)
-s {email program to use} (mail,ssmtp,msmtp,sendEmail,mailext) (default: mail)
--timestamps Prefix messages with timestamps (default: no)
-u "{excludeList}" List of directories to exclude from tar and rsync backup
-U current script version will be replaced by the most recent version. Current version will be saved and can be restored with parameter -V
-v verbose output of backup tools (default: no)
-V restore a previous version
-z compress backup file with gzip (default: yes)

 -Backup options-
-a "{commands to execute after Backup}" (default: systemctl start udisks2 && systemctl start triggerhappy && systemctl start ssh && systemctl start skybupmx && systemctl start rsyslog && systemctl start rpi-eeprom-update && systemctl start rng-tools && systemctl start rc-local && systemctl start rbfeeder && systemctl start raspi-config && systemctl start polkit && systemctl start piaware && systemctl start pfclient && systemctl start opensky-feeder && systemctl start nmbd && systemctl start networking && systemctl start mm2 && systemctl start mlat-client && systemctl start lighttpd && systemctl start kmod-static-nodes && systemctl start keyboard-setup && systemctl start ifupdown-pre && systemctl start hciuart && systemctl start graphs1090 && systemctl start gldriver-test && systemctl start generate-pirehose-cert && systemctl start fr24feed && systemctl start fake-hwclock && systemctl start dump1090-fa && systemctl start dphys-swapfile && systemctl start dhcpcd && systemctl start cron && systemctl start dbus && systemctl start console-setup && systemctl start collectd && systemctl start bluetooth && systemctl start bluealsa && systemctl start avahi-daemon && systemctl start alsa-state && systemctl start alsa-restore && systemctl start adsbx-socat && systemctl start adsbx-mlat)
-B Save bootpartition in tar file (Default: 0)
-F Backup is simulated
-k {backupsToKeep} (default: 3)
-o "{commands to execute before Backup}" (default: systemctl stop adsbx-mlat && systemctl stop adsbx-socat && systemctl stop alsa-restore && systemctl stop alsa-state && systemctl stop avahi-daemon && systemctl stop bluealsa && systemctl stop bluetooth && systemctl stop collectd && systemctl stop console-setup && systemctl stop dbus && systemctl stop cron && systemctl stop dhcpcd && systemctl stop dphys-swapfile && systemctl stop dump1090-fa && systemctl stop fake-hwclock && systemctl stop fr24feed && systemctl stop generate-pirehose-cert && systemctl stop gldriver-test && systemctl stop graphs1090 && systemctl stop hciuart && systemctl stop ifupdown-pre && systemctl stop keyboard-setup && systemctl stop kmod-static-nodes && systemctl stop lighttpd && systemctl stop mlat-client && systemctl stop mm2 && systemctl stop networking && systemctl stop nmbd && systemctl stop opensky-feeder && systemctl stop pfclient && systemctl stop piaware && systemctl stop polkit && systemctl stop raspi-config && systemctl stop rbfeeder && systemctl stop rc-local && systemctl stop rng-tools && systemctl stop rpi-eeprom-update && systemctl stop rsyslog && systemctl stop skybupmx && systemctl stop ssh && systemctl stop triggerhappy && systemctl stop udisks2)
-t {backupType} (dd|rsync|tar) (default: dd)
-T "{List of partitions to save}" (Partition numbers, e.g. "1 2 3"). Only valid with parameter -P (default: *)
 -Restore options-
-0 SD card will not be formatted
-1 Formatting errors on SD card will be ignored
-C Formating of the restorepartitions will check for badblocks (Standard: 0)
-d {restoreDevice} (default: no) (Example: /dev/sda)
-R {rootPartition} (default: restoreDevice) (Example: /dev/sdb1)
--resizeRootFS (Default: yes)

1 Like

Cloned 8 Gb microSD card on RPi model 2

Conclusion: must reboot Pi after cloning.

(1) Took about 55 minutes
(2) Filled up memory. When I noted, rebooted to clear memory.
(3) CPU temp rose from 50°C to 60°C and stayed during cloning

Time of start of cloning = 19:45
Time of end of cloning = 20:35
Time of reboot = 01:00 hrs

 

 

I don’t think it did fill your memory, just because memory is ‘buffered’, doesn’t mean it’s in use. Have a quick read of this to see what I mean.

Here’s mine from my 3B last night and I have no intention on rebooting :slight_smile:

This one took an hour and a half but was running simultaensously with my Pi4 which took 52 minutes so again, the NAS could have been a slight bottleneck. I’ve just got the Pi4 running overnight tonight so will see how long it takes.

I’ve now got them running on alternate nights just so I can monitor them until I’m happy everything is good and then I’ll reduce them to once a week.

What do i do with that Spam?
Did you register yourself just for that?

This is an awesome idea but I’m struggling to get to work. Seems to work but then throws up an error after a while…

--- RBK0085I: Backup of type ddz started. Please be patient.
20200922-132944 DBG 1431:                      --> executeCmd Command: dd if=/dev/mmcblk0 bs=1M  | 
gzip  > "/home/@USER@/@HOSTNAME@-ddz-backup-20200922-132937.img.gz"
20200922-132944 DBG 1432:                      --- Redirect: 2 - Skips:
14804+0 records in
14804+0 records out
15523119104 bytes (16 GB, 14 GiB) copied, 3629.71 s, 4.3 MB/s

gzip: stdout: Input/output error
20200922-143014 DBG 1451:                      <-- executeCmd 1
20200922-143014 DBG 3949:                  <-- ddBackup 1
20200922-143014 DBG 2862:                  --> colorAnnotation 1
20200922-143014 DBG 2886:                  <-- colorAnnotation
??? RBK0021E: Backupprogram for type ddz failed with RC 1.
20200922-143014 DBG 1406:                  --> exitError 109
20200922-143014 DBG 1413:                  <-- exitError 109
20200922-143014 DBG 3308:                  --> cleanup
20200922-143014 DBG 2862:                      --> colorAnnotation 1
20200922-143014 DBG 2886:                      <-- colorAnnotation
--- RBK0033I: Please wait until cleanup has finished.

What have I missed?

@NeoDuder The gzip “Input/output error” seems to be the key. Could be many things. How much free space is on your sdcard? Do you have other processes beyond ADS-B stuff that might be interfering?

It’s been running weekly on my two RPIs for me with no problems. Well it does cause Airspy to drop a few messages for a few seconds each time in runs, but no big deal…

@davidinjp Just ADSB Stuff running, 16GB card with 12.7GB free.

For me cloning this over the network taking an hour doesn’t make really sense. Such a card can be copied to an image within 10 Minutes by using the SDCard slot on a computer/laptop.

@foxhunter That would involve me having to gain access to my Pi and shut it down every week. Kinda defeats the point.

Tried it again without the Zip option and got this…

dd: closing output file '/home/pi/backup/piaware/piaware-dd-backup-20200923-081205/piaware-dd- 
backup-20200923-081205.img': Input/output error
??? RBK0021E: Backupprogram for type dd failed with RC 1.

That’s my logic here as well. I have seven Pis which I backup to my NAS, each one gets done once a week and I keep six weeks worth of backups. I check them every few weeks just to make sure that they’re still backing up and that the .msg file created shows a success. I don’t want to have to do this manually and the installs on these Pis would take too long to rebuild if I had to do them by scratch,

It makes me a lot more comfortable knowing I have backups that I can restore if I need.

If it’s ADS-B only, why is a weekly full backup required? Do you have so many changes?
Image it once and then run the incremental backup via network

I would understand it for other devices with a different purpose. But a Flightfeeder?

My devices are running without backup. I only copy once a day the VRS database.
If one fails, well, then i reinstall it with all feeders in approx 90 Minutes

I am not complaining, just asking for my understanding. You can of course do however you like to do it.

@foxhunter I do frequent manual updates of my databases. I run multiple ADSB feeders as well as TAR1090, Timelapse1090, Graphs1090. My Dump1090 map has lots of customisations which I am constantly adding to and adjusting. Just makes sense to have a full backup every week.

Sidenote: I got this working. Rebooted my pi and now it works no problem.

1 Like

Got it, thanks

I am using 64GB cards and the backup is running endless, that’s why i decided to not do it over the network using imaging option

64 Gb microSD card is too big for ADS-B, unless you are using your Pi for something else also.

I use my 3 Pis for adsb only, and all have 8 Gb microSD card. These are running without any issue due to small card size since last 5 years,

There is merit to using a large SD card however - writes get spread across the medium creating a longer MTBF. If anyone is using a Pi, they can run the entire thing from a USB stick or attached HDD/SSD and not worry about SD card degradation at all as they aren’t required to boot with the proper firmware.

There is never a “too big” :slight_smile:

As in our shops there are almost no cards 16GB or smaller available and the 64GB versions are always the faster ones, the decision for it was easy.
And several tests showed that larger cards can have a longer reliable runtime

But that now exceed the original discussion.