Dead and/or Dying SD Card...

FlightAware ADS-B Flight Tracking
1

So that last couple of times that my main production Raspberry Pi2 has been restarted it, it has failed to boot until ‘maintenance’ was performed on the SD card.

So using the commands fsck & e2fsck I have been able to get it to restart. But I’ve pretty much resigned myself to having to rebuild the system from scratch. And this is where I’ve questions.

  1. Should I start with a new SD card and build on it as usual, or…

  2. Use the new SD card to boot, but move the operating system to a USB drive

Would the USB drive be more reliable in the long term?

Any incite from others would be greatly appreciated, Tom

2

USB or MicroSD I don’t believe you will find one is better then the other in the long run. You best bet is to buy a Quality SD card, not these cheap ones. Also buy a larger one then you need so that as parts of your SD card go bad there is more room for your data to be moved to good parts of the SD card, this will extend the life as well. Here is a good read makeuseof.com/tag/extend-lif … s-sd-card/

3

Use a tool to take an image of the SD card. Then you can just copy it to a new card very quickly. If you do it once you have everything set up how you want it, then you have a ready to go installation. Just set up some way of backing up your dynamic data, maybe keep it on a USB disk instead of the sd card.

4

tjowen,
Out of curiosity, what brand, model, class, and size this failing card?

I have had no problems with the current models of Samsung cards. I prefer the simplicity and low power of the SD card.

Cheers!
LitterBug

You can use DD to do a quick unbuffered benchmark of your card on your Pi to compare performance of cards. Here is a quickie 4K Write/read and 512K write/read example:


dd if=/dev/zero of=~/test.tmp bs=4k count=2000 oflag=direct | tail -1
dd if=~/test.tmp of=/dev/null bs=4K count=2000 iflag=direct | tail -1
rm ~/test.tmp
dd if=/dev/zero of=~/test.tmp bs=512k count=128 oflag=direct | tail -1
dd if=~/test.tmp of=/dev/null bs=512k count=128 iflag=direct | tail -1
rm ~/test.tmp

Sample output from my Pi:
pi@raspberrypi ~ $ dd if=/dev/zero of=~/test.tmp bs=4k count=2000 oflag=direct | tail -1
2000+0 records in
2000+0 records out
8192000 bytes (8.2 MB) copied, 3.06458 s, 2.7 MB/s
pi@raspberrypi ~ $ dd if=~/test.tmp of=/dev/null bs=4K count=2000 iflag=direct | tail -1
2000+0 records in
2000+0 records out
8192000 bytes (8.2 MB) copied, 1.00369 s, 8.2 MB/s
pi@raspberrypi ~ $ rm ~/test.tmp
pi@raspberrypi ~ $ dd if=/dev/zero of=~/test.tmp bs=512k count=128 oflag=direct | tail -1
128+0 records in
128+0 records out
67108864 bytes (67 MB) copied, 3.80506 s, 17.6 MB/s
pi@raspberrypi ~ $ dd if=~/test.tmp of=/dev/null bs=512k count=128 iflag=direct | tail -1
128+0 records in
128+0 records out
67108864 bytes (67 MB) copied, 3.53463 s, 19.0 MB/s

Summary:
My boot time is about 15 seconds to get to the command prompt
4K write 2.7 MB/s 4K Read 8.2 MB/s
512k write 17.6 MB/s 512k read 19.0 MB/s

5

This is exactly what I do. Along with an image of the SD card, I keep a USB stick in the Pi. A cron job backs up all my customized files and data at 2:00 am local.

6

The SD card at issue is a Kingston 16GB Class 10 card that I purchased new locally in February. I am using the same card in both 8G & 16G versions in other Pi media systems without issue.

Just got back from Staples with a new Lexar 64GB Class 10 card that I’m imaging and will give it a try.

…Tom

7

I’ve been doing this manually and will have to set it up automatically going forward.
Thanks, Tom

8

Here’s my script. It’s not at all sophisticated - just a bunch of tar commands and a couple of others. Maybe someone here can suggest some improvements. It assumes your USB stick is mounted on /mnt/usb. It runs in under 10 seconds, even quicker if you don’t compress the tarballs.


#!/bin/sh

/bin/tar cvzf /mnt/usb/home_pi.tar.gz /home/pi/dump-tools && \
/bin/tar cvzf /mnt/usb/var_www_collectd.tar.gz /var/www/collectd && \
/bin/tar cvzf /mnt/usb/var_lib_collectd.tar.gz /var/lib/collectd && \
/bin/tar cvzf /mnt/usb/usr_share_dump1090-mutability_html.tar.gz /usr/share/dump1090-mutability/html && \
/usr/bin/crontab -l > /mnt/usb/root.crontab && \
/bin/cp /etc/collectd/collectd.conf /mnt/usb/
9

I’ve been using several Rpi’s for about 2 1/2 years. I’ve found SD card corruption mainly to be caused by loss of power and using cheap SD cards. Add a UPS, if you don’t already have one. I’m using SanDisk SD cards direct from Amazon. There are big scams going around eBay and third party sellers on Amazon selling counterfeit cards. They are virtually impossible to tell apart. They have the correct labeling and packaging, but they quickly fail.

10

Well, this time, fsck nor e2fsck were able to recover the partition to being able to boot. So I replaced the Kingston 16G Class 10 card with a new from Staples Lexar 64G Class 10 card this morning.

After rebuilding the apps, which didn’t really take too long and I was able to restore from my backup USB drive my reconfigured files.

We’ll see how long this one lasts.

And jepolch, I am going to use your suggestion of a daily backup script for the my own data, thank you.
…Tom

11

jepolch’s method of backing up the essential bits is definitely the most efficient and space-saving backup method but I have a full image backup script running on a couple of my 24/7 ethernet connected RPis if anyone is interested. Adapted from one found on the RPi forums.

It basically creates a full image and writes it to a network share. If it is successful it deletes the backup image on the system that is 5 days old.

Very useful to have a full backup image from the same day available so you can write it to a new SD card and be back up and running quickly. The bigger the SD card the longer it will take, and you obviously need enough space to store a few images.

I wouldn’t ever use this on an RPi using WiFi, it just isn’t reliable enough.


#!/bin/bash
# Backup OS to the network share

# Create a filename with datestamp for our current backup (without .img suffix)
ipaddr="`ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'`"
currdate="`date +%Y-%m-%d.%s`"
olddate="`date --date="5 days ago" +%Y-%m-%d`"
stamp="$ipaddr-$currdate"
oldstamp="$ipaddr-$olddate"

ofile="/mnt/server/rpibackup/rpi_$stamp"

# Create final filename, with suffix
ofilefinal=$ofile.img

# Begin the backup process, should take about 1 hour from 8Gb SD card to HDD

dd if="/dev/mmcblk0" of=$ofile bs=1M

# Collect result of backup procedure
result=$?

# If command has completed successfully, delete previous backups and exit
if  $result = 0 ]; then
   rm -f /mnt/server/rpibackup/*$oldstamp.*
   mv $ofile $ofilefinal
   exit 0

# Else remove attempted backup file
else
   rm -f $ofile
   exit 1
fi
12

Thanks for sharing this. I implemented your script on one of my Pi2’s earlier and ran it manually as a test before leaving it for cron to play with every morning.

Not sure which Pi model you’re using (or if that’s even your note?), but the script only took it around 12-minutes to complete on a Pi2 with an 8GB card. Ethernet interface was pretty much pegged the entire time, but it didn’t disrupt Piaware, FR24feeder, etc… If only these things had Gigabit, it would be even quicker. :slight_smile:

Matt

–

13

I too want to say thanks for the script. I implemented a similar one and verified I could replicate an image to another card. Flawless. My times are in line w/ Matt’s; a 16GB card in a Pi2 is taking me just under 24 minutes.

14

I have a similar script running on my Mac but I have it compress the data so it creates a smaller image.

If you modify these lines from:
ofilefinal=$ofile.img

dd if=“/dev/mmcblk0” of=$ofile bs=1M

to:
ofilefinal=$ofile.img.gz

dd if=“/dev/mmcblk0” bs=1M | gzip > $ofile

You may get similar results. I was able to compress a 32GB SD card down to 1.8GB during the backup.

Restoring is basically the same:

gzip -dc changetofilename.gz | sudo dd of=/dev/mmcblk0 bs=1m

Disclaimer, I didn’t run this on mine yet as I don’t have any network drives mounted on my PI.

15

It’s not a problem, like I said, not my script at first, just tweaked it to write to my network share rather than a USB HDD as it was originally created for.

It wasn’t my note but I think it is totally dependant on SD card speed and Pi1 or Pi2.

I’m doing a 2GB SD backup on a Pi1 and it takes about 18 mins (slow SD card).

I also have a Pi2 used for something else and an 8GB SD backup only takes 11 mins by the looks of my previous backup times.

16

Hi all.

I confirmed that the gzip options works on my pi to a SMB network mount. It compressed my 32GB image to 1.8GB. I suspect it will do pretty well on compressing smaller SD cards as well.

It’s a nice script, I like the auto delete of older archives.

Thanks for sharing.

Here’s the gzip version.


#!/bin/bash
# Backup OS to the network share

# Create a filename with datestamp for our current backup (without .img suffix)
ipaddr="`ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'`"
currdate="`date +%Y-%m-%d.%s`"
olddate="`date --date="5 days ago" +%Y-%m-%d`"
stamp="$ipaddr-$currdate"
oldstamp="$ipaddr-$olddate"

ofile="/mnt/rpibackup/rpi_$stamp"

# Create final filename, with suffix
ofilefinal=$ofile.img.gz

# Begin the backup process, should take about 1 hour from 8Gb SD card to HDD

# dd if="/dev/mmcblk0" of=$ofile bs=1M
dd if="/dev/mmcblk0" bs=1M | gzip > $ofile

# Collect result of backup procedure
result=$?

# If command has completed successfully, delete previous backups and exit
if  $result = 0 ]; then
   rm -f /mnt/server/rpibackup/*$oldstamp.*
   mv $ofile $ofilefinal
   exit 0

# Else remove attempted backup file
else
   rm -f $ofile
   exit 1
fi
17

Do you have an idea how much longer it takes the script to run when using compression for your 32G card (and how long it took w/out compression)?

18

I just did a test and a 16G card in a Pi2 takes 23 min. without compression and 40 min. with compression. Not much of a time penalty

19

Not bad at all. At the very least you’re saving storage space by not having to store the “empty” space that is easily compressed. I might have to implement this for my Pis running larger SD cards.

20

I was originally writing an uncompressed image of an 8GB card from a Pi2 over to a share drive on a Win7 Pro box in around 12 minutes, with it now taking around 40 minutes to write the same image compressed. File size went from 7.8GB down to 2.4GB. Card in use in that Pi2 is a Samsung 8GB Class 6 Micro SDHC.

The fact that I went down to 2.4GB upon compressing an 8GB card image has me a bit intrigued as to how the 32GB card mentioned above compressed down even smaller to 1.8GB? I guess the answer there would involve how much of my 8GB card is actually in use and how much of that 32GB card is actually in use.

Either way - excellent work here, guys.

Matt

–