Running Raspbian from a USB hard drive.

FlightAware ADS-B Flight Tracking
1

UPDATE:
Apparently when I mentioned USB Drive this caused a bit of confusion as to the exact type of device I am using. I should have been more clear on this and referred to “USB Drive” as “USB Hard Drive”. My set up is using one of the following drives I had replaced on my Xbox One when going to a bigger one for it. amazon.com//dp/B00CI3BLPA/. I am quite pleased with it. It has been running for over a year straight 24/7 on the Xbox One and now on the RPi.

Once again had to replace a burned out SD card on my receiver today. Kind of getting sick of swapping out SD cards every month or so, so this time I decided to move the root partition off the SD card and on to a USB drive. Supposedly this should give the set up some much needed durability as well as a performance gain. I am starting over from scratch but this method of moving the root directory over to a USB drive should work with any existing setup. If you decide to do this with an existing setup please BACKUP OR IMAGE YOUR EXISTING SD CARD BEFORE STARTING.

Boot up your running Raspberry Pi and plug in your USB drive and execute the following command.


sudo fdisk –l

The USB drive will generally show up as /dev/sda but make sure by comparing it to the list presented by the previous command.

Now we will need to partition the USB drive which we will do so by running the following command.


sudo fdisk /dev/sda

Delete the partitions by entering “d”. If more than one partition is present you will need to specify the number of the partition repeating this until they are all gone. Do not exit fdisk after doing this we are not done using it yet.

Next we will create a couple new partitions one to being an ext4 file system which will contain the root partition from here on out and another which will be used as a swap partition by the operating system.

First create the partition to be used for swap:

Enter “n” to create the partition, “p” to set it as a primary partition and then “2” to set the partition number to two. Press enter to select the default start sector then enter +2G to specify the size.

Now create the soon to be root partition:

Enter “n” to create the partition, “p” to set it as a primary partition and then “1” to set the partition number to one. Press enter to select the default start sector then select the default end sector to use up the entire disk.

Now finish setting up the swap partition by entering “t” after which we will need to enter “2” as the partition number we wish to work with the enter “82” to make this a Linux Swap partition.

Enter “p” to list the partitions to be written and confirm everything is right.
Enter “w” to write the new partition table to the drive once everything is confirmed to be in order.

Now format the new root partition by using the following command.


sudo mkfs.ext4 /dev/sda1

Initialize the new swap partition.


sudo mkswap /dev/sda2

Mount the new root partition.


sudo mount /dev/sda1 /mnt

Install rsync and copy the files from the SD card to the USB drive.


sudo apt-get install rsync
sudo rsync -avz --exclude '/mnt' / /mnt

Now open the file /mnt/etc/fstab in your favorite text editor.


sudo nano /mnt/etc/fstab

Change the the root device from /dev/mmcblk0p2 to /dev/sda1 then add the following line to the file.


/dev/sda2 none swap sw 0 0

Next stop the system from using the swap file it has in the past.


sudo rm /mnt/etc/rc?.d/*dphys-swapfile

Then delete the old swap file.


sudo rm /mnt/var/swap

Open the file /boot/cmdline.txt in your favorite text editor.


sudo nano /boot/cmdline.txt

And change /dev/mmcblk0p2 to /dev/sda1


sudo reboot

I ran through this twice both times on different hardware with things booting up great. No activity on my SD card with the USB drive flashing away happily instead.

2

I had a memory card go bad a few days ago after running fine for the last 6 months. Very frustrating, even with backups.

I’d be interested in hearing updates on how the drive works out for you.

3

raspbian-ua-netinst has a mode where it will put the root FS on a USB drive too.

4

Im using a 32 Gb USB 3.0 with my RPi 2 with no issues. Works pretty good so far.


Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G  3.5G   24G  13% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           463M  4.0K  463M   1% /dev/shm
tmpfs           463M   54M  410M  12% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           463M     0  463M   0% /sys/fs/cgroup
/dev/mmcblk0p1  1.9G   25M  1.9G   2% /boot
tmpfs            93M     0   93M   0% /run/user/1000
/dev/sda1        60M   20M   41M  34% /media/pi/USBDRIVE
5

Been running about a week now with no writes to the SD drive worth mentioning as expected. Anything having to do with disk IO including the web interface seems faster and a bit snappier as well. If I run into any issues I will be sure to report them but looks like this is the way I will be setting things up from here on out.

6

I have been using the ADS-B Receiver Project Setup Scripts and I have also lost a couple SD cards that wouldn’t reboot so I recently upgraded my receiver with a WD 314GB HDD and everything is working fine. I wasn’t using the DB feature before but now that I have a HDD I am logging flights to the DB.

My question is, what do I need to change so the Performance Graphs - Disk I/O graphs show HDD I/O activity?

http://flightaware.com/adsb/stats/user/tbirdrv#stats-19516

Thanks for the great scripts,

7

@TbirdRV
This is taking into account the disk is known as /dev/sda by the operating system.
This may be different check to make sure on your device.

In the file /etc/collectd/collectd.conf…


<Plugin "disk"> 
    Disk "mmcblk0" 
    IgnoreSelected false 
</Plugin>

Should look like…


<Plugin "disk"> 
    Disk "sda" 
    IgnoreSelected false 
</Plugin>

In the file ~/adsb-receiver/build/portal/graphs/make-collectd-graphs.sh…


  disk_io_iops_graph ${DOCUMENTROOT}/graphs/system-$2-disk_io_iops-$4.png /var/lib/collectd/rrd/$1/disk-mmcblk0 "$3" "$4" "$5"
  disk_io_octets_graph ${DOCUMENTROOT}/graphs/system-$2-disk_io_octets-$4.png /var/lib/collectd/rrd/$1/disk-mmcblk0 "$3" "$4" "$5"

Should be changed to look like…


  disk_io_iops_graph ${DOCUMENTROOT}/graphs/system-$2-disk_io_iops-$4.png /var/lib/collectd/rrd/$1/disk-sda "$3" "$4" "$5"
  disk_io_octets_graph ${DOCUMENTROOT}/graphs/system-$2-disk_io_octets-$4.png /var/lib/collectd/rrd/$1/disk-sda "$3" "$4" "$5"

That should do it.

8

That worked perfectly.

Thanks!

9

I 2nd those thanks! Hats off to Mr. Prochazka for this (and all the other) work that make these productive tasks easy. This external drive idea is saving my bacon after a lot of bad MicroSD card luck in the past…

10

Hi,

Thanks for these instructions jprochazka. I’m trying to follow them to switch to / setup a 32GB SSD drive running in a USB2 enclosure attached to my Pi 3. I am running your ADS-B Receiver v2.0.3 image (set up last week), and have a few questions / issues.

When doing the “sudo fdisk /dev/sda” command and creating a new partition, I never get asked the question about if I want it to be a primary partition, does this matter.

When setting up partition 2 to be the Linux Swap partition, it wasn’t “82”, but (I think) it was 14.

When doing “sudo rsync -avz --exclude ‘/mnt’ / /mnt“ it finishes with errors with something like:

ERROR: address failed verification – update discarded.
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c

Adding “| grep failed” seems to show files under the /sys folder. I googled around (I know next to nothing about Linux) and it shows that /sys (and /proc) are “Virtual directory for system information”, and as such shouldn’t be copied as contain “run time” information, so should those be excluded from the rsync. Think on another thread someone mentioned errors with rsync, and using another method for copying the data, but I can’t find it now.

Thanks.

11

I am running an ODROID and they have a eMMC module that goes right on the board

hardkernel.com/main/products … 8750148750

12

I got my USB Hard disk working. From the ADS-B Receiver Forums someone linked to a script here: github.com/adafruit/Adafruit-Pi … oot-Helper

The instructions say to do the following:

git clone git@github.com:adafruit/Adafruit-Pi-ExternalRoot-Helper.git
cd Adafruit-Pi-ExternalRoot-Helper
sudo ./adafruit-pi-externalroot-helper -d /dev/sda

However I have to do use the following in place of the 1st line:

git clone github.com/adafruit/Adafruit-Pi … Helper.git

All went ok, and I’m now running off the USB Drive with no activity on the SD card. The script doesn’t create the Swap partition that the original instructions in this thread does, but from what I can tell the swap file that is in use with the ADS-B Receiver build is moved to the USB Disk.

13

So, I have been running off the External SSD disk (in USB Enclosure) for a week now, but have had an issue 2 times with it. The symptoms are that I can’t connect to the dump1090 web page, and SSH’ing to the Pi doesn’t connect, however the feeding of data to Flightarare continues ok. To resolve the issue I have to disconnect power from the Pi and then reconnect it, then after it’s booted all is ok again, but I am missing “data” for the graphs from the time the issue occurred to when it’s restarted.

I assume what’s happening is that the “USB” disk is disconnecting, and from that moment on nothing can write to it. It may come back shortly after, but from googling around it seems that it may come back as a different device id (I can’t check this as the Pi is headless with no monitor / keyboard) and because the id is different it doesn’t mount as before.

I assume as dump1090 / piaware etc are running in memory they carry on “working” it’s just the logging of data stops and being able to serve web pages. I can’t look in any log files as (I assume) as they are on the external drive nothing can be written to them.

Looking around people do have issues with disks attached with USB disconnecting, and it’s related to power. It may be the issue in my case, however the SSD is rated at 200mA, so not that high. With the Flighaware stick running at 500mA that’s a total of 700mA, but I have a Pi 3 and that’s capable of outputting 1.2A across it’s usb ports. I’m powering over a TP-Link TL-200 POE kit, and that should output 2A at it’s 5V setting, so that should be enough to power the Pi and the usb devices.

The little usb enclosure I got came with a Y cable, but seeing as the SSD was only rated at 200mA I only connected one side of the Y, I have now connected the other and will wait and see.

Few questions.

Has anyone else had similar issues. Running of a powered USB hub is mentioned, but seeing as I have everything in a waterproof box on a wall I don’t think that’s an option, also it would have to be powered from the same POE kit, so would just be supplying the same 5V 2A to it.

I’m using the ADS-B Receiver Project build, would it be possible to just move the folders relating to that or just the more disk intense folders to the usb disk such as the folders where the data for the web graphs etc is written too, and leave the rest of the root partition on the SD card.

Or, can the folders where the system logs get written to be moved back to the SD card, that way I may be able to get more info on what’s happening when this occurs.

Also, my ADS-B Receiver Project install is in light mode. If I re-install in Advanced mode, again can I leave the root on the SD card, but have the more instance write folders / SQL DB etc on the usb disk.

Thanks.

14

So, despite running with the extra “power” bit of the Y cable plugged in, the disk “disappeared” for a 3rd time, this time stopping the feed to Flightaware as well as the local web service. Once the Pi came back up I noticed the time it went down, and noticed that it was while a heavy rain / thunder / lightning storm was happening, then thinking back about it I realised that the other 2 times were also when there was a thunder / lightning storm. We haven’t had a storm for over a week now and its been without issue again, so maybe the lightning was causing the issues. Not sure what can really be done about it though.

15

Had the disk i/o working but now it is empty showing nothing. Also the portal sqlite Database has stopped to collect data and i can’t see why.
Are there some log files that i could check?

Regards

16

Perfect clarification
worked for the second time