Installing multiple external USB hard drives
by Matt Giwer, © 2007 [May], 2010 [June]

Storage increases to fill the space available. It is a fundamental rule of nature. Don't try to fight it.


So much for hope, Aug 2011
Two more big changes here.

/sbin/mkfs.ext4 -j -m 0 -L FRED /dev/sdX1

After a very long time I have these changes. Changing from mke2fs to mkfs.ext4 guarantees an ext4 file system instead of the system default. If you want the system default use mke2fs.

The really big one is the -m 0 which increases the file size by 5%. Not that much with 320G drives but with the present 3TB drives this means 93GB more storage or 30 gigs per terabyte. It is 5% of the formatted space -- here ext4 -- not the unformatted space.

Notice also the -c is gone for good. for a 2TB drives the checking takes some 18 hours AND until I figure how to use it, ignores the bad sectors it finds and includes them in the formatting.

I had a drive reporting bad sectors. I reformatted it with -c and still bad sectors are reported. Makes no sense at all.


Hopefully the Last Version
Seems while I was developing the methods below, Redhat was changing things to automatically do what I was trying to accomplish. My fixes were screwing up the improvements. Hopefully this is the last word on how to do it.

Use the -L option on mke2fs or tune2fs to give the drive a unique label.

/sbin/mke2fs -j -L FRED /dev/sdX1

It can't be simpler. Then unplug and plug in again or power cycle the drive and you will find it mounted on /media. For example if the label is FRED you will find

/media/FRED/

If you have more than one partition on FRED they will be named

FRED_ 
FRED__
and so forth. Note the additional underscore. These mount points are the same every boot so you can even

ln -s /media/FRED/home home

Add to /etc/fstab something like these.

# /etc/fstab
# Created by anaconda on Sun Feb 14 16:28:17 2010
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_dawn-lv_root /                       ext4    defaults        1 1
UUID=8c083301-b241-450d-9388-08cc4427f81d /boot                   ext4    defaults        1 2
/dev/mapper/vg_dawn-lv_swap swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=1000	/media/1000	ext3	defaults	0 0
LABEL=2150	/media/2150	ext4	defaults	0 0
LABEL=3200	/media/3200	ext4	defaults	0 0
LABEL=4320	/media/4320	ext3	defaults	0 0
LABEL=6100	/media/6100	ext3	defaults	0 0
LABEL=7500	/media/7500	ext3	defaults	0 0
LABEL=8500	/media/8500	ext3	defaults	0 0
LABEL=9100	/media/9100	ext3	defaults	0 0
But if you still have problems...
When all else fails these are three comands to use for information about your USB drives to help solve the problem.
cat /etc/mtab
ls /dev/sd*
ls -l /dev/disk/by-*


The Shortest Version so far
I don't remember exactly when it happened or which package caused the changes but as a result of a yum upgrade fstab switched from using the UUIDs to the traditional /dev/sdX1 identifier but apparently with the same functionality.


The Shorter Version
With Fedora 9 fstab is created using the UUID of the drive in place of /dev/sdX1. They always mount is the right place with every boot.
The Short Version
Having done this now on several computers and having gained much experience the way to do it is as follows.

After booting read /etc/mtab and then edit /etc/fstab to match the entries in /etc/mtab. That is the quickest and easiest way to make the drives compatible with the boot process.

If that does not work for you and want to know more, continue. Else, "Go thy way. Thy faith has saved Thee." So sayeth the middle English Jesus.


Why would you want to do a stupid thing like this?
You are a home user. You only have four slots for drives. At least two are your main hard drive and your CD or DVD drive. So you have two empty slots. Unless you are richer than the average user you can afford maybe two 500MB internal drives.

You do not want to do this if the speed of any application depends upon disk access time.

The boring part
For the first time since PCs came out there have not gone to machines speed increasing as fast as my budget for computer hardware. The speed increase has maxed for two years while Intel and AMD developed multi-core and 64bit CPUs. There is no point to switching to those until your most used applications can make use of them. In the mean time my budget has gone into mass storage.

Obviously a very dated paragraph. As of June 2010 2 terabyte drives have fallen into the USD $100 to $120 range.

It started innocently enough with some unused drives from old machines and finding an add-on for an external enclosure that would hold a drive and plug into a USB port. These were 40 GB drives and my computer had a 160 GB drive. I had the idea that for the first time I could afford to back up all data of any serious importance without the hassle of putting it on floppies or CD/DVDs. That is a nuisance and I was constantly having to decide what was really, really important and keeping track of all the disks and update intervals. You know the drill.

At first I had to switch the two 40s back and forth but it worked. Then I got a sale notice from Tigerdirect on a 250 GB USB drive for something like $110 or so. Whatever the price it was the first affordable drive that got me to look further. I bought and in one night backed up everything in my home directory and had an update scheme in formulated for the future.

The problem was backup was only 80 GB out of 230 GB available after formatting. All that free space ate at me. I found other things to collect from the web. In a few weeks I started running out of disk space and bought another and another and found a 120GB I had forgotten about to put in the original purchase holder. If only I had started with that drive I might never have learned the fundamental rule of nature.

Soon I had hundreds of GB of data and no way to back up all of it. Things to be stored expand to fill the space available.

The price of hard drives is dropping rapidly. I assume this is to empty inventory while switching over to the denser vertical polarization technology.

The bottom line is I have a terabyte of external storage for about $US400. The original machine came with a 160 GB drive to which I have added a 320 GB drive for nearly 1.5 TB total storage. The added internal 320 GB drive was also about $US100. So for about $US1000 I have a 3 GHz machine with 1.5 TB of storage.

End of boring part. Beginning of useful part.

I have always bought by best price per gigabyte. (I see no reason to consider the size of the buffer because of the low data rate through the USB port.) They are all different different vendors. They all work with nothing but plugging them in. I use Redhat linux FC6 and KDE.

To satisfy the Microsoft "plug n play" advertising they come with VFAT file format. Yes, most Linuxes will read and write that format but it is very unreliable and can be wiped out by a simple power glitch. Put a real file system on it.

I format them to ext3. Were I to use ext2 the available space would be larger. How much larger I have no idea. However, a power glitch can force a filecheck with ext2 format. Since a filecheck takes about an hour per 100 GB I'll take the hit. With ext3 the advertised size reduces as follows.

120 GB	=> 113 GB	ext3
250 GB 	=> 230 GB	ext3
320 GB 	=> 294 GB	ext3
500 GB	=> 459 GB	ext3
1 TB  	=> 917 GB	ext3
1.5 TB 	=> 1.4 TB	ext4
2 TB 	=> 1.8 TB	ext4
3 TB 	=> 2.7 TB	ext4

For formatting I use this as root.

/sbin/mke2fs -c -j -L 320 /dev/sdX1
If you want to change things later use tune2fs.

The -c does a read-only check for bad sectors. This takes about an hour per 100 GB on a 3 GHz machine. I have read this isn't really necessary these days. I don't know. I am not in a hurry.

The -j creates an ext3 filesystem. The j is for journaling. The -L is creates a label on the disk in this case the label is 320. I have not found a use for this yet but if I ever do, there it is. Of course make all the labels different.

There are many other options for mke2fs detailed on the man page. I cannot find any that are worth getting a handle on to see if there is significantly more usable space. But if you are going to use them for more than offline storage you likely do want the defaults without special configuration. But if you do make a special configuration, write it down on paper and put that in a safe place. I guarantee you a month from now you will not remember what you did.

The first problem, formatting
Specifically finding the dev number to use. This is the /dev/sdX1 in the format command above. Using KDE when you plug it in a browser window will open and show its contents. There will be no contents at this point but the browser window will give you how it is mounted showing the sdX1 device name. (I assume something similar happens in Gnome.) Write that down, close the browser and go root.

Alternatively doing cat /etc/mtab should give you the last line as the sdX1 device name. As this is done in a window it is independent of the desktop you are using.

Just in case as root do

umount /dev/sdX1
/sbin/mke2fs -c -j -L 320 /dev/sdX1 with the right letter for the X. Get a cup of coffee, have lunch, make love to your spouse. Whatever you do this will take an hour or three. As this does not interfere with anything else you can just get on with whatever it is you do on the computer. If you are in a hurry and are confident the bad sector check is not needed these days leave out the -c and it will only take a minute or two.

The second problem, mounting
Unless you use it only for occasional backup you want these drives to mount themselves and behave just like an internal drive. The problem is when they are automatically mounted at boot by fstab they are not given the same drive number as when they are plugged into running computer. I am certain there is a way to do with without rebooting but I have no idea what it might be.

KDE to the rescue again. On the desktop there will be an icon for each drive after boot. It you gave it a label, that will be the name under the icon otherwise it will be the amount of storage on the device. Now you click on the icon and that browser will appear again. The navigation window will show you the device number it was assigned at boot.

Here are the relevant entries from my /etc/fstab file

/dev/hda	/media/cdrecorder iso9660 defaults 0 0
/dev/hdb	/media/cdrom	  iso9660 defaults 0 0
/dev/sde	/media/120	ext3	defaults 0 0
/dev/sdc1	/media/250	ext3	defaults 0 0
#/dev/sdd1	/media/250	ext3	defaults 0 0
/dev/sdb	/media/320	ext3	defaults 0 0
/dev/sdd1	/media/4320	ext3	defaults 0 0
#/dev/sdc1	/media/4320	ext3	defaults 0 0
As you can see I labeled my drives by the size and also created mountpoints of the same name in /media. At first none of the /dev/sdX existed. Once finding their device number assigned at boot time I add a line to put it at the right mountpoint. (The second 320 is named 4320 simply to make use of tab completion.

To create the mountpoints you have to be root. But after creating them do

chown user:user /media/120
so you can read and write them without being root. There is probably a better owner designation. I am the only one using this computer. This works for me.

It if does not bother you you can also do

chown -R user:user /media/
which will make everything mounted under /media/ the property of the user. I have done this and find no problem with doing so.

Known problem
Notice from my FSTAB two drives are devices with an appended 1 and two are not. When I had only three, only one had an appended 1 and there was no problem. With the fourth two have the appended 1 and they occasionally switch at boot time. That is not good.

Here is what happens. If you try to format say /dev/sdc1 as /dev/sdc you get a warning that reads dire. I have ignored it and no problems. I assume adding a partition to those drives now is impossible without a complete reformat of the disk. If that is the worst penalty so be it.

At my first opportunity meaning when I can offload one of the appended 1 drives I will reformat it without the 1 and see if the switching stops.

In the mean time I have created a file at the top of each drive with its label as the file name and then a simple script

echo
echo '120'
ls -lhtd /media/120/120 
echo 
echo '250' 
ls -lhtd /media/250/250
echo 
echo '320-TV' 
ls -lhtd /media/320/320
echo 
echo '4320' 
ls -lhtd /media/4320/4320
echo 

If I do not get a reply from 250 and 4320 then they have switched. The fix is another short script.

umount /dev/sdc1 &&
umount /dev/sdd1 &&
mount -t ext3 /dev/sdc1 /media/4320 &&
mount -t ext3 /dev/sdd1 /media/250

This has to be done as root.

Other considerations

External USB drives are much slower than an internal drive mounted on the bus. Also when moving data from one external to another the speed is only half of that already slow speed. This speed is more than bearable to me as I only use them for storage. If you have an application needing a lot of disk accesses do not put that data on an external drive. Of course this doubly slow speed is also true of the internal bus but that is so fast you barely notice it unless moving a very large file. USB is so slow that when you read and write to the USB the difference is very obvious.

Also because the access is slow if you are running other applications which also have reads and writes to an external drive and there is clearly interference one of them has to be run with nice.


Above I mentioned using Fedora Core 6. I just upgraded (June 03, 2007) to FC7 and the problem with the drives switching mount points has disappeared. Apparently I am not the only person who has noticed the problem and Redhat made some change to their booting process that corrects the problem. It does take much longer in the early boot process to mount all the USB drives than it did before.

Also when upgrading to RH7 turn your external drives off before upgrading. RC7 seems to recognize them and even give the option to ignore them but on the second "ignore" mouse click it responded press to reboot system. So I turned them off and then edited /etc/fstab to comment out all references to them. Then I started the install process from the top.

After that I turned them back on and verified they were remounting from the appearance of icons on the KDE desktops. Mounting verified I rebooted and followed the above procedure as described for FC6 to create the /etc/fstab lines.

It was all a clean and simple process. Overall I have done what appeared to be obvious and it worked. You may have to live with linux for a few years to see what I did as obvious.

Avoiding an awshit
When installing a new linux instead of upgrading shut off or remove every drive you do not want to become part of Logical Volume 00. If you let it become part of the logical volume it will be reformatted and all data will be lost.


Update October 30, 2007
Through a series of Lemoney Snickety events I have had the opportunity to repeat this process on a new computer. The one I used for the above had six USB ports so all were direct connections. This time I had to use a four port USB hub of the trivial kind. It was under $10 from Walgreen or something like that.

So when I first booted I had to manually put them at the desired mount points in /media. On next boot they were jumbled and using MTAB and the automount features as described above I got them all mounted correctly.

Then I looked at the associations in /etc/mtab and modified /etc/fstab to match mtab. That means associating the correct /dev/sdXy with the desired /media directories. On the next boot all five drives mounted as desired.

Your mileage WILL vary but this has worked for me.


A strange thing happened after a short on a different house circuit. One of the external drives failed to register and I rebooted. It would not reboot. It hung on the Compaq logo not even going to BIOS. After trying everything obvious I sat back to think. After about 20 minutes it started to boot but would hang part way through.

Finally I get the idea to reboot everything including the USB drives. That worked. Everything was back to normal.

Page reads: 14690