Backup problem when spanning multiple removable disks
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Backup problem when spanning multiple removable disks
While testing a new removable hard-disk drive, I ran into some interesting problems while trying to create an archive which span multiple removable disks using tar. I didn't some research on the web (googling). But pretty much all references I came up with are about spanning multiple tapes or floppies. So I am posting my results here, hoping to hear some insights from tar experts.
1. My removable disk is /dev/sda1 (single portition) in the system. It's mounted under /media/disk (or not mounted at all).
2. I created the archive by writing directly to the portition or to a tar file on the portition using the following commands:
- "tar -Mcvf /dev/sda1 file-list" (with the disk portition NOT mounted).
- "tar -Mcvf /media/disk/tar_file_name file-list" (with the portition mounted)
In both cases, tar prompted for the second volume after it has filled the first disk. After I ejected the first disk and inserted the second (mounted at the same point), tar completed the archive without error. However, when I tried to extract contents of the archive, it always gives an errors on the second volume: "tar: This volume is out of sequence".
I checked the second portition (tar -t ..), and the archive looks fine. I also tested it on FAT32, ext2, ext3. All failed the same way. However, it worked on FAT16!
3. I was able to get it working by adding a "-L N" option to the tar command (and writing to a tar file), where N is the number of blocks smaller than the size of the portition. But this is not ideal since it's an hassle to decide what value to use on the fly (especially if disks of different sizes are used.
Does anybody know what could have caused such difference between the file systems, and how I could get it working without having to use the -L option?
You might want to try dar instead. It does multiple disk spanning and not only compresses, but compresses individual files within a dar archive file, and has built-in CRC checking. I've been creating multi-disk dar hard drive images for the last year. I'm not the only one, the docs on the dar site describe somebody who saved 1.4T over 200 DVDs... and he got it all back when his disk array crashed a weel later.
You can get probably get it via whatever automated installer your distro uses, if that doesn't work, go to http://dar.linux.free.fr/doc/Features.html , you'll want to read the docs in any case.
Dar doesn't care what it archives to, you create your backup .dar files at whatever size is right for the backup media you use. (limited by the largest individual file size your system can handle, mine was 2G... so I save 3 x 1.5G dar files per DVD)
This is probably not the answer you wanted, but it might do a better job than what you've been trying.
Quote:
Originally Posted by wayne.liu
While testing a new removable hard-disk drive, I ran into some interesting problems while trying to create an archive which span multiple removable disks using tar. I didn't some research on the web (googling). But pretty much all references I came up with are about spanning multiple tapes or floppies. So I am posting my results here, hoping to hear some insights from tar experts.
1. My removable disk is /dev/sda1 (single portition) in the system. It's mounted under /media/disk (or not mounted at all).
2. I created the archive by writing directly to the portition or to a tar file on the portition using the following commands:
- "tar -Mcvf /dev/sda1 file-list" (with the disk portition NOT mounted).
- "tar -Mcvf /media/disk/tar_file_name file-list" (with the portition mounted)
In both cases, tar prompted for the second volume after it has filled the first disk. After I ejected the first disk and inserted the second (mounted at the same point), tar completed the archive without error. However, when I tried to extract contents of the archive, it always gives an errors on the second volume: "tar: This volume is out of sequence".
I checked the second portition (tar -t ..), and the archive looks fine. I also tested it on FAT32, ext2, ext3. All failed the same way. However, it worked on FAT16!
3. I was able to get it working by adding a "-L N" option to the tar command (and writing to a tar file), where N is the number of blocks smaller than the size of the portition. But this is not ideal since it's an hassle to decide what value to use on the fly (especially if disks of different sizes are used.
Does anybody know what could have caused such difference between the file systems, and how I could get it working without having to use the -L option?
Thanks for the suggestions. I might give it a try later. At this point, I am really looking for a way to get tar working on removable hard disks as with tape drives (i.e. without having to explicitly specify the file size).
Thanks for the suggestions. I might give it a try later. At this point, I am really looking for a way to get tar working on removable hard disks as with tape drives (i.e. without having to explicitly specify the file size).
Wayne
Are you trying to create a drive mirror (image backup) on a removable drive? Rsync is a far better backup solution for this than tar. (and dar doesn't make sense in this context... as I said, I don't quite know what you're trying to do.)
I rejected tar for myself and didn't recommend it to the readers in the how-to article I sold on Linux drive backups because it doesn't provide built-in error checking and it doesn't handle differential backups. The time to find out that something didn't go right with a file transfer is during, not when your primary hard drive is down for the count.
The only good reasons to learn the ins and outs of tar I can think of is if you are learning Linux because you want to be a sysadmin or if you're taking CS classes where the instructor expects you to learn it... the only way I use tar is when I'm decompressing a tarball for a build-from-source. However, my approach to my box need not be yours, of course, in my case, this is the only computer I've got and I do business with it... which makes me pretty conservative with respect to what I will and won't do with the workstation.
Rsync (for remote synchronization) is a far better solution for removable drive mirrors... it can be scripted to provide full image backup the first time around and differential backups afterwards as I set it up, you can set yours up to back up only the directories that interest you. The difference between a full image backup and differential is the difference between several hours and a few minutes when I do my every-other-day drive backup. And I can state from experience that a full restore done with an rsynced mirror works... when my HD failed a few months ago, I dropped in the mirror and had my computer back up within a few minutes.
If you are interested in trying this, you might check my rsync script which has been backing up my own system to a mirror drive for the last year or so in my how-to article on backups here: http://www.techbuilder.org/showArtic...leID=171200478
I'm using Fedora Core and you probably aren't. Just modify the top level directories in the script to match whatever's in your distro except for /proc ... presumably, that shouldn't be backed up on your distro, either...
Sorry I couldn't give you what you asked for... but if what you really want is simply to back up your drive without extra hassle or concern about "are the files really there and not corrupt?", what's here is what I think you actually want.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.