...making Linux just a little more fun!
Amit Saha [amitsaha.in at gmail.com]
Hello all:
I have a simple requirement- I want to back up all my lab's data (~ 1TB and increasing) to a 2TB hard-disk on another machine. How best should I partition/create volumes on my 2 TB hard-disk so as to enable my backups? I have no special requirements. Just simple dumping of the contents from one to another. In the same vein, I would like to hear how you do something similar.
(I plan to use rsync for the same)
Best,
Amit
-- Journal: http://amitksaha.wordpress.com, µ-blog: http://twitter.com/amitsaha Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python
Mulyadi Santosa [mulyadi.santosa at gmail.com]
On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote:
> Hello all: > > I have a simple requirement- I want to back up all my lab's data (~ > 1TB and increasing) to a 2TB hard-disk on another machine. How best > should I partition/create volumes on my 2 TB hard-disk so as to enable > my backups? I have no special requirements.
Sounds like rsync would do the job for you, something rsync -avPH to maintain all the permission and the ownership to the target disk.
And you said "increasing"? You mean, you plan to implement to create LVM? Then go ahead...it's a good choice IMO. IMO Double it with RAID 1 for data redundancy...
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Thomas Adam [thomas at xteddy.org]
On Sat, Mar 06, 2010 at 10:07:15PM +0700, Mulyadi Santosa wrote:
> On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote: > > Hello all: > > > > I have a simple requirement- I want to back up all my lab's data (~ > > 1TB and increasing) to a 2TB hard-disk on another machine. How best > > should I partition/create volumes on my 2 TB hard-disk so as to enable > > my backups? I have no special requirements. > > Sounds like rsync would do the job for you, something rsync -avPH to > maintain all the permission and the ownership to the target disk.
rdiff-backup. The tool of choice for this sort of thing.
-- Thomas Adam
-- "It was the cruelest game I've ever played and it's played inside my head." -- "Hush The Warmth", Gorky's Zygotic Mynci.
Amit Saha [amitsaha.in at gmail.com]
On Sat, Mar 6, 2010 at 9:38 PM, Thomas Adam <thomas@xteddy.org> wrote:
> On Sat, Mar 06, 2010 at 10:07:15PM +0700, Mulyadi Santosa wrote: >> On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote: >> > Hello all: >> > >> > I have a simple requirement- I want to back up all my lab's data (~ >> > 1TB and increasing) to a 2TB hard-disk on another machine. How best >> > should I partition/create volumes on my 2 TB hard-disk so as to enable >> > my backups? I have no special requirements.
Pls. direct your answers more towards the partitioning of the 2 TB disk 
>> >> Sounds like rsync would do the job for you, something rsync -avPH to >> maintain all the permission and the ownership to the target disk. > > rdiff-backup. The tool of choice for this sort of thing.
Thanks for the help, already!
Best, Amit
-- Journal: http://amitksaha.wordpress.com, µ-blog: http://twitter.com/amitsaha Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python
John Karns [johnkarns at gmail.com]
On Sat, Mar 6, 2010 at 9:10 AM, Amit Saha <amitsaha.in@gmail.com> wrote:
> Pls. direct your answers more towards the partitioning of the 2 TB disk
You should be a little more specific in wording your question. IOW, specify whether:
- you will be using the 2 TB drive as a secondary drive, (dedicated to backups),
or,
- you are asking for recommendations for a partitioning scheme which will serve the dual purpose of using the 2TB drive as both a system drive, and a backup drive?
-- John
Amit Saha [amitsaha.in at gmail.com]
On Sun, Mar 7, 2010 at 2:27 AM, John Karns <johnkarns@gmail.com> wrote:
> On Sat, Mar 6, 2010 at 9:10 AM, Amit Saha <amitsaha.in@gmail.com> wrote: >> Pls. direct your answers more towards the partitioning of the 2 TB disk
I am going to use it as a dedicated backup drive. Thanks in advance!
Best, Amit
-- Journal: http://amitksaha.wordpress.com, µ-blog: http://twitter.com/amitsaha Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python
Karl-Heinz Herrmann [kh1 at khherrmann.de]
On Sun, 7 Mar 2010 02:53:20 +0530 Amit Saha <amitsaha.in@gmail.com> wrote:
> I am going to use it as a dedicated backup drive. Thanks in advance!
Well then use one 2TB partition. You might need cfdisk or similar instead of the regular fdisk which cant handle filesystems beyond a certain size.
If you would like to keep the option of further enlarging the backup storage beyond the 2TB and still see it as one huge partition use lvm, use the 2TB drive as the first physical member of that LVM volume and create your filesystem on that virtual disk.
Once the 2TB are used up buy a 4TB drive and add it as a second physical drive to the volume, enlarge LVM and filesystem and go on.
With that amount of data you do want protection against single disk failure. So if the primary disk is not already a raid system you should make the backup a raid 1 or 5.
If your amount of data ist typically growing by adding new files you might want to have a look into backuppc which can give you daily views of the data (if they change) while storing identical files only once aand hardlinking them.
K.-H.
René Pfeiffer [lynx at luchs.at]
On Mar 19, 2010 at 2209 +0100, Karl-Heinz Herrmann appeared and said:
> On Sun, 7 Mar 2010 02:53:20 +0530 > Amit Saha <amitsaha.in@gmail.com> wrote: > > > I am going to use it as a dedicated backup drive. Thanks in advance! > > [...] > With that amount of data you do want protection against single disk > failure. So if the primary disk is not already a raid system you should > make the backup a raid 1 or 5.
Don't use RAID5 with disks containing a large block number. The probability for a double-disk failure gets higher with the amount of blocks.
http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5_disk_failure_rate
Consider RAID6 or using RAID1 block devices as building blocks for logical volumes.
Best,
René,
jumping to mention this, no harm intended.
Jim Jackson [jj at franjam.org.uk]
On Fri, 19 Mar 2010, Ren? Pfeiffer wrote:
> On Mar 19, 2010 at 2209 +0100, Karl-Heinz Herrmann appeared and said: >> On Sun, 7 Mar 2010 02:53:20 +0530 >> Amit Saha <amitsaha.in@gmail.com> wrote: >> >>> I am going to use it as a dedicated backup drive. Thanks in advance! >> >> [...] >> With that amount of data you do want protection against single disk >> failure. So if the primary disk is not already a raid system you should >> make the backup a raid 1 or 5. > > Don't use RAID5 with disks containing a large block number. The > probability for a double-disk failure gets higher with the amount of > blocks. > > http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html
I think you will find, if you read all the comments on this article, that many people, who appear reasonably knowledgable, think this is bullshit^H^H^H^H^H^H^H^Hfanciful.
Given the article was written in 2008 and disks have indeed got a lot bigger, and are used in large raid5 arrays, I think we'd be hearing a lot of squealing in the industry right now!
René Pfeiffer [lynx at luchs.at]
On Mar 20, 2010 at 2207 +0000, Jim Jackson appeared and said:
> On Fri, 19 Mar 2010, Ren? Pfeiffer wrote: > > [...] > >Don't use RAID5 with disks containing a large block number. The > >probability for a double-disk failure gets higher with the amount of > >blocks. > > > >http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html > > I think you will find, if you read all the comments on this article, > that many people, who appear reasonably knowledgable, think this is > bullshit^H^H^H^H^H^H^H^Hfanciful. > > Given the article was written in 2008 and disks have indeed got a lot > bigger, and are used in large raid5 arrays, I think we'd be hearing a > lot of squealing in the industry right now!
The source might not have been the best one, but I cooperate with a data recovery company whose "war stories" tell a different tale. Of course the URE event is rare, but it does happen.
Best, René.
Anderson Silva [afsilva at gmail.com]
On Sat, Mar 6, 2010 at 9:59 AM, Amit Saha <amitsaha.in@gmail.com> wrote:
> Hello all: > > I have a simple requirement- I want to back up all my lab's data (~ > 1TB and increasing) to a 2TB hard-disk on another machine. How best > should I partition/create volumes on my 2 TB hard-disk so as to enable > my backups? I have no special requirements. Just simple dumping of the > contents from one to another. In the same vein, I would like to hear > how you do something similar. > > (I plan to use rsync for the same)
If you want to try rsync with a GUI, try backintime, it's pretty cool.
AS
-- http://www.the-silvas.com