Back it up

[swestrup]

After having gone through a number of trials and tribulations, I think I finally have the hardware for the Ten Ton Tabulator sorted out. That's the name I finally gave to the 12 TB backup server I bought recently, mainly because its the heaviest computer I've ever owned, and the heaviest that the store has ever sold. The dang thing must weight 50+ Kg.

Now that it seems to provide reliable storage (two raid 6s) I just need to make USE of the damn thing. In the long run, this is likely to cause me to upgrade my internal network to gigabit speeds, otherwise it will take *forever* to backup all the house systems.

For the moment I am testing things out with an "easy" case: backing up my main computer and laptop. Only this turns out to be far from easy, as I seem to have 4 not-necessarily exclusive categories that a file might want to be in:

1) Junk or System files that I don't really need or want backed up.
2) Files that I just want backed up.
3) Files that I want to share between all OS's on a given machine
4) Files that I want to sync between different machines.

Seeing as my laptop runs 3 different OSes (Gentoo for customizability, Ubuntu 10.04 LTS for work, and Win7 for games), while my Desktop runs 6 at last count (my dev environment plus one each of the target distros for my work software), and there are a large number of development files that need to be sync'ed between my laptop and dev system, especially when I'm on a business trip, it all gets kinda... messy.

So right now I'm going through my desktop and my laptop and moving files around so that they are in folders that make the above classifications clear, so that I can setup automated backup and sync services. This involves a large amount of reconfiguring software, as things move around, and in some cases rebooting a number of times to ensure that the software in every OS I run agrees on where the files have been moved to. Oy.

Comments

[joenotcharles.livejournal.com]

With 12 TB, I recommend you just suck it up and back up the junk + system files. These things don't change often so you'll only pay a one-time penalty - they won't be included in most incrementals. The only things that change often should be in /tmp and /var, which you can exclude at the root.

Sharing and syncing files aren't really the job of your backup system. Again with 12 TB you can probably get away with backing up a separate copy of each of the shared files for each computer.

[swestrup]

Well, when I do a "Full Backup" I will, indeed back up everything including most of the junk. But when I am sync'ing stuff between my laptop and my desktop, I really do need to distinguish between application data that genuinely needs to be synced (like firefox bookmarks) and what can be ignored (like firefox caches).

And seeing that I need to sync my desktop and laptop, and Linda likely will want to do something similar, and seeing how that is usually easier when using a server for it, the backup server seems the opportune choice.

Oh, and 12 TB is NOT MUCH STORAGE. It will not let me keep more than one copy of every unique file in the house. We really DO have LOTS of stuff stored.