Re: [gentoo-server] Re: [OT] Mirroring/backing-up a large

[kashani]

MIkey wrote:
> The second step is to ditch storing everything on a single 9TB system that
> cannot be backed up efficiently.  Distribute the storage of the images on
> clusters or whatever.  For example peel of 1TB of images onto a single
> server, then update the database (or apache/squid mapping) to point to the
> new location.  9 1TB boxes would be far less prone to catastrophic failure
> and much easier to replicate/mirror/backup than a single 9TB box.  This is
> what I call the "google approach" ;)  Use cheap commodity hardware and
> smart implementation to distribute/scale the load.
>
> Of course the ultimate solution would some sort of cluster or san
> approach...
>

I'm not sold on the Google approach.

	Assuming someone was to build nine data servers we're talking roughly 
$3k per server (dual CPU, 4GB ram, raid 5 sata) or $30k with shipping 
and tax. On top of that I now have to manage nine boxes and manage my 
data in nine different places. These 9 servers are going to pull 18A of 
power and uses 18U of rack space. Whereas $35k gets me an NFS/iSCSI/cifs 
head (of admittedly third tier storage) and two 16 x 500GB shelves or 
12TB usable if I split each shelf into two RAID 6 partitions. This setup 
pulls 14A, uses 8U, has volume management, snapshots, can expand easily, 
and eventually cluster the heads if I'm willing to buy the license later.
	The 9 x 1TB setup might be worth the pain if you had the application 
written to deal with that and needed more of your data in RAM. For a 
community photo site I'm not sure you do. Additionally I don't think it 
helps solve the original problem of backing data up somewhere, but maybe 
I'm missing something.

	In any case I'm not saying you need to spend $30k to fix the problem, 
but if you plan to drop some money on the problem really sit down a 
figure initial cost, cost to expand, rack space, power, cooling, 
maintenance costs, administration costs, etc and relate it all back to a 
$/GB so you can compare apples to apples.


In order to get better backups you might consider hashing your data a 
bit more on the filesystem.

What you've got now
/data/00000-50000/file|thumb|etc

what might work better
/data/1e/01ac/cdd98a910ca1d4e37b39a9197e/file|thumb|etc

And then you can run through each tree and only sync the subdirs you 
need. I'm not certain this idea is the right way to go long term, but 
might be easy to implement now. I would not use more than three layers 
of directories.

kashani
-- 
gentoo-server@g.o mailing list