| 1 |
First of all, Thank you for the many reply's and interesting discussions. |
| 2 |
|
| 3 |
Let me tell you what we concluded, after some tests it was obvious that |
| 4 |
10k files per directory was far better then the 50k we use now. |
| 5 |
The longest rsync on 10K files took 15 seconds, avarage took about 8 |
| 6 |
seconds. That is with the 9TB system using jfs and the 4TB using reiserfs. |
| 7 |
We intend to use rsync in a combination with marking the folders dirty. |
| 8 |
|
| 9 |
This method should scale well enough for us, figures indicate we might |
| 10 |
have 100TB by the end of the year. |
| 11 |
|
| 12 |
I believe there is no ultimate solution for a company like ours, we are |
| 13 |
constantly trying to find better solutions, Some new website feature |
| 14 |
requires other hardware setups for optimality, Bottle necks are common. |
| 15 |
therefor what would now be the ultimate solution, might not be so in a |
| 16 |
few weeks. |
| 17 |
But we will keep looking to better solutions for storage, backup and all |
| 18 |
other areas. But we will have to do it as problems arise, resources are |
| 19 |
spread a little thin. |
| 20 |
The Just in Time concept has penetrated to systemmanagement. |
| 21 |
|
| 22 |
About spreading the storage over several smaller systems, i do not think |
| 23 |
this is the way to go in our case. We are allready managing 100+ |
| 24 |
servers, and storage needs will be ever increasing. Where is the end? |
| 25 |
20x 5TB systems by the end of the year? |
| 26 |
point is, it is not just the 20 servers you need to maintain, but all |
| 27 |
the clients as wel (about half the servers in our case adding up to a |
| 28 |
1000 nfs mounts that could crash and have proofed to crash for no |
| 29 |
apparent reason over time). |
| 30 |
|
| 31 |
I also really liked the overlay idea, i never knew it existed :). |
| 32 |
|
| 33 |
With regards, |
| 34 |
|
| 35 |
Jos Houtman |
| 36 |
-- |
| 37 |
gentoo-server@g.o mailing list |
Archives