Gentoo Archives: gentoo-user

From: Florian Philipp <lists@×××××××××××.net>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Fast file system for cache directory with lot's of files
Date: Mon, 13 Aug 2012 17:16:46
Message-Id: 502935D8.9080803@binarywings.net
In Reply to: Re: [gentoo-user] Fast file system for cache directory with lot's of files by Michael Mol
1 Am 13.08.2012 16:52, schrieb Michael Mol:
2 > On Mon, Aug 13, 2012 at 10:42 AM, Michael Hampicke
3 > <mgehampicke@×××××.com <mailto:mgehampicke@×××××.com>> wrote:
4 >
5 > Have you indexed your ext4 partition?
6 >
7 > # tune2fs -O dir_index /dev/your_partition
8 > # e2fsck -D /dev/your_partition
9 >
10 > Hi, the dir_index is active. I guess that's why delete operations
11 > take as long as they take (index has to be updated every time)
12 >
13 >
14 > 1) Scan for files to remove
15 > 2) disable index
16 > 3) Remove files
17 > 4) enable index
18 >
19 > ?
20 >
21 > --
22 > :wq
23
24 Other things to think about:
25
26 1. Play around with data=journal/writeback/ordered. IIRC, data=journal
27 actually used to improve performance depending on the workload as it
28 delays random IO in favor of sequential IO (when updating the journal).
29
30 2. Increase the journal size.
31
32 3. Take a look at `man 1 chattr`. Especially the 'T' attribute. Of
33 course this only helps after re-allocating everything.
34
35 4. Try parallelizing. Ext4 requires relatively few locks nowadays (since
36 2.6.39 IIRC). For example:
37 find $TOP_DIR -mindepth 1 -maxdepth 1 -print0 | \
38 xargs -0 -n 1 -r -P 4 -I '{}' find '{}' -type f
39
40 5. Use a separate device for the journal.
41
42 6. Temporarily deactivate the journal with tune2fs similar to MM's idea.
43
44 Regards,
45 Florian Philipp

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies

Subject Author
Re: [gentoo-user] Fast file system for cache directory with lot's of files Michael Hampicke <mgehampicke@×××××.com>