1 |
Am Sun, 17 Sep 2017 01:20:45 -0500 |
2 |
schrieb Dan Douglas <ormaaj@×××××.com>: |
3 |
|
4 |
> On 09/16/2017 07:06 AM, Kai Krakow wrote: |
5 |
> > Am Fri, 15 Sep 2017 14:28:49 -0400 |
6 |
> > schrieb Rich Freeman <rich0@g.o>: |
7 |
> > |
8 |
> >> On Fri, Sep 8, 2017 at 3:16 PM, Kai Krakow <hurikhan77@×××××.com> |
9 |
> >> wrote: |
10 |
> [...] |
11 |
> >> |
12 |
> >> True, but keep in mind that this applies in general in btrfs to any |
13 |
> >> kind of modification to a file. If you modify 1MB in the middle |
14 |
> >> of a 10GB file on ext4 you end up it taking up 10GB of space. If |
15 |
> >> you do the same thing in btrfs you'll probably end up with the |
16 |
> >> file taking up 10.001GB. Since btrfs doesn't overwrite files |
17 |
> >> in-place it will typically allocate a new extent for the |
18 |
> >> additional 1MB, and the original content at that position within |
19 |
> >> the file is still on disk in the original extent. It works a bit |
20 |
> >> like a log-based filesystem in this regard (which is also |
21 |
> >> effectively copy on write). |
22 |
> > |
23 |
> > Good point, this makes sense. I never thought about that. |
24 |
> > |
25 |
> > But I guess that btrfs doesn't use 10G sized extents? And I also |
26 |
> > guess, this is where autodefrag jumps in. |
27 |
> |
28 |
> According to btrfs-filesystem(8), defragmentation breaks reflinks, in |
29 |
> all but a few old kernel versions where I guess they tried to fix the |
30 |
> problem and apparently failed. |
31 |
|
32 |
It was splitting and splicing all the reflinks which is actually a tree |
33 |
walk with more and more extents coming into the equation, and ended up |
34 |
doing a lot of small IO and needing a lot of memory. I think you really |
35 |
cannot fix this when working with extents. |
36 |
|
37 |
|
38 |
> This really makes much of what btrfs |
39 |
> does altogether pointless if you ever defragment manually or have |
40 |
> autodefrag enabled. Deduplication is broken for the same reason. |
41 |
|
42 |
It's much easier to fix this for deduplication: Just write your common |
43 |
denominator of an extent to a tmp file, then walk all the reflinks and |
44 |
share them with parts of this extent. |
45 |
|
46 |
If you carefully select what to defragment, there should be no problem. |
47 |
A defrag tool could simply skip all the shared extents. A few fragments |
48 |
do not hurt performance at all, but what's important is spatial |
49 |
locality. A lot small fragments may hurt performance a lot, so one |
50 |
could give the defragger a hint when to ignore the rule and still |
51 |
defragment the extent. Also, when your deduplication window is 1M you |
52 |
could probably safely defrag all extents smaller than 1M. |
53 |
|
54 |
|
55 |
-- |
56 |
Regards, |
57 |
Kai |
58 |
|
59 |
Replies to list-only preferred. |