Gentoo Archives: gentoo-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-dev@l.g.o
Cc: heroxbd@g.o
Subject: Re: [gentoo-dev] Re: RFC: Hosting daily gx86 squashfs images and deltas
Date: Sat, 18 Jan 2014 08:15:16
Message-Id: 20140118091456.19afcfd6@pomiot.lan
In Reply to: Re: [gentoo-dev] Re: RFC: Hosting daily gx86 squashfs images and deltas by heroxbd@gentoo.org
1 Dnia 2014-01-18, o godz. 10:08:13
2 heroxbd@g.o napisał(a):
3
4 > Michał Górny <mgorny@g.o> writes:
5 >
6 > > However, it may be actually beneficial to provide other durations, like
7 > > weekly deltas. In my tests, the daily updates for this week summed up
8 > > to almost 50M while the weekly was barely 20M.
9 >
10 > Is there a way to merge the deltas without the original squashfs?
11
12 You mean get two deltas and merge them to a single one? I have no idea,
13 honestly :). It may be possible but xdelta3 is hard to use.
14
15 > how fast is the delta generation?
16
17 With max compression on 2 GHz amd64, about 15s for a daily delta.
18 A weekly takes 25s.
19
20 > Or we provide a set of 16,8,4,2,1 day deltas, then
21 >
22 > 16d: 1 piece needed
23 > 8d: 2 needed
24 > 4d: 4 needed
25 > 2d: 8 needed
26 > 1d: 16 needed
27 >
28 > The total of 31 pieces will cover a month (31 days) with at most 5
29 > deltas to be downloaded.
30 >
31 > e.g. If the system is 19 days old, then we download a 1d, 2d, and 16d.
32
33 Unnecessarily complex and not worth the effort.
34
35 Please remember that we're generating deltas against compressed image
36 with metadata cache included. Those are far from optimal, and with your
37 solution you'll quickly run over the original squashfs size.
38
39 I'm currently working on another solution that should be a bit harder
40 for the server yet much better for users. It assumes we're keeping
41 deltas from N days back to today, i.e.: n-1..n, n-2..n, n-3..n.
42
43 Any of those deltas is much smaller than two dailies (I'll provide
44 exact numbers when my playground finishes, that is in few hours). You
45 just download the one matching your current image and apply it,
46 and TADAAM, you've got today's snapshot :).
47
48 --
49 Best regards,
50 Michał Górny

Attachments

File name MIME type
signature.asc application/pgp-signature