Gentoo Archives: gentoo-dev

From: "Michał Górny" <mgorny@g.o>
To: gentoo-dev@l.g.o
Cc: 1i5t5.duncan@×××.net
Subject: Re: [gentoo-dev] Re: RFC: Hosting daily gx86 squashfs images and deltas
Date: Fri, 17 Jan 2014 23:36:42
Message-Id: 20140118003625.66e25e13@pomiot.lan
In Reply to: [gentoo-dev] Re: RFC: Hosting daily gx86 squashfs images and deltas by Duncan <1i5t5.duncan@cox.net>
1 Dnia 2014-01-17, o godz. 23:31:32
2 Duncan <1i5t5.duncan@×××.net> napisał(a):
3
4 > Michał Górny posted on Fri, 17 Jan 2014 20:30:00 +0100 as excerpted:
5 >
6 > > Dnia 2014-01-17, o godz. 19:19:14 Duncan <1i5t5.duncan@×××.net>
7 > > napisał(a):
8 > >
9 > >> Michał Górny posted on Fri, 17 Jan 2014 17:27:30 +0100 as excerpted:
10 > >>
11 > >> > 96M portage-20140108.sqfs
12 >
13 > >> > For deltas [...]
14 > >> >
15 > >> > 6,3M portage-20140109.sqfs-portage-20140110.sqfs.vcdiff.djw
16 >
17 > >> > applying it takes ~2.5 second on my 2 GHz Athlon64.
18 > >>
19 > >> diffs are ~1/16 the full squashfs size[.] So people updating once a
20 > >> week [or] 10 days would see a bandwidth savings, provided the sync
21 > >> script was intelligent enough to apply updates serially.
22 > >>
23 > >> The breakover point would be roughly an update every two weeks, or
24 > >> twice a month
25 > >
26 > > However, it may be actually beneficial to provide other durations, like
27 > > weekly deltas. In my tests, the daily updates for this week summed up to
28 > > almost 50M while the weekly was barely 20M.
29 >
30 > That's useful additional data. Thanks.
31 >
32 > And yes, a weekly delta would be quite useful, taking the breakover point
33 > out to about a month or so. Practically speaking, I'd guess most
34 > gentooers update once a month or more, so that should cover the vast
35 > majority. Beyond a month, just downloading a new full squashfs makes as
36 > much sense anyway, and as the cutover would be automated, users on the
37 > borderline wouldn't have to worry about whether they should just do the
38 > normal sync or download an entirely new tarball, as they now need to do,
39 > if they even bother at all. For those users, it'd be an even BIGGER win.
40
41 Well, I've got even a better idea and I'll try to provide direct deltas
42 from N days back to the newest snapshot. Since the tree changes a lot,
43 those deltas will be smaller to download ;).
44
45 Now, does anyone have an old portage-XXXXYYZZ.tar.{bz2,xz} snapshot? I
46 need the official one from our mirrors, preferably 3-4 months old.
47
48 --
49 Best regards,
50 Michał Górny

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies

Subject Author
Re: [gentoo-dev] Re: RFC: Hosting daily gx86 squashfs images and deltas "Thomas D." <whissi@××××××.de>