Gentoo Archives: gentoo-portage-dev

From: W-Mark Kubacki <wmark@×××××××××.de>
To: gentoo-portage-dev@l.g.o
Subject: Re: [gentoo-portage-dev] [PATCH] portage: HTTP if-modified-since and compression
Date: Fri, 03 Aug 2012 12:08:37
Message-Id: 20120803093304.GA26033@3-0.hurrikane.de
In Reply to: Re: [gentoo-portage-dev] [PATCH] portage: HTTP if-modified-since and compression by Brian Dolbec
1 On Thu, Aug 02, 2012 at 06:29:31PM -0700, Brian Dolbec wrote:
2 > On Thu, 2012-08-02 at 21:57 +0200, Mark Kubacki wrote:
3 > >
4 > > Regarding functionality – there is still some room for more
5 > > optimizations and more features. For example, if the local copy is no
6 > > older than x seconds then there's no need to contact any remote
7 > > server. Expect patches.
8 > >
9 >
10 > Mark, I did similar for the layman-2.0 code which has been running with
11 > the header info for quite a while now. After it had been running for a
12 > good amount of time I put in a request to infra for some usage stats.
13 >
14 > The If-Modified-Since header does make a big difference for layman.
15 > Now I just really need to make a good blog post with a few graphs of the
16 > data.
17 > You can view the results on this bug if your interested:
18 > https://bugs.gentoo.org/show_bug.cgi?id=398465
19 >
20
21 Brian, thanks for the stats and the pointer to layman. I guess we both
22 see the opportunity to share some experiences and code. Layman can
23 benefit from adding compression and I need to integrate your notices
24 about Py2/Py3 compatibility.
25
26 If it is okay with Zac I will refactor and improve the URL-fetching some
27 more. Following redirects, a proper auth-handler and 'identificator'
28 comes to mind. You could copy the final handlers, then.
29
30 Portage's 'emerge' currently contacts remote hosts whenever it is run
31 and this adds a noticeable delay. In the best case even the 304 (not
32 modified) responses are avoid wherever possible. So in the end success
33 of Portage's caching will not be measurable by a 200-to-304 ratio.
34
35 --
36 Mark
37
38 [1] http://trac.nginx.org/nginx/ticket/93 – discussion about unintuitive
39 but valid handling of the If-Modified-Since header

Replies