On Thu, May 03, 2007 at 04:34:19PM -0700, Robin H. Johnson wrote:
> Would all mirrors that carry the experimental tree please see my
> follow-up email about BitTorrent HTTP-seeding?

Hi folks,

So now stuff on BitTorrent for the 2007.0 release.

There's two portions to this email, firstly stuff distributed by
BitTorrent only, and then everything else.

We have 94 torrents in total, 2 of which are only media that is only
being distributed by BitTorrent.

We needs seeds for all of it, but the bittorrent-only stuff more than
the rest. I have one sponsor offering 50Mbit * 2 weeks for seeding
everything so far.

In terms of basic structure, there is a .torrent for every .tar.bz2 or
.iso in releases or experimental, containing the core file, plus it's
DIGESTS, signature, and CONTENTS files.

BitTorrent-only:
3.7G	livedvd-amd64-installer-2007.0/
3.8G	livedvd-i686-installer-2007.0/

If you're interested in direct BitTorrent seeding (esp. for the above
two, even if you aren't seeding anything else), please give me a shout
so that I can arrange to get them to you.

Now for the experimental part...

To help out long-term seeding of the majority of torrents (those that
aren't bittorrent-only), I'd like to take advantage of the newish
HTTP-seeding support of BitTorrent [1]. 

HTTP-seeding kicks in at a MUCH lower priority than regular BitTorrent
peering, basically helping clients that cannot contact any other source
for missing pieces. It effectively converts the BitTorrent client to a
slightly-smart HTTP fetcher that then shares the blocks back over
BitTorrent to other clients.

For all experimental/ mirrors, that means we're adding a directory tree
under experimental/ that has directories containing only symlinks. Your
rsync options will need to have the various symlink-transfer options,
which I think should be in place already.

Here's an example of one.
experimental/torrent-symlinks/stage3-x86-2007.0:
lrwxrwxrwx stage3-x86-2007.0.tar.bz2 -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2
lrwxrwxrwx stage3-x86-2007.0.tar.bz2.CONTENTS -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.CONTENTS
lrwxrwxrwx stage3-x86-2007.0.tar.bz2.DIGESTS -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.DIGESTS
lrwxrwxrwx stage3-x86-2007.0.tar.bz2.asc -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.asc

This in itself doesn't make the BitTorrent clients contact you, so even
if you aren't participating, it doesn't cause harm to have it.

For part two of the HTTP-seeding (eg making the BT clients actually use
the mirrors), I'd like to know which of the experimental mirrors are
interested in being used for HTTP seeding. 

Your webservers will need to support HTTP Range requests for this to
work. The BitTorrent clients will send range requests for 32K..256K
blocks (powers of two only).

I'll be placing the base potion of your URLs into the .torrent files, as
"$BASEURL/experimental/torrent-symlinks/"

[1] The actual specification for the HTTP-seeding is here:
    http://www.getright.com/seedtorrent.html

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Council Member
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85