Re: [gentoo-soc] Sort of "ondemand" mirror for distfiles

[Auke Booij]

What are you describing is just a caching proxy. Try Squid.

2010/3/28 £ukasz Zubkowicz <lukasek@××××××××××××××××××.pl>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'm a CS student from Poland interested in taking part in GSoC 2010.
> I've been using Gentoo for about 5 years. I've expanded my home network
> recently and found some difficulties with keeping things tidy on 5
> different boxes with Gentoo. And that's where my idea for this year's
> GSoC has come from.
> Most of packages I install are common to all of my machines and
> therefore I would be happy to download them only once. It would save
> bandwidth and time for me and others. Trying to solve this issue with
> some ready software I've found two or three solutions, but none of them
> satisfying. Starting with the simplest solution, I could just emerge
> package on one machine and then copy sources to the others, but it's at
> least a little bit stupid. Being smarter, I could export
> /usr/portage/distfiles to other machines through NFS, but as far as I
> know something about NFS it could raise some synchronization errors if
> tried to merge the same packages on more than one machine
> simultaneously. Finally, the designated solution: I could just install a
> mirror on one of my local machines and use in within my network (just
> like I've done it with rsyncing portage tree), but... isn't it to
> expensive? It would be fine if I had a big network with plenty of
> storage, but (keeping in mind my small home network) spending about
> 200GB (that number I've found somewhere) on accumulating all packages
> doesn't seem to me a good idea. It's quite substantial amount of disk
> space for me.
> I feel like there was lack of solution for people like me and I bet
> there are more such users who have about 5 to 15 machines under their
> control. My idea is to develop kind of mirror which would act just like
> the real "big" mirror, but not keep all of packages. Instead of that it
> would download a package when it's needed for the first time and make
> somehow wait machine which is asking for it (and others which will ask
> for it in time before it's fully downloaded). Of course it's only a
> draft, but I just wanted you to feel my idea.
> I wait for your comments and responses. I realize that I may be terribly
> wrong and there is already such thing which I need, but I have spent
> some time on seeking and haven't found it. In such case I'm sorry for
> wasting your time and... please let me know about it  :) .
>
> - --
> Best wishes, Lukasz Zubkowicz
> http://students.mimuw.edu.pl/~lz248258
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkuvO1QACgkQ+6A5fji6urEOAgCgigzFa+WY+nz86lA0ErZPpxlj
> ZH4AoO9UctF51PAOpLVOzjAibbUYiCSE
> =wj/o
> -----END PGP SIGNATURE-----
>
>