Gentoo Logo
Gentoo Spaceship

Installation:
Gentoo Handbook
Installation Docs

Documentation:
Home
Listing
About Gentoo
Philosophy
Social Contract

Resources:
Bug Tracker
Developer List
Discussion Forums
Gentoo BitTorrents
Gentoo Linux Enhancement Proposals
IRC Channels
Mailing Lists
Mirrors
Name and Logo Guidelines
Online Package Database
Security Announcements
Staffing Needs
Supporting Vendors
View our CVS

Graphics:
Logos and themes
Icons
ScreenShots

Miscellaneous Resources:
Gentoo Linux Store
Gentoo-hosted projects
IBM dW/Intel article archive




List Archive: gentoo-soc
Navigation:
Lists: gentoo-soc: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: gentoo-soc@g.o
From: Arne Babenhauserheide <arne_bab@...>
Subject: Re: [GSoC-status] Tree-wide collision checking and files database
Date: Mon, 15 Jun 2009 08:23:30 +0200
Am Freitag, 12. Juni 2009 17:05:33 schrieb Jeremy Olexa:
> > It seems that whole process could be sped up by hosting binary
> > packages on one central server (Binary host). Obviously various versions
> > of the same package would be created and therefore unique names could be
> > created by using some metadata to create hash part of filename. On a
> > first thought I would use USE flags and DEPEND as metadata to hash.
>
> This is a cool aspect of the project, I hope you can work with solar
> and zmedico to improve binpkgs. USE flags seem to be the trouble spot
> of binpkgs.

Maybe this Bug could help with that: 

"Include more info about a binpkg"
- http://bugs.gentoo.org/show_bug.cgi?id=150031#c7

I wanted to tackle that myself some time ago but got stuck at trying to find 
my way within portage so I postponed it a few times... 

It would only need three things: 


* Find SLOT and USE for any ebuild (SLOT is everything which affects all 
packages on one machine, USE are the *active* USE-flags of the package). 

* Save binpackages in a directory structure which contains a hash over the 
SLOT and a hash over the active USE flags. 

* Locate binpackages via the same hashes. 


The path to a binpackage would then look like this: 

$PKGDIR/$CATEGORY/$PN/$SLOTS_HASH/$USE_HASH/python-2.5.2-r8.tbz2


You can make the USE hash more user-friendly by only hashing a USE string, if 
the unhashed string would be longer than the hashed string. 

As hash you can simply use sha.sha() - maybe in BASE32 encoding to make it 
URL- and filename-safe. 

--- !python
# hashing a USE string
from sha import sha
from base64 import b32encode

hash = sha()
use_string = "STRING OF THE ACTIVE USE FLAGS"
# if the string is longer than a base32 encoded sha1 hash, 
# hash it to have an upper bound on its length
if len(use_string) > 32: 
	hash.update(use_string)
	use_string = b32encode(hash.digest())
...

One missing piece is a nice human readable delimiter for the USE flags in the 
USE string which can safely be used in URLs. If there is none, always hashing 
USE flags is a simple fallback option. 

Since "-" and "_" are already being used in USE-flags I don't know about more 
possible delimiters - except just using the space and escaping it in URLs as 
%20.  

The bash won't like that (require escaping), but since portage is written in 
Python and ebuilds don't need to access binpackages directly that should be a 
minor problem. 

Best wishes, 
Arne

--- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- 
   - singing a part of the history of free software -
              http://infinite-hands.draketo.de

Attachment:
signature.asc (This is a digitally signed message part.)
Replies:
Re: [GSoC-status] Tree-wide collision checking and files database
-- Arne Babenhauserheide
References:
[GSoC-status] Tree-wide collision checking and files database
-- Stanislav Ochotnicky
Re: [GSoC-status] Tree-wide collision checking and files database
-- Jeremy Olexa
Navigation:
Lists: gentoo-soc: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: [GSoC-status] Tree-wide collision checking and files database
Next by thread:
Re: [GSoC-status] Tree-wide collision checking and files database
Previous by date:
Re: [GSoC-status] Tree-wide collision checking and files database
Next by date:
Re: [GSoC-status] Tree-wide collision checking and files database


Updated Jun 17, 2009

Donate to support our development efforts.

Gentoo Centric Hosting: vr.org

VR Hosted

Tek Alchemy

Tek Alchemy

SevenL.net

SevenL.net

php|architect

php|architect

Copyright 2001-2007 Gentoo Foundation, Inc. Questions, Comments? Email www@gentoo.org.