[gentoo-soc] GSOC weekly report #1: g-pypi

[Domen Kožar]

Greetings, following is the weekly update about g-pypi (currently named
gpypi2).

Quick links to project info:
* http://bitbucket.org/iElectric/g-pypi2- Repository, issue tracker
* http://hudson.fubar.si/job/g-pypi2/ - Hudson CI instance
* http://docs.fubar.si/gpypi2/ - Sphinx documentation generated by
hudson on each commit
* http://neurogeek.ath.cx/g-pypi - wiki/user stories for scrum
development


Previous week (24-30 May)
=========================

In Montenegro (19-14 May) I was unable to do any coding, but the project
was prepared in any case.

Task: port ebuild.py and cli.py
-------------------------------

bitbucket.org/iElectric/g-pypi2/src/tip/gpypi2/ebuild.py

Ebuild was ported in some manner, but no additional work has been done. 

cli.py was not yet touched, as I am still designing the idea how it
should be best programmed (currently thinking of doing action commands
as Python classes)

Task: migrate enamer functions to a class that can be later pluggable
(for sake of modularity)
----------------------------------------------------------------------

Done. Later, each class will be pluggable in .gpypi2 rc configuration
file. Something like this is what I had in mind:

[gpypi2]
enamer = gpypi2.enamer:Enamer
ebuild = gpypi2.ebuild:Ebuild
...

This makes it easier for someone to subclass Enamer/Ebuild/Younameit and
point it to the right import path.

A lot of work has been done on Enamer. Documentation is almost perfect,
I wrote more than 500LOC of tests (totaling in 1000+ lines of unittests
for enamer).

About 150 packages out of 10k on pypi fail due to version conversion. In
appendix there are version which fail, somehow categorised. If you think
one of those should be covered, please respond!

Task: write class that tries to get SRC_URI (currently implementation
will
query pypi and sourceforge by issuing HEAD HTTP request)
-----------------------------------------------------------------------

Not yet touched. This will be done in upcoming week.

Task: test enamer functions against whole pypi mirror (specially version
parsing) 
------------------------------------------------------------------------

Done -
bitbucket.org/iElectric/g-pypi2/src/53e09a1968ee/gpypi2/tests/test_pypi.py

to run tests against pypi:

$ TEST_LIVE_PYPI=1 nosetests

Unfortunately I had fever whole saturday and partly in sunday, after
almost 2 years of being healty:)


Upcoming week (01-06 June)
==========================

Task: write class that tries to get SRC_URI (currently implementation
will
query pypi and sourceforge by issuing HEAD HTTP request)
---------------------------------------------------------------------

I have now more clear picture how this class should be written. Each
mirror will inherit from Base class, which will supply utility methods.
Class should be able to determine variables out of URI like MY_P and
issue HEAD HTTP request to confirm that link exists.

Add docs and tests of course.

Task: generate basic structure of templates (we are using jinja2)
-----------------------------------------------------------------

In proposal I advised to use Mako (since I'm very familiar with it
because of Pylons), but jinja2 is currently activly developed and very
similar to Mako.

Task: refactor and get familiar with ebuild.py, also write tests and
docs
---------------------------------------------------------------------

The title says everything, I will organize ebuild.py and report next
week my findings and how it will be refactored.


------------------------------------------------------------------------
APPENDIX - here are the mentioned packages that fail version/packagename
conversion
------------------------------------------------------------------------

# too much separators
zope.event-3.5.0-1
ispmanccp-0.0.1alpha3.1 -> 0.0.1_alpha3_p1
yapps2-2.1.1-17.1
bombardier_cli-1.00-748
zw.jsmath-3.6a-0.9
collective-recipe-linktally-0.1.0-3
dradm-1-0.3.3_alpha
iqpp-plone-commenting-0.5.beta.2
django-mptt-2-0.3.0
iccommunity-core-1.0-_pre371
django-cms-2.1.0._beta3
collective-dexteritypaste-1.0alpha1.1
collective-portlet-postit-1.0-_beta1

# too much release indicators
bycyclecore-0.4a0_pre1054
byCycleTripPlanner-0.4a0.dev-r1056
plonetheme-notredame-1.2-beta2_beta
plonetheme-peerstheme-1.0b4dev
id3encodingconverter-0.1alpha.svn20080501
iscorpio-plonepm-2.0beta2_pre382
cjktools-data-0.2.0-2009-12-28
kaa-base-0.99.0dev-r4259-20091102
collective-captchaforms-0.1-20090814
collective-z3cform-kss-0.1b1dev
divdialogs-0.1a1_pre610
django-checksum-0.1.rec68f42a975e
django-chronograph-svn-r21-git.2
products-andreas09theme-0.1-step10
products-clockserver-0.2-Zope2.9
pylucid-boot-0.0.1.git-6e4b6cc
fellowiki-0.01a1_pre36
pyosc-0.3.5b-5294
se-portlet-gallery-1.2.1bugfix_pre8
lymon-0.2r58_pre58
fuxi-1.0-rc-II_pre
tg-ext-silverplate-0.1rc3_pre47
skunk-web-4.0.0a2_pre227
quecksilber-0.1a1_pre20081205
pymoneris-0.1r2_pre2
pyggel-0.08-alpha4_beta
inplace-0.1a0_pre1518
pylucid-0.9.0.RC3.git-fbba4_rc7
pylucid-0.9.0.RC2.svn2609

# unknown
ftd2xx-bzr14
pyopencv-2.1.0.wr1.0.2
plotkit-0.9.1w3
objectroles-0.6X
django-dbpreferences-0.3.2.svn34
ete2-2.0rev89
kmpp-1.0.0b1_wrapper2
z-base-36-1.0.1
nawa-1.0.0._beta090714164121
openallure-0.1d17
htmlserver-2010.02.07.appengine

# EASTEREGG
sflib-1.0dev-BZR-r45-panta-elasticworld.org-20091021145839-1oceeh3stpvyl04t
softwarefabrica-django-appserver-1.0dev-BZR-r10-panta-elasticworld.org-20091023132843-vitk6k7e5qlvhe
pynq-0.4.0.Pineapple
netlogger-trunk
llsd-unreleased.unofficial
asciiporn3k-2009.11.16.py3k.cpp
ast2src-2010.01.21.ast2src
freetype-Milestone1
gdm2setup-0.5.3-karmic