Re: [gentoo-soc] Weekly Report: Big Data Infrastructure and Maven Overlay in Week 5

[Miroslav Šulc]

hi,

i was not on this list (now i am) so thanks for cc'ing me.

Dne 06. 07. 20 v 3:09 Benda Xu napsal(a):
> Zhang Zongyu <zzy2529420793@×××××.com> writes:
>
>> This week I have done a lot of repetitive work. That is, to go through
>> all of the ebuild files in Gentoo Repository and to get them the
>> equivalent GroupId and ArtifactId in Maven Central. There are more
>> than 500 packages in dev-java/ and java-virtual/ category, and
>> sometimes one Gentoo package may correspond to more than one Maven
>> artifact, so it took me a lot of time to complete them. And here is my
>> lookup up table which covers all packages in dev-java/ and maps
>> "category/package:slot" into "groupId:artifactId".[1] As a result,
>> spark-core can make use of more than 100 Gentoo native packages now.
i just would like to bring to your attention the content of 
~/.java-ebuilder/cache. it contains all packages that are based at least 
on one of java eclasses, and those eclasses are also listed at the end 
of each row. might help you to find all packages that are java packages, 
even outside dev-java/ category.
> My previous approach was to `ebuild xxxx prepare` the packages in
> ::gentoo and inspect the pom.xml file inside ${WORKDIR}.  Does that work
> for you?
>
>    https://github.com/gentoo/java-ebuilder/blob/master/scripts/meta.sh
this seems to be a really clever idea :-)
>> Besides, I have also added java-ebuilder some features. It can check
>> the metadata of a package and get the correct SLOT, and it can make
>> ebuild unpack "Maven Central distributed source files" into proper
>> ${JAVA_SRC_DIR} now.
> Good!

i wanted to merge your commits to the main repo but your commits are not 
signed (which is required by gentoo) so the merge failed:

$ git push --signed
Enumerating objects: 160, done.
Counting objects: 100% (160/160), done.
Delta compression using up to 8 threads
Compressing objects: 100% (32/32), done.
Writing objects: 100% (140/140), 10.91 KiB | 10.91 MiB/s, done.
Total 140 (delta 48), reused 140 (delta 48), pack-reused 0
remote: Resolving deltas: 100% (48/48), completed with 13 local objects.
remote: FATAL: VREF/proj-gentoo-06-copyright: helper program exit status 256
remote: dd579db2a1505cec77815479ae6a3d6ef3a0c5ce: missing Signed-off-by 
on commit
remote: 99097c6f99c248ccd30fce4519215c257c0c7a89: missing Signed-off-by 
on commit
remote: 58874c99da6c32a2956b8ce4bb4b53478634c0da: missing Signed-off-by 
on commit
remote: 85f88068c6fe41ce0984f90818715967b9b4d15c: missing Signed-off-by 
on commit
remote: d93842e5ba03c6169cc040bab72b5abfc6626d68: missing Signed-off-by 
on commit
remote: ad49c323e6f023c2ddd4935dd1dee1f4a3ad0b60: missing Signed-off-by 
on commit
remote: 0ef403839d003795bb0afcca122e10b6bd0a8c44: missing Signed-off-by 
on commit
remote: e22c64c1c277b3a882cc30d7664cdf508f7f05eb: missing Signed-off-by 
on commit
remote: 974268aa05144c8f28a7b5872e85b4ef2059fb94: missing Signed-off-by 
on commit
remote: a1843f1c17aad889d5b6614bd14eeb992abf9925: missing Signed-off-by 
on commit
remote: 58ee12330f57eeb9b6772ef44889de8e93af4f1e: missing Signed-off-by 
on commit
remote: 2ff7dd8b48f5bc71636e9cedb9f03a95f234208e: missing Signed-off-by 
on commit
remote: f3a06517706afe5868f15508ead6d94c0c9c2080: missing Signed-off-by 
on commit
remote:
remote: Please make sure to read the copyright policy before adding 
Signed-off-by!
remote:   https://www.gentoo.org/glep/glep-0076.html
remote: error: hook declined to update refs/heads/master
To git+ssh://git.gentoo.org/proj/java-ebuilder.git
  ! [remote rejected] master -> master (hook declined)
error: failed to push some refs to 
'git+ssh://git.gentoo.org/proj/java-ebuilder.git'

generally, using `git commit -s ...` adds the required line on new 
commits. you can also rebase your commit messages ( `git rebase -i 
c3b197f0e69c06f58f982f8d8dd16c88e2d5d518`) and push the commits again 
afaik.

>> Next week I am going to test and add jaxb (javax.xml.bind:jaxb-api and
>> javax.xml.bind:jaxb-impl) and some newer version packages which are
>> required by spark-core, and
> Good to know we are getting back jaxb.  It is strange to me that without
> such a seemingly central package packages in ::gentoo still work.  Is it
> now part of icedtea jdk?  Please dig more into the story behind jaxb in
> the #gentoo-java IRC and bugzilla.
>
> Maybe fordfrog have a better answer (@fordfrog, not sure if you have
> subscribed to gentoo-soc, I am Ccing you).
javax/xml/bind is part of jdk:8 but was dropped later and is not present 
in jdk:11 anymore which causes some packages to fail to compile against 
jdk:11 (you can easily check with something like this: `for jdk in 
icedtea:8 openjdk:8 openjdk:11; do echo; echo $jdk; for jar in $(equery 
files $jdk | grep -E ".*\.jar$"); do echo $jar; unzip -l $jar | grep 
javax/xml/bind; done; done`). you have to have those jdk installed 
before running this command.
>> I hope the newer version ebuild files could be accepted by Gentoo-Java
>> team. The prototype of the ebuild files that are just generated by
>> java-ebuilder are listed in [2].
> Please start the code review processes of the new java packages by
> submitting bug reports and pull requests when you see fit.

do these work with java-pkg-simple.eclass as is in the tree or it needs 
your updated eclass? getting the changes in the eclass into the main 
tree requires review and approval by devs.

also, before getting the ebuilds into the tree, i would really like to 
see an automated test to verify that the content of our jar files 
matches the content of the maven repo jar files. without this, we might 
get issues because of missing resources and maybe even other causes.

> Benda

thank you for the work you do! :-)


miroslav

>> [1] lookup table containing all packages from dev-java/
>> https://github.com/6-6-6/JEbuilder/blob/master/app-portage/java-ebuilder/files/scripts/LUT/LUT
>> [2] a prototype overlay
>> https://github.com/6-6-6/spark-overlay/tree/master/dev-java