Gentoo Archives: gentoo-soc

From: "Yuan Liao (Leo)" <liaoyuan@×××××.com>
To: gentoo-soc@l.g.o
Subject: [gentoo-soc] Week 6 Report for Big Data Infrastructure and H2O ebuilds Project
Date: Mon, 19 Jul 2021 05:53:11
Message-Id: CACk4dkvsuNt4s+iHJQsmbwyxi_R8S785bPYrO5vrprGt4uVMfw@mail.gmail.com
1 Hi folks,
2
3 This week, I have been busy with an eclectic mix of tasks. First of
4 all, I continued to make improvements to ebuild-commander so it can
5 run well both in CI environments and on any developers' personal
6 workstations. Another related accomplishment was a new ebuild
7 installation test case to be run in GitHub Actions that would have
8 coverage over all ebuilds in the Spark overlay. Last but not least, I
9 further expanded the documentation for the Kotlin packages I had
10 created a few weeks ago with the addition of a maintainer-oriented
11 wiki page.
12
13 The bulk of the improvements to ebuild-commander are new features
14 designed to optimize its user experience in an interactive
15 environment. While I was developing ebuild-commander and creating new
16 test cases for ebuilds in the Spark overlay, I would often run the
17 test cases from an interactive shell on my own computer before
18 creating a Git commit. Sometimes I realized something was wrong
19 immediately after the tests were launched, so I would interrupt
20 ebuild-commander at once, fix the problem, and re-run the tests. The
21 interruption would almost always be followed by a deletion of the
22 Docker container created by ebuild-commander, which was tedious when
23 it was done manually. So, I added SIGINT handling to ebuild-commander
24 to let it automatically clean up the container it created upon
25 interruption. By contrast, ebuild-commander might also perform an
26 automatic clean-up when the user might want the container to be
27 retained. For example, if the test fails, the user probably wishes to
28 open up a shell in the container to inspect it, in which case the
29 container should not be cleaned up. To deal with this situation, I
30 added a new command-line option to ebuild-commander for controlling
31 the automatic clean-up behavior. By default, the clean-up would be
32 skipped if the test failed, but the user can also choose to keep the
33 container even if the test succeeded or always remove it even upon
34 test failure.
35
36 As per my original project proposal, I am also adding a test case for
37 the ebuild installation tests which will ensure every package in the
38 Spark overlay can be installed at least once. Adding every package to
39 the emerge command theoretically works, but the command would be too
40 long. Invoking emerge separately for each package would resolve this
41 problem, but the overhead of emerge's dependency calculation would
42 seriously impact the test runtime. I came up with a solution that
43 could address both issues: write a script to compute a list of leaf
44 packages in the Spark overlay and pass the packages in the list to
45 emerge, so every package in the overlay would be installed, and the
46 emerge command can be simplified to have a shorter length too. The
47 script can also act as a helpful tool for any ebuild repository's
48 maintainers to find out all leaf packages in the repository for
49 maintenance tasks like last-rite and package clean-up. After some
50 initial optimization and tuning, the script (written in Python) can
51 compute a list of leaf packages among about 500 packages in the Spark
52 overlay within only a few minutes. The optimization and tuning is
53 also the topic for this week's blog post of mine [1]. This post
54 covers some knowledge and topics from computer science, including
55 graph theory, graph algorithms, data structure, and time complexity.
56 If you are interested in any of those subjects, make sure you don't
57 miss it!
58
59 Meanwhile, I created a new page on Gentoo Wiki for the Kotlin Package
60 Maintainer Guide [2] and added explanations for how a Kotlin package
61 is built by the upstream, instructions to find the commands that
62 should be used to compile the Kotlin packages, and a tutorial for
63 using the eclasses I authored to support the Kotlin ebuilds. I hope
64 it can be useful to both any future developers who would like to help
65 maintain those Kotlin packages and myself after a few months in case I
66 forget the details of my own work.
67
68 This concludes my work during the past week and this report. Thank
69 you for reading it (and my blog post in case you are checking it out)!
70
71 Best regards,
72 Leo
73
74 [1]: https://leo3418.github.io/2021/07/18/find-leaf-packages.html
75 [2]: https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin/Package_Maintainer_Guide

Replies