From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 76C6C13933E for ; Mon, 19 Jul 2021 05:53:11 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9FCD0E0B4E; Mon, 19 Jul 2021 05:53:10 +0000 (UTC) Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 2DC2CE0B4E for ; Mon, 19 Jul 2021 05:53:10 +0000 (UTC) Received: by mail-vs1-xe2a.google.com with SMTP id x22so6882223vsq.5 for ; Sun, 18 Jul 2021 22:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=8kpPt6L2E79scKP6+g6TvyRdl5XuHJ+K7tIQ5rs82cw=; b=StVRcwQWRhOLeOdGMSsFzk/gEX3g3Cu8pIC2pD0qJPrhaP6boWnBEot5mkvoaKDxvy Sz6SOweLk35fsp5blvtNeJj5ZqJFXvLxOYXclJj/+L4nWcEJtgctiWCGsTHiAfHdrm0E AWOMTihD2j9Ic/IKk54yqJHcGgIKZ5rxsRgSafik4aIULxwqLBiJckC5ER6RM3jHTz2s 40ozKdQbJ7GpsN53luQXD4wgG2vJpOrsPR0gPJObn5zORZVknVV/cyRa4FJkGtjuttxd PZSRUns0cPSXDUhfctZyq+mTQup/a9YkcBkQTYRsRwThTuRCI19DWNOWStYlaMIHN1vg GPPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=8kpPt6L2E79scKP6+g6TvyRdl5XuHJ+K7tIQ5rs82cw=; b=cU9MhCeWDJ0pLHarmPe1NMvRqleNu+ytBEH6VYrSMWZvGlAxHRFotrKQqpJ8HqNmk6 vkPhuxrzH/nCv75Ge15qASO2+6BuBVG7IX4vhnfNAPADxaCG1IJwEkn/JZLQfTDLPKSf OTOVwmrl6zGPzTHMR7FJprUShsOv8Ey8DL/MQvMdBl5rynlYYqMvSOoJaxg7nub2imHq TNo2/Kg1Ie32NfwIcaciv+IzaGc5KiYJNhBjPwBgQ2h7WEL6R+65zkmW1qZBIeLyB4s7 qGJmz8jzYBq8fSKoVBJMgytKgdkMy4wHLZUCJiDn19o1naptwwKoD7ZPeeNJYijnwA/Y vAuA== X-Gm-Message-State: AOAM530oE9Pfu71K8XXc2BRX3T5+0axP5c1Inyhzjw1Mdzbdl5XvZjDB y4FC41nXWTgVvGDlpb7/M7fdzkXd5vxkM8FK/BFCjfdQIRiVJA== X-Google-Smtp-Source: ABdhPJzPb3yoXXde5akZ+wIDXj3S/S1HNo7HMSFofBV25gyBh99d6emtlewnXkOFgNvQ9nNTe8ynoJw+s1R2pJis6gA= X-Received: by 2002:a67:7142:: with SMTP id m63mr22716171vsc.8.1626673988358; Sun, 18 Jul 2021 22:53:08 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-soc@lists.gentoo.org Reply-to: gentoo-soc@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 From: "Yuan Liao (Leo)" Date: Sun, 18 Jul 2021 22:52:32 -0700 Message-ID: Subject: [gentoo-soc] Week 6 Report for Big Data Infrastructure and H2O ebuilds Project To: gentoo-soc@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: bfa8d0b8-4cdd-4f9b-a689-40e183d8c8a2 X-Archives-Hash: 9e59ff8df3e824bc0c8e106c829f8b0d Hi folks, This week, I have been busy with an eclectic mix of tasks. First of all, I continued to make improvements to ebuild-commander so it can run well both in CI environments and on any developers' personal workstations. Another related accomplishment was a new ebuild installation test case to be run in GitHub Actions that would have coverage over all ebuilds in the Spark overlay. Last but not least, I further expanded the documentation for the Kotlin packages I had created a few weeks ago with the addition of a maintainer-oriented wiki page. The bulk of the improvements to ebuild-commander are new features designed to optimize its user experience in an interactive environment. While I was developing ebuild-commander and creating new test cases for ebuilds in the Spark overlay, I would often run the test cases from an interactive shell on my own computer before creating a Git commit. Sometimes I realized something was wrong immediately after the tests were launched, so I would interrupt ebuild-commander at once, fix the problem, and re-run the tests. The interruption would almost always be followed by a deletion of the Docker container created by ebuild-commander, which was tedious when it was done manually. So, I added SIGINT handling to ebuild-commander to let it automatically clean up the container it created upon interruption. By contrast, ebuild-commander might also perform an automatic clean-up when the user might want the container to be retained. For example, if the test fails, the user probably wishes to open up a shell in the container to inspect it, in which case the container should not be cleaned up. To deal with this situation, I added a new command-line option to ebuild-commander for controlling the automatic clean-up behavior. By default, the clean-up would be skipped if the test failed, but the user can also choose to keep the container even if the test succeeded or always remove it even upon test failure. As per my original project proposal, I am also adding a test case for the ebuild installation tests which will ensure every package in the Spark overlay can be installed at least once. Adding every package to the emerge command theoretically works, but the command would be too long. Invoking emerge separately for each package would resolve this problem, but the overhead of emerge's dependency calculation would seriously impact the test runtime. I came up with a solution that could address both issues: write a script to compute a list of leaf packages in the Spark overlay and pass the packages in the list to emerge, so every package in the overlay would be installed, and the emerge command can be simplified to have a shorter length too. The script can also act as a helpful tool for any ebuild repository's maintainers to find out all leaf packages in the repository for maintenance tasks like last-rite and package clean-up. After some initial optimization and tuning, the script (written in Python) can compute a list of leaf packages among about 500 packages in the Spark overlay within only a few minutes. The optimization and tuning is also the topic for this week's blog post of mine [1]. This post covers some knowledge and topics from computer science, including graph theory, graph algorithms, data structure, and time complexity. If you are interested in any of those subjects, make sure you don't miss it! Meanwhile, I created a new page on Gentoo Wiki for the Kotlin Package Maintainer Guide [2] and added explanations for how a Kotlin package is built by the upstream, instructions to find the commands that should be used to compile the Kotlin packages, and a tutorial for using the eclasses I authored to support the Kotlin ebuilds. I hope it can be useful to both any future developers who would like to help maintain those Kotlin packages and myself after a few months in case I forget the details of my own work. This concludes my work during the past week and this report. Thank you for reading it (and my blog post in case you are checking it out)! Best regards, Leo [1]: https://leo3418.github.io/2021/07/18/find-leaf-packages.html [2]: https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin/Package_Maintainer_Guide