From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id AA94E1381F3 for ; Mon, 2 Aug 2021 07:59:50 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 7D08AE0886; Mon, 2 Aug 2021 07:59:48 +0000 (UTC) Received: from mail-vs1-xe2f.google.com (mail-vs1-xe2f.google.com [IPv6:2607:f8b0:4864:20::e2f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 091BCE0886 for ; Mon, 2 Aug 2021 07:59:47 +0000 (UTC) Received: by mail-vs1-xe2f.google.com with SMTP id ba4so6160258vsb.5 for ; Mon, 02 Aug 2021 00:59:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Jh3WWfggwRbYcWSJBK1Kfckf5GQlZuBuIO5tj2Vs/t8=; b=WeWjSJ8Zy2V5JX2uhA1Lx4mUxABSn0l6wEeARCFZtHGVqlVGsdqADyEITgGqJmyZKO lTZeZzE7iWAp394/uYR/co8kZsLwjahAR8gU/iik7dqHO7xPGTZWTmODzxBHwUwj4yxB t6F5L0aT5378SvRq1Q3a6NZAKAUXXd7XKAIwvnVvHiQsucCkh0TNJ6XjDuarhPDgUcYT pFA0mGRX29nU2PjgQ204h+d4zsrMDEio+PmyxqtEcLMzRfelzGQSiQbUBrMwQxmevlPK FHj3JeH77K602/KII7Mq2VmX0sfWPs/DGBBOnqi6ggQ9y5AUR497q77R6mAucODJzRE1 rZvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Jh3WWfggwRbYcWSJBK1Kfckf5GQlZuBuIO5tj2Vs/t8=; b=SxMrQgbGSIMxyULU9f3rf3vkTXmI6dKF56SsTMc5/l8YSyPRBCKB0K9LLwCe6hk9Y0 0YOzPBuFbjUe6UW6QJCVMg5NtlZ3NzWYqxqGWjgC48liYmudw8RlKwyDz+jumwNdGMM/ B44NwQ7OmlFa7PKlWd5IC+GN8vmY7L6Q6QsWmXuhF/2GS7u66ewPOocYqAT7g7lY7Dys aL8KJV3S7FEVn5OGbapk4DEK3mnPaMfejSmsBXdWUS2zwtdlUf3uKVicGuK4+o6w7wNW WEBNyFNDzeicBMwlob/biueQ3dMSrId0VOs3OE8hh1QcbcRS3ZAntyDrwYiZieqyuDi6 G0Qw== X-Gm-Message-State: AOAM533osdDX0bQDM6GouHeiYjIBz6rjCgjBDILThUyifvzovMEWfnoB 0BLB0LU0ANyhSawvLlBP/kfZLcahty7rVLyhrg3v/Q0f2xPyJw== X-Google-Smtp-Source: ABdhPJz+bM6TQy7hD1UkZL2L+p10xAuYUcoj8M+TBR2L5ZFhrP3SMW1D/Ot32e4hCnuaHOFJMIqWXuXKN4a2MBoHJdg= X-Received: by 2002:a67:c911:: with SMTP id w17mr8981171vsk.37.1627891186925; Mon, 02 Aug 2021 00:59:46 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-soc@lists.gentoo.org Reply-to: gentoo-soc@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 From: "Yuan Liao (Leo)" Date: Mon, 2 Aug 2021 00:59:11 -0700 Message-ID: Subject: [gentoo-soc] Week 8 Report for Big Data Infrastructure and H2O ebuilds Project To: gentoo-soc@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: 8514b1ce-da36-4337-ba90-0a08b65dd7c4 X-Archives-Hash: 3d7dd209eb40642dc39258af17d6151b Hi folks, The past week was largely a week of stabilization. First and foremost, I tried to use the Kotlin eclasses I had worked on about a week ago to retrofit existing ebuilds for Kotlin packages including okhttp, okio and reactor-core, all of which were successful. After tinkering with those eclasses, I believed they were ready for creation of more Kotlin packages on Gentoo. eselect-kotlin was also stabilized a few days later with an initial v0.1.0 release and a specification for any Kotlin compiler packages on Gentoo that are intended to be recognized by the eselect module [1]. Then, I continued to fix broken ebuilds in the Spark overlay and finally reached a state where all packages can be installed with the default USE flag settings, and the changes have been merged into Zongyu's main repository created last year. During this process, I also helped resolve a bug in the dev-java/zstd-jni package (#776910) when it was being pulled as a dependency of packages in the Spark overlay and bump dev-java/jansi in ::gentoo to 1.13 after fixing an ebuild in the Spark overlay for the same version of the package. Before I started to revamp existing ebuild in the Spark overlay, there were about 60 out of 500 packages that could not be built from source and thus would require USE="binary" to be explicitly enabled for them by the users. This week, I went through each of them one by one in the hope to make them installable without the 'binary' USE flag. The most common reason that they could not be built from source was issues within dependencies, and one of the following resolution was applied to each ebuild depending on the actual situation: - A dependency's version was incorrect, and the correct version was available in either ::gentoo or the Spark overlay. If the correct version was in a different slot, then the slot would be changed in the problematic ebuild. Otherwise, a new slot would be created for the correct version so that the problematic ebuild could use the version without affecting other ebuilds. - A dependency's version was incorrect, and the correct version is too old. This often happened to packages requiring Guava. If an ebuild in the Spark overlay would need Guava, it usually would depend on Guava 29.0; but, the correct Guava version needed to build some of the packages could be as old as 18.0. In this case, the reason the ebuild could not be compiled from source was that some APIs in Guava had been removed. Fortunately, for every removed API, Google would state its replacement API(s) in Guava's documentation, so this kind of issue could be resolved by patching the problematic ebuild's source files to switch to the replacement APIs. - A package that existed in either ::gentoo or the Spark overlay was missing in the build-time classpath. In this case, the package would be added to the problematic ebuild as a dependency. - A package that did not exist was required for building, and an ebuild for the package could be easily created with java-ebuilder or by hand. The new ebuild would be added to the Spark overlay, and the problematic ebuild would depend on it. - A package that did not exist was required, and it was hard to create an ebuild for it; or, the build failure was caused by an issue not related to dependencies and could not be trivially fixed. To save time for the remaining final parts of my GSoC project, I just simply added IUSE="+binary" to the ebuild to enable the USE flag by default with a comment describing the obstacles to compile the package from source. My new blog post for this week is now available [2]. The topic for this post is the ebuild testing infrastructure I created and have been using to verify the working status of ebuilds in the Spark overlay. An interesting fact for the testing framework is that I used it to test my fix for the zstd-jni bug and the new jansi-1.13 ebuild too, so it is not only suitable for testing small ebuild repositories but probably can be used to test a subset of ebuilds in ::gentoo as well. Thanks for reading, Leo [1]: https://github.com/Leo3418/eselect-kotlin/blob/v0.1.0/README.md#gentoo-kotlin-compiler-package-specification [2]: https://leo3418.github.io/2021/08/01/ebuild-repos-testing-solution.html