From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1JYkuB-0007dE-P4 for garchives@archives.gentoo.org; Mon, 10 Mar 2008 16:31:16 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0E189E0558; Mon, 10 Mar 2008 16:31:15 +0000 (UTC) Received: from indigo.cs.bgu.ac.il (indigo.cs.bgu.ac.il [132.72.42.23]) by pigeon.gentoo.org (Postfix) with ESMTP id 96FDCE0558 for ; Mon, 10 Mar 2008 16:31:14 +0000 (UTC) Received: from lesbinux (lesbinux [132.72.41.93]) by indigo.cs.bgu.ac.il (8.13.6/8.13.6) with ESMTP id m2AGVsp1026811; Mon, 10 Mar 2008 18:31:54 +0200 (IST) Date: Mon, 10 Mar 2008 18:31:54 +0200 (IST) From: Alexander Piavka X-X-Sender: piavka@lesbinux To: Justin Bronder cc: gentoo-cluster@lists.gentoo.org Subject: Re: [gentoo-cluster] Installing and using multiple MPI implementations at the same time. In-Reply-To: <20080208031923.GA27361@mejis.cold-front> Message-ID: References: <20080208031923.GA27361@mejis.cold-front> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-cluster@lists.gentoo.org Reply-to: gentoo-cluster@lists.gentoo.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: ClamAV version 0.87, clamav-milter version 0.87 on ha-rs2 X-Virus-Status: Clean X-Spam-Status: No, score=-1.3 required=6.5 tests=ALL_TRUSTED,BAYES_50 autolearn=no version=3.2.3 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on gandalf X-Archives-Salt: 37ed214b-6a2a-4130-8b7b-4be10b96e9c6 X-Archives-Hash: 93e52a97b3ff4f1572f43c2d0d874086 Hi Justin, I've started playing with your empi implementation. Some problems & suggestions: 1)'eselect mpi set ...' does not check for existance of ~/.env.d dir and fails if one does not exists. It creates ~/.env.d/mpi which looks like this: ---------------------- PATH="/usr/lib64/mpi/mpi-openmpi/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.1.2:/opt/blackdown-jdk-1.4.2.03/bin:/opt/blackdown-jdk-1.4.2.03/jre/bin" MANPATH="/usr/lib64/mpi/mpi-openmpi/usr/share/man:/etc/java-config-2/current-system-vm/man:/usr/local/share/man:/usr/share/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.18/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.1.2/man:/opt/blackdown-jdk-1.4.2.03/man:/etc/java-config/system-vm/man/" LD_LIBRARY_PATH="/usr/lib64/mpi/mpi-openmpi/usr/lib64:" ESELECT_MPI_IMP="mpi-openmpi" export LD_LIBRARY_PATH export PATH export MANPATH export ESELECT_MPI_IMP ---------------------- while the following would be better: ---------------------- PATH="/usr/lib64/mpi/mpi-openmpi/usr/bin:${PATH}" MANPATH="/usr/lib64/mpi/mpi-openmpi/usr/share/man:${MANPATH}" LD_LIBRARY_PATH="/usr/lib64/mpi/mpi-openmpi/usr/lib64:${LD_LIBRARY_PATH}" ESELECT_MPI_IMP="mpi-openmpi" export LD_LIBRARY_PATH export PATH export MANPATH export ESELECT_MPI_IMP ---------------------- maybe even ---------------------- if [ "X${PATH}" != "X" ]; then export PATH="/usr/lib64/mpi/mpi-openmpi/usr/bin:${PATH}" else export PATH="/usr/lib64/mpi/mpi-openmpi/usr/bin" fi if [ "X${MANPATH}" != "X" ]; then export MANPATH="/usr/lib64/mpi/mpi-openmpi/usr/share/man:${MANPATH}" else export MANPATH="/usr/lib64/mpi/mpi-openmpi/usr/share/man" fi if [ "X${LD_LIBRARY_PATH}" != "X" ]; then export LD_LIBRARY_PATH="/usr/lib64/mpi/mpi-openmpi/usr/lib64:${LD_LIBRARY_PATH}" else export LD_LIBRARY_PATH="/usr/lib64/mpi/mpi-openmpi/usr/lib64" fi export ESELECT_MPI_IMP ---------------------- Also, probably . besides /etc/env.d/mpi/mpi-openmpi the /etc/env.d/XXmpi file should also be created with the default empi profile then 'eselect mpi set ' is run. 2)another problem is a failure to install binpkg of openmpi on another identical systems, the error is * * ERROR: mpi-openmpi/openmpi-1.2.5-r1 failed. * Call stack: * ebuild.sh, line 1717: Called dyn_setup * ebuild.sh, line 768: Called qa_call 'pkg_setup' * ebuild.sh, line 44: Called pkg_setup * openmpi-1.2.5-r1.ebuild, line 23: Called mpi_pkg_setup * mpi.eclass, line 306: Called die * The specific snippet of code: * [[ -f "${FILESDIR}"/${MPI_ESELECT_FILE} ]] \ * || die "MPI_ESELECT_FILE is not defined/found. ${MPI_ESELECT_FILE}" * The die message: * MPI_ESELECT_FILE is not defined/found. eselect.mpi.openmpi * * If you need support, post the topmost build error, and the call stack if relevant. * A complete build log is located at '/var/tmp/portage/mpi-openmpi/openmpi-1.2.5-r1/temp/build.log'. * I thinks this is due to MPI_ESELECT_FILE being defined in pkg_setup() of openmpi ebuild and not in top of ebuild (will check if this would help later) 3) If i have PORTDIR_OVERLAY="/usr/local/overlays/csbgu /tmp/empi" empi --create --implementation mpi-openmpi =sys-cluster/openmpi-1.2.5-r1 would create mpi-openmpi category tree under /usr/local/overlays/csbgu/mpi-openmpi since it's first overlay in PORTDIR_OVERLAY, it would be nice if it could ALWAYS be created under the empi overlay i.e /tmp/empi/mpi-openmpi Of couse i can put the empi overlay first in PORTDIR_OVERLAY instead but i want to avoid manual tweaking as much as possible. With all mpi-implementation residing in the same overlay tree as empi it would much more convienient , for me, to auto distribute single overlay among cluster hosts and avoid possible need for commands like 'empi --create --implementation mpi-openmpi ...' Thanks Alex On Thu, 7 Feb 2008, Justin Bronder wrote: > Well bug 44132 [1] has been a real thorn in my side for some time. The > problem being that all mpi implementations currently in portage block each > other. This is a real pain for any serious administrators, makes testing > new/other implementations too time-consuming, and in my opinion has probably > slowed any progress related to getting cluster applications into portage or > the overlays. > > All that being said, I set out to do something about it. Donnie Berkholz was > nice enough to help me out early on and ended up suggesting that I look > at crossdev. I shamelessly ripped off the aforementioned script and tuned > it for mpi implementations and packages. It should be noted that all the > changes I've implemented do [should, hopefully will] not effect the way the > packages behave now, rather it just adds another option. The idea is as > follows. > > sys-cluster/empi: Does the same stuff that crossdev does. You create a new > implementation root by specifying a name and mpi implementation package to > build it with. [2] Empi adds these to an overlay under a new category with > the name you gave. The ebuilds inherit mpi.eclass which handles pushing all > the files to /usr/lib/mpi/, and providing files for eselect-mpi. > > Later, you can add packages by specifying the category name and packages to > install. Empi grabs the ones that inherit mpi (including deps), adds them to > the overlay and emerges. The deps stuff hasn't been tested all that much. > > app-admin/eselect-mpi: Used to setup a user's environment to call a named > implementation install. Basically does this by dropping a bash-sourceable > file in ${HOME}/.env.d/mpi. The not-so-fun part is that the user then has to > remember to source that file. Also used a fair amount by mpi.eclass. > > I threw some quick notes showing commands and output up in my devspace. [3] > If anyone is at all interested, please try this out and let me know what you > think. Even if you don't try it out but have ideas/comments/rants, I'm all ears. > > Current overlay has the following ready to be tried out. > eselect-mpi/eselect-mpi-0.0.1 > sys-cluster/empi-0.1 > sys-cluster/hpl-1.0-r3 > sys-cluster/lam-mpi-7.1.4-r1 > sys-cluster/mpi-examples-1.0 > sys-cluster/openmpi-1.2.5-r1 > > A couple of final words, hpl and mpi-examples currently wouldn't work without > empi, mainly because I'm lazy :) Also I still haven't figured out a good way > to handle do{man,doc,www} etcetera, ideas are welcome. There's still a fair > amount of work to be done, but I wanted to see if anyone had any feedback > regarding the general concept first before pushing on. > > You can pull the overlay from rsync://cold-front.ath.cx/mpi (for now...) > > Thanks, > > > [1] http://bugs.gentoo.org/show_bug.cgi?id=44132 > [2] Currently openmpi-1.2.5-r1 or lam-mpi-7.1.4-r1 > [3] http://dev.gentoo.org/~jsbronder/README.empi.txt > > -- > Justin Bronder > -- gentoo-cluster@lists.gentoo.org mailing list