[gentoo-amd64] Re: doomsday

[Duncan]

Chris posted on Sat, 13 Mar 2010 16:21:59 -0500 as excerpted:

> Hi all. I'm a  bit of a newb on gentoo/emerge.. so forgive me if I'm
> reviving the dead horse for yet another flogging. I just subscribed and
> posted without lurking/searching. I'm trying to get this package to
> compile on my amd64 box to run in dedicated server mode (no GUI
> required). My kernel has IA32 support compiled in.
>
> The package in question is hard masked ~amd64, but can be worked around
> with
>
> ACCEPT_KEYWORDS="~amd64" emerge doomsday

That's generally not a recommended way of emerging something, because the 
next update will want to unmerge it.  Instead, add a line like this to 
your /etc/portage/package.keywords file:

=games-fps/doomsday-1.9.0_beta67	~amd64

That would limit the keyword to that specific version.  If you want to do 
it for the package in general (so when say 1.9.0_beta69 comes out, 
presumably keyworded ~amd64, portage will automatically upgrade to it), do 
it this way:

games-fps/doomsday			~amd64

That way (either one), portage knows that it's OK for it to keep that 
package at ~amd64 and won't attempt to unmerge it at the next update.  You 
can then simply "emerge doomsday" and it should work.

FWIW, the term /hard/ mask generally refers to packages in package.mask.  
Such an entry requires a parallel unmask entry in /etc/portage/
package.unmask , if you wish to use it anyway.

What you're referring to here is /keyword/ masking.  It's rather 
different.  There's arch-stable (amd64 for us), arch-testing (aka ~arch, 
~amd64 for us), and unkeyworded (for us, no amd64 at all in keywords).  If 
a package is known /not/ to work on an arch or profile (such as
amd64/no-multilib), it's also often hard masked but only for that profile.

Unkeyworded only for amd64 generally means it simply hasn't been tested on 
amd64 yet, while no keywords at all indicate an early testing version, 
often a live sources version or early alpha upstream, or an early ebuild 
that still has known issues in general.

~arch aka arch-testing, ~amd64 in our case, by Gentoo policy means it's 
reasonably stable upstream and is generally on the path to stable for 
Gentoo, so is considered appropriate for ~arch users, but it hasn't been 
fully tested with arch-stable yet.  Since by policy a package may not be 
stabilized until all its dependencies are also stable, the package itself 
may be stable, but a (perhaps USE flag optional) dependency is still ~arch 
and not yet ready for stable.  The /general/ policy (individual 
maintainers and packages can differ) is that a package must be in ~arch 
for 30 days without (significant) bugs before it can be considered for 
stable.

It can be noted that while a package maintainer controls the initial 
introduction of a new version ebuild and requests stabilization, it's up 
to the individual arch teams (gentoo/amd64 in our case) to decide whether 
a package is /actually/ stable on a particular arch.  This may take some 
time as it involves testing of its own, the result being that a keyworded 
stable package has been tested and found stable by both its maintainer in 
general, and by the specific arch keywording it.

Also note that not every version is necessarily stabilized.  For fast 
developing and releasing packages, several ~arch versions may go by before 
the maintainer asks for one to be stabilized -- and of course, especially 
on the minor archs, more may go by and another stable candidate or two may 
be requested, before the arch team actually gets around to stabilizing one 
of them.

So ~arch is generally far more fast changing than arch-stable.

Finally, note that in general (thus, there are exceptions), packages not 
considered stable upstream are not even candidates for ~arch, since in 
theory, every ~arch package should be a a candidate for arch-stable.  And 
if there are serious known bugs, a package is normally not even allowed in 
~arch (and can be masked if already in ~arch).

> This package compiles fine, but then segfaults immediately due some
> memory allocation issues in the code, thus the mask.

This sounds suspicious to me.  Wigames-fps/doomsday-1.9.0_beta67th that 
serious a bug, what is it even doing in ~arch?  It sounds to me like it /
should/ be hard-masked, thus requiring an entry in package.unmask in 
ordered to merge it at all, regardless of its keywording.  (Or, if that's 
only an issue on amd64, it shouldn't have the ~amd64 keyword at all and/or 
should be package.masked in the amd64 profiles.)

Unless it's only occurring for some people, presumably due to other issues 
on their machine.  (Maybe it compiles fine for full ~amd64 users, but not 
for stable amd64 users only ~arch keywording individual packages, due to 
older gcc or something.)  /Then/ it might be only ~amd64, but your 
implication is that it's doing this for /all/ amd64 users and possibly all 
users, period, in which case it shouldn't even be ~arch keyworded.

> However I should be able to force this package to compile into 32bit
> mode. Can anyone take a peek at it for me and give me direction?
>
> run attempt:
> root@lbox:/usr/src/linux# emerge doomsday
>
> Calculating dependencies... done!
>
> !!! All ebuilds that could satisfy "games-fps/doomsday" have been
> masked. !!! One of the following masked packages is required to complete
> your request: - games-fps/doomsday-1.9.0_beta67 (masked by: ~amd64
> keyword) - games-fps/doomsday-1.9.0_beta62 (masked by: missing keyword)
>
>
> Do I require multilib to force x86?
> Portage 2.1.6.13 (default/linux/amd64/10.0/no-multilib, gcc-4.3.4,
> glibc-2.9_p20081201-r2, 2.6.31-gentoo-r6 x86_64)

You don't want to force x86 (32-bit) compile, and it'll likely be 
considerably more complicated than you're making out to do so.

As explained above, the ~amd64 keyword indicates (or /should/ indicate) 
that it's 64-bit compatible.  If it's 32-bit only (generally because it's 
a precompiled binary, tho it could be simply that the code isn't 64-bit 
clean, it hasn't been ported yet), it should be package.masked for
no-multilib profiles.

So you /should/ be able to compile it as 64-bit.  If you can't, it's a bug.

Meanwhile, if you want to try 32-bit anyway (or just for general 
information), keep reading:

For multilib amd64 users, their toolchain should do both 32-bit and 64-
bit, so in theory, they can simply add -m32 to the gcc CFLAGS and force 32-
bit compilation.  However, since portage doesn't track multiple archs, it 
won't understand that and will thus consider it a 64-bit package.  This 
quickly leads to breakage so it's NOT recommended.  Instead of using 
portage to do 32-bit compiles (at least those beyond the few that are 
specially designed for it, that generally do both 32- and 64- bit versions 
with one package, gcc, glibc, sandbox, etc), therefore, it's recommended 
that you do all 32-bit builds manually -- that is, outside of portage.  Of 
course that means tracking all the dependencies yourself, which might be 
OK for a package or two, but quickly gets out of hand if you're doing much 
more than that.

Additionally (still for multilib), there's only a limited number of 32-bit 
libraries installed to build against, and only a few more available as the 
prebuild binary emul-linux packages.  Once you get beyond these 
dependencies you have to start manually building and installing 32-bit 
versions of all the other libraries you need as well as the executables 
you're trying to install.   Building one executable may require a half 
dozen or more additional dependencies, all manually installed and 
tracked.  This is why it so quickly gets out of hand as the number of 
packages you want to install this way goes up.

For no-multilib users, the only way to compile 32-bit packages is to use a 
32-bit chroot, as the whole /point/ of no-multilib is that it simplifies 
things by omitting the 32-bit toolchain and base libraries.  Without the 
chroot, it's 64-bit only.

Fortunately, the gentoo/amd64 project maintains a 32-bit chroot guide.  
Here's a direct link.

http://www.gentoo.org/proj/en/base/amd64/howtos/index.xml?part=1&chap=2

The idea is a reasonably simple, but there's a few tricks that make things 
easier and prevent a few gotchas, thus the guide with the step-by-step, 
above.

In general, you prepare a separate empty directory (I use an entirely 
different partition, but that's not required) for the chroot, untar a 
normal x86 stage-3 tarball within, and sort of go about things as if you 
were building a whole separate 32-bit system to multiboot into.  Only 
unless you actually /do/ want a 32-bit multiboot, you don't have to build 
things like syslog and the kernel, because the main 64-bit system is your 
host; you only chroot into the 32-bit side from your 64-bit host.  There's 
a few other tricks and shortcuts as well, like mount-binding various 
existing directories from your 64-bit host under the 32-bit chroot, so 
they can be accessed from the chroot as well.  This is what the chroot 
guide covers.

But in general, using the chroot, you are then using an entirely separate 
instance of portage, tracking 32-bit while your main system portage still 
tracks 64-bit. and likewise, configured for 32-bit (separate CFLAGS, USE 
flags, the whole bit) as opposed to the 64-bit main system.  Since you're 
using an entirely separate 32-bit instance, you can now let it manage all 
the dependencies as it normally would, without worrying about it screwing 
up the 64-bit tracking.  Thus, in the chroot, you can emerge 32-bit 
packages pretty much as you would on an entirely separate machine, and not 
have to worry about screwing up the main 64-bit system on the one hand or 
manually tracking all the 32-bit dependencies on the other.  The down side 
is that it's a lot of work, because it's as if you /are/ managing, almost 
(say 80% of), an entirely separate system.  But once you get beyond a 
couple 32-bit packages, it's still far less trouble than attempting to 
manage all those compiles and their the dependencies manually. =:^)

FWIW, I run a 32-bit chroot here, but fully configured and built out, 
kernel, syslog, /etc configured, everything, included.  But I don't boot 
it on the main machine and actually couldn't, as the 32-bit kernel isn't 
configured for that.  Instead, I rsync it to my much less powerful 
netbook, so I can run Gentoo on it while compiling everything on my more 
powerful main system.  The netbook doesn't even have a portage tree on it, 
tho it was less trouble just keeping portage and the rest of the build-
only system in the rsynced image than trying to figure out how to exclude 
it (tho I do exclude some things).

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman