Re: [gentoo-user] xz memory hungry?

[Michael Mol]

On Wed, Aug 22, 2012 at 2:52 PM, Jorge Almeida <jjalmeida@×××××.com> wrote:
> # tar -xJvf /usr/portage/distfiles/m4-1.4.16.tar.xz
> xz: (stdin): Cannot allocate memory
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
>
> The box has 2G ram + 1G swap. I'm installing Gentoo from an existing distro.
> Emerging fails on m4. tar xJvf fails both from within the chroot and from the
> host system. top shows that nothing is using any amount of memory worth
> mentioning. Extracting libtool-2.4.tar.xz works. I can extract
> m4-1.4.16.tar.xz in a computer with 4G ram. This is ridiculous. Not gentoo
> related, except that I have no choice, as m4 is pulled by other packages.
> What to do?
>
> app-arch/xz-utils-5.0.3 in chroot
> xz 5.0.4 in host system (Archlinux)

How much do you have free? From xz's manpage:

 Memory usage
       The memory usage of xz varies from a  few  hundred  kilobytes
to  several  gigabytes
       depending  on  the  compression  settings.  The settings used
when compressing a file
       determine the memory requirements of the decompressor.
Typically  the  decompressor
       needs  5 %  to  20 % of the amount of memory that the
compressor needed when creating
       the file.  For example, decompressing a file created with xz
-9  currently  requires
       65 MiB of memory.  Still, it is possible to have .xz files that
require several giga‐
       bytes of memory to decompress.


Three things come to mind:

1) You may not have enough memory free
2) There may be a bug (either compile/link-induced or code-induced) in
the copy of xz you're using
3) Upstream used some insane settings, causing a massive increase in
the amount of RAM required to decompress that stream.


You could download the .tar.xz file, decompress it on a different box,
and then recompress it with lighter settings.

unxz filename.tar.xz
xz -1 filename.tar

-- 
:wq