Re: [gentoo-user] xz memory hungry?

[Volker Armin Hemmann]

Am Mittwoch, 22. August 2012, 22:16:19 schrieb Jorge Almeida:
> On Wed, Aug 22, 2012 at 9:39 PM, Florian Philipp <lists@×××××××××××.net>
wrote:
> > Am 22.08.2012 20:52, schrieb Jorge Almeida:
> >
> > This should not happen, especially on such a small archive. I've tried
> > `strace xz -t m4-1.4.16.tar.xz` and looked for calls to mmap (e.g.
> > memory allocations). They never were larger than 68 MB
> >
> > Try it yourself. The second parameter in mmap is the allocated size in
> > byte.
> In the box where it works:
>
> $ strace -e trace=mmap2 xz -t m4-1.4.16.tar.xz
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7746000
> mmap2(NULL, 134226, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7725000
> mmap2(NULL, 155888, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0xb76fe000
> mmap2(0xb7723000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x24) = 0xb7723000
> mmap2(NULL, 107004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0xb76e3000
> mmap2(0xb76fa000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16) = 0xb76fa000
> mmap2(0xb76fc000, 4604, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76fc000
> mmap2(NULL, 1727172, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
> 3, 0) = 0xb753d000
> mmap2(0xb76dd000, 12288, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19f) = 0xb76dd000
> mmap2(0xb76e0000, 10948, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76e0000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb753c000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb753b000
> mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb733b000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7745000
> mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = 0xb723a000
> +++ exited with 0 +++
>
>
> In the other box, in the gentoo chroot:
>
> # strace -e trace=mmap2 xz -t /usr/portage/distfiles/m4-1.4.16.tar.xz
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb779e000
> mmap2(NULL, 12143, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb779b000
> mmap2(NULL, 143600, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> 0) = 0xb7777000
> mmap2(0xb7799000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7799000
> mmap2(NULL, 1448488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
> 3, 0) = 0xb7615000
> mmap2(0xb7771000, 12288, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15c) = 0xb7771000
> mmap2(0xb7774000, 10792, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7774000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7614000
> mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7414000
> mmap2(NULL, 4096, PROT_READ, MAP_PRIVATE, 3, 0x102a) = 0xb779d000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb779c000
> mmap2(NULL, 67112960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 67244032, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 2097152, PROT_NONE,
> MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xb7214000
> mmap2(NULL, 67112960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb779c000
> xz: /usr/portage/distfiles/m4-1.4.16.tar.xz: Cannot allocate memory
> +++ exited with 1 +++
>
>
> Thanks,
>
> Jorge Almeida


maybe a memory fragmentation problem? It tries to allocate 65mb continous 
chunk - and can't find one?
-- 
#163933