Re: [gentoo-dev] Re: Move x86/amd64 CPU extensions USE flags to a new USE_EXPAND variable

[Francesco R]

2010/12/13 Konstantin Tokarev <annulen@××××××.ru>

>
>
> 13.12.2010, 18:53, "Francesco R" <vivo75@×××××.com>:
> > 2010/12/13 Ryan Hill <dirtyepic@g.o>
> >> On Sun, 12 Dec 2010 09:01:13 -0400
> >> "Sergio D. Rodríguez Inclan" <srinclan@×××××.com> wrote:
> >>
> >>>   El 12/12/2010 02:46 a.m., Ryan Hill escribió:
> >>> > I think the fewer sources of magic USE flags the better.  Maybe we
> could
> >>> > document how to figure out what instruction sets a processor supports
> in the
> >>> > handbook instead.
> >>
> >>> A good manual would be greatly appreciated :)
> >>
> >> I wrote a guide a couple weeks ago that might be a good starting point.
> >>
> >> http://en.gentoo-wiki.com/wiki/Hardware_CFLAGS
> >
> > if I read correctly the article on the wiki it does circa what the script
> reported below does.
> > would be possible to adopt something similar for automatic C*FLAGS
> selection if someone step in willing to take the pain to mantain it.
> >
> > #!/usr/bin/python
> > # Copyright 2010 Gentoo Foundation
> > # Distributed under the terms of the GNU General Public License v2
> > # Author: Francesco Riosa
> > # extrapolated from http://en.gentoo-wiki.com/wiki/Hardware_CFLAGS,
> errors are mine
> >
> > # kate: encoding utf-8; eol unix
> > # kate: indent-width 4; mixedindent off; replace-tabs on;
> > # kate: remove-trailing-space on; space-indent on
> >
> > # echo "int main() { return 0; }" | gcc -march=native -v -E - 2>&1 | grep
> march
> > # echo "int main() { return 0; }" | gcc -march=core2 -v -Q -x c - 2>&1
> >
> > """
> > example output:
> > ./hw-cflags.py
> > extrapolating flags for gcc-4.4.5
> >   useful flags: -march=core2 -msse4.1 --param l1-cache-size=32 --param
> l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=generic
> >   redundant:    -mcx16 -msahf
> >
> > extrapolating flags for gcc-4.5.1
> >   useful flags: -march=core2 -msse4.1 --param l1-cache-size=32 --param
> l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=core2
> >   redundant:    -mcx16 -msahf
> > """
> >
> > import os
> > import time
> > import fnmatch
> > from subprocess import Popen, PIPE
> >
> > GCC_PATH = '/usr/bin/'
> > GCC_LIST = fnmatch.filter(os.listdir(GCC_PATH), 'gcc-[0-9].*')
> > GCC_LIST.sort()
> >
> > def extract_flags(gcccmd, header):
> >     # get output from gcc
> >     buf = ''
> >     devnul = open('/dev/null', 'w')
> >     p = Popen(gcccmd, stdin=PIPE, stdout=devnul, stderr=PIPE)
> >     p.stdin.write("""int main() { return 0; }""")
> >     p.stdin.close()
> >     while p.poll() is None:
> >         t = p.stderr.read()
> >         buf = "buf%s" % t
> >         time.sleep(0.01)
> >     p.stderr.close()
> >     devnul.close()
> >
> >     # parse it
> >     flags = []
> >     add = False
> >     for line in buf.split('\n'):
> >         if line.startswith(header):
> >             add = True
> >             flags += line.strip().split(' ')
> >             continue
> >         if add:
> >             if line.startswith(' '):
> >                 flags += line.strip().split(' ')
> >             else:
> >                 break
> >
> >     # extract flags we are interested in
> >     t = []
> >     march = ''
> >     mtune = '-mtune=generic'
> >     for i in xrange(len(flags)):
> >         if flags[i].startswith('-m'):
> >             if flags[i].startswith('-mtune'):
> >                 mtune = flags[i]
> >             elif flags[i].startswith('-march'):
> >                 march = flags[i]
> >             else:
> >                 t.append(flags[i])
> >         elif flags[i] == '--param':
> >             t.append("%s %s" % (flags[i], flags[i+1]))
> >     flags = t
> >
> >     return march, mtune, flags
> >
> > for gcc in GCC_LIST:
> >
> >     print "extrapolating flags for %s" % gcc
> >
> >     gcccmd = [ GCC_PATH + gcc, '-march=native', '-v', '-E', '-', ]
> >     header='COLLECT_GCC_OPTIONS'
> >     march, mtune, flags_native = extract_flags(gcccmd, header)
> >
> >     gcccmd = [ GCC_PATH + gcc, march, '-v', '-Q', '-x', 'c', '-', ]
> >     header='options enabled:'
> >     t, t, flags_enabled = extract_flags(gcccmd, header)
> >
> >     redundant_flags = []
> >     useful_flags = []
> >
> >     for x in flags_native:
> >         if x in flags_enabled:
> >             redundant_flags.append(x)
> >         else:
> >             useful_flags.append(x)
> >
> >     if gcc < "gcc-4.5.0":
> >         mtune = '-mtune=generic'
> >
> >     print "  useful flags: %s %s %s " % (march, " ".join(useful_flags),
> mtune)
> >     print "  redundant:    %s" % " ".join(redundant_flags)
> >     print
>
> Note that for some architectures (e.g, PowerPC) you need to add "-mcpu"
> case
> (-mcpu has the same meaning as -march on x86)
>
>
> --
> Regards,
> Konstantin
>
>
Assuming there is some possibility of Gentoo using some automated method of
choice of CFLAGS the arch maintainers should really step in and tell what
works on their cpus and be able to modify/disable it.

There are other shortcomings in the script, for example it does not work
with <gcc-4.2 because it use -march=native.
Also it should at least consider what's in /etc/make.conf C*FLAGS/CHOST.
It's really a proof of concept