Re: [gentoo-user] OT: GCC 5 Offloading

[Fernando Rodriguez]

On Tuesday, September 15, 2015 3:11:37 AM james wrote:
> Fernando Rodriguez <frodriguez.developer <at> outlook.com> writes:
>
> > Do you know of any plans to enable offloading on the gentoo toolchain?
>
> NO, but I'm sure some devs are keenly aware of this feature in gcc-5.
>
>
> > I was able to build the offloading compiler using crossdev with a few hacks
> > and wrote an ebuild for Intel's simulator[2]. I will work on enabling the
> > host compiler tomorrow and may open a feature request and post patches
> > once I get it working. The changes needed to enable it on the host are
> > pretty trivial.
>
> Sorry Fernando, I just now saw this thread on an old thread. I think that
> 'sys-cluster/ceph' is where I'd like to test your spin on the gcc-5.
> Ceph has RDMA (RoCE) in the 0.94 branch (in portage). You are definately
> ahead of me on practical gcc-5 experiments with offloading and other
> new features.
>
> You did not list your second reference. Where I can I get/git your
> compiler and some brief suggestions on taking it for a test drive.
>
> I'm not much interested in the Intel simulator, atm. I like to test on
> old gear running gentoo:: borking is no big deal, if it happens.
> Other codes keen to test gcc-5 (offloading) on are Apache-mesos and
> Apache-spark and mesos-distcc.
>
> Also, per this doc [1] you can get your own gentoo overlay to put
> things up for wider experimentation, if you like.
>
> [1] https://wiki.gentoo.org/wiki/Project:Overlays/Dev_Guide
>
>
> Very cool, what you have done,
> James

The link was for the emulator ebuild[1]. I got distracted with other stuff and 
didn't make the host compiler, will do it this week. I built the offloading 
(accel) compiler as follows:

1. Use crossdev like when building a regular cross-compiler:

# crossdev -v -t x86_64-intelmicemul-linux-gnu     #for intel MIC
or
# crossdev -v -t nvptx-none  # for nvidia ptx

2. As soon as it start building the 1st stage, hit CTRL-C 

3. cd to your crossdev overlay, create an eclass directory and copy the 
attached eclass to it.

4. cd to the category directory for your compiler and delete the gcc symlink 
and copy over the gcc directory from your portage tree.

5. delete all ebuild versions except the one you want to install and apply the 
attached patch to it.

6. Emerge it with crossdev again, it should succeed. Technically you don't 
need to build the whole thing so you can use crossdev options to just build 
the compiler (I didn't try that).

This works for Intel MCI. For nvidia ptx you need to install nvptx-tools and 
tell the compiler how to find it by adding this on line 800 of the eclass: --
with-build-time-tools=[install-nvptx-tools]/nvptx-none/bin --disable-sjlj-
exceptions --enable-newlib-io-long-long. I will patch crossdev to do it 
automatically when I get time. I won't bother with an ebuild for nvptx-tools 
cause I don't have a card to test and don't want to buy one.

For the host compiler, it just need one configure option so it can find the 
offloading/accelerator compiler. I plan to add GCC_OFFLOAD_TARGETS to make.conf 
and patch toolchain.eclass to add it to the configure command. It's not 
something I'll want to put on an overlay because it affects the whole toolchain 
so I'll just make it a patch and apply it to the toolchain.eclass on the 
portage tree when I want to build a compiler.

I want to test with opencv stuff since I got an embedded project that will do a 
lot of it. I'm hoping to use the emulator to get a good idea of how much can 
be offloaded and if it's worth getting one of those.

[1] https://github.com/fernando-rodriguez/portage-overlay/tree/master/dev-util/intel-sde-external
[2] https://github.com/MentorEmbedded/nvptx-tools/


-- 
Fernando Rodriguez