Re: [gentoo-server] (Hardened) Converting production Gentoo mail/web server to

[Kerin Millar]

On 25/01/06, Thilo Bangert <thilo.bangert@×××.net> wrote:
> > I have often considered and even tried a couple of times to setup a
> > hardened box however I get confused between all the different options
> > and all the different implications. What with Selinux Grsecurity 1/2
> > RSBAC PIE etc. etc.
> >
> >
>
> yeah - same here. although i am interested and wouldn't even mind a small
> performance hit i have not had the guts to follow through...

I'll try to summarise some of the most interesting aspects thereof:

* grsecurity refers to a single kernel patch which provides a series
of miscellaneous and very valuable "hardening" options, all of which
are *completely* optional.

* These options are fall within any of the following 8 categories:
address space protection, role based access control, filesystem
protections, kernel auditing, executable protections, network
protections, sysctl support and logging options.

* The options are all quite clearly documented and provide a diverse
range of useful protections (even for those not running fully
"hardened" systems). Highlights include denying writes to /dev/kmem,
rendering kernel symbols invisible to regular users, chroot
restrictions, /proc visibility restrictions, exec logging, restriction
of access to dmesg, randomised TCP source ports and many more.

* This patch also includes and provides PaX for optional and
comprehensive W^X memory protections, even on architectures that do
not support a so-called NX bit. The premise of W^X is that memory
regions can be marked writable or executable but *not* both at the
same time. PaX also allows for full randomisation of the address space
belonging to a process. The stated goal of the project is to put an
end to "arbitrary code execution". Highly recommended reading:
http://en.wikipedia.org/wiki/PaX.

* To best leverage the protections offered by PaX, it should be
judiciously combined with various grsec options. Also, the user-land
should be (at the very least) built with a PIE enabled gcc profile in
order to build position independent executables. For more information
on PIE and other related topics I recommend:
http://www.gentoo.org/proj/en/hardened/primer.xml.

* Although I'm not going to touch upon it much here (mainly because I
have no real experience with it), grsecurity offers a full RBAC system
for those how want to use it. The user-land tool used to manage and
control policy is called "sys-apps/gradm" and features a remarkable
learning mode in order to reduce the burden of developing a sensible
policy in the first instance. Note that this is not the only RBAC
mechanism at the disposal of hardened gentoo users (consider
rsbac-sources and selinux as alternatives for instance).

>
> the craziest thing is, that i seem to get a hardened toolchain built by
> default - without using the hardened profile

Yes, that's absolutely correct. However if gcc is built with
USE="hardened" then the fully hardened specs are just set to be the
default (see below) and the profile that enables the conventional
specs have the term "-vanilla" appended instead.

For other packages the "hardened" USE flag has varying effects. For
example, syslog-ng will install a policy that doesn't suck and creates
a log file specifically to track pax policy violations. The "pic" USE
flag is also quite important. Actually, there are not many stable
packages left in the tree that care about this flag but some still
need it so that special patches are applied to make sure that the
results are fully position independent (or to work around quirks in
the code in lieu of this topic). The hardened team usually try to get
such work accepted upstream.

Of course, the hardened portage profile sets these flags as necessary
and also exhibits a no-nonsence USE flag policy which is very well
suited to servers. Compare this to the standard 2005.1 profile which
is flag soup frankly!

>
> marsupilami ~ # gcc-config -l

[snip]

>  [6] i686-pc-linux-gnu-3.4.4

This profile enables the standard gcc specs.

>  [7] i686-pc-linux-gnu-3.4.4-hardened

This profile enables the fully hardened specs. PIE support is enabled
to force ELF executables to be built as position independent (normally
this would occur only for ELF libraries). Furthermore, SSP is enabled
(-fstack-protector basically) and the linker is always invoked with
the "now" and "relro" options. Resulting binaries that are dynamically
linked force the dynamic linker to resolve all symbols as soon as the
program is started rather than to be deferred (which is sometimes
referred to as "lazy" run-time binding). See the ld manpage for more
information. Basically it works out more or less as: CFLAGS="-fPIE
-fstack-protector-all" LDFLAGS="-Wl,-z,now -Wl,-z,relro". Further
reading: http://www.gentoo.org/proj/en/hardened/hardenedfaq.xml.

>  [8] i686-pc-linux-gnu-3.4.4-hardenednopie

Same, but no PIE.

>  [9] i686-pc-linux-gnu-3.4.4-hardenednopiessp

Same but no PIE and no SSP (i.e. pretty much only the linker flags
options are effected). I would actually say this is a good idea for
all server setups - even if you have no interest in any of the other
stuff.

>  [10] i686-pc-linux-gnu-3.4.4-hardenednossp

Same, but no SSP.

>
> somewhere it says that on x86 the performance penalty for PIE is
> considerable... guess i have to get some AMD64 boxes...

On x86 there is indeed a penalty. In practice I cannot say that I have
noticed it at all. This leads me on to another interesting point.
There is a performance penalty associated with the W^X protection
offered by PaX on processors that do *not* offer a so-called "nx" bit
(recent P4 models, and AMD processors do offer this bit, check the
flags in /proc/cpuinfo). There are two mechanisms to enforce this form
of protection in PaX, PAGEEXEC and SEGMEXEC. PAGEXEC is the method
that should be chosen if either (1) your processor has an "nx" bit or
(2) you are *not* using a P4. Unfortunately, PAGEEXEC seems to be
perform quite poorly on the P4 architecture so in that case I would
recommend SEGMEXEC which is what I am using. I've found that it works
great and the performance penalty is minimal. The only caveat is that
it halves the memory address space visible to any running process from
3G to 1.5G (*not* the amount of memory available fortunately). Here is
a very interesting study on the topic:
http://www.pjvenda.org/linux/doc/pax-performance/. Note that recent P4
cores do support an NX bit and also work fine and dandy with amd64
gentoo setups :)

For those who are concerned that these features may break things,
please bear in mind that all features and options that have been
described are optional. With regard to PaX, specific protections can
be enabled/disabled on a per-executable basis by using the paxctl tool
to assert suitable "markings" in the header of the executable
concerned. As for grsec, it is possible to run it in such a way that
any option can be enabled/disabled via the /proc interface. This makes
it very easy to play with the various options (eventually you can
"echo 1 > /proc/sys/kernel/grsecurity/grsec_lock" which will render
all of the options immutable until the next boot). Furthermore, 3
basic profiles are selectable when configuring the kernel - low,
medium and high. In my experience it is extremely unlikely that any
applications will break when using the "low" or "medium" option
profiles. In fact, as of recent releases I have yet to find an
application that falls foul of the "high" profile. That's not to say
that they don't exist of course.

Those interested in running a fully hardened toolchain and building a
PIE enabled user-land and leveraging the full extent of the memory
protections offered by PaX should note that one of the most irksome
problems faced by the hardened developers (and users alike) are
binaries that exhibit TEXTRELs. It's quite a complex topic which is
expounded upon furthere by Kevin Quinn here:
http://www.nabble.com/Re:-Textrels-in-packages-policy-p1936024.html.
TEXTRELs are highly undesirable and make it impossible to take the
fullest advantage of PaX protections. In some cases, the solution is
to relax the policy applied to the application/binaries concerned.

Personally, I've configured my kernel in such a way that textrels
simply won't be tolerated. Since running a server-based hardened setup
the only packages I've encountered thus far that had any problems in
this regard were hylafax and elinks. In the case of hylafax I realised
that installing a newer version of the jbigkit library then rebuilding
hylafax solved the issue (I filed a bug which was attended to). In the
case of elinks I found that the spidermonkey library used to support
javascript (if you set USE="javascript" has many textrels. So I'm
using links instead for now.

The reason I mention the above is that there are caveats and that,
depending on the number of protections you enable and the packages you
run, you may sometimes need to assert exemptions for affected packages
or do a bit of research to find out what the situation is. I have to
say though, that these problems really tend to bite desktop users much
more. Avid users may have noticed that portage has recently begun
depending on pax-utils and now flags textrels as a QA issue. The
hardened team are ever vigilant in tracking and dealing with this
problem (which is undesirable for all users for a number of reasons).

>
> perhaps some hardened and server people should get together and write a
> short overview... i am in!
>

That's a good idea. I hope that the above has proved to be partially
informative at least! I'd also like to say that I jumped on the
hardened bandwagon not so long ago and am not looking back. The level
of protection offered by the grsec/pax combo is simply incredible and
gentoo is probably now the leading distribution in terms of
pro-actively applying these remarkable technologies. I strongly
recommend anyone who is interested to investigate further and see what
it can do for them. For those who are nervous to experiment I
recommend qemu is a great conduit for doing just that.

Cheers,

--Kerin Millar

-- 
gentoo-server@g.o mailing list