Re: [gentoo-dev] Stack smash protected daemons

[Jason Wever]

On Sun, 26 Sep 2004 02:14:35 -0400
John Richard Moser <nigelenki@×××××××.net> wrote:

> |>2) The risk is real and errors against this seem common.
> |
> |
> | Sure, there is risk in almost everything too.  However just because
> | driving an automobile can be dangerous doesn't mean I'll buy a tank or
> | stay inside just to feel safe.
>
> No, but you'll get one with strong bars in the doors so taht side
> impacts don't crush you to death, but rather push your car (crack open a
> fiberglass car door once, you'll see 'em).  Also, rollbars on cars with
> no hard-top, etc.

And to make analogies, wouldn't technologies such as ACLs,
firewalls, VPNs, etc provide us with our side-impact bars, airbags, etc? 

Additionally just because wood burns doesn't mean people don't live in
houses made from wood.

> | Doesn't this exist already?  if people didn't trust Gentoo then why
> are| they using it?  We can't be held ultimately responsible for
> software we| didn't write.  If you can knock over service foo-1.2.3 on
> Gentoo, chances| are you can knock it over on another Linux or possibly
> any other platform| it runs on either.
>
> I trust Gentoo in the security sense as much as I trust Mandrake, or
> Debian, or SuSE.  The difference is that after taking notice of the
> hardened project, I learned about all kinds of neat stuff like
> - -fstack-protector and PaX, and now don't really trust anything else.

That is where you chose to define reasonable security for yourself and
your applications.  Imposing these options on the unknowing or unwary is
not the best course of action, regardless of the "transparency" of those
options to the user.

> You can't be held responsible for others' security holes, but you can
> take simple steps to mitigate the damages.

This is something that each person needs to evaluate for themselves. 
Whether I agree or disagree with a given individual on what a default
level of reasonable security is, the ultimate decision is up to the end
user.

Gentoo's role is similar to that of a consultant.  You have the
consultants research or investigate a problem, concept, etc and then they
present you with the results, and the possible solutions to go
with. The decision is still up to the end user, and with a consultant,
they*know* what the pros and cons before making that decision.

> |>What I propose to do (pick the low hanging fruit):
> |>1) Add stack protector and and any similar 'features' stable in
> hardened|>to the default CLFAGS of the gentoo install/profiles. By
> stable I mean|>things which do not break the majority of functionality.
> |
> |
> | Feel free to take on the ownership of making this work on every arch's
> | toolchain then.  Also feel free to deal with all upstream authors who
> | start instantly dismissing any bugs from Gentoo due to the fact that
> the| toolchain is quite modified to accomplish this task.  Take the
> current| stance the GAIM team has with us as an example of what would be
> to come.
>
> SSP works on all architectures.

The concept of SSP may work on all architectures, but the implementation
requires a fair amount of work to make it happen.  Getting the toolchain
and/or commonly used applications to build and/or run properly when SSP is
in use has proven to be a problem in the past. If you want exact
historical evidence, just search our bugzilla server, particularly for
SPARC (i.e. bug #.39725).

Granted support for SSP gets better as time goes on, but using the Gentoo
user base (unless they choose to be) as the QA for this is not a good
idea.

> As for upstream authors, guess what?
>
>
> usr/lib/python2.3/site-packages/mx/TextTools/Examples/pytag.py:47:
> SyntaxWarning: name 'debugging' is used prior to global declaration
>
> python: stack smashing attack in function symtable_node()
>
> error: command '/usr/bin/python' terminated by signal 6
>
>
> Well SHIT!  Looks like I've got some MEANINGFUL information about a bug!
> ~ I mean damn, I can tell you right in what function the CHARACTER ARRAY
> that got overflowed was created, and that it was created on the stack
> (char a[n]); this leads you of course to the fact that it had to be
> overflowed by some function that got called from this function AND
> passed a character array, or somewhere along the line where that
> character array was passed.
>
> This is much more useful than "OMFG PYTHON R CRASH N STUFF WHEN I DO
> STFUF!!!1@!2151!"

Perhaps I need to clarify what I said earlier so it isn't taken the wrong
way.  The support issue I am thinking of here is more of the day to day
problems with a package rather than discovered security vulnerabilities in
a package.  For instance, in the past we (being the SPARC team) have had
issues with certain packages, even in the system profile, failing to build
with -fstack-protector enabled.  When running into problems like this
where a package with a non-modified toolchain normally works, software
authors are probably less likely to work to fix the problem unless the fix
is given to them.

I can't say I've heard of a case where SSP helped to find a
vulnerability and an author chose not to fix it.

Also while SSP will provide useful debug information, as you have
illustrated above, to help work towards a solution to the problem, the
average user who would have this enabled would have no real clue what
these messages mean, or even file a bug report about them.  And when a
bug report is filed, a large majority of those bug reports due come in the
form you've indicated above.


> |>2) broken ebuilds can filter-flags until fixed (better approach since
> |>you are only fixing up ebuilds for broken stuff and may also prompt
> the|>devs to try and make the protection work).
> |
> |
> | The protection itself is a work-around to the original problem.  You
> want| to continue to work around the problem even more?
> |
>
> SSP is a work-around for buffer overflows that may or may not lead to
> security exploits.
>
> Filtering in ebuilds is a work-around for bad software which gets killed
> because it's self-overflowing; or for software that triggers potential
> bugs in SSP (yes, SSP is software, written by humans, geniouses yes but
> humans, and can have bugs).  In the first case, of course, SSP tells you
> almost exactly where to start looking.

To reference this with the concern about trust, I would hope that part of
the decision for the people with high levels of security in mind would be
the security of the packages they choose to install.  For instance, I'm
probably less likely to install wu-ftpd over other ftp daemons because of
it's historical problems with security bugs.
 
> |
> |>3) People who prefer not to be protected can remove the settings from
> |>their CFLAGS
> |
> |
> | Personally, I don't think opting out is the way to do this.  Having
> CFLAGS| that are in by default that may or may not work across all
> architectures| is not a good thing.
>
> Opting out of a feature which in usage you're normally not going to
> really notice is there is no the way to do things?  Dude, that's like
> saying you should make locks on windows optional.  They can be unlocked.
>
> And ssp is supposed to be portable.  Etoh and Yoda's paper[1] says that
> The IBM stack smash protection method (ProPolice) is CPU and OS
> independent[2].  I think that you'd be within reason to complain to them
> if it didn't work accross all archs.
>
> [1] http://www.trl.ibm.com/projects/security/ssp/main.html
> [2]
> http://www.trl.ibm.com/projects/security/ssp/node4.html#SECTION00045000000000000000

See my above reply to your comment of SSP working on all architectures. 
Additionally, thanks for the links on SSP.  
 
> |>4) get stuff virus, spam, etc protection functional and leveraged into
> |>the defaults in other words turn on those USE flags in the standard
> |>profiles
> |
> |
> | No thanks.  I don't want to have to spend the first 24 hours or so of
> | using my new"trusted" operating system opting out of all of the overly
> | paranoid defaults.  If I'm looking for a high level of security out of
> the| box, I'll use hardened or OpenBSD.
> |
> |
> |>A personal opinion:
> |>Anyone who thinks that a speed tradeoff is too much for better
> |>protection is crazy. Do us all a favor and play a go night of russian
> |>roulette by yourself to get your thrills.
> |
>
> 1.  Within reason
> 2.  It has to actually affect speed.  Overhead != speed
>
> |
> | OK seriously, this kind of comment isn't needed.  I'm not sure what
> you| hoped to have accomplished by it, but I'm fairly sure it didn't
> work.|
> | As anyone who has spent 5 minutes in the security world knows, there's
> a| fine line between good security and paranoia.  You can lock your
> system| down to high heaven and be sure that people won't get into it,
> but then| you won't be able to do a damn thing either.  What good is
> that?|
>
> Yes, exactly.  -fstack-protector is one of those things you put there
> and never notice, but it does its job.

As someone who supports our end users.  I definitely have noticed it with
regards to the afore mentioned build problems.

> | Right now I have a choice to use these features if I want to.  I don't
> | have to "opt-out" and I would rather keep it that way.  The support
> | nightmare this will create is not worth the potential advantages.
>
> Yes and about 99.9999999% of your user base is probably going to say
> "wha?  SSP?  Wassat?" if you ask them if they use SSP.

This is a strong part of my point.

One of the reasons I originally chose to run Slackware Linux and later
Gentoo Linux is that they did not attempt to do things for the user by
default.  They assumed a certain level of knowledge by default, and let
the users make their own choices.  In having talked with many users of
Gentoo either online or in the real world, this is one of the important
reasons they chose to use Gentoo.

To me, every time a proposal like this comes up that influences what the
defaults are, it takes us one step farther away from that. 

> |>There's more to be said on security but I feel too lazy right now to
> |>type it so if this isn't convinving you let us know.
> |
> |
> | And as this list has shown historically, we can all argue security to
> high| heaven with each party feeling they have the right answer and
> never| accomplish anything.
> |
> | Now please don't get me wrong.  As someone who's day job is in the
> | security field, I very much like and appreciate the efforts that have
> gone| into making secure toolchains and hardened systems a reality in
> Gentoo.| In some cases I do use them as well.
> |
>
> As someone who is passively absorbing this information, I find your
> ignorance combined with your claim of being a security expert to
> indicate that you're full of shit.

You are certainly entitled to that opinion.  I never claimed to be a
security expert, I only said I worked in the field.  And just because I
work in the field doesn't mean I'm intricately familiar with everything in
it.

> You've repetedly referred to the issue of cross-platform portability
> with SSP in here, for example; and I've pointed out once a link that
> shows that SSP is OS and CPU independent.  Do your research, read what's
> out there.

See my above mentioned historical problems with using -fstack-protector. 
What is written on paper and what actually happens in the real world are
two entirely different things.

> I also pointed out how SSP helps you find AND fix the problems, where
> you just said it's a work-around and that upstream maintainers will use
> it as an excuse to fix bugs.  Instead of whining to them that shit
> breaks and that they need to now find some way to reproduce the problem
> OR just go audit their entire codebase over and over until they find it,
> we can tell them EXACTLY where to start looking.  We can ALMOST tell
> them where the bug is.  Doesn't this make life easier for all of us?

Being able to give bug reports to upstream authors as much detail as
possible is always a good thing, be it a regular type bug or a security
bug.

Yes I said SSP is a work-around.  While it can help mitigate the risks of
certain forms of attacks, and help to find a solution for them, it does
not inherently fix them.  The original problem itself still lies in the
original daemon code.

As for upstream maintainers issues with using the modified toolchain, the
problem more revolves around build issues with -fstack-protector enabled
for example.  If it builds for the author and a majority of the users with
their unmodified toolchains, but ours with -fstack-protector enabled fails
to build the package, chances are the problem isn't going to receive a
high level of attention unless the fix is provided.

> This tells me you haven't done your research, haven't tested the system
> and triggered it intentionally to see what happens, have only a rough
> idea if any what you're dealing with, and thus have no place commenting.

As perhaps I should have more clearly indicated in my first email, I'm not
considering this from strictly a security standpoint.  I'm considering
this more from the perspective of someone who's role is to maintain an
architecture for Gentoo.  This requires myself and my team, as well as the
other architecture teams, to extensively test a very large number of
packages.

As you get farther away from the more commonly supported architectures
(x86 and PPC), the problems in the base toolchain, kernel or even a large
number of packages grows quite significantly.  However, the developer base
for these other architectures is quite small (either with regards to
Gentoo or without). With the introduction of this and possibly security
technologies in the future into the default profiles, it leaves an already
tight time schedule even more stressed.

This is also one of the reasons we have herds, and not everyone is in the
hardened herd.  Enabling options such as this by default essentially does
that to some extent in my mind, particularly for the Gentoo architecture
teams.  In an ideal world, it would be nice to have one or two developers
from each architecture who could help to focus on improving security in
Gentoo, but at the current time that isn't possible.

Thankfully we have had some people in the past such as Method,
solar and pappy who have done a tremendous job at first incorporating
these technologies into Gentoo, and then help make them work for
additional architectures.  They can attest to the amount of work needed to
actually get these options to fly as often toolchain components like gcc
or glibc need some patching in order for this to happen.

> I'm no security expert, I don't claim to be; but I at least know the
> subjet matter here better than you, for some strange and unknown reason.

That is entirely possible.  I think some of this is due to us looking at
it from different angles as well.

> | However, I do not believe it is our place nor our job to make that
> choice| for our users.  You cannot protect people from themselves,
> regardless of| the perceived benefits.
> |
>
> I can say that faster.  "General security is a lost cause; only security
> experts have any business having security, even if it's transparent to
> them."

Like a lot of things in life, it pays to do your homework first.  That's
why I'm advocating the opt-in approach here.  Similarly to the fact that
we do not require users to use a syslog daemon (unless other
applications they chose to use require it), but suggest it in the
Installation Handbook, this could be an item to be put at the beginning.
Both to inform the user as to what SSP is and how to enable it if they
choose.

This level of enhanced security can easily be negated by something as
simple as social engineering too.  Choosing to enable one form of security
protection by default can quickly snowball into many forms, which may or
may not be the right decisions for our regular, non-security minded users.

Only by providing users with information and strongly urging them to
read it can they be in the proper position to evaluate that.

And while security is important to some people, it is not to others (yes
this is an endlessly debatable topic, so lets leave it at that and not
contribute to it). . If we are seriously thinking of implementing this, I
would ask that we poll our end users first to see if this is a
default option a majority of them would want or not.

Regards,
-- 
Jason Wever
Gentoo/Sparc Team Co-Lead