Re: [gentoo-dev] Guidelines for IUSE defaults

[Daniel Campbell]

On 02/02/2017 12:35 PM, james wrote:
> On 02/02/2017 01:01 PM, Rich Freeman wrote:
>> On Thu, Feb 2, 2017 at 11:25 AM, Michael Orlitzky <mjo@g.o> wrote:
>>>
>>> If (base == minimal), then all of the upstream defaults need to be added
>>> to package.use for the upstream-defaults profile. That's bad,
>>
>> I'll go further and say that it is unacceptably bad.
>>
>>> but if
>>> (base == upstream-defaults), then the important IUSE defaults need to be
>>> copy/pasted from our ebuilds into the minimal profile. The latter is
>>> more spiritually damning =)
>>>
>>
>> So, I'll admit I've never been one that cared a great deal about
>> minimalism so I appreciate that I may not be the best one to judge
>> this, so let's go ahead and embrace your statement for the purpose of
>> debate.
>>
>> Is there a better way we can have our cake and eat it too?  I'll admit
>> that a huge package.use on the minimal profile isn't a whole lot
>> better than a huge package.use on all the other profiles.
>>
>> Do we need another form of syntax in individual ebuilds to try to
>> separate out the various cases you cite?  Does anybody care to
>> actually suggest one?
>
> I think that unikernels are something everyone should be aware of
> as they purport to be the latest trend in securing all sorts of systems.
> (a brief read).
>
> http://unikernel.org/
>
>
> Perhaps we should not have a 'minimal profile', as it means so many
> different things to so many different people. We have not even surveyed
> the user base...
>
> So if we think of all the possible profiles and sub-profiles as being
> organized in a tree structure, it is better to figure out logical
> organization of all profiles, imho.
>
> So how about profiles that are either under the embedded or basic
> 'moniker' in the profile tree.  So embedded is the least number of
> packages possible,  regardless of upstream, where folks build up from
> there to what they want as a finished system. Embedded, clusters, HPC,
> and such are compatible enough for what I'm building to be under the
> same branch of the profile tree. If other folks want their cluster works
> under the basic  part of the profile tree that is concerned with
> upstream, then  they have their part of the profile tree located there.
>
> And the 'basic' part of the tree is similar to what we now call
> 'default' and the names build up in whatever schema those target builds
> desire, like basic+headless, basic+kde, basic+?+whatever..... ?
>
> And is there any reason, if necessary that other needs cannot be
> branched off, as necessary form the profile tree?  Perhaps the main root
> is a state diagram of what is need and has links to  relevant documents.
> That way the profile tree is a live system that can  be enhance or
> modified to serve all of Gentoo's diverse visions.

That's an awful lot of change for only slight benefit. The mixin idea
that mgorny's been proposing may become a great way for people to state
their needs explicitly in something that profiles are able to handle,
once that work is finished. I believe Funtoo uses this extensively, and
they might be worth contacting to see how we can best use them for our
needs as well, assuming the design is the same.

>
>
>> I still think that we shouldn't encourage users to lightly deviate
>> from all the upstream defaults.  There are of course legitimate
>> reasons for doing so, and you and I can probably appreciate when we
>> should do this, but for somebody starting out we're giving them a lot
>> of rope to hang themselves with.
>
> This is only the case because profiles are in general in a mess and
> there are little in the way of conventions. What is so sacrosanct about
> upstream for a truly embedded gentoo system or a gentoo based IoT
> device? How many of the gentoo-embedded devs even bother to read
> gentoo-dev? Your vision seems to me to be tainted by the major distros
> and their visions, not be impolite, but they are way off course for
> secure systems, embedded systems, IoT, HPC and numerous other active
> areas of Linux, imho.
>
>
> Think about it, if upstream is so brilliant, and has our needs 'at
> heart' then why have not the kernel-geniuses bothered to build a
> logical, coherent semantic for kernel configurations, management,
> security testing and such?  If fact, if I were to be critical of
> upstream, I'd say those and many other issues should have been addressed
> before the adventures of systemd were dictated upon the larger user
> base. Upstream is an 'ad-hoc' mess, in the old days we called such
> entities a clusterF*.

Profiles may be a bit of a mess. I'm not qualified to determine that
since I don't touch profiles, but the brief browsing I've done indicates
that *some* care was put into it, or there wouldn't be such a large
profile tree.

The thing about Gentoo is that no single use case rules all, but as a
distribution we kinda owe it to the greater user audience to adopt
upstream defaults. The entire point of USE flags is so users can deviate
from that, if they need or want to. Gentoo needs to be usable out of the
box; it can't be if we're dismantling the work we've put into figuring
out sane USE flags across the tree.

I'm not saying profiles are perfect, just that they make a decent
blueprint. With mixins likely coming to the tree, I'm excited to see
what's possible with them.
>
>
> So those of us going back to minimal and basics and such venues, even
> with clusters, could not care less about Mozilla, systemd and thousands
> of other upstream folks  that have lost their pathway forward. (deepest
> apologies, but I have very strong feelings about "upstream*".

I do too, which is why I deal with it with my USE flags. It is, however,
upstream's software, and most of the time they're the most qualified to
determine what the default should be. Defaults aren't set in stone in
Gentoo; that's part of its beauty.

>
>
> Many of the profiles in this and other recent threads, are just 'ad-hoc'
> naming structures and locations, and that historical flexibility extend
> to the devs is fantastic. This enhance/cleaning need of gentoo profiles
> needs to be well thought out and as flexible semantically as possible.

Agreed.

>
>
> It is absolutely a superior approach to not care at all what upstream
> does, to provide a pathway for embedded gentoo. That is a fundamental
> characteristic of what an embedded system is. Thanke the code, purge
> 90+ percent of this, and then install it on a embedded system so
> only what is needed is encluded. On a distro, you can pile on tons
> of uncessary codes, for convenience and not care, but embedded,
> it does matter. That is why so much of Iot is hacked and owned
> by folks with nefarious intentions. Much of 'upstream' is growing
> dumber by the day, and corporate manager and government interlopers
> are just loving the cruft of 'upstream'.

I think that largely depends on your purpose for a given system. IoT
devices get cracked and abused mostly due to misconfiguration and a
failure to update when CVEs and whatnot are released. Being completely
Spartan may help against that, but as we all know... old code tends to
rot if not taken care of with at least backported patches.

Upstream is important because they supply 99% of the software we use. We
are just the distributors (hence "distribution"). We ideally shouldn't
have too strong of an opinion as maintainers; expose enough USE flags to
make a given package configurable, and apply '+' to upstream's default
stuff. I'd be surprised if a considerable number of packages on Gentoo
*weren't* that way.

Again, package.use (and maybe mixins) are the way to fix this.
>
>
> Minimal is a close cousin to embedded, and now unikernels and aggressive
> HPC clusters are going that direction too in the name of performance,
> sane-management and reducing attack surfaces for the cloud or HPC
> cluster (not all, but many).
>
>
> Surely a branch of the gentoo profiles tree, could rigorously be focused
> on compatibility with upstream, but that inflexibility, if imposed
> on everyone else, will only serve to further alienate gentoo-embedded
> and serve as a unnecessary wedge between some minimal, HPC and unikernel
> types of gentoo builds.
>

Is there not an embedded profile? If that's where the beef is, perhaps
you should be talking to the embedded team rather than the greater
Gentoo dev community. They're likely better versed, and they *should* be
paying attention to gentoo-dev, as is expected from us. None of us works
in a vacuum, and we should be aware of what's happening with the whole
distribution, if only to see that there's activity somewhere.

A default Gentoo installation is not inflexible. It's merely a default
that can later be changed; even before your first call to 'emerge'.
>
>
>> It is like building a kernel
>> answering no to the largest number of questions possible while still
>> actually building something.  I'd actually be curious as to what that
>> kernel would even be capable of doing (there are a lot of fairly
>> essential things you can turn off in the kernel).
>
>
> This is a whole other area of valid concern. Their was a GSoC project
> last summer to work on organizing gentooish kernel builds, but that is a
> very big topic. Perhaps the profiles will have to be proposed but not
> formalized until folks go out and do some kernel builds and testing
> associated with a proposed profile organization?

afaik building the kernel is completely outside of Portage's reach. It
merely installs the source files needed to build it. Everything else is
up to you and/or genkernel.

A special kernel fork designed for embedded sounds good, though. I'm
sure we've got something like that in the tree (or an overlay) somewhere.
>
>
>> In the same way, we shouldn't be too quick to deviate from upstream
>> defaults ourselves (#4 in your example), beyond actual integration
>> work.
>
>> I'll admit the current state is a bit of a compromise, but I don't
>> think we should change it unless we're changing it to something
>> significantly better.  This is a pretty big-impact change.
>
>
> Just formalize some new parts of the profile tree to not be required to
> be upstream compliant. Isn't that part of being a 'meta-distribution'?

Don't we already do that? KDE, GNOME, and XFCE aren't "upstream"
compliant because there *isn't* a single, default upstream DE, so we
have profiles for that. If you or others would like to create or improve
existing profiles, then that's awesome and you should try to put
together some patches or a pull request so we can take a look.

>
> In my future (and the future of many others) there will be minimalistic
> builds on clusters where any number to constructs including
> compatibility  which will all be solved, at the framework level, not the
> base distro level, to the extent possible. Folks are now running a
> myriad of windows OS versions on linux (&clusters), so I have just read
> about a few days ago. So I'm proposing and working on gentoo
> heterogeneous cluster where one can partition part to be for systemd
> stuffage, part to be HPC, part to be extraordinarily secure, part to be
> aligned with a particularly commercial linux distro, and many other
> differing needs based frameworks.
>
>
> The minimal (lowest level) should be clear of all of those distro
> encumbrances. CoreOS is taking this approach, as their bare metal
> bootstrapping occurs completely and well before systemd or any other
> PID1 schema is invoked or becomes a defacto requirement.  Gentoo is all
> about freedom, right?

If we need a new profile, then certainly those who are going to use it
should be best equipped to know what needs to be in it, right? This is a
great case for building what you need and then sharing it so everyone
can benefit. I don't do embedded (though I might tinker with it some
day), so I'm definitely not able to know what needs to be in such a
profile. We need people who actually work in that sort of stuff to do
the work, because if someone like me does it, then it may have too many
packages in it, or it won't account for X or Y use case that's super
common in embedded, etc.

Embedded is an itch, and non-embedded Gentoo people can't scratch it for
you because we aren't qualified. If you or others ever manage to make
that profile, please share it so others can benefit too. :)
>
>
> hth,
> James
>

Thanks for reading,
zlg
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6