Re: [gentoo-user] Beta test Gentoo with mdev instead of udev; version 5 - failure :-(

[Michael Mol]

On Wed, Mar 14, 2012 at 2:45 PM, Canek Peláez Valdés <caneko@×××××.com> wrote:
> On Wed, Mar 14, 2012 at 12:09 PM, Michael Mol <mikemol@×××××.com> wrote:
>> On Wed, Mar 14, 2012 at 1:22 PM, Canek Peláez Valdés <caneko@×××××.com> wrote:
>>> On Wed, Mar 14, 2012 at 9:16 AM, Alan Mackenzie <acm@×××.de> wrote:
>>>> Hello, Canek
>>>>
>>>> On Tue, Mar 13, 2012 at 06:07:32PM -0600, Canek Peláez Valdés wrote:
>>>>> On Tue, Mar 13, 2012 at 5:03 PM, Alan Mackenzie <acm@×××.de> wrote:
>>>>
>>>>> > The new hardware will "just work" if there are the correct drivers
>>>>> >built in.  That's as true of udev as it is of mdev as it is of the old
>>>>> >static /dev with mknod.
>>>>
>>>>> No, it is not. You are letting out the sine qua non of the matter: the
>>>>> device has to be built, *and the /dev file should exists*. I hope you
>>>>> are not suggesting that we put *ALL* the possible files under /dev,
>>>>> because that was the idea before devfs, and it doesn't work *IN
>>>>> GENERAL*.
>>>>
>>>> Previously you made appropriate /dev entries with mknod, giving the
>>>> device major and minor numbers as parameters.  This appeared to work in
>>>> general - I'm not aware of any device it didn't work for.
>>>
>>> Again, I believe you are not following me. In *general* the number of
>>> potential device files under /dev is not bounded. Given N device
>>> filess, I can give you an example where you would need N+1 device
>>> files. With your experience, I assume you know about huge arrays of
>>> SCSI disks, for example; add to that whatever number of USB devices
>>> (and the hubs necessary to connect them), whatever number of Bluetooth
>>> thingies, etc., etc.
>>>
>>>  Therefore, mknod doesn't solve the problem in general, because I can
>>> always give an example where the preset device files on  /dev are not
>>> enough.
>>
>> And I can always give an example where there can't be enough inodes
>> (or perhaps even RAM) to contain enough device nodes. "General Case"
>> is a beautiful thing for a theoretical system, but my computer is not
>> a theoretical system. Neither is my phone, or my server.
>
> You are arguing that the mknod method should be used? Because that
> dicussion happened ten years ago; that train is long gone. If you want
> to argue with someone about it, it would not be me.

No, I was taking your argument to its perceived end result. You want
the universal solution, but that requires limitless resources in
things like memory and integer sizes. The software doesn't exist
within such an environment. The assumptions which it's already
depending on limit its utility in lower-end hardware.

>
>>
>>>
>>>>> So, you need something to handle device files on /dev, so you don't
>>>>> need every possible device file for every possible piece of hardware.
>>>>> But then you want to handle the same device with the same device name,
>>>>> so you need some kind of database. Then for the majority of users,
>>>>> they want to see *something* happen when they connect aa piece of
>>>>> hardware to their computers.
>>>>
>>>> That happened under the old static /dev system.  What was that /dev
>>>> system, if not a database matching /dev names to device numbers?  I'm not
>>>> sure what you mean by "same device" and "same device name".
>>>
>>> That if I connect a USB wi-fi dongle, and it appears with the name
>>> wlan23, I want *every* time that dongle to have the wlan23 name .Good
>>> luck doing that without a database.
>>
>> udev does something nice here, and I believe mdev is capable of
>> similar. sysfs exports device attributes such as model and serial
>> number, and you could trivially derive the node name from that.
>
> I think (as does the udev maintainers) that there should be a strong
> coupling between the device manager, the database handling, and the
> firing of scripts. Otherwise. we get back to devfs, which again, that
> train is long gone.

From the sound of it, it sounds like mdev matches that description.
mdev supports the renaming of devices, so there's your database. It
supports firing scripts.

>
>>>
>>>>> So you need to handle the events associated with the connections (or
>>>>> discovery, for things like Bluetooth) of the devices, and since udev is
>>>>> already handling the database and the detection of
>>>>> connections/discovery, I agree with the decision of leting udev to
>>>>> execute programs when something gets connected. You could get that
>>>>> function in another program, but you are only moving the problem, *and
>>>>> it can also happen very early at boot time*, so lets udev handle it all
>>>>> the time.
>>>>
>>>> Early in boot time, you only need things like disk drives, graphic cards
>>>> and keyboards.  Anything else can be postponed till late boot time.
>>>
>>> Bluetooth keyboards. Done, you made my system unbootable when I need
>>> to run fsck by hand after a power failure.
>>
>> The userland bluetooth stack is an abomination. (Actually, the _whole_
>> bluetooth stack is an abomination. You don't want to know what
>> embedded developers who build car stereos and the like have to go
>> through to try to test things. Or what real packets fundamentally look
>> like 'on the wire'.)
>>
>> It needs a real overhaul. I used to use a bluetooth keyboard, but I
>> found it to be a real mess. I even joined the Linux Documentation
>> Project with the intent of getting bluetooth profiles, apps and stacks
>> indexed and cross-referenced, but there's just way too much going
>> wrong with bluetooth. I eventually switched to using a propriety
>> wireless keyboard, and relegated the bluetooth keyboard to secondary
>> access and to controlling the PS3.
>>
>> Besides, your BIOS isn't going to support bluetooth, either; if you're
>> concerned about failure-case recovery, and you don't have a keyboard
>> you can navigate your BIOS with, you're very probably doing something
>> wrong.
>
> BIOS is going the way of the dodo too, but that's besides the point.

No, it's really not. Replace 'BIOS' with 'UEFI' or whatever you like;
you still need _something_ to handle initial hardware initialization
which has device-specific knowledge. And if your UEFI firmware
supports Bluetooth, whether the OS supports it or not becomes a moot
point; you can use a legacy support mechanism such as the ones that
allowed me to use USB keyboards in DOS. (Yes, I've done that.)

> I'm actually quite happy with the Linux bluetooth stack (which, if I'm
> not mistaken, is used by Android). I have several bluetooth thingies,
> they all work great.

Go look at the spec. Go look at the list of existing Bluetooth
profiles. Sit down and _study_ this thing which you keep holding up as
an example. You've had someone else do all the work for you, and I
don't think you understand how much of a mess it really is. I would
have loved to set up a huge amount of stuff piped and routed via
bluetooth as a PAN, but outside of a few core profiles with de facto
support, nothing really interoperates well.

>
>
>>>
>>>>> I hope you see where I'm going. As I said before, mdev could (in
>>>>> theory) do the same that udev does. But then it will be as complicated
>>>>> as udev, *because it is a complicated problem* in general. And I again
>>>>> use my fuel injection analogy: it is not *necessary*. It is just very
>>>>> damn convenient.
>>>>
>>>> It may be a complicated problem in general, but many people do not need
>>>> that generality.
>>>
>>> ^^^^^ That's your mistake! (IMHO). I explain below.
>>>
>>>> I suspect the vast majority don't need it.  Neither the
>>>> typical desktop, the typical server, nor typical embedded devices like
>>>> routers.
>>>
>>> Alan, the "vast majority" of Linux users right now are phone users.
>>
>> Phone users are _excellent_ examples of embedded environments. You
>> have a static hardware set, where you can predict darn near everything
>> about it that you might possibly need during a boot sequence. You're
>> not expected or intended to cope with boot failures. Your bluetooth
>> earpiece and input devices can wait until the OS has loaded.
>>
>> This is exactly the kind of environment where I would expect udev to
>> _not_ be necessary during boot.
>
> The phones/tablets of now have everything but a "static hardware set".
> The connect (wirelessly usually) with a lot of things, and they have
> USB and SD besides. So I would expect udev to *be* necessary with
> them.

For boot?

Also, you want udev to handle wifi? Bluetooth, I could see, but wifi
belongs elsewhere; AFAIK, the kernel doesn't send hotplug events for
visible networks.

USB (client mode, not host mode) and SD, sure, you want a hotplug
handler. But that's not necessary during boot, and there's nothing
dynamic about them apart from they're either active or they're not.
They'll have static hardware positions. You won't be adding and
removing them.

USB host most (which few mobile devices have, but some do) is a
different animal, but, again, not something typical (your mainstream,
mass market, majority of Linux users) mobile and embedded devices need
to care about at boot.

>
> I'm still wainting for someone to tell me how mdev handles bluetooth.

I'm still waiting for you to explain why bluetooth makes sense for
your boot-failed device.

>
>>> That was my initial point some mails ago. What *you* believe are
>>> "regular" users (people like you, or maybe even me), stopped being
>>> true a couple of years ago. The days of the Unix admin and workstation
>>> user are going the way of the dodo.
>>>
>>> At least, that's how I see it.
>>
>> I can only assume you've run a server. I expect you currently run at
>> least one. If you haven't, then you _really_ have no standing to argue
>> that server environments are irrelevant; you'd be belying a massive
>> lack of understanding of systemic, practical and operational context.
>> I can't _imagine_ you expect the client/server architecture of the
>> Internet to go away in the near future. You're very explicitly talking
>> about sacrificing Linux's efficiency in a server environment in favor
>> of The General Case, which is a case which should only be needed once
>> the system has already booted.
>
> "sacrificing Linux's efficiency in a server environment"? What are you
> talking about? I run my servers with udev... *and* systemd. Have you
> tried it? Are you *sure* it's less efficient, or it seems so to you
> "in theory"?

The more complicated the software, the more RAM, CPU and disk it must
consume. Ergo, by increasing resource usage in one area, you decrease
its availability in another area.

There's the theory. And, yes, I do run udev on my servers, because my
servers aren't such that I want to poke and prod with alternatives
quite yet. Someday, I very probably will run mdev, because I see the
memory, CPU and RAM counters for the three udev instances on my
server, and I want those resources for something else, because I
_care_ about the performance of my server. And I'd care even more if I
was trying to fit things inside 2MB of RAM instead of 2GB or 4GB of
RAM.

>
> More complex doesn't mean less efficient: again, look at the fuel
> injection analogy. It's incredible complex, but it gives you *more*
> fuel efficiency.

In a dynamic system, perhaps. A server is certainly not a dynamic
system. The boot sequence of my phone does not need to be a dynamic
system. The boot sequence of my tablet does not need to be a dynamic
system.

>
> The general  solution is *specially* important for the server case.
> Kay and Lennart work for RedHat, and be sure RHEL will have udev and
> systemd at some point. And guess what? I'm willing to bet another beer
> that it would be more efficient.

I think you and I are at some strong disagreement by which we count efficiency.

>
>> We used to joke that emacs would be a great operating system if only
>> it had drivers. From your arguments and positions, it's looking like
>> udev is setting itself to be the next target of similar jokes.
>
> Maybe it's coincidence, but I still use Emacs ;)
>
> Read the last mail from Greg KH in the -dev list: more coupling is
> what most of the devs are looking for, because *it solves more cases*
> of the general problem.

Sure. People like integration, because standardizing interoperability
of discrete pieces in a system requires cooperation, and is thus more
difficult. Making a monolith, though, almost invariably makes
maintenance more difficult in the long term. Going back to cars, I
can't use an engine from a Ford in a Hyundai, because their interfaces
aren't standardized. I can take a video card from a Dell and put it in
a Lenovo, though, because those interfaces _are_ standardized.

>
>>
>>>
>>>>> I have a really time understanding why you don't see the complexity on
>>>>> the problem. I explained above: it is a complicated problem (when
>>>>> dealing with the general case), and therefore the (general) solution is
>>>>> bound to be also complicated.
>>>>
>>>> I've had a hard time understanding, because up till now, nobody's
>>>> described the problem in detail - there's only been hand-waving.
>>>>
>>>>> You want it simple? Tha'ts fine, it is possible. It's just that it
>>>>> will not solve the general problem, just a very specific subset of it.
>>>>
>>>> That subset used by the vast majority of Linux users.
>>>
>>> Again, think about phones. And tablets. And TVs. And
>>> [insert-here-cool-gadgets-from-the-future].
>>
>> See my argument above about static, predictable hardware,
>> strictly-defined boot process and clear stages of operation.
>
> Again, if it's simpler/easier, why has anybody implemented it?

Not sure exactly what you're asking/saying. What is "it" here? udev?
Because someone didn't like the status quo and came up with a
solution. And then the solution snowballed into something massive and
nowhere near as simple and elegant as its initial versions.

Yes, udev was once a pretty decent piece of software, but it's
reaching to cover too many different use paradigms, and *that's*
what's making it so complex. But people didn't really care until udev
decided that it was going to step away from trying to be
self-contained, and declared restrictions on the hosting environment.

*That* is taking a direct movement away from The General Case; they
redefined The General Case to something that was easier for them to
cope with.

> And
> please, don't say "mdev", because (as I have said), it doesn't work
> *right now* for all my use cases. When it does, I keep betting that it
> would be as complex as udev.

I'd bet it won't. And shouldn't. Because your use cases are outside
its target. (And, actually, your use cases are outside what your
hardware was designed to do. By forcing it out of its niche, you're
injecting inelegence into your own system.)

>
>>>
>>>>  And yes, I do want
>>>> it simple, because elegant simplicity is the best way, IMAO.  You, on the
>>>> other hand, seem to love complicated solutions because they are "the way
>>>> forward".  We'll have to agree to disagree on this one.
>>>
>>> No, it's not a matter of "that's the way forward". It's a matter of
>>> trying to solve the general problem. And since the general solution
>>> also solves the simple cases, I don't see a reason to waste my
>>> time/resources in a solution that in the end will not solve the
>>> general problem.
>>
>> Honestly, you're too dead set on finding a solution to the general
>> problem, because to solve the general problem, you must redefine
>> reality to cope with the limits of your theory. In order to handle the
>> burden of your general-case-software, hardware itself must become more
>> powerful and more expensive, which in turn will lead to greater
>> complexity for your general case. Old hardware, no longer capable of
>> running the general case software, even though it's a specific-case
>> device, is wastefully discarded.
>
> What are you talking about man? The "general problem", by definition,
> includes old hardware. That's why it's "general".

But the more complex you make the software, the more difficult it
becomes to fit it into old hardware.

>
> I think you guys are not seeing the forest because of all the trees.

I think you're forgetting that a forest is made of trees.

>
>>
>>>
>>> Of course, as I have been saying from the beginning, this is
>>> OpenSource. Want to pour some effort into solving the simple cases
>>> that will not help all the community, and that it will only help in
>>> fact a minority, that's your prerogative (and Walt's, and Vapier's,
>>> and everyone else that don't like the "complex" but complete
>>> solution). Go nuts with it if you want.
>>>
>>> But please don't dismiss the general solution as "unnecessary" complex
>>> when it's not the case, and don't think that the "simple" setups
>>> (whatever that means) are the majority. Again, think phones and
>>> tablets: those *are* the majority.
>>
>> Again, phones and tablets are simple cases.
>
> OK; call me when they use mdev.
>
>>>
>>>>> Just as mdev is doing; Walt just posted an email explaining that if
>>>>> you use GNOME, KDE, XFCE, or LVM2, mdev is not for you.
>>>>
>>>> Walter is, I believe, mistaken here.  I can mount and use my LVM2
>>>> partitions.  Gnome looks like it comes up OK, but that could be moot,
>>>> since right now I haven't got keyboard/mouse drivers under the X server.
>>>
>>> Oh, for sure you can modify LVM2 to work under mdev. Also
>>> GNOME/KDE/XFCE, and everything under the sun. You have the source; you
>>> can do *anything* you want with it.
>>>
>>> But the effort wasted^^^^^Hpoured in that excercise will only serve a
>>> handful of users, and it will be never accepted upstream, because
>>> upstream is (rightfully) concerned with the general problem.
>>
>> And here is the biggest reason I get frustrated with you. You say "go
>> ahead and try, you're wasting your effort. You're going to fail.
>> Everyone will leave you behind." It's an incredibly negative,
>> aggressive attitude which isn't geared toward finding a solution, but
>> is rather geared at reducing the problem.
>
> I'm just stating my opinion; if you find it "negative", well sorry.
> Disagree with me if you want, but (obviously) I don't think or see the
> problem as you guys. I'm not being "negative"; I'm just call'em as I
> see'em.
>
> And besides, it matters not what I say or don't say, because what
> matters is the *code*. And the code of udev works right now in the
> general case; mdev (as cute hack as it could be) doesn't. That's all
> what I'm saying.

Every time someone starts actually doing something about it, you tell
them they're wasting their time, and that they're doomed, but,
whatever, it's their time to waste.

That's pretty negative and condescending, and is polarizing the issue.


>
>
>>>
>>> Again, this is OpenSource: go nuts with any problem you find
>>> "interesting" (I really don't understand why solving a particular case
>>> of the general problem will be more interesting that solving the
>>> former, but that is maybe the Computer Scientist in me).
>>
>> That's exactly what it is. I have a _lot_ of interest in (and respect
>> for) computer science[1], but I also have several years of practical,
>> real-world software development. I have the privilege of being allowed
>> to see and participate in both the realms of theory and of practice.
>>
>> I'm not the smartest or most knowledgeable guy on this mailing list,
>> but I think I might know enough to say that, yes, you have too strong
>> a focus on the general case.
>
> Wellm, yes. That's what I have been saying.

You're saying you're too focused? I haven't seen that except for one paragraph.

>
>>
>>>
>>> I'm more interested in the general solution that will work not only
>>> for my current machines, but also for the ones I'm planning to have in
>>> the future. I'm dying to get a tablet where I can put GNOME 3 on it; I
>>> can bet you another beer that mdev will be not enough to handle that.
>>
>> The fundamental problem, as I see it, is that udev's role currently
>> encompasses both runtime hotplug events and boot-time
>> responsibilities. Those need to be separated. How? I don't have a
>> quick or easy answer, but that's where I see the systemic problem.
>> (And, no, I won't suggest that HAL is the solution.)
>
> Again, read about devfs. Tighly coupling is the path the developers
> (in general) are taking. I agree with them.

I remember devfs. Never wound up using it, myself, but I followed the
drama via kerneltrap and diff -u.

There is no such thing as a one-size-fits-all solution, but that's not
something you seem to grasp yet.

>
>
>>
>>>
>>>>> I will not be surprised if in the future the list of programs "not for
>>>>> mdev" only grows.
>>>>
>>>> There's a difference between "needed by portage" and "doesn't work under
>>>> mdev".  As I say, it will all be moot if the evdev driver won't work
>>>> under mdev.
>>>
>>> With all due respect, Alan (and this is completely sincere, in this
>>> list you are of the guys I respect the most), I believe you are
>>> thinking too small.
>>
>> And I think you're thinking too big.
>
> Thank you.
>
>> Every case you would reach for to
>> cover with your general case, there will be a case just outside your
>> reach. At some point, you need to express limits. Limits are fine.
>> Limits can be practical. Heck, reasonable limits make practice easier,
>> because known constraints help one predict the behavior of the system.
>
> I know limits. udev has a lot of limitations; but it can do anything
> mdev can, and more. I'm not talking from a pure theoretical POV.

You're thinking about algorithmic limits. I'm talking about practical
limits like bytes, counts and CPU cycles.

>
>>>
>>> Right now Linux runs in my phone, my TV's, my routers and every
>>> computer I own. I have a couple of Windows installations, which I use
>>> once or twice every three months (I ported a PyGTK program to Windows
>>> last week, so I had to boot into Windows for the first time this
>>> year). I want Linux running on *everything*, and what is more: I don't
>>> want android in my handhelds, I want the full GNOME experience.
>>
>> BTW, you've put yourself into a niche here; you want a general-purpose
>> platform in embedded devices, but that's not the environment you've
>> argued the majory of Linux users exist in. I agree the majority of
>> Linux users are probably running Android at this point. They're not
>> running a general-purpose platform.
>
> And the devs are planning on changing that ;) That's the whole point
> of all this udev mess.

You're getting impossible to argue with. You've gone from

"most people using Linux are using Linux via Android"

to

"well, I don't want to use my embedded devices as embedded devices, I
want something more general than Android"

to

"the udev developers are working on changing embedded devices from
being embedded devices to being general-purpose devices."

Each time, you've acted as though the new stance is what you've been
arguing from all along, but because you haven't communicated that,
it's impossible to reasonably discuss specifics in practicality. I
think I'm done with this particular discussion.

-- 
:wq