Gentoo Archives: gentoo-dev

From: Max Kalika <max@g.o>
To: stuart@g.o, Max Kalika <max@g.o>, Troy Dack <tad@g.o>, gentoo-dev@g.o
Subject: Re: [gentoo-dev] [GLEP] Web Application Installation
Date: Mon, 04 Aug 2003 23:16:19
Message-Id: 86960000.1060038977@valkyrie.lsit.ucsb.edu
Quoting Stuart Herbert <stuart@g.o>:

> Here's an example. Imagine hosting (oh, I don't know) www.iammax.com and > www.maxisgreat.com on the same physical box. Document roots for each are > /var/www/<host>/public_html/ for arguments sake.
Love those names. I wonder if they're taken? ;-)
> a) A bit of mod_alias magic, and you make the /phpbb/ directories aliases > for /usr/share/webapps/phpBB-<version>/files/. If you do this, though, > how do you get each copy of phpBB to use a separate set of configuration > files? What happens when the app needs write-access to the directories? > The directories are on /usr, and we're all agreed that /usr should be > mountable read-only. And does every webserver have something like > mod_alias in the first place?
If not mod_alias, then symlink to the DocRoot, but then we're back to not knowing where the user configured his/her/virtualhosts's DocRoot.
> If I've understood your eclass correctly, this is what it tries to do, > yes?
Yes.
> b) A bit of .htaccess magic, and you have the directory structure of the > webapp on /var, but directives telling PHP where to find the .php files > by setting the include_path. The config file problem is solved, because > you can drop in local config files, and there are real directories that > can be made writable. Only problem with this approach is that not every > type of web server has the equivalent of .htaccess files - and the PHP > SAPI for each type of web server doesn't necessarily support > configuration directives in config files either. And that's before we > think about Perl, Python and other ways that webapps can be implemented.
Local config files? Where would these be stored? How are these handled for virtual hosts?
> c) A bit of find and cp -l, and you've got the directory structure of the > webapp on /var, plus links back to the original on /usr. Again, the > config file problem is solved, because local copies can be parachuted > in, and again there are real directories that can be made writable. No > webserver-specific tricks are required - as far as the web server is > concerned, each domain has its own installation of phpBB.
Ok, I'll pool some replies below...
>> Why base it on the flag? If the webserver is installed and is supported, >> configure for it. > > Because that's what USE flags are there for. If the user puts '-apache2' > in their USE flags, it's the job of the ebuild to respect that. > Otherwise, the ebuild is broken - and probably in breach of policy too.
Ok, this is debatable, but I'm willing to change the eclass to take -apache2 into effect.
>> This whole thing needs a separate discussion. There's no portage support >> for dealing with more than one installation of the same version of the >> same package. > > Why a separate discussion? If this GLEP isn't here to address this > fundamental issue, then I'd say that the GLEP has the wrong scope. > > You're right - Portage currently can't do all of this on its own. > Perhaps it never will be able to. As I understand it, that's why > Robin's volunteered to write and maintain additional tools to bridge the > gap.
Keep reading, going to answer this below. :-)
>> Keep in mind it is not completely accurate if the per-package >> USE flags goes into portage >> (http://bugs.gentoo.org/show_bug.cgi?id=13616). Things in PUSE will also >> have to be taken into account. But this is a completely different >> matter and will be easy to integrate. > > It's a nice start though. Ideally, we need a programatic interface to > portage, an API we can use to handle these types of queries. karltk is > working on one ;-)
Cool!
>> Thats just more to call from the ebuild. My goal was to have very small >> ebuilds -- just declare a few variables and inherit the eclass. > > Hmm ... there's a tradeoff here. Smaller ebuilds and less flexible (and > re-usable) eclasses. Or larger ebuilds, but more re-usable eclasses. I > guess I prefer the latter.
We can still achieve flexibility along with smaller ebuilds. If we break down the eclass install function into smaller functions and optionally call them from install, we've got ourself a nice tidy system.
>> If config >> files aren't stored in /etc/webapps/${PN}, then we need to have a way of >> generating an env.d/${PN} which contains a CONFIG_PROTECT line, otherwise >> we're forcing sysadins to reconfigure the application at every upgrade. > > Yeah, but if each instance of the installed app has its own config files, > then what's the relevance of /etc/webapps at all? > >> And as I mentioned before, there's no way for portage to handle >> multiple-copy installs, so I'm not sure the best way to go about >> achieving this goal. > > This is the problem that I think we should be solving - how to support > the installation, configuration, and upgrading of multiple-copy installs.
There's nothing I can think of that will make this work, but see below for some arguments...
>> Why should this be handled any different then the rest of apps handled by >> portage? > > Because most of the rest of the apps handled by portage don't run in a > virtual domain environment. Webapps really are a different beast.
They certainly are. :-)
> Here's an example. Imagine you're running your own hosting firm, and you > have a non-trivial number of customers using the same webapp (say, phpBB > for arguments sake). A new version of phpBB comes out. Some customers > will ask for the upgrade, and some explicitly will ask you not to > upgrade. So, in this situation, you need to have two copies of phpBB > installed on the same box at the same time. > > Now let's look at what happens when you run 'emerge -u phpBB', with the > appropriate ACCEPT_KEYWORDS of course. Portage goes and installs the new > version of phpBB over the top of the old phpBB files. The old version of > phpBB gets overwritten, yes? I don't see how AUTOCLEAN will prevent that > from happening. > > The whole point of SLOTing apps (as I understand it) is to allow you to > have multiple versions installed alongside each other. This is the > mechanism that Portage offers us.
Here's the problem. First of all, SLOTing won't help that because it assumes that the versions of the packages *and* the files you are installing are _different_. I.e. db4 and db3 are SLOTted: they install libdb-4.so and libdb-3.so respectively and thus can run simultaneously. Now, wrt to the whole virtual server thing. Personally if I was running a server with, as you say, a non-trivial number of virtual domains who have different needs, I probably wouldn't use a package management system to begin with, or truly virtualize everything and use something like user-mode-linux in which case we're back to single-domain installs. Going along with your example, how can you do an 'emerge -u phpBB' _for a particular domain_? You're telling portage to maintain a package, and upgrade parts of it randomly. I simply don't think portage will ever have any way of handling multiple installs of a package in this manner. How about we (Gentoo) just provide single domain/host installs and maybe SLOTing different versions so they won't be overwritten on upgrades. Beyhond that, we have to let the sysadmins know to cp -l the installed files into their vhosts. AFAIK, no other distro has support for virtual hosting because of this very issue.
> That's up to you. If you're maintaining webapp ebuilds currently in > portage, though, I'd urge you not to stop maintaining them just because > you're waiting for a design solution via this GLEP.
Not in charge of anything else in portage. Though I'd like to help out with Horde down the road (seeing as I'm on the horde dev team as well).
>> Right, but as we already agreed, not all apps need to have their files >> owned by apache:apache. This should be configurable in the eclass. >> Correct? > > Do we need a new user/group to own most of the files? And then we just > make the files that need write-access owned by the webserver?
The most robust solution would be to create a new group (web?) and make all the webserver users be a part of that group and make these directory group-owned by said group. (Similar to the "mail" group for many mail-related services)
> In his email in that thread you pointed me at, Robin was explicitly > talking about /var being a read-only NFS mount at install time.
Ok, install-time aside, the installed, writeable directories cannot exist on a readonly mount for day-to-day operations. Which takes me back to wanting to support a single instance install of an app that an admin has to copy in place if the default location is unacceptable.
>> So in any case, we have to pull the >> apache-specific things out into a separate framework. Therefore things >> like DocumentRoot can't even be considered. So a central location for >> webapps must be once again taken into account. > > Yep. How does this sound as the design of the central layout? Let's > agree a design, so that it can be added to the GLEP. > > * /usr/webapps/<app-name> as the main directory. > * /usr/webapps/<app-name>/public_html/ for files served by the web server > * /usr/webapps/<app-name>/cgi-bin/ for CGI-BIN files > * /etc/webapps/<app-name>/ to hold the box-default config files > * <app-name> is ${PN} for non-slotted packages > * <app-name> is ${P} for slotted packages
I'd say <app-name> should just be ${PF} for both and not worry about what is SLOTed and what is not -- it works itself out. Otherwise, this seems ok to me and is easy to implement in the eclass. The only thing I would change is the names public_html and cgi-bin. There may be no html and not binary files in them (respectively). How's about 'public' and 'cgi'? (I know, I know, nitpicking).
> a) you need a standard way of detecting which one is installed, and
Is the approach currently in the eclass unacceptable? (Check for the existance of the directory/configfile)
> b) you need to honour the 'apache2' USE flag
Yup, see above...no higher...no, higher than that. Ok there, stop! --mk -- gentoo-dev@g.o mailing list

Replies

Subject Author
Re: [gentoo-dev] [GLEP] Web Application Installation Stuart Herbert <stuart@g.o>