[gentoo-commits] portage r11694 - in main/branches/prefix: bin pym/_emerge pym/portage pym/portage/dbapi pym/portage/elog

[Fabian Groffen (grobian)]

Author: grobian
Date: 2008-10-15 20:33:21 +0000 (Wed, 15 Oct 2008)
New Revision: 11694

Modified:
   main/branches/prefix/bin/ebuild.sh
   main/branches/prefix/bin/repoman
   main/branches/prefix/pym/_emerge/__init__.py
   main/branches/prefix/pym/portage/__init__.py
   main/branches/prefix/pym/portage/dbapi/bintree.py
   main/branches/prefix/pym/portage/dbapi/porttree.py
   main/branches/prefix/pym/portage/dbapi/vartree.py
   main/branches/prefix/pym/portage/elog/__init__.py
Log:
   Merged from trunk -r11675:11692

   | 11676   | Bug #241204 - Fix dbapi.invalidentry() calls inside          |
   | zmedico | vardbapi.cpv_all() to correctly join the path via            |
   |         | self.getpath().                                              |
   
   | 11677   | When warning about package categories that haven't been      |
   | zmedico | configured via /etc/portage/categories, join the path with   |
   |         | PORTAGE_CONFIGROOT. Thanks to Ned Ludd <solar@g.o> for       |
   |         | reporting.                                                   |
   
   | 11678   | In EbuildFetcher._start(), in order to ensure that a new log |
   | zmedico | is created when appropriate, clean up PORTAGE_BUILDDIR if    |
   |         | appropriate.                                                 |
   
   | 11679   | Make the Scheduler class treat failed fetches the same as    |
   | zmedico | other failed packages, and don't bail out due to a failure   |
   |         | when in --fetchonly mode. This fixes an issue with           |
   |         | --fetchonly bailing out instead of continuing to fetch.      |
   
   | 11680   | Buge #241100 - Make EbuildFetcher use a pty when             |
   | zmedico | appropriate, so that fetcher progress bars, like wget has,   |
   |         | will work properly.                                          |
   
   | 11681   | Remove the prefetch logic in EbuildFetcher._pipe() since     |
   | zmedico | prefetchers are always in the background anyway.             |
   
   | 11682   | Now that only prefetch (parallel-fetch) output goes to the   |
   | zmedico | global fetch log, update code to remove inappropriate        |
   |         | references to this log.                                      |
   
   | 11683   | Inside fetch(), don't call pkg_nofetch when in               |
   | zmedico | parallel-fetch mode since it will get call again later       |
   |         | anyway.                                                      |
   
   | 11684   | Add an explicit note about bug #239560 in the relevant code. |
   | zmedico |                                                              |
   
   | 11685   | Bug #241118 - Re-enable pkg_nofetch for emerge --pretend     |
   | zmedico | --fetch, but direct output to stderr. This requires          |
   |         | PORTAGE_BUILDDIR locking for bug #239560 and also for elog   |
   |         | message storage.                                             |
   
   | 11686   | Bug #234301 - When fetch fails, include the path of the log  |
   | zmedico | file in the eerror message that's generated. This way it's   |
   |         | easy for the user to find the output from the fetcher or     |
   |         | from the pkg_nofetch phase.                                  |
   
   | 11687   | Don't return from fetch() due to fetch restriction when in   |
   | zmedico | "listonly" mode.                                             |
   
   | 11688   | Make fetch() fetch as many files as possible, even if some   |
   | zmedico | fail, when in fetchonly mode.                                |
   
   | 11689   | Make sure the pkg_nofetch phase is only called once when in  |
   | zmedico | fetchonly mode.                                              |
   
   | 11690   | When emerge --keep-going bails due to missing dependencies,  |
   | zmedico | show the error message after the mod_echo output since       |
   |         | otherwise the mod_echo output can sweep the error message    |
   |         | out of view. Thanks to Donnie Berkholz <dberkholz@g.o> for   |
   |         | reporting.                                                   |
   
   | 11691   | Bug #241366 - Only suggest to run `emaint --check world`     |
   | zmedico | when the atom is a direct member of the world set (rather    |
   |         | than from a nested set).                                     |
   
   | 11692   | Bug #229033 - When a Manifest is added automatically by      |
   | zmedico | repoman, it's in the "mynew" list, so move manifest paths    |
   |         | from that list to the "mymanifests" list when necessary.     |


Modified: main/branches/prefix/bin/ebuild.sh
===================================================================
--- main/branches/prefix/bin/ebuild.sh	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/bin/ebuild.sh	2008-10-15 20:33:21 UTC (rev 11694)
@@ -272,7 +272,9 @@
 	export EBUILD_DEATH_HOOKS="${EBUILD_DEATH_HOOKS} $*"
 }
 
-# Ensure that $PWD is sane whenever possible.
+# Ensure that $PWD is sane whenever possible, to protect against
+# exploitation of insecure search path for python -c in ebuilds.
+# See bug #239560.
 if ! hasq "$EBUILD_PHASE" clean depend help ; then
 	cd "$PORTAGE_BUILDDIR" || \
 		die "PORTAGE_BUILDDIR does not exist: '$PORTAGE_BUILDDIR'"

Modified: main/branches/prefix/bin/repoman
===================================================================
--- main/branches/prefix/bin/repoman	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/bin/repoman	2008-10-15 20:33:21 UTC (rev 11694)
@@ -1738,8 +1738,23 @@
 
 	# Manifests need to be regenerated after all other commits, so don't commit
 	# them now even if they have changed.
-	mymanifests = [f for f in mychanged if "Manifest" == os.path.basename(f)]
-	mychanged = [f for f in mychanged if "Manifest" != os.path.basename(f)]
+	mymanifests = set()
+	changed_set = set()
+	new_set = set()
+	for f in mychanged:
+		if "Manifest" == os.path.basename(f):
+			mymanifests.add(f)
+		else:
+			changed_set.add(f)
+	for f in mynew:
+		if "Manifest" == os.path.basename(f):
+			mymanifests.add(f)
+		else:
+			new_set.add(f)
+	mychanged = list(changed_set)
+	mynew =  list(new_set)
+	mymanifests = list(mymanifests)
+	del changed_set, new_set
 	myupdates = mychanged + mynew
 	myheaders = []
 	mydirty = []

Modified: main/branches/prefix/pym/_emerge/__init__.py
===================================================================
--- main/branches/prefix/pym/_emerge/__init__.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/_emerge/__init__.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -1615,6 +1615,13 @@
 	__slots__ = ("fetch_all", "pkg", "settings")
 
 	def execute(self):
+		# To spawn pkg_nofetch requires PORTAGE_BUILDDIR for
+		# ensuring sane $PWD (bug #239560) and storing elog
+		# messages.
+		build_dir = EbuildBuildDir(pkg=self.pkg, settings=self.settings)
+		build_dir.lock()
+		build_dir.clean()
+		portage.prepare_build_dirs(self.pkg.root, self.settings, 0)
 		portdb = self.pkg.root_config.trees["porttree"].dbapi
 		ebuild_path = portdb.findname(self.pkg.cpv)
 		debug = self.settings.get("PORTAGE_DEBUG") == "1"
@@ -1623,6 +1630,10 @@
 			self.settings["ROOT"], self.settings, debug=debug,
 			listonly=1, fetchonly=1, fetchall=self.fetch_all,
 			mydbapi=portdb, tree="porttree")
+
+		portage.elog.elog_process(self.pkg.cpv, self.settings)
+		build_dir.clean()
+		build_dir.unlock()
 		return retval
 
 class AsynchronousTask(SlotObject):
@@ -2275,6 +2286,8 @@
 		settings = self.config_pool.allocate()
 		self._build_dir = EbuildBuildDir(pkg=self.pkg, settings=settings)
 		self._build_dir.lock()
+		self._build_dir.clean()
+		portage.prepare_build_dirs(self.pkg.root, self._build_dir.settings, 0)
 		if self.logfile is None:
 			self.logfile = settings.get("PORTAGE_LOG_FILE")
 
@@ -2304,6 +2317,18 @@
 		self.env = fetch_env
 		SpawnProcess._start(self)
 
+	def _pipe(self, fd_pipes):
+		"""When appropriate, use a pty so that fetcher progress bars,
+		like wget has, will work properly."""
+		if self.background or not sys.stdout.isatty():
+			# When the output only goes to a log file,
+			# there's no point in creating a pty.
+			return os.pipe()
+		stdout_pipe = fd_pipes.get(1)
+		got_pty, master_fd, slave_fd = \
+			portage._create_pty_or_pipe(copy_term_size=stdout_pipe)
+		return (master_fd, slave_fd)
+
 	def _set_returncode(self, wait_retval):
 		SpawnProcess._set_returncode(self, wait_retval)
 		# Collect elog messages that might have been
@@ -2315,21 +2340,20 @@
 				if self.logfile is not None:
 					if self.background:
 						elog_out = open(self.logfile, 'a')
-				eerror("Fetch failed for '%s'" % self.pkg.cpv,
-					phase="unpack", key=self.pkg.cpv, out=elog_out)
+				msg = "Fetch failed for '%s'" % (self.pkg.cpv,)
+				if self.logfile is not None:
+					msg += ", Log file:"
+				eerror(msg, phase="unpack", key=self.pkg.cpv, out=elog_out)
+				if self.logfile is not None:
+					eerror(" '%s'" % (self.logfile,),
+						phase="unpack", key=self.pkg.cpv, out=elog_out)
 				if elog_out is not None:
 					elog_out.close()
 			if not self.prefetch:
 				portage.elog.elog_process(self.pkg.cpv, self._build_dir.settings)
 			features = self._build_dir.settings.features
-			if (self.fetchonly or self.returncode == os.EX_OK) and \
-				not ("keepwork" in features or "keeptemp" in features):
-				try:
-					shutil.rmtree(self._build_dir.settings["PORTAGE_BUILDDIR"])
-				except EnvironmentError, e:
-					if e.errno != errno.ENOENT:
-						raise
-					del e
+			if self.returncode == os.EX_OK:
+				self._build_dir.clean()
 			self._build_dir.unlock()
 			self.config_pool.deallocate(self._build_dir.settings)
 			self._build_dir = None
@@ -2365,7 +2389,6 @@
 			portage.doebuild_environment(ebuild_path, "setup", root_config.root,
 				self.settings, debug, use_cache, portdb)
 			dir_path = self.settings["PORTAGE_BUILDDIR"]
-			portage.prepare_build_dirs(self.pkg.root, self.settings, 0)
 
 		catdir = os.path.dirname(dir_path)
 		self._catdir = catdir
@@ -2385,6 +2408,19 @@
 			if catdir_lock is not None:
 				portage.locks.unlockdir(catdir_lock)
 
+	def clean(self):
+		"""Uses shutil.rmtree() rather than spawning a 'clean' phase. Disabled
+		by keepwork or keeptemp in FEATURES."""
+		settings = self.settings
+		features = settings.features
+		if not ("keepwork" in features or "keeptemp" in features):
+			try:
+				shutil.rmtree(settings["PORTAGE_BUILDDIR"])
+			except EnvironmentError, e:
+				if e.errno != errno.ENOENT:
+					raise
+				del e
+
 	def unlock(self):
 		if self._lock_obj is None:
 			return
@@ -3504,7 +3540,7 @@
 
 	__slots__ = ("args_set",
 		"binpkg_opts", "build_opts", "config_pool", "emerge_opts",
-		"failed_fetches", "find_blockers", "logger", "mtimedb", "pkg",
+		"find_blockers", "logger", "mtimedb", "pkg",
 		"pkg_count", "pkg_to_replace", "prefetcher",
 		"settings", "statusMessage", "world_atom") + \
 		("_install_task",)
@@ -3564,7 +3600,7 @@
 				settings=settings, world_atom=world_atom)
 
 			self._install_task = build
-			self._start_task(build, self._ebuild_exit)
+			self._start_task(build, self._default_final_exit)
 			return
 
 		elif pkg.type_name == "binary":
@@ -3580,12 +3616,6 @@
 			self._start_task(binpkg, self._default_final_exit)
 			return
 
-	def _ebuild_exit(self, build):
-		if self._final_exit(build) != os.EX_OK:
-			if self.build_opts.fetchonly:
-				self.failed_fetches.append(self.pkg.cpv)
-		self.wait()
-
 	def _poll(self):
 		self._install_task.poll()
 		return self.returncode
@@ -3598,7 +3628,6 @@
 
 		pkg = self.pkg
 		build_opts = self.build_opts
-		failed_fetches = self.failed_fetches
 		find_blockers = self.find_blockers
 		logger = self.logger
 		mtimedb = self.mtimedb
@@ -7567,8 +7596,11 @@
 		if self._missing_args:
 			world_problems = False
 			if "world" in self._sets:
+				# Filter out indirect members of world (from nested sets)
+				# since only direct members of world are desired here.
+				world_set = self.roots[self.target_root].sets["world"]
 				for arg, atom in self._missing_args:
-					if arg.name == "world":
+					if arg.name == "world" and atom in world_set:
 						world_problems = True
 						break
 
@@ -9022,8 +9054,7 @@
 			emergelog(self.xterm_titles, *pargs, **kwargs)
 
 	class _failed_pkg(SlotObject):
-		__slots__ = ("build_dir", "build_log",
-			"fetch_log", "pkg", "returncode")
+		__slots__ = ("build_dir", "build_log", "pkg", "returncode")
 
 	class _ConfigPool(object):
 		"""Interface for a task to temporarily allocate a config
@@ -9116,7 +9147,7 @@
 		self._failed_pkgs = []
 		self._failed_pkgs_all = []
 		self._failed_pkgs_die_msgs = []
-		self._failed_fetches = []
+		self._post_mod_echo_msgs = []
 		self._parallel_fetch = False
 		merge_count = len([x for x in mergelist \
 			if isinstance(x, Package) and x.operation == "merge"])
@@ -9156,7 +9187,7 @@
 			elif len(mergelist) > 1:
 				self._parallel_fetch = True
 
-		if background or self._parallel_fetch:
+		if self._parallel_fetch:
 				# clear out existing fetch log if it exists
 				try:
 					open(self._fetch_log, 'w')
@@ -9475,34 +9506,6 @@
 
 		return prefetcher
 
-	def _show_failed_fetches(self):
-		failed_fetches = self._failed_fetches
-		if not failed_fetches or not \
-			("--fetchonly" in self.myopts or \
-			"--fetch-all-uri" in self.myopts):
-			return
-
-		if self._background:
-			msg = "Some fetch errors were " + \
-				"encountered. Please see %s for details." % \
-				self._fetch_log
-		else:
-			msg = "Some fetch errors were " + \
-				"encountered. Please see above for details."
-
-		prefix = bad(" * ")
-		msg = "".join("%s%s\n" % (prefix, line) \
-			for line in textwrap.wrap(msg, 70))
-		writemsg_level(msg, level=logging.ERROR, noiselevel=-1)
-
-		msg = []
-		msg.append("")
-		for cpv in failed_fetches:
-			msg.append("  %s" % cpv)
-		msg.append("")
-		writemsg_level("".join("%s%s\n" % (prefix, line) \
-			for line in msg), level=logging.ERROR, noiselevel=-1)
-
 	def _is_restart_scheduled(self):
 		"""
 		Check if the merge list contains a replacement
@@ -9602,15 +9605,13 @@
 			return rval
 
 		keep_going = "--keep-going" in self.myopts
+		fetchonly = self._build_opts.fetchonly
 		mtimedb = self._mtimedb
 		failed_pkgs = self._failed_pkgs
 
 		while True:
 			rval = self._merge()
-			self._show_failed_fetches()
-			del self._failed_fetches[:]
-
-			if rval == os.EX_OK or not keep_going:
+			if rval == os.EX_OK or fetchonly or not keep_going:
 				break
 			if "resume" not in mtimedb:
 				break
@@ -9630,29 +9631,13 @@
 			if not mergelist:
 				break
 
-			dropped_tasks = self._calc_resume_list()
-			if dropped_tasks is None:
+			if not self._calc_resume_list():
 				break
 
 			clear_caches(self.trees)
 			if not self._mergelist:
 				break
 
-			if dropped_tasks:
-
-				def _eerror(lines):
-					for l in lines:
-						eerror(l, phase="other", key=failed_pkg.pkg.cpv)
-
-				msg = []
-				msg.append("One or more packages have been " + \
-					"dropped due to unsatisfied dependencies:")
-				msg.append("")
-				msg.extend("  " + str(task) for task in dropped_tasks)
-				msg.append("")
-				_eerror(msg)
-				del _eerror, msg
-			del dropped_tasks
 			self._save_resume_list()
 			self._pkg_count.curval = 0
 			self._pkg_count.maxval = len([x for x in self._mergelist \
@@ -9691,10 +9676,15 @@
 					log_file.close()
 				failure_log_shown = True
 
+		# Dump mod_echo output now since it tends to flood the terminal.
+		# This allows us to avoid having more important output, generated
+		# later, from being swept away by the mod_echo output.
+		mod_echo_output =  _flush_elog_mod_echo()
+
 		if background and not failure_log_shown and \
 			self._failed_pkgs_all and \
 			self._failed_pkgs_die_msgs and \
-			not _flush_elog_mod_echo():
+			not mod_echo_output:
 
 			printer = portage.output.EOutput()
 			for mysettings, key, logentries in self._failed_pkgs_die_msgs:
@@ -9714,8 +9704,11 @@
 						for line in msgcontent:
 							printer.eerror(line.strip("\n"))
 
+		if self._post_mod_echo_msgs:
+			for msg in self._post_mod_echo_msgs:
+				msg()
+
 		if len(self._failed_pkgs_all) > 1:
-			_flush_elog_mod_echo()
 			msg = "The following packages have " + \
 				"failed to build or install:"
 			prefix = bad(" * ")
@@ -9745,9 +9738,6 @@
 
 		log_paths = [failed_pkg.build_log]
 
-		if not (build_dir and os.path.isdir(build_dir)):
-			log_paths.append(failed_pkg.fetch_log)
-
 		for log_path in log_paths:
 			if not log_path:
 				continue
@@ -9787,11 +9777,10 @@
 			settings = merge.merge.settings
 			build_dir = settings.get("PORTAGE_BUILDDIR")
 			build_log = settings.get("PORTAGE_LOG_FILE")
-			fetch_log = self._fetch_log
 
 			self._failed_pkgs.append(self._failed_pkg(
 				build_dir=build_dir, build_log=build_log,
-				fetch_log=fetch_log, pkg=pkg,
+				pkg=pkg,
 				returncode=merge.returncode))
 			self._failed_pkg_msg(self._failed_pkgs[-1], "install", "to")
 
@@ -9831,12 +9820,11 @@
 		else:
 			settings = build.settings
 			build_dir = settings.get("PORTAGE_BUILDDIR")
-			fetch_log = self._fetch_log
 			build_log = settings.get("PORTAGE_LOG_FILE")
 
 			self._failed_pkgs.append(self._failed_pkg(
 				build_dir=build_dir, build_log=build_log,
-				fetch_log=fetch_log, pkg=build.pkg,
+				pkg=build.pkg,
 				returncode=build.returncode))
 			self._failed_pkg_msg(self._failed_pkgs[-1], "emerge", "for")
 
@@ -9998,6 +9986,10 @@
 			if self._poll_event_handlers:
 				self._poll_loop()
 
+	def _keep_scheduling(self):
+		return bool(self._pkg_queue and \
+			not (self._failed_pkgs and not self._build_opts.fetchonly))
+
 	def _schedule_tasks(self):
 		self._schedule_tasks_imp()
 		self._status_display.display()
@@ -10009,7 +10001,7 @@
 
 		# Cancel prefetchers if they're the only reason
 		# the main poll loop is still running.
-		if self._failed_pkgs and \
+		if self._failed_pkgs and not self._build_opts.fetchonly and \
 			not (self._jobs or self._task_queues.merge) and \
 			self._task_queues.fetch:
 			self._task_queues.fetch.clear()
@@ -10019,7 +10011,7 @@
 			self._schedule_tasks_imp()
 			self._status_display.display()
 
-		return bool(self._pkg_queue and not self._failed_pkgs)
+		return self._keep_scheduling()
 
 	def _job_delay(self):
 		"""
@@ -10049,7 +10041,7 @@
 
 		while True:
 
-			if not self._pkg_queue or self._failed_pkgs:
+			if not self._keep_scheduling():
 				return bool(state_change)
 
 			if self._choose_pkg_return_early or \
@@ -10106,7 +10098,6 @@
 			config_pool=self._ConfigPool(pkg.root,
 			self._allocate_config, self._deallocate_config),
 			emerge_opts=self.myopts,
-			failed_fetches=self._failed_fetches,
 			find_blockers=self._find_blockers(pkg), logger=self._logger,
 			mtimedb=self._mtimedb, pkg=pkg, pkg_count=self._pkg_count.copy(),
 			pkg_to_replace=pkg_to_replace,
@@ -10165,9 +10156,8 @@
 		"""
 		Use the current resume list to calculate a new one,
 		dropping any packages with unsatisfied deps.
-		@rtype: set
-		@returns: a possibly empty set of dropped tasks, or
-			None if an error occurs.
+		@rtype: bool
+		@returns: True if successful, False otherwise.
 		"""
 		print colorize("GOOD", "*** Resuming merge...")
 
@@ -10203,42 +10193,50 @@
 			print "\b\b... done!"
 
 		if e is not None:
-			mydepgraph.display_problems()
-			out = portage.output.EOutput()
-			out.eerror("One or more packages are either masked or " + \
-				"have missing dependencies:")
-			out.eerror("")
-			indent = "  "
-			for dep in e.value:
-				if dep.atom is None:
-					out.eerror(indent + "Masked package:")
-					out.eerror(2 * indent + str(dep.parent))
-					out.eerror("")
-				else:
-					out.eerror(indent + str(dep.atom) + " pulled in by:")
-					out.eerror(2 * indent + str(dep.parent))
-					out.eerror("")
-			msg = "The resume list contains packages " + \
-				"that are either masked or have " + \
-				"unsatisfied dependencies. " + \
-				"Please restart/continue " + \
-				"the operation manually, or use --skipfirst " + \
-				"to skip the first package in the list and " + \
-				"any other packages that may be " + \
-				"masked or have missing dependencies."
-			for line in textwrap.wrap(msg, 72):
-				out.eerror(line)
-			return None
+			def unsatisfied_resume_dep_msg():
+				mydepgraph.display_problems()
+				out = portage.output.EOutput()
+				out.eerror("One or more packages are either masked or " + \
+					"have missing dependencies:")
+				out.eerror("")
+				indent = "  "
+				show_parents = set()
+				for dep in e.value:
+					if dep.parent in show_parents:
+						continue
+					show_parents.add(dep.parent)
+					if dep.atom is None:
+						out.eerror(indent + "Masked package:")
+						out.eerror(2 * indent + str(dep.parent))
+						out.eerror("")
+					else:
+						out.eerror(indent + str(dep.atom) + " pulled in by:")
+						out.eerror(2 * indent + str(dep.parent))
+						out.eerror("")
+				msg = "The resume list contains packages " + \
+					"that are either masked or have " + \
+					"unsatisfied dependencies. " + \
+					"Please restart/continue " + \
+					"the operation manually, or use --skipfirst " + \
+					"to skip the first package in the list and " + \
+					"any other packages that may be " + \
+					"masked or have missing dependencies."
+				for line in textwrap.wrap(msg, 72):
+					out.eerror(line)
+			self._post_mod_echo_msgs.append(unsatisfied_resume_dep_msg)
+			return False
 
-		if self._show_list():
+		if success and self._show_list():
 			mylist = mydepgraph.altlist()
-			if "--tree" in self.myopts:
-				mylist.reverse()
-			mydepgraph.display(mylist, favorites=self._favorites)
+			if mylist:
+				if "--tree" in self.myopts:
+					mylist.reverse()
+				mydepgraph.display(mylist, favorites=self._favorites)
 
+		if not success:
+			self._post_mod_echo_msgs.append(mydepgraph.display_problems)
+			return False
 		mydepgraph.display_problems()
-		if not success:
-			return None
 
 		mylist = mydepgraph.altlist()
 		mydepgraph.break_refs(mylist)
@@ -10247,8 +10245,28 @@
 
 		self._mergelist = mylist
 		self._set_digraph(mydepgraph.digraph)
-		return dropped_tasks
 
+		msg_width = 75
+		for task in dropped_tasks:
+			if not (isinstance(task, Package) and task.operation == "merge"):
+				continue
+			pkg = task
+			msg = "emerge --keep-going:" + \
+				" %s" % (pkg.cpv,)
+			if pkg.root != "/":
+				msg += " for %s" % (pkg.root,)
+			msg += " dropped due to unsatisfied dependency."
+			for line in textwrap.wrap(msg, msg_width):
+				eerror(line, phase="other", key=pkg.cpv)
+			settings = mydepgraph.pkgsettings[pkg.root]
+			# Ensure that log collection from $T is disabled inside
+			# elog_process(), since any logs that might exist are
+			# not valid here.
+			settings.pop("T", None)
+			portage.elog.elog_process(pkg.cpv, settings)
+
+		return True
+
 	def _show_list(self):
 		myopts = self.myopts
 		if "--quiet" not in myopts and \

Modified: main/branches/prefix/pym/portage/__init__.py
===================================================================
--- main/branches/prefix/pym/portage/__init__.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/portage/__init__.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -3643,6 +3643,7 @@
 			del distlocks_subdir
 
 	distdir_writable = can_fetch and not fetch_to_ro
+	failed_files = set()
 
 	for myfile in filedict:
 		"""
@@ -4080,31 +4081,57 @@
 
 		if listonly:
 			writemsg_stdout("\n", noiselevel=-1)
-		elif fetched != 2:
+		if fetched != 2:
 			if restrict_fetch:
-				print "\n!!!", mysettings["CATEGORY"] + "/" + \
-					mysettings["PF"], "has fetch restriction turned on."
-				print "!!! This probably means that this " + \
-					"ebuild's files must be downloaded"
-				print "!!! manually.  See the comments in" + \
-					" the ebuild for more information.\n"
-				ebuild_phase = mysettings.get("EBUILD_PHASE")
-				try:
-					mysettings["EBUILD_PHASE"] = "nofetch"
-					spawn(_shell_quote(EBUILD_SH_BINARY) + \
-						" nofetch", mysettings)
-				finally:
-					if ebuild_phase is None:
-						mysettings.pop("EBUILD_PHASE", None)
-					else:
-						mysettings["EBUILD_PHASE"] = ebuild_phase
+				msg = ("\n!!! %s/%s" + \
+					" has fetch restriction turned on.\n" + \
+					"!!! This probably means that this " + \
+					"ebuild's files must be downloaded\n" + \
+					"!!! manually.  See the comments in" + \
+					" the ebuild for more information.\n\n") % \
+					(mysettings["CATEGORY"], mysettings["PF"])
+				portage.util.writemsg_level(msg,
+					level=logging.ERROR, noiselevel=-1)
+				if not parallel_fetchonly:
+					# To spawn pkg_nofetch requires PORTAGE_BUILDDIR for
+					# ensuring sane $PWD (bug #239560) and storing elog
+					# messages. Therefore, calling code needs to ensure that
+					# PORTAGE_BUILDDIR is already clean and locked here.
+
+					# All the pkg_nofetch goes to stderr since it's considered
+					# to be an error message.
+					fd_pipes = {
+						0 : sys.stdin.fileno(),
+						1 : sys.stderr.fileno(),
+						2 : sys.stderr.fileno(),
+					}
+
+					ebuild_phase = mysettings.get("EBUILD_PHASE")
+					try:
+						mysettings["EBUILD_PHASE"] = "nofetch"
+						spawn(_shell_quote(EBUILD_SH_BINARY) + \
+							" nofetch", mysettings, fd_pipes=fd_pipes)
+					finally:
+						if ebuild_phase is None:
+							mysettings.pop("EBUILD_PHASE", None)
+						else:
+							mysettings["EBUILD_PHASE"] = ebuild_phase
+				if listonly:
+					continue
+			elif listonly:
+				continue
 			elif not filedict[myfile]:
 				writemsg("Warning: No mirrors available for file" + \
 					" '%s'\n" % (myfile), noiselevel=-1)
 			else:
 				writemsg("!!! Couldn't download '%s'. Aborting.\n" % myfile,
 					noiselevel=-1)
+			if fetchonly and not restrict_fetch:
+				failed_files.add(myfile)
+				continue
 			return 0
+	if failed_files:
+		return 0
 	return 1
 
 def digestgen(myarchives, mysettings, overwrite=1, manifestonly=0, myportdb=None):
@@ -5479,8 +5506,7 @@
 
 		# Build directory creation isn't required for any of these.
 		have_build_dirs = False
-		if not (mydo in ("digest", "help", "manifest") or \
-			(mydo == "fetch" and listonly)):
+		if not mydo in ("digest", "help", "manifest"):
 			mystatus = prepare_build_dirs(myroot, mysettings, cleanup)
 			if mystatus:
 				return mystatus

Modified: main/branches/prefix/pym/portage/dbapi/bintree.py
===================================================================
--- main/branches/prefix/pym/portage/dbapi/bintree.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/portage/dbapi/bintree.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -555,7 +555,8 @@
 							"unrecognized category: '%s'\n") % full_path,
 							noiselevel=-1)
 						writemsg(("!!! '%s' has a category that is not" + \
-							" listed in /etc/portage/categories\n") % mycpv,
+							" listed in %setc/portage/categories\n") % \
+							(mycpv, self.settings["PORTAGE_CONFIGROOT"]),
 							noiselevel=-1)
 						continue
 					pkg_paths[mycpv] = mypath
@@ -723,7 +724,8 @@
 						"unrecognized category: '%s'\n") % fullpkg,
 						noiselevel=-1)
 					writemsg(("!!! '%s' has a category that is not" + \
-						" listed in /etc/portage/categories\n") % fullpkg,
+						" listed in %setc/portage/categories\n") % \
+						(fullpkg, self.settings["PORTAGE_CONFIGROOT"]),
 						noiselevel=-1)
 					continue
 				mykey = dep_getkey(fullpkg)

Modified: main/branches/prefix/pym/portage/dbapi/porttree.py
===================================================================
--- main/branches/prefix/pym/portage/dbapi/porttree.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/portage/dbapi/porttree.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -728,7 +728,8 @@
 					d[mysplit[0]+"/"+pf] = None
 		if invalid_category and d:
 			writemsg(("\n!!! '%s' has a category that is not listed in " + \
-				"/etc/portage/categories\n") % mycp, noiselevel=-1)
+				"%setc/portage/categories\n") % \
+				(mycp, self.mysettings["PORTAGE_CONFIGROOT"]), noiselevel=-1)
 			mylist = []
 		else:
 			mylist = d.keys()

Modified: main/branches/prefix/pym/portage/dbapi/vartree.py
===================================================================
--- main/branches/prefix/pym/portage/dbapi/vartree.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/portage/dbapi/vartree.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -1295,10 +1295,10 @@
 				# -MERGING- should never be a cpv, nor should files.
 				try:
 					if catpkgsplit(subpath) is None:
-						self.invalidentry(os.path.join(self.root, subpath))
+						self.invalidentry(self.getpath(subpath))
 						continue
 				except InvalidData:
-					self.invalidentry(os.path.join(self.root, subpath))
+					self.invalidentry(self.getpath(subpath))
 					continue
 				returnme.append(subpath)
 		return returnme

Modified: main/branches/prefix/pym/portage/elog/__init__.py
===================================================================
--- main/branches/prefix/pym/portage/elog/__init__.py	2008-10-15 19:39:42 UTC (rev 11693)
+++ main/branches/prefix/pym/portage/elog/__init__.py	2008-10-15 20:33:21 UTC (rev 11694)
@@ -72,7 +72,13 @@
 		except ImportError:
 			pass
 
-	ebuild_logentries = collect_ebuild_messages(os.path.join(mysettings["T"], "logging"))
+	if "T" in mysettings:
+		ebuild_logentries = collect_ebuild_messages(
+			os.path.join(mysettings["T"], "logging"))
+	else:
+		# A build dir isn't necessarily required since the messages.e*
+		# functions allow messages to be generated in-memory.
+		ebuild_logentries = {}
 	all_logentries = collect_messages()
 	if cpv in all_logentries:
 		all_logentries[cpv] = _merge_logentries(ebuild_logentries, all_logentries[cpv])