[gentoo-portage-dev] plugin-sync progress report

[Brian Dolbec]

I've started working on the repository/config.py changes needed for the
plugin-sync system.

First:
  The following class performed checks on the
repos.conf entries for the sync variables regardless if they were being
used or not.  I would like to remove that code block completely from
this loader/parser.  I would prefer to have the sync module perform
it's own checks and only when syncing or from a future emaint module
with that specific 'check-config' option.

I've sectioned off the code I'd like to remove below. 
Is there any reason they must be done here and during initial parsing?

Advantages:
  - No need to import and initialize the sync module and the plugin system 
    for other emerge/portage operations.
  - Should speed things up slightly with less to do.
  - Makes the sync module responsible the validation of any sync specific variables.


Second:
  - Should all the repos.conf entires to be synced be validated prior to syncing and fail
    if there are any errors?
  - Or, just call each sync module's sync() and let each fail individually 
    and continue syncing remaining repos?

Third:
  - I would like to add a new config item for all repos.
    It is a variable to define which sources of sync operations which
    the repo will be allowed to be synced from.  It could be a space separated list
    which could be used to prevent it from being synced via certain commands.

    i.e:

    [repo1] 
    sync-from = emerge emaint

    [repo2]
    sync-from = emerge emaint layman

    [repo3]
    sync-from = None

  - Thoughts?  
  - Opinions?

############

class RepoConfigLoader(object):
	"""Loads and store config of several repositories, loaded from PORTDIR_OVERLAY or repos.conf"""

	@staticmethod
	def _parse(paths, prepos, ignored_map, ignored_location_map, local_config, portdir):
		"""Parse files in paths to load config"""
		parser = SafeConfigParser()

		# use read_file/readfp in order to control decoding of unicode
		try:
			# Python >=3.2
			read_file = parser.read_file
			source_kwarg = 'source'
		except AttributeError:
			read_file = parser.readfp
			source_kwarg = 'filename'

		recursive_paths = []
		for p in paths:
			if isinstance(p, basestring):
				recursive_paths.extend(_recursive_file_list(p))
			else:
				recursive_paths.append(p)

		for p in recursive_paths:
			if isinstance(p, basestring):
				f = None
				try:
					f = io.open(_unicode_encode(p,
						encoding=_encodings['fs'], errors='strict'),
						mode='r', encoding=_encodings['repo.content'],
						errors='replace')
				except EnvironmentError:
					pass
				else:
					# The 'source' keyword argument is needed since otherwise
					# ConfigParser in Python <3.3.3 may throw a TypeError
					# because it assumes that f.name is a native string rather
					# than binary when constructing error messages.
					kwargs = {source_kwarg: p}
					read_file(f, **portage._native_kwargs(kwargs))
				finally:
					if f is not None:
						f.close()
			elif isinstance(p, io.StringIO):
				kwargs = {source_kwarg: "<io.StringIO>"}
				read_file(p, **portage._native_kwargs(kwargs))
			else:
				raise TypeError("Unsupported type %r of element %r of 'paths' argument" % (type(p), p))

		prepos['DEFAULT'] = RepoConfig("DEFAULT",
			parser.defaults(), local_config=local_config)

		for sname in parser.sections():
			optdict = {}
			for oname in parser.options(sname):
				optdict[oname] = parser.get(sname, oname)

			repo = RepoConfig(sname, optdict, local_config=local_config)

###################################
# remove these checks
###################################
			if repo.sync_type is not None and repo.sync_uri is None:
				writemsg_level("!!! %s\n" % _("Repository '%s' has sync-type attribute, but is missing sync-uri attribute") %
					sname, level=logging.ERROR, noiselevel=-1)
				continue

			if repo.sync_uri is not None and repo.sync_type is None:
				writemsg_level("!!! %s\n" % _("Repository '%s' has sync-uri attribute, but is missing sync-type attribute") %
					sname, level=logging.ERROR, noiselevel=-1)
				continue

			if repo.sync_type not in portage.sync.module_names + [None]:
				writemsg_level("!!! %s\n" % _("Repository '%s' has sync-type attribute set to unsupported value: '%s'") %
					(sname, repo.sync_type), level=logging.ERROR, noiselevel=-1)
				continue

			if repo.sync_type == "cvs" and repo.sync_cvs_repo is None:
				writemsg_level("!!! %s\n" % _("Repository '%s' has sync-type=cvs, but is missing sync-cvs-repo attribute") %
					sname, level=logging.ERROR, noiselevel=-1)
				continue
###################################
# end remove
###################################

			# For backward compatibility with locations set via PORTDIR and
			# PORTDIR_OVERLAY, delay validation of the location and repo.name
			# until after PORTDIR and PORTDIR_OVERLAY have been processed.
			prepos[sname] = repo


-- 
Brian Dolbec <dolsen>