subrepos: support adding files in git subrepos

This support includes correct matching, so includes,
excludes and patterns are all supported.

mercurial/help/subrepos.txt

@@ -1,160 +1,160 b''
 Subrepositories let you nest external repositories or projects into a
 parent Mercurial repository, and make commands operate on them as a
 group.
 
 Mercurial currently supports Mercurial, Git, and Subversion
 subrepositories.
 
 Subrepositories are made of three components:
 
 1. Nested repository checkouts. They can appear anywhere in the
    parent working directory.
 
 2. Nested repository references. They are defined in ``.hgsub``, which
    should be placed in the root of working directory, and
    tell where the subrepository checkouts come from. Mercurial
    subrepositories are referenced like::
 
      path/to/nested = https://example.com/nested/repo/path
 
    Git and Subversion subrepos are also supported::
 
      path/to/nested = [git]git://example.com/nested/repo/path
      path/to/nested = [svn]https://example.com/nested/trunk/path
 
    where ``path/to/nested`` is the checkout location relatively to the
    parent Mercurial root, and ``https://example.com/nested/repo/path``
    is the source repository path. The source can also reference a
    filesystem path.
 
    Note that ``.hgsub`` does not exist by default in Mercurial
    repositories, you have to create and add it to the parent
    repository before using subrepositories.
 
 3. Nested repository states. They are defined in ``.hgsubstate``, which
    is placed in the root of working directory, and
    capture whatever information is required to restore the
    subrepositories to the state they were committed in a parent
    repository changeset. Mercurial automatically record the nested
    repositories states when committing in the parent repository.
 
    .. note::
 
       The ``.hgsubstate`` file should not be edited manually.
 
 
 Adding a Subrepository
 ======================
 
 If ``.hgsub`` does not exist, create it and add it to the parent
 repository. Clone or checkout the external projects where you want it
 to live in the parent repository. Edit ``.hgsub`` and add the
 subrepository entry as described above. At this point, the
 subrepository is tracked and the next commit will record its state in
 ``.hgsubstate`` and bind it to the committed changeset.
 
 Synchronizing a Subrepository
 =============================
 
 Subrepos do not automatically track the latest changeset of their
 sources. Instead, they are updated to the changeset that corresponds
 with the changeset checked out in the top-level changeset. This is so
 developers always get a consistent set of compatible code and
 libraries when they update.
 
 Thus, updating subrepos is a manual process. Simply check out target
 subrepo at the desired revision, test in the top-level repo, then
 commit in the parent repository to record the new combination.
 
 Deleting a Subrepository
 ========================
 
 To remove a subrepository from the parent repository, delete its
 reference from ``.hgsub``, then remove its files.
 
 Interaction with Mercurial Commands
 ===================================
 
 :add: add does not recurse in subrepos unless -S/--subrepos is
     specified.  However, if you specify the full path of a file in a
     subrepo, it will be added even without -S/--subrepos specified.
-    Git and Subversion subrepositories are currently silently
+    Subversion subrepositories are currently silently
     ignored.
 
 :addremove: addremove does not recurse into subrepos unless
     -S/--subrepos is specified.  However, if you specify the full
     path of a directory in a subrepo, addremove will be performed on
     it even without -S/--subrepos being specified.  Git and
     Subversion subrepositories will print a warning and continue.
 
 :archive: archive does not recurse in subrepositories unless
     -S/--subrepos is specified.
 
 :cat: cat currently only handles exact file matches in subrepos.
     Subversion subrepositories are currently ignored.
 
 :commit: commit creates a consistent snapshot of the state of the
     entire project and its subrepositories. If any subrepositories
     have been modified, Mercurial will abort.  Mercurial can be made
     to instead commit all modified subrepositories by specifying
     -S/--subrepos, or setting "ui.commitsubrepos=True" in a
     configuration file (see :hg:`help config`).  After there are no
     longer any modified subrepositories, it records their state and
     finally commits it in the parent repository.  The --addremove
     option also honors the -S/--subrepos option.  However, Git and
     Subversion subrepositories will print a warning and abort.
 
 :diff: diff does not recurse in subrepos unless -S/--subrepos is
     specified. Changes are displayed as usual, on the subrepositories
     elements. Git subrepositories do not support --include/--exclude.
     Subversion subrepositories are currently silently ignored.
 
 :forget: forget currently only handles exact file matches in subrepos.
     Git and Subversion subrepositories are currently silently ignored.
 
 :incoming: incoming does not recurse in subrepos unless -S/--subrepos
     is specified. Git and Subversion subrepositories are currently
     silently ignored.
 
 :outgoing: outgoing does not recurse in subrepos unless -S/--subrepos
     is specified. Git and Subversion subrepositories are currently
     silently ignored.
 
 :pull: pull is not recursive since it is not clear what to pull prior
     to running :hg:`update`. Listing and retrieving all
     subrepositories changes referenced by the parent repository pulled
     changesets is expensive at best, impossible in the Subversion
     case.
 
 :push: Mercurial will automatically push all subrepositories first
     when the parent repository is being pushed. This ensures new
     subrepository changes are available when referenced by top-level
     repositories.  Push is a no-op for Subversion subrepositories.
 
 :status: status does not recurse into subrepositories unless
     -S/--subrepos is specified. Subrepository changes are displayed as
     regular Mercurial changes on the subrepository
     elements. Subversion subrepositories are currently silently
     ignored.
 
 :remove: remove does not recurse into subrepositories unless
     -S/--subrepos is specified.  However, if you specify a file or
     directory path in a subrepo, it will be removed even without
     -S/--subrepos.  Git and Subversion subrepositories are currently
     silently ignored.
 
 :update: update restores the subrepos in the state they were
     originally committed in target changeset. If the recorded
     changeset is not available in the current subrepository, Mercurial
     will pull it in first before updating.  This means that updating
     can require network access when using subrepositories.
 
 Remapping Subrepositories Sources
 =================================
 
 A subrepository source location may change during a project life,
 invalidating references stored in the parent repository history. To
 fix this, rewriting rules can be defined in parent repository ``hgrc``
 file or in Mercurial configuration. See the ``[subpaths]`` section in
 hgrc(5) for more details.
 

mercurial/subrepo.py

@@ -1,1705 +1,1728 b''
 # subrepo.py - sub-repository handling for Mercurial
 #
 # Copyright 2009-2010 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 import copy
 import errno, os, re, shutil, posixpath, sys
 import xml.dom.minidom
 import stat, subprocess, tarfile
 from i18n import _
 import config, util, node, error, cmdutil, scmutil, match as matchmod
 import phases
 import pathutil
 import exchange
 hg = None
 propertycache = util.propertycache
 
 nullstate = ('', '', 'empty')
 
 def _expandedabspath(path):
     '''
     get a path or url and if it is a path expand it and return an absolute path
     '''
     expandedpath = util.urllocalpath(util.expandpath(path))
     u = util.url(expandedpath)
     if not u.scheme:
         path = util.normpath(os.path.abspath(u.path))
     return path
 
 def _getstorehashcachename(remotepath):
     '''get a unique filename for the store hash cache of a remote repository'''
     return util.sha1(_expandedabspath(remotepath)).hexdigest()[0:12]
 
 class SubrepoAbort(error.Abort):
     """Exception class used to avoid handling a subrepo error more than once"""
     def __init__(self, *args, **kw):
         error.Abort.__init__(self, *args, **kw)
         self.subrepo = kw.get('subrepo')
         self.cause = kw.get('cause')
 
 def annotatesubrepoerror(func):
     def decoratedmethod(self, *args, **kargs):
         try:
             res = func(self, *args, **kargs)
         except SubrepoAbort, ex:
             # This exception has already been handled
             raise ex
         except error.Abort, ex:
             subrepo = subrelpath(self)
             errormsg = str(ex) + ' ' + _('(in subrepo %s)') % subrepo
             # avoid handling this exception by raising a SubrepoAbort exception
             raise SubrepoAbort(errormsg, hint=ex.hint, subrepo=subrepo,
                                cause=sys.exc_info())
         return res
     return decoratedmethod
 
 def state(ctx, ui):
     """return a state dict, mapping subrepo paths configured in .hgsub
     to tuple: (source from .hgsub, revision from .hgsubstate, kind
     (key in types dict))
     """
     p = config.config()
     def read(f, sections=None, remap=None):
         if f in ctx:
             try:
                 data = ctx[f].data()
             except IOError, err:
                 if err.errno != errno.ENOENT:
                     raise
                 # handle missing subrepo spec files as removed
                 ui.warn(_("warning: subrepo spec file %s not found\n") % f)
                 return
             p.parse(f, data, sections, remap, read)
         else:
             raise util.Abort(_("subrepo spec file %s not found") % f)
 
     if '.hgsub' in ctx:
         read('.hgsub')
 
     for path, src in ui.configitems('subpaths'):
         p.set('subpaths', path, src, ui.configsource('subpaths', path))
 
     rev = {}
     if '.hgsubstate' in ctx:
         try:
             for i, l in enumerate(ctx['.hgsubstate'].data().splitlines()):
                 l = l.lstrip()
                 if not l:
                     continue
                 try:
                     revision, path = l.split(" ", 1)
                 except ValueError:
                     raise util.Abort(_("invalid subrepository revision "
                                        "specifier in .hgsubstate line %d")
                                      % (i + 1))
                 rev[path] = revision
         except IOError, err:
             if err.errno != errno.ENOENT:
                 raise
 
     def remap(src):
         for pattern, repl in p.items('subpaths'):
             # Turn r'C:\foo\bar' into r'C:\\foo\\bar' since re.sub
             # does a string decode.
             repl = repl.encode('string-escape')
             # However, we still want to allow back references to go
             # through unharmed, so we turn r'\\1' into r'\1'. Again,
             # extra escapes are needed because re.sub string decodes.
             repl = re.sub(r'\\\\([0-9]+)', r'\\\1', repl)
             try:
                 src = re.sub(pattern, repl, src, 1)
             except re.error, e:
                 raise util.Abort(_("bad subrepository pattern in %s: %s")
                                  % (p.source('subpaths', pattern), e))
         return src
 
     state = {}
     for path, src in p[''].items():
         kind = 'hg'
         if src.startswith('['):
             if ']' not in src:
                 raise util.Abort(_('missing ] in subrepo source'))
             kind, src = src.split(']', 1)
             kind = kind[1:]
             src = src.lstrip() # strip any extra whitespace after ']'
 
         if not util.url(src).isabs():
             parent = _abssource(ctx._repo, abort=False)
             if parent:
                 parent = util.url(parent)
                 parent.path = posixpath.join(parent.path or '', src)
                 parent.path = posixpath.normpath(parent.path)
                 joined = str(parent)
                 # Remap the full joined path and use it if it changes,
                 # else remap the original source.
                 remapped = remap(joined)
                 if remapped == joined:
                     src = remap(src)
                 else:
                     src = remapped
 
         src = remap(src)
         state[util.pconvert(path)] = (src.strip(), rev.get(path, ''), kind)
 
     return state
 
 def writestate(repo, state):
     """rewrite .hgsubstate in (outer) repo with these subrepo states"""
     lines = ['%s %s\n' % (state[s][1], s) for s in sorted(state)]
     repo.wwrite('.hgsubstate', ''.join(lines), '')
 
 def submerge(repo, wctx, mctx, actx, overwrite):
     """delegated from merge.applyupdates: merging of .hgsubstate file
     in working context, merging context and ancestor context"""
     if mctx == actx: # backwards?
         actx = wctx.p1()
     s1 = wctx.substate
     s2 = mctx.substate
     sa = actx.substate
     sm = {}
 
     repo.ui.debug("subrepo merge %s %s %s\n" % (wctx, mctx, actx))
 
     def debug(s, msg, r=""):
         if r:
             r = "%s:%s:%s" % r
         repo.ui.debug("  subrepo %s: %s %s\n" % (s, msg, r))
 
     for s, l in sorted(s1.iteritems()):
         a = sa.get(s, nullstate)
         ld = l # local state with possible dirty flag for compares
         if wctx.sub(s).dirty():
             ld = (l[0], l[1] + "+")
         if wctx == actx: # overwrite
             a = ld
 
         if s in s2:
             r = s2[s]
             if ld == r or r == a: # no change or local is newer
                 sm[s] = l
                 continue
             elif ld == a: # other side changed
                 debug(s, "other changed, get", r)
                 wctx.sub(s).get(r, overwrite)
                 sm[s] = r
             elif ld[0] != r[0]: # sources differ
                 if repo.ui.promptchoice(
                     _(' subrepository sources for %s differ\n'
                       'use (l)ocal source (%s) or (r)emote source (%s)?'
                       '$$ &Local $$ &Remote') % (s, l[0], r[0]), 0):
                     debug(s, "prompt changed, get", r)
                     wctx.sub(s).get(r, overwrite)
                     sm[s] = r
             elif ld[1] == a[1]: # local side is unchanged
                 debug(s, "other side changed, get", r)
                 wctx.sub(s).get(r, overwrite)
                 sm[s] = r
             else:
                 debug(s, "both sides changed")
                 srepo = wctx.sub(s)
                 option = repo.ui.promptchoice(
                     _(' subrepository %s diverged (local revision: %s, '
                       'remote revision: %s)\n'
                       '(M)erge, keep (l)ocal or keep (r)emote?'
                       '$$ &Merge $$ &Local $$ &Remote')
                     % (s, srepo.shortid(l[1]), srepo.shortid(r[1])), 0)
                 if option == 0:
                     wctx.sub(s).merge(r)
                     sm[s] = l
                     debug(s, "merge with", r)
                 elif option == 1:
                     sm[s] = l
                     debug(s, "keep local subrepo revision", l)
                 else:
                     wctx.sub(s).get(r, overwrite)
                     sm[s] = r
                     debug(s, "get remote subrepo revision", r)
         elif ld == a: # remote removed, local unchanged
             debug(s, "remote removed, remove")
             wctx.sub(s).remove()
         elif a == nullstate: # not present in remote or ancestor
             debug(s, "local added, keep")
             sm[s] = l
             continue
         else:
             if repo.ui.promptchoice(
                 _(' local changed subrepository %s which remote removed\n'
                   'use (c)hanged version or (d)elete?'
                   '$$ &Changed $$ &Delete') % s, 0):
                 debug(s, "prompt remove")
                 wctx.sub(s).remove()
 
     for s, r in sorted(s2.items()):
         if s in s1:
             continue
         elif s not in sa:
             debug(s, "remote added, get", r)
             mctx.sub(s).get(r)
             sm[s] = r
         elif r != sa[s]:
             if repo.ui.promptchoice(
                 _(' remote changed subrepository %s which local removed\n'
                   'use (c)hanged version or (d)elete?'
                   '$$ &Changed $$ &Delete') % s, 0) == 0:
                 debug(s, "prompt recreate", r)
                 mctx.sub(s).get(r)
                 sm[s] = r
 
     # record merged .hgsubstate
     writestate(repo, sm)
     return sm
 
 def _updateprompt(ui, sub, dirty, local, remote):
     if dirty:
         msg = (_(' subrepository sources for %s differ\n'
                  'use (l)ocal source (%s) or (r)emote source (%s)?'
                  '$$ &Local $$ &Remote')
                % (subrelpath(sub), local, remote))
     else:
         msg = (_(' subrepository sources for %s differ (in checked out '
                  'version)\n'
                  'use (l)ocal source (%s) or (r)emote source (%s)?'
                  '$$ &Local $$ &Remote')
                % (subrelpath(sub), local, remote))
     return ui.promptchoice(msg, 0)
 
 def reporelpath(repo):
     """return path to this (sub)repo as seen from outermost repo"""
     parent = repo
     while util.safehasattr(parent, '_subparent'):
         parent = parent._subparent
     return repo.root[len(pathutil.normasprefix(parent.root)):]
 
 def subrelpath(sub):
     """return path to this subrepo as seen from outermost repo"""
     if util.safehasattr(sub, '_relpath'):
         return sub._relpath
     if not util.safehasattr(sub, '_repo'):
         return sub._path
     return reporelpath(sub._repo)
 
 def _abssource(repo, push=False, abort=True):
     """return pull/push path of repo - either based on parent repo .hgsub info
     or on the top repo config. Abort or return None if no source found."""
     if util.safehasattr(repo, '_subparent'):
         source = util.url(repo._subsource)
         if source.isabs():
             return str(source)
         source.path = posixpath.normpath(source.path)
         parent = _abssource(repo._subparent, push, abort=False)
         if parent:
             parent = util.url(util.pconvert(parent))
             parent.path = posixpath.join(parent.path or '', source.path)
             parent.path = posixpath.normpath(parent.path)
             return str(parent)
     else: # recursion reached top repo
         if util.safehasattr(repo, '_subtoppath'):
             return repo._subtoppath
         if push and repo.ui.config('paths', 'default-push'):
             return repo.ui.config('paths', 'default-push')
         if repo.ui.config('paths', 'default'):
             return repo.ui.config('paths', 'default')
         if repo.shared():
             # chop off the .hg component to get the default path form
             return os.path.dirname(repo.sharedpath)
     if abort:
         raise util.Abort(_("default path for subrepository not found"))
 
 def _sanitize(ui, path, ignore):
     for dirname, dirs, names in os.walk(path):
         for i, d in enumerate(dirs):
             if d.lower() == ignore:
                 del dirs[i]
                 break
         if os.path.basename(dirname).lower() != '.hg':
             continue
         for f in names:
             if f.lower() == 'hgrc':
                 ui.warn(_("warning: removing potentially hostile 'hgrc' "
                           "in '%s'\n") % dirname)
                 os.unlink(os.path.join(dirname, f))
 
 def subrepo(ctx, path):
     """return instance of the right subrepo class for subrepo in path"""
     # subrepo inherently violates our import layering rules
     # because it wants to make repo objects from deep inside the stack
     # so we manually delay the circular imports to not break
     # scripts that don't use our demand-loading
     global hg
     import hg as h
     hg = h
 
     pathutil.pathauditor(ctx._repo.root)(path)
     state = ctx.substate[path]
     if state[2] not in types:
         raise util.Abort(_('unknown subrepo type %s') % state[2])
     return types[state[2]](ctx, path, state[:2])
 
 def newcommitphase(ui, ctx):
     commitphase = phases.newcommitphase(ui)
     substate = getattr(ctx, "substate", None)
     if not substate:
         return commitphase
     check = ui.config('phases', 'checksubrepos', 'follow')
     if check not in ('ignore', 'follow', 'abort'):
         raise util.Abort(_('invalid phases.checksubrepos configuration: %s')
                          % (check))
     if check == 'ignore':
         return commitphase
     maxphase = phases.public
     maxsub = None
     for s in sorted(substate):
         sub = ctx.sub(s)
         subphase = sub.phase(substate[s][1])
         if maxphase < subphase:
             maxphase = subphase
             maxsub = s
     if commitphase < maxphase:
         if check == 'abort':
             raise util.Abort(_("can't commit in %s phase"
                                " conflicting %s from subrepository %s") %
                              (phases.phasenames[commitphase],
                               phases.phasenames[maxphase], maxsub))
         ui.warn(_("warning: changes are committed in"
                   " %s phase from subrepository %s\n") %
                 (phases.phasenames[maxphase], maxsub))
         return maxphase
     return commitphase
 
 # subrepo classes need to implement the following abstract class:
 
 class abstractsubrepo(object):
 
     def __init__(self, ui):
         self.ui = ui
 
     def storeclean(self, path):
         """
         returns true if the repository has not changed since it was last
         cloned from or pushed to a given repository.
         """
         return False
 
     def dirty(self, ignoreupdate=False):
         """returns true if the dirstate of the subrepo is dirty or does not
         match current stored state. If ignoreupdate is true, only check
         whether the subrepo has uncommitted changes in its dirstate.
         """
         raise NotImplementedError
 
     def basestate(self):
         """current working directory base state, disregarding .hgsubstate
         state and working directory modifications"""
         raise NotImplementedError
 
     def checknested(self, path):
         """check if path is a subrepository within this repository"""
         return False
 
     def commit(self, text, user, date):
         """commit the current changes to the subrepo with the given
         log message. Use given user and date if possible. Return the
         new state of the subrepo.
         """
         raise NotImplementedError
 
     def phase(self, state):
         """returns phase of specified state in the subrepository.
         """
         return phases.public
 
     def remove(self):
         """remove the subrepo
 
         (should verify the dirstate is not dirty first)
         """
         raise NotImplementedError
 
     def get(self, state, overwrite=False):
         """run whatever commands are needed to put the subrepo into
         this state
         """
         raise NotImplementedError
 
     def merge(self, state):
         """merge currently-saved state with the new state."""
         raise NotImplementedError
 
     def push(self, opts):
         """perform whatever action is analogous to 'hg push'
 
         This may be a no-op on some systems.
         """
         raise NotImplementedError
 
     def add(self, ui, match, prefix, explicitonly, **opts):
         return []
 
     def addremove(self, matcher, prefix, opts, dry_run, similarity):
         self.ui.warn("%s: %s" % (prefix, _("addremove is not supported")))
         return 1
 
     def cat(self, match, prefix, **opts):
         return 1
 
     def status(self, rev2, **opts):
         return scmutil.status([], [], [], [], [], [], [])
 
     def diff(self, ui, diffopts, node2, match, prefix, **opts):
         pass
 
     def outgoing(self, ui, dest, opts):
         return 1
 
     def incoming(self, ui, source, opts):
         return 1
 
     def files(self):
         """return filename iterator"""
         raise NotImplementedError
 
     def filedata(self, name):
         """return file data"""
         raise NotImplementedError
 
     def fileflags(self, name):
         """return file flags"""
         return ''
 
     def archive(self, archiver, prefix, match=None):
         if match is not None:
             files = [f for f in self.files() if match(f)]
         else:
             files = self.files()
         total = len(files)
         relpath = subrelpath(self)
         self.ui.progress(_('archiving (%s)') % relpath, 0,
                          unit=_('files'), total=total)
         for i, name in enumerate(files):
             flags = self.fileflags(name)
             mode = 'x' in flags and 0755 or 0644
             symlink = 'l' in flags
             archiver.addfile(os.path.join(prefix, self._path, name),
                              mode, symlink, self.filedata(name))
             self.ui.progress(_('archiving (%s)') % relpath, i + 1,
                              unit=_('files'), total=total)
         self.ui.progress(_('archiving (%s)') % relpath, None)
         return total
 
     def walk(self, match):
         '''
         walk recursively through the directory tree, finding all files
         matched by the match function
         '''
         pass
 
     def forget(self, match, prefix):
         return ([], [])
 
     def removefiles(self, matcher, prefix, after, force, subrepos):
         """remove the matched files from the subrepository and the filesystem,
         possibly by force and/or after the file has been removed from the
         filesystem.  Return 0 on success, 1 on any warning.
         """
         return 1
 
     def revert(self, substate, *pats, **opts):
         self.ui.warn('%s: reverting %s subrepos is unsupported\n' \
             % (substate[0], substate[2]))
         return []
 
     def shortid(self, revid):
         return revid
 
 class hgsubrepo(abstractsubrepo):
     def __init__(self, ctx, path, state):
         super(hgsubrepo, self).__init__(ctx._repo.ui)
         self._path = path
         self._state = state
         r = ctx._repo
         root = r.wjoin(path)
         create = not r.wvfs.exists('%s/.hg' % path)
         self._repo = hg.repository(r.baseui, root, create=create)
         self.ui = self._repo.ui
         for s, k in [('ui', 'commitsubrepos')]:
             v = r.ui.config(s, k)
             if v:
                 self.ui.setconfig(s, k, v, 'subrepo')
         self.ui.setconfig('ui', '_usedassubrepo', 'True', 'subrepo')
         self._initrepo(r, state[0], create)
 
     def storeclean(self, path):
         lock = self._repo.lock()
         try:
             return self._storeclean(path)
         finally:
             lock.release()
 
     def _storeclean(self, path):
         clean = True
         itercache = self._calcstorehash(path)
         try:
             for filehash in self._readstorehashcache(path):
                 if filehash != itercache.next():
                     clean = False
                     break
         except StopIteration:
             # the cached and current pull states have a different size
             clean = False
         if clean:
             try:
                 itercache.next()
                 # the cached and current pull states have a different size
                 clean = False
             except StopIteration:
                 pass
         return clean
 
     def _calcstorehash(self, remotepath):
         '''calculate a unique "store hash"
 
         This method is used to to detect when there are changes that may
         require a push to a given remote path.'''
         # sort the files that will be hashed in increasing (likely) file size
         filelist = ('bookmarks', 'store/phaseroots', 'store/00changelog.i')
         yield '# %s\n' % _expandedabspath(remotepath)
         vfs = self._repo.vfs
         for relname in filelist:
             filehash = util.sha1(vfs.tryread(relname)).hexdigest()
             yield '%s = %s\n' % (relname, filehash)
 
     @propertycache
     def _cachestorehashvfs(self):
         return scmutil.vfs(self._repo.join('cache/storehash'))
 
     def _readstorehashcache(self, remotepath):
         '''read the store hash cache for a given remote repository'''
         cachefile = _getstorehashcachename(remotepath)
         return self._cachestorehashvfs.tryreadlines(cachefile, 'r')
 
     def _cachestorehash(self, remotepath):
         '''cache the current store hash
 
         Each remote repo requires its own store hash cache, because a subrepo
         store may be "clean" versus a given remote repo, but not versus another
         '''
         cachefile = _getstorehashcachename(remotepath)
         lock = self._repo.lock()
         try:
             storehash = list(self._calcstorehash(remotepath))
             vfs = self._cachestorehashvfs
             vfs.writelines(cachefile, storehash, mode='w', notindexed=True)
         finally:
             lock.release()
 
     @annotatesubrepoerror
     def _initrepo(self, parentrepo, source, create):
         self._repo._subparent = parentrepo
         self._repo._subsource = source
 
         if create:
             lines = ['[paths]\n']
 
             def addpathconfig(key, value):
                 if value:
                     lines.append('%s = %s\n' % (key, value))
                     self.ui.setconfig('paths', key, value, 'subrepo')
 
             defpath = _abssource(self._repo, abort=False)
             defpushpath = _abssource(self._repo, True, abort=False)
             addpathconfig('default', defpath)
             if defpath != defpushpath:
                 addpathconfig('default-push', defpushpath)
 
             fp = self._repo.vfs("hgrc", "w", text=True)
             try:
                 fp.write(''.join(lines))
             finally:
                 fp.close()
 
     @annotatesubrepoerror
     def add(self, ui, match, prefix, explicitonly, **opts):
         return cmdutil.add(ui, self._repo, match,
                            os.path.join(prefix, self._path), explicitonly,
                            **opts)
 
     @annotatesubrepoerror
     def addremove(self, m, prefix, opts, dry_run, similarity):
         # In the same way as sub directories are processed, once in a subrepo,
         # always entry any of its subrepos.  Don't corrupt the options that will
         # be used to process sibling subrepos however.
         opts = copy.copy(opts)
         opts['subrepos'] = True
         return scmutil.addremove(self._repo, m,
                                  os.path.join(prefix, self._path), opts,
                                  dry_run, similarity)
 
     @annotatesubrepoerror
     def cat(self, match, prefix, **opts):
         rev = self._state[1]
         ctx = self._repo[rev]
         return cmdutil.cat(self.ui, self._repo, ctx, match, prefix, **opts)
 
     @annotatesubrepoerror
     def status(self, rev2, **opts):
         try:
             rev1 = self._state[1]
             ctx1 = self._repo[rev1]
             ctx2 = self._repo[rev2]
             return self._repo.status(ctx1, ctx2, **opts)
         except error.RepoLookupError, inst:
             self.ui.warn(_('warning: error "%s" in subrepository "%s"\n')
                          % (inst, subrelpath(self)))
             return scmutil.status([], [], [], [], [], [], [])
 
     @annotatesubrepoerror
     def diff(self, ui, diffopts, node2, match, prefix, **opts):
         try:
             node1 = node.bin(self._state[1])
             # We currently expect node2 to come from substate and be
             # in hex format
             if node2 is not None:
                 node2 = node.bin(node2)
             cmdutil.diffordiffstat(ui, self._repo, diffopts,
                                    node1, node2, match,
                                    prefix=posixpath.join(prefix, self._path),
                                    listsubrepos=True, **opts)
         except error.RepoLookupError, inst:
             self.ui.warn(_('warning: error "%s" in subrepository "%s"\n')
                           % (inst, subrelpath(self)))
 
     @annotatesubrepoerror
     def archive(self, archiver, prefix, match=None):
         self._get(self._state + ('hg',))
         total = abstractsubrepo.archive(self, archiver, prefix, match)
         rev = self._state[1]
         ctx = self._repo[rev]
         for subpath in ctx.substate:
             s = subrepo(ctx, subpath)
             submatch = matchmod.narrowmatcher(subpath, match)
             total += s.archive(
                 archiver, os.path.join(prefix, self._path), submatch)
         return total
 
     @annotatesubrepoerror
     def dirty(self, ignoreupdate=False):
         r = self._state[1]
         if r == '' and not ignoreupdate: # no state recorded
             return True
         w = self._repo[None]
         if r != w.p1().hex() and not ignoreupdate:
             # different version checked out
             return True
         return w.dirty() # working directory changed
 
     def basestate(self):
         return self._repo['.'].hex()
 
     def checknested(self, path):
         return self._repo._checknested(self._repo.wjoin(path))
 
     @annotatesubrepoerror
     def commit(self, text, user, date):
         # don't bother committing in the subrepo if it's only been
         # updated
         if not self.dirty(True):
             return self._repo['.'].hex()
         self.ui.debug("committing subrepo %s\n" % subrelpath(self))
         n = self._repo.commit(text, user, date)
         if not n:
             return self._repo['.'].hex() # different version checked out
         return node.hex(n)
 
     @annotatesubrepoerror
     def phase(self, state):
         return self._repo[state].phase()
 
     @annotatesubrepoerror
     def remove(self):
         # we can't fully delete the repository as it may contain
         # local-only history
         self.ui.note(_('removing subrepo %s\n') % subrelpath(self))
         hg.clean(self._repo, node.nullid, False)
 
     def _get(self, state):
         source, revision, kind = state
         if revision in self._repo.unfiltered():
             return True
         self._repo._subsource = source
         srcurl = _abssource(self._repo)
         other = hg.peer(self._repo, {}, srcurl)
         if len(self._repo) == 0:
             self.ui.status(_('cloning subrepo %s from %s\n')
                            % (subrelpath(self), srcurl))
             parentrepo = self._repo._subparent
             shutil.rmtree(self._repo.path)
             other, cloned = hg.clone(self._repo._subparent.baseui, {},
                                      other, self._repo.root,
                                      update=False)
             self._repo = cloned.local()
             self._initrepo(parentrepo, source, create=True)
             self._cachestorehash(srcurl)
         else:
             self.ui.status(_('pulling subrepo %s from %s\n')
                            % (subrelpath(self), srcurl))
             cleansub = self.storeclean(srcurl)
             exchange.pull(self._repo, other)
             if cleansub:
                 # keep the repo clean after pull
                 self._cachestorehash(srcurl)
         return False
 
     @annotatesubrepoerror
     def get(self, state, overwrite=False):
         inrepo = self._get(state)
         source, revision, kind = state
         repo = self._repo
         repo.ui.debug("getting subrepo %s\n" % self._path)
         if inrepo:
             urepo = repo.unfiltered()
             ctx = urepo[revision]
             if ctx.hidden():
                 urepo.ui.warn(
                     _('revision %s in subrepo %s is hidden\n') \
                     % (revision[0:12], self._path))
                 repo = urepo
         hg.updaterepo(repo, revision, overwrite)
 
     @annotatesubrepoerror
     def merge(self, state):
         self._get(state)
         cur = self._repo['.']
         dst = self._repo[state[1]]
         anc = dst.ancestor(cur)
 
         def mergefunc():
             if anc == cur and dst.branch() == cur.branch():
                 self.ui.debug("updating subrepo %s\n" % subrelpath(self))
                 hg.update(self._repo, state[1])
             elif anc == dst:
                 self.ui.debug("skipping subrepo %s\n" % subrelpath(self))
             else:
                 self.ui.debug("merging subrepo %s\n" % subrelpath(self))
                 hg.merge(self._repo, state[1], remind=False)
 
         wctx = self._repo[None]
         if self.dirty():
             if anc != dst:
                 if _updateprompt(self.ui, self, wctx.dirty(), cur, dst):
                     mergefunc()
             else:
                 mergefunc()
         else:
             mergefunc()
 
     @annotatesubrepoerror
     def push(self, opts):
         force = opts.get('force')
         newbranch = opts.get('new_branch')
         ssh = opts.get('ssh')
 
         # push subrepos depth-first for coherent ordering
         c = self._repo['']
         subs = c.substate # only repos that are committed
         for s in sorted(subs):
             if c.sub(s).push(opts) == 0:
                 return False
 
         dsturl = _abssource(self._repo, True)
         if not force:
             if self.storeclean(dsturl):
                 self.ui.status(
                     _('no changes made to subrepo %s since last push to %s\n')
                     % (subrelpath(self), dsturl))
                 return None
         self.ui.status(_('pushing subrepo %s to %s\n') %
             (subrelpath(self), dsturl))
         other = hg.peer(self._repo, {'ssh': ssh}, dsturl)
         res = exchange.push(self._repo, other, force, newbranch=newbranch)
 
         # the repo is now clean
         self._cachestorehash(dsturl)
         return res.cgresult
 
     @annotatesubrepoerror
     def outgoing(self, ui, dest, opts):
         return hg.outgoing(ui, self._repo, _abssource(self._repo, True), opts)
 
     @annotatesubrepoerror
     def incoming(self, ui, source, opts):
         return hg.incoming(ui, self._repo, _abssource(self._repo, False), opts)
 
     @annotatesubrepoerror
     def files(self):
         rev = self._state[1]
         ctx = self._repo[rev]
         return ctx.manifest().keys()
 
     def filedata(self, name):
         rev = self._state[1]
         return self._repo[rev][name].data()
 
     def fileflags(self, name):
         rev = self._state[1]
         ctx = self._repo[rev]
         return ctx.flags(name)
 
     def walk(self, match):
         ctx = self._repo[None]
         return ctx.walk(match)
 
     @annotatesubrepoerror
     def forget(self, match, prefix):
         return cmdutil.forget(self.ui, self._repo, match,
                               os.path.join(prefix, self._path), True)
 
     @annotatesubrepoerror
     def removefiles(self, matcher, prefix, after, force, subrepos):
         return cmdutil.remove(self.ui, self._repo, matcher,
                               os.path.join(prefix, self._path), after, force,
                               subrepos)
 
     @annotatesubrepoerror
     def revert(self, substate, *pats, **opts):
         # reverting a subrepo is a 2 step process:
         # 1. if the no_backup is not set, revert all modified
         #    files inside the subrepo
         # 2. update the subrepo to the revision specified in
         #    the corresponding substate dictionary
         self.ui.status(_('reverting subrepo %s\n') % substate[0])
         if not opts.get('no_backup'):
             # Revert all files on the subrepo, creating backups
             # Note that this will not recursively revert subrepos
             # We could do it if there was a set:subrepos() predicate
             opts = opts.copy()
             opts['date'] = None
             opts['rev'] = substate[1]
 
             self.filerevert(*pats, **opts)
 
         # Update the repo to the revision specified in the given substate
         if not opts.get('dry_run'):
             self.get(substate, overwrite=True)
 
     def filerevert(self, *pats, **opts):
         ctx = self._repo[opts['rev']]
         parents = self._repo.dirstate.parents()
         if opts.get('all'):
             pats = ['set:modified()']
         else:
             pats = []
         cmdutil.revert(self.ui, self._repo, ctx, parents, *pats, **opts)
 
     def shortid(self, revid):
         return revid[:12]
 
 class svnsubrepo(abstractsubrepo):
     def __init__(self, ctx, path, state):
         super(svnsubrepo, self).__init__(ctx._repo.ui)
         self._path = path
         self._state = state
         self._ctx = ctx
         self._exe = util.findexe('svn')
         if not self._exe:
             raise util.Abort(_("'svn' executable not found for subrepo '%s'")
                              % self._path)
 
     def _svncommand(self, commands, filename='', failok=False):
         cmd = [self._exe]
         extrakw = {}
         if not self.ui.interactive():
             # Making stdin be a pipe should prevent svn from behaving
             # interactively even if we can't pass --non-interactive.
             extrakw['stdin'] = subprocess.PIPE
             # Starting in svn 1.5 --non-interactive is a global flag
             # instead of being per-command, but we need to support 1.4 so
             # we have to be intelligent about what commands take
             # --non-interactive.
             if commands[0] in ('update', 'checkout', 'commit'):
                 cmd.append('--non-interactive')
         cmd.extend(commands)
         if filename is not None:
             path = os.path.join(self._ctx._repo.origroot, self._path, filename)
             cmd.append(path)
         env = dict(os.environ)
         # Avoid localized output, preserve current locale for everything else.
         lc_all = env.get('LC_ALL')
         if lc_all:
             env['LANG'] = lc_all
             del env['LC_ALL']
         env['LC_MESSAGES'] = 'C'
         p = subprocess.Popen(cmd, bufsize=-1, close_fds=util.closefds,
                              stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                               universal_newlines=True, env=env, **extrakw)
         stdout, stderr = p.communicate()
         stderr = stderr.strip()
         if not failok:
             if p.returncode:
                 raise util.Abort(stderr or 'exited with code %d' % p.returncode)
             if stderr:
                 self.ui.warn(stderr + '\n')
         return stdout, stderr
 
     @propertycache
     def _svnversion(self):
         output, err = self._svncommand(['--version', '--quiet'], filename=None)
         m = re.search(r'^(\d+)\.(\d+)', output)
         if not m:
             raise util.Abort(_('cannot retrieve svn tool version'))
         return (int(m.group(1)), int(m.group(2)))
 
     def _wcrevs(self):
         # Get the working directory revision as well as the last
         # commit revision so we can compare the subrepo state with
         # both. We used to store the working directory one.
         output, err = self._svncommand(['info', '--xml'])
         doc = xml.dom.minidom.parseString(output)
         entries = doc.getElementsByTagName('entry')
         lastrev, rev = '0', '0'
         if entries:
             rev = str(entries[0].getAttribute('revision')) or '0'
             commits = entries[0].getElementsByTagName('commit')
             if commits:
                 lastrev = str(commits[0].getAttribute('revision')) or '0'
         return (lastrev, rev)
 
     def _wcrev(self):
         return self._wcrevs()[0]
 
     def _wcchanged(self):
         """Return (changes, extchanges, missing) where changes is True
         if the working directory was changed, extchanges is
         True if any of these changes concern an external entry and missing
         is True if any change is a missing entry.
         """
         output, err = self._svncommand(['status', '--xml'])
         externals, changes, missing = [], [], []
         doc = xml.dom.minidom.parseString(output)
         for e in doc.getElementsByTagName('entry'):
             s = e.getElementsByTagName('wc-status')
             if not s:
                 continue
             item = s[0].getAttribute('item')
             props = s[0].getAttribute('props')
             path = e.getAttribute('path')
             if item == 'external':
                 externals.append(path)
             elif item == 'missing':
                 missing.append(path)
             if (item not in ('', 'normal', 'unversioned', 'external')
                 or props not in ('', 'none', 'normal')):
                 changes.append(path)
         for path in changes:
             for ext in externals:
                 if path == ext or path.startswith(ext + os.sep):
                     return True, True, bool(missing)
         return bool(changes), False, bool(missing)
 
     def dirty(self, ignoreupdate=False):
         if not self._wcchanged()[0]:
             if self._state[1] in self._wcrevs() or ignoreupdate:
                 return False
         return True
 
     def basestate(self):
         lastrev, rev = self._wcrevs()
         if lastrev != rev:
             # Last committed rev is not the same than rev. We would
             # like to take lastrev but we do not know if the subrepo
             # URL exists at lastrev.  Test it and fallback to rev it
             # is not there.
             try:
                 self._svncommand(['list', '%s@%s' % (self._state[0], lastrev)])
                 return lastrev
             except error.Abort:
                 pass
         return rev
 
     @annotatesubrepoerror
     def commit(self, text, user, date):
         # user and date are out of our hands since svn is centralized
         changed, extchanged, missing = self._wcchanged()
         if not changed:
             return self.basestate()
         if extchanged:
             # Do not try to commit externals
             raise util.Abort(_('cannot commit svn externals'))
         if missing:
             # svn can commit with missing entries but aborting like hg
             # seems a better approach.
             raise util.Abort(_('cannot commit missing svn entries'))
         commitinfo, err = self._svncommand(['commit', '-m', text])
         self.ui.status(commitinfo)
         newrev = re.search('Committed revision ([0-9]+).', commitinfo)
         if not newrev:
             if not commitinfo.strip():
                 # Sometimes, our definition of "changed" differs from
                 # svn one. For instance, svn ignores missing files
                 # when committing. If there are only missing files, no
                 # commit is made, no output and no error code.
                 raise util.Abort(_('failed to commit svn changes'))
             raise util.Abort(commitinfo.splitlines()[-1])
         newrev = newrev.groups()[0]
         self.ui.status(self._svncommand(['update', '-r', newrev])[0])
         return newrev
 
     @annotatesubrepoerror
     def remove(self):
         if self.dirty():
             self.ui.warn(_('not removing repo %s because '
                            'it has changes.\n') % self._path)
             return
         self.ui.note(_('removing subrepo %s\n') % self._path)
 
         def onerror(function, path, excinfo):
             if function is not os.remove:
                 raise
             # read-only files cannot be unlinked under Windows
             s = os.stat(path)
             if (s.st_mode & stat.S_IWRITE) != 0:
                 raise
             os.chmod(path, stat.S_IMODE(s.st_mode) | stat.S_IWRITE)
             os.remove(path)
 
         path = self._ctx._repo.wjoin(self._path)
         shutil.rmtree(path, onerror=onerror)
         try:
             os.removedirs(os.path.dirname(path))
         except OSError:
             pass
 
     @annotatesubrepoerror
     def get(self, state, overwrite=False):
         if overwrite:
             self._svncommand(['revert', '--recursive'])
         args = ['checkout']
         if self._svnversion >= (1, 5):
             args.append('--force')
         # The revision must be specified at the end of the URL to properly
         # update to a directory which has since been deleted and recreated.
         args.append('%s@%s' % (state[0], state[1]))
         status, err = self._svncommand(args, failok=True)
         _sanitize(self.ui, self._ctx._repo.wjoin(self._path), '.svn')
         if not re.search('Checked out revision [0-9]+.', status):
             if ('is already a working copy for a different URL' in err
                 and (self._wcchanged()[:2] == (False, False))):
                 # obstructed but clean working copy, so just blow it away.
                 self.remove()
                 self.get(state, overwrite=False)
                 return
             raise util.Abort((status or err).splitlines()[-1])
         self.ui.status(status)
 
     @annotatesubrepoerror
     def merge(self, state):
         old = self._state[1]
         new = state[1]
         wcrev = self._wcrev()
         if new != wcrev:
             dirty = old == wcrev or self._wcchanged()[0]
             if _updateprompt(self.ui, self, dirty, wcrev, new):
                 self.get(state, False)
 
     def push(self, opts):
         # push is a no-op for SVN
         return True
 
     @annotatesubrepoerror
     def files(self):
         output = self._svncommand(['list', '--recursive', '--xml'])[0]
         doc = xml.dom.minidom.parseString(output)
         paths = []
         for e in doc.getElementsByTagName('entry'):
             kind = str(e.getAttribute('kind'))
             if kind != 'file':
                 continue
             name = ''.join(c.data for c
                            in e.getElementsByTagName('name')[0].childNodes
                            if c.nodeType == c.TEXT_NODE)
             paths.append(name.encode('utf-8'))
         return paths
 
     def filedata(self, name):
         return self._svncommand(['cat'], name)[0]
 
 
 class gitsubrepo(abstractsubrepo):
     def __init__(self, ctx, path, state):
         super(gitsubrepo, self).__init__(ctx._repo.ui)
         self._state = state
         self._ctx = ctx
         self._path = path
         self._relpath = os.path.join(reporelpath(ctx._repo), path)
         self._abspath = ctx._repo.wjoin(path)
         self._subparent = ctx._repo
         self._ensuregit()
 
     def _ensuregit(self):
         try:
             self._gitexecutable = 'git'
             out, err = self._gitnodir(['--version'])
         except OSError, e:
             if e.errno != 2 or os.name != 'nt':
                 raise
             self._gitexecutable = 'git.cmd'
             out, err = self._gitnodir(['--version'])
         versionstatus = self._checkversion(out)
         if versionstatus == 'unknown':
             self.ui.warn(_('cannot retrieve git version\n'))
         elif versionstatus == 'abort':
             raise util.Abort(_('git subrepo requires at least 1.6.0 or later'))
         elif versionstatus == 'warning':
             self.ui.warn(_('git subrepo requires at least 1.6.0 or later\n'))
 
     @staticmethod
     def _gitversion(out):
         m = re.search(r'^git version (\d+)\.(\d+)\.(\d+)', out)
         if m:
             return (int(m.group(1)), int(m.group(2)), int(m.group(3)))
 
         m = re.search(r'^git version (\d+)\.(\d+)', out)
         if m:
             return (int(m.group(1)), int(m.group(2)), 0)
 
         return -1
 
     @staticmethod
     def _checkversion(out):
         '''ensure git version is new enough
 
         >>> _checkversion = gitsubrepo._checkversion
         >>> _checkversion('git version 1.6.0')
         'ok'
         >>> _checkversion('git version 1.8.5')
         'ok'
         >>> _checkversion('git version 1.4.0')
         'abort'
         >>> _checkversion('git version 1.5.0')
         'warning'
         >>> _checkversion('git version 1.9-rc0')
         'ok'
         >>> _checkversion('git version 1.9.0.265.g81cdec2')
         'ok'
         >>> _checkversion('git version 1.9.0.GIT')
         'ok'
         >>> _checkversion('git version 12345')
         'unknown'
         >>> _checkversion('no')
         'unknown'
         '''
         version = gitsubrepo._gitversion(out)
         # git 1.4.0 can't work at all, but 1.5.X can in at least some cases,
         # despite the docstring comment.  For now, error on 1.4.0, warn on
         # 1.5.0 but attempt to continue.
         if version == -1:
             return 'unknown'
         if version < (1, 5, 0):
             return 'abort'
         elif version < (1, 6, 0):
             return 'warning'
         return 'ok'
 
     def _gitcommand(self, commands, env=None, stream=False):
         return self._gitdir(commands, env=env, stream=stream)[0]
 
     def _gitdir(self, commands, env=None, stream=False):
         return self._gitnodir(commands, env=env, stream=stream,
                               cwd=self._abspath)
 
     def _gitnodir(self, commands, env=None, stream=False, cwd=None):
         """Calls the git command
 
         The methods tries to call the git command. versions prior to 1.6.0
         are not supported and very probably fail.
         """
         self.ui.debug('%s: git %s\n' % (self._relpath, ' '.join(commands)))
         # unless ui.quiet is set, print git's stderr,
         # which is mostly progress and useful info
         errpipe = None
         if self.ui.quiet:
             errpipe = open(os.devnull, 'w')
         p = subprocess.Popen([self._gitexecutable] + commands, bufsize=-1,
                              cwd=cwd, env=env, close_fds=util.closefds,
                              stdout=subprocess.PIPE, stderr=errpipe)
         if stream:
             return p.stdout, None
 
         retdata = p.stdout.read().strip()
         # wait for the child to exit to avoid race condition.
         p.wait()
 
         if p.returncode != 0 and p.returncode != 1:
             # there are certain error codes that are ok
             command = commands[0]
             if command in ('cat-file', 'symbolic-ref'):
                 return retdata, p.returncode
             # for all others, abort
             raise util.Abort('git %s error %d in %s' %
                              (command, p.returncode, self._relpath))
 
         return retdata, p.returncode
 
     def _gitmissing(self):
         return not os.path.exists(os.path.join(self._abspath, '.git'))
 
     def _gitstate(self):
         return self._gitcommand(['rev-parse', 'HEAD'])
 
     def _gitcurrentbranch(self):
         current, err = self._gitdir(['symbolic-ref', 'HEAD', '--quiet'])
         if err:
             current = None
         return current
 
     def _gitremote(self, remote):
         out = self._gitcommand(['remote', 'show', '-n', remote])
         line = out.split('\n')[1]
         i = line.index('URL: ') + len('URL: ')
         return line[i:]
 
     def _githavelocally(self, revision):
         out, code = self._gitdir(['cat-file', '-e', revision])
         return code == 0
 
     def _gitisancestor(self, r1, r2):
         base = self._gitcommand(['merge-base', r1, r2])
         return base == r1
 
     def _gitisbare(self):
         return self._gitcommand(['config', '--bool', 'core.bare']) == 'true'
 
     def _gitupdatestat(self):
         """This must be run before git diff-index.
         diff-index only looks at changes to file stat;
         this command looks at file contents and updates the stat."""
         self._gitcommand(['update-index', '-q', '--refresh'])
 
     def _gitbranchmap(self):
         '''returns 2 things:
         a map from git branch to revision
         a map from revision to branches'''
         branch2rev = {}
         rev2branch = {}
 
         out = self._gitcommand(['for-each-ref', '--format',
                                 '%(objectname) %(refname)'])
         for line in out.split('\n'):
             revision, ref = line.split(' ')
             if (not ref.startswith('refs/heads/') and
                 not ref.startswith('refs/remotes/')):
                 continue
             if ref.startswith('refs/remotes/') and ref.endswith('/HEAD'):
                 continue # ignore remote/HEAD redirects
             branch2rev[ref] = revision
             rev2branch.setdefault(revision, []).append(ref)
         return branch2rev, rev2branch
 
     def _gittracking(self, branches):
         'return map of remote branch to local tracking branch'
         # assumes no more than one local tracking branch for each remote
         tracking = {}
         for b in branches:
             if b.startswith('refs/remotes/'):
                 continue
             bname = b.split('/', 2)[2]
             remote = self._gitcommand(['config', 'branch.%s.remote' % bname])
             if remote:
                 ref = self._gitcommand(['config', 'branch.%s.merge' % bname])
                 tracking['refs/remotes/%s/%s' %
                          (remote, ref.split('/', 2)[2])] = b
         return tracking
 
     def _abssource(self, source):
         if '://' not in source:
             # recognize the scp syntax as an absolute source
             colon = source.find(':')
             if colon != -1 and '/' not in source[:colon]:
                 return source
         self._subsource = source
         return _abssource(self)
 
     def _fetch(self, source, revision):
         if self._gitmissing():
             source = self._abssource(source)
             self.ui.status(_('cloning subrepo %s from %s\n') %
                             (self._relpath, source))
             self._gitnodir(['clone', source, self._abspath])
         if self._githavelocally(revision):
             return
         self.ui.status(_('pulling subrepo %s from %s\n') %
                         (self._relpath, self._gitremote('origin')))
         # try only origin: the originally cloned repo
         self._gitcommand(['fetch'])
         if not self._githavelocally(revision):
             raise util.Abort(_("revision %s does not exist in subrepo %s\n") %
                                (revision, self._relpath))
 
     @annotatesubrepoerror
     def dirty(self, ignoreupdate=False):
         if self._gitmissing():
             return self._state[1] != ''
         if self._gitisbare():
             return True
         if not ignoreupdate and self._state[1] != self._gitstate():
             # different version checked out
             return True
         # check for staged changes or modified files; ignore untracked files
         self._gitupdatestat()
         out, code = self._gitdir(['diff-index', '--quiet', 'HEAD'])
         return code == 1
 
     def basestate(self):
         return self._gitstate()
 
     @annotatesubrepoerror
     def get(self, state, overwrite=False):
         source, revision, kind = state
         if not revision:
             self.remove()
             return
         self._fetch(source, revision)
         # if the repo was set to be bare, unbare it
         if self._gitisbare():
             self._gitcommand(['config', 'core.bare', 'false'])
             if self._gitstate() == revision:
                 self._gitcommand(['reset', '--hard', 'HEAD'])
                 return
         elif self._gitstate() == revision:
             if overwrite:
                 # first reset the index to unmark new files for commit, because
                 # reset --hard will otherwise throw away files added for commit,
                 # not just unmark them.
                 self._gitcommand(['reset', 'HEAD'])
                 self._gitcommand(['reset', '--hard', 'HEAD'])
             return
         branch2rev, rev2branch = self._gitbranchmap()
 
         def checkout(args):
             cmd = ['checkout']
             if overwrite:
                 # first reset the index to unmark new files for commit, because
                 # the -f option will otherwise throw away files added for
                 # commit, not just unmark them.
                 self._gitcommand(['reset', 'HEAD'])
                 cmd.append('-f')
             self._gitcommand(cmd + args)
             _sanitize(self.ui, self._abspath, '.git')
 
         def rawcheckout():
             # no branch to checkout, check it out with no branch
             self.ui.warn(_('checking out detached HEAD in subrepo %s\n') %
                           self._relpath)
             self.ui.warn(_('check out a git branch if you intend '
                             'to make changes\n'))
             checkout(['-q', revision])
 
         if revision not in rev2branch:
             rawcheckout()
             return
         branches = rev2branch[revision]
         firstlocalbranch = None
         for b in branches:
             if b == 'refs/heads/master':
                 # master trumps all other branches
                 checkout(['refs/heads/master'])
                 return
             if not firstlocalbranch and not b.startswith('refs/remotes/'):
                 firstlocalbranch = b
         if firstlocalbranch:
             checkout([firstlocalbranch])
             return
 
         tracking = self._gittracking(branch2rev.keys())
         # choose a remote branch already tracked if possible
         remote = branches[0]
         if remote not in tracking:
             for b in branches:
                 if b in tracking:
                     remote = b
                     break
 
         if remote not in tracking:
             # create a new local tracking branch
             local = remote.split('/', 3)[3]
             checkout(['-b', local, remote])
         elif self._gitisancestor(branch2rev[tracking[remote]], remote):
             # When updating to a tracked remote branch,
             # if the local tracking branch is downstream of it,
             # a normal `git pull` would have performed a "fast-forward merge"
             # which is equivalent to updating the local branch to the remote.
             # Since we are only looking at branching at update, we need to
             # detect this situation and perform this action lazily.
             if tracking[remote] != self._gitcurrentbranch():
                 checkout([tracking[remote]])
             self._gitcommand(['merge', '--ff', remote])
             _sanitize(self.ui, self._abspath, '.git')
         else:
             # a real merge would be required, just checkout the revision
             rawcheckout()
 
     @annotatesubrepoerror
     def commit(self, text, user, date):
         if self._gitmissing():
             raise util.Abort(_("subrepo %s is missing") % self._relpath)
         cmd = ['commit', '-a', '-m', text]
         env = os.environ.copy()
         if user:
             cmd += ['--author', user]
         if date:
             # git's date parser silently ignores when seconds < 1e9
             # convert to ISO8601
             env['GIT_AUTHOR_DATE'] = util.datestr(date,
                                                   '%Y-%m-%dT%H:%M:%S %1%2')
         self._gitcommand(cmd, env=env)
         # make sure commit works otherwise HEAD might not exist under certain
         # circumstances
         return self._gitstate()
 
     @annotatesubrepoerror
     def merge(self, state):
         source, revision, kind = state
         self._fetch(source, revision)
         base = self._gitcommand(['merge-base', revision, self._state[1]])
         self._gitupdatestat()
         out, code = self._gitdir(['diff-index', '--quiet', 'HEAD'])
 
         def mergefunc():
             if base == revision:
                 self.get(state) # fast forward merge
             elif base != self._state[1]:
                 self._gitcommand(['merge', '--no-commit', revision])
             _sanitize(self.ui, self._abspath, '.git')
 
         if self.dirty():
             if self._gitstate() != revision:
                 dirty = self._gitstate() == self._state[1] or code != 0
                 if _updateprompt(self.ui, self, dirty,
                                  self._state[1][:7], revision[:7]):
                     mergefunc()
         else:
             mergefunc()
 
     @annotatesubrepoerror
     def push(self, opts):
         force = opts.get('force')
 
         if not self._state[1]:
             return True
         if self._gitmissing():
             raise util.Abort(_("subrepo %s is missing") % self._relpath)
         # if a branch in origin contains the revision, nothing to do
         branch2rev, rev2branch = self._gitbranchmap()
         if self._state[1] in rev2branch:
             for b in rev2branch[self._state[1]]:
                 if b.startswith('refs/remotes/origin/'):
                     return True
         for b, revision in branch2rev.iteritems():
             if b.startswith('refs/remotes/origin/'):
                 if self._gitisancestor(self._state[1], revision):
                     return True
         # otherwise, try to push the currently checked out branch
         cmd = ['push']
         if force:
             cmd.append('--force')
 
         current = self._gitcurrentbranch()
         if current:
             # determine if the current branch is even useful
             if not self._gitisancestor(self._state[1], current):
                 self.ui.warn(_('unrelated git branch checked out '
                                 'in subrepo %s\n') % self._relpath)
                 return False
             self.ui.status(_('pushing branch %s of subrepo %s\n') %
                            (current.split('/', 2)[2], self._relpath))
             ret = self._gitdir(cmd + ['origin', current])
             return ret[1] == 0
         else:
             self.ui.warn(_('no branch checked out in subrepo %s\n'
                            'cannot push revision %s\n') %
                           (self._relpath, self._state[1]))
             return False
 
     @annotatesubrepoerror
+    def add(self, ui, match, prefix, explicitonly, **opts):
+        if self._gitmissing():
+            return []
+        if match.files():
+            files = match.files()
+        else:
+            (modified, added, removed,
+             deleted, unknown, ignored, clean) = self.status(None)
+            files = unknown
+
+        files = [f for f in files if match(f)]
+        for f in files:
+            exact = match.exact(f)
+            command = ["add"]
+            if exact:
+                command.append("-f") #should be added, even if ignored
+            if ui.verbose or not exact:
+                ui.status(_('adding %s\n') % match.rel(f))
+            if not opts.get('dry_run'):
+                self._gitcommand(command + [f])
+        return []
+
+    @annotatesubrepoerror
     def remove(self):
         if self._gitmissing():
             return
         if self.dirty():
             self.ui.warn(_('not removing repo %s because '
                            'it has changes.\n') % self._relpath)
             return
         # we can't fully delete the repository as it may contain
         # local-only history
         self.ui.note(_('removing subrepo %s\n') % self._relpath)
         self._gitcommand(['config', 'core.bare', 'true'])
         for f in os.listdir(self._abspath):
             if f == '.git':
                 continue
             path = os.path.join(self._abspath, f)
             if os.path.isdir(path) and not os.path.islink(path):
                 shutil.rmtree(path)
             else:
                 os.remove(path)
 
     def archive(self, archiver, prefix, match=None):
         total = 0
         source, revision = self._state
         if not revision:
             return total
         self._fetch(source, revision)
 
         # Parse git's native archive command.
         # This should be much faster than manually traversing the trees
         # and objects with many subprocess calls.
         tarstream = self._gitcommand(['archive', revision], stream=True)
         tar = tarfile.open(fileobj=tarstream, mode='r|')
         relpath = subrelpath(self)
         self.ui.progress(_('archiving (%s)') % relpath, 0, unit=_('files'))
         for i, info in enumerate(tar):
             if info.isdir():
                 continue
             if match and not match(info.name):
                 continue
             if info.issym():
                 data = info.linkname
             else:
                 data = tar.extractfile(info).read()
             archiver.addfile(os.path.join(prefix, self._path, info.name),
                              info.mode, info.issym(), data)
             total += 1
             self.ui.progress(_('archiving (%s)') % relpath, i + 1,
                              unit=_('files'))
         self.ui.progress(_('archiving (%s)') % relpath, None)
         return total
 
 
     @annotatesubrepoerror
     def cat(self, match, prefix, **opts):
         rev = self._state[1]
         if match.anypats():
             return 1 #No support for include/exclude yet
 
         if not match.files():
             return 1
 
         for f in match.files():
             output = self._gitcommand(["show", "%s:%s" % (rev, f)])
             fp = cmdutil.makefileobj(self._subparent, opts.get('output'),
                                      self._ctx.node(),
                                      pathname=os.path.join(prefix, f))
             fp.write(output)
             fp.close()
         return 0
 
 
     @annotatesubrepoerror
     def status(self, rev2, **opts):
         rev1 = self._state[1]
         if self._gitmissing() or not rev1:
             # if the repo is missing, return no results
             return [], [], [], [], [], [], []
         modified, added, removed = [], [], []
         self._gitupdatestat()
         if rev2:
             command = ['diff-tree', rev1, rev2]
         else:
             command = ['diff-index', rev1]
         out = self._gitcommand(command)
         for line in out.split('\n'):
             tab = line.find('\t')
             if tab == -1:
                 continue
             status, f = line[tab - 1], line[tab + 1:]
             if status == 'M':
                 modified.append(f)
             elif status == 'A':
                 added.append(f)
             elif status == 'D':
                 removed.append(f)
 
         deleted, unknown, ignored, clean = [], [], [], []
 
         if not rev2:
             command = ['ls-files', '--others', '--exclude-standard']
             out = self._gitcommand(command)
             for line in out.split('\n'):
                 if len(line) == 0:
                     continue
                 unknown.append(line)
 
         return scmutil.status(modified, added, removed, deleted,
                               unknown, ignored, clean)
 
     @annotatesubrepoerror
     def diff(self, ui, diffopts, node2, match, prefix, **opts):
         node1 = self._state[1]
         cmd = ['diff']
         if opts['stat']:
             cmd.append('--stat')
         else:
             # for Git, this also implies '-p'
             cmd.append('-U%d' % diffopts.context)
 
         gitprefix = os.path.join(prefix, self._path)
 
         if diffopts.noprefix:
             cmd.extend(['--src-prefix=%s/' % gitprefix,
                         '--dst-prefix=%s/' % gitprefix])
         else:
             cmd.extend(['--src-prefix=a/%s/' % gitprefix,
                         '--dst-prefix=b/%s/' % gitprefix])
 
         if diffopts.ignorews:
             cmd.append('--ignore-all-space')
         if diffopts.ignorewsamount:
             cmd.append('--ignore-space-change')
         if self._gitversion(self._gitcommand(['--version'])) >= (1, 8, 4) \
                 and diffopts.ignoreblanklines:
             cmd.append('--ignore-blank-lines')
 
         cmd.append(node1)
         if node2:
             cmd.append(node2)
 
         if match.anypats():
             return #No support for include/exclude yet
 
         output = ""
         if match.always():
             output += self._gitcommand(cmd) + '\n'
         elif match.files():
             for f in match.files():
                 output += self._gitcommand(cmd + [f]) + '\n'
         elif match(gitprefix): #Subrepo is matched
             output += self._gitcommand(cmd) + '\n'
 
         if output.strip():
             ui.write(output)
 
     @annotatesubrepoerror
     def revert(self, substate, *pats, **opts):
         self.ui.status(_('reverting subrepo %s\n') % substate[0])
         if not opts.get('no_backup'):
             status = self.status(None)
             names = status.modified
             for name in names:
                 bakname = "%s.orig" % name
                 self.ui.note(_('saving current version of %s as %s\n') %
                         (name, bakname))
                 util.rename(os.path.join(self._abspath, name),
                             os.path.join(self._abspath, bakname))
 
         if not opts.get('dry_run'):
             self.get(substate, overwrite=True)
         return []
 
     def shortid(self, revid):
         return revid[:7]
 
 types = {
     'hg': hgsubrepo,
     'svn': svnsubrepo,
     'git': gitsubrepo,
     }

tests/test-subrepo-git.t

@@ -1,853 +1,980 b''
 #require git
 
 make git commits repeatable
 
   $ echo "[core]" >> $HOME/.gitconfig
   $ echo "autocrlf = false" >> $HOME/.gitconfig
   $ GIT_AUTHOR_NAME='test'; export GIT_AUTHOR_NAME
   $ GIT_AUTHOR_EMAIL='[e-mail]'; export GIT_AUTHOR_EMAIL
   $ GIT_AUTHOR_DATE='1234567891 +0000'; export GIT_AUTHOR_DATE
   $ GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME"; export GIT_COMMITTER_NAME
   $ GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL"; export GIT_COMMITTER_EMAIL
   $ GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"; export GIT_COMMITTER_DATE
   $ GIT_CONFIG_NOSYSTEM=1; export GIT_CONFIG_NOSYSTEM
 
 root hg repo
 
   $ hg init t
   $ cd t
   $ echo a > a
   $ hg add a
   $ hg commit -m a
   $ cd ..
 
 new external git repo
 
   $ mkdir gitroot
   $ cd gitroot
   $ git init -q
   $ echo g > g
   $ git add g
   $ git commit -q -m g
 
 add subrepo clone
 
   $ cd ../t
   $ echo 's = [git]../gitroot' > .hgsub
   $ git clone -q ../gitroot s
   $ hg add .hgsub
   $ hg commit -m 'new git subrepo'
   $ hg debugsub
   path s
    source   ../gitroot
    revision da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
 
 record a new commit from upstream from a different branch
 
   $ cd ../gitroot
   $ git checkout -q -b testing
   $ echo gg >> g
   $ git commit -q -a -m gg
 
   $ cd ../t/s
   $ git pull -q >/dev/null 2>/dev/null
   $ git checkout -q -b testing origin/testing >/dev/null
 
   $ cd ..
   $ hg status --subrepos
   M s/g
   $ hg commit -m 'update git subrepo'
   $ hg debugsub
   path s
    source   ../gitroot
    revision 126f2a14290cd5ce061fdedc430170e8d39e1c5a
 
 make $GITROOT pushable, by replacing it with a clone with nothing checked out
 
   $ cd ..
   $ git clone gitroot gitrootbare --bare -q
   $ rm -rf gitroot
   $ mv gitrootbare gitroot
 
 clone root
 
   $ cd t
   $ hg clone . ../tc 2> /dev/null
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cd ../tc
   $ hg debugsub
   path s
    source   ../gitroot
    revision 126f2a14290cd5ce061fdedc430170e8d39e1c5a
 
 update to previous substate
 
   $ hg update 1 -q
   $ cat s/g
   g
   $ hg debugsub
   path s
    source   ../gitroot
    revision da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
 
 clone root, make local change
 
   $ cd ../t
   $ hg clone . ../ta 2> /dev/null
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ cd ../ta
   $ echo ggg >> s/g
   $ hg status --subrepos
   M s/g
   $ hg diff --subrepos
   diff --git a/s/g b/s/g
   index 089258f..85341ee 100644
   --- a/s/g
   +++ b/s/g
   @@ -1,2 +1,3 @@
    g
    gg
   +ggg
   $ hg commit --subrepos -m ggg
   committing subrepository s
   $ hg debugsub
   path s
    source   ../gitroot
    revision 79695940086840c99328513acbe35f90fcd55e57
 
 clone root separately, make different local change
 
   $ cd ../t
   $ hg clone . ../tb 2> /dev/null
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ cd ../tb/s
   $ hg status --subrepos
   $ echo f > f
   $ hg status --subrepos
   ? s/f
   $ hg add .
+  adding f
   $ git add f
   $ cd ..
 
   $ hg status --subrepos
   A s/f
   $ hg commit --subrepos -m f
   committing subrepository s
   $ hg debugsub
   path s
    source   ../gitroot
    revision aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
 
 user b push changes
 
   $ hg push 2>/dev/null
   pushing to $TESTTMP/t (glob)
   pushing branch testing of subrepo s
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
 
 user a pulls, merges, commits
 
   $ cd ../ta
   $ hg pull
   pulling from $TESTTMP/t (glob)
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files (+1 heads)
   (run 'hg heads' to see heads, 'hg merge' to merge)
   $ hg merge 2>/dev/null
    subrepository s diverged (local revision: 7969594, remote revision: aa84837)
   (M)erge, keep (l)ocal or keep (r)emote? m
   pulling subrepo s from $TESTTMP/gitroot
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   (branch merge, don't forget to commit)
   $ cat s/f
   f
   $ cat s/g
   g
   gg
   ggg
   $ hg commit --subrepos -m 'merge'
   committing subrepository s
   $ hg status --subrepos --rev 1:5
   M .hgsubstate
   M s/g
   A s/f
   $ hg debugsub
   path s
    source   ../gitroot
    revision f47b465e1bce645dbf37232a00574aa1546ca8d3
   $ hg push 2>/dev/null
   pushing to $TESTTMP/t (glob)
   pushing branch testing of subrepo s
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 2 changesets with 2 changes to 1 files
 
 make upstream git changes
 
   $ cd ..
   $ git clone -q gitroot gitclone
   $ cd gitclone
   $ echo ff >> f
   $ git commit -q -a -m ff
   $ echo fff >> f
   $ git commit -q -a -m fff
   $ git push origin testing 2>/dev/null
 
 make and push changes to hg without updating the subrepo
 
   $ cd ../t
   $ hg clone . ../td 2>&1 | egrep -v '^Cloning into|^done\.'
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   checking out detached HEAD in subrepo s
   check out a git branch if you intend to make changes
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cd ../td
   $ echo aa >> a
   $ hg commit -m aa
   $ hg push
   pushing to $TESTTMP/t (glob)
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
 
 sync to upstream git, distribute changes
 
   $ cd ../ta
   $ hg pull -u -q
   $ cd s
   $ git pull -q >/dev/null 2>/dev/null
   $ cd ..
   $ hg commit -m 'git upstream sync'
   $ hg debugsub
   path s
    source   ../gitroot
    revision 32a343883b74769118bb1d3b4b1fbf9156f4dddc
   $ hg push -q
 
   $ cd ../tb
   $ hg pull -q
   $ hg update 2>/dev/null
   pulling subrepo s from $TESTTMP/gitroot
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg debugsub
   path s
    source   ../gitroot
    revision 32a343883b74769118bb1d3b4b1fbf9156f4dddc
 
 create a new git branch
 
   $ cd s
   $ git checkout -b b2
   Switched to a new branch 'b2'
   $ echo a>a
   $ git add a
   $ git commit -qm 'add a'
   $ cd ..
   $ hg commit -m 'add branch in s'
 
 pulling new git branch should not create tracking branch named 'origin/b2'
 (issue3870)
   $ cd ../td/s
   $ git remote set-url origin $TESTTMP/tb/s
   $ git branch --no-track oldtesting
   $ cd ..
   $ hg pull -q ../tb
   $ hg up
   From $TESTTMP/tb/s
    * [new branch]      b2         -> origin/b2
   Previous HEAD position was f47b465... merge
   Switched to a new branch 'b2'
   pulling subrepo s from $TESTTMP/tb/s
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
 update to a revision without the subrepo, keeping the local git repository
 
   $ cd ../t
   $ hg up 0
   0 files updated, 0 files merged, 2 files removed, 0 files unresolved
   $ ls -a s
   .
   ..
   .git
 
   $ hg up 2
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ ls -a s
   .
   ..
   .git
   g
 
 archive subrepos
 
   $ cd ../tc
   $ hg pull -q
   $ hg archive --subrepos -r 5 ../archive 2>/dev/null
   pulling subrepo s from $TESTTMP/gitroot
   $ cd ../archive
   $ cat s/f
   f
   $ cat s/g
   g
   gg
   ggg
 
   $ hg -R ../tc archive --subrepo -r 5 -X ../tc/**f ../archive_x 2>/dev/null
   $ find ../archive_x | sort | grep -v pax_global_header
   ../archive_x
   ../archive_x/.hg_archival.txt
   ../archive_x/.hgsub
   ../archive_x/.hgsubstate
   ../archive_x/a
   ../archive_x/s
   ../archive_x/s/g
 
 create nested repo
 
   $ cd ..
   $ hg init outer
   $ cd outer
   $ echo b>b
   $ hg add b
   $ hg commit -m b
 
   $ hg clone ../t inner 2> /dev/null
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ echo inner = inner > .hgsub
   $ hg add .hgsub
   $ hg commit -m 'nested sub'
 
 nested commit
 
   $ echo ffff >> inner/s/f
   $ hg status --subrepos
   M inner/s/f
   $ hg commit --subrepos -m nested
   committing subrepository inner
   committing subrepository inner/s (glob)
 
 nested archive
 
   $ hg archive --subrepos ../narchive
   $ ls ../narchive/inner/s | grep -v pax_global_header
   f
   g
 
 relative source expansion
 
   $ cd ..
   $ mkdir d
   $ hg clone t d/t 2> /dev/null
   updating to branch default
   cloning subrepo s from $TESTTMP/gitroot
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
 Don't crash if the subrepo is missing
 
   $ hg clone t missing -q
   $ cd missing
   $ rm -rf s
   $ hg status -S
   $ hg sum | grep commit
   commit: 1 subrepos
   $ hg push -q
   abort: subrepo s is missing (in subrepo s)
   [255]
   $ hg commit --subrepos -qm missing
   abort: subrepo s is missing (in subrepo s)
   [255]
   $ hg update -C 2> /dev/null
   cloning subrepo s from $TESTTMP/gitroot
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg sum | grep commit
   commit: (clean)
 
 Don't crash if the .hgsubstate entry is missing
 
   $ hg update 1 -q
   $ hg rm .hgsubstate
   $ hg commit .hgsubstate -m 'no substate'
   nothing changed
   [1]
   $ hg tag -l nosubstate
   $ hg manifest
   .hgsub
   .hgsubstate
   a
 
   $ hg status -S
   R .hgsubstate
   $ hg sum | grep commit
   commit: 1 removed, 1 subrepos (new branch head)
 
   $ hg commit -m 'restore substate'
   nothing changed
   [1]
   $ hg manifest
   .hgsub
   .hgsubstate
   a
   $ hg sum | grep commit
   commit: 1 removed, 1 subrepos (new branch head)
 
   $ hg update -qC nosubstate
   $ ls s
   g
 
 issue3109: false positives in git diff-index
 
   $ hg update -q
   $ touch -t 200001010000 s/g
   $ hg status --subrepos
   $ touch -t 200001010000 s/g
   $ hg sum | grep commit
   commit: (clean)
 
 Check hg update --clean
   $ cd $TESTTMP/ta
   $ echo  > s/g
   $ cd s
   $ echo c1 > f1
   $ echo c1 > f2
   $ git add f1
   $ cd ..
   $ hg status -S
   M s/g
   A s/f1
   ? s/f2
   $ ls s
   f
   f1
   f2
   g
   $ hg update --clean
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg status -S
   ? s/f1
   ? s/f2
   $ ls s
   f
   f1
   f2
   g
 
 Sticky subrepositories, no changes
   $ cd $TESTTMP/ta
   $ hg id -n
   7
   $ cd s
   $ git rev-parse HEAD
   32a343883b74769118bb1d3b4b1fbf9156f4dddc
   $ cd ..
   $ hg update 1 > /dev/null 2>&1
   $ hg id -n
   1
   $ cd s
   $ git rev-parse HEAD
   da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
   $ cd ..
 
 Sticky subrepositories, file changes
   $ touch s/f1
   $ cd s
   $ git add f1
   $ cd ..
   $ hg id -n
   1+
   $ cd s
   $ git rev-parse HEAD
   da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
   $ cd ..
   $ hg update 4
    subrepository s diverged (local revision: da5f5b1, remote revision: aa84837)
   (M)erge, keep (l)ocal or keep (r)emote? m
    subrepository sources for s differ
   use (l)ocal source (da5f5b1) or (r)emote source (aa84837)? l
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n
   4+
   $ cd s
   $ git rev-parse HEAD
   da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
   $ cd ..
   $ hg update --clean tip > /dev/null 2>&1
 
 Sticky subrepository, revision updates
   $ hg id -n
   7
   $ cd s
   $ git rev-parse HEAD
   32a343883b74769118bb1d3b4b1fbf9156f4dddc
   $ cd ..
   $ cd s
   $ git checkout aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
   Previous HEAD position was 32a3438... fff
   HEAD is now at aa84837... f
   $ cd ..
   $ hg update 1
    subrepository s diverged (local revision: 32a3438, remote revision: da5f5b1)
   (M)erge, keep (l)ocal or keep (r)emote? m
    subrepository sources for s differ (in checked out version)
   use (l)ocal source (32a3438) or (r)emote source (da5f5b1)? l
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n
   1+
   $ cd s
   $ git rev-parse HEAD
   aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
   $ cd ..
 
 Sticky subrepository, file changes and revision updates
   $ touch s/f1
   $ cd s
   $ git add f1
   $ git rev-parse HEAD
   aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
   $ cd ..
   $ hg id -n
   1+
   $ hg update 7
    subrepository s diverged (local revision: 32a3438, remote revision: 32a3438)
   (M)erge, keep (l)ocal or keep (r)emote? m
    subrepository sources for s differ
   use (l)ocal source (32a3438) or (r)emote source (32a3438)? l
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n
   7+
   $ cd s
   $ git rev-parse HEAD
   aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
   $ cd ..
 
 Sticky repository, update --clean
   $ hg update --clean tip 2>/dev/null
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n
   7
   $ cd s
   $ git rev-parse HEAD
   32a343883b74769118bb1d3b4b1fbf9156f4dddc
   $ cd ..
 
 Test subrepo already at intended revision:
   $ cd s
   $ git checkout 32a343883b74769118bb1d3b4b1fbf9156f4dddc
   HEAD is now at 32a3438... fff
   $ cd ..
   $ hg update 1
   Previous HEAD position was 32a3438... fff
   HEAD is now at da5f5b1... g
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n
   1
   $ cd s
   $ git rev-parse HEAD
   da5f5b1d8ffcf62fb8327bcd3c89a4367a6018e7
   $ cd ..
 
 Test forgetting files, not implemented in git subrepo, used to
 traceback
 #if no-windows
   $ hg forget 'notafile*'
   notafile*: No such file or directory
   [1]
 #else
   $ hg forget 'notafile'
   notafile: * (glob)
   [1]
 #endif
 
   $ cd ..
 
 Test sanitizing ".hg/hgrc" in subrepo
 
   $ cd t
   $ hg tip -q
   7:af6d2edbb0d3
   $ hg update -q -C af6d2edbb0d3
   $ cd s
   $ git checkout -q -b sanitize-test
   $ mkdir .hg
   $ echo '.hg/hgrc in git repo' > .hg/hgrc
   $ mkdir -p sub/.hg
   $ echo 'sub/.hg/hgrc in git repo' > sub/.hg/hgrc
   $ git add .hg sub
   $ git commit -qm 'add .hg/hgrc to be sanitized at hg update'
   $ git push -q origin sanitize-test
   $ cd ..
   $ grep ' s$' .hgsubstate
   32a343883b74769118bb1d3b4b1fbf9156f4dddc s
   $ hg commit -qm 'commit with git revision including .hg/hgrc'
   $ hg parents -q
   8:3473d20bddcf
   $ grep ' s$' .hgsubstate
   c4069473b459cf27fd4d7c2f50c4346b4e936599 s
   $ cd ..
 
   $ hg -R tc pull -q
   $ hg -R tc update -q -C 3473d20bddcf 2>&1 | sort
   warning: removing potentially hostile 'hgrc' in '$TESTTMP/tc/s/.hg' (glob)
   warning: removing potentially hostile 'hgrc' in '$TESTTMP/tc/s/sub/.hg' (glob)
   $ cd tc
   $ hg parents -q
   8:3473d20bddcf
   $ grep ' s$' .hgsubstate
   c4069473b459cf27fd4d7c2f50c4346b4e936599 s
   $ test -f s/.hg/hgrc
   [1]
   $ test -f s/sub/.hg/hgrc
   [1]
   $ cd ..
 
 additional test for "git merge --ff" route:
 
   $ cd t
   $ hg tip -q
   8:3473d20bddcf
   $ hg update -q -C af6d2edbb0d3
   $ cd s
   $ git checkout -q testing
   $ mkdir .hg
   $ echo '.hg/hgrc in git repo' > .hg/hgrc
   $ mkdir -p sub/.hg
   $ echo 'sub/.hg/hgrc in git repo' > sub/.hg/hgrc
   $ git add .hg sub
   $ git commit -qm 'add .hg/hgrc to be sanitized at hg update (git merge --ff)'
   $ git push -q origin testing
   $ cd ..
   $ grep ' s$' .hgsubstate
   32a343883b74769118bb1d3b4b1fbf9156f4dddc s
   $ hg commit -qm 'commit with git revision including .hg/hgrc'
   $ hg parents -q
   9:ed23f7fe024e
   $ grep ' s$' .hgsubstate
   f262643c1077219fbd3858d54e78ef050ef84fbf s
   $ cd ..
 
   $ cd tc
   $ hg update -q -C af6d2edbb0d3
   $ test -f s/.hg/hgrc
   [1]
   $ test -f s/sub/.hg/hgrc
   [1]
   $ cd ..
   $ hg -R tc pull -q
   $ hg -R tc update -q -C ed23f7fe024e 2>&1 | sort
   warning: removing potentially hostile 'hgrc' in '$TESTTMP/tc/s/.hg' (glob)
   warning: removing potentially hostile 'hgrc' in '$TESTTMP/tc/s/sub/.hg' (glob)
   $ cd tc
   $ hg parents -q
   9:ed23f7fe024e
   $ grep ' s$' .hgsubstate
   f262643c1077219fbd3858d54e78ef050ef84fbf s
   $ test -f s/.hg/hgrc
   [1]
   $ test -f s/sub/.hg/hgrc
   [1]
 
 Test that sanitizing is omitted in meta data area:
 
   $ mkdir s/.git/.hg
   $ echo '.hg/hgrc in git metadata area' > s/.git/.hg/hgrc
   $ hg update -q -C af6d2edbb0d3
   checking out detached HEAD in subrepo s
   check out a git branch if you intend to make changes
 
 check differences made by most recent change
   $ cd s
   $ cat > foobar << EOF
   > woopwoop
   > 
   > foo
   > bar
   > EOF
   $ git add foobar
   $ cd ..
 
   $ hg diff --subrepos
   diff --git a/s/foobar b/s/foobar
   new file mode 100644
   index 0000000..8a5a5e2
   --- /dev/null
   +++ b/s/foobar
   @@ -0,0 +1,4 @@
   +woopwoop
   +
   +foo
   +bar
 
   $ hg commit --subrepos -m "Added foobar"
   committing subrepository s
   created new head
 
   $ hg diff -c . --subrepos --nodates
   diff -r af6d2edbb0d3 -r 255ee8cf690e .hgsubstate
   --- a/.hgsubstate
   +++ b/.hgsubstate
   @@ -1,1 +1,1 @@
   -32a343883b74769118bb1d3b4b1fbf9156f4dddc s
   +fd4dbf828a5b2fcd36b2bcf21ea773820970d129 s
   diff --git a/s/foobar b/s/foobar
   new file mode 100644
   index 0000000..8a5a5e2
   --- /dev/null
   +++ b/s/foobar
   @@ -0,0 +1,4 @@
   +woopwoop
   +
   +foo
   +bar
 
 check output when only diffing the subrepository
   $ hg diff -c . --subrepos s
   diff --git a/s/foobar b/s/foobar
   new file mode 100644
   index 0000000..8a5a5e2
   --- /dev/null
   +++ b/s/foobar
   @@ -0,0 +1,4 @@
   +woopwoop
   +
   +foo
   +bar
 
 check output when diffing something else
   $ hg diff -c . --subrepos .hgsubstate --nodates
   diff -r af6d2edbb0d3 -r 255ee8cf690e .hgsubstate
   --- a/.hgsubstate
   +++ b/.hgsubstate
   @@ -1,1 +1,1 @@
   -32a343883b74769118bb1d3b4b1fbf9156f4dddc s
   +fd4dbf828a5b2fcd36b2bcf21ea773820970d129 s
 
 add new changes, including whitespace
   $ cd s
   $ cat > foobar << EOF
   > woop    woop
   > 
   > foo
   > bar
   > EOF
   $ echo foo > barfoo
   $ git add barfoo
   $ cd ..
 
   $ hg diff --subrepos --ignore-all-space
   diff --git a/s/barfoo b/s/barfoo
   new file mode 100644
   index 0000000..257cc56
   --- /dev/null
   +++ b/s/barfoo
   @@ -0,0 +1 @@
   +foo
   $ hg diff --subrepos s/foobar
   diff --git a/s/foobar b/s/foobar
   index 8a5a5e2..bd5812a 100644
   --- a/s/foobar
   +++ b/s/foobar
   @@ -1,4 +1,4 @@
   -woopwoop
   +woop    woop
    
    foo
    bar
 
 execute a diffstat
 the output contains a regex, because git 1.7.10 and 1.7.11
  change the amount of whitespace
   $ hg diff --subrepos --stat
   \s*barfoo |\s*1 + (re)
   \s*foobar |\s*2 +- (re)
    2 files changed, 2 insertions\(\+\), 1 deletions?\(-\) (re)
 
 ensure adding include/exclude ignores the subrepo
   $ hg diff --subrepos -I s/foobar
   $ hg diff --subrepos -X s/foobar
 
 revert the subrepository
   $ hg revert --all
   reverting subrepo ../gitroot
 
   $ hg status --subrepos
   ? s/barfoo
   ? s/foobar.orig
 
   $ mv s/foobar.orig s/foobar
 
   $ hg revert --no-backup s
   reverting subrepo ../gitroot
 
   $ hg status --subrepos
   ? s/barfoo
 
 show file at specific revision
   $ cat > s/foobar << EOF
   > woop    woop
   > fooo bar
   > EOF
   $ hg commit --subrepos -m "updated foobar"
   committing subrepository s
   $ cat > s/foobar << EOF
   > current foobar
   > (should not be visible using hg cat)
   > EOF
 
   $ hg cat -r . s/foobar
   woop    woop
   fooo bar (no-eol)
   $ hg cat -r "parents(.)" s/foobar > catparents
 
   $ mkdir -p tmp/s
 
   $ hg cat -r "parents(.)" --output tmp/%% s/foobar
   $ diff tmp/% catparents
 
   $ hg cat -r "parents(.)" --output tmp/%s s/foobar
   $ diff tmp/foobar catparents
 
   $ hg cat -r "parents(.)" --output tmp/%d/otherfoobar s/foobar
   $ diff tmp/s/otherfoobar catparents
 
   $ hg cat -r "parents(.)" --output tmp/%p s/foobar
   $ diff tmp/s/foobar catparents
 
   $ hg cat -r "parents(.)" --output tmp/%H s/foobar
   $ diff tmp/255ee8cf690ec86e99b1e80147ea93ece117cd9d catparents
 
   $ hg cat -r "parents(.)" --output tmp/%R s/foobar
   $ diff tmp/10 catparents
 
   $ hg cat -r "parents(.)" --output tmp/%h s/foobar
   $ diff tmp/255ee8cf690e catparents
 
   $ rm tmp/10
   $ hg cat -r "parents(.)" --output tmp/%r s/foobar
   $ diff tmp/10 catparents
 
   $ mkdir tmp/tc
   $ hg cat -r "parents(.)" --output tmp/%b/foobar s/foobar
   $ diff tmp/tc/foobar catparents
 
+cleanup
+  $ rm -r tmp
+  $ rm catparents
+
+add git files, using either files or patterns
+  $ echo "hsss! hsssssssh!" > s/snake.python
+  $ echo "ccc" > s/c.c
+  $ echo "cpp" > s/cpp.cpp
+
+  $ hg add s/snake.python s/c.c s/cpp.cpp
+  $ hg st --subrepos s
+  M s/foobar
+  A s/c.c
+  A s/cpp.cpp
+  A s/snake.python
+  ? s/barfoo
+  $ hg revert s
+  reverting subrepo ../gitroot
+
+  $ hg add --subrepos "glob:**.python"
+  adding s/snake.python (glob)
+  $ hg st --subrepos s
+  A s/snake.python
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  $ hg revert s
+  reverting subrepo ../gitroot
+
+  $ hg add --subrepos s
+  adding s/barfoo (glob)
+  adding s/c.c (glob)
+  adding s/cpp.cpp (glob)
+  adding s/foobar.orig (glob)
+  adding s/snake.python (glob)
+  $ hg st --subrepos s
+  A s/barfoo
+  A s/c.c
+  A s/cpp.cpp
+  A s/foobar.orig
+  A s/snake.python
+  $ hg revert s
+  reverting subrepo ../gitroot
+make sure everything is reverted correctly
+  $ hg st --subrepos s
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  ? s/snake.python
+
+  $ hg add --subrepos --exclude "path:s/c.c"
+  adding s/barfoo (glob)
+  adding s/cpp.cpp (glob)
+  adding s/foobar.orig (glob)
+  adding s/snake.python (glob)
+  $ hg st --subrepos s
+  A s/barfoo
+  A s/cpp.cpp
+  A s/foobar.orig
+  A s/snake.python
+  ? s/c.c
+  $ hg revert --all -q
+
+.hgignore should not have influence in subrepos
+  $ cat > .hgignore << EOF
+  > syntax: glob
+  > *.python
+  > EOF
+  $ hg add .hgignore
+  $ hg add --subrepos "glob:**.python"
+  adding s/snake.python (glob)
+  $ hg st --subrepos s
+  A s/snake.python
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  $ hg revert --all -q
+
+.gitignore should have influence,
+except for explicitly added files (no patterns)
+  $ cat > s/.gitignore << EOF
+  > *.python
+  > EOF
+  $ hg add s/.gitignore
+  $ hg st --subrepos s
+  A s/.gitignore
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  $ hg add --subrepos "glob:**.python"
+  $ hg st --subrepos s
+  A s/.gitignore
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  $ hg add --subrepos s/snake.python
+  $ hg st --subrepos s
+  A s/.gitignore
+  A s/snake.python
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+
+correctly do a dry run
+  $ hg add --subrepos s --dry-run
+  adding s/barfoo (glob)
+  adding s/c.c (glob)
+  adding s/cpp.cpp (glob)
+  adding s/foobar.orig (glob)
+  $ hg st --subrepos s
+  A s/.gitignore
+  A s/snake.python
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+
+currently no error given when adding an already tracked file
+  $ hg add s/.gitignore
+
   $ cd ..