repoview: discard filtered changelog if index isn't shared with unfiltered

Before this patch, revisions rollbacked at failure of previous
transaction might be visible at subsequent operations unintentionally,
if repoview object is reused even after failure of transaction:
e.g. command server and HTTP server are typical cases.

'repoview' uses the tuple of values below of unfiltered changelog as
"the key" to examine validity of filtered changelog cache.

     - length
     - tip node
     - filtered revisions (as hashed value)
     - '_delayed' field

'repoview' compares between "the key" of unfiltered changelog at
previous caching and now, and reuses filtered changelog cache if no
change is detected.

But this comparison indicates only that there is no change between
unfiltered 'repo.changelog' at last caching and now, but not that
filtered changelog cache is valid for current unfiltered one.

'repoview' uses "shallow copy" of unfiltered changelog to create
filtered changelog cache. In this case, 'index' buffer of unfiltered
changelog is also referred by filtered changelog.

At failure of transaction, unfiltered changelog itself is invalidated
(= un-referred) on the 'repo' side (see 0a7610758c42 also). But
'index' of it still contains revisions to be rollbacked at this
failure, and is referred by filtered changelog.

Therefore, even if there is no change between unfiltered
'repo.changelog' at last caching and now, steps below makes rollbacked
revisions visible via filtered changelog unintentionally.

  1. instantiate unfiltered changelog as 'repo.changelog'
     (call it CL1)

  2. make filtered (= shallow copy of) CL1
     (call it FCL1)

  3. cache FCL1 with "the key" of CL1

  4. revisions are appended to 'index', which is shared by CL1 and FCL1

  5. invalidate 'repo.changelog' (= CL1) at failure of transaction

  6. instantiate 'repo.changelog' again at next operation
     (call it CL2)

     CL2 doesn't have revisions added at (4), because it is
     instantiated from '00changelog.i', which isn't changed while
     failed transaction.

  7. compare between "the key" of CL1 and CL2

  8. FCL1 cached at (3) is reused, because comparison at (7) doesn't
     detect change between CL1 at (1) and CL2

  9. revisions rollbacked at (5) are visible via FCL1 unintentionally,
     because FCL1 still refers 'index' changed at (4)

The root cause of this issue is that there is no examination about
validity of filtered changelog cache against current unfiltered one.

This patch discards filtered changelog cache, if its 'index' object
isn't shared with unfiltered one.

BTW, at the time of this patch, redundant truncation of
'00changelog.i' at failure of transaction (see 0a7610758c42 for
detail) often prevents "hg serve" from making already rollbacked
revisions visible, because updating timestamps of '00changelog.i' by
truncation makes "hg serve" discard old repoview object with invalid
filtered changelog cache.

This is reason why this issue is overlooked before this patch, even
though test-bundle2-exchange.t has tests in similar situation: failure
of "hg push" via HTTP by pretxnclose hook on server side doesn't
prevent subsequent commands from looking up outgoing revisions
correctly.

But timestamp on the filesystem doesn't have enough resolution for
recent computation power, and it can't be assumed that this avoidance
always works as expected.

Therefore, without this patch, this issue might appear occasionally.

mercurial/repoview.py

@@ -1,353 +1,356 b''
 # repoview.py - Filtered view of a localrepo object
 #
 # Copyright 2012 Pierre-Yves David <[e-mail]>
 #                Logilab SA        <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 from __future__ import absolute_import
 
 import copy
 import heapq
 import struct
 
 from .node import nullrev
 from . import (
     error,
     obsolete,
     phases,
     tags as tagsmod,
     util,
 )
 
 def hideablerevs(repo):
     """Revisions candidates to be hidden
 
     This is a standalone function to help extensions to wrap it."""
     return obsolete.getrevs(repo, 'obsolete')
 
 def _getstatichidden(repo):
     """Revision to be hidden (disregarding dynamic blocker)
 
     To keep a consistent graph, we cannot hide any revisions with
     non-hidden descendants. This function computes the set of
     revisions that could be hidden while keeping the graph consistent.
 
     A second pass will be done to apply "dynamic blocker" like bookmarks or
     working directory parents.
 
     """
     assert not repo.changelog.filteredrevs
     hidden = set(hideablerevs(repo))
     if hidden:
         getphase = repo._phasecache.phase
         getparentrevs = repo.changelog.parentrevs
         # Skip heads which are public (guaranteed to not be hidden)
         heap = [-r for r in repo.changelog.headrevs() if getphase(repo, r)]
         heapq.heapify(heap)
         heappop = heapq.heappop
         heappush = heapq.heappush
         seen = set() # no need to init it with heads, they have no children
         while heap:
             rev = -heappop(heap)
             # All children have been processed so at that point, if no children
             # removed 'rev' from the 'hidden' set, 'rev' is going to be hidden.
             blocker = rev not in hidden
             for parent in getparentrevs(rev):
                 if parent == nullrev:
                     continue
                 if blocker:
                     # If visible, ensure parent will be visible too
                     hidden.discard(parent)
                 # - Avoid adding the same revision twice
                 # - Skip nodes which are public (guaranteed to not be hidden)
                 pre = len(seen)
                 seen.add(parent)
                 if pre < len(seen) and getphase(repo, rev):
                     heappush(heap, -parent)
     return hidden
 
 def _getdynamicblockers(repo):
     """Non-cacheable revisions blocking hidden changesets from being filtered.
 
     Get revisions that will block hidden changesets and are likely to change,
     but unlikely to create hidden blockers. They won't be cached, so be careful
     with adding additional computation."""
 
     cl = repo.changelog
     blockers = set()
     blockers.update([par.rev() for par in repo[None].parents()])
     blockers.update([cl.rev(bm) for bm in repo._bookmarks.values()])
 
     tags = {}
     tagsmod.readlocaltags(repo.ui, repo, tags, {})
     if tags:
         rev, nodemap = cl.rev, cl.nodemap
         blockers.update(rev(t[0]) for t in tags.values() if t[0] in nodemap)
     return blockers
 
 cacheversion = 1
 cachefile = 'cache/hidden'
 
 def cachehash(repo, hideable):
     """return sha1 hash of repository data to identify a valid cache.
 
     We calculate a sha1 of repo heads and the content of the obsstore and write
     it to the cache. Upon reading we can easily validate by checking the hash
     against the stored one and discard the cache in case the hashes don't match.
     """
     h = util.sha1()
     h.update(''.join(repo.heads()))
     h.update(str(hash(frozenset(hideable))))
     return h.digest()
 
 def _writehiddencache(cachefile, cachehash, hidden):
     """write hidden data to a cache file"""
     data = struct.pack('>%ii' % len(hidden), *sorted(hidden))
     cachefile.write(struct.pack(">H", cacheversion))
     cachefile.write(cachehash)
     cachefile.write(data)
 
 def trywritehiddencache(repo, hideable, hidden):
     """write cache of hidden changesets to disk
 
     Will not write the cache if a wlock cannot be obtained lazily.
     The cache consists of a head of 22byte:
        2 byte    version number of the cache
       20 byte    sha1 to validate the cache
      n*4 byte    hidden revs
     """
     wlock = fh = None
     try:
         wlock = repo.wlock(wait=False)
         # write cache to file
         newhash = cachehash(repo, hideable)
         fh = repo.vfs.open(cachefile, 'w+b', atomictemp=True)
         _writehiddencache(fh, newhash, hidden)
     except (IOError, OSError):
         repo.ui.debug('error writing hidden changesets cache\n')
     except error.LockHeld:
         repo.ui.debug('cannot obtain lock to write hidden changesets cache\n')
     finally:
         if fh:
             fh.close()
         if wlock:
             wlock.release()
 
 def tryreadcache(repo, hideable):
     """read a cache if the cache exists and is valid, otherwise returns None."""
     hidden = fh = None
     try:
         if repo.vfs.exists(cachefile):
             fh = repo.vfs.open(cachefile, 'rb')
             version, = struct.unpack(">H", fh.read(2))
             oldhash = fh.read(20)
             newhash = cachehash(repo, hideable)
             if (cacheversion, oldhash) == (version, newhash):
                 # cache is valid, so we can start reading the hidden revs
                 data = fh.read()
                 count = len(data) / 4
                 hidden = frozenset(struct.unpack('>%ii' % count, data))
         return hidden
     except struct.error:
         repo.ui.debug('corrupted hidden cache\n')
         # No need to fix the content as it will get rewritten
         return None
     except (IOError, OSError):
         repo.ui.debug('cannot read hidden cache\n')
         return None
     finally:
         if fh:
             fh.close()
 
 def computehidden(repo):
     """compute the set of hidden revision to filter
 
     During most operation hidden should be filtered."""
     assert not repo.changelog.filteredrevs
 
     hidden = frozenset()
     hideable = hideablerevs(repo)
     if hideable:
         cl = repo.changelog
         hidden = tryreadcache(repo, hideable)
         if hidden is None:
             hidden = frozenset(_getstatichidden(repo))
             trywritehiddencache(repo, hideable, hidden)
 
         # check if we have wd parents, bookmarks or tags pointing to hidden
         # changesets and remove those.
         dynamic = hidden & _getdynamicblockers(repo)
         if dynamic:
             blocked = cl.ancestors(dynamic, inclusive=True)
             hidden = frozenset(r for r in hidden if r not in blocked)
     return hidden
 
 def computeunserved(repo):
     """compute the set of revision that should be filtered when used a server
 
     Secret and hidden changeset should not pretend to be here."""
     assert not repo.changelog.filteredrevs
     # fast path in simple case to avoid impact of non optimised code
     hiddens = filterrevs(repo, 'visible')
     if phases.hassecret(repo):
         cl = repo.changelog
         secret = phases.secret
         getphase = repo._phasecache.phase
         first = min(cl.rev(n) for n in repo._phasecache.phaseroots[secret])
         revs = cl.revs(start=first)
         secrets = set(r for r in revs if getphase(repo, r) >= secret)
         return frozenset(hiddens | secrets)
     else:
         return hiddens
 
 def computemutable(repo):
     """compute the set of revision that should be filtered when used a server
 
     Secret and hidden changeset should not pretend to be here."""
     assert not repo.changelog.filteredrevs
     # fast check to avoid revset call on huge repo
     if any(repo._phasecache.phaseroots[1:]):
         getphase = repo._phasecache.phase
         maymutable = filterrevs(repo, 'base')
         return frozenset(r for r in maymutable if getphase(repo, r))
     return frozenset()
 
 def computeimpactable(repo):
     """Everything impactable by mutable revision
 
     The immutable filter still have some chance to get invalidated. This will
     happen when:
 
     - you garbage collect hidden changeset,
     - public phase is moved backward,
     - something is changed in the filtering (this could be fixed)
 
     This filter out any mutable changeset and any public changeset that may be
     impacted by something happening to a mutable revision.
 
     This is achieved by filtered everything with a revision number egal or
     higher than the first mutable changeset is filtered."""
     assert not repo.changelog.filteredrevs
     cl = repo.changelog
     firstmutable = len(cl)
     for roots in repo._phasecache.phaseroots[1:]:
         if roots:
             firstmutable = min(firstmutable, min(cl.rev(r) for r in roots))
     # protect from nullrev root
     firstmutable = max(0, firstmutable)
     return frozenset(xrange(firstmutable, len(cl)))
 
 # function to compute filtered set
 #
 # When adding a new filter you MUST update the table at:
 #     mercurial.branchmap.subsettable
 # Otherwise your filter will have to recompute all its branches cache
 # from scratch (very slow).
 filtertable = {'visible': computehidden,
                'served': computeunserved,
                'immutable':  computemutable,
                'base':  computeimpactable}
 
 def filterrevs(repo, filtername):
     """returns set of filtered revision for this filter name"""
     if filtername not in repo.filteredrevcache:
         func = filtertable[filtername]
         repo.filteredrevcache[filtername] = func(repo.unfiltered())
     return repo.filteredrevcache[filtername]
 
 class repoview(object):
     """Provide a read/write view of a repo through a filtered changelog
 
     This object is used to access a filtered version of a repository without
     altering the original repository object itself. We can not alter the
     original object for two main reasons:
     - It prevents the use of a repo with multiple filters at the same time. In
       particular when multiple threads are involved.
     - It makes scope of the filtering harder to control.
 
     This object behaves very closely to the original repository. All attribute
     operations are done on the original repository:
     - An access to `repoview.someattr` actually returns `repo.someattr`,
     - A write to `repoview.someattr` actually sets value of `repo.someattr`,
     - A deletion of `repoview.someattr` actually drops `someattr`
       from `repo.__dict__`.
 
     The only exception is the `changelog` property. It is overridden to return
     a (surface) copy of `repo.changelog` with some revisions filtered. The
     `filtername` attribute of the view control the revisions that need to be
     filtered.  (the fact the changelog is copied is an implementation detail).
 
     Unlike attributes, this object intercepts all method calls. This means that
     all methods are run on the `repoview` object with the filtered `changelog`
     property. For this purpose the simple `repoview` class must be mixed with
     the actual class of the repository. This ensures that the resulting
     `repoview` object have the very same methods than the repo object. This
     leads to the property below.
 
         repoview.method() --> repo.__class__.method(repoview)
 
     The inheritance has to be done dynamically because `repo` can be of any
     subclasses of `localrepo`. Eg: `bundlerepo` or `statichttprepo`.
     """
 
     def __init__(self, repo, filtername):
         object.__setattr__(self, '_unfilteredrepo', repo)
         object.__setattr__(self, 'filtername', filtername)
         object.__setattr__(self, '_clcachekey', None)
         object.__setattr__(self, '_clcache', None)
 
     # not a propertycache on purpose we shall implement a proper cache later
     @property
     def changelog(self):
         """return a filtered version of the changeset
 
         this changelog must not be used for writing"""
         # some cache may be implemented later
         unfi = self._unfilteredrepo
         unfichangelog = unfi.changelog
         # bypass call to changelog.method
         unfiindex = unfichangelog.index
         unfilen = len(unfiindex) - 1
         unfinode = unfiindex[unfilen - 1][7]
 
         revs = filterrevs(unfi, self.filtername)
         cl = self._clcache
         newkey = (unfilen, unfinode, hash(revs), unfichangelog._delayed)
-        if cl is not None and newkey != self._clcachekey:
+        # if cl.index is not unfiindex, unfi.changelog would be
+        # recreated, and our clcache refers to garbage object
+        if (cl is not None and
+            (cl.index is not unfiindex or newkey != self._clcachekey)):
             cl = None
         # could have been made None by the previous if
         if cl is None:
             cl = copy.copy(unfichangelog)
             cl.filteredrevs = revs
             object.__setattr__(self, '_clcache', cl)
             object.__setattr__(self, '_clcachekey', newkey)
         return cl
 
     def unfiltered(self):
         """Return an unfiltered version of a repo"""
         return self._unfilteredrepo
 
     def filtered(self, name):
         """Return a filtered version of a repository"""
         if name == self.filtername:
             return self
         return self.unfiltered().filtered(name)
 
     # everything access are forwarded to the proxied repo
     def __getattr__(self, attr):
         return getattr(self._unfilteredrepo, attr)
 
     def __setattr__(self, attr, value):
         return setattr(self._unfilteredrepo, attr, value)
 
     def __delattr__(self, attr):
         return delattr(self._unfilteredrepo, attr)
 
     # The `requirements` attribute is initialized during __init__. But
     # __getattr__ won't be called as it also exists on the class. We need
     # explicit forwarding to main repo here
     @property
     def requirements(self):
         return self._unfilteredrepo.requirements

tests/test-commandserver.t

@@ -1,727 +1,857 b''
 #if windows
   $ PYTHONPATH="$TESTDIR/../contrib;$PYTHONPATH"
 #else
   $ PYTHONPATH="$TESTDIR/../contrib:$PYTHONPATH"
 #endif
   $ export PYTHONPATH
 
 typical client does not want echo-back messages, so test without it:
 
   $ grep -v '^promptecho ' < $HGRCPATH >> $HGRCPATH.new
   $ mv $HGRCPATH.new $HGRCPATH
 
   $ hg init repo
   $ cd repo
 
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def hellomessage(server):
   ...     ch, data = readchannel(server)
   ...     print '%c, %r' % (ch, data)
   ...     # run an arbitrary command to make sure the next thing the server
   ...     # sends isn't part of the hello message
   ...     runcommand(server, ['id'])
   o, 'capabilities: getencoding runcommand\nencoding: *\npid: *' (glob)
   *** runcommand id
   000000000000 tip
 
   >>> from hgclient import check
   >>> @check
   ... def unknowncommand(server):
   ...     server.stdin.write('unknowncommand\n')
   abort: unknown command unknowncommand
 
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def checkruncommand(server):
   ...     # hello block
   ...     readchannel(server)
   ... 
   ...     # no args
   ...     runcommand(server, [])
   ... 
   ...     # global options
   ...     runcommand(server, ['id', '--quiet'])
   ... 
   ...     # make sure global options don't stick through requests
   ...     runcommand(server, ['id'])
   ... 
   ...     # --config
   ...     runcommand(server, ['id', '--config', 'ui.quiet=True'])
   ... 
   ...     # make sure --config doesn't stick
   ...     runcommand(server, ['id'])
   ... 
   ...     # negative return code should be masked
   ...     runcommand(server, ['id', '-runknown'])
   *** runcommand 
   Mercurial Distributed SCM
   
   basic commands:
   
    add           add the specified files on the next commit
    annotate      show changeset information by line for each file
    clone         make a copy of an existing repository
    commit        commit the specified files or all outstanding changes
    diff          diff repository (or selected files)
    export        dump the header and diffs for one or more changesets
    forget        forget the specified files on the next commit
    init          create a new repository in the given directory
    log           show revision history of entire repository or files
    merge         merge another revision into working directory
    pull          pull changes from the specified source
    push          push changes to the specified destination
    remove        remove the specified files on the next commit
    serve         start stand-alone webserver
    status        show changed files in the working directory
    summary       summarize working directory state
    update        update working directory (or switch revisions)
   
   (use "hg help" for the full list of commands or "hg -v" for details)
   *** runcommand id --quiet
   000000000000
   *** runcommand id
   000000000000 tip
   *** runcommand id --config ui.quiet=True
   000000000000
   *** runcommand id
   000000000000 tip
   *** runcommand id -runknown
   abort: unknown revision 'unknown'!
    [255]
 
   >>> from hgclient import readchannel, check
   >>> @check
   ... def inputeof(server):
   ...     readchannel(server)
   ...     server.stdin.write('runcommand\n')
   ...     # close stdin while server is waiting for input
   ...     server.stdin.close()
   ... 
   ...     # server exits with 1 if the pipe closed while reading the command
   ...     print 'server exit code =', server.wait()
   server exit code = 1
 
   >>> import cStringIO
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def serverinput(server):
   ...     readchannel(server)
   ... 
   ...     patch = """
   ... # HG changeset patch
   ... # User test
   ... # Date 0 0
   ... # Node ID c103a3dec114d882c98382d684d8af798d09d857
   ... # Parent  0000000000000000000000000000000000000000
   ... 1
   ... 
   ... diff -r 000000000000 -r c103a3dec114 a
   ... --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
   ... +++ b/a	Thu Jan 01 00:00:00 1970 +0000
   ... @@ -0,0 +1,1 @@
   ... +1
   ... """
   ... 
   ...     runcommand(server, ['import', '-'], input=cStringIO.StringIO(patch))
   ...     runcommand(server, ['log'])
   *** runcommand import -
   applying patch from stdin
   *** runcommand log
   changeset:   0:eff892de26ec
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     1
   
 
 check that --cwd doesn't persist between requests:
 
   $ mkdir foo
   $ touch foo/bar
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def cwd(server):
   ...     readchannel(server)
   ...     runcommand(server, ['--cwd', 'foo', 'st', 'bar'])
   ...     runcommand(server, ['st', 'foo/bar'])
   *** runcommand --cwd foo st bar
   ? bar
   *** runcommand st foo/bar
   ? foo/bar
 
   $ rm foo/bar
 
 
 check that local configs for the cached repo aren't inherited when -R is used:
 
   $ cat <<EOF >> .hg/hgrc
   > [ui]
   > foo = bar
   > EOF
 
   >>> from hgclient import readchannel, sep, runcommand, check
   >>> @check
   ... def localhgrc(server):
   ...     readchannel(server)
   ... 
   ...     # the cached repo local hgrc contains ui.foo=bar, so showconfig should
   ...     # show it
   ...     runcommand(server, ['showconfig'], outfilter=sep)
   ... 
   ...     # but not for this repo
   ...     runcommand(server, ['init', 'foo'])
   ...     runcommand(server, ['-R', 'foo', 'showconfig', 'ui', 'defaults'])
   *** runcommand showconfig
   bundle.mainreporoot=$TESTTMP/repo
   defaults.backout=-d "0 0"
   defaults.commit=-d "0 0"
   defaults.shelve=--date "0 0"
   defaults.tag=-d "0 0"
   devel.all-warnings=true
   largefiles.usercache=$TESTTMP/.cache/largefiles
   ui.slash=True
   ui.interactive=False
   ui.mergemarkers=detailed
   ui.foo=bar
   ui.nontty=true
   *** runcommand init foo
   *** runcommand -R foo showconfig ui defaults
   defaults.backout=-d "0 0"
   defaults.commit=-d "0 0"
   defaults.shelve=--date "0 0"
   defaults.tag=-d "0 0"
   ui.slash=True
   ui.interactive=False
   ui.mergemarkers=detailed
   ui.nontty=true
 
   $ rm -R foo
 
 #if windows
   $ PYTHONPATH="$TESTTMP/repo;$PYTHONPATH"
 #else
   $ PYTHONPATH="$TESTTMP/repo:$PYTHONPATH"
 #endif
 
   $ cat <<EOF > hook.py
   > import sys
   > def hook(**args):
   >     print 'hook talking'
   >     print 'now try to read something: %r' % sys.stdin.read()
   > EOF
 
   >>> import cStringIO
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def hookoutput(server):
   ...     readchannel(server)
   ...     runcommand(server, ['--config',
   ...                         'hooks.pre-identify=python:hook.hook',
   ...                         'id'],
   ...                input=cStringIO.StringIO('some input'))
   *** runcommand --config hooks.pre-identify=python:hook.hook id
   hook talking
   now try to read something: 'some input'
   eff892de26ec tip
 
   $ rm hook.py*
 
   $ echo a >> a
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def outsidechanges(server):
   ...     readchannel(server)
   ...     runcommand(server, ['status'])
   ...     os.system('hg ci -Am2')
   ...     runcommand(server, ['tip'])
   ...     runcommand(server, ['status'])
   *** runcommand status
   M a
   *** runcommand tip
   changeset:   1:d3a0a68be6de
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     2
   
   *** runcommand status
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def bookmarks(server):
   ...     readchannel(server)
   ...     runcommand(server, ['bookmarks'])
   ... 
   ...     # changes .hg/bookmarks
   ...     os.system('hg bookmark -i bm1')
   ...     os.system('hg bookmark -i bm2')
   ...     runcommand(server, ['bookmarks'])
   ... 
   ...     # changes .hg/bookmarks.current
   ...     os.system('hg upd bm1 -q')
   ...     runcommand(server, ['bookmarks'])
   ... 
   ...     runcommand(server, ['bookmarks', 'bm3'])
   ...     f = open('a', 'ab')
   ...     f.write('a\n')
   ...     f.close()
   ...     runcommand(server, ['commit', '-Amm'])
   ...     runcommand(server, ['bookmarks'])
   *** runcommand bookmarks
   no bookmarks set
   *** runcommand bookmarks
      bm1                       1:d3a0a68be6de
      bm2                       1:d3a0a68be6de
   *** runcommand bookmarks
    * bm1                       1:d3a0a68be6de
      bm2                       1:d3a0a68be6de
   *** runcommand bookmarks bm3
   *** runcommand commit -Amm
   *** runcommand bookmarks
      bm1                       1:d3a0a68be6de
      bm2                       1:d3a0a68be6de
    * bm3                       2:aef17e88f5f0
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def tagscache(server):
   ...     readchannel(server)
   ...     runcommand(server, ['id', '-t', '-r', '0'])
   ...     os.system('hg tag -r 0 foo')
   ...     runcommand(server, ['id', '-t', '-r', '0'])
   *** runcommand id -t -r 0
   
   *** runcommand id -t -r 0
   foo
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def setphase(server):
   ...     readchannel(server)
   ...     runcommand(server, ['phase', '-r', '.'])
   ...     os.system('hg phase -r . -p')
   ...     runcommand(server, ['phase', '-r', '.'])
   *** runcommand phase -r .
   3: draft
   *** runcommand phase -r .
   3: public
 
   $ echo a >> a
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def rollback(server):
   ...     readchannel(server)
   ...     runcommand(server, ['phase', '-r', '.', '-p'])
   ...     runcommand(server, ['commit', '-Am.'])
   ...     runcommand(server, ['rollback'])
   ...     runcommand(server, ['phase', '-r', '.'])
   *** runcommand phase -r . -p
   no phases changed
   *** runcommand commit -Am.
   *** runcommand rollback
   repository tip rolled back to revision 3 (undo commit)
   working directory now based on revision 3
   *** runcommand phase -r .
   3: public
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def branch(server):
   ...     readchannel(server)
   ...     runcommand(server, ['branch'])
   ...     os.system('hg branch foo')
   ...     runcommand(server, ['branch'])
   ...     os.system('hg branch default')
   *** runcommand branch
   default
   marked working directory as branch foo
   (branches are permanent and global, did you want a bookmark?)
   *** runcommand branch
   foo
   marked working directory as branch default
   (branches are permanent and global, did you want a bookmark?)
 
   $ touch .hgignore
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def hgignore(server):
   ...     readchannel(server)
   ...     runcommand(server, ['commit', '-Am.'])
   ...     f = open('ignored-file', 'ab')
   ...     f.write('')
   ...     f.close()
   ...     f = open('.hgignore', 'ab')
   ...     f.write('ignored-file')
   ...     f.close()
   ...     runcommand(server, ['status', '-i', '-u'])
   *** runcommand commit -Am.
   adding .hgignore
   *** runcommand status -i -u
   I ignored-file
 
 cache of non-public revisions should be invalidated on repository change
 (issue4855):
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def phasesetscacheaftercommit(server):
   ...     readchannel(server)
   ...     # load _phasecache._phaserevs and _phasesets
   ...     runcommand(server, ['log', '-qr', 'draft()'])
   ...     # create draft commits by another process
   ...     for i in xrange(5, 7):
   ...         f = open('a', 'ab')
   ...         f.seek(0, os.SEEK_END)
   ...         f.write('a\n')
   ...         f.close()
   ...         os.system('hg commit -Aqm%d' % i)
   ...     # new commits should be listed as draft revisions
   ...     runcommand(server, ['log', '-qr', 'draft()'])
   *** runcommand log -qr draft()
   4:7966c8e3734d
   *** runcommand log -qr draft()
   4:7966c8e3734d
   5:41f6602d1c4f
   6:10501e202c35
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def phasesetscacheafterstrip(server):
   ...     readchannel(server)
   ...     # load _phasecache._phaserevs and _phasesets
   ...     runcommand(server, ['log', '-qr', 'draft()'])
   ...     # strip cached revisions by another process
   ...     os.system('hg --config extensions.strip= strip -q 5')
   ...     # shouldn't abort by "unknown revision '6'"
   ...     runcommand(server, ['log', '-qr', 'draft()'])
   *** runcommand log -qr draft()
   4:7966c8e3734d
   5:41f6602d1c4f
   6:10501e202c35
   *** runcommand log -qr draft()
   4:7966c8e3734d
 
 cache of phase roots should be invalidated on strip (issue3827):
 
   >>> import os
   >>> from hgclient import readchannel, sep, runcommand, check
   >>> @check
   ... def phasecacheafterstrip(server):
   ...     readchannel(server)
   ... 
   ...     # create new head, 5:731265503d86
   ...     runcommand(server, ['update', '-C', '0'])
   ...     f = open('a', 'ab')
   ...     f.write('a\n')
   ...     f.close()
   ...     runcommand(server, ['commit', '-Am.', 'a'])
   ...     runcommand(server, ['log', '-Gq'])
   ... 
   ...     # make it public; draft marker moves to 4:7966c8e3734d
   ...     runcommand(server, ['phase', '-p', '.'])
   ...     # load _phasecache.phaseroots
   ...     runcommand(server, ['phase', '.'], outfilter=sep)
   ... 
   ...     # strip 1::4 outside server
   ...     os.system('hg -q --config extensions.mq= strip 1')
   ... 
   ...     # shouldn't raise "7966c8e3734d: no node!"
   ...     runcommand(server, ['branches'])
   *** runcommand update -C 0
   1 files updated, 0 files merged, 2 files removed, 0 files unresolved
   (leaving bookmark bm3)
   *** runcommand commit -Am. a
   created new head
   *** runcommand log -Gq
   @  5:731265503d86
   |
   | o  4:7966c8e3734d
   | |
   | o  3:b9b85890c400
   | |
   | o  2:aef17e88f5f0
   | |
   | o  1:d3a0a68be6de
   |/
   o  0:eff892de26ec
   
   *** runcommand phase -p .
   *** runcommand phase .
   5: public
   *** runcommand branches
   default                        1:731265503d86
 
 in-memory cache must be reloaded if transaction is aborted. otherwise
 changelog and manifest would have invalid node:
 
   $ echo a >> a
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def txabort(server):
   ...     readchannel(server)
   ...     runcommand(server, ['commit', '--config', 'hooks.pretxncommit=false',
   ...                         '-mfoo'])
   ...     runcommand(server, ['verify'])
   *** runcommand commit --config hooks.pretxncommit=false -mfoo
   transaction abort!
   rollback completed
   abort: pretxncommit hook exited with status 1
    [255]
   *** runcommand verify
   checking changesets
   checking manifests
   crosschecking files in changesets and manifests
   checking files
   1 files, 2 changesets, 2 total revisions
   $ hg revert --no-backup -aq
 
   $ cat >> .hg/hgrc << EOF
   > [experimental]
   > evolution=createmarkers
   > EOF
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def obsolete(server):
   ...     readchannel(server)
   ... 
   ...     runcommand(server, ['up', 'null'])
   ...     runcommand(server, ['phase', '-df', 'tip'])
   ...     cmd = 'hg debugobsolete `hg log -r tip --template {node}`'
   ...     if os.name == 'nt':
   ...         cmd = 'sh -c "%s"' % cmd # run in sh, not cmd.exe
   ...     os.system(cmd)
   ...     runcommand(server, ['log', '--hidden'])
   ...     runcommand(server, ['log'])
   *** runcommand up null
   0 files updated, 0 files merged, 1 files removed, 0 files unresolved
   *** runcommand phase -df tip
   *** runcommand log --hidden
   changeset:   1:731265503d86
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     .
   
   changeset:   0:eff892de26ec
   bookmark:    bm1
   bookmark:    bm2
   bookmark:    bm3
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     1
   
   *** runcommand log
   changeset:   0:eff892de26ec
   bookmark:    bm1
   bookmark:    bm2
   bookmark:    bm3
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     1
   
 
   $ cat <<EOF >> .hg/hgrc
   > [extensions]
   > mq =
   > EOF
 
   >>> import os
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def mqoutsidechanges(server):
   ...     readchannel(server)
   ... 
   ...     # load repo.mq
   ...     runcommand(server, ['qapplied'])
   ...     os.system('hg qnew 0.diff')
   ...     # repo.mq should be invalidated
   ...     runcommand(server, ['qapplied'])
   ... 
   ...     runcommand(server, ['qpop', '--all'])
   ...     os.system('hg qqueue --create foo')
   ...     # repo.mq should be recreated to point to new queue
   ...     runcommand(server, ['qqueue', '--active'])
   *** runcommand qapplied
   *** runcommand qapplied
   0.diff
   *** runcommand qpop --all
   popping 0.diff
   patch queue now empty
   *** runcommand qqueue --active
   foo
 
   $ cat <<EOF > dbgui.py
   > import os, sys
   > from mercurial import cmdutil, commands
   > cmdtable = {}
   > command = cmdutil.command(cmdtable)
   > @command("debuggetpass", norepo=True)
   > def debuggetpass(ui):
   >     ui.write("%s\\n" % ui.getpass())
   > @command("debugprompt", norepo=True)
   > def debugprompt(ui):
   >     ui.write("%s\\n" % ui.prompt("prompt:"))
   > @command("debugreadstdin", norepo=True)
   > def debugreadstdin(ui):
   >     ui.write("read: %r\n" % sys.stdin.read(1))
   > @command("debugwritestdout", norepo=True)
   > def debugwritestdout(ui):
   >     os.write(1, "low-level stdout fd and\n")
   >     sys.stdout.write("stdout should be redirected to /dev/null\n")
   >     sys.stdout.flush()
   > EOF
   $ cat <<EOF >> .hg/hgrc
   > [extensions]
   > dbgui = dbgui.py
   > EOF
 
   >>> import cStringIO
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def getpass(server):
   ...     readchannel(server)
   ...     runcommand(server, ['debuggetpass', '--config',
   ...                         'ui.interactive=True'],
   ...                input=cStringIO.StringIO('1234\n'))
   ...     runcommand(server, ['debugprompt', '--config',
   ...                         'ui.interactive=True'],
   ...                input=cStringIO.StringIO('5678\n'))
   ...     runcommand(server, ['debugreadstdin'])
   ...     runcommand(server, ['debugwritestdout'])
   *** runcommand debuggetpass --config ui.interactive=True
   password: 1234
   *** runcommand debugprompt --config ui.interactive=True
   prompt: 5678
   *** runcommand debugreadstdin
   read: ''
   *** runcommand debugwritestdout
 
 
 run commandserver in commandserver, which is silly but should work:
 
   >>> import cStringIO
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def nested(server):
   ...     print '%c, %r' % readchannel(server)
   ...     class nestedserver(object):
   ...         stdin = cStringIO.StringIO('getencoding\n')
   ...         stdout = cStringIO.StringIO()
   ...     runcommand(server, ['serve', '--cmdserver', 'pipe'],
   ...                output=nestedserver.stdout, input=nestedserver.stdin)
   ...     nestedserver.stdout.seek(0)
   ...     print '%c, %r' % readchannel(nestedserver)  # hello
   ...     print '%c, %r' % readchannel(nestedserver)  # getencoding
   o, 'capabilities: getencoding runcommand\nencoding: *\npid: *' (glob)
   *** runcommand serve --cmdserver pipe
   o, 'capabilities: getencoding runcommand\nencoding: *\npid: *' (glob)
   r, '*' (glob)
 
 
 start without repository:
 
   $ cd ..
 
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def hellomessage(server):
   ...     ch, data = readchannel(server)
   ...     print '%c, %r' % (ch, data)
   ...     # run an arbitrary command to make sure the next thing the server
   ...     # sends isn't part of the hello message
   ...     runcommand(server, ['id'])
   o, 'capabilities: getencoding runcommand\nencoding: *\npid: *' (glob)
   *** runcommand id
   abort: there is no Mercurial repository here (.hg not found)
    [255]
 
   >>> from hgclient import readchannel, runcommand, check
   >>> @check
   ... def startwithoutrepo(server):
   ...     readchannel(server)
   ...     runcommand(server, ['init', 'repo2'])
   ...     runcommand(server, ['id', '-R', 'repo2'])
   *** runcommand init repo2
   *** runcommand id -R repo2
   000000000000 tip
 
 
 don't fall back to cwd if invalid -R path is specified (issue4805):
 
   $ cd repo
   $ hg serve --cmdserver pipe -R ../nonexistent
   abort: repository ../nonexistent not found!
   [255]
   $ cd ..
 
 
 unix domain socket:
 
   $ cd repo
   $ hg update -q
 
 #if unix-socket unix-permissions
 
   >>> import cStringIO
   >>> from hgclient import unixserver, readchannel, runcommand, check
   >>> server = unixserver('.hg/server.sock', '.hg/server.log')
   >>> def hellomessage(conn):
   ...     ch, data = readchannel(conn)
   ...     print '%c, %r' % (ch, data)
   ...     runcommand(conn, ['id'])
   >>> check(hellomessage, server.connect)
   o, 'capabilities: getencoding runcommand\nencoding: *\npid: *' (glob)
   *** runcommand id
   eff892de26ec tip bm1/bm2/bm3
   >>> def unknowncommand(conn):
   ...     readchannel(conn)
   ...     conn.stdin.write('unknowncommand\n')
   >>> check(unknowncommand, server.connect)  # error sent to server.log
   >>> def serverinput(conn):
   ...     readchannel(conn)
   ...     patch = """
   ... # HG changeset patch
   ... # User test
   ... # Date 0 0
   ... 2
   ... 
   ... diff -r eff892de26ec -r 1ed24be7e7a0 a
   ... --- a/a
   ... +++ b/a
   ... @@ -1,1 +1,2 @@
   ...  1
   ... +2
   ... """
   ...     runcommand(conn, ['import', '-'], input=cStringIO.StringIO(patch))
   ...     runcommand(conn, ['log', '-rtip', '-q'])
   >>> check(serverinput, server.connect)
   *** runcommand import -
   applying patch from stdin
   *** runcommand log -rtip -q
   2:1ed24be7e7a0
   >>> server.shutdown()
 
   $ cat .hg/server.log
   listening at .hg/server.sock
   abort: unknown command unknowncommand
   killed!
 #endif
 #if no-unix-socket
 
   $ hg serve --cmdserver unix -a .hg/server.sock
   abort: unsupported platform
   [255]
 
 #endif
+
+  $ cd ..
+
+Test that accessing to invalid changelog cache is avoided at
+subsequent operations even if repo object is reused even after failure
+of transaction (see 0a7610758c42 also)
+
+"hg log" after failure of transaction is needed to detect invalid
+cache in repoview: this can't detect by "hg verify" only.
+
+Combination of "finalization" and "empty-ness of changelog" (2 x 2 =
+4) are tested, because '00changelog.i' are differently changed in each
+cases.
+
+  $ cat > $TESTTMP/failafterfinalize.py <<EOF
+  > # extension to abort transaction after finalization forcibly
+  > from mercurial import commands, error, extensions, lock as lockmod
+  > def fail(tr):
+  >     raise error.Abort('fail after finalization')
+  > def reposetup(ui, repo):
+  >     class failrepo(repo.__class__):
+  >         def commitctx(self, ctx, error=False):
+  >             if self.ui.configbool('failafterfinalize', 'fail'):
+  >                 # 'sorted()' by ASCII code on category names causes
+  >                 # invoking 'fail' after finalization of changelog
+  >                 # using "'cl-%i' % id(self)" as category name
+  >                 self.currenttransaction().addfinalize('zzzzzzzz', fail)
+  >             return super(failrepo, self).commitctx(ctx, error)
+  >     repo.__class__ = failrepo
+  > EOF
+
+  $ hg init repo3
+  $ cd repo3
+
+  $ cat <<EOF >> $HGRCPATH
+  > [ui]
+  > logtemplate = {rev} {desc|firstline} ({files})\n
+  > 
+  > [extensions]
+  > failafterfinalize = $TESTTMP/failafterfinalize.py
+  > EOF
+
+- test failure with "empty changelog"
+
+  $ echo foo > foo
+  $ hg add foo
+
+(failuer before finalization)
+
+  >>> from hgclient import readchannel, runcommand, check
+  >>> @check
+  ... def abort(server):
+  ...     readchannel(server)
+  ...     runcommand(server, ['commit',
+  ...                         '--config', 'hooks.pretxncommit=false',
+  ...                         '-mfoo'])
+  ...     runcommand(server, ['log'])
+  ...     runcommand(server, ['verify', '-q'])
+  *** runcommand commit --config hooks.pretxncommit=false -mfoo
+  transaction abort!
+  rollback completed
+  abort: pretxncommit hook exited with status 1
+   [255]
+  *** runcommand log
+  *** runcommand verify -q
+
+(failuer after finalization)
+
+  >>> from hgclient import readchannel, runcommand, check
+  >>> @check
+  ... def abort(server):
+  ...     readchannel(server)
+  ...     runcommand(server, ['commit',
+  ...                         '--config', 'failafterfinalize.fail=true',
+  ...                         '-mfoo'])
+  ...     runcommand(server, ['log'])
+  ...     runcommand(server, ['verify', '-q'])
+  *** runcommand commit --config failafterfinalize.fail=true -mfoo
+  transaction abort!
+  rollback completed
+  abort: fail after finalization
+   [255]
+  *** runcommand log
+  *** runcommand verify -q
+
+- test failure with "not-empty changelog"
+
+  $ echo bar > bar
+  $ hg add bar
+  $ hg commit -mbar bar
+
+(failure before finalization)
+
+  >>> from hgclient import readchannel, runcommand, check
+  >>> @check
+  ... def abort(server):
+  ...     readchannel(server)
+  ...     runcommand(server, ['commit',
+  ...                         '--config', 'hooks.pretxncommit=false',
+  ...                         '-mfoo', 'foo'])
+  ...     runcommand(server, ['log'])
+  ...     runcommand(server, ['verify', '-q'])
+  *** runcommand commit --config hooks.pretxncommit=false -mfoo foo
+  transaction abort!
+  rollback completed
+  abort: pretxncommit hook exited with status 1
+   [255]
+  *** runcommand log
+  0 bar (bar)
+  *** runcommand verify -q
+
+(failure after finalization)
+
+  >>> from hgclient import readchannel, runcommand, check
+  >>> @check
+  ... def abort(server):
+  ...     readchannel(server)
+  ...     runcommand(server, ['commit',
+  ...                         '--config', 'failafterfinalize.fail=true',
+  ...                         '-mfoo', 'foo'])
+  ...     runcommand(server, ['log'])
+  ...     runcommand(server, ['verify', '-q'])
+  *** runcommand commit --config failafterfinalize.fail=true -mfoo foo
+  transaction abort!
+  rollback completed
+  abort: fail after finalization
+   [255]
+  *** runcommand log
+  0 bar (bar)
+  *** runcommand verify -q