tagcache: distinguish between invalid and missing entries

The TortoiseHg repo has typically not had a newly applied tag accessible by name
for recent releases, for unknown reasons.  Deleting and rebuilding the tag cache
doesn't fix it, though deleting the cache and running `hg log -r $new_tag` does.
Eventually the situation does sort itself out for new clones from the server.
In an effort to figure out what the issue is, Pierre-Yves David suggested
listing these entries in the debug output more specifically.

This isn't complete yet- the second test change that says "missing" is more like
"invalid", since it was truncated.  The problem there is the code that reads the
raw array truncates any partial records and then fills it with 0xFF, which
signifies that it is missing.  As a side note, that means the check for the
length when validating an existing entry never fails.

Differential Revision: https://phab.mercurial-scm.org/D9811

mercurial/bundle2.py

@@ -1,2561 +1,2561 b''
 # bundle2.py - generic container format to transmit arbitrary data.
 #
 # Copyright 2013 Facebook, Inc.
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 """Handling of the new bundle2 format
 
 The goal of bundle2 is to act as an atomically packet to transmit a set of
 payloads in an application agnostic way. It consist in a sequence of "parts"
 that will be handed to and processed by the application layer.
 
 
 General format architecture
 ===========================
 
 The format is architectured as follow
 
  - magic string
  - stream level parameters
  - payload parts (any number)
  - end of stream marker.
 
 the Binary format
 ============================
 
 All numbers are unsigned and big-endian.
 
 stream level parameters
 ------------------------
 
 Binary format is as follow
 
 :params size: int32
 
   The total number of Bytes used by the parameters
 
 :params value: arbitrary number of Bytes
 
   A blob of `params size` containing the serialized version of all stream level
   parameters.
 
   The blob contains a space separated list of parameters. Parameters with value
   are stored in the form `<name>=<value>`. Both name and value are urlquoted.
 
   Empty name are obviously forbidden.
 
   Name MUST start with a letter. If this first letter is lower case, the
   parameter is advisory and can be safely ignored. However when the first
   letter is capital, the parameter is mandatory and the bundling process MUST
   stop if he is not able to proceed it.
 
   Stream parameters use a simple textual format for two main reasons:
 
   - Stream level parameters should remain simple and we want to discourage any
     crazy usage.
   - Textual data allow easy human inspection of a bundle2 header in case of
     troubles.
 
   Any Applicative level options MUST go into a bundle2 part instead.
 
 Payload part
 ------------------------
 
 Binary format is as follow
 
 :header size: int32
 
   The total number of Bytes used by the part header. When the header is empty
   (size = 0) this is interpreted as the end of stream marker.
 
 :header:
 
     The header defines how to interpret the part. It contains two piece of
     data: the part type, and the part parameters.
 
     The part type is used to route an application level handler, that can
     interpret payload.
 
     Part parameters are passed to the application level handler.  They are
     meant to convey information that will help the application level object to
     interpret the part payload.
 
     The binary format of the header is has follow
 
     :typesize: (one byte)
 
     :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
 
     :partid: A 32bits integer (unique in the bundle) that can be used to refer
              to this part.
 
     :parameters:
 
         Part's parameter may have arbitrary content, the binary structure is::
 
             <mandatory-count><advisory-count><param-sizes><param-data>
 
         :mandatory-count: 1 byte, number of mandatory parameters
 
         :advisory-count:  1 byte, number of advisory parameters
 
         :param-sizes:
 
             N couple of bytes, where N is the total number of parameters. Each
             couple contains (<size-of-key>, <size-of-value) for one parameter.
 
         :param-data:
 
             A blob of bytes from which each parameter key and value can be
             retrieved using the list of size couples stored in the previous
             field.
 
             Mandatory parameters comes first, then the advisory ones.
 
             Each parameter's key MUST be unique within the part.
 
 :payload:
 
     payload is a series of `<chunksize><chunkdata>`.
 
     `chunksize` is an int32, `chunkdata` are plain bytes (as much as
     `chunksize` says)` The payload part is concluded by a zero size chunk.
 
     The current implementation always produces either zero or one chunk.
     This is an implementation limitation that will ultimately be lifted.
 
     `chunksize` can be negative to trigger special case processing. No such
     processing is in place yet.
 
 Bundle processing
 ============================
 
 Each part is processed in order using a "part handler". Handler are registered
 for a certain part type.
 
 The matching of a part to its handler is case insensitive. The case of the
 part type is used to know if a part is mandatory or advisory. If the Part type
 contains any uppercase char it is considered mandatory. When no handler is
 known for a Mandatory part, the process is aborted and an exception is raised.
 If the part is advisory and no handler is known, the part is ignored. When the
 process is aborted, the full bundle is still read from the stream to keep the
 channel usable. But none of the part read from an abort are processed. In the
 future, dropping the stream may become an option for channel we do not care to
 preserve.
 """
 
 from __future__ import absolute_import, division
 
 import collections
 import errno
 import os
 import re
 import string
 import struct
 import sys
 
 from .i18n import _
 from .node import (
     hex,
     nullid,
     short,
 )
 from . import (
     bookmarks,
     changegroup,
     encoding,
     error,
     obsolete,
     phases,
     pushkey,
     pycompat,
     requirements,
     scmutil,
     streamclone,
     tags,
     url,
     util,
 )
 from .utils import stringutil
 
 urlerr = util.urlerr
 urlreq = util.urlreq
 
 _pack = struct.pack
 _unpack = struct.unpack
 
 _fstreamparamsize = b'>i'
 _fpartheadersize = b'>i'
 _fparttypesize = b'>B'
 _fpartid = b'>I'
 _fpayloadsize = b'>i'
 _fpartparamcount = b'>BB'
 
 preferedchunksize = 32768
 
 _parttypeforbidden = re.compile(b'[^a-zA-Z0-9_:-]')
 
 
 def outdebug(ui, message):
     """debug regarding output stream (bundling)"""
     if ui.configbool(b'devel', b'bundle2.debug'):
         ui.debug(b'bundle2-output: %s\n' % message)
 
 
 def indebug(ui, message):
     """debug on input stream (unbundling)"""
     if ui.configbool(b'devel', b'bundle2.debug'):
         ui.debug(b'bundle2-input: %s\n' % message)
 
 
 def validateparttype(parttype):
     """raise ValueError if a parttype contains invalid character"""
     if _parttypeforbidden.search(parttype):
         raise ValueError(parttype)
 
 
 def _makefpartparamsizes(nbparams):
     """return a struct format to read part parameter sizes
 
     The number parameters is variable so we need to build that format
     dynamically.
     """
     return b'>' + (b'BB' * nbparams)
 
 
 parthandlermapping = {}
 
 
 def parthandler(parttype, params=()):
     """decorator that register a function as a bundle2 part handler
 
     eg::
 
         @parthandler('myparttype', ('mandatory', 'param', 'handled'))
         def myparttypehandler(...):
             '''process a part of type "my part".'''
             ...
     """
     validateparttype(parttype)
 
     def _decorator(func):
         lparttype = parttype.lower()  # enforce lower case matching.
         assert lparttype not in parthandlermapping
         parthandlermapping[lparttype] = func
         func.params = frozenset(params)
         return func
 
     return _decorator
 
 
 class unbundlerecords(object):
     """keep record of what happens during and unbundle
 
     New records are added using `records.add('cat', obj)`. Where 'cat' is a
     category of record and obj is an arbitrary object.
 
     `records['cat']` will return all entries of this category 'cat'.
 
     Iterating on the object itself will yield `('category', obj)` tuples
     for all entries.
 
     All iterations happens in chronological order.
     """
 
     def __init__(self):
         self._categories = {}
         self._sequences = []
         self._replies = {}
 
     def add(self, category, entry, inreplyto=None):
         """add a new record of a given category.
 
         The entry can then be retrieved in the list returned by
         self['category']."""
         self._categories.setdefault(category, []).append(entry)
         self._sequences.append((category, entry))
         if inreplyto is not None:
             self.getreplies(inreplyto).add(category, entry)
 
     def getreplies(self, partid):
         """get the records that are replies to a specific part"""
         return self._replies.setdefault(partid, unbundlerecords())
 
     def __getitem__(self, cat):
         return tuple(self._categories.get(cat, ()))
 
     def __iter__(self):
         return iter(self._sequences)
 
     def __len__(self):
         return len(self._sequences)
 
     def __nonzero__(self):
         return bool(self._sequences)
 
     __bool__ = __nonzero__
 
 
 class bundleoperation(object):
     """an object that represents a single bundling process
 
     Its purpose is to carry unbundle-related objects and states.
 
     A new object should be created at the beginning of each bundle processing.
     The object is to be returned by the processing function.
 
     The object has very little content now it will ultimately contain:
     * an access to the repo the bundle is applied to,
     * a ui object,
     * a way to retrieve a transaction to add changes to the repo,
     * a way to record the result of processing each part,
     * a way to construct a bundle response when applicable.
     """
 
     def __init__(self, repo, transactiongetter, captureoutput=True, source=b''):
         self.repo = repo
         self.ui = repo.ui
         self.records = unbundlerecords()
         self.reply = None
         self.captureoutput = captureoutput
         self.hookargs = {}
         self._gettransaction = transactiongetter
         # carries value that can modify part behavior
         self.modes = {}
         self.source = source
 
     def gettransaction(self):
         transaction = self._gettransaction()
 
         if self.hookargs:
             # the ones added to the transaction supercede those added
             # to the operation.
             self.hookargs.update(transaction.hookargs)
             transaction.hookargs = self.hookargs
 
         # mark the hookargs as flushed.  further attempts to add to
         # hookargs will result in an abort.
         self.hookargs = None
 
         return transaction
 
     def addhookargs(self, hookargs):
         if self.hookargs is None:
             raise error.ProgrammingError(
                 b'attempted to add hookargs to '
                 b'operation after transaction started'
             )
         self.hookargs.update(hookargs)
 
 
 class TransactionUnavailable(RuntimeError):
     pass
 
 
 def _notransaction():
     """default method to get a transaction while processing a bundle
 
     Raise an exception to highlight the fact that no transaction was expected
     to be created"""
     raise TransactionUnavailable()
 
 
 def applybundle(repo, unbundler, tr, source, url=None, **kwargs):
     # transform me into unbundler.apply() as soon as the freeze is lifted
     if isinstance(unbundler, unbundle20):
         tr.hookargs[b'bundle2'] = b'1'
         if source is not None and b'source' not in tr.hookargs:
             tr.hookargs[b'source'] = source
         if url is not None and b'url' not in tr.hookargs:
             tr.hookargs[b'url'] = url
         return processbundle(repo, unbundler, lambda: tr, source=source)
     else:
         # the transactiongetter won't be used, but we might as well set it
         op = bundleoperation(repo, lambda: tr, source=source)
         _processchangegroup(op, unbundler, tr, source, url, **kwargs)
         return op
 
 
 class partiterator(object):
     def __init__(self, repo, op, unbundler):
         self.repo = repo
         self.op = op
         self.unbundler = unbundler
         self.iterator = None
         self.count = 0
         self.current = None
 
     def __enter__(self):
         def func():
             itr = enumerate(self.unbundler.iterparts(), 1)
             for count, p in itr:
                 self.count = count
                 self.current = p
                 yield p
                 p.consume()
                 self.current = None
 
         self.iterator = func()
         return self.iterator
 
     def __exit__(self, type, exc, tb):
         if not self.iterator:
             return
 
         # Only gracefully abort in a normal exception situation. User aborts
         # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
         # and should not gracefully cleanup.
         if isinstance(exc, Exception):
             # Any exceptions seeking to the end of the bundle at this point are
             # almost certainly related to the underlying stream being bad.
             # And, chances are that the exception we're handling is related to
             # getting in that bad state. So, we swallow the seeking error and
             # re-raise the original error.
             seekerror = False
             try:
                 if self.current:
                     # consume the part content to not corrupt the stream.
                     self.current.consume()
 
                 for part in self.iterator:
                     # consume the bundle content
                     part.consume()
             except Exception:
                 seekerror = True
 
             # Small hack to let caller code distinguish exceptions from bundle2
             # processing from processing the old format. This is mostly needed
             # to handle different return codes to unbundle according to the type
             # of bundle. We should probably clean up or drop this return code
             # craziness in a future version.
             exc.duringunbundle2 = True
             salvaged = []
             replycaps = None
             if self.op.reply is not None:
                 salvaged = self.op.reply.salvageoutput()
                 replycaps = self.op.reply.capabilities
             exc._replycaps = replycaps
             exc._bundle2salvagedoutput = salvaged
 
             # Re-raising from a variable loses the original stack. So only use
             # that form if we need to.
             if seekerror:
                 raise exc
 
         self.repo.ui.debug(
             b'bundle2-input-bundle: %i parts total\n' % self.count
         )
 
 
 def processbundle(repo, unbundler, transactiongetter=None, op=None, source=b''):
     """This function process a bundle, apply effect to/from a repo
 
     It iterates over each part then searches for and uses the proper handling
     code to process the part. Parts are processed in order.
 
     Unknown Mandatory part will abort the process.
 
     It is temporarily possible to provide a prebuilt bundleoperation to the
     function. This is used to ensure output is properly propagated in case of
     an error during the unbundling. This output capturing part will likely be
     reworked and this ability will probably go away in the process.
     """
     if op is None:
         if transactiongetter is None:
             transactiongetter = _notransaction
         op = bundleoperation(repo, transactiongetter, source=source)
     # todo:
     # - replace this is a init function soon.
     # - exception catching
     unbundler.params
     if repo.ui.debugflag:
         msg = [b'bundle2-input-bundle:']
         if unbundler.params:
             msg.append(b' %i params' % len(unbundler.params))
         if op._gettransaction is None or op._gettransaction is _notransaction:
             msg.append(b' no-transaction')
         else:
             msg.append(b' with-transaction')
         msg.append(b'\n')
         repo.ui.debug(b''.join(msg))
 
     processparts(repo, op, unbundler)
 
     return op
 
 
 def processparts(repo, op, unbundler):
     with partiterator(repo, op, unbundler) as parts:
         for part in parts:
             _processpart(op, part)
 
 
 def _processchangegroup(op, cg, tr, source, url, **kwargs):
     ret = cg.apply(op.repo, tr, source, url, **kwargs)
     op.records.add(
         b'changegroup',
         {
             b'return': ret,
         },
     )
     return ret
 
 
 def _gethandler(op, part):
     status = b'unknown'  # used by debug output
     try:
         handler = parthandlermapping.get(part.type)
         if handler is None:
             status = b'unsupported-type'
             raise error.BundleUnknownFeatureError(parttype=part.type)
         indebug(op.ui, b'found a handler for part %s' % part.type)
         unknownparams = part.mandatorykeys - handler.params
         if unknownparams:
             unknownparams = list(unknownparams)
             unknownparams.sort()
             status = b'unsupported-params (%s)' % b', '.join(unknownparams)
             raise error.BundleUnknownFeatureError(
                 parttype=part.type, params=unknownparams
             )
         status = b'supported'
     except error.BundleUnknownFeatureError as exc:
         if part.mandatory:  # mandatory parts
             raise
         indebug(op.ui, b'ignoring unsupported advisory part %s' % exc)
         return  # skip to part processing
     finally:
         if op.ui.debugflag:
             msg = [b'bundle2-input-part: "%s"' % part.type]
             if not part.mandatory:
                 msg.append(b' (advisory)')
             nbmp = len(part.mandatorykeys)
             nbap = len(part.params) - nbmp
             if nbmp or nbap:
                 msg.append(b' (params:')
                 if nbmp:
                     msg.append(b' %i mandatory' % nbmp)
                 if nbap:
                     msg.append(b' %i advisory' % nbmp)
                 msg.append(b')')
             msg.append(b' %s\n' % status)
             op.ui.debug(b''.join(msg))
 
     return handler
 
 
 def _processpart(op, part):
     """process a single part from a bundle
 
     The part is guaranteed to have been fully consumed when the function exits
     (even if an exception is raised)."""
     handler = _gethandler(op, part)
     if handler is None:
         return
 
     # handler is called outside the above try block so that we don't
     # risk catching KeyErrors from anything other than the
     # parthandlermapping lookup (any KeyError raised by handler()
     # itself represents a defect of a different variety).
     output = None
     if op.captureoutput and op.reply is not None:
         op.ui.pushbuffer(error=True, subproc=True)
         output = b''
     try:
         handler(op, part)
     finally:
         if output is not None:
             output = op.ui.popbuffer()
         if output:
             outpart = op.reply.newpart(b'output', data=output, mandatory=False)
             outpart.addparam(
                 b'in-reply-to', pycompat.bytestr(part.id), mandatory=False
             )
 
 
 def decodecaps(blob):
     """decode a bundle2 caps bytes blob into a dictionary
 
     The blob is a list of capabilities (one per line)
     Capabilities may have values using a line of the form::
 
         capability=value1,value2,value3
 
     The values are always a list."""
     caps = {}
     for line in blob.splitlines():
         if not line:
             continue
         if b'=' not in line:
             key, vals = line, ()
         else:
             key, vals = line.split(b'=', 1)
             vals = vals.split(b',')
         key = urlreq.unquote(key)
         vals = [urlreq.unquote(v) for v in vals]
         caps[key] = vals
     return caps
 
 
 def encodecaps(caps):
     """encode a bundle2 caps dictionary into a bytes blob"""
     chunks = []
     for ca in sorted(caps):
         vals = caps[ca]
         ca = urlreq.quote(ca)
         vals = [urlreq.quote(v) for v in vals]
         if vals:
             ca = b"%s=%s" % (ca, b','.join(vals))
         chunks.append(ca)
     return b'\n'.join(chunks)
 
 
 bundletypes = {
     b"": (b"", b'UN'),  # only when using unbundle on ssh and old http servers
     # since the unification ssh accepts a header but there
     # is no capability signaling it.
     b"HG20": (),  # special-cased below
     b"HG10UN": (b"HG10UN", b'UN'),
     b"HG10BZ": (b"HG10", b'BZ'),
     b"HG10GZ": (b"HG10GZ", b'GZ'),
 }
 
 # hgweb uses this list to communicate its preferred type
 bundlepriority = [b'HG10GZ', b'HG10BZ', b'HG10UN']
 
 
 class bundle20(object):
     """represent an outgoing bundle2 container
 
     Use the `addparam` method to add stream level parameter. and `newpart` to
     populate it. Then call `getchunks` to retrieve all the binary chunks of
     data that compose the bundle2 container."""
 
     _magicstring = b'HG20'
 
     def __init__(self, ui, capabilities=()):
         self.ui = ui
         self._params = []
         self._parts = []
         self.capabilities = dict(capabilities)
         self._compengine = util.compengines.forbundletype(b'UN')
         self._compopts = None
         # If compression is being handled by a consumer of the raw
         # data (e.g. the wire protocol), unsetting this flag tells
         # consumers that the bundle is best left uncompressed.
         self.prefercompressed = True
 
     def setcompression(self, alg, compopts=None):
         """setup core part compression to <alg>"""
         if alg in (None, b'UN'):
             return
         assert not any(n.lower() == b'compression' for n, v in self._params)
         self.addparam(b'Compression', alg)
         self._compengine = util.compengines.forbundletype(alg)
         self._compopts = compopts
 
     @property
     def nbparts(self):
         """total number of parts added to the bundler"""
         return len(self._parts)
 
     # methods used to defines the bundle2 content
     def addparam(self, name, value=None):
         """add a stream level parameter"""
         if not name:
             raise error.ProgrammingError(b'empty parameter name')
         if name[0:1] not in pycompat.bytestr(
             string.ascii_letters  # pytype: disable=wrong-arg-types
         ):
             raise error.ProgrammingError(
                 b'non letter first character: %s' % name
             )
         self._params.append((name, value))
 
     def addpart(self, part):
         """add a new part to the bundle2 container
 
         Parts contains the actual applicative payload."""
         assert part.id is None
         part.id = len(self._parts)  # very cheap counter
         self._parts.append(part)
 
     def newpart(self, typeid, *args, **kwargs):
         """create a new part and add it to the containers
 
         As the part is directly added to the containers. For now, this means
         that any failure to properly initialize the part after calling
         ``newpart`` should result in a failure of the whole bundling process.
 
         You can still fall back to manually create and add if you need better
         control."""
         part = bundlepart(typeid, *args, **kwargs)
         self.addpart(part)
         return part
 
     # methods used to generate the bundle2 stream
     def getchunks(self):
         if self.ui.debugflag:
             msg = [b'bundle2-output-bundle: "%s",' % self._magicstring]
             if self._params:
                 msg.append(b' (%i params)' % len(self._params))
             msg.append(b' %i parts total\n' % len(self._parts))
             self.ui.debug(b''.join(msg))
         outdebug(self.ui, b'start emission of %s stream' % self._magicstring)
         yield self._magicstring
         param = self._paramchunk()
         outdebug(self.ui, b'bundle parameter: %s' % param)
         yield _pack(_fstreamparamsize, len(param))
         if param:
             yield param
         for chunk in self._compengine.compressstream(
             self._getcorechunk(), self._compopts
         ):
             yield chunk
 
     def _paramchunk(self):
         """return a encoded version of all stream parameters"""
         blocks = []
         for par, value in self._params:
             par = urlreq.quote(par)
             if value is not None:
                 value = urlreq.quote(value)
                 par = b'%s=%s' % (par, value)
             blocks.append(par)
         return b' '.join(blocks)
 
     def _getcorechunk(self):
         """yield chunk for the core part of the bundle
 
         (all but headers and parameters)"""
         outdebug(self.ui, b'start of parts')
         for part in self._parts:
             outdebug(self.ui, b'bundle part: "%s"' % part.type)
             for chunk in part.getchunks(ui=self.ui):
                 yield chunk
         outdebug(self.ui, b'end of bundle')
         yield _pack(_fpartheadersize, 0)
 
     def salvageoutput(self):
         """return a list with a copy of all output parts in the bundle
 
         This is meant to be used during error handling to make sure we preserve
         server output"""
         salvaged = []
         for part in self._parts:
             if part.type.startswith(b'output'):
                 salvaged.append(part.copy())
         return salvaged
 
 
 class unpackermixin(object):
     """A mixin to extract bytes and struct data from a stream"""
 
     def __init__(self, fp):
         self._fp = fp
 
     def _unpack(self, format):
         """unpack this struct format from the stream
 
         This method is meant for internal usage by the bundle2 protocol only.
         They directly manipulate the low level stream including bundle2 level
         instruction.
 
         Do not use it to implement higher-level logic or methods."""
         data = self._readexact(struct.calcsize(format))
         return _unpack(format, data)
 
     def _readexact(self, size):
         """read exactly <size> bytes from the stream
 
         This method is meant for internal usage by the bundle2 protocol only.
         They directly manipulate the low level stream including bundle2 level
         instruction.
 
         Do not use it to implement higher-level logic or methods."""
         return changegroup.readexactly(self._fp, size)
 
 
 def getunbundler(ui, fp, magicstring=None):
     """return a valid unbundler object for a given magicstring"""
     if magicstring is None:
         magicstring = changegroup.readexactly(fp, 4)
     magic, version = magicstring[0:2], magicstring[2:4]
     if magic != b'HG':
         ui.debug(
             b"error: invalid magic: %r (version %r), should be 'HG'\n"
             % (magic, version)
         )
         raise error.Abort(_(b'not a Mercurial bundle'))
     unbundlerclass = formatmap.get(version)
     if unbundlerclass is None:
         raise error.Abort(_(b'unknown bundle version %s') % version)
     unbundler = unbundlerclass(ui, fp)
     indebug(ui, b'start processing of %s stream' % magicstring)
     return unbundler
 
 
 class unbundle20(unpackermixin):
     """interpret a bundle2 stream
 
     This class is fed with a binary stream and yields parts through its
     `iterparts` methods."""
 
     _magicstring = b'HG20'
 
     def __init__(self, ui, fp):
         """If header is specified, we do not read it out of the stream."""
         self.ui = ui
         self._compengine = util.compengines.forbundletype(b'UN')
         self._compressed = None
         super(unbundle20, self).__init__(fp)
 
     @util.propertycache
     def params(self):
         """dictionary of stream level parameters"""
         indebug(self.ui, b'reading bundle2 stream parameters')
         params = {}
         paramssize = self._unpack(_fstreamparamsize)[0]
         if paramssize < 0:
             raise error.BundleValueError(
                 b'negative bundle param size: %i' % paramssize
             )
         if paramssize:
             params = self._readexact(paramssize)
             params = self._processallparams(params)
         return params
 
     def _processallparams(self, paramsblock):
         """"""
         params = util.sortdict()
         for p in paramsblock.split(b' '):
             p = p.split(b'=', 1)
             p = [urlreq.unquote(i) for i in p]
             if len(p) < 2:
                 p.append(None)
             self._processparam(*p)
             params[p[0]] = p[1]
         return params
 
     def _processparam(self, name, value):
         """process a parameter, applying its effect if needed
 
         Parameter starting with a lower case letter are advisory and will be
         ignored when unknown.  Those starting with an upper case letter are
         mandatory and will this function will raise a KeyError when unknown.
 
         Note: no option are currently supported. Any input will be either
               ignored or failing.
         """
         if not name:
             raise ValueError('empty parameter name')
         if name[0:1] not in pycompat.bytestr(
             string.ascii_letters  # pytype: disable=wrong-arg-types
         ):
             raise ValueError('non letter first character: %s' % name)
         try:
             handler = b2streamparamsmap[name.lower()]
         except KeyError:
             if name[0:1].islower():
                 indebug(self.ui, b"ignoring unknown parameter %s" % name)
             else:
                 raise error.BundleUnknownFeatureError(params=(name,))
         else:
             handler(self, name, value)
 
     def _forwardchunks(self):
         """utility to transfer a bundle2 as binary
 
         This is made necessary by the fact the 'getbundle' command over 'ssh'
         have no way to know then the reply end, relying on the bundle to be
         interpreted to know its end. This is terrible and we are sorry, but we
         needed to move forward to get general delta enabled.
         """
         yield self._magicstring
         assert 'params' not in vars(self)
         paramssize = self._unpack(_fstreamparamsize)[0]
         if paramssize < 0:
             raise error.BundleValueError(
                 b'negative bundle param size: %i' % paramssize
             )
         if paramssize:
             params = self._readexact(paramssize)
             self._processallparams(params)
             # The payload itself is decompressed below, so drop
             # the compression parameter passed down to compensate.
             outparams = []
             for p in params.split(b' '):
                 k, v = p.split(b'=', 1)
                 if k.lower() != b'compression':
                     outparams.append(p)
             outparams = b' '.join(outparams)
             yield _pack(_fstreamparamsize, len(outparams))
             yield outparams
         else:
             yield _pack(_fstreamparamsize, paramssize)
         # From there, payload might need to be decompressed
         self._fp = self._compengine.decompressorreader(self._fp)
         emptycount = 0
         while emptycount < 2:
             # so we can brainlessly loop
             assert _fpartheadersize == _fpayloadsize
             size = self._unpack(_fpartheadersize)[0]
             yield _pack(_fpartheadersize, size)
             if size:
                 emptycount = 0
             else:
                 emptycount += 1
                 continue
             if size == flaginterrupt:
                 continue
             elif size < 0:
                 raise error.BundleValueError(b'negative chunk size: %i')
             yield self._readexact(size)
 
     def iterparts(self, seekable=False):
         """yield all parts contained in the stream"""
         cls = seekableunbundlepart if seekable else unbundlepart
         # make sure param have been loaded
         self.params
         # From there, payload need to be decompressed
         self._fp = self._compengine.decompressorreader(self._fp)
         indebug(self.ui, b'start extraction of bundle2 parts')
         headerblock = self._readpartheader()
         while headerblock is not None:
             part = cls(self.ui, headerblock, self._fp)
             yield part
             # Ensure part is fully consumed so we can start reading the next
             # part.
             part.consume()
 
             headerblock = self._readpartheader()
         indebug(self.ui, b'end of bundle2 stream')
 
     def _readpartheader(self):
         """reads a part header size and return the bytes blob
 
         returns None if empty"""
         headersize = self._unpack(_fpartheadersize)[0]
         if headersize < 0:
             raise error.BundleValueError(
                 b'negative part header size: %i' % headersize
             )
         indebug(self.ui, b'part header size: %i' % headersize)
         if headersize:
             return self._readexact(headersize)
         return None
 
     def compressed(self):
         self.params  # load params
         return self._compressed
 
     def close(self):
         """close underlying file"""
         if util.safehasattr(self._fp, 'close'):
             return self._fp.close()
 
 
 formatmap = {b'20': unbundle20}
 
 b2streamparamsmap = {}
 
 
 def b2streamparamhandler(name):
     """register a handler for a stream level parameter"""
 
     def decorator(func):
         assert name not in formatmap
         b2streamparamsmap[name] = func
         return func
 
     return decorator
 
 
 @b2streamparamhandler(b'compression')
 def processcompression(unbundler, param, value):
     """read compression parameter and install payload decompression"""
     if value not in util.compengines.supportedbundletypes:
         raise error.BundleUnknownFeatureError(params=(param,), values=(value,))
     unbundler._compengine = util.compengines.forbundletype(value)
     if value is not None:
         unbundler._compressed = True
 
 
 class bundlepart(object):
     """A bundle2 part contains application level payload
 
     The part `type` is used to route the part to the application level
     handler.
 
     The part payload is contained in ``part.data``. It could be raw bytes or a
     generator of byte chunks.
 
     You can add parameters to the part using the ``addparam`` method.
     Parameters can be either mandatory (default) or advisory. Remote side
     should be able to safely ignore the advisory ones.
 
     Both data and parameters cannot be modified after the generation has begun.
     """
 
     def __init__(
         self,
         parttype,
         mandatoryparams=(),
         advisoryparams=(),
         data=b'',
         mandatory=True,
     ):
         validateparttype(parttype)
         self.id = None
         self.type = parttype
         self._data = data
         self._mandatoryparams = list(mandatoryparams)
         self._advisoryparams = list(advisoryparams)
         # checking for duplicated entries
         self._seenparams = set()
         for pname, __ in self._mandatoryparams + self._advisoryparams:
             if pname in self._seenparams:
                 raise error.ProgrammingError(b'duplicated params: %s' % pname)
             self._seenparams.add(pname)
         # status of the part's generation:
         # - None: not started,
         # - False: currently generated,
         # - True: generation done.
         self._generated = None
         self.mandatory = mandatory
 
     def __repr__(self):
         cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
         return '<%s object at %x; id: %s; type: %s; mandatory: %s>' % (
             cls,
             id(self),
             self.id,
             self.type,
             self.mandatory,
         )
 
     def copy(self):
         """return a copy of the part
 
         The new part have the very same content but no partid assigned yet.
         Parts with generated data cannot be copied."""
         assert not util.safehasattr(self.data, 'next')
         return self.__class__(
             self.type,
             self._mandatoryparams,
             self._advisoryparams,
             self._data,
             self.mandatory,
         )
 
     # methods used to defines the part content
     @property
     def data(self):
         return self._data
 
     @data.setter
     def data(self, data):
         if self._generated is not None:
             raise error.ReadOnlyPartError(b'part is being generated')
         self._data = data
 
     @property
     def mandatoryparams(self):
         # make it an immutable tuple to force people through ``addparam``
         return tuple(self._mandatoryparams)
 
     @property
     def advisoryparams(self):
         # make it an immutable tuple to force people through ``addparam``
         return tuple(self._advisoryparams)
 
     def addparam(self, name, value=b'', mandatory=True):
         """add a parameter to the part
 
         If 'mandatory' is set to True, the remote handler must claim support
         for this parameter or the unbundling will be aborted.
 
         The 'name' and 'value' cannot exceed 255 bytes each.
         """
         if self._generated is not None:
             raise error.ReadOnlyPartError(b'part is being generated')
         if name in self._seenparams:
             raise ValueError(b'duplicated params: %s' % name)
         self._seenparams.add(name)
         params = self._advisoryparams
         if mandatory:
             params = self._mandatoryparams
         params.append((name, value))
 
     # methods used to generates the bundle2 stream
     def getchunks(self, ui):
         if self._generated is not None:
             raise error.ProgrammingError(b'part can only be consumed once')
         self._generated = False
 
         if ui.debugflag:
             msg = [b'bundle2-output-part: "%s"' % self.type]
             if not self.mandatory:
                 msg.append(b' (advisory)')
             nbmp = len(self.mandatoryparams)
             nbap = len(self.advisoryparams)
             if nbmp or nbap:
                 msg.append(b' (params:')
                 if nbmp:
                     msg.append(b' %i mandatory' % nbmp)
                 if nbap:
                     msg.append(b' %i advisory' % nbmp)
                 msg.append(b')')
             if not self.data:
                 msg.append(b' empty payload')
             elif util.safehasattr(self.data, 'next') or util.safehasattr(
                 self.data, b'__next__'
             ):
                 msg.append(b' streamed payload')
             else:
                 msg.append(b' %i bytes payload' % len(self.data))
             msg.append(b'\n')
             ui.debug(b''.join(msg))
 
         #### header
         if self.mandatory:
             parttype = self.type.upper()
         else:
             parttype = self.type.lower()
         outdebug(ui, b'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
         ## parttype
         header = [
             _pack(_fparttypesize, len(parttype)),
             parttype,
             _pack(_fpartid, self.id),
         ]
         ## parameters
         # count
         manpar = self.mandatoryparams
         advpar = self.advisoryparams
         header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
         # size
         parsizes = []
         for key, value in manpar:
             parsizes.append(len(key))
             parsizes.append(len(value))
         for key, value in advpar:
             parsizes.append(len(key))
             parsizes.append(len(value))
         paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
         header.append(paramsizes)
         # key, value
         for key, value in manpar:
             header.append(key)
             header.append(value)
         for key, value in advpar:
             header.append(key)
             header.append(value)
         ## finalize header
         try:
             headerchunk = b''.join(header)
         except TypeError:
             raise TypeError(
                 'Found a non-bytes trying to '
                 'build bundle part header: %r' % header
             )
         outdebug(ui, b'header chunk size: %i' % len(headerchunk))
         yield _pack(_fpartheadersize, len(headerchunk))
         yield headerchunk
         ## payload
         try:
             for chunk in self._payloadchunks():
                 outdebug(ui, b'payload chunk size: %i' % len(chunk))
                 yield _pack(_fpayloadsize, len(chunk))
                 yield chunk
         except GeneratorExit:
             # GeneratorExit means that nobody is listening for our
             # results anyway, so just bail quickly rather than trying
             # to produce an error part.
             ui.debug(b'bundle2-generatorexit\n')
             raise
         except BaseException as exc:
             bexc = stringutil.forcebytestr(exc)
             # backup exception data for later
             ui.debug(
                 b'bundle2-input-stream-interrupt: encoding exception %s' % bexc
             )
             tb = sys.exc_info()[2]
             msg = b'unexpected error: %s' % bexc
             interpart = bundlepart(
                 b'error:abort', [(b'message', msg)], mandatory=False
             )
             interpart.id = 0
             yield _pack(_fpayloadsize, -1)
             for chunk in interpart.getchunks(ui=ui):
                 yield chunk
             outdebug(ui, b'closing payload chunk')
             # abort current part payload
             yield _pack(_fpayloadsize, 0)
             pycompat.raisewithtb(exc, tb)
         # end of payload
         outdebug(ui, b'closing payload chunk')
         yield _pack(_fpayloadsize, 0)
         self._generated = True
 
     def _payloadchunks(self):
         """yield chunks of a the part payload
 
         Exists to handle the different methods to provide data to a part."""
         # we only support fixed size data now.
         # This will be improved in the future.
         if util.safehasattr(self.data, 'next') or util.safehasattr(
             self.data, b'__next__'
         ):
             buff = util.chunkbuffer(self.data)
             chunk = buff.read(preferedchunksize)
             while chunk:
                 yield chunk
                 chunk = buff.read(preferedchunksize)
         elif len(self.data):
             yield self.data
 
 
 flaginterrupt = -1
 
 
 class interrupthandler(unpackermixin):
     """read one part and process it with restricted capability
 
     This allows to transmit exception raised on the producer size during part
     iteration while the consumer is reading a part.
 
     Part processed in this manner only have access to a ui object,"""
 
     def __init__(self, ui, fp):
         super(interrupthandler, self).__init__(fp)
         self.ui = ui
 
     def _readpartheader(self):
         """reads a part header size and return the bytes blob
 
         returns None if empty"""
         headersize = self._unpack(_fpartheadersize)[0]
         if headersize < 0:
             raise error.BundleValueError(
                 b'negative part header size: %i' % headersize
             )
         indebug(self.ui, b'part header size: %i\n' % headersize)
         if headersize:
             return self._readexact(headersize)
         return None
 
     def __call__(self):
 
         self.ui.debug(
             b'bundle2-input-stream-interrupt: opening out of band context\n'
         )
         indebug(self.ui, b'bundle2 stream interruption, looking for a part.')
         headerblock = self._readpartheader()
         if headerblock is None:
             indebug(self.ui, b'no part found during interruption.')
             return
         part = unbundlepart(self.ui, headerblock, self._fp)
         op = interruptoperation(self.ui)
         hardabort = False
         try:
             _processpart(op, part)
         except (SystemExit, KeyboardInterrupt):
             hardabort = True
             raise
         finally:
             if not hardabort:
                 part.consume()
         self.ui.debug(
             b'bundle2-input-stream-interrupt: closing out of band context\n'
         )
 
 
 class interruptoperation(object):
     """A limited operation to be use by part handler during interruption
 
     It only have access to an ui object.
     """
 
     def __init__(self, ui):
         self.ui = ui
         self.reply = None
         self.captureoutput = False
 
     @property
     def repo(self):
         raise error.ProgrammingError(b'no repo access from stream interruption')
 
     def gettransaction(self):
         raise TransactionUnavailable(b'no repo access from stream interruption')
 
 
 def decodepayloadchunks(ui, fh):
     """Reads bundle2 part payload data into chunks.
 
     Part payload data consists of framed chunks. This function takes
     a file handle and emits those chunks.
     """
     dolog = ui.configbool(b'devel', b'bundle2.debug')
     debug = ui.debug
 
     headerstruct = struct.Struct(_fpayloadsize)
     headersize = headerstruct.size
     unpack = headerstruct.unpack
 
     readexactly = changegroup.readexactly
     read = fh.read
 
     chunksize = unpack(readexactly(fh, headersize))[0]
     indebug(ui, b'payload chunk size: %i' % chunksize)
 
     # changegroup.readexactly() is inlined below for performance.
     while chunksize:
         if chunksize >= 0:
             s = read(chunksize)
             if len(s) < chunksize:
                 raise error.Abort(
                     _(
                         b'stream ended unexpectedly '
                         b' (got %d bytes, expected %d)'
                     )
                     % (len(s), chunksize)
                 )
 
             yield s
         elif chunksize == flaginterrupt:
             # Interrupt "signal" detected. The regular stream is interrupted
             # and a bundle2 part follows. Consume it.
             interrupthandler(ui, fh)()
         else:
             raise error.BundleValueError(
                 b'negative payload chunk size: %s' % chunksize
             )
 
         s = read(headersize)
         if len(s) < headersize:
             raise error.Abort(
                 _(b'stream ended unexpectedly  (got %d bytes, expected %d)')
                 % (len(s), chunksize)
             )
 
         chunksize = unpack(s)[0]
 
         # indebug() inlined for performance.
         if dolog:
             debug(b'bundle2-input: payload chunk size: %i\n' % chunksize)
 
 
 class unbundlepart(unpackermixin):
     """a bundle part read from a bundle"""
 
     def __init__(self, ui, header, fp):
         super(unbundlepart, self).__init__(fp)
         self._seekable = util.safehasattr(fp, 'seek') and util.safehasattr(
             fp, b'tell'
         )
         self.ui = ui
         # unbundle state attr
         self._headerdata = header
         self._headeroffset = 0
         self._initialized = False
         self.consumed = False
         # part data
         self.id = None
         self.type = None
         self.mandatoryparams = None
         self.advisoryparams = None
         self.params = None
         self.mandatorykeys = ()
         self._readheader()
         self._mandatory = None
         self._pos = 0
 
     def _fromheader(self, size):
         """return the next <size> byte from the header"""
         offset = self._headeroffset
         data = self._headerdata[offset : (offset + size)]
         self._headeroffset = offset + size
         return data
 
     def _unpackheader(self, format):
         """read given format from header
 
         This automatically compute the size of the format to read."""
         data = self._fromheader(struct.calcsize(format))
         return _unpack(format, data)
 
     def _initparams(self, mandatoryparams, advisoryparams):
         """internal function to setup all logic related parameters"""
         # make it read only to prevent people touching it by mistake.
         self.mandatoryparams = tuple(mandatoryparams)
         self.advisoryparams = tuple(advisoryparams)
         # user friendly UI
         self.params = util.sortdict(self.mandatoryparams)
         self.params.update(self.advisoryparams)
         self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
 
     def _readheader(self):
         """read the header and setup the object"""
         typesize = self._unpackheader(_fparttypesize)[0]
         self.type = self._fromheader(typesize)
         indebug(self.ui, b'part type: "%s"' % self.type)
         self.id = self._unpackheader(_fpartid)[0]
         indebug(self.ui, b'part id: "%s"' % pycompat.bytestr(self.id))
         # extract mandatory bit from type
         self.mandatory = self.type != self.type.lower()
         self.type = self.type.lower()
         ## reading parameters
         # param count
         mancount, advcount = self._unpackheader(_fpartparamcount)
         indebug(self.ui, b'part parameters: %i' % (mancount + advcount))
         # param size
         fparamsizes = _makefpartparamsizes(mancount + advcount)
         paramsizes = self._unpackheader(fparamsizes)
         # make it a list of couple again
         paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
         # split mandatory from advisory
         mansizes = paramsizes[:mancount]
         advsizes = paramsizes[mancount:]
         # retrieve param value
         manparams = []
         for key, value in mansizes:
             manparams.append((self._fromheader(key), self._fromheader(value)))
         advparams = []
         for key, value in advsizes:
             advparams.append((self._fromheader(key), self._fromheader(value)))
         self._initparams(manparams, advparams)
         ## part payload
         self._payloadstream = util.chunkbuffer(self._payloadchunks())
         # we read the data, tell it
         self._initialized = True
 
     def _payloadchunks(self):
         """Generator of decoded chunks in the payload."""
         return decodepayloadchunks(self.ui, self._fp)
 
     def consume(self):
         """Read the part payload until completion.
 
         By consuming the part data, the underlying stream read offset will
         be advanced to the next part (or end of stream).
         """
         if self.consumed:
             return
 
         chunk = self.read(32768)
         while chunk:
             self._pos += len(chunk)
             chunk = self.read(32768)
 
     def read(self, size=None):
         """read payload data"""
         if not self._initialized:
             self._readheader()
         if size is None:
             data = self._payloadstream.read()
         else:
             data = self._payloadstream.read(size)
         self._pos += len(data)
         if size is None or len(data) < size:
             if not self.consumed and self._pos:
                 self.ui.debug(
                     b'bundle2-input-part: total payload size %i\n' % self._pos
                 )
             self.consumed = True
         return data
 
 
 class seekableunbundlepart(unbundlepart):
     """A bundle2 part in a bundle that is seekable.
 
     Regular ``unbundlepart`` instances can only be read once. This class
     extends ``unbundlepart`` to enable bi-directional seeking within the
     part.
 
     Bundle2 part data consists of framed chunks. Offsets when seeking
     refer to the decoded data, not the offsets in the underlying bundle2
     stream.
 
     To facilitate quickly seeking within the decoded data, instances of this
     class maintain a mapping between offsets in the underlying stream and
     the decoded payload. This mapping will consume memory in proportion
     to the number of chunks within the payload (which almost certainly
     increases in proportion with the size of the part).
     """
 
     def __init__(self, ui, header, fp):
         # (payload, file) offsets for chunk starts.
         self._chunkindex = []
 
         super(seekableunbundlepart, self).__init__(ui, header, fp)
 
     def _payloadchunks(self, chunknum=0):
         '''seek to specified chunk and start yielding data'''
         if len(self._chunkindex) == 0:
             assert chunknum == 0, b'Must start with chunk 0'
             self._chunkindex.append((0, self._tellfp()))
         else:
             assert chunknum < len(self._chunkindex), (
                 b'Unknown chunk %d' % chunknum
             )
             self._seekfp(self._chunkindex[chunknum][1])
 
         pos = self._chunkindex[chunknum][0]
 
         for chunk in decodepayloadchunks(self.ui, self._fp):
             chunknum += 1
             pos += len(chunk)
             if chunknum == len(self._chunkindex):
                 self._chunkindex.append((pos, self._tellfp()))
 
             yield chunk
 
     def _findchunk(self, pos):
         '''for a given payload position, return a chunk number and offset'''
         for chunk, (ppos, fpos) in enumerate(self._chunkindex):
             if ppos == pos:
                 return chunk, 0
             elif ppos > pos:
                 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
         raise ValueError(b'Unknown chunk')
 
     def tell(self):
         return self._pos
 
     def seek(self, offset, whence=os.SEEK_SET):
         if whence == os.SEEK_SET:
             newpos = offset
         elif whence == os.SEEK_CUR:
             newpos = self._pos + offset
         elif whence == os.SEEK_END:
             if not self.consumed:
                 # Can't use self.consume() here because it advances self._pos.
                 chunk = self.read(32768)
                 while chunk:
                     chunk = self.read(32768)
             newpos = self._chunkindex[-1][0] - offset
         else:
             raise ValueError(b'Unknown whence value: %r' % (whence,))
 
         if newpos > self._chunkindex[-1][0] and not self.consumed:
             # Can't use self.consume() here because it advances self._pos.
             chunk = self.read(32768)
             while chunk:
                 chunk = self.read(32668)
 
         if not 0 <= newpos <= self._chunkindex[-1][0]:
             raise ValueError(b'Offset out of range')
 
         if self._pos != newpos:
             chunk, internaloffset = self._findchunk(newpos)
             self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
             adjust = self.read(internaloffset)
             if len(adjust) != internaloffset:
                 raise error.Abort(_(b'Seek failed\n'))
             self._pos = newpos
 
     def _seekfp(self, offset, whence=0):
         """move the underlying file pointer
 
         This method is meant for internal usage by the bundle2 protocol only.
         They directly manipulate the low level stream including bundle2 level
         instruction.
 
         Do not use it to implement higher-level logic or methods."""
         if self._seekable:
             return self._fp.seek(offset, whence)
         else:
             raise NotImplementedError(_(b'File pointer is not seekable'))
 
     def _tellfp(self):
         """return the file offset, or None if file is not seekable
 
         This method is meant for internal usage by the bundle2 protocol only.
         They directly manipulate the low level stream including bundle2 level
         instruction.
 
         Do not use it to implement higher-level logic or methods."""
         if self._seekable:
             try:
                 return self._fp.tell()
             except IOError as e:
                 if e.errno == errno.ESPIPE:
                     self._seekable = False
                 else:
                     raise
         return None
 
 
 # These are only the static capabilities.
 # Check the 'getrepocaps' function for the rest.
 capabilities = {
     b'HG20': (),
     b'bookmarks': (),
     b'error': (b'abort', b'unsupportedcontent', b'pushraced', b'pushkey'),
     b'listkeys': (),
     b'pushkey': (),
     b'digests': tuple(sorted(util.DIGESTS.keys())),
     b'remote-changegroup': (b'http', b'https'),
     b'hgtagsfnodes': (),
     b'rev-branch-cache': (),
     b'phases': (b'heads',),
     b'stream': (b'v2',),
 }
 
 
 def getrepocaps(repo, allowpushback=False, role=None):
     """return the bundle2 capabilities for a given repo
 
     Exists to allow extensions (like evolution) to mutate the capabilities.
 
     The returned value is used for servers advertising their capabilities as
     well as clients advertising their capabilities to servers as part of
     bundle2 requests. The ``role`` argument specifies which is which.
     """
     if role not in (b'client', b'server'):
         raise error.ProgrammingError(b'role argument must be client or server')
 
     caps = capabilities.copy()
     caps[b'changegroup'] = tuple(
         sorted(changegroup.supportedincomingversions(repo))
     )
     if obsolete.isenabled(repo, obsolete.exchangeopt):
         supportedformat = tuple(b'V%i' % v for v in obsolete.formats)
         caps[b'obsmarkers'] = supportedformat
     if allowpushback:
         caps[b'pushback'] = ()
     cpmode = repo.ui.config(b'server', b'concurrent-push-mode')
     if cpmode == b'check-related':
         caps[b'checkheads'] = (b'related',)
     if b'phases' in repo.ui.configlist(b'devel', b'legacy.exchange'):
         caps.pop(b'phases')
 
     # Don't advertise stream clone support in server mode if not configured.
     if role == b'server':
         streamsupported = repo.ui.configbool(
             b'server', b'uncompressed', untrusted=True
         )
         featuresupported = repo.ui.configbool(b'server', b'bundle2.stream')
 
         if not streamsupported or not featuresupported:
             caps.pop(b'stream')
     # Else always advertise support on client, because payload support
     # should always be advertised.
 
     return caps
 
 
 def bundle2caps(remote):
     """return the bundle capabilities of a peer as dict"""
     raw = remote.capable(b'bundle2')
     if not raw and raw != b'':
         return {}
     capsblob = urlreq.unquote(remote.capable(b'bundle2'))
     return decodecaps(capsblob)
 
 
 def obsmarkersversion(caps):
     """extract the list of supported obsmarkers versions from a bundle2caps dict"""
     obscaps = caps.get(b'obsmarkers', ())
     return [int(c[1:]) for c in obscaps if c.startswith(b'V')]
 
 
 def writenewbundle(
     ui,
     repo,
     source,
     filename,
     bundletype,
     outgoing,
     opts,
     vfs=None,
     compression=None,
     compopts=None,
 ):
     if bundletype.startswith(b'HG10'):
         cg = changegroup.makechangegroup(repo, outgoing, b'01', source)
         return writebundle(
             ui,
             cg,
             filename,
             bundletype,
             vfs=vfs,
             compression=compression,
             compopts=compopts,
         )
     elif not bundletype.startswith(b'HG20'):
         raise error.ProgrammingError(b'unknown bundle type: %s' % bundletype)
 
     caps = {}
     if b'obsolescence' in opts:
         caps[b'obsmarkers'] = (b'V1',)
     bundle = bundle20(ui, caps)
     bundle.setcompression(compression, compopts)
     _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
     chunkiter = bundle.getchunks()
 
     return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
 
 
 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
     # We should eventually reconcile this logic with the one behind
     # 'exchange.getbundle2partsgenerator'.
     #
     # The type of input from 'getbundle' and 'writenewbundle' are a bit
     # different right now. So we keep them separated for now for the sake of
     # simplicity.
 
     # we might not always want a changegroup in such bundle, for example in
     # stream bundles
     if opts.get(b'changegroup', True):
         cgversion = opts.get(b'cg.version')
         if cgversion is None:
             cgversion = changegroup.safeversion(repo)
         cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
         part = bundler.newpart(b'changegroup', data=cg.getchunks())
         part.addparam(b'version', cg.version)
         if b'clcount' in cg.extras:
             part.addparam(
                 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
             )
         if opts.get(b'phases') and repo.revs(
             b'%ln and secret()', outgoing.ancestorsof
         ):
             part.addparam(
                 b'targetphase', b'%d' % phases.secret, mandatory=False
             )
         if b'exp-sidedata-flag' in repo.requirements:
             part.addparam(b'exp-sidedata', b'1')
 
     if opts.get(b'streamv2', False):
         addpartbundlestream2(bundler, repo, stream=True)
 
     if opts.get(b'tagsfnodescache', True):
         addparttagsfnodescache(repo, bundler, outgoing)
 
     if opts.get(b'revbranchcache', True):
         addpartrevbranchcache(repo, bundler, outgoing)
 
     if opts.get(b'obsolescence', False):
         obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
         buildobsmarkerspart(
             bundler,
             obsmarkers,
             mandatory=opts.get(b'obsolescence-mandatory', True),
         )
 
     if opts.get(b'phases', False):
         headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
         phasedata = phases.binaryencode(headsbyphase)
         bundler.newpart(b'phase-heads', data=phasedata)
 
 
 def addparttagsfnodescache(repo, bundler, outgoing):
     # we include the tags fnode cache for the bundle changeset
     # (as an optional parts)
     cache = tags.hgtagsfnodescache(repo.unfiltered())
     chunks = []
 
     # .hgtags fnodes are only relevant for head changesets. While we could
     # transfer values for all known nodes, there will likely be little to
     # no benefit.
     #
     # We don't bother using a generator to produce output data because
     # a) we only have 40 bytes per head and even esoteric numbers of heads
     # consume little memory (1M heads is 40MB) b) we don't want to send the
     # part if we don't have entries and knowing if we have entries requires
     # cache lookups.
     for node in outgoing.ancestorsof:
         # Don't compute missing, as this may slow down serving.
         fnode = cache.getfnode(node, computemissing=False)
-        if fnode is not None:
+        if fnode:
             chunks.extend([node, fnode])
 
     if chunks:
         bundler.newpart(b'hgtagsfnodes', data=b''.join(chunks))
 
 
 def addpartrevbranchcache(repo, bundler, outgoing):
     # we include the rev branch cache for the bundle changeset
     # (as an optional parts)
     cache = repo.revbranchcache()
     cl = repo.unfiltered().changelog
     branchesdata = collections.defaultdict(lambda: (set(), set()))
     for node in outgoing.missing:
         branch, close = cache.branchinfo(cl.rev(node))
         branchesdata[branch][close].add(node)
 
     def generate():
         for branch, (nodes, closed) in sorted(branchesdata.items()):
             utf8branch = encoding.fromlocal(branch)
             yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
             yield utf8branch
             for n in sorted(nodes):
                 yield n
             for n in sorted(closed):
                 yield n
 
     bundler.newpart(b'cache:rev-branch-cache', data=generate(), mandatory=False)
 
 
 def _formatrequirementsspec(requirements):
     requirements = [req for req in requirements if req != b"shared"]
     return urlreq.quote(b','.join(sorted(requirements)))
 
 
 def _formatrequirementsparams(requirements):
     requirements = _formatrequirementsspec(requirements)
     params = b"%s%s" % (urlreq.quote(b"requirements="), requirements)
     return params
 
 
 def addpartbundlestream2(bundler, repo, **kwargs):
     if not kwargs.get('stream', False):
         return
 
     if not streamclone.allowservergeneration(repo):
         raise error.Abort(
             _(
                 b'stream data requested but server does not allow '
                 b'this feature'
             ),
             hint=_(
                 b'well-behaved clients should not be '
                 b'requesting stream data from servers not '
                 b'advertising it; the client may be buggy'
             ),
         )
 
     # Stream clones don't compress well. And compression undermines a
     # goal of stream clones, which is to be fast. Communicate the desire
     # to avoid compression to consumers of the bundle.
     bundler.prefercompressed = False
 
     # get the includes and excludes
     includepats = kwargs.get('includepats')
     excludepats = kwargs.get('excludepats')
 
     narrowstream = repo.ui.configbool(
         b'experimental', b'server.stream-narrow-clones'
     )
 
     if (includepats or excludepats) and not narrowstream:
         raise error.Abort(_(b'server does not support narrow stream clones'))
 
     includeobsmarkers = False
     if repo.obsstore:
         remoteversions = obsmarkersversion(bundler.capabilities)
         if not remoteversions:
             raise error.Abort(
                 _(
                     b'server has obsolescence markers, but client '
                     b'cannot receive them via stream clone'
                 )
             )
         elif repo.obsstore._version in remoteversions:
             includeobsmarkers = True
 
     filecount, bytecount, it = streamclone.generatev2(
         repo, includepats, excludepats, includeobsmarkers
     )
     requirements = _formatrequirementsspec(repo.requirements)
     part = bundler.newpart(b'stream2', data=it)
     part.addparam(b'bytecount', b'%d' % bytecount, mandatory=True)
     part.addparam(b'filecount', b'%d' % filecount, mandatory=True)
     part.addparam(b'requirements', requirements, mandatory=True)
 
 
 def buildobsmarkerspart(bundler, markers, mandatory=True):
     """add an obsmarker part to the bundler with <markers>
 
     No part is created if markers is empty.
     Raises ValueError if the bundler doesn't support any known obsmarker format.
     """
     if not markers:
         return None
 
     remoteversions = obsmarkersversion(bundler.capabilities)
     version = obsolete.commonversion(remoteversions)
     if version is None:
         raise ValueError(b'bundler does not support common obsmarker format')
     stream = obsolete.encodemarkers(markers, True, version=version)
     return bundler.newpart(b'obsmarkers', data=stream, mandatory=mandatory)
 
 
 def writebundle(
     ui, cg, filename, bundletype, vfs=None, compression=None, compopts=None
 ):
     """Write a bundle file and return its filename.
 
     Existing files will not be overwritten.
     If no filename is specified, a temporary file is created.
     bz2 compression can be turned off.
     The bundle file will be deleted in case of errors.
     """
 
     if bundletype == b"HG20":
         bundle = bundle20(ui)
         bundle.setcompression(compression, compopts)
         part = bundle.newpart(b'changegroup', data=cg.getchunks())
         part.addparam(b'version', cg.version)
         if b'clcount' in cg.extras:
             part.addparam(
                 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
             )
         chunkiter = bundle.getchunks()
     else:
         # compression argument is only for the bundle2 case
         assert compression is None
         if cg.version != b'01':
             raise error.Abort(
                 _(b'old bundle types only supports v1 changegroups')
             )
         header, comp = bundletypes[bundletype]
         if comp not in util.compengines.supportedbundletypes:
             raise error.Abort(_(b'unknown stream compression type: %s') % comp)
         compengine = util.compengines.forbundletype(comp)
 
         def chunkiter():
             yield header
             for chunk in compengine.compressstream(cg.getchunks(), compopts):
                 yield chunk
 
         chunkiter = chunkiter()
 
     # parse the changegroup data, otherwise we will block
     # in case of sshrepo because we don't know the end of the stream
     return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
 
 
 def combinechangegroupresults(op):
     """logic to combine 0 or more addchangegroup results into one"""
     results = [r.get(b'return', 0) for r in op.records[b'changegroup']]
     changedheads = 0
     result = 1
     for ret in results:
         # If any changegroup result is 0, return 0
         if ret == 0:
             result = 0
             break
         if ret < -1:
             changedheads += ret + 1
         elif ret > 1:
             changedheads += ret - 1
     if changedheads > 0:
         result = 1 + changedheads
     elif changedheads < 0:
         result = -1 + changedheads
     return result
 
 
 @parthandler(
     b'changegroup',
     (
         b'version',
         b'nbchanges',
         b'exp-sidedata',
         b'treemanifest',
         b'targetphase',
     ),
 )
 def handlechangegroup(op, inpart):
     """apply a changegroup part on the repo"""
     from . import localrepo
 
     tr = op.gettransaction()
     unpackerversion = inpart.params.get(b'version', b'01')
     # We should raise an appropriate exception here
     cg = changegroup.getunbundler(unpackerversion, inpart, None)
     # the source and url passed here are overwritten by the one contained in
     # the transaction.hookargs argument. So 'bundle2' is a placeholder
     nbchangesets = None
     if b'nbchanges' in inpart.params:
         nbchangesets = int(inpart.params.get(b'nbchanges'))
     if b'treemanifest' in inpart.params and not scmutil.istreemanifest(op.repo):
         if len(op.repo.changelog) != 0:
             raise error.Abort(
                 _(
                     b"bundle contains tree manifests, but local repo is "
                     b"non-empty and does not use tree manifests"
                 )
             )
         op.repo.requirements.add(requirements.TREEMANIFEST_REQUIREMENT)
         op.repo.svfs.options = localrepo.resolvestorevfsoptions(
             op.repo.ui, op.repo.requirements, op.repo.features
         )
         scmutil.writereporequirements(op.repo)
 
     bundlesidedata = bool(b'exp-sidedata' in inpart.params)
     reposidedata = bool(b'exp-sidedata-flag' in op.repo.requirements)
     if reposidedata and not bundlesidedata:
         msg = b"repository is using sidedata but the bundle source do not"
         hint = b'this is currently unsupported'
         raise error.Abort(msg, hint=hint)
 
     extrakwargs = {}
     targetphase = inpart.params.get(b'targetphase')
     if targetphase is not None:
         extrakwargs['targetphase'] = int(targetphase)
     ret = _processchangegroup(
         op,
         cg,
         tr,
         b'bundle2',
         b'bundle2',
         expectedtotal=nbchangesets,
         **extrakwargs
     )
     if op.reply is not None:
         # This is definitely not the final form of this
         # return. But one need to start somewhere.
         part = op.reply.newpart(b'reply:changegroup', mandatory=False)
         part.addparam(
             b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
         )
         part.addparam(b'return', b'%i' % ret, mandatory=False)
     assert not inpart.read()
 
 
 _remotechangegroupparams = tuple(
     [b'url', b'size', b'digests']
     + [b'digest:%s' % k for k in util.DIGESTS.keys()]
 )
 
 
 @parthandler(b'remote-changegroup', _remotechangegroupparams)
 def handleremotechangegroup(op, inpart):
     """apply a bundle10 on the repo, given an url and validation information
 
     All the information about the remote bundle to import are given as
     parameters. The parameters include:
       - url: the url to the bundle10.
       - size: the bundle10 file size. It is used to validate what was
         retrieved by the client matches the server knowledge about the bundle.
       - digests: a space separated list of the digest types provided as
         parameters.
       - digest:<digest-type>: the hexadecimal representation of the digest with
         that name. Like the size, it is used to validate what was retrieved by
         the client matches what the server knows about the bundle.
 
     When multiple digest types are given, all of them are checked.
     """
     try:
         raw_url = inpart.params[b'url']
     except KeyError:
         raise error.Abort(_(b'remote-changegroup: missing "%s" param') % b'url')
     parsed_url = util.url(raw_url)
     if parsed_url.scheme not in capabilities[b'remote-changegroup']:
         raise error.Abort(
             _(b'remote-changegroup does not support %s urls')
             % parsed_url.scheme
         )
 
     try:
         size = int(inpart.params[b'size'])
     except ValueError:
         raise error.Abort(
             _(b'remote-changegroup: invalid value for param "%s"') % b'size'
         )
     except KeyError:
         raise error.Abort(
             _(b'remote-changegroup: missing "%s" param') % b'size'
         )
 
     digests = {}
     for typ in inpart.params.get(b'digests', b'').split():
         param = b'digest:%s' % typ
         try:
             value = inpart.params[param]
         except KeyError:
             raise error.Abort(
                 _(b'remote-changegroup: missing "%s" param') % param
             )
         digests[typ] = value
 
     real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
 
     tr = op.gettransaction()
     from . import exchange
 
     cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
     if not isinstance(cg, changegroup.cg1unpacker):
         raise error.Abort(
             _(b'%s: not a bundle version 1.0') % util.hidepassword(raw_url)
         )
     ret = _processchangegroup(op, cg, tr, b'bundle2', b'bundle2')
     if op.reply is not None:
         # This is definitely not the final form of this
         # return. But one need to start somewhere.
         part = op.reply.newpart(b'reply:changegroup')
         part.addparam(
             b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
         )
         part.addparam(b'return', b'%i' % ret, mandatory=False)
     try:
         real_part.validate()
     except error.Abort as e:
         raise error.Abort(
             _(b'bundle at %s is corrupted:\n%s')
             % (util.hidepassword(raw_url), e.message)
         )
     assert not inpart.read()
 
 
 @parthandler(b'reply:changegroup', (b'return', b'in-reply-to'))
 def handlereplychangegroup(op, inpart):
     ret = int(inpart.params[b'return'])
     replyto = int(inpart.params[b'in-reply-to'])
     op.records.add(b'changegroup', {b'return': ret}, replyto)
 
 
 @parthandler(b'check:bookmarks')
 def handlecheckbookmarks(op, inpart):
     """check location of bookmarks
 
     This part is to be used to detect push race regarding bookmark, it
     contains binary encoded (bookmark, node) tuple. If the local state does
     not marks the one in the part, a PushRaced exception is raised
     """
     bookdata = bookmarks.binarydecode(inpart)
 
     msgstandard = (
         b'remote repository changed while pushing - please try again '
         b'(bookmark "%s" move from %s to %s)'
     )
     msgmissing = (
         b'remote repository changed while pushing - please try again '
         b'(bookmark "%s" is missing, expected %s)'
     )
     msgexist = (
         b'remote repository changed while pushing - please try again '
         b'(bookmark "%s" set on %s, expected missing)'
     )
     for book, node in bookdata:
         currentnode = op.repo._bookmarks.get(book)
         if currentnode != node:
             if node is None:
                 finalmsg = msgexist % (book, short(currentnode))
             elif currentnode is None:
                 finalmsg = msgmissing % (book, short(node))
             else:
                 finalmsg = msgstandard % (
                     book,
                     short(node),
                     short(currentnode),
                 )
             raise error.PushRaced(finalmsg)
 
 
 @parthandler(b'check:heads')
 def handlecheckheads(op, inpart):
     """check that head of the repo did not change
 
     This is used to detect a push race when using unbundle.
     This replaces the "heads" argument of unbundle."""
     h = inpart.read(20)
     heads = []
     while len(h) == 20:
         heads.append(h)
         h = inpart.read(20)
     assert not h
     # Trigger a transaction so that we are guaranteed to have the lock now.
     if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
         op.gettransaction()
     if sorted(heads) != sorted(op.repo.heads()):
         raise error.PushRaced(
             b'remote repository changed while pushing - please try again'
         )
 
 
 @parthandler(b'check:updated-heads')
 def handlecheckupdatedheads(op, inpart):
     """check for race on the heads touched by a push
 
     This is similar to 'check:heads' but focus on the heads actually updated
     during the push. If other activities happen on unrelated heads, it is
     ignored.
 
     This allow server with high traffic to avoid push contention as long as
     unrelated parts of the graph are involved."""
     h = inpart.read(20)
     heads = []
     while len(h) == 20:
         heads.append(h)
         h = inpart.read(20)
     assert not h
     # trigger a transaction so that we are guaranteed to have the lock now.
     if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
         op.gettransaction()
 
     currentheads = set()
     for ls in op.repo.branchmap().iterheads():
         currentheads.update(ls)
 
     for h in heads:
         if h not in currentheads:
             raise error.PushRaced(
                 b'remote repository changed while pushing - '
                 b'please try again'
             )
 
 
 @parthandler(b'check:phases')
 def handlecheckphases(op, inpart):
     """check that phase boundaries of the repository did not change
 
     This is used to detect a push race.
     """
     phasetonodes = phases.binarydecode(inpart)
     unfi = op.repo.unfiltered()
     cl = unfi.changelog
     phasecache = unfi._phasecache
     msg = (
         b'remote repository changed while pushing - please try again '
         b'(%s is %s expected %s)'
     )
     for expectedphase, nodes in pycompat.iteritems(phasetonodes):
         for n in nodes:
             actualphase = phasecache.phase(unfi, cl.rev(n))
             if actualphase != expectedphase:
                 finalmsg = msg % (
                     short(n),
                     phases.phasenames[actualphase],
                     phases.phasenames[expectedphase],
                 )
                 raise error.PushRaced(finalmsg)
 
 
 @parthandler(b'output')
 def handleoutput(op, inpart):
     """forward output captured on the server to the client"""
     for line in inpart.read().splitlines():
         op.ui.status(_(b'remote: %s\n') % line)
 
 
 @parthandler(b'replycaps')
 def handlereplycaps(op, inpart):
     """Notify that a reply bundle should be created
 
     The payload contains the capabilities information for the reply"""
     caps = decodecaps(inpart.read())
     if op.reply is None:
         op.reply = bundle20(op.ui, caps)
 
 
 class AbortFromPart(error.Abort):
     """Sub-class of Abort that denotes an error from a bundle2 part."""
 
 
 @parthandler(b'error:abort', (b'message', b'hint'))
 def handleerrorabort(op, inpart):
     """Used to transmit abort error over the wire"""
     raise AbortFromPart(
         inpart.params[b'message'], hint=inpart.params.get(b'hint')
     )
 
 
 @parthandler(
     b'error:pushkey',
     (b'namespace', b'key', b'new', b'old', b'ret', b'in-reply-to'),
 )
 def handleerrorpushkey(op, inpart):
     """Used to transmit failure of a mandatory pushkey over the wire"""
     kwargs = {}
     for name in (b'namespace', b'key', b'new', b'old', b'ret'):
         value = inpart.params.get(name)
         if value is not None:
             kwargs[name] = value
     raise error.PushkeyFailed(
         inpart.params[b'in-reply-to'], **pycompat.strkwargs(kwargs)
     )
 
 
 @parthandler(b'error:unsupportedcontent', (b'parttype', b'params'))
 def handleerrorunsupportedcontent(op, inpart):
     """Used to transmit unknown content error over the wire"""
     kwargs = {}
     parttype = inpart.params.get(b'parttype')
     if parttype is not None:
         kwargs[b'parttype'] = parttype
     params = inpart.params.get(b'params')
     if params is not None:
         kwargs[b'params'] = params.split(b'\0')
 
     raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
 
 
 @parthandler(b'error:pushraced', (b'message',))
 def handleerrorpushraced(op, inpart):
     """Used to transmit push race error over the wire"""
     raise error.ResponseError(_(b'push failed:'), inpart.params[b'message'])
 
 
 @parthandler(b'listkeys', (b'namespace',))
 def handlelistkeys(op, inpart):
     """retrieve pushkey namespace content stored in a bundle2"""
     namespace = inpart.params[b'namespace']
     r = pushkey.decodekeys(inpart.read())
     op.records.add(b'listkeys', (namespace, r))
 
 
 @parthandler(b'pushkey', (b'namespace', b'key', b'old', b'new'))
 def handlepushkey(op, inpart):
     """process a pushkey request"""
     dec = pushkey.decode
     namespace = dec(inpart.params[b'namespace'])
     key = dec(inpart.params[b'key'])
     old = dec(inpart.params[b'old'])
     new = dec(inpart.params[b'new'])
     # Grab the transaction to ensure that we have the lock before performing the
     # pushkey.
     if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
         op.gettransaction()
     ret = op.repo.pushkey(namespace, key, old, new)
     record = {b'namespace': namespace, b'key': key, b'old': old, b'new': new}
     op.records.add(b'pushkey', record)
     if op.reply is not None:
         rpart = op.reply.newpart(b'reply:pushkey')
         rpart.addparam(
             b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
         )
         rpart.addparam(b'return', b'%i' % ret, mandatory=False)
     if inpart.mandatory and not ret:
         kwargs = {}
         for key in (b'namespace', b'key', b'new', b'old', b'ret'):
             if key in inpart.params:
                 kwargs[key] = inpart.params[key]
         raise error.PushkeyFailed(
             partid=b'%d' % inpart.id, **pycompat.strkwargs(kwargs)
         )
 
 
 @parthandler(b'bookmarks')
 def handlebookmark(op, inpart):
     """transmit bookmark information
 
     The part contains binary encoded bookmark information.
 
     The exact behavior of this part can be controlled by the 'bookmarks' mode
     on the bundle operation.
 
     When mode is 'apply' (the default) the bookmark information is applied as
     is to the unbundling repository. Make sure a 'check:bookmarks' part is
     issued earlier to check for push races in such update. This behavior is
     suitable for pushing.
 
     When mode is 'records', the information is recorded into the 'bookmarks'
     records of the bundle operation. This behavior is suitable for pulling.
     """
     changes = bookmarks.binarydecode(inpart)
 
     pushkeycompat = op.repo.ui.configbool(
         b'server', b'bookmarks-pushkey-compat'
     )
     bookmarksmode = op.modes.get(b'bookmarks', b'apply')
 
     if bookmarksmode == b'apply':
         tr = op.gettransaction()
         bookstore = op.repo._bookmarks
         if pushkeycompat:
             allhooks = []
             for book, node in changes:
                 hookargs = tr.hookargs.copy()
                 hookargs[b'pushkeycompat'] = b'1'
                 hookargs[b'namespace'] = b'bookmarks'
                 hookargs[b'key'] = book
                 hookargs[b'old'] = hex(bookstore.get(book, b''))
                 hookargs[b'new'] = hex(node if node is not None else b'')
                 allhooks.append(hookargs)
 
             for hookargs in allhooks:
                 op.repo.hook(
                     b'prepushkey', throw=True, **pycompat.strkwargs(hookargs)
                 )
 
         for book, node in changes:
             if bookmarks.isdivergent(book):
                 msg = _(b'cannot accept divergent bookmark %s!') % book
                 raise error.Abort(msg)
 
         bookstore.applychanges(op.repo, op.gettransaction(), changes)
 
         if pushkeycompat:
 
             def runhook(unused_success):
                 for hookargs in allhooks:
                     op.repo.hook(b'pushkey', **pycompat.strkwargs(hookargs))
 
             op.repo._afterlock(runhook)
 
     elif bookmarksmode == b'records':
         for book, node in changes:
             record = {b'bookmark': book, b'node': node}
             op.records.add(b'bookmarks', record)
     else:
         raise error.ProgrammingError(
             b'unkown bookmark mode: %s' % bookmarksmode
         )
 
 
 @parthandler(b'phase-heads')
 def handlephases(op, inpart):
     """apply phases from bundle part to repo"""
     headsbyphase = phases.binarydecode(inpart)
     phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
 
 
 @parthandler(b'reply:pushkey', (b'return', b'in-reply-to'))
 def handlepushkeyreply(op, inpart):
     """retrieve the result of a pushkey request"""
     ret = int(inpart.params[b'return'])
     partid = int(inpart.params[b'in-reply-to'])
     op.records.add(b'pushkey', {b'return': ret}, partid)
 
 
 @parthandler(b'obsmarkers')
 def handleobsmarker(op, inpart):
     """add a stream of obsmarkers to the repo"""
     tr = op.gettransaction()
     markerdata = inpart.read()
     if op.ui.config(b'experimental', b'obsmarkers-exchange-debug'):
         op.ui.writenoi18n(
             b'obsmarker-exchange: %i bytes received\n' % len(markerdata)
         )
     # The mergemarkers call will crash if marker creation is not enabled.
     # we want to avoid this if the part is advisory.
     if not inpart.mandatory and op.repo.obsstore.readonly:
         op.repo.ui.debug(
             b'ignoring obsolescence markers, feature not enabled\n'
         )
         return
     new = op.repo.obsstore.mergemarkers(tr, markerdata)
     op.repo.invalidatevolatilesets()
     op.records.add(b'obsmarkers', {b'new': new})
     if op.reply is not None:
         rpart = op.reply.newpart(b'reply:obsmarkers')
         rpart.addparam(
             b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
         )
         rpart.addparam(b'new', b'%i' % new, mandatory=False)
 
 
 @parthandler(b'reply:obsmarkers', (b'new', b'in-reply-to'))
 def handleobsmarkerreply(op, inpart):
     """retrieve the result of a pushkey request"""
     ret = int(inpart.params[b'new'])
     partid = int(inpart.params[b'in-reply-to'])
     op.records.add(b'obsmarkers', {b'new': ret}, partid)
 
 
 @parthandler(b'hgtagsfnodes')
 def handlehgtagsfnodes(op, inpart):
     """Applies .hgtags fnodes cache entries to the local repo.
 
     Payload is pairs of 20 byte changeset nodes and filenodes.
     """
     # Grab the transaction so we ensure that we have the lock at this point.
     if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
         op.gettransaction()
     cache = tags.hgtagsfnodescache(op.repo.unfiltered())
 
     count = 0
     while True:
         node = inpart.read(20)
         fnode = inpart.read(20)
         if len(node) < 20 or len(fnode) < 20:
             op.ui.debug(b'ignoring incomplete received .hgtags fnodes data\n')
             break
         cache.setfnode(node, fnode)
         count += 1
 
     cache.write()
     op.ui.debug(b'applied %i hgtags fnodes cache entries\n' % count)
 
 
 rbcstruct = struct.Struct(b'>III')
 
 
 @parthandler(b'cache:rev-branch-cache')
 def handlerbc(op, inpart):
     """Legacy part, ignored for compatibility with bundles from or
     for Mercurial before 5.7. Newer Mercurial computes the cache
     efficiently enough during unbundling that the additional transfer
     is unnecessary."""
 
 
 @parthandler(b'pushvars')
 def bundle2getvars(op, part):
     '''unbundle a bundle2 containing shellvars on the server'''
     # An option to disable unbundling on server-side for security reasons
     if op.ui.configbool(b'push', b'pushvars.server'):
         hookargs = {}
         for key, value in part.advisoryparams:
             key = key.upper()
             # We want pushed variables to have USERVAR_ prepended so we know
             # they came from the --pushvar flag.
             key = b"USERVAR_" + key
             hookargs[key] = value
         op.addhookargs(hookargs)
 
 
 @parthandler(b'stream2', (b'requirements', b'filecount', b'bytecount'))
 def handlestreamv2bundle(op, part):
 
     requirements = urlreq.unquote(part.params[b'requirements']).split(b',')
     filecount = int(part.params[b'filecount'])
     bytecount = int(part.params[b'bytecount'])
 
     repo = op.repo
     if len(repo):
         msg = _(b'cannot apply stream clone to non empty repository')
         raise error.Abort(msg)
 
     repo.ui.debug(b'applying stream bundle\n')
     streamclone.applybundlev2(repo, part, filecount, bytecount, requirements)
 
 
 def widen_bundle(
     bundler, repo, oldmatcher, newmatcher, common, known, cgversion, ellipses
 ):
     """generates bundle2 for widening a narrow clone
 
     bundler is the bundle to which data should be added
     repo is the localrepository instance
     oldmatcher matches what the client already has
     newmatcher matches what the client needs (including what it already has)
     common is set of common heads between server and client
     known is a set of revs known on the client side (used in ellipses)
     cgversion is the changegroup version to send
     ellipses is boolean value telling whether to send ellipses data or not
 
     returns bundle2 of the data required for extending
     """
     commonnodes = set()
     cl = repo.changelog
     for r in repo.revs(b"::%ln", common):
         commonnodes.add(cl.node(r))
     if commonnodes:
         packer = changegroup.getbundler(
             cgversion,
             repo,
             oldmatcher=oldmatcher,
             matcher=newmatcher,
             fullnodes=commonnodes,
         )
         cgdata = packer.generate(
             {nullid},
             list(commonnodes),
             False,
             b'narrow_widen',
             changelog=False,
         )
 
         part = bundler.newpart(b'changegroup', data=cgdata)
         part.addparam(b'version', cgversion)
         if scmutil.istreemanifest(repo):
             part.addparam(b'treemanifest', b'1')
         if b'exp-sidedata-flag' in repo.requirements:
             part.addparam(b'exp-sidedata', b'1')
 
     return bundler

mercurial/debugcommands.py

@@ -1,4749 +1,4755 b''
 # debugcommands.py - command processing for debug* commands
 #
 # Copyright 2005-2016 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 from __future__ import absolute_import
 
 import codecs
 import collections
 import difflib
 import errno
 import glob
 import operator
 import os
 import platform
 import random
 import re
 import socket
 import ssl
 import stat
 import string
 import subprocess
 import sys
 import time
 
 from .i18n import _
 from .node import (
     bin,
     hex,
     nullid,
     nullrev,
     short,
 )
 from .pycompat import (
     getattr,
     open,
 )
 from . import (
     bundle2,
     bundlerepo,
     changegroup,
     cmdutil,
     color,
     context,
     copies,
     dagparser,
     encoding,
     error,
     exchange,
     extensions,
     filemerge,
     filesetlang,
     formatter,
     hg,
     httppeer,
     localrepo,
     lock as lockmod,
     logcmdutil,
     mergestate as mergestatemod,
     metadata,
     obsolete,
     obsutil,
     pathutil,
     phases,
     policy,
     pvec,
     pycompat,
     registrar,
     repair,
     repoview,
     revlog,
     revset,
     revsetlang,
     scmutil,
     setdiscovery,
     simplemerge,
     sshpeer,
     sslutil,
     streamclone,
     strip,
     tags as tagsmod,
     templater,
     treediscovery,
     upgrade,
     url as urlmod,
     util,
     vfs as vfsmod,
     wireprotoframing,
     wireprotoserver,
     wireprotov2peer,
 )
 from .utils import (
     cborutil,
     compression,
     dateutil,
     procutil,
     stringutil,
 )
 
 from .revlogutils import (
     deltas as deltautil,
     nodemap,
     sidedata,
 )
 
 release = lockmod.release
 
 table = {}
 table.update(strip.command._table)
 command = registrar.command(table)
 
 
 @command(b'debugancestor', [], _(b'[INDEX] REV1 REV2'), optionalrepo=True)
 def debugancestor(ui, repo, *args):
     """find the ancestor revision of two revisions in a given index"""
     if len(args) == 3:
         index, rev1, rev2 = args
         r = revlog.revlog(vfsmod.vfs(encoding.getcwd(), audit=False), index)
         lookup = r.lookup
     elif len(args) == 2:
         if not repo:
             raise error.Abort(
                 _(b'there is no Mercurial repository here (.hg not found)')
             )
         rev1, rev2 = args
         r = repo.changelog
         lookup = repo.lookup
     else:
         raise error.Abort(_(b'either two or three arguments required'))
     a = r.ancestor(lookup(rev1), lookup(rev2))
     ui.write(b'%d:%s\n' % (r.rev(a), hex(a)))
 
 
 @command(b'debugantivirusrunning', [])
 def debugantivirusrunning(ui, repo):
     """attempt to trigger an antivirus scanner to see if one is active"""
     with repo.cachevfs.open('eicar-test-file.com', b'wb') as f:
         f.write(
             util.b85decode(
                 # This is a base85-armored version of the EICAR test file. See
                 # https://en.wikipedia.org/wiki/EICAR_test_file for details.
                 b'ST#=}P$fV?P+K%yP+C|uG$>GBDK|qyDK~v2MM*<JQY}+dK~6+LQba95P'
                 b'E<)&Nm5l)EmTEQR4qnHOhq9iNGnJx'
             )
         )
     # Give an AV engine time to scan the file.
     time.sleep(2)
     util.unlink(repo.cachevfs.join('eicar-test-file.com'))
 
 
 @command(b'debugapplystreamclonebundle', [], b'FILE')
 def debugapplystreamclonebundle(ui, repo, fname):
     """apply a stream clone bundle file"""
     f = hg.openpath(ui, fname)
     gen = exchange.readbundle(ui, f, fname)
     gen.apply(repo)
 
 
 @command(
     b'debugbuilddag',
     [
         (
             b'm',
             b'mergeable-file',
             None,
             _(b'add single file mergeable changes'),
         ),
         (
             b'o',
             b'overwritten-file',
             None,
             _(b'add single file all revs overwrite'),
         ),
         (b'n', b'new-file', None, _(b'add new file at each rev')),
     ],
     _(b'[OPTION]... [TEXT]'),
 )
 def debugbuilddag(
     ui,
     repo,
     text=None,
     mergeable_file=False,
     overwritten_file=False,
     new_file=False,
 ):
     """builds a repo with a given DAG from scratch in the current empty repo
 
     The description of the DAG is read from stdin if not given on the
     command line.
 
     Elements:
 
      - "+n" is a linear run of n nodes based on the current default parent
      - "." is a single node based on the current default parent
      - "$" resets the default parent to null (implied at the start);
            otherwise the default parent is always the last node created
      - "<p" sets the default parent to the backref p
      - "*p" is a fork at parent p, which is a backref
      - "*p1/p2" is a merge of parents p1 and p2, which are backrefs
      - "/p2" is a merge of the preceding node and p2
      - ":tag" defines a local tag for the preceding node
      - "@branch" sets the named branch for subsequent nodes
      - "#...\\n" is a comment up to the end of the line
 
     Whitespace between the above elements is ignored.
 
     A backref is either
 
      - a number n, which references the node curr-n, where curr is the current
        node, or
      - the name of a local tag you placed earlier using ":tag", or
      - empty to denote the default parent.
 
     All string valued-elements are either strictly alphanumeric, or must
     be enclosed in double quotes ("..."), with "\\" as escape character.
     """
 
     if text is None:
         ui.status(_(b"reading DAG from stdin\n"))
         text = ui.fin.read()
 
     cl = repo.changelog
     if len(cl) > 0:
         raise error.Abort(_(b'repository is not empty'))
 
     # determine number of revs in DAG
     total = 0
     for type, data in dagparser.parsedag(text):
         if type == b'n':
             total += 1
 
     if mergeable_file:
         linesperrev = 2
         # make a file with k lines per rev
         initialmergedlines = [
             b'%d' % i for i in pycompat.xrange(0, total * linesperrev)
         ]
         initialmergedlines.append(b"")
 
     tags = []
     progress = ui.makeprogress(
         _(b'building'), unit=_(b'revisions'), total=total
     )
     with progress, repo.wlock(), repo.lock(), repo.transaction(b"builddag"):
         at = -1
         atbranch = b'default'
         nodeids = []
         id = 0
         progress.update(id)
         for type, data in dagparser.parsedag(text):
             if type == b'n':
                 ui.note((b'node %s\n' % pycompat.bytestr(data)))
                 id, ps = data
 
                 files = []
                 filecontent = {}
 
                 p2 = None
                 if mergeable_file:
                     fn = b"mf"
                     p1 = repo[ps[0]]
                     if len(ps) > 1:
                         p2 = repo[ps[1]]
                         pa = p1.ancestor(p2)
                         base, local, other = [
                             x[fn].data() for x in (pa, p1, p2)
                         ]
                         m3 = simplemerge.Merge3Text(base, local, other)
                         ml = [l.strip() for l in m3.merge_lines()]
                         ml.append(b"")
                     elif at > 0:
                         ml = p1[fn].data().split(b"\n")
                     else:
                         ml = initialmergedlines
                     ml[id * linesperrev] += b" r%i" % id
                     mergedtext = b"\n".join(ml)
                     files.append(fn)
                     filecontent[fn] = mergedtext
 
                 if overwritten_file:
                     fn = b"of"
                     files.append(fn)
                     filecontent[fn] = b"r%i\n" % id
 
                 if new_file:
                     fn = b"nf%i" % id
                     files.append(fn)
                     filecontent[fn] = b"r%i\n" % id
                     if len(ps) > 1:
                         if not p2:
                             p2 = repo[ps[1]]
                         for fn in p2:
                             if fn.startswith(b"nf"):
                                 files.append(fn)
                                 filecontent[fn] = p2[fn].data()
 
                 def fctxfn(repo, cx, path):
                     if path in filecontent:
                         return context.memfilectx(
                             repo, cx, path, filecontent[path]
                         )
                     return None
 
                 if len(ps) == 0 or ps[0] < 0:
                     pars = [None, None]
                 elif len(ps) == 1:
                     pars = [nodeids[ps[0]], None]
                 else:
                     pars = [nodeids[p] for p in ps]
                 cx = context.memctx(
                     repo,
                     pars,
                     b"r%i" % id,
                     files,
                     fctxfn,
                     date=(id, 0),
                     user=b"debugbuilddag",
                     extra={b'branch': atbranch},
                 )
                 nodeid = repo.commitctx(cx)
                 nodeids.append(nodeid)
                 at = id
             elif type == b'l':
                 id, name = data
                 ui.note((b'tag %s\n' % name))
                 tags.append(b"%s %s\n" % (hex(repo.changelog.node(id)), name))
             elif type == b'a':
                 ui.note((b'branch %s\n' % data))
                 atbranch = data
             progress.update(id)
 
         if tags:
             repo.vfs.write(b"localtags", b"".join(tags))
 
 
 def _debugchangegroup(ui, gen, all=None, indent=0, **opts):
     indent_string = b' ' * indent
     if all:
         ui.writenoi18n(
             b"%sformat: id, p1, p2, cset, delta base, len(delta)\n"
             % indent_string
         )
 
         def showchunks(named):
             ui.write(b"\n%s%s\n" % (indent_string, named))
             for deltadata in gen.deltaiter():
                 node, p1, p2, cs, deltabase, delta, flags = deltadata
                 ui.write(
                     b"%s%s %s %s %s %s %d\n"
                     % (
                         indent_string,
                         hex(node),
                         hex(p1),
                         hex(p2),
                         hex(cs),
                         hex(deltabase),
                         len(delta),
                     )
                 )
 
         gen.changelogheader()
         showchunks(b"changelog")
         gen.manifestheader()
         showchunks(b"manifest")
         for chunkdata in iter(gen.filelogheader, {}):
             fname = chunkdata[b'filename']
             showchunks(fname)
     else:
         if isinstance(gen, bundle2.unbundle20):
             raise error.Abort(_(b'use debugbundle2 for this file'))
         gen.changelogheader()
         for deltadata in gen.deltaiter():
             node, p1, p2, cs, deltabase, delta, flags = deltadata
             ui.write(b"%s%s\n" % (indent_string, hex(node)))
 
 
 def _debugobsmarkers(ui, part, indent=0, **opts):
     """display version and markers contained in 'data'"""
     opts = pycompat.byteskwargs(opts)
     data = part.read()
     indent_string = b' ' * indent
     try:
         version, markers = obsolete._readmarkers(data)
     except error.UnknownVersion as exc:
         msg = b"%sunsupported version: %s (%d bytes)\n"
         msg %= indent_string, exc.version, len(data)
         ui.write(msg)
     else:
         msg = b"%sversion: %d (%d bytes)\n"
         msg %= indent_string, version, len(data)
         ui.write(msg)
         fm = ui.formatter(b'debugobsolete', opts)
         for rawmarker in sorted(markers):
             m = obsutil.marker(None, rawmarker)
             fm.startitem()
             fm.plain(indent_string)
             cmdutil.showmarker(fm, m)
         fm.end()
 
 
 def _debugphaseheads(ui, data, indent=0):
     """display version and markers contained in 'data'"""
     indent_string = b' ' * indent
     headsbyphase = phases.binarydecode(data)
     for phase in phases.allphases:
         for head in headsbyphase[phase]:
             ui.write(indent_string)
             ui.write(b'%s %s\n' % (hex(head), phases.phasenames[phase]))
 
 
 def _quasirepr(thing):
     if isinstance(thing, (dict, util.sortdict, collections.OrderedDict)):
         return b'{%s}' % (
             b', '.join(b'%s: %s' % (k, thing[k]) for k in sorted(thing))
         )
     return pycompat.bytestr(repr(thing))
 
 
 def _debugbundle2(ui, gen, all=None, **opts):
     """lists the contents of a bundle2"""
     if not isinstance(gen, bundle2.unbundle20):
         raise error.Abort(_(b'not a bundle2 file'))
     ui.write((b'Stream params: %s\n' % _quasirepr(gen.params)))
     parttypes = opts.get('part_type', [])
     for part in gen.iterparts():
         if parttypes and part.type not in parttypes:
             continue
         msg = b'%s -- %s (mandatory: %r)\n'
         ui.write((msg % (part.type, _quasirepr(part.params), part.mandatory)))
         if part.type == b'changegroup':
             version = part.params.get(b'version', b'01')
             cg = changegroup.getunbundler(version, part, b'UN')
             if not ui.quiet:
                 _debugchangegroup(ui, cg, all=all, indent=4, **opts)
         if part.type == b'obsmarkers':
             if not ui.quiet:
                 _debugobsmarkers(ui, part, indent=4, **opts)
         if part.type == b'phase-heads':
             if not ui.quiet:
                 _debugphaseheads(ui, part, indent=4)
 
 
 @command(
     b'debugbundle',
     [
         (b'a', b'all', None, _(b'show all details')),
         (b'', b'part-type', [], _(b'show only the named part type')),
         (b'', b'spec', None, _(b'print the bundlespec of the bundle')),
     ],
     _(b'FILE'),
     norepo=True,
 )
 def debugbundle(ui, bundlepath, all=None, spec=None, **opts):
     """lists the contents of a bundle"""
     with hg.openpath(ui, bundlepath) as f:
         if spec:
             spec = exchange.getbundlespec(ui, f)
             ui.write(b'%s\n' % spec)
             return
 
         gen = exchange.readbundle(ui, f, bundlepath)
         if isinstance(gen, bundle2.unbundle20):
             return _debugbundle2(ui, gen, all=all, **opts)
         _debugchangegroup(ui, gen, all=all, **opts)
 
 
 @command(b'debugcapabilities', [], _(b'PATH'), norepo=True)
 def debugcapabilities(ui, path, **opts):
     """lists the capabilities of a remote peer"""
     opts = pycompat.byteskwargs(opts)
     peer = hg.peer(ui, opts, path)
     caps = peer.capabilities()
     ui.writenoi18n(b'Main capabilities:\n')
     for c in sorted(caps):
         ui.write(b'  %s\n' % c)
     b2caps = bundle2.bundle2caps(peer)
     if b2caps:
         ui.writenoi18n(b'Bundle2 capabilities:\n')
         for key, values in sorted(pycompat.iteritems(b2caps)):
             ui.write(b'  %s\n' % key)
             for v in values:
                 ui.write(b'    %s\n' % v)
 
 
 @command(
     b'debugchangedfiles',
     [
         (
             b'',
             b'compute',
             False,
             b"compute information instead of reading it from storage",
         ),
     ],
     b'REV',
 )
 def debugchangedfiles(ui, repo, rev, **opts):
     """list the stored files changes for a revision"""
     ctx = scmutil.revsingle(repo, rev, None)
     files = None
 
     if opts['compute']:
         files = metadata.compute_all_files_changes(ctx)
     else:
         sd = repo.changelog.sidedata(ctx.rev())
         files_block = sd.get(sidedata.SD_FILES)
         if files_block is not None:
             files = metadata.decode_files_sidedata(sd)
     if files is not None:
         for f in sorted(files.touched):
             if f in files.added:
                 action = b"added"
             elif f in files.removed:
                 action = b"removed"
             elif f in files.merged:
                 action = b"merged"
             elif f in files.salvaged:
                 action = b"salvaged"
             else:
                 action = b"touched"
 
             copy_parent = b""
             copy_source = b""
             if f in files.copied_from_p1:
                 copy_parent = b"p1"
                 copy_source = files.copied_from_p1[f]
             elif f in files.copied_from_p2:
                 copy_parent = b"p2"
                 copy_source = files.copied_from_p2[f]
 
             data = (action, copy_parent, f, copy_source)
             template = b"%-8s %2s: %s, %s;\n"
             ui.write(template % data)
 
 
 @command(b'debugcheckstate', [], b'')
 def debugcheckstate(ui, repo):
     """validate the correctness of the current dirstate"""
     parent1, parent2 = repo.dirstate.parents()
     m1 = repo[parent1].manifest()
     m2 = repo[parent2].manifest()
     errors = 0
     for f in repo.dirstate:
         state = repo.dirstate[f]
         if state in b"nr" and f not in m1:
             ui.warn(_(b"%s in state %s, but not in manifest1\n") % (f, state))
             errors += 1
         if state in b"a" and f in m1:
             ui.warn(_(b"%s in state %s, but also in manifest1\n") % (f, state))
             errors += 1
         if state in b"m" and f not in m1 and f not in m2:
             ui.warn(
                 _(b"%s in state %s, but not in either manifest\n") % (f, state)
             )
             errors += 1
     for f in m1:
         state = repo.dirstate[f]
         if state not in b"nrm":
             ui.warn(_(b"%s in manifest1, but listed as state %s") % (f, state))
             errors += 1
     if errors:
         errstr = _(b".hg/dirstate inconsistent with current parent's manifest")
         raise error.Abort(errstr)
 
 
 @command(
     b'debugcolor',
     [(b'', b'style', None, _(b'show all configured styles'))],
     b'hg debugcolor',
 )
 def debugcolor(ui, repo, **opts):
     """show available color, effects or style"""
     ui.writenoi18n(b'color mode: %s\n' % stringutil.pprint(ui._colormode))
     if opts.get('style'):
         return _debugdisplaystyle(ui)
     else:
         return _debugdisplaycolor(ui)
 
 
 def _debugdisplaycolor(ui):
     ui = ui.copy()
     ui._styles.clear()
     for effect in color._activeeffects(ui).keys():
         ui._styles[effect] = effect
     if ui._terminfoparams:
         for k, v in ui.configitems(b'color'):
             if k.startswith(b'color.'):
                 ui._styles[k] = k[6:]
             elif k.startswith(b'terminfo.'):
                 ui._styles[k] = k[9:]
     ui.write(_(b'available colors:\n'))
     # sort label with a '_' after the other to group '_background' entry.
     items = sorted(ui._styles.items(), key=lambda i: (b'_' in i[0], i[0], i[1]))
     for colorname, label in items:
         ui.write(b'%s\n' % colorname, label=label)
 
 
 def _debugdisplaystyle(ui):
     ui.write(_(b'available style:\n'))
     if not ui._styles:
         return
     width = max(len(s) for s in ui._styles)
     for label, effects in sorted(ui._styles.items()):
         ui.write(b'%s' % label, label=label)
         if effects:
             # 50
             ui.write(b': ')
             ui.write(b' ' * (max(0, width - len(label))))
             ui.write(b', '.join(ui.label(e, e) for e in effects.split()))
         ui.write(b'\n')
 
 
 @command(b'debugcreatestreamclonebundle', [], b'FILE')
 def debugcreatestreamclonebundle(ui, repo, fname):
     """create a stream clone bundle file
 
     Stream bundles are special bundles that are essentially archives of
     revlog files. They are commonly used for cloning very quickly.
     """
     # TODO we may want to turn this into an abort when this functionality
     # is moved into `hg bundle`.
     if phases.hassecret(repo):
         ui.warn(
             _(
                 b'(warning: stream clone bundle will contain secret '
                 b'revisions)\n'
             )
         )
 
     requirements, gen = streamclone.generatebundlev1(repo)
     changegroup.writechunks(ui, gen, fname)
 
     ui.write(_(b'bundle requirements: %s\n') % b', '.join(sorted(requirements)))
 
 
 @command(
     b'debugdag',
     [
         (b't', b'tags', None, _(b'use tags as labels')),
         (b'b', b'branches', None, _(b'annotate with branch names')),
         (b'', b'dots', None, _(b'use dots for runs')),
         (b's', b'spaces', None, _(b'separate elements by spaces')),
     ],
     _(b'[OPTION]... [FILE [REV]...]'),
     optionalrepo=True,
 )
 def debugdag(ui, repo, file_=None, *revs, **opts):
     """format the changelog or an index DAG as a concise textual description
 
     If you pass a revlog index, the revlog's DAG is emitted. If you list
     revision numbers, they get labeled in the output as rN.
 
     Otherwise, the changelog DAG of the current repo is emitted.
     """
     spaces = opts.get('spaces')
     dots = opts.get('dots')
     if file_:
         rlog = revlog.revlog(vfsmod.vfs(encoding.getcwd(), audit=False), file_)
         revs = {int(r) for r in revs}
 
         def events():
             for r in rlog:
                 yield b'n', (r, list(p for p in rlog.parentrevs(r) if p != -1))
                 if r in revs:
                     yield b'l', (r, b"r%i" % r)
 
     elif repo:
         cl = repo.changelog
         tags = opts.get('tags')
         branches = opts.get('branches')
         if tags:
             labels = {}
             for l, n in repo.tags().items():
                 labels.setdefault(cl.rev(n), []).append(l)
 
         def events():
             b = b"default"
             for r in cl:
                 if branches:
                     newb = cl.read(cl.node(r))[5][b'branch']
                     if newb != b:
                         yield b'a', newb
                         b = newb
                 yield b'n', (r, list(p for p in cl.parentrevs(r) if p != -1))
                 if tags:
                     ls = labels.get(r)
                     if ls:
                         for l in ls:
                             yield b'l', (r, l)
 
     else:
         raise error.Abort(_(b'need repo for changelog dag'))
 
     for line in dagparser.dagtextlines(
         events(),
         addspaces=spaces,
         wraplabels=True,
         wrapannotations=True,
         wrapnonlinear=dots,
         usedots=dots,
         maxlinewidth=70,
     ):
         ui.write(line)
         ui.write(b"\n")
 
 
 @command(b'debugdata', cmdutil.debugrevlogopts, _(b'-c|-m|FILE REV'))
 def debugdata(ui, repo, file_, rev=None, **opts):
     """dump the contents of a data file revision"""
     opts = pycompat.byteskwargs(opts)
     if opts.get(b'changelog') or opts.get(b'manifest') or opts.get(b'dir'):
         if rev is not None:
             raise error.CommandError(b'debugdata', _(b'invalid arguments'))
         file_, rev = None, file_
     elif rev is None:
         raise error.CommandError(b'debugdata', _(b'invalid arguments'))
     r = cmdutil.openstorage(repo, b'debugdata', file_, opts)
     try:
         ui.write(r.rawdata(r.lookup(rev)))
     except KeyError:
         raise error.Abort(_(b'invalid revision identifier %s') % rev)
 
 
 @command(
     b'debugdate',
     [(b'e', b'extended', None, _(b'try extended date formats'))],
     _(b'[-e] DATE [RANGE]'),
     norepo=True,
     optionalrepo=True,
 )
 def debugdate(ui, date, range=None, **opts):
     """parse and display a date"""
     if opts["extended"]:
         d = dateutil.parsedate(date, dateutil.extendeddateformats)
     else:
         d = dateutil.parsedate(date)
     ui.writenoi18n(b"internal: %d %d\n" % d)
     ui.writenoi18n(b"standard: %s\n" % dateutil.datestr(d))
     if range:
         m = dateutil.matchdate(range)
         ui.writenoi18n(b"match: %s\n" % m(d[0]))
 
 
 @command(
     b'debugdeltachain',
     cmdutil.debugrevlogopts + cmdutil.formatteropts,
     _(b'-c|-m|FILE'),
     optionalrepo=True,
 )
 def debugdeltachain(ui, repo, file_=None, **opts):
     """dump information about delta chains in a revlog
 
     Output can be templatized. Available template keywords are:
 
     :``rev``:       revision number
     :``chainid``:   delta chain identifier (numbered by unique base)
     :``chainlen``:  delta chain length to this revision
     :``prevrev``:   previous revision in delta chain
     :``deltatype``: role of delta / how it was computed
     :``compsize``:  compressed size of revision
     :``uncompsize``: uncompressed size of revision
     :``chainsize``: total size of compressed revisions in chain
     :``chainratio``: total chain size divided by uncompressed revision size
                     (new delta chains typically start at ratio 2.00)
     :``lindist``:   linear distance from base revision in delta chain to end
                     of this revision
     :``extradist``: total size of revisions not part of this delta chain from
                     base of delta chain to end of this revision; a measurement
                     of how much extra data we need to read/seek across to read
                     the delta chain for this revision
     :``extraratio``: extradist divided by chainsize; another representation of
                     how much unrelated data is needed to load this delta chain
 
     If the repository is configured to use the sparse read, additional keywords
     are available:
 
     :``readsize``:     total size of data read from the disk for a revision
                        (sum of the sizes of all the blocks)
     :``largestblock``: size of the largest block of data read from the disk
     :``readdensity``:  density of useful bytes in the data read from the disk
     :``srchunks``:  in how many data hunks the whole revision would be read
 
     The sparse read can be enabled with experimental.sparse-read = True
     """
     opts = pycompat.byteskwargs(opts)
     r = cmdutil.openrevlog(repo, b'debugdeltachain', file_, opts)
     index = r.index
     start = r.start
     length = r.length
     generaldelta = r.version & revlog.FLAG_GENERALDELTA
     withsparseread = getattr(r, '_withsparseread', False)
 
     def revinfo(rev):
         e = index[rev]
         compsize = e[1]
         uncompsize = e[2]
         chainsize = 0
 
         if generaldelta:
             if e[3] == e[5]:
                 deltatype = b'p1'
             elif e[3] == e[6]:
                 deltatype = b'p2'
             elif e[3] == rev - 1:
                 deltatype = b'prev'
             elif e[3] == rev:
                 deltatype = b'base'
             else:
                 deltatype = b'other'
         else:
             if e[3] == rev:
                 deltatype = b'base'
             else:
                 deltatype = b'prev'
 
         chain = r._deltachain(rev)[0]
         for iterrev in chain:
             e = index[iterrev]
             chainsize += e[1]
 
         return compsize, uncompsize, deltatype, chain, chainsize
 
     fm = ui.formatter(b'debugdeltachain', opts)
 
     fm.plain(
         b'    rev  chain# chainlen     prev   delta       '
         b'size    rawsize  chainsize     ratio   lindist extradist '
         b'extraratio'
     )
     if withsparseread:
         fm.plain(b'   readsize largestblk rddensity srchunks')
     fm.plain(b'\n')
 
     chainbases = {}
     for rev in r:
         comp, uncomp, deltatype, chain, chainsize = revinfo(rev)
         chainbase = chain[0]
         chainid = chainbases.setdefault(chainbase, len(chainbases) + 1)
         basestart = start(chainbase)
         revstart = start(rev)
         lineardist = revstart + comp - basestart
         extradist = lineardist - chainsize
         try:
             prevrev = chain[-2]
         except IndexError:
             prevrev = -1
 
         if uncomp != 0:
             chainratio = float(chainsize) / float(uncomp)
         else:
             chainratio = chainsize
 
         if chainsize != 0:
             extraratio = float(extradist) / float(chainsize)
         else:
             extraratio = extradist
 
         fm.startitem()
         fm.write(
             b'rev chainid chainlen prevrev deltatype compsize '
             b'uncompsize chainsize chainratio lindist extradist '
             b'extraratio',
             b'%7d %7d %8d %8d %7s %10d %10d %10d %9.5f %9d %9d %10.5f',
             rev,
             chainid,
             len(chain),
             prevrev,
             deltatype,
             comp,
             uncomp,
             chainsize,
             chainratio,
             lineardist,
             extradist,
             extraratio,
             rev=rev,
             chainid=chainid,
             chainlen=len(chain),
             prevrev=prevrev,
             deltatype=deltatype,
             compsize=comp,
             uncompsize=uncomp,
             chainsize=chainsize,
             chainratio=chainratio,
             lindist=lineardist,
             extradist=extradist,
             extraratio=extraratio,
         )
         if withsparseread:
             readsize = 0
             largestblock = 0
             srchunks = 0
 
             for revschunk in deltautil.slicechunk(r, chain):
                 srchunks += 1
                 blkend = start(revschunk[-1]) + length(revschunk[-1])
                 blksize = blkend - start(revschunk[0])
 
                 readsize += blksize
                 if largestblock < blksize:
                     largestblock = blksize
 
             if readsize:
                 readdensity = float(chainsize) / float(readsize)
             else:
                 readdensity = 1
 
             fm.write(
                 b'readsize largestblock readdensity srchunks',
                 b' %10d %10d %9.5f %8d',
                 readsize,
                 largestblock,
                 readdensity,
                 srchunks,
                 readsize=readsize,
                 largestblock=largestblock,
                 readdensity=readdensity,
                 srchunks=srchunks,
             )
 
         fm.plain(b'\n')
 
     fm.end()
 
 
 @command(
     b'debugdirstate|debugstate',
     [
         (
             b'',
             b'nodates',
             None,
             _(b'do not display the saved mtime (DEPRECATED)'),
         ),
         (b'', b'dates', True, _(b'display the saved mtime')),
         (b'', b'datesort', None, _(b'sort by saved mtime')),
     ],
     _(b'[OPTION]...'),
 )
 def debugstate(ui, repo, **opts):
     """show the contents of the current dirstate"""
 
     nodates = not opts['dates']
     if opts.get('nodates') is not None:
         nodates = True
     datesort = opts.get('datesort')
 
     if datesort:
         keyfunc = lambda x: (x[1][3], x[0])  # sort by mtime, then by filename
     else:
         keyfunc = None  # sort by filename
     for file_, ent in sorted(pycompat.iteritems(repo.dirstate), key=keyfunc):
         if ent[3] == -1:
             timestr = b'unset               '
         elif nodates:
             timestr = b'set                 '
         else:
             timestr = time.strftime(
                 "%Y-%m-%d %H:%M:%S ", time.localtime(ent[3])
             )
             timestr = encoding.strtolocal(timestr)
         if ent[1] & 0o20000:
             mode = b'lnk'
         else:
             mode = b'%3o' % (ent[1] & 0o777 & ~util.umask)
         ui.write(b"%c %s %10d %s%s\n" % (ent[0], mode, ent[2], timestr, file_))
     for f in repo.dirstate.copies():
         ui.write(_(b"copy: %s -> %s\n") % (repo.dirstate.copied(f), f))
 
 
 @command(
     b'debugdiscovery',
     [
         (b'', b'old', None, _(b'use old-style discovery')),
         (
             b'',
             b'nonheads',
             None,
             _(b'use old-style discovery with non-heads included'),
         ),
         (b'', b'rev', [], b'restrict discovery to this set of revs'),
         (b'', b'seed', b'12323', b'specify the random seed use for discovery'),
         (
             b'',
             b'local-as-revs',
             "",
             'treat local has having these revisions only',
         ),
         (
             b'',
             b'remote-as-revs',
             "",
             'use local as remote, with only these these revisions',
         ),
     ]
     + cmdutil.remoteopts,
     _(b'[--rev REV] [OTHER]'),
 )
 def debugdiscovery(ui, repo, remoteurl=b"default", **opts):
     """runs the changeset discovery protocol in isolation
 
     The local peer can be "replaced" by a subset of the local repository by
     using the `--local-as-revs` flag. Int he same way, usual `remote` peer can
     be "replaced" by a subset of the local repository using the
     `--local-as-revs` flag. This is useful to efficiently debug pathological
     discovery situation.
     """
     opts = pycompat.byteskwargs(opts)
     unfi = repo.unfiltered()
 
     # setup potential extra filtering
     local_revs = opts[b"local_as_revs"]
     remote_revs = opts[b"remote_as_revs"]
 
     # make sure tests are repeatable
     random.seed(int(opts[b'seed']))
 
     if not remote_revs:
 
         remoteurl, branches = hg.parseurl(ui.expandpath(remoteurl))
         remote = hg.peer(repo, opts, remoteurl)
         ui.status(_(b'comparing with %s\n') % util.hidepassword(remoteurl))
     else:
         branches = (None, [])
         remote_filtered_revs = scmutil.revrange(
             unfi, [b"not (::(%s))" % remote_revs]
         )
         remote_filtered_revs = frozenset(remote_filtered_revs)
 
         def remote_func(x):
             return remote_filtered_revs
 
         repoview.filtertable[b'debug-discovery-remote-filter'] = remote_func
 
         remote = repo.peer()
         remote._repo = remote._repo.filtered(b'debug-discovery-remote-filter')
 
     if local_revs:
         local_filtered_revs = scmutil.revrange(
             unfi, [b"not (::(%s))" % local_revs]
         )
         local_filtered_revs = frozenset(local_filtered_revs)
 
         def local_func(x):
             return local_filtered_revs
 
         repoview.filtertable[b'debug-discovery-local-filter'] = local_func
         repo = repo.filtered(b'debug-discovery-local-filter')
 
     data = {}
     if opts.get(b'old'):
 
         def doit(pushedrevs, remoteheads, remote=remote):
             if not util.safehasattr(remote, b'branches'):
                 # enable in-client legacy support
                 remote = localrepo.locallegacypeer(remote.local())
             common, _in, hds = treediscovery.findcommonincoming(
                 repo, remote, force=True, audit=data
             )
             common = set(common)
             if not opts.get(b'nonheads'):
                 ui.writenoi18n(
                     b"unpruned common: %s\n"
                     % b" ".join(sorted(short(n) for n in common))
                 )
 
                 clnode = repo.changelog.node
                 common = repo.revs(b'heads(::%ln)', common)
                 common = {clnode(r) for r in common}
             return common, hds
 
     else:
 
         def doit(pushedrevs, remoteheads, remote=remote):
             nodes = None
             if pushedrevs:
                 revs = scmutil.revrange(repo, pushedrevs)
                 nodes = [repo[r].node() for r in revs]
             common, any, hds = setdiscovery.findcommonheads(
                 ui, repo, remote, ancestorsof=nodes, audit=data
             )
             return common, hds
 
     remoterevs, _checkout = hg.addbranchrevs(repo, remote, branches, revs=None)
     localrevs = opts[b'rev']
     with util.timedcm('debug-discovery') as t:
         common, hds = doit(localrevs, remoterevs)
 
     # compute all statistics
     heads_common = set(common)
     heads_remote = set(hds)
     heads_local = set(repo.heads())
     # note: they cannot be a local or remote head that is in common and not
     # itself a head of common.
     heads_common_local = heads_common & heads_local
     heads_common_remote = heads_common & heads_remote
     heads_common_both = heads_common & heads_remote & heads_local
 
     all = repo.revs(b'all()')
     common = repo.revs(b'::%ln', common)
     roots_common = repo.revs(b'roots(::%ld)', common)
     missing = repo.revs(b'not ::%ld', common)
     heads_missing = repo.revs(b'heads(%ld)', missing)
     roots_missing = repo.revs(b'roots(%ld)', missing)
     assert len(common) + len(missing) == len(all)
 
     initial_undecided = repo.revs(
         b'not (::%ln or %ln::)', heads_common_remote, heads_common_local
     )
     heads_initial_undecided = repo.revs(b'heads(%ld)', initial_undecided)
     roots_initial_undecided = repo.revs(b'roots(%ld)', initial_undecided)
     common_initial_undecided = initial_undecided & common
     missing_initial_undecided = initial_undecided & missing
 
     data[b'elapsed'] = t.elapsed
     data[b'nb-common-heads'] = len(heads_common)
     data[b'nb-common-heads-local'] = len(heads_common_local)
     data[b'nb-common-heads-remote'] = len(heads_common_remote)
     data[b'nb-common-heads-both'] = len(heads_common_both)
     data[b'nb-common-roots'] = len(roots_common)
     data[b'nb-head-local'] = len(heads_local)
     data[b'nb-head-local-missing'] = len(heads_local) - len(heads_common_local)
     data[b'nb-head-remote'] = len(heads_remote)
     data[b'nb-head-remote-unknown'] = len(heads_remote) - len(
         heads_common_remote
     )
     data[b'nb-revs'] = len(all)
     data[b'nb-revs-common'] = len(common)
     data[b'nb-revs-missing'] = len(missing)
     data[b'nb-missing-heads'] = len(heads_missing)
     data[b'nb-missing-roots'] = len(roots_missing)
     data[b'nb-ini_und'] = len(initial_undecided)
     data[b'nb-ini_und-heads'] = len(heads_initial_undecided)
     data[b'nb-ini_und-roots'] = len(roots_initial_undecided)
     data[b'nb-ini_und-common'] = len(common_initial_undecided)
     data[b'nb-ini_und-missing'] = len(missing_initial_undecided)
 
     # display discovery summary
     ui.writenoi18n(b"elapsed time:  %(elapsed)f seconds\n" % data)
     ui.writenoi18n(b"round-trips:           %(total-roundtrips)9d\n" % data)
     ui.writenoi18n(b"heads summary:\n")
     ui.writenoi18n(b"  total common heads:  %(nb-common-heads)9d\n" % data)
     ui.writenoi18n(
         b"    also local heads:  %(nb-common-heads-local)9d\n" % data
     )
     ui.writenoi18n(
         b"    also remote heads: %(nb-common-heads-remote)9d\n" % data
     )
     ui.writenoi18n(b"    both:              %(nb-common-heads-both)9d\n" % data)
     ui.writenoi18n(b"  local heads:         %(nb-head-local)9d\n" % data)
     ui.writenoi18n(
         b"    common:            %(nb-common-heads-local)9d\n" % data
     )
     ui.writenoi18n(
         b"    missing:           %(nb-head-local-missing)9d\n" % data
     )
     ui.writenoi18n(b"  remote heads:        %(nb-head-remote)9d\n" % data)
     ui.writenoi18n(
         b"    common:            %(nb-common-heads-remote)9d\n" % data
     )
     ui.writenoi18n(
         b"    unknown:           %(nb-head-remote-unknown)9d\n" % data
     )
     ui.writenoi18n(b"local changesets:      %(nb-revs)9d\n" % data)
     ui.writenoi18n(b"  common:              %(nb-revs-common)9d\n" % data)
     ui.writenoi18n(b"    heads:             %(nb-common-heads)9d\n" % data)
     ui.writenoi18n(b"    roots:             %(nb-common-roots)9d\n" % data)
     ui.writenoi18n(b"  missing:             %(nb-revs-missing)9d\n" % data)
     ui.writenoi18n(b"    heads:             %(nb-missing-heads)9d\n" % data)
     ui.writenoi18n(b"    roots:             %(nb-missing-roots)9d\n" % data)
     ui.writenoi18n(b"  first undecided set: %(nb-ini_und)9d\n" % data)
     ui.writenoi18n(b"    heads:             %(nb-ini_und-heads)9d\n" % data)
     ui.writenoi18n(b"    roots:             %(nb-ini_und-roots)9d\n" % data)
     ui.writenoi18n(b"    common:            %(nb-ini_und-common)9d\n" % data)
     ui.writenoi18n(b"    missing:           %(nb-ini_und-missing)9d\n" % data)
 
     if ui.verbose:
         ui.writenoi18n(
             b"common heads: %s\n"
             % b" ".join(sorted(short(n) for n in heads_common))
         )
 
 
 _chunksize = 4 << 10
 
 
 @command(
     b'debugdownload',
     [
         (b'o', b'output', b'', _(b'path')),
     ],
     optionalrepo=True,
 )
 def debugdownload(ui, repo, url, output=None, **opts):
     """download a resource using Mercurial logic and config"""
     fh = urlmod.open(ui, url, output)
 
     dest = ui
     if output:
         dest = open(output, b"wb", _chunksize)
     try:
         data = fh.read(_chunksize)
         while data:
             dest.write(data)
             data = fh.read(_chunksize)
     finally:
         if output:
             dest.close()
 
 
 @command(b'debugextensions', cmdutil.formatteropts, [], optionalrepo=True)
 def debugextensions(ui, repo, **opts):
     '''show information about active extensions'''
     opts = pycompat.byteskwargs(opts)
     exts = extensions.extensions(ui)
     hgver = util.version()
     fm = ui.formatter(b'debugextensions', opts)
     for extname, extmod in sorted(exts, key=operator.itemgetter(0)):
         isinternal = extensions.ismoduleinternal(extmod)
         extsource = None
 
         if util.safehasattr(extmod, '__file__'):
             extsource = pycompat.fsencode(extmod.__file__)
         elif getattr(sys, 'oxidized', False):
             extsource = pycompat.sysexecutable
         if isinternal:
             exttestedwith = []  # never expose magic string to users
         else:
             exttestedwith = getattr(extmod, 'testedwith', b'').split()
         extbuglink = getattr(extmod, 'buglink', None)
 
         fm.startitem()
 
         if ui.quiet or ui.verbose:
             fm.write(b'name', b'%s\n', extname)
         else:
             fm.write(b'name', b'%s', extname)
             if isinternal or hgver in exttestedwith:
                 fm.plain(b'\n')
             elif not exttestedwith:
                 fm.plain(_(b' (untested!)\n'))
             else:
                 lasttestedversion = exttestedwith[-1]
                 fm.plain(b' (%s!)\n' % lasttestedversion)
 
         fm.condwrite(
             ui.verbose and extsource,
             b'source',
             _(b'  location: %s\n'),
             extsource or b"",
         )
 
         if ui.verbose:
             fm.plain(_(b'  bundled: %s\n') % [b'no', b'yes'][isinternal])
         fm.data(bundled=isinternal)
 
         fm.condwrite(
             ui.verbose and exttestedwith,
             b'testedwith',
             _(b'  tested with: %s\n'),
             fm.formatlist(exttestedwith, name=b'ver'),
         )
 
         fm.condwrite(
             ui.verbose and extbuglink,
             b'buglink',
             _(b'  bug reporting: %s\n'),
             extbuglink or b"",
         )
 
     fm.end()
 
 
 @command(
     b'debugfileset',
     [
         (
             b'r',
             b'rev',
             b'',
             _(b'apply the filespec on this revision'),
             _(b'REV'),
         ),
         (
             b'',
             b'all-files',
             False,
             _(b'test files from all revisions and working directory'),
         ),
         (
             b's',
             b'show-matcher',
             None,
             _(b'print internal representation of matcher'),
         ),
         (
             b'p',
             b'show-stage',
             [],
             _(b'print parsed tree at the given stage'),
             _(b'NAME'),
         ),
     ],
     _(b'[-r REV] [--all-files] [OPTION]... FILESPEC'),
 )
 def debugfileset(ui, repo, expr, **opts):
     '''parse and apply a fileset specification'''
     from . import fileset
 
     fileset.symbols  # force import of fileset so we have predicates to optimize
     opts = pycompat.byteskwargs(opts)
     ctx = scmutil.revsingle(repo, opts.get(b'rev'), None)
 
     stages = [
         (b'parsed', pycompat.identity),
         (b'analyzed', filesetlang.analyze),
         (b'optimized', filesetlang.optimize),
     ]
     stagenames = {n for n, f in stages}
 
     showalways = set()
     if ui.verbose and not opts[b'show_stage']:
         # show parsed tree by --verbose (deprecated)
         showalways.add(b'parsed')
     if opts[b'show_stage'] == [b'all']:
         showalways.update(stagenames)
     else:
         for n in opts[b'show_stage']:
             if n not in stagenames:
                 raise error.Abort(_(b'invalid stage name: %s') % n)
         showalways.update(opts[b'show_stage'])
 
     tree = filesetlang.parse(expr)
     for n, f in stages:
         tree = f(tree)
         if n in showalways:
             if opts[b'show_stage'] or n != b'parsed':
                 ui.write(b"* %s:\n" % n)
             ui.write(filesetlang.prettyformat(tree), b"\n")
 
     files = set()
     if opts[b'all_files']:
         for r in repo:
             c = repo[r]
             files.update(c.files())
             files.update(c.substate)
     if opts[b'all_files'] or ctx.rev() is None:
         wctx = repo[None]
         files.update(
             repo.dirstate.walk(
                 scmutil.matchall(repo),
                 subrepos=list(wctx.substate),
                 unknown=True,
                 ignored=True,
             )
         )
         files.update(wctx.substate)
     else:
         files.update(ctx.files())
         files.update(ctx.substate)
 
     m = ctx.matchfileset(repo.getcwd(), expr)
     if opts[b'show_matcher'] or (opts[b'show_matcher'] is None and ui.verbose):
         ui.writenoi18n(b'* matcher:\n', stringutil.prettyrepr(m), b'\n')
     for f in sorted(files):
         if not m(f):
             continue
         ui.write(b"%s\n" % f)
 
 
 @command(b'debugformat', [] + cmdutil.formatteropts)
 def debugformat(ui, repo, **opts):
     """display format information about the current repository
 
     Use --verbose to get extra information about current config value and
     Mercurial default."""
     opts = pycompat.byteskwargs(opts)
     maxvariantlength = max(len(fv.name) for fv in upgrade.allformatvariant)
     maxvariantlength = max(len(b'format-variant'), maxvariantlength)
 
     def makeformatname(name):
         return b'%s:' + (b' ' * (maxvariantlength - len(name)))
 
     fm = ui.formatter(b'debugformat', opts)
     if fm.isplain():
 
         def formatvalue(value):
             if util.safehasattr(value, b'startswith'):
                 return value
             if value:
                 return b'yes'
             else:
                 return b'no'
 
     else:
         formatvalue = pycompat.identity
 
     fm.plain(b'format-variant')
     fm.plain(b' ' * (maxvariantlength - len(b'format-variant')))
     fm.plain(b' repo')
     if ui.verbose:
         fm.plain(b' config default')
     fm.plain(b'\n')
     for fv in upgrade.allformatvariant:
         fm.startitem()
         repovalue = fv.fromrepo(repo)
         configvalue = fv.fromconfig(repo)
 
         if repovalue != configvalue:
             namelabel = b'formatvariant.name.mismatchconfig'
             repolabel = b'formatvariant.repo.mismatchconfig'
         elif repovalue != fv.default:
             namelabel = b'formatvariant.name.mismatchdefault'
             repolabel = b'formatvariant.repo.mismatchdefault'
         else:
             namelabel = b'formatvariant.name.uptodate'
             repolabel = b'formatvariant.repo.uptodate'
 
         fm.write(b'name', makeformatname(fv.name), fv.name, label=namelabel)
         fm.write(b'repo', b' %3s', formatvalue(repovalue), label=repolabel)
         if fv.default != configvalue:
             configlabel = b'formatvariant.config.special'
         else:
             configlabel = b'formatvariant.config.default'
         fm.condwrite(
             ui.verbose,
             b'config',
             b' %6s',
             formatvalue(configvalue),
             label=configlabel,
         )
         fm.condwrite(
             ui.verbose,
             b'default',
             b' %7s',
             formatvalue(fv.default),
             label=b'formatvariant.default',
         )
         fm.plain(b'\n')
     fm.end()
 
 
 @command(b'debugfsinfo', [], _(b'[PATH]'), norepo=True)
 def debugfsinfo(ui, path=b"."):
     """show information detected about current filesystem"""
     ui.writenoi18n(b'path: %s\n' % path)
     ui.writenoi18n(
         b'mounted on: %s\n' % (util.getfsmountpoint(path) or b'(unknown)')
     )
     ui.writenoi18n(b'exec: %s\n' % (util.checkexec(path) and b'yes' or b'no'))
     ui.writenoi18n(b'fstype: %s\n' % (util.getfstype(path) or b'(unknown)'))
     ui.writenoi18n(
         b'symlink: %s\n' % (util.checklink(path) and b'yes' or b'no')
     )
     ui.writenoi18n(
         b'hardlink: %s\n' % (util.checknlink(path) and b'yes' or b'no')
     )
     casesensitive = b'(unknown)'
     try:
         with pycompat.namedtempfile(prefix=b'.debugfsinfo', dir=path) as f:
             casesensitive = util.fscasesensitive(f.name) and b'yes' or b'no'
     except OSError:
         pass
     ui.writenoi18n(b'case-sensitive: %s\n' % casesensitive)
 
 
 @command(
     b'debuggetbundle',
     [
         (b'H', b'head', [], _(b'id of head node'), _(b'ID')),
         (b'C', b'common', [], _(b'id of common node'), _(b'ID')),
         (
             b't',
             b'type',
             b'bzip2',
             _(b'bundle compression type to use'),
             _(b'TYPE'),
         ),
     ],
     _(b'REPO FILE [-H|-C ID]...'),
     norepo=True,
 )
 def debuggetbundle(ui, repopath, bundlepath, head=None, common=None, **opts):
     """retrieves a bundle from a repo
 
     Every ID must be a full-length hex node id string. Saves the bundle to the
     given file.
     """
     opts = pycompat.byteskwargs(opts)
     repo = hg.peer(ui, opts, repopath)
     if not repo.capable(b'getbundle'):
         raise error.Abort(b"getbundle() not supported by target repository")
     args = {}
     if common:
         args['common'] = [bin(s) for s in common]
     if head:
         args['heads'] = [bin(s) for s in head]
     # TODO: get desired bundlecaps from command line.
     args['bundlecaps'] = None
     bundle = repo.getbundle(b'debug', **args)
 
     bundletype = opts.get(b'type', b'bzip2').lower()
     btypes = {
         b'none': b'HG10UN',
         b'bzip2': b'HG10BZ',
         b'gzip': b'HG10GZ',
         b'bundle2': b'HG20',
     }
     bundletype = btypes.get(bundletype)
     if bundletype not in bundle2.bundletypes:
         raise error.Abort(_(b'unknown bundle type specified with --type'))
     bundle2.writebundle(ui, bundle, bundlepath, bundletype)
 
 
 @command(b'debugignore', [], b'[FILE]')
 def debugignore(ui, repo, *files, **opts):
     """display the combined ignore pattern and information about ignored files
 
     With no argument display the combined ignore pattern.
 
     Given space separated file names, shows if the given file is ignored and
     if so, show the ignore rule (file and line number) that matched it.
     """
     ignore = repo.dirstate._ignore
     if not files:
         # Show all the patterns
         ui.write(b"%s\n" % pycompat.byterepr(ignore))
     else:
         m = scmutil.match(repo[None], pats=files)
         uipathfn = scmutil.getuipathfn(repo, legacyrelativevalue=True)
         for f in m.files():
             nf = util.normpath(f)
             ignored = None
             ignoredata = None
             if nf != b'.':
                 if ignore(nf):
                     ignored = nf
                     ignoredata = repo.dirstate._ignorefileandline(nf)
                 else:
                     for p in pathutil.finddirs(nf):
                         if ignore(p):
                             ignored = p
                             ignoredata = repo.dirstate._ignorefileandline(p)
                             break
             if ignored:
                 if ignored == nf:
                     ui.write(_(b"%s is ignored\n") % uipathfn(f))
                 else:
                     ui.write(
                         _(
                             b"%s is ignored because of "
                             b"containing directory %s\n"
                         )
                         % (uipathfn(f), ignored)
                     )
                 ignorefile, lineno, line = ignoredata
                 ui.write(
                     _(b"(ignore rule in %s, line %d: '%s')\n")
                     % (ignorefile, lineno, line)
                 )
             else:
                 ui.write(_(b"%s is not ignored\n") % uipathfn(f))
 
 
 @command(
     b'debugindex',
     cmdutil.debugrevlogopts + cmdutil.formatteropts,
     _(b'-c|-m|FILE'),
 )
 def debugindex(ui, repo, file_=None, **opts):
     """dump index data for a storage primitive"""
     opts = pycompat.byteskwargs(opts)
     store = cmdutil.openstorage(repo, b'debugindex', file_, opts)
 
     if ui.debugflag:
         shortfn = hex
     else:
         shortfn = short
 
     idlen = 12
     for i in store:
         idlen = len(shortfn(store.node(i)))
         break
 
     fm = ui.formatter(b'debugindex', opts)
     fm.plain(
         b'   rev linkrev %s %s p2\n'
         % (b'nodeid'.ljust(idlen), b'p1'.ljust(idlen))
     )
 
     for rev in store:
         node = store.node(rev)
         parents = store.parents(node)
 
         fm.startitem()
         fm.write(b'rev', b'%6d ', rev)
         fm.write(b'linkrev', b'%7d ', store.linkrev(rev))
         fm.write(b'node', b'%s ', shortfn(node))
         fm.write(b'p1', b'%s ', shortfn(parents[0]))
         fm.write(b'p2', b'%s', shortfn(parents[1]))
         fm.plain(b'\n')
 
     fm.end()
 
 
 @command(
     b'debugindexdot',
     cmdutil.debugrevlogopts,
     _(b'-c|-m|FILE'),
     optionalrepo=True,
 )
 def debugindexdot(ui, repo, file_=None, **opts):
     """dump an index DAG as a graphviz dot file"""
     opts = pycompat.byteskwargs(opts)
     r = cmdutil.openstorage(repo, b'debugindexdot', file_, opts)
     ui.writenoi18n(b"digraph G {\n")
     for i in r:
         node = r.node(i)
         pp = r.parents(node)
         ui.write(b"\t%d -> %d\n" % (r.rev(pp[0]), i))
         if pp[1] != nullid:
             ui.write(b"\t%d -> %d\n" % (r.rev(pp[1]), i))
     ui.write(b"}\n")
 
 
 @command(b'debugindexstats', [])
 def debugindexstats(ui, repo):
     """show stats related to the changelog index"""
     repo.changelog.shortest(nullid, 1)
     index = repo.changelog.index
     if not util.safehasattr(index, b'stats'):
         raise error.Abort(_(b'debugindexstats only works with native code'))
     for k, v in sorted(index.stats().items()):
         ui.write(b'%s: %d\n' % (k, v))
 
 
 @command(b'debuginstall', [] + cmdutil.formatteropts, b'', norepo=True)
 def debuginstall(ui, **opts):
     """test Mercurial installation
 
     Returns 0 on success.
     """
     opts = pycompat.byteskwargs(opts)
 
     problems = 0
 
     fm = ui.formatter(b'debuginstall', opts)
     fm.startitem()
 
     # encoding might be unknown or wrong. don't translate these messages.
     fm.write(b'encoding', b"checking encoding (%s)...\n", encoding.encoding)
     err = None
     try:
         codecs.lookup(pycompat.sysstr(encoding.encoding))
     except LookupError as inst:
         err = stringutil.forcebytestr(inst)
         problems += 1
     fm.condwrite(
         err,
         b'encodingerror',
         b" %s\n (check that your locale is properly set)\n",
         err,
     )
 
     # Python
     pythonlib = None
     if util.safehasattr(os, '__file__'):
         pythonlib = os.path.dirname(pycompat.fsencode(os.__file__))
     elif getattr(sys, 'oxidized', False):
         pythonlib = pycompat.sysexecutable
 
     fm.write(
         b'pythonexe',
         _(b"checking Python executable (%s)\n"),
         pycompat.sysexecutable or _(b"unknown"),
     )
     fm.write(
         b'pythonimplementation',
         _(b"checking Python implementation (%s)\n"),
         pycompat.sysbytes(platform.python_implementation()),
     )
     fm.write(
         b'pythonver',
         _(b"checking Python version (%s)\n"),
         (b"%d.%d.%d" % sys.version_info[:3]),
     )
     fm.write(
         b'pythonlib',
         _(b"checking Python lib (%s)...\n"),
         pythonlib or _(b"unknown"),
     )
 
     try:
         from . import rustext
 
         rustext.__doc__  # trigger lazy import
     except ImportError:
         rustext = None
 
     security = set(sslutil.supportedprotocols)
     if sslutil.hassni:
         security.add(b'sni')
 
     fm.write(
         b'pythonsecurity',
         _(b"checking Python security support (%s)\n"),
         fm.formatlist(sorted(security), name=b'protocol', fmt=b'%s', sep=b','),
     )
 
     # These are warnings, not errors. So don't increment problem count. This
     # may change in the future.
     if b'tls1.2' not in security:
         fm.plain(
             _(
                 b'  TLS 1.2 not supported by Python install; '
                 b'network connections lack modern security\n'
             )
         )
     if b'sni' not in security:
         fm.plain(
             _(
                 b'  SNI not supported by Python install; may have '
                 b'connectivity issues with some servers\n'
             )
         )
 
     fm.plain(
         _(
             b"checking Rust extensions (%s)\n"
             % (b'missing' if rustext is None else b'installed')
         ),
     )
 
     # TODO print CA cert info
 
     # hg version
     hgver = util.version()
     fm.write(
         b'hgver', _(b"checking Mercurial version (%s)\n"), hgver.split(b'+')[0]
     )
     fm.write(
         b'hgverextra',
         _(b"checking Mercurial custom build (%s)\n"),
         b'+'.join(hgver.split(b'+')[1:]),
     )
 
     # compiled modules
     hgmodules = None
     if util.safehasattr(sys.modules[__name__], '__file__'):
         hgmodules = os.path.dirname(pycompat.fsencode(__file__))
     elif getattr(sys, 'oxidized', False):
         hgmodules = pycompat.sysexecutable
 
     fm.write(
         b'hgmodulepolicy', _(b"checking module policy (%s)\n"), policy.policy
     )
     fm.write(
         b'hgmodules',
         _(b"checking installed modules (%s)...\n"),
         hgmodules or _(b"unknown"),
     )
 
     rustandc = policy.policy in (b'rust+c', b'rust+c-allow')
     rustext = rustandc  # for now, that's the only case
     cext = policy.policy in (b'c', b'allow') or rustandc
     nopure = cext or rustext
     if nopure:
         err = None
         try:
             if cext:
                 from .cext import (  # pytype: disable=import-error
                     base85,
                     bdiff,
                     mpatch,
                     osutil,
                 )
 
                 # quiet pyflakes
                 dir(bdiff), dir(mpatch), dir(base85), dir(osutil)
             if rustext:
                 from .rustext import (  # pytype: disable=import-error
                     ancestor,
                     dirstate,
                 )
 
                 dir(ancestor), dir(dirstate)  # quiet pyflakes
         except Exception as inst:
             err = stringutil.forcebytestr(inst)
             problems += 1
         fm.condwrite(err, b'extensionserror', b" %s\n", err)
 
     compengines = util.compengines._engines.values()
     fm.write(
         b'compengines',
         _(b'checking registered compression engines (%s)\n'),
         fm.formatlist(
             sorted(e.name() for e in compengines),
             name=b'compengine',
             fmt=b'%s',
             sep=b', ',
         ),
     )
     fm.write(
         b'compenginesavail',
         _(b'checking available compression engines (%s)\n'),
         fm.formatlist(
             sorted(e.name() for e in compengines if e.available()),
             name=b'compengine',
             fmt=b'%s',
             sep=b', ',
         ),
     )
     wirecompengines = compression.compengines.supportedwireengines(
         compression.SERVERROLE
     )
     fm.write(
         b'compenginesserver',
         _(
             b'checking available compression engines '
             b'for wire protocol (%s)\n'
         ),
         fm.formatlist(
             [e.name() for e in wirecompengines if e.wireprotosupport()],
             name=b'compengine',
             fmt=b'%s',
             sep=b', ',
         ),
     )
     re2 = b'missing'
     if util._re2:
         re2 = b'available'
     fm.plain(_(b'checking "re2" regexp engine (%s)\n') % re2)
     fm.data(re2=bool(util._re2))
 
     # templates
     p = templater.templatedir()
     fm.write(b'templatedirs', b'checking templates (%s)...\n', p or b'')
     fm.condwrite(not p, b'', _(b" no template directories found\n"))
     if p:
         (m, fp) = templater.try_open_template(b"map-cmdline.default")
         if m:
             # template found, check if it is working
             err = None
             try:
                 templater.templater.frommapfile(m)
             except Exception as inst:
                 err = stringutil.forcebytestr(inst)
                 p = None
             fm.condwrite(err, b'defaulttemplateerror', b" %s\n", err)
         else:
             p = None
         fm.condwrite(
             p, b'defaulttemplate', _(b"checking default template (%s)\n"), m
         )
         fm.condwrite(
             not m,
             b'defaulttemplatenotfound',
             _(b" template '%s' not found\n"),
             b"default",
         )
     if not p:
         problems += 1
     fm.condwrite(
         not p, b'', _(b" (templates seem to have been installed incorrectly)\n")
     )
 
     # editor
     editor = ui.geteditor()
     editor = util.expandpath(editor)
     editorbin = procutil.shellsplit(editor)[0]
     fm.write(b'editor', _(b"checking commit editor... (%s)\n"), editorbin)
     cmdpath = procutil.findexe(editorbin)
     fm.condwrite(
         not cmdpath and editor == b'vi',
         b'vinotfound',
         _(
             b" No commit editor set and can't find %s in PATH\n"
             b" (specify a commit editor in your configuration"
             b" file)\n"
         ),
         not cmdpath and editor == b'vi' and editorbin,
     )
     fm.condwrite(
         not cmdpath and editor != b'vi',
         b'editornotfound',
         _(
             b" Can't find editor '%s' in PATH\n"
             b" (specify a commit editor in your configuration"
             b" file)\n"
         ),
         not cmdpath and editorbin,
     )
     if not cmdpath and editor != b'vi':
         problems += 1
 
     # check username
     username = None
     err = None
     try:
         username = ui.username()
     except error.Abort as e:
         err = e.message
         problems += 1
 
     fm.condwrite(
         username, b'username', _(b"checking username (%s)\n"), username
     )
     fm.condwrite(
         err,
         b'usernameerror',
         _(
             b"checking username...\n %s\n"
             b" (specify a username in your configuration file)\n"
         ),
         err,
     )
 
     for name, mod in extensions.extensions():
         handler = getattr(mod, 'debuginstall', None)
         if handler is not None:
             problems += handler(ui, fm)
 
     fm.condwrite(not problems, b'', _(b"no problems detected\n"))
     if not problems:
         fm.data(problems=problems)
     fm.condwrite(
         problems,
         b'problems',
         _(b"%d problems detected, please check your install!\n"),
         problems,
     )
     fm.end()
 
     return problems
 
 
 @command(b'debugknown', [], _(b'REPO ID...'), norepo=True)
 def debugknown(ui, repopath, *ids, **opts):
     """test whether node ids are known to a repo
 
     Every ID must be a full-length hex node id string. Returns a list of 0s
     and 1s indicating unknown/known.
     """
     opts = pycompat.byteskwargs(opts)
     repo = hg.peer(ui, opts, repopath)
     if not repo.capable(b'known'):
         raise error.Abort(b"known() not supported by target repository")
     flags = repo.known([bin(s) for s in ids])
     ui.write(b"%s\n" % (b"".join([f and b"1" or b"0" for f in flags])))
 
 
 @command(b'debuglabelcomplete', [], _(b'LABEL...'))
 def debuglabelcomplete(ui, repo, *args):
     '''backwards compatibility with old bash completion scripts (DEPRECATED)'''
     debugnamecomplete(ui, repo, *args)
 
 
 @command(
     b'debuglocks',
     [
         (b'L', b'force-free-lock', None, _(b'free the store lock (DANGEROUS)')),
         (
             b'W',
             b'force-free-wlock',
             None,
             _(b'free the working state lock (DANGEROUS)'),
         ),
         (b's', b'set-lock', None, _(b'set the store lock until stopped')),
         (
             b'S',
             b'set-wlock',
             None,
             _(b'set the working state lock until stopped'),
         ),
     ],
     _(b'[OPTION]...'),
 )
 def debuglocks(ui, repo, **opts):
     """show or modify state of locks
 
     By default, this command will show which locks are held. This
     includes the user and process holding the lock, the amount of time
     the lock has been held, and the machine name where the process is
     running if it's not local.
 
     Locks protect the integrity of Mercurial's data, so should be
     treated with care. System crashes or other interruptions may cause
     locks to not be properly released, though Mercurial will usually
     detect and remove such stale locks automatically.
 
     However, detecting stale locks may not always be possible (for
     instance, on a shared filesystem). Removing locks may also be
     blocked by filesystem permissions.
 
     Setting a lock will prevent other commands from changing the data.
     The command will wait until an interruption (SIGINT, SIGTERM, ...) occurs.
     The set locks are removed when the command exits.
 
     Returns 0 if no locks are held.
 
     """
 
     if opts.get('force_free_lock'):
         repo.svfs.unlink(b'lock')
     if opts.get('force_free_wlock'):
         repo.vfs.unlink(b'wlock')
     if opts.get('force_free_lock') or opts.get('force_free_wlock'):
         return 0
 
     locks = []
     try:
         if opts.get('set_wlock'):
             try:
                 locks.append(repo.wlock(False))
             except error.LockHeld:
                 raise error.Abort(_(b'wlock is already held'))
         if opts.get('set_lock'):
             try:
                 locks.append(repo.lock(False))
             except error.LockHeld:
                 raise error.Abort(_(b'lock is already held'))
         if len(locks):
             ui.promptchoice(_(b"ready to release the lock (y)? $$ &Yes"))
             return 0
     finally:
         release(*locks)
 
     now = time.time()
     held = 0
 
     def report(vfs, name, method):
         # this causes stale locks to get reaped for more accurate reporting
         try:
             l = method(False)
         except error.LockHeld:
             l = None
 
         if l:
             l.release()
         else:
             try:
                 st = vfs.lstat(name)
                 age = now - st[stat.ST_MTIME]
                 user = util.username(st.st_uid)
                 locker = vfs.readlock(name)
                 if b":" in locker:
                     host, pid = locker.split(b':')
                     if host == socket.gethostname():
                         locker = b'user %s, process %s' % (user or b'None', pid)
                     else:
                         locker = b'user %s, process %s, host %s' % (
                             user or b'None',
                             pid,
                             host,
                         )
                 ui.writenoi18n(b"%-6s %s (%ds)\n" % (name + b":", locker, age))
                 return 1
             except OSError as e:
                 if e.errno != errno.ENOENT:
                     raise
 
         ui.writenoi18n(b"%-6s free\n" % (name + b":"))
         return 0
 
     held += report(repo.svfs, b"lock", repo.lock)
     held += report(repo.vfs, b"wlock", repo.wlock)
 
     return held
 
 
 @command(
     b'debugmanifestfulltextcache',
     [
         (b'', b'clear', False, _(b'clear the cache')),
         (
             b'a',
             b'add',
             [],
             _(b'add the given manifest nodes to the cache'),
             _(b'NODE'),
         ),
     ],
     b'',
 )
 def debugmanifestfulltextcache(ui, repo, add=(), **opts):
     """show, clear or amend the contents of the manifest fulltext cache"""
 
     def getcache():
         r = repo.manifestlog.getstorage(b'')
         try:
             return r._fulltextcache
         except AttributeError:
             msg = _(
                 b"Current revlog implementation doesn't appear to have a "
                 b"manifest fulltext cache\n"
             )
             raise error.Abort(msg)
 
     if opts.get('clear'):
         with repo.wlock():
             cache = getcache()
             cache.clear(clear_persisted_data=True)
             return
 
     if add:
         with repo.wlock():
             m = repo.manifestlog
             store = m.getstorage(b'')
             for n in add:
                 try:
                     manifest = m[store.lookup(n)]
                 except error.LookupError as e:
                     raise error.Abort(e, hint=b"Check your manifest node id")
                 manifest.read()  # stores revisision in cache too
             return
 
     cache = getcache()
     if not len(cache):
         ui.write(_(b'cache empty\n'))
     else:
         ui.write(
             _(
                 b'cache contains %d manifest entries, in order of most to '
                 b'least recent:\n'
             )
             % (len(cache),)
         )
         totalsize = 0
         for nodeid in cache:
             # Use cache.get to not update the LRU order
             data = cache.peek(nodeid)
             size = len(data)
             totalsize += size + 24  # 20 bytes nodeid, 4 bytes size
             ui.write(
                 _(b'id: %s, size %s\n') % (hex(nodeid), util.bytecount(size))
             )
         ondisk = cache._opener.stat(b'manifestfulltextcache').st_size
         ui.write(
             _(b'total cache data size %s, on-disk %s\n')
             % (util.bytecount(totalsize), util.bytecount(ondisk))
         )
 
 
 @command(b'debugmergestate', [] + cmdutil.templateopts, b'')
 def debugmergestate(ui, repo, *args, **opts):
     """print merge state
 
     Use --verbose to print out information about whether v1 or v2 merge state
     was chosen."""
 
     if ui.verbose:
         ms = mergestatemod.mergestate(repo)
 
         # sort so that reasonable information is on top
         v1records = ms._readrecordsv1()
         v2records = ms._readrecordsv2()
 
         if not v1records and not v2records:
             pass
         elif not v2records:
             ui.writenoi18n(b'no version 2 merge state\n')
         elif ms._v1v2match(v1records, v2records):
             ui.writenoi18n(b'v1 and v2 states match: using v2\n')
         else:
             ui.writenoi18n(b'v1 and v2 states mismatch: using v1\n')
 
     opts = pycompat.byteskwargs(opts)
     if not opts[b'template']:
         opts[b'template'] = (
             b'{if(commits, "", "no merge state found\n")}'
             b'{commits % "{name}{if(label, " ({label})")}: {node}\n"}'
             b'{files % "file: {path} (state \\"{state}\\")\n'
             b'{if(local_path, "'
             b'  local path: {local_path} (hash {local_key}, flags \\"{local_flags}\\")\n'
             b'  ancestor path: {ancestor_path} (node {ancestor_node})\n'
             b'  other path: {other_path} (node {other_node})\n'
             b'")}'
             b'{if(rename_side, "'
             b'  rename side: {rename_side}\n'
             b'  renamed path: {renamed_path}\n'
             b'")}'
             b'{extras % "  extra: {key} = {value}\n"}'
             b'"}'
             b'{extras % "extra: {file} ({key} = {value})\n"}'
         )
 
     ms = mergestatemod.mergestate.read(repo)
 
     fm = ui.formatter(b'debugmergestate', opts)
     fm.startitem()
 
     fm_commits = fm.nested(b'commits')
     if ms.active():
         for name, node, label_index in (
             (b'local', ms.local, 0),
             (b'other', ms.other, 1),
         ):
             fm_commits.startitem()
             fm_commits.data(name=name)
             fm_commits.data(node=hex(node))
             if ms._labels and len(ms._labels) > label_index:
                 fm_commits.data(label=ms._labels[label_index])
     fm_commits.end()
 
     fm_files = fm.nested(b'files')
     if ms.active():
         for f in ms:
             fm_files.startitem()
             fm_files.data(path=f)
             state = ms._state[f]
             fm_files.data(state=state[0])
             if state[0] in (
                 mergestatemod.MERGE_RECORD_UNRESOLVED,
                 mergestatemod.MERGE_RECORD_RESOLVED,
             ):
                 fm_files.data(local_key=state[1])
                 fm_files.data(local_path=state[2])
                 fm_files.data(ancestor_path=state[3])
                 fm_files.data(ancestor_node=state[4])
                 fm_files.data(other_path=state[5])
                 fm_files.data(other_node=state[6])
                 fm_files.data(local_flags=state[7])
             elif state[0] in (
                 mergestatemod.MERGE_RECORD_UNRESOLVED_PATH,
                 mergestatemod.MERGE_RECORD_RESOLVED_PATH,
             ):
                 fm_files.data(renamed_path=state[1])
                 fm_files.data(rename_side=state[2])
             fm_extras = fm_files.nested(b'extras')
             for k, v in sorted(ms.extras(f).items()):
                 fm_extras.startitem()
                 fm_extras.data(key=k)
                 fm_extras.data(value=v)
             fm_extras.end()
 
     fm_files.end()
 
     fm_extras = fm.nested(b'extras')
     for f, d in sorted(pycompat.iteritems(ms.allextras())):
         if f in ms:
             # If file is in mergestate, we have already processed it's extras
             continue
         for k, v in pycompat.iteritems(d):
             fm_extras.startitem()
             fm_extras.data(file=f)
             fm_extras.data(key=k)
             fm_extras.data(value=v)
     fm_extras.end()
 
     fm.end()
 
 
 @command(b'debugnamecomplete', [], _(b'NAME...'))
 def debugnamecomplete(ui, repo, *args):
     '''complete "names" - tags, open branch names, bookmark names'''
 
     names = set()
     # since we previously only listed open branches, we will handle that
     # specially (after this for loop)
     for name, ns in pycompat.iteritems(repo.names):
         if name != b'branches':
             names.update(ns.listnames(repo))
     names.update(
         tag
         for (tag, heads, tip, closed) in repo.branchmap().iterbranches()
         if not closed
     )
     completions = set()
     if not args:
         args = [b'']
     for a in args:
         completions.update(n for n in names if n.startswith(a))
     ui.write(b'\n'.join(sorted(completions)))
     ui.write(b'\n')
 
 
 @command(
     b'debugnodemap',
     [
         (
             b'',
             b'dump-new',
             False,
             _(b'write a (new) persistent binary nodemap on stdout'),
         ),
         (b'', b'dump-disk', False, _(b'dump on-disk data on stdout')),
         (
             b'',
             b'check',
             False,
             _(b'check that the data on disk data are correct.'),
         ),
         (
             b'',
             b'metadata',
             False,
             _(b'display the on disk meta data for the nodemap'),
         ),
     ],
 )
 def debugnodemap(ui, repo, **opts):
     """write and inspect on disk nodemap"""
     if opts['dump_new']:
         unfi = repo.unfiltered()
         cl = unfi.changelog
         if util.safehasattr(cl.index, "nodemap_data_all"):
             data = cl.index.nodemap_data_all()
         else:
             data = nodemap.persistent_data(cl.index)
         ui.write(data)
     elif opts['dump_disk']:
         unfi = repo.unfiltered()
         cl = unfi.changelog
         nm_data = nodemap.persisted_data(cl)
         if nm_data is not None:
             docket, data = nm_data
             ui.write(data[:])
     elif opts['check']:
         unfi = repo.unfiltered()
         cl = unfi.changelog
         nm_data = nodemap.persisted_data(cl)
         if nm_data is not None:
             docket, data = nm_data
             return nodemap.check_data(ui, cl.index, data)
     elif opts['metadata']:
         unfi = repo.unfiltered()
         cl = unfi.changelog
         nm_data = nodemap.persisted_data(cl)
         if nm_data is not None:
             docket, data = nm_data
             ui.write((b"uid: %s\n") % docket.uid)
             ui.write((b"tip-rev: %d\n") % docket.tip_rev)
             ui.write((b"tip-node: %s\n") % hex(docket.tip_node))
             ui.write((b"data-length: %d\n") % docket.data_length)
             ui.write((b"data-unused: %d\n") % docket.data_unused)
             unused_perc = docket.data_unused * 100.0 / docket.data_length
             ui.write((b"data-unused: %2.3f%%\n") % unused_perc)
 
 
 @command(
     b'debugobsolete',
     [
         (b'', b'flags', 0, _(b'markers flag')),
         (
             b'',
             b'record-parents',
             False,
             _(b'record parent information for the precursor'),
         ),
         (b'r', b'rev', [], _(b'display markers relevant to REV')),
         (
             b'',
             b'exclusive',
             False,
             _(b'restrict display to markers only relevant to REV'),
         ),
         (b'', b'index', False, _(b'display index of the marker')),
         (b'', b'delete', [], _(b'delete markers specified by indices')),
     ]
     + cmdutil.commitopts2
     + cmdutil.formatteropts,
     _(b'[OBSOLETED [REPLACEMENT ...]]'),
 )
 def debugobsolete(ui, repo, precursor=None, *successors, **opts):
     """create arbitrary obsolete marker
 
     With no arguments, displays the list of obsolescence markers."""
 
     opts = pycompat.byteskwargs(opts)
 
     def parsenodeid(s):
         try:
             # We do not use revsingle/revrange functions here to accept
             # arbitrary node identifiers, possibly not present in the
             # local repository.
             n = bin(s)
             if len(n) != len(nullid):
                 raise TypeError()
             return n
         except TypeError:
             raise error.InputError(
                 b'changeset references must be full hexadecimal '
                 b'node identifiers'
             )
 
     if opts.get(b'delete'):
         indices = []
         for v in opts.get(b'delete'):
             try:
                 indices.append(int(v))
             except ValueError:
                 raise error.InputError(
                     _(b'invalid index value: %r') % v,
                     hint=_(b'use integers for indices'),
                 )
 
         if repo.currenttransaction():
             raise error.Abort(
                 _(b'cannot delete obsmarkers in the middle of transaction.')
             )
 
         with repo.lock():
             n = repair.deleteobsmarkers(repo.obsstore, indices)
             ui.write(_(b'deleted %i obsolescence markers\n') % n)
 
         return
 
     if precursor is not None:
         if opts[b'rev']:
             raise error.InputError(
                 b'cannot select revision when creating marker'
             )
         metadata = {}
         metadata[b'user'] = encoding.fromlocal(opts[b'user'] or ui.username())
         succs = tuple(parsenodeid(succ) for succ in successors)
         l = repo.lock()
         try:
             tr = repo.transaction(b'debugobsolete')
             try:
                 date = opts.get(b'date')
                 if date:
                     date = dateutil.parsedate(date)
                 else:
                     date = None
                 prec = parsenodeid(precursor)
                 parents = None
                 if opts[b'record_parents']:
                     if prec not in repo.unfiltered():
                         raise error.Abort(
                             b'cannot used --record-parents on '
                             b'unknown changesets'
                         )
                     parents = repo.unfiltered()[prec].parents()
                     parents = tuple(p.node() for p in parents)
                 repo.obsstore.create(
                     tr,
                     prec,
                     succs,
                     opts[b'flags'],
                     parents=parents,
                     date=date,
                     metadata=metadata,
                     ui=ui,
                 )
                 tr.close()
             except ValueError as exc:
                 raise error.Abort(
                     _(b'bad obsmarker input: %s') % pycompat.bytestr(exc)
                 )
             finally:
                 tr.release()
         finally:
             l.release()
     else:
         if opts[b'rev']:
             revs = scmutil.revrange(repo, opts[b'rev'])
             nodes = [repo[r].node() for r in revs]
             markers = list(
                 obsutil.getmarkers(
                     repo, nodes=nodes, exclusive=opts[b'exclusive']
                 )
             )
             markers.sort(key=lambda x: x._data)
         else:
             markers = obsutil.getmarkers(repo)
 
         markerstoiter = markers
         isrelevant = lambda m: True
         if opts.get(b'rev') and opts.get(b'index'):
             markerstoiter = obsutil.getmarkers(repo)
             markerset = set(markers)
             isrelevant = lambda m: m in markerset
 
         fm = ui.formatter(b'debugobsolete', opts)
         for i, m in enumerate(markerstoiter):
             if not isrelevant(m):
                 # marker can be irrelevant when we're iterating over a set
                 # of markers (markerstoiter) which is bigger than the set
                 # of markers we want to display (markers)
                 # this can happen if both --index and --rev options are
                 # provided and thus we need to iterate over all of the markers
                 # to get the correct indices, but only display the ones that
                 # are relevant to --rev value
                 continue
             fm.startitem()
             ind = i if opts.get(b'index') else None
             cmdutil.showmarker(fm, m, index=ind)
         fm.end()
 
 
 @command(
     b'debugp1copies',
     [(b'r', b'rev', b'', _(b'revision to debug'), _(b'REV'))],
     _(b'[-r REV]'),
 )
 def debugp1copies(ui, repo, **opts):
     """dump copy information compared to p1"""
 
     opts = pycompat.byteskwargs(opts)
     ctx = scmutil.revsingle(repo, opts.get(b'rev'), default=None)
     for dst, src in ctx.p1copies().items():
         ui.write(b'%s -> %s\n' % (src, dst))
 
 
 @command(
     b'debugp2copies',
     [(b'r', b'rev', b'', _(b'revision to debug'), _(b'REV'))],
     _(b'[-r REV]'),
 )
 def debugp1copies(ui, repo, **opts):
     """dump copy information compared to p2"""
 
     opts = pycompat.byteskwargs(opts)
     ctx = scmutil.revsingle(repo, opts.get(b'rev'), default=None)
     for dst, src in ctx.p2copies().items():
         ui.write(b'%s -> %s\n' % (src, dst))
 
 
 @command(
     b'debugpathcomplete',
     [
         (b'f', b'full', None, _(b'complete an entire path')),
         (b'n', b'normal', None, _(b'show only normal files')),
         (b'a', b'added', None, _(b'show only added files')),
         (b'r', b'removed', None, _(b'show only removed files')),
     ],
     _(b'FILESPEC...'),
 )
 def debugpathcomplete(ui, repo, *specs, **opts):
     """complete part or all of a tracked path
 
     This command supports shells that offer path name completion. It
     currently completes only files already known to the dirstate.
 
     Completion extends only to the next path segment unless
     --full is specified, in which case entire paths are used."""
 
     def complete(path, acceptable):
         dirstate = repo.dirstate
         spec = os.path.normpath(os.path.join(encoding.getcwd(), path))
         rootdir = repo.root + pycompat.ossep
         if spec != repo.root and not spec.startswith(rootdir):
             return [], []
         if os.path.isdir(spec):
             spec += b'/'
         spec = spec[len(rootdir) :]
         fixpaths = pycompat.ossep != b'/'
         if fixpaths:
             spec = spec.replace(pycompat.ossep, b'/')
         speclen = len(spec)
         fullpaths = opts['full']
         files, dirs = set(), set()
         adddir, addfile = dirs.add, files.add
         for f, st in pycompat.iteritems(dirstate):
             if f.startswith(spec) and st[0] in acceptable:
                 if fixpaths:
                     f = f.replace(b'/', pycompat.ossep)
                 if fullpaths:
                     addfile(f)
                     continue
                 s = f.find(pycompat.ossep, speclen)
                 if s >= 0:
                     adddir(f[:s])
                 else:
                     addfile(f)
         return files, dirs
 
     acceptable = b''
     if opts['normal']:
         acceptable += b'nm'
     if opts['added']:
         acceptable += b'a'
     if opts['removed']:
         acceptable += b'r'
     cwd = repo.getcwd()
     if not specs:
         specs = [b'.']
 
     files, dirs = set(), set()
     for spec in specs:
         f, d = complete(spec, acceptable or b'nmar')
         files.update(f)
         dirs.update(d)
     files.update(dirs)
     ui.write(b'\n'.join(repo.pathto(p, cwd) for p in sorted(files)))
     ui.write(b'\n')
 
 
 @command(
     b'debugpathcopies',
     cmdutil.walkopts,
     b'hg debugpathcopies REV1 REV2 [FILE]',
     inferrepo=True,
 )
 def debugpathcopies(ui, repo, rev1, rev2, *pats, **opts):
     """show copies between two revisions"""
     ctx1 = scmutil.revsingle(repo, rev1)
     ctx2 = scmutil.revsingle(repo, rev2)
     m = scmutil.match(ctx1, pats, opts)
     for dst, src in sorted(copies.pathcopies(ctx1, ctx2, m).items()):
         ui.write(b'%s -> %s\n' % (src, dst))
 
 
 @command(b'debugpeer', [], _(b'PATH'), norepo=True)
 def debugpeer(ui, path):
     """establish a connection to a peer repository"""
     # Always enable peer request logging. Requires --debug to display
     # though.
     overrides = {
         (b'devel', b'debug.peer-request'): True,
     }
 
     with ui.configoverride(overrides):
         peer = hg.peer(ui, {}, path)
 
         local = peer.local() is not None
         canpush = peer.canpush()
 
         ui.write(_(b'url: %s\n') % peer.url())
         ui.write(_(b'local: %s\n') % (_(b'yes') if local else _(b'no')))
         ui.write(_(b'pushable: %s\n') % (_(b'yes') if canpush else _(b'no')))
 
 
 @command(
     b'debugpickmergetool',
     [
         (b'r', b'rev', b'', _(b'check for files in this revision'), _(b'REV')),
         (b'', b'changedelete', None, _(b'emulate merging change and delete')),
     ]
     + cmdutil.walkopts
     + cmdutil.mergetoolopts,
     _(b'[PATTERN]...'),
     inferrepo=True,
 )
 def debugpickmergetool(ui, repo, *pats, **opts):
     """examine which merge tool is chosen for specified file
 
     As described in :hg:`help merge-tools`, Mercurial examines
     configurations below in this order to decide which merge tool is
     chosen for specified file.
 
     1. ``--tool`` option
     2. ``HGMERGE`` environment variable
     3. configurations in ``merge-patterns`` section
     4. configuration of ``ui.merge``
     5. configurations in ``merge-tools`` section
     6. ``hgmerge`` tool (for historical reason only)
     7. default tool for fallback (``:merge`` or ``:prompt``)
 
     This command writes out examination result in the style below::
 
         FILE = MERGETOOL
 
     By default, all files known in the first parent context of the
     working directory are examined. Use file patterns and/or -I/-X
     options to limit target files. -r/--rev is also useful to examine
     files in another context without actual updating to it.
 
     With --debug, this command shows warning messages while matching
     against ``merge-patterns`` and so on, too. It is recommended to
     use this option with explicit file patterns and/or -I/-X options,
     because this option increases amount of output per file according
     to configurations in hgrc.
 
     With -v/--verbose, this command shows configurations below at
     first (only if specified).
 
     - ``--tool`` option
     - ``HGMERGE`` environment variable
     - configuration of ``ui.merge``
 
     If merge tool is chosen before matching against
     ``merge-patterns``, this command can't show any helpful
     information, even with --debug. In such case, information above is
     useful to know why a merge tool is chosen.
     """
     opts = pycompat.byteskwargs(opts)
     overrides = {}
     if opts[b'tool']:
         overrides[(b'ui', b'forcemerge')] = opts[b'tool']
         ui.notenoi18n(b'with --tool %r\n' % (pycompat.bytestr(opts[b'tool'])))
 
     with ui.configoverride(overrides, b'debugmergepatterns'):
         hgmerge = encoding.environ.get(b"HGMERGE")
         if hgmerge is not None:
             ui.notenoi18n(b'with HGMERGE=%r\n' % (pycompat.bytestr(hgmerge)))
         uimerge = ui.config(b"ui", b"merge")
         if uimerge:
             ui.notenoi18n(b'with ui.merge=%r\n' % (pycompat.bytestr(uimerge)))
 
         ctx = scmutil.revsingle(repo, opts.get(b'rev'))
         m = scmutil.match(ctx, pats, opts)
         changedelete = opts[b'changedelete']
         for path in ctx.walk(m):
             fctx = ctx[path]
             try:
                 if not ui.debugflag:
                     ui.pushbuffer(error=True)
                 tool, toolpath = filemerge._picktool(
                     repo,
                     ui,
                     path,
                     fctx.isbinary(),
                     b'l' in fctx.flags(),
                     changedelete,
                 )
             finally:
                 if not ui.debugflag:
                     ui.popbuffer()
             ui.write(b'%s = %s\n' % (path, tool))
 
 
 @command(b'debugpushkey', [], _(b'REPO NAMESPACE [KEY OLD NEW]'), norepo=True)
 def debugpushkey(ui, repopath, namespace, *keyinfo, **opts):
     """access the pushkey key/value protocol
 
     With two args, list the keys in the given namespace.
 
     With five args, set a key to new if it currently is set to old.
     Reports success or failure.
     """
 
     target = hg.peer(ui, {}, repopath)
     if keyinfo:
         key, old, new = keyinfo
         with target.commandexecutor() as e:
             r = e.callcommand(
                 b'pushkey',
                 {
                     b'namespace': namespace,
                     b'key': key,
                     b'old': old,
                     b'new': new,
                 },
             ).result()
 
         ui.status(pycompat.bytestr(r) + b'\n')
         return not r
     else:
         for k, v in sorted(pycompat.iteritems(target.listkeys(namespace))):
             ui.write(
                 b"%s\t%s\n" % (stringutil.escapestr(k), stringutil.escapestr(v))
             )
 
 
 @command(b'debugpvec', [], _(b'A B'))
 def debugpvec(ui, repo, a, b=None):
     ca = scmutil.revsingle(repo, a)
     cb = scmutil.revsingle(repo, b)
     pa = pvec.ctxpvec(ca)
     pb = pvec.ctxpvec(cb)
     if pa == pb:
         rel = b"="
     elif pa > pb:
         rel = b">"
     elif pa < pb:
         rel = b"<"
     elif pa | pb:
         rel = b"|"
     ui.write(_(b"a: %s\n") % pa)
     ui.write(_(b"b: %s\n") % pb)
     ui.write(_(b"depth(a): %d depth(b): %d\n") % (pa._depth, pb._depth))
     ui.write(
         _(b"delta: %d hdist: %d distance: %d relation: %s\n")
         % (
             abs(pa._depth - pb._depth),
             pvec._hamming(pa._vec, pb._vec),
             pa.distance(pb),
             rel,
         )
     )
 
 
 @command(
     b'debugrebuilddirstate|debugrebuildstate',
     [
         (b'r', b'rev', b'', _(b'revision to rebuild to'), _(b'REV')),
         (
             b'',
             b'minimal',
             None,
             _(
                 b'only rebuild files that are inconsistent with '
                 b'the working copy parent'
             ),
         ),
     ],
     _(b'[-r REV]'),
 )
 def debugrebuilddirstate(ui, repo, rev, **opts):
     """rebuild the dirstate as it would look like for the given revision
 
     If no revision is specified the first current parent will be used.
 
     The dirstate will be set to the files of the given revision.
     The actual working directory content or existing dirstate
     information such as adds or removes is not considered.
 
     ``minimal`` will only rebuild the dirstate status for files that claim to be
     tracked but are not in the parent manifest, or that exist in the parent
     manifest but are not in the dirstate. It will not change adds, removes, or
     modified files that are in the working copy parent.
 
     One use of this command is to make the next :hg:`status` invocation
     check the actual file content.
     """
     ctx = scmutil.revsingle(repo, rev)
     with repo.wlock():
         dirstate = repo.dirstate
         changedfiles = None
         # See command doc for what minimal does.
         if opts.get('minimal'):
             manifestfiles = set(ctx.manifest().keys())
             dirstatefiles = set(dirstate)
             manifestonly = manifestfiles - dirstatefiles
             dsonly = dirstatefiles - manifestfiles
             dsnotadded = {f for f in dsonly if dirstate[f] != b'a'}
             changedfiles = manifestonly | dsnotadded
 
         dirstate.rebuild(ctx.node(), ctx.manifest(), changedfiles)
 
 
 @command(b'debugrebuildfncache', [], b'')
 def debugrebuildfncache(ui, repo):
     """rebuild the fncache file"""
     repair.rebuildfncache(ui, repo)
 
 
 @command(
     b'debugrename',
     [(b'r', b'rev', b'', _(b'revision to debug'), _(b'REV'))],
     _(b'[-r REV] [FILE]...'),
 )
 def debugrename(ui, repo, *pats, **opts):
     """dump rename information"""
 
     opts = pycompat.byteskwargs(opts)
     ctx = scmutil.revsingle(repo, opts.get(b'rev'))
     m = scmutil.match(ctx, pats, opts)
     for abs in ctx.walk(m):
         fctx = ctx[abs]
         o = fctx.filelog().renamed(fctx.filenode())
         rel = repo.pathto(abs)
         if o:
             ui.write(_(b"%s renamed from %s:%s\n") % (rel, o[0], hex(o[1])))
         else:
             ui.write(_(b"%s not renamed\n") % rel)
 
 
 @command(b'debugrequires|debugrequirements', [], b'')
 def debugrequirements(ui, repo):
     """ print the current repo requirements """
     for r in sorted(repo.requirements):
         ui.write(b"%s\n" % r)
 
 
 @command(
     b'debugrevlog',
     cmdutil.debugrevlogopts + [(b'd', b'dump', False, _(b'dump index data'))],
     _(b'-c|-m|FILE'),
     optionalrepo=True,
 )
 def debugrevlog(ui, repo, file_=None, **opts):
     """show data and statistics about a revlog"""
     opts = pycompat.byteskwargs(opts)
     r = cmdutil.openrevlog(repo, b'debugrevlog', file_, opts)
 
     if opts.get(b"dump"):
         numrevs = len(r)
         ui.write(
             (
                 b"# rev p1rev p2rev start   end deltastart base   p1   p2"
                 b" rawsize totalsize compression heads chainlen\n"
             )
         )
         ts = 0
         heads = set()
 
         for rev in pycompat.xrange(numrevs):
             dbase = r.deltaparent(rev)
             if dbase == -1:
                 dbase = rev
             cbase = r.chainbase(rev)
             clen = r.chainlen(rev)
             p1, p2 = r.parentrevs(rev)
             rs = r.rawsize(rev)
             ts = ts + rs
             heads -= set(r.parentrevs(rev))
             heads.add(rev)
             try:
                 compression = ts / r.end(rev)
             except ZeroDivisionError:
                 compression = 0
             ui.write(
                 b"%5d %5d %5d %5d %5d %10d %4d %4d %4d %7d %9d "
                 b"%11d %5d %8d\n"
                 % (
                     rev,
                     p1,
                     p2,
                     r.start(rev),
                     r.end(rev),
                     r.start(dbase),
                     r.start(cbase),
                     r.start(p1),
                     r.start(p2),
                     rs,
                     ts,
                     compression,
                     len(heads),
                     clen,
                 )
             )
         return 0
 
     v = r.version
     format = v & 0xFFFF
     flags = []
     gdelta = False
     if v & revlog.FLAG_INLINE_DATA:
         flags.append(b'inline')
     if v & revlog.FLAG_GENERALDELTA:
         gdelta = True
         flags.append(b'generaldelta')
     if not flags:
         flags = [b'(none)']
 
     ### tracks merge vs single parent
     nummerges = 0
 
     ### tracks ways the "delta" are build
     # nodelta
     numempty = 0
     numemptytext = 0
     numemptydelta = 0
     # full file content
     numfull = 0
     # intermediate snapshot against a prior snapshot
     numsemi = 0
     # snapshot count per depth
     numsnapdepth = collections.defaultdict(lambda: 0)
     # delta against previous revision
     numprev = 0
     # delta against first or second parent (not prev)
     nump1 = 0
     nump2 = 0
     # delta against neither prev nor parents
     numother = 0
     # delta against prev that are also first or second parent
     # (details of `numprev`)
     nump1prev = 0
     nump2prev = 0
 
     # data about delta chain of each revs
     chainlengths = []
     chainbases = []
     chainspans = []
 
     # data about each revision
     datasize = [None, 0, 0]
     fullsize = [None, 0, 0]
     semisize = [None, 0, 0]
     # snapshot count per depth
     snapsizedepth = collections.defaultdict(lambda: [None, 0, 0])
     deltasize = [None, 0, 0]
     chunktypecounts = {}
     chunktypesizes = {}
 
     def addsize(size, l):
         if l[0] is None or size < l[0]:
             l[0] = size
         if size > l[1]:
             l[1] = size
         l[2] += size
 
     numrevs = len(r)
     for rev in pycompat.xrange(numrevs):
         p1, p2 = r.parentrevs(rev)
         delta = r.deltaparent(rev)
         if format > 0:
             addsize(r.rawsize(rev), datasize)
         if p2 != nullrev:
             nummerges += 1
         size = r.length(rev)
         if delta == nullrev:
             chainlengths.append(0)
             chainbases.append(r.start(rev))
             chainspans.append(size)
             if size == 0:
                 numempty += 1
                 numemptytext += 1
             else:
                 numfull += 1
                 numsnapdepth[0] += 1
                 addsize(size, fullsize)
                 addsize(size, snapsizedepth[0])
         else:
             chainlengths.append(chainlengths[delta] + 1)
             baseaddr = chainbases[delta]
             revaddr = r.start(rev)
             chainbases.append(baseaddr)
             chainspans.append((revaddr - baseaddr) + size)
             if size == 0:
                 numempty += 1
                 numemptydelta += 1
             elif r.issnapshot(rev):
                 addsize(size, semisize)
                 numsemi += 1
                 depth = r.snapshotdepth(rev)
                 numsnapdepth[depth] += 1
                 addsize(size, snapsizedepth[depth])
             else:
                 addsize(size, deltasize)
                 if delta == rev - 1:
                     numprev += 1
                     if delta == p1:
                         nump1prev += 1
                     elif delta == p2:
                         nump2prev += 1
                 elif delta == p1:
                     nump1 += 1
                 elif delta == p2:
                     nump2 += 1
                 elif delta != nullrev:
                     numother += 1
 
         # Obtain data on the raw chunks in the revlog.
         if util.safehasattr(r, b'_getsegmentforrevs'):
             segment = r._getsegmentforrevs(rev, rev)[1]
         else:
             segment = r._revlog._getsegmentforrevs(rev, rev)[1]
         if segment:
             chunktype = bytes(segment[0:1])
         else:
             chunktype = b'empty'
 
         if chunktype not in chunktypecounts:
             chunktypecounts[chunktype] = 0
             chunktypesizes[chunktype] = 0
 
         chunktypecounts[chunktype] += 1
         chunktypesizes[chunktype] += size
 
     # Adjust size min value for empty cases
     for size in (datasize, fullsize, semisize, deltasize):
         if size[0] is None:
             size[0] = 0
 
     numdeltas = numrevs - numfull - numempty - numsemi
     numoprev = numprev - nump1prev - nump2prev
     totalrawsize = datasize[2]
     datasize[2] /= numrevs
     fulltotal = fullsize[2]
     if numfull == 0:
         fullsize[2] = 0
     else:
         fullsize[2] /= numfull
     semitotal = semisize[2]
     snaptotal = {}
     if numsemi > 0:
         semisize[2] /= numsemi
     for depth in snapsizedepth:
         snaptotal[depth] = snapsizedepth[depth][2]
         snapsizedepth[depth][2] /= numsnapdepth[depth]
 
     deltatotal = deltasize[2]
     if numdeltas > 0:
         deltasize[2] /= numdeltas
     totalsize = fulltotal + semitotal + deltatotal
     avgchainlen = sum(chainlengths) / numrevs
     maxchainlen = max(chainlengths)
     maxchainspan = max(chainspans)
     compratio = 1
     if totalsize:
         compratio = totalrawsize / totalsize
 
     basedfmtstr = b'%%%dd\n'
     basepcfmtstr = b'%%%dd %s(%%5.2f%%%%)\n'
 
     def dfmtstr(max):
         return basedfmtstr % len(str(max))
 
     def pcfmtstr(max, padding=0):
         return basepcfmtstr % (len(str(max)), b' ' * padding)
 
     def pcfmt(value, total):
         if total:
             return (value, 100 * float(value) / total)
         else:
             return value, 100.0
 
     ui.writenoi18n(b'format : %d\n' % format)
     ui.writenoi18n(b'flags  : %s\n' % b', '.join(flags))
 
     ui.write(b'\n')
     fmt = pcfmtstr(totalsize)
     fmt2 = dfmtstr(totalsize)
     ui.writenoi18n(b'revisions     : ' + fmt2 % numrevs)
     ui.writenoi18n(b'    merges    : ' + fmt % pcfmt(nummerges, numrevs))
     ui.writenoi18n(
         b'    normal    : ' + fmt % pcfmt(numrevs - nummerges, numrevs)
     )
     ui.writenoi18n(b'revisions     : ' + fmt2 % numrevs)
     ui.writenoi18n(b'    empty     : ' + fmt % pcfmt(numempty, numrevs))
     ui.writenoi18n(
         b'                   text  : '
         + fmt % pcfmt(numemptytext, numemptytext + numemptydelta)
     )
     ui.writenoi18n(
         b'                   delta : '
         + fmt % pcfmt(numemptydelta, numemptytext + numemptydelta)
     )
     ui.writenoi18n(
         b'    snapshot  : ' + fmt % pcfmt(numfull + numsemi, numrevs)
     )
     for depth in sorted(numsnapdepth):
         ui.write(
             (b'      lvl-%-3d :       ' % depth)
             + fmt % pcfmt(numsnapdepth[depth], numrevs)
         )
     ui.writenoi18n(b'    deltas    : ' + fmt % pcfmt(numdeltas, numrevs))
     ui.writenoi18n(b'revision size : ' + fmt2 % totalsize)
     ui.writenoi18n(
         b'    snapshot  : ' + fmt % pcfmt(fulltotal + semitotal, totalsize)
     )
     for depth in sorted(numsnapdepth):
         ui.write(
             (b'      lvl-%-3d :       ' % depth)
             + fmt % pcfmt(snaptotal[depth], totalsize)
         )
     ui.writenoi18n(b'    deltas    : ' + fmt % pcfmt(deltatotal, totalsize))
 
     def fmtchunktype(chunktype):
         if chunktype == b'empty':
             return b'    %s     : ' % chunktype
         elif chunktype in pycompat.bytestr(string.ascii_letters):
             return b'    0x%s (%s)  : ' % (hex(chunktype), chunktype)
         else:
             return b'    0x%s      : ' % hex(chunktype)
 
     ui.write(b'\n')
     ui.writenoi18n(b'chunks        : ' + fmt2 % numrevs)
     for chunktype in sorted(chunktypecounts):
         ui.write(fmtchunktype(chunktype))
         ui.write(fmt % pcfmt(chunktypecounts[chunktype], numrevs))
     ui.writenoi18n(b'chunks size   : ' + fmt2 % totalsize)
     for chunktype in sorted(chunktypecounts):
         ui.write(fmtchunktype(chunktype))
         ui.write(fmt % pcfmt(chunktypesizes[chunktype], totalsize))
 
     ui.write(b'\n')
     fmt = dfmtstr(max(avgchainlen, maxchainlen, maxchainspan, compratio))
     ui.writenoi18n(b'avg chain length  : ' + fmt % avgchainlen)
     ui.writenoi18n(b'max chain length  : ' + fmt % maxchainlen)
     ui.writenoi18n(b'max chain reach   : ' + fmt % maxchainspan)
     ui.writenoi18n(b'compression ratio : ' + fmt % compratio)
 
     if format > 0:
         ui.write(b'\n')
         ui.writenoi18n(
             b'uncompressed data size (min/max/avg) : %d / %d / %d\n'
             % tuple(datasize)
         )
     ui.writenoi18n(
         b'full revision size (min/max/avg)     : %d / %d / %d\n'
         % tuple(fullsize)
     )
     ui.writenoi18n(
         b'inter-snapshot size (min/max/avg)    : %d / %d / %d\n'
         % tuple(semisize)
     )
     for depth in sorted(snapsizedepth):
         if depth == 0:
             continue
         ui.writenoi18n(
             b'    level-%-3d (min/max/avg)          : %d / %d / %d\n'
             % ((depth,) + tuple(snapsizedepth[depth]))
         )
     ui.writenoi18n(
         b'delta size (min/max/avg)             : %d / %d / %d\n'
         % tuple(deltasize)
     )
 
     if numdeltas > 0:
         ui.write(b'\n')
         fmt = pcfmtstr(numdeltas)
         fmt2 = pcfmtstr(numdeltas, 4)
         ui.writenoi18n(
             b'deltas against prev  : ' + fmt % pcfmt(numprev, numdeltas)
         )
         if numprev > 0:
             ui.writenoi18n(
                 b'    where prev = p1  : ' + fmt2 % pcfmt(nump1prev, numprev)
             )
             ui.writenoi18n(
                 b'    where prev = p2  : ' + fmt2 % pcfmt(nump2prev, numprev)
             )
             ui.writenoi18n(
                 b'    other            : ' + fmt2 % pcfmt(numoprev, numprev)
             )
         if gdelta:
             ui.writenoi18n(
                 b'deltas against p1    : ' + fmt % pcfmt(nump1, numdeltas)
             )
             ui.writenoi18n(
                 b'deltas against p2    : ' + fmt % pcfmt(nump2, numdeltas)
             )
             ui.writenoi18n(
                 b'deltas against other : ' + fmt % pcfmt(numother, numdeltas)
             )
 
 
 @command(
     b'debugrevlogindex',
     cmdutil.debugrevlogopts
     + [(b'f', b'format', 0, _(b'revlog format'), _(b'FORMAT'))],
     _(b'[-f FORMAT] -c|-m|FILE'),
     optionalrepo=True,
 )
 def debugrevlogindex(ui, repo, file_=None, **opts):
     """dump the contents of a revlog index"""
     opts = pycompat.byteskwargs(opts)
     r = cmdutil.openrevlog(repo, b'debugrevlogindex', file_, opts)
     format = opts.get(b'format', 0)
     if format not in (0, 1):
         raise error.Abort(_(b"unknown format %d") % format)
 
     if ui.debugflag:
         shortfn = hex
     else:
         shortfn = short
 
     # There might not be anything in r, so have a sane default
     idlen = 12
     for i in r:
         idlen = len(shortfn(r.node(i)))
         break
 
     if format == 0:
         if ui.verbose:
             ui.writenoi18n(
                 b"   rev    offset  length linkrev %s %s p2\n"
                 % (b"nodeid".ljust(idlen), b"p1".ljust(idlen))
             )
         else:
             ui.writenoi18n(
                 b"   rev linkrev %s %s p2\n"
                 % (b"nodeid".ljust(idlen), b"p1".ljust(idlen))
             )
     elif format == 1:
         if ui.verbose:
             ui.writenoi18n(
                 (
                     b"   rev flag   offset   length     size   link     p1"
                     b"     p2 %s\n"
                 )
                 % b"nodeid".rjust(idlen)
             )
         else:
             ui.writenoi18n(
                 b"   rev flag     size   link     p1     p2 %s\n"
                 % b"nodeid".rjust(idlen)
             )
 
     for i in r:
         node = r.node(i)
         if format == 0:
             try:
                 pp = r.parents(node)
             except Exception:
                 pp = [nullid, nullid]
             if ui.verbose:
                 ui.write(
                     b"% 6d % 9d % 7d % 7d %s %s %s\n"
                     % (
                         i,
                         r.start(i),
                         r.length(i),
                         r.linkrev(i),
                         shortfn(node),
                         shortfn(pp[0]),
                         shortfn(pp[1]),
                     )
                 )
             else:
                 ui.write(
                     b"% 6d % 7d %s %s %s\n"
                     % (
                         i,
                         r.linkrev(i),
                         shortfn(node),
                         shortfn(pp[0]),
                         shortfn(pp[1]),
                     )
                 )
         elif format == 1:
             pr = r.parentrevs(i)
             if ui.verbose:
                 ui.write(
                     b"% 6d %04x % 8d % 8d % 8d % 6d % 6d % 6d %s\n"
                     % (
                         i,
                         r.flags(i),
                         r.start(i),
                         r.length(i),
                         r.rawsize(i),
                         r.linkrev(i),
                         pr[0],
                         pr[1],
                         shortfn(node),
                     )
                 )
             else:
                 ui.write(
                     b"% 6d %04x % 8d % 6d % 6d % 6d %s\n"
                     % (
                         i,
                         r.flags(i),
                         r.rawsize(i),
                         r.linkrev(i),
                         pr[0],
                         pr[1],
                         shortfn(node),
                     )
                 )
 
 
 @command(
     b'debugrevspec',
     [
         (
             b'',
             b'optimize',
             None,
             _(b'print parsed tree after optimizing (DEPRECATED)'),
         ),
         (
             b'',
             b'show-revs',
             True,
             _(b'print list of result revisions (default)'),
         ),
         (
             b's',
             b'show-set',
             None,
             _(b'print internal representation of result set'),
         ),
         (
             b'p',
             b'show-stage',
             [],
             _(b'print parsed tree at the given stage'),
             _(b'NAME'),
         ),
         (b'', b'no-optimized', False, _(b'evaluate tree without optimization')),
         (b'', b'verify-optimized', False, _(b'verify optimized result')),
     ],
     b'REVSPEC',
 )
 def debugrevspec(ui, repo, expr, **opts):
     """parse and apply a revision specification
 
     Use -p/--show-stage option to print the parsed tree at the given stages.
     Use -p all to print tree at every stage.
 
     Use --no-show-revs option with -s or -p to print only the set
     representation or the parsed tree respectively.
 
     Use --verify-optimized to compare the optimized result with the unoptimized
     one. Returns 1 if the optimized result differs.
     """
     opts = pycompat.byteskwargs(opts)
     aliases = ui.configitems(b'revsetalias')
     stages = [
         (b'parsed', lambda tree: tree),
         (
             b'expanded',
             lambda tree: revsetlang.expandaliases(tree, aliases, ui.warn),
         ),
         (b'concatenated', revsetlang.foldconcat),
         (b'analyzed', revsetlang.analyze),
         (b'optimized', revsetlang.optimize),
     ]
     if opts[b'no_optimized']:
         stages = stages[:-1]
     if opts[b'verify_optimized'] and opts[b'no_optimized']:
         raise error.Abort(
             _(b'cannot use --verify-optimized with --no-optimized')
         )
     stagenames = {n for n, f in stages}
 
     showalways = set()
     showchanged = set()
     if ui.verbose and not opts[b'show_stage']:
         # show parsed tree by --verbose (deprecated)
         showalways.add(b'parsed')
         showchanged.update([b'expanded', b'concatenated'])
         if opts[b'optimize']:
             showalways.add(b'optimized')
     if opts[b'show_stage'] and opts[b'optimize']:
         raise error.Abort(_(b'cannot use --optimize with --show-stage'))
     if opts[b'show_stage'] == [b'all']:
         showalways.update(stagenames)
     else:
         for n in opts[b'show_stage']:
             if n not in stagenames:
                 raise error.Abort(_(b'invalid stage name: %s') % n)
         showalways.update(opts[b'show_stage'])
 
     treebystage = {}
     printedtree = None
     tree = revsetlang.parse(expr, lookup=revset.lookupfn(repo))
     for n, f in stages:
         treebystage[n] = tree = f(tree)
         if n in showalways or (n in showchanged and tree != printedtree):
             if opts[b'show_stage'] or n != b'parsed':
                 ui.write(b"* %s:\n" % n)
             ui.write(revsetlang.prettyformat(tree), b"\n")
             printedtree = tree
 
     if opts[b'verify_optimized']:
         arevs = revset.makematcher(treebystage[b'analyzed'])(repo)
         brevs = revset.makematcher(treebystage[b'optimized'])(repo)
         if opts[b'show_set'] or (opts[b'show_set'] is None and ui.verbose):
             ui.writenoi18n(
                 b"* analyzed set:\n", stringutil.prettyrepr(arevs), b"\n"
             )
             ui.writenoi18n(
                 b"* optimized set:\n", stringutil.prettyrepr(brevs), b"\n"
             )
         arevs = list(arevs)
         brevs = list(brevs)
         if arevs == brevs:
             return 0
         ui.writenoi18n(b'--- analyzed\n', label=b'diff.file_a')
         ui.writenoi18n(b'+++ optimized\n', label=b'diff.file_b')
         sm = difflib.SequenceMatcher(None, arevs, brevs)
         for tag, alo, ahi, blo, bhi in sm.get_opcodes():
             if tag in ('delete', 'replace'):
                 for c in arevs[alo:ahi]:
                     ui.write(b'-%d\n' % c, label=b'diff.deleted')
             if tag in ('insert', 'replace'):
                 for c in brevs[blo:bhi]:
                     ui.write(b'+%d\n' % c, label=b'diff.inserted')
             if tag == 'equal':
                 for c in arevs[alo:ahi]:
                     ui.write(b' %d\n' % c)
         return 1
 
     func = revset.makematcher(tree)
     revs = func(repo)
     if opts[b'show_set'] or (opts[b'show_set'] is None and ui.verbose):
         ui.writenoi18n(b"* set:\n", stringutil.prettyrepr(revs), b"\n")
     if not opts[b'show_revs']:
         return
     for c in revs:
         ui.write(b"%d\n" % c)
 
 
 @command(
     b'debugserve',
     [
         (
             b'',
             b'sshstdio',
             False,
             _(b'run an SSH server bound to process handles'),
         ),
         (b'', b'logiofd', b'', _(b'file descriptor to log server I/O to')),
         (b'', b'logiofile', b'', _(b'file to log server I/O to')),
     ],
     b'',
 )
 def debugserve(ui, repo, **opts):
     """run a server with advanced settings
 
     This command is similar to :hg:`serve`. It exists partially as a
     workaround to the fact that ``hg serve --stdio`` must have specific
     arguments for security reasons.
     """
     opts = pycompat.byteskwargs(opts)
 
     if not opts[b'sshstdio']:
         raise error.Abort(_(b'only --sshstdio is currently supported'))
 
     logfh = None
 
     if opts[b'logiofd'] and opts[b'logiofile']:
         raise error.Abort(_(b'cannot use both --logiofd and --logiofile'))
 
     if opts[b'logiofd']:
         # Ideally we would be line buffered. But line buffering in binary
         # mode isn't supported and emits a warning in Python 3.8+. Disabling
         # buffering could have performance impacts. But since this isn't
         # performance critical code, it should be fine.
         try:
             logfh = os.fdopen(int(opts[b'logiofd']), 'ab', 0)
         except OSError as e:
             if e.errno != errno.ESPIPE:
                 raise
             # can't seek a pipe, so `ab` mode fails on py3
             logfh = os.fdopen(int(opts[b'logiofd']), 'wb', 0)
     elif opts[b'logiofile']:
         logfh = open(opts[b'logiofile'], b'ab', 0)
 
     s = wireprotoserver.sshserver(ui, repo, logfh=logfh)
     s.serve_forever()
 
 
 @command(b'debugsetparents', [], _(b'REV1 [REV2]'))
 def debugsetparents(ui, repo, rev1, rev2=None):
     """manually set the parents of the current working directory (DANGEROUS)
 
     This command is not what you are looking for and should not be used. Using
     this command will most certainly results in slight corruption of the file
     level histories withing your repository. DO NOT USE THIS COMMAND.
 
     The command update the p1 and p2 field in the dirstate, and not touching
     anything else. This useful for writing repository conversion tools, but
     should be used with extreme care. For example, neither the working
     directory nor the dirstate is updated, so file status may be incorrect
     after running this command. Only used if you are one of the few people that
     deeply unstand both conversion tools and file level histories. If you are
     reading this help, you are not one of this people (most of them sailed west
     from Mithlond anyway.
 
     So one last time DO NOT USE THIS COMMAND.
 
     Returns 0 on success.
     """
 
     node1 = scmutil.revsingle(repo, rev1).node()
     node2 = scmutil.revsingle(repo, rev2, b'null').node()
 
     with repo.wlock():
         repo.setparents(node1, node2)
 
 
 @command(b'debugsidedata', cmdutil.debugrevlogopts, _(b'-c|-m|FILE REV'))
 def debugsidedata(ui, repo, file_, rev=None, **opts):
     """dump the side data for a cl/manifest/file revision
 
     Use --verbose to dump the sidedata content."""
     opts = pycompat.byteskwargs(opts)
     if opts.get(b'changelog') or opts.get(b'manifest') or opts.get(b'dir'):
         if rev is not None:
             raise error.CommandError(b'debugdata', _(b'invalid arguments'))
         file_, rev = None, file_
     elif rev is None:
         raise error.CommandError(b'debugdata', _(b'invalid arguments'))
     r = cmdutil.openstorage(repo, b'debugdata', file_, opts)
     r = getattr(r, '_revlog', r)
     try:
         sidedata = r.sidedata(r.lookup(rev))
     except KeyError:
         raise error.Abort(_(b'invalid revision identifier %s') % rev)
     if sidedata:
         sidedata = list(sidedata.items())
         sidedata.sort()
         ui.writenoi18n(b'%d sidedata entries\n' % len(sidedata))
         for key, value in sidedata:
             ui.writenoi18n(b' entry-%04o size %d\n' % (key, len(value)))
             if ui.verbose:
                 ui.writenoi18n(b'  %s\n' % stringutil.pprint(value))
 
 
 @command(b'debugssl', [], b'[SOURCE]', optionalrepo=True)
 def debugssl(ui, repo, source=None, **opts):
     """test a secure connection to a server
 
     This builds the certificate chain for the server on Windows, installing the
     missing intermediates and trusted root via Windows Update if necessary.  It
     does nothing on other platforms.
 
     If SOURCE is omitted, the 'default' path will be used.  If a URL is given,
     that server is used. See :hg:`help urls` for more information.
 
     If the update succeeds, retry the original operation.  Otherwise, the cause
     of the SSL error is likely another issue.
     """
     if not pycompat.iswindows:
         raise error.Abort(
             _(b'certificate chain building is only possible on Windows')
         )
 
     if not source:
         if not repo:
             raise error.Abort(
                 _(
                     b"there is no Mercurial repository here, and no "
                     b"server specified"
                 )
             )
         source = b"default"
 
     source, branches = hg.parseurl(ui.expandpath(source))
     url = util.url(source)
 
     defaultport = {b'https': 443, b'ssh': 22}
     if url.scheme in defaultport:
         try:
             addr = (url.host, int(url.port or defaultport[url.scheme]))
         except ValueError:
             raise error.Abort(_(b"malformed port number in URL"))
     else:
         raise error.Abort(_(b"only https and ssh connections are supported"))
 
     from . import win32
 
     s = ssl.wrap_socket(
         socket.socket(),
         ssl_version=ssl.PROTOCOL_TLS,
         cert_reqs=ssl.CERT_NONE,
         ca_certs=None,
     )
 
     try:
         s.connect(addr)
         cert = s.getpeercert(True)
 
         ui.status(_(b'checking the certificate chain for %s\n') % url.host)
 
         complete = win32.checkcertificatechain(cert, build=False)
 
         if not complete:
             ui.status(_(b'certificate chain is incomplete, updating... '))
 
             if not win32.checkcertificatechain(cert):
                 ui.status(_(b'failed.\n'))
             else:
                 ui.status(_(b'done.\n'))
         else:
             ui.status(_(b'full certificate chain is available\n'))
     finally:
         s.close()
 
 
 @command(
     b"debugbackupbundle",
     [
         (
             b"",
             b"recover",
             b"",
             b"brings the specified changeset back into the repository",
         )
     ]
     + cmdutil.logopts,
     _(b"hg debugbackupbundle [--recover HASH]"),
 )
 def debugbackupbundle(ui, repo, *pats, **opts):
     """lists the changesets available in backup bundles
 
     Without any arguments, this command prints a list of the changesets in each
     backup bundle.
 
     --recover takes a changeset hash and unbundles the first bundle that
     contains that hash, which puts that changeset back in your repository.
 
     --verbose will print the entire commit message and the bundle path for that
     backup.
     """
     backups = list(
         filter(
             os.path.isfile, glob.glob(repo.vfs.join(b"strip-backup") + b"/*.hg")
         )
     )
     backups.sort(key=lambda x: os.path.getmtime(x), reverse=True)
 
     opts = pycompat.byteskwargs(opts)
     opts[b"bundle"] = b""
     opts[b"force"] = None
     limit = logcmdutil.getlimit(opts)
 
     def display(other, chlist, displayer):
         if opts.get(b"newest_first"):
             chlist.reverse()
         count = 0
         for n in chlist:
             if limit is not None and count >= limit:
                 break
             parents = [True for p in other.changelog.parents(n) if p != nullid]
             if opts.get(b"no_merges") and len(parents) == 2:
                 continue
             count += 1
             displayer.show(other[n])
 
     recovernode = opts.get(b"recover")
     if recovernode:
         if scmutil.isrevsymbol(repo, recovernode):
             ui.warn(_(b"%s already exists in the repo\n") % recovernode)
             return
     elif backups:
         msg = _(
             b"Recover changesets using: hg debugbackupbundle --recover "
             b"<changeset hash>\n\nAvailable backup changesets:"
         )
         ui.status(msg, label=b"status.removed")
     else:
         ui.status(_(b"no backup changesets found\n"))
         return
 
     for backup in backups:
         # Much of this is copied from the hg incoming logic
         source = ui.expandpath(os.path.relpath(backup, encoding.getcwd()))
         source, branches = hg.parseurl(source, opts.get(b"branch"))
         try:
             other = hg.peer(repo, opts, source)
         except error.LookupError as ex:
             msg = _(b"\nwarning: unable to open bundle %s") % source
             hint = _(b"\n(missing parent rev %s)\n") % short(ex.name)
             ui.warn(msg, hint=hint)
             continue
         revs, checkout = hg.addbranchrevs(
             repo, other, branches, opts.get(b"rev")
         )
 
         if revs:
             revs = [other.lookup(rev) for rev in revs]
 
         quiet = ui.quiet
         try:
             ui.quiet = True
             other, chlist, cleanupfn = bundlerepo.getremotechanges(
                 ui, repo, other, revs, opts[b"bundle"], opts[b"force"]
             )
         except error.LookupError:
             continue
         finally:
             ui.quiet = quiet
 
         try:
             if not chlist:
                 continue
             if recovernode:
                 with repo.lock(), repo.transaction(b"unbundle") as tr:
                     if scmutil.isrevsymbol(other, recovernode):
                         ui.status(_(b"Unbundling %s\n") % (recovernode))
                         f = hg.openpath(ui, source)
                         gen = exchange.readbundle(ui, f, source)
                         if isinstance(gen, bundle2.unbundle20):
                             bundle2.applybundle(
                                 repo,
                                 gen,
                                 tr,
                                 source=b"unbundle",
                                 url=b"bundle:" + source,
                             )
                         else:
                             gen.apply(repo, b"unbundle", b"bundle:" + source)
                         break
             else:
                 backupdate = encoding.strtolocal(
                     time.strftime(
                         "%a %H:%M, %Y-%m-%d",
                         time.localtime(os.path.getmtime(source)),
                     )
                 )
                 ui.status(b"\n%s\n" % (backupdate.ljust(50)))
                 if ui.verbose:
                     ui.status(b"%s%s\n" % (b"bundle:".ljust(13), source))
                 else:
                     opts[
                         b"template"
                     ] = b"{label('status.modified', node|short)} {desc|firstline}\n"
                 displayer = logcmdutil.changesetdisplayer(
                     ui, other, opts, False
                 )
                 display(other, chlist, displayer)
                 displayer.close()
         finally:
             cleanupfn()
 
 
 @command(
     b'debugsub',
     [(b'r', b'rev', b'', _(b'revision to check'), _(b'REV'))],
     _(b'[-r REV] [REV]'),
 )
 def debugsub(ui, repo, rev=None):
     ctx = scmutil.revsingle(repo, rev, None)
     for k, v in sorted(ctx.substate.items()):
         ui.writenoi18n(b'path %s\n' % k)
         ui.writenoi18n(b' source   %s\n' % v[0])
         ui.writenoi18n(b' revision %s\n' % v[1])
 
 
 @command(b'debugshell', optionalrepo=True)
 def debugshell(ui, repo):
     """run an interactive Python interpreter
 
     The local namespace is provided with a reference to the ui and
     the repo instance (if available).
     """
     import code
 
     imported_objects = {
         'ui': ui,
         'repo': repo,
     }
 
     code.interact(local=imported_objects)
 
 
 @command(
     b'debugsuccessorssets',
     [(b'', b'closest', False, _(b'return closest successors sets only'))],
     _(b'[REV]'),
 )
 def debugsuccessorssets(ui, repo, *revs, **opts):
     """show set of successors for revision
 
     A successors set of changeset A is a consistent group of revisions that
     succeed A. It contains non-obsolete changesets only unless closests
     successors set is set.
 
     In most cases a changeset A has a single successors set containing a single
     successor (changeset A replaced by A').
 
     A changeset that is made obsolete with no successors are called "pruned".
     Such changesets have no successors sets at all.
 
     A changeset that has been "split" will have a successors set containing
     more than one successor.
 
     A changeset that has been rewritten in multiple different ways is called
     "divergent". Such changesets have multiple successor sets (each of which
     may also be split, i.e. have multiple successors).
 
     Results are displayed as follows::
 
         <rev1>
             <successors-1A>
         <rev2>
             <successors-2A>
             <successors-2B1> <successors-2B2> <successors-2B3>
 
     Here rev2 has two possible (i.e. divergent) successors sets. The first
     holds one element, whereas the second holds three (i.e. the changeset has
     been split).
     """
     # passed to successorssets caching computation from one call to another
     cache = {}
     ctx2str = bytes
     node2str = short
     for rev in scmutil.revrange(repo, revs):
         ctx = repo[rev]
         ui.write(b'%s\n' % ctx2str(ctx))
         for succsset in obsutil.successorssets(
             repo, ctx.node(), closest=opts['closest'], cache=cache
         ):
             if succsset:
                 ui.write(b'    ')
                 ui.write(node2str(succsset[0]))
                 for node in succsset[1:]:
                     ui.write(b' ')
                     ui.write(node2str(node))
             ui.write(b'\n')
 
 
 @command(b'debugtagscache', [])
 def debugtagscache(ui, repo):
     """display the contents of .hg/cache/hgtagsfnodes1"""
     cache = tagsmod.hgtagsfnodescache(repo.unfiltered())
     for r in repo:
         node = repo[r].node()
         tagsnode = cache.getfnode(node, computemissing=False)
-        tagsnodedisplay = hex(tagsnode) if tagsnode else b'missing/invalid'
+        if tagsnode:
+            tagsnodedisplay = hex(tagsnode)
+        elif tagsnode is False:
+            tagsnodedisplay = b'invalid'
+        else:
+            tagsnodedisplay = b'missing'
+
         ui.write(b'%d %s %s\n' % (r, hex(node), tagsnodedisplay))
 
 
 @command(
     b'debugtemplate',
     [
         (b'r', b'rev', [], _(b'apply template on changesets'), _(b'REV')),
         (b'D', b'define', [], _(b'define template keyword'), _(b'KEY=VALUE')),
     ],
     _(b'[-r REV]... [-D KEY=VALUE]... TEMPLATE'),
     optionalrepo=True,
 )
 def debugtemplate(ui, repo, tmpl, **opts):
     """parse and apply a template
 
     If -r/--rev is given, the template is processed as a log template and
     applied to the given changesets. Otherwise, it is processed as a generic
     template.
 
     Use --verbose to print the parsed tree.
     """
     revs = None
     if opts['rev']:
         if repo is None:
             raise error.RepoError(
                 _(b'there is no Mercurial repository here (.hg not found)')
             )
         revs = scmutil.revrange(repo, opts['rev'])
 
     props = {}
     for d in opts['define']:
         try:
             k, v = (e.strip() for e in d.split(b'=', 1))
             if not k or k == b'ui':
                 raise ValueError
             props[k] = v
         except ValueError:
             raise error.Abort(_(b'malformed keyword definition: %s') % d)
 
     if ui.verbose:
         aliases = ui.configitems(b'templatealias')
         tree = templater.parse(tmpl)
         ui.note(templater.prettyformat(tree), b'\n')
         newtree = templater.expandaliases(tree, aliases)
         if newtree != tree:
             ui.notenoi18n(
                 b"* expanded:\n", templater.prettyformat(newtree), b'\n'
             )
 
     if revs is None:
         tres = formatter.templateresources(ui, repo)
         t = formatter.maketemplater(ui, tmpl, resources=tres)
         if ui.verbose:
             kwds, funcs = t.symbolsuseddefault()
             ui.writenoi18n(b"* keywords: %s\n" % b', '.join(sorted(kwds)))
             ui.writenoi18n(b"* functions: %s\n" % b', '.join(sorted(funcs)))
         ui.write(t.renderdefault(props))
     else:
         displayer = logcmdutil.maketemplater(ui, repo, tmpl)
         if ui.verbose:
             kwds, funcs = displayer.t.symbolsuseddefault()
             ui.writenoi18n(b"* keywords: %s\n" % b', '.join(sorted(kwds)))
             ui.writenoi18n(b"* functions: %s\n" % b', '.join(sorted(funcs)))
         for r in revs:
             displayer.show(repo[r], **pycompat.strkwargs(props))
         displayer.close()
 
 
 @command(
     b'debuguigetpass',
     [
         (b'p', b'prompt', b'', _(b'prompt text'), _(b'TEXT')),
     ],
     _(b'[-p TEXT]'),
     norepo=True,
 )
 def debuguigetpass(ui, prompt=b''):
     """show prompt to type password"""
     r = ui.getpass(prompt)
     if r is None:
         r = b"<default response>"
     ui.writenoi18n(b'response: %s\n' % r)
 
 
 @command(
     b'debuguiprompt',
     [
         (b'p', b'prompt', b'', _(b'prompt text'), _(b'TEXT')),
     ],
     _(b'[-p TEXT]'),
     norepo=True,
 )
 def debuguiprompt(ui, prompt=b''):
     """show plain prompt"""
     r = ui.prompt(prompt)
     ui.writenoi18n(b'response: %s\n' % r)
 
 
 @command(b'debugupdatecaches', [])
 def debugupdatecaches(ui, repo, *pats, **opts):
     """warm all known caches in the repository"""
     with repo.wlock(), repo.lock():
         repo.updatecaches(full=True)
 
 
 @command(
     b'debugupgraderepo',
     [
         (
             b'o',
             b'optimize',
             [],
             _(b'extra optimization to perform'),
             _(b'NAME'),
         ),
         (b'', b'run', False, _(b'performs an upgrade')),
         (b'', b'backup', True, _(b'keep the old repository content around')),
         (b'', b'changelog', None, _(b'select the changelog for upgrade')),
         (b'', b'manifest', None, _(b'select the manifest for upgrade')),
         (b'', b'filelogs', None, _(b'select all filelogs for upgrade')),
     ],
 )
 def debugupgraderepo(ui, repo, run=False, optimize=None, backup=True, **opts):
     """upgrade a repository to use different features
 
     If no arguments are specified, the repository is evaluated for upgrade
     and a list of problems and potential optimizations is printed.
 
     With ``--run``, a repository upgrade is performed. Behavior of the upgrade
     can be influenced via additional arguments. More details will be provided
     by the command output when run without ``--run``.
 
     During the upgrade, the repository will be locked and no writes will be
     allowed.
 
     At the end of the upgrade, the repository may not be readable while new
     repository data is swapped in. This window will be as long as it takes to
     rename some directories inside the ``.hg`` directory. On most machines, this
     should complete almost instantaneously and the chances of a consumer being
     unable to access the repository should be low.
 
     By default, all revlog will be upgraded. You can restrict this using flag
     such as `--manifest`:
 
       * `--manifest`: only optimize the manifest
       * `--no-manifest`: optimize all revlog but the manifest
       * `--changelog`: optimize the changelog only
       * `--no-changelog --no-manifest`: optimize filelogs only
       * `--filelogs`: optimize the filelogs only
       * `--no-changelog --no-manifest --no-filelogs`: skip all revlog optimizations
     """
     return upgrade.upgraderepo(
         ui, repo, run=run, optimize=set(optimize), backup=backup, **opts
     )
 
 
 @command(
     b'debugwalk', cmdutil.walkopts, _(b'[OPTION]... [FILE]...'), inferrepo=True
 )
 def debugwalk(ui, repo, *pats, **opts):
     """show how files match on given patterns"""
     opts = pycompat.byteskwargs(opts)
     m = scmutil.match(repo[None], pats, opts)
     if ui.verbose:
         ui.writenoi18n(b'* matcher:\n', stringutil.prettyrepr(m), b'\n')
     items = list(repo[None].walk(m))
     if not items:
         return
     f = lambda fn: fn
     if ui.configbool(b'ui', b'slash') and pycompat.ossep != b'/':
         f = lambda fn: util.normpath(fn)
     fmt = b'f  %%-%ds  %%-%ds  %%s' % (
         max([len(abs) for abs in items]),
         max([len(repo.pathto(abs)) for abs in items]),
     )
     for abs in items:
         line = fmt % (
             abs,
             f(repo.pathto(abs)),
             m.exact(abs) and b'exact' or b'',
         )
         ui.write(b"%s\n" % line.rstrip())
 
 
 @command(b'debugwhyunstable', [], _(b'REV'))
 def debugwhyunstable(ui, repo, rev):
     """explain instabilities of a changeset"""
     for entry in obsutil.whyunstable(repo, scmutil.revsingle(repo, rev)):
         dnodes = b''
         if entry.get(b'divergentnodes'):
             dnodes = (
                 b' '.join(
                     b'%s (%s)' % (ctx.hex(), ctx.phasestr())
                     for ctx in entry[b'divergentnodes']
                 )
                 + b' '
             )
         ui.write(
             b'%s: %s%s %s\n'
             % (entry[b'instability'], dnodes, entry[b'reason'], entry[b'node'])
         )
 
 
 @command(
     b'debugwireargs',
     [
         (b'', b'three', b'', b'three'),
         (b'', b'four', b'', b'four'),
         (b'', b'five', b'', b'five'),
     ]
     + cmdutil.remoteopts,
     _(b'REPO [OPTIONS]... [ONE [TWO]]'),
     norepo=True,
 )
 def debugwireargs(ui, repopath, *vals, **opts):
     opts = pycompat.byteskwargs(opts)
     repo = hg.peer(ui, opts, repopath)
     for opt in cmdutil.remoteopts:
         del opts[opt[1]]
     args = {}
     for k, v in pycompat.iteritems(opts):
         if v:
             args[k] = v
     args = pycompat.strkwargs(args)
     # run twice to check that we don't mess up the stream for the next command
     res1 = repo.debugwireargs(*vals, **args)
     res2 = repo.debugwireargs(*vals, **args)
     ui.write(b"%s\n" % res1)
     if res1 != res2:
         ui.warn(b"%s\n" % res2)
 
 
 def _parsewirelangblocks(fh):
     activeaction = None
     blocklines = []
     lastindent = 0
 
     for line in fh:
         line = line.rstrip()
         if not line:
             continue
 
         if line.startswith(b'#'):
             continue
 
         if not line.startswith(b' '):
             # New block. Flush previous one.
             if activeaction:
                 yield activeaction, blocklines
 
             activeaction = line
             blocklines = []
             lastindent = 0
             continue
 
         # Else we start with an indent.
 
         if not activeaction:
             raise error.Abort(_(b'indented line outside of block'))
 
         indent = len(line) - len(line.lstrip())
 
         # If this line is indented more than the last line, concatenate it.
         if indent > lastindent and blocklines:
             blocklines[-1] += line.lstrip()
         else:
             blocklines.append(line)
             lastindent = indent
 
     # Flush last block.
     if activeaction:
         yield activeaction, blocklines
 
 
 @command(
     b'debugwireproto',
     [
         (b'', b'localssh', False, _(b'start an SSH server for this repo')),
         (b'', b'peer', b'', _(b'construct a specific version of the peer')),
         (
             b'',
             b'noreadstderr',
             False,
             _(b'do not read from stderr of the remote'),
         ),
         (
             b'',
             b'nologhandshake',
             False,
             _(b'do not log I/O related to the peer handshake'),
         ),
     ]
     + cmdutil.remoteopts,
     _(b'[PATH]'),
     optionalrepo=True,
 )
 def debugwireproto(ui, repo, path=None, **opts):
     """send wire protocol commands to a server
 
     This command can be used to issue wire protocol commands to remote
     peers and to debug the raw data being exchanged.
 
     ``--localssh`` will start an SSH server against the current repository
     and connect to that. By default, the connection will perform a handshake
     and establish an appropriate peer instance.
 
     ``--peer`` can be used to bypass the handshake protocol and construct a
     peer instance using the specified class type. Valid values are ``raw``,
     ``http2``, ``ssh1``, and ``ssh2``. ``raw`` instances only allow sending
     raw data payloads and don't support higher-level command actions.
 
     ``--noreadstderr`` can be used to disable automatic reading from stderr
     of the peer (for SSH connections only). Disabling automatic reading of
     stderr is useful for making output more deterministic.
 
     Commands are issued via a mini language which is specified via stdin.
     The language consists of individual actions to perform. An action is
     defined by a block. A block is defined as a line with no leading
     space followed by 0 or more lines with leading space. Blocks are
     effectively a high-level command with additional metadata.
 
     Lines beginning with ``#`` are ignored.
 
     The following sections denote available actions.
 
     raw
     ---
 
     Send raw data to the server.
 
     The block payload contains the raw data to send as one atomic send
     operation. The data may not actually be delivered in a single system
     call: it depends on the abilities of the transport being used.
 
     Each line in the block is de-indented and concatenated. Then, that
     value is evaluated as a Python b'' literal. This allows the use of
     backslash escaping, etc.
 
     raw+
     ----
 
     Behaves like ``raw`` except flushes output afterwards.
 
     command <X>
     -----------
 
     Send a request to run a named command, whose name follows the ``command``
     string.
 
     Arguments to the command are defined as lines in this block. The format of
     each line is ``<key> <value>``. e.g.::
 
        command listkeys
            namespace bookmarks
 
     If the value begins with ``eval:``, it will be interpreted as a Python
     literal expression. Otherwise values are interpreted as Python b'' literals.
     This allows sending complex types and encoding special byte sequences via
     backslash escaping.
 
     The following arguments have special meaning:
 
     ``PUSHFILE``
         When defined, the *push* mechanism of the peer will be used instead
         of the static request-response mechanism and the content of the
         file specified in the value of this argument will be sent as the
         command payload.
 
         This can be used to submit a local bundle file to the remote.
 
     batchbegin
     ----------
 
     Instruct the peer to begin a batched send.
 
     All ``command`` blocks are queued for execution until the next
     ``batchsubmit`` block.
 
     batchsubmit
     -----------
 
     Submit previously queued ``command`` blocks as a batch request.
 
     This action MUST be paired with a ``batchbegin`` action.
 
     httprequest <method> <path>
     ---------------------------
 
     (HTTP peer only)
 
     Send an HTTP request to the peer.
 
     The HTTP request line follows the ``httprequest`` action. e.g. ``GET /foo``.
 
     Arguments of the form ``<key>: <value>`` are interpreted as HTTP request
     headers to add to the request. e.g. ``Accept: foo``.
 
     The following arguments are special:
 
     ``BODYFILE``
         The content of the file defined as the value to this argument will be
         transferred verbatim as the HTTP request body.
 
     ``frame <type> <flags> <payload>``
         Send a unified protocol frame as part of the request body.
 
         All frames will be collected and sent as the body to the HTTP
         request.
 
     close
     -----
 
     Close the connection to the server.
 
     flush
     -----
 
     Flush data written to the server.
 
     readavailable
     -------------
 
     Close the write end of the connection and read all available data from
     the server.
 
     If the connection to the server encompasses multiple pipes, we poll both
     pipes and read available data.
 
     readline
     --------
 
     Read a line of output from the server. If there are multiple output
     pipes, reads only the main pipe.
 
     ereadline
     ---------
 
     Like ``readline``, but read from the stderr pipe, if available.
 
     read <X>
     --------
 
     ``read()`` N bytes from the server's main output pipe.
 
     eread <X>
     ---------
 
     ``read()`` N bytes from the server's stderr pipe, if available.
 
     Specifying Unified Frame-Based Protocol Frames
     ----------------------------------------------
 
     It is possible to emit a *Unified Frame-Based Protocol* by using special
     syntax.
 
     A frame is composed as a type, flags, and payload. These can be parsed
     from a string of the form:
 
        <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
 
     ``request-id`` and ``stream-id`` are integers defining the request and
     stream identifiers.
 
     ``type`` can be an integer value for the frame type or the string name
     of the type. The strings are defined in ``wireprotoframing.py``. e.g.
     ``command-name``.
 
     ``stream-flags`` and ``flags`` are a ``|`` delimited list of flag
     components. Each component (and there can be just one) can be an integer
     or a flag name for stream flags or frame flags, respectively. Values are
     resolved to integers and then bitwise OR'd together.
 
     ``payload`` represents the raw frame payload. If it begins with
     ``cbor:``, the following string is evaluated as Python code and the
     resulting object is fed into a CBOR encoder. Otherwise it is interpreted
     as a Python byte string literal.
     """
     opts = pycompat.byteskwargs(opts)
 
     if opts[b'localssh'] and not repo:
         raise error.Abort(_(b'--localssh requires a repository'))
 
     if opts[b'peer'] and opts[b'peer'] not in (
         b'raw',
         b'http2',
         b'ssh1',
         b'ssh2',
     ):
         raise error.Abort(
             _(b'invalid value for --peer'),
             hint=_(b'valid values are "raw", "ssh1", and "ssh2"'),
         )
 
     if path and opts[b'localssh']:
         raise error.Abort(_(b'cannot specify --localssh with an explicit path'))
 
     if ui.interactive():
         ui.write(_(b'(waiting for commands on stdin)\n'))
 
     blocks = list(_parsewirelangblocks(ui.fin))
 
     proc = None
     stdin = None
     stdout = None
     stderr = None
     opener = None
 
     if opts[b'localssh']:
         # We start the SSH server in its own process so there is process
         # separation. This prevents a whole class of potential bugs around
         # shared state from interfering with server operation.
         args = procutil.hgcmd() + [
             b'-R',
             repo.root,
             b'debugserve',
             b'--sshstdio',
         ]
         proc = subprocess.Popen(
             pycompat.rapply(procutil.tonativestr, args),
             stdin=subprocess.PIPE,
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
             bufsize=0,
         )
 
         stdin = proc.stdin
         stdout = proc.stdout
         stderr = proc.stderr
 
         # We turn the pipes into observers so we can log I/O.
         if ui.verbose or opts[b'peer'] == b'raw':
             stdin = util.makeloggingfileobject(
                 ui, proc.stdin, b'i', logdata=True
             )
             stdout = util.makeloggingfileobject(
                 ui, proc.stdout, b'o', logdata=True
             )
             stderr = util.makeloggingfileobject(
                 ui, proc.stderr, b'e', logdata=True
             )
 
         # --localssh also implies the peer connection settings.
 
         url = b'ssh://localserver'
         autoreadstderr = not opts[b'noreadstderr']
 
         if opts[b'peer'] == b'ssh1':
             ui.write(_(b'creating ssh peer for wire protocol version 1\n'))
             peer = sshpeer.sshv1peer(
                 ui,
                 url,
                 proc,
                 stdin,
                 stdout,
                 stderr,
                 None,
                 autoreadstderr=autoreadstderr,
             )
         elif opts[b'peer'] == b'ssh2':
             ui.write(_(b'creating ssh peer for wire protocol version 2\n'))
             peer = sshpeer.sshv2peer(
                 ui,
                 url,
                 proc,
                 stdin,
                 stdout,
                 stderr,
                 None,
                 autoreadstderr=autoreadstderr,
             )
         elif opts[b'peer'] == b'raw':
             ui.write(_(b'using raw connection to peer\n'))
             peer = None
         else:
             ui.write(_(b'creating ssh peer from handshake results\n'))
             peer = sshpeer.makepeer(
                 ui,
                 url,
                 proc,
                 stdin,
                 stdout,
                 stderr,
                 autoreadstderr=autoreadstderr,
             )
 
     elif path:
         # We bypass hg.peer() so we can proxy the sockets.
         # TODO consider not doing this because we skip
         # ``hg.wirepeersetupfuncs`` and potentially other useful functionality.
         u = util.url(path)
         if u.scheme != b'http':
             raise error.Abort(_(b'only http:// paths are currently supported'))
 
         url, authinfo = u.authinfo()
         openerargs = {
             'useragent': b'Mercurial debugwireproto',
         }
 
         # Turn pipes/sockets into observers so we can log I/O.
         if ui.verbose:
             openerargs.update(
                 {
                     'loggingfh': ui,
                     'loggingname': b's',
                     'loggingopts': {
                         'logdata': True,
                         'logdataapis': False,
                     },
                 }
             )
 
         if ui.debugflag:
             openerargs['loggingopts']['logdataapis'] = True
 
         # Don't send default headers when in raw mode. This allows us to
         # bypass most of the behavior of our URL handling code so we can
         # have near complete control over what's sent on the wire.
         if opts[b'peer'] == b'raw':
             openerargs['sendaccept'] = False
 
         opener = urlmod.opener(ui, authinfo, **openerargs)
 
         if opts[b'peer'] == b'http2':
             ui.write(_(b'creating http peer for wire protocol version 2\n'))
             # We go through makepeer() because we need an API descriptor for
             # the peer instance to be useful.
             with ui.configoverride(
                 {(b'experimental', b'httppeer.advertise-v2'): True}
             ):
                 if opts[b'nologhandshake']:
                     ui.pushbuffer()
 
                 peer = httppeer.makepeer(ui, path, opener=opener)
 
                 if opts[b'nologhandshake']:
                     ui.popbuffer()
 
             if not isinstance(peer, httppeer.httpv2peer):
                 raise error.Abort(
                     _(
                         b'could not instantiate HTTP peer for '
                         b'wire protocol version 2'
                     ),
                     hint=_(
                         b'the server may not have the feature '
                         b'enabled or is not allowing this '
                         b'client version'
                     ),
                 )
 
         elif opts[b'peer'] == b'raw':
             ui.write(_(b'using raw connection to peer\n'))
             peer = None
         elif opts[b'peer']:
             raise error.Abort(
                 _(b'--peer %s not supported with HTTP peers') % opts[b'peer']
             )
         else:
             peer = httppeer.makepeer(ui, path, opener=opener)
 
         # We /could/ populate stdin/stdout with sock.makefile()...
     else:
         raise error.Abort(_(b'unsupported connection configuration'))
 
     batchedcommands = None
 
     # Now perform actions based on the parsed wire language instructions.
     for action, lines in blocks:
         if action in (b'raw', b'raw+'):
             if not stdin:
                 raise error.Abort(_(b'cannot call raw/raw+ on this peer'))
 
             # Concatenate the data together.
             data = b''.join(l.lstrip() for l in lines)
             data = stringutil.unescapestr(data)
             stdin.write(data)
 
             if action == b'raw+':
                 stdin.flush()
         elif action == b'flush':
             if not stdin:
                 raise error.Abort(_(b'cannot call flush on this peer'))
             stdin.flush()
         elif action.startswith(b'command'):
             if not peer:
                 raise error.Abort(
                     _(
                         b'cannot send commands unless peer instance '
                         b'is available'
                     )
                 )
 
             command = action.split(b' ', 1)[1]
 
             args = {}
             for line in lines:
                 # We need to allow empty values.
                 fields = line.lstrip().split(b' ', 1)
                 if len(fields) == 1:
                     key = fields[0]
                     value = b''
                 else:
                     key, value = fields
 
                 if value.startswith(b'eval:'):
                     value = stringutil.evalpythonliteral(value[5:])
                 else:
                     value = stringutil.unescapestr(value)
 
                 args[key] = value
 
             if batchedcommands is not None:
                 batchedcommands.append((command, args))
                 continue
 
             ui.status(_(b'sending %s command\n') % command)
 
             if b'PUSHFILE' in args:
                 with open(args[b'PUSHFILE'], 'rb') as fh:
                     del args[b'PUSHFILE']
                     res, output = peer._callpush(
                         command, fh, **pycompat.strkwargs(args)
                     )
                     ui.status(_(b'result: %s\n') % stringutil.escapestr(res))
                     ui.status(
                         _(b'remote output: %s\n') % stringutil.escapestr(output)
                     )
             else:
                 with peer.commandexecutor() as e:
                     res = e.callcommand(command, args).result()
 
                 if isinstance(res, wireprotov2peer.commandresponse):
                     val = res.objects()
                     ui.status(
                         _(b'response: %s\n')
                         % stringutil.pprint(val, bprefix=True, indent=2)
                     )
                 else:
                     ui.status(
                         _(b'response: %s\n')
                         % stringutil.pprint(res, bprefix=True, indent=2)
                     )
 
         elif action == b'batchbegin':
             if batchedcommands is not None:
                 raise error.Abort(_(b'nested batchbegin not allowed'))
 
             batchedcommands = []
         elif action == b'batchsubmit':
             # There is a batching API we could go through. But it would be
             # difficult to normalize requests into function calls. It is easier
             # to bypass this layer and normalize to commands + args.
             ui.status(
                 _(b'sending batch with %d sub-commands\n')
                 % len(batchedcommands)
             )
             assert peer is not None
             for i, chunk in enumerate(peer._submitbatch(batchedcommands)):
                 ui.status(
                     _(b'response #%d: %s\n') % (i, stringutil.escapestr(chunk))
                 )
 
             batchedcommands = None
 
         elif action.startswith(b'httprequest '):
             if not opener:
                 raise error.Abort(
                     _(b'cannot use httprequest without an HTTP peer')
                 )
 
             request = action.split(b' ', 2)
             if len(request) != 3:
                 raise error.Abort(
                     _(
                         b'invalid httprequest: expected format is '
                         b'"httprequest <method> <path>'
                     )
                 )
 
             method, httppath = request[1:]
             headers = {}
             body = None
             frames = []
             for line in lines:
                 line = line.lstrip()
                 m = re.match(b'^([a-zA-Z0-9_-]+): (.*)$', line)
                 if m:
                     # Headers need to use native strings.
                     key = pycompat.strurl(m.group(1))
                     value = pycompat.strurl(m.group(2))
                     headers[key] = value
                     continue
 
                 if line.startswith(b'BODYFILE '):
                     with open(line.split(b' ', 1), b'rb') as fh:
                         body = fh.read()
                 elif line.startswith(b'frame '):
                     frame = wireprotoframing.makeframefromhumanstring(
                         line[len(b'frame ') :]
                     )
 
                     frames.append(frame)
                 else:
                     raise error.Abort(
                         _(b'unknown argument to httprequest: %s') % line
                     )
 
             url = path + httppath
 
             if frames:
                 body = b''.join(bytes(f) for f in frames)
 
             req = urlmod.urlreq.request(pycompat.strurl(url), body, headers)
 
             # urllib.Request insists on using has_data() as a proxy for
             # determining the request method. Override that to use our
             # explicitly requested method.
             req.get_method = lambda: pycompat.sysstr(method)
 
             try:
                 res = opener.open(req)
                 body = res.read()
             except util.urlerr.urlerror as e:
                 # read() method must be called, but only exists in Python 2
                 getattr(e, 'read', lambda: None)()
                 continue
 
             ct = res.headers.get('Content-Type')
             if ct == 'application/mercurial-cbor':
                 ui.write(
                     _(b'cbor> %s\n')
                     % stringutil.pprint(
                         cborutil.decodeall(body), bprefix=True, indent=2
                     )
                 )
 
         elif action == b'close':
             assert peer is not None
             peer.close()
         elif action == b'readavailable':
             if not stdout or not stderr:
                 raise error.Abort(
                     _(b'readavailable not available on this peer')
                 )
 
             stdin.close()
             stdout.read()
             stderr.read()
 
         elif action == b'readline':
             if not stdout:
                 raise error.Abort(_(b'readline not available on this peer'))
             stdout.readline()
         elif action == b'ereadline':
             if not stderr:
                 raise error.Abort(_(b'ereadline not available on this peer'))
             stderr.readline()
         elif action.startswith(b'read '):
             count = int(action.split(b' ', 1)[1])
             if not stdout:
                 raise error.Abort(_(b'read not available on this peer'))
             stdout.read(count)
         elif action.startswith(b'eread '):
             count = int(action.split(b' ', 1)[1])
             if not stderr:
                 raise error.Abort(_(b'eread not available on this peer'))
             stderr.read(count)
         else:
             raise error.Abort(_(b'unknown action: %s') % action)
 
     if batchedcommands is not None:
         raise error.Abort(_(b'unclosed "batchbegin" request'))
 
     if peer:
         peer.close()
 
     if proc:
         proc.kill()

mercurial/tags.py

@@ -1,876 +1,880 b''
 # tags.py - read tag info from local repository
 #
 # Copyright 2009 Matt Mackall <[e-mail]>
 # Copyright 2009 Greg Ward <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 # Currently this module only deals with reading and caching tags.
 # Eventually, it could take care of updating (adding/removing/moving)
 # tags too.
 
 from __future__ import absolute_import
 
 import errno
 import io
 
 from .node import (
     bin,
     hex,
     nullid,
     nullrev,
     short,
 )
 from .i18n import _
 from . import (
     encoding,
     error,
     match as matchmod,
     pycompat,
     scmutil,
     util,
 )
 from .utils import stringutil
 
 # Tags computation can be expensive and caches exist to make it fast in
 # the common case.
 #
 # The "hgtagsfnodes1" cache file caches the .hgtags filenode values for
 # each revision in the repository. The file is effectively an array of
 # fixed length records. Read the docs for "hgtagsfnodescache" for technical
 # details.
 #
 # The .hgtags filenode cache grows in proportion to the length of the
 # changelog. The file is truncated when the # changelog is stripped.
 #
 # The purpose of the filenode cache is to avoid the most expensive part
 # of finding global tags, which is looking up the .hgtags filenode in the
 # manifest for each head. This can take dozens or over 100ms for
 # repositories with very large manifests. Multiplied by dozens or even
 # hundreds of heads and there is a significant performance concern.
 #
 # There also exist a separate cache file for each repository filter.
 # These "tags-*" files store information about the history of tags.
 #
 # The tags cache files consists of a cache validation line followed by
 # a history of tags.
 #
 # The cache validation line has the format:
 #
 #   <tiprev> <tipnode> [<filteredhash>]
 #
 # <tiprev> is an integer revision and <tipnode> is a 40 character hex
 # node for that changeset. These redundantly identify the repository
 # tip from the time the cache was written. In addition, <filteredhash>,
 # if present, is a 40 character hex hash of the contents of the filtered
 # revisions for this filter. If the set of filtered revs changes, the
 # hash will change and invalidate the cache.
 #
 # The history part of the tags cache consists of lines of the form:
 #
 #   <node> <tag>
 #
 # (This format is identical to that of .hgtags files.)
 #
 # <tag> is the tag name and <node> is the 40 character hex changeset
 # the tag is associated with.
 #
 # Tags are written sorted by tag name.
 #
 # Tags associated with multiple changesets have an entry for each changeset.
 # The most recent changeset (in terms of revlog ordering for the head
 # setting it) for each tag is last.
 
 
 def fnoderevs(ui, repo, revs):
     """return the list of '.hgtags' fnodes used in a set revisions
 
     This is returned as list of unique fnodes. We use a list instead of a set
     because order matters when it comes to tags."""
     unfi = repo.unfiltered()
     tonode = unfi.changelog.node
     nodes = [tonode(r) for r in revs]
     fnodes = _getfnodes(ui, repo, nodes)
     fnodes = _filterfnodes(fnodes, nodes)
     return fnodes
 
 
 def _nulltonone(value):
     """convert nullid to None
 
     For tag value, nullid means "deleted". This small utility function helps
     translating that to None."""
     if value == nullid:
         return None
     return value
 
 
 def difftags(ui, repo, oldfnodes, newfnodes):
     """list differences between tags expressed in two set of file-nodes
 
     The list contains entries in the form: (tagname, oldvalue, new value).
     None is used to expressed missing value:
         ('foo', None, 'abcd') is a new tag,
         ('bar', 'ef01', None) is a deletion,
         ('baz', 'abcd', 'ef01') is a tag movement.
     """
     if oldfnodes == newfnodes:
         return []
     oldtags = _tagsfromfnodes(ui, repo, oldfnodes)
     newtags = _tagsfromfnodes(ui, repo, newfnodes)
 
     # list of (tag, old, new): None means missing
     entries = []
     for tag, (new, __) in newtags.items():
         new = _nulltonone(new)
         old, __ = oldtags.pop(tag, (None, None))
         old = _nulltonone(old)
         if old != new:
             entries.append((tag, old, new))
     # handle deleted tags
     for tag, (old, __) in oldtags.items():
         old = _nulltonone(old)
         if old is not None:
             entries.append((tag, old, None))
     entries.sort()
     return entries
 
 
 def writediff(fp, difflist):
     """write tags diff information to a file.
 
     Data are stored with a line based format:
 
         <action> <hex-node> <tag-name>\n
 
     Action are defined as follow:
        -R tag is removed,
        +A tag is added,
        -M tag is moved (old value),
        +M tag is moved (new value),
 
     Example:
 
          +A 875517b4806a848f942811a315a5bce30804ae85 t5
 
     See documentation of difftags output for details about the input.
     """
     add = b'+A %s %s\n'
     remove = b'-R %s %s\n'
     updateold = b'-M %s %s\n'
     updatenew = b'+M %s %s\n'
     for tag, old, new in difflist:
         # translate to hex
         if old is not None:
             old = hex(old)
         if new is not None:
             new = hex(new)
         # write to file
         if old is None:
             fp.write(add % (new, tag))
         elif new is None:
             fp.write(remove % (old, tag))
         else:
             fp.write(updateold % (old, tag))
             fp.write(updatenew % (new, tag))
 
 
 def findglobaltags(ui, repo):
     """Find global tags in a repo: return a tagsmap
 
     tagsmap: tag name to (node, hist) 2-tuples.
 
     The tags cache is read and updated as a side-effect of calling.
     """
     (heads, tagfnode, valid, cachetags, shouldwrite) = _readtagcache(ui, repo)
     if cachetags is not None:
         assert not shouldwrite
         # XXX is this really 100% correct?  are there oddball special
         # cases where a global tag should outrank a local tag but won't,
         # because cachetags does not contain rank info?
         alltags = {}
         _updatetags(cachetags, alltags)
         return alltags
 
     for head in reversed(heads):  # oldest to newest
         assert repo.changelog.index.has_node(
             head
         ), b"tag cache returned bogus head %s" % short(head)
     fnodes = _filterfnodes(tagfnode, reversed(heads))
     alltags = _tagsfromfnodes(ui, repo, fnodes)
 
     # and update the cache (if necessary)
     if shouldwrite:
         _writetagcache(ui, repo, valid, alltags)
     return alltags
 
 
 def _filterfnodes(tagfnode, nodes):
     """return a list of unique fnodes
 
     The order of this list matches the order of "nodes". Preserving this order
     is important as reading tags in different order provides different
     results."""
     seen = set()  # set of fnode
     fnodes = []
     for no in nodes:  # oldest to newest
         fnode = tagfnode.get(no)
         if fnode and fnode not in seen:
             seen.add(fnode)
             fnodes.append(fnode)
     return fnodes
 
 
 def _tagsfromfnodes(ui, repo, fnodes):
     """return a tagsmap from a list of file-node
 
     tagsmap: tag name to (node, hist) 2-tuples.
 
     The order of the list matters."""
     alltags = {}
     fctx = None
     for fnode in fnodes:
         if fctx is None:
             fctx = repo.filectx(b'.hgtags', fileid=fnode)
         else:
             fctx = fctx.filectx(fnode)
         filetags = _readtags(ui, repo, fctx.data().splitlines(), fctx)
         _updatetags(filetags, alltags)
     return alltags
 
 
 def readlocaltags(ui, repo, alltags, tagtypes):
     '''Read local tags in repo. Update alltags and tagtypes.'''
     try:
         data = repo.vfs.read(b"localtags")
     except IOError as inst:
         if inst.errno != errno.ENOENT:
             raise
         return
 
     # localtags is in the local encoding; re-encode to UTF-8 on
     # input for consistency with the rest of this module.
     filetags = _readtags(
         ui, repo, data.splitlines(), b"localtags", recode=encoding.fromlocal
     )
 
     # remove tags pointing to invalid nodes
     cl = repo.changelog
     for t in list(filetags):
         try:
             cl.rev(filetags[t][0])
         except (LookupError, ValueError):
             del filetags[t]
 
     _updatetags(filetags, alltags, b'local', tagtypes)
 
 
 def _readtaghist(ui, repo, lines, fn, recode=None, calcnodelines=False):
     """Read tag definitions from a file (or any source of lines).
 
     This function returns two sortdicts with similar information:
 
     - the first dict, bintaghist, contains the tag information as expected by
       the _readtags function, i.e. a mapping from tag name to (node, hist):
         - node is the node id from the last line read for that name,
         - hist is the list of node ids previously associated with it (in file
           order). All node ids are binary, not hex.
 
     - the second dict, hextaglines, is a mapping from tag name to a list of
       [hexnode, line number] pairs, ordered from the oldest to the newest node.
 
     When calcnodelines is False the hextaglines dict is not calculated (an
     empty dict is returned). This is done to improve this function's
     performance in cases where the line numbers are not needed.
     """
 
     bintaghist = util.sortdict()
     hextaglines = util.sortdict()
     count = 0
 
     def dbg(msg):
         ui.debug(b"%s, line %d: %s\n" % (fn, count, msg))
 
     for nline, line in enumerate(lines):
         count += 1
         if not line:
             continue
         try:
             (nodehex, name) = line.split(b" ", 1)
         except ValueError:
             dbg(b"cannot parse entry")
             continue
         name = name.strip()
         if recode:
             name = recode(name)
         try:
             nodebin = bin(nodehex)
         except TypeError:
             dbg(b"node '%s' is not well formed" % nodehex)
             continue
 
         # update filetags
         if calcnodelines:
             # map tag name to a list of line numbers
             if name not in hextaglines:
                 hextaglines[name] = []
             hextaglines[name].append([nodehex, nline])
             continue
         # map tag name to (node, hist)
         if name not in bintaghist:
             bintaghist[name] = []
         bintaghist[name].append(nodebin)
     return bintaghist, hextaglines
 
 
 def _readtags(ui, repo, lines, fn, recode=None, calcnodelines=False):
     """Read tag definitions from a file (or any source of lines).
 
     Returns a mapping from tag name to (node, hist).
 
     "node" is the node id from the last line read for that name. "hist"
     is the list of node ids previously associated with it (in file order).
     All node ids are binary, not hex.
     """
     filetags, nodelines = _readtaghist(
         ui, repo, lines, fn, recode=recode, calcnodelines=calcnodelines
     )
     # util.sortdict().__setitem__ is much slower at replacing then inserting
     # new entries. The difference can matter if there are thousands of tags.
     # Create a new sortdict to avoid the performance penalty.
     newtags = util.sortdict()
     for tag, taghist in filetags.items():
         newtags[tag] = (taghist[-1], taghist[:-1])
     return newtags
 
 
 def _updatetags(filetags, alltags, tagtype=None, tagtypes=None):
     """Incorporate the tag info read from one file into dictionnaries
 
     The first one, 'alltags', is a "tagmaps" (see 'findglobaltags' for details).
 
     The second one, 'tagtypes', is optional and will be updated to track the
     "tagtype" of entries in the tagmaps. When set, the 'tagtype' argument also
     needs to be set."""
     if tagtype is None:
         assert tagtypes is None
 
     for name, nodehist in pycompat.iteritems(filetags):
         if name not in alltags:
             alltags[name] = nodehist
             if tagtype is not None:
                 tagtypes[name] = tagtype
             continue
 
         # we prefer alltags[name] if:
         #  it supersedes us OR
         #  mutual supersedes and it has a higher rank
         # otherwise we win because we're tip-most
         anode, ahist = nodehist
         bnode, bhist = alltags[name]
         if (
             bnode != anode
             and anode in bhist
             and (bnode not in ahist or len(bhist) > len(ahist))
         ):
             anode = bnode
         elif tagtype is not None:
             tagtypes[name] = tagtype
         ahist.extend([n for n in bhist if n not in ahist])
         alltags[name] = anode, ahist
 
 
 def _filename(repo):
     """name of a tagcache file for a given repo or repoview"""
     filename = b'tags2'
     if repo.filtername:
         filename = b'%s-%s' % (filename, repo.filtername)
     return filename
 
 
 def _readtagcache(ui, repo):
     """Read the tag cache.
 
     Returns a tuple (heads, fnodes, validinfo, cachetags, shouldwrite).
 
     If the cache is completely up-to-date, "cachetags" is a dict of the
     form returned by _readtags() and "heads", "fnodes", and "validinfo" are
     None and "shouldwrite" is False.
 
     If the cache is not up to date, "cachetags" is None. "heads" is a list
     of all heads currently in the repository, ordered from tip to oldest.
     "validinfo" is a tuple describing cache validation info. This is used
     when writing the tags cache. "fnodes" is a mapping from head to .hgtags
     filenode. "shouldwrite" is True.
 
     If the cache is not up to date, the caller is responsible for reading tag
     info from each returned head. (See findglobaltags().)
     """
     try:
         cachefile = repo.cachevfs(_filename(repo), b'r')
         # force reading the file for static-http
         cachelines = iter(cachefile)
     except IOError:
         cachefile = None
 
     cacherev = None
     cachenode = None
     cachehash = None
     if cachefile:
         try:
             validline = next(cachelines)
             validline = validline.split()
             cacherev = int(validline[0])
             cachenode = bin(validline[1])
             if len(validline) > 2:
                 cachehash = bin(validline[2])
         except Exception:
             # corruption of the cache, just recompute it.
             pass
 
     tipnode = repo.changelog.tip()
     tiprev = len(repo.changelog) - 1
 
     # Case 1 (common): tip is the same, so nothing has changed.
     # (Unchanged tip trivially means no changesets have been added.
     # But, thanks to localrepository.destroyed(), it also means none
     # have been destroyed by strip or rollback.)
     if (
         cacherev == tiprev
         and cachenode == tipnode
         and cachehash == scmutil.filteredhash(repo, tiprev)
     ):
         tags = _readtags(ui, repo, cachelines, cachefile.name)
         cachefile.close()
         return (None, None, None, tags, False)
     if cachefile:
         cachefile.close()  # ignore rest of file
 
     valid = (tiprev, tipnode, scmutil.filteredhash(repo, tiprev))
 
     repoheads = repo.heads()
     # Case 2 (uncommon): empty repo; get out quickly and don't bother
     # writing an empty cache.
     if repoheads == [nullid]:
         return ([], {}, valid, {}, False)
 
     # Case 3 (uncommon): cache file missing or empty.
 
     # Case 4 (uncommon): tip rev decreased.  This should only happen
     # when we're called from localrepository.destroyed().  Refresh the
     # cache so future invocations will not see disappeared heads in the
     # cache.
 
     # Case 5 (common): tip has changed, so we've added/replaced heads.
 
     # As it happens, the code to handle cases 3, 4, 5 is the same.
 
     # N.B. in case 4 (nodes destroyed), "new head" really means "newly
     # exposed".
     if not len(repo.file(b'.hgtags')):
         # No tags have ever been committed, so we can avoid a
         # potentially expensive search.
         return ([], {}, valid, None, True)
 
     # Now we have to lookup the .hgtags filenode for every new head.
     # This is the most expensive part of finding tags, so performance
     # depends primarily on the size of newheads.  Worst case: no cache
     # file, so newheads == repoheads.
     # Reversed order helps the cache ('repoheads' is in descending order)
     cachefnode = _getfnodes(ui, repo, reversed(repoheads))
 
     # Caller has to iterate over all heads, but can use the filenodes in
     # cachefnode to get to each .hgtags revision quickly.
     return (repoheads, cachefnode, valid, None, True)
 
 
 def _getfnodes(ui, repo, nodes):
     """return .hgtags fnodes for a list of changeset nodes
 
     Return value is a {node: fnode} mapping. There will be no entry for nodes
     without a '.hgtags' file.
     """
     starttime = util.timer()
     fnodescache = hgtagsfnodescache(repo.unfiltered())
     cachefnode = {}
     for node in nodes:
         fnode = fnodescache.getfnode(node)
         if fnode != nullid:
             cachefnode[node] = fnode
 
     fnodescache.write()
 
     duration = util.timer() - starttime
     ui.log(
         b'tagscache',
         b'%d/%d cache hits/lookups in %0.4f seconds\n',
         fnodescache.hitcount,
         fnodescache.lookupcount,
         duration,
     )
     return cachefnode
 
 
 def _writetagcache(ui, repo, valid, cachetags):
     filename = _filename(repo)
     try:
         cachefile = repo.cachevfs(filename, b'w', atomictemp=True)
     except (OSError, IOError):
         return
 
     ui.log(
         b'tagscache',
         b'writing .hg/cache/%s with %d tags\n',
         filename,
         len(cachetags),
     )
 
     if valid[2]:
         cachefile.write(
             b'%d %s %s\n' % (valid[0], hex(valid[1]), hex(valid[2]))
         )
     else:
         cachefile.write(b'%d %s\n' % (valid[0], hex(valid[1])))
 
     # Tag names in the cache are in UTF-8 -- which is the whole reason
     # we keep them in UTF-8 throughout this module.  If we converted
     # them local encoding on input, we would lose info writing them to
     # the cache.
     for (name, (node, hist)) in sorted(pycompat.iteritems(cachetags)):
         for n in hist:
             cachefile.write(b"%s %s\n" % (hex(n), name))
         cachefile.write(b"%s %s\n" % (hex(node), name))
 
     try:
         cachefile.close()
     except (OSError, IOError):
         pass
 
 
 def tag(repo, names, node, message, local, user, date, editor=False):
     """tag a revision with one or more symbolic names.
 
     names is a list of strings or, when adding a single tag, names may be a
     string.
 
     if local is True, the tags are stored in a per-repository file.
     otherwise, they are stored in the .hgtags file, and a new
     changeset is committed with the change.
 
     keyword arguments:
 
     local: whether to store tags in non-version-controlled file
     (default False)
 
     message: commit message to use if committing
 
     user: name of user to use if committing
 
     date: date tuple to use if committing"""
 
     if not local:
         m = matchmod.exact([b'.hgtags'])
         st = repo.status(match=m, unknown=True, ignored=True)
         if any(
             (
                 st.modified,
                 st.added,
                 st.removed,
                 st.deleted,
                 st.unknown,
                 st.ignored,
             )
         ):
             raise error.Abort(
                 _(b'working copy of .hgtags is changed'),
                 hint=_(b'please commit .hgtags manually'),
             )
 
     with repo.wlock():
         repo.tags()  # instantiate the cache
         _tag(repo, names, node, message, local, user, date, editor=editor)
 
 
 def _tag(
     repo, names, node, message, local, user, date, extra=None, editor=False
 ):
     if isinstance(names, bytes):
         names = (names,)
 
     branches = repo.branchmap()
     for name in names:
         repo.hook(b'pretag', throw=True, node=hex(node), tag=name, local=local)
         if name in branches:
             repo.ui.warn(
                 _(b"warning: tag %s conflicts with existing branch name\n")
                 % name
             )
 
     def writetags(fp, names, munge, prevtags):
         fp.seek(0, io.SEEK_END)
         if prevtags and not prevtags.endswith(b'\n'):
             fp.write(b'\n')
         for name in names:
             if munge:
                 m = munge(name)
             else:
                 m = name
 
             if repo._tagscache.tagtypes and name in repo._tagscache.tagtypes:
                 old = repo.tags().get(name, nullid)
                 fp.write(b'%s %s\n' % (hex(old), m))
             fp.write(b'%s %s\n' % (hex(node), m))
         fp.close()
 
     prevtags = b''
     if local:
         try:
             fp = repo.vfs(b'localtags', b'r+')
         except IOError:
             fp = repo.vfs(b'localtags', b'a')
         else:
             prevtags = fp.read()
 
         # local tags are stored in the current charset
         writetags(fp, names, None, prevtags)
         for name in names:
             repo.hook(b'tag', node=hex(node), tag=name, local=local)
         return
 
     try:
         fp = repo.wvfs(b'.hgtags', b'rb+')
     except IOError as e:
         if e.errno != errno.ENOENT:
             raise
         fp = repo.wvfs(b'.hgtags', b'ab')
     else:
         prevtags = fp.read()
 
     # committed tags are stored in UTF-8
     writetags(fp, names, encoding.fromlocal, prevtags)
 
     fp.close()
 
     repo.invalidatecaches()
 
     if b'.hgtags' not in repo.dirstate:
         repo[None].add([b'.hgtags'])
 
     m = matchmod.exact([b'.hgtags'])
     tagnode = repo.commit(
         message, user, date, extra=extra, match=m, editor=editor
     )
 
     for name in names:
         repo.hook(b'tag', node=hex(node), tag=name, local=local)
 
     return tagnode
 
 
 _fnodescachefile = b'hgtagsfnodes1'
 _fnodesrecsize = 4 + 20  # changeset fragment + filenode
 _fnodesmissingrec = b'\xff' * 24
 
 
 class hgtagsfnodescache(object):
     """Persistent cache mapping revisions to .hgtags filenodes.
 
     The cache is an array of records. Each item in the array corresponds to
     a changelog revision. Values in the array contain the first 4 bytes of
     the node hash and the 20 bytes .hgtags filenode for that revision.
 
     The first 4 bytes are present as a form of verification. Repository
     stripping and rewriting may change the node at a numeric revision in the
     changelog. The changeset fragment serves as a verifier to detect
     rewriting. This logic is shared with the rev branch cache (see
     branchmap.py).
 
     The instance holds in memory the full cache content but entries are
     only parsed on read.
 
     Instances behave like lists. ``c[i]`` works where i is a rev or
     changeset node. Missing indexes are populated automatically on access.
     """
 
     def __init__(self, repo):
         assert repo.filtername is None
 
         self._repo = repo
 
         # Only for reporting purposes.
         self.lookupcount = 0
         self.hitcount = 0
 
         try:
             data = repo.cachevfs.read(_fnodescachefile)
         except (OSError, IOError):
             data = b""
         self._raw = bytearray(data)
 
         # The end state of self._raw is an array that is of the exact length
         # required to hold a record for every revision in the repository.
         # We truncate or extend the array as necessary. self._dirtyoffset is
         # defined to be the start offset at which we need to write the output
         # file. This offset is also adjusted when new entries are calculated
         # for array members.
         cllen = len(repo.changelog)
         wantedlen = cllen * _fnodesrecsize
         rawlen = len(self._raw)
 
         self._dirtyoffset = None
 
         rawlentokeep = min(
             wantedlen, (rawlen // _fnodesrecsize) * _fnodesrecsize
         )
         if rawlen > rawlentokeep:
             # There's no easy way to truncate array instances. This seems
             # slightly less evil than copying a potentially large array slice.
             for i in range(rawlen - rawlentokeep):
                 self._raw.pop()
             rawlen = len(self._raw)
             self._dirtyoffset = rawlen
         if rawlen < wantedlen:
             if self._dirtyoffset is None:
                 self._dirtyoffset = rawlen
+            # TODO: zero fill entire record, because it's invalid not missing?
             self._raw.extend(b'\xff' * (wantedlen - rawlen))
 
     def getfnode(self, node, computemissing=True):
         """Obtain the filenode of the .hgtags file at a specified revision.
 
         If the value is in the cache, the entry will be validated and returned.
         Otherwise, the filenode will be computed and returned unless
-        "computemissing" is False, in which case None will be returned without
+        "computemissing" is False.  In that case, None will be returned if
+        the entry is missing or False if the entry is invalid without
         any potentially expensive computation being performed.
 
         If an .hgtags does not exist at the specified revision, nullid is
         returned.
         """
         if node == nullid:
             return nullid
 
         ctx = self._repo[node]
         rev = ctx.rev()
 
         self.lookupcount += 1
 
         offset = rev * _fnodesrecsize
         record = b'%s' % self._raw[offset : offset + _fnodesrecsize]
         properprefix = node[0:4]
 
         # Validate and return existing entry.
         if record != _fnodesmissingrec and len(record) == _fnodesrecsize:
             fileprefix = record[0:4]
 
             if fileprefix == properprefix:
                 self.hitcount += 1
                 return record[4:]
 
             # Fall through.
 
         # If we get here, the entry is either missing or invalid.
 
         if not computemissing:
+            if record != _fnodesmissingrec:
+                return False
             return None
 
         fnode = None
         cl = self._repo.changelog
         p1rev, p2rev = cl._uncheckedparentrevs(rev)
         p1node = cl.node(p1rev)
         p1fnode = self.getfnode(p1node, computemissing=False)
         if p2rev != nullrev:
             # There is some no-merge changeset where p1 is null and p2 is set
             # Processing them as merge is just slower, but still gives a good
             # result.
             p2node = cl.node(p1rev)
             p2fnode = self.getfnode(p2node, computemissing=False)
             if p1fnode != p2fnode:
                 # we cannot rely on readfast because we don't know against what
                 # parent the readfast delta is computed
                 p1fnode = None
-        if p1fnode is not None:
+        if p1fnode:
             mctx = ctx.manifestctx()
             fnode = mctx.readfast().get(b'.hgtags')
             if fnode is None:
                 fnode = p1fnode
         if fnode is None:
             # Populate missing entry.
             try:
                 fnode = ctx.filenode(b'.hgtags')
             except error.LookupError:
                 # No .hgtags file on this revision.
                 fnode = nullid
 
         self._writeentry(offset, properprefix, fnode)
         return fnode
 
     def setfnode(self, node, fnode):
         """Set the .hgtags filenode for a given changeset."""
         assert len(fnode) == 20
         ctx = self._repo[node]
 
         # Do a lookup first to avoid writing if nothing has changed.
         if self.getfnode(ctx.node(), computemissing=False) == fnode:
             return
 
         self._writeentry(ctx.rev() * _fnodesrecsize, node[0:4], fnode)
 
     def _writeentry(self, offset, prefix, fnode):
         # Slices on array instances only accept other array.
         entry = bytearray(prefix + fnode)
         self._raw[offset : offset + _fnodesrecsize] = entry
         # self._dirtyoffset could be None.
         self._dirtyoffset = min(self._dirtyoffset or 0, offset or 0)
 
     def write(self):
         """Perform all necessary writes to cache file.
 
         This may no-op if no writes are needed or if a write lock could
         not be obtained.
         """
         if self._dirtyoffset is None:
             return
 
         data = self._raw[self._dirtyoffset :]
         if not data:
             return
 
         repo = self._repo
 
         try:
             lock = repo.lock(wait=False)
         except error.LockError:
             repo.ui.log(
                 b'tagscache',
                 b'not writing .hg/cache/%s because '
                 b'lock cannot be acquired\n' % _fnodescachefile,
             )
             return
 
         try:
             f = repo.cachevfs.open(_fnodescachefile, b'ab')
             try:
                 # if the file has been truncated
                 actualoffset = f.tell()
                 if actualoffset < self._dirtyoffset:
                     self._dirtyoffset = actualoffset
                     data = self._raw[self._dirtyoffset :]
                 f.seek(self._dirtyoffset)
                 f.truncate()
                 repo.ui.log(
                     b'tagscache',
                     b'writing %d bytes to cache/%s\n'
                     % (len(data), _fnodescachefile),
                 )
                 f.write(data)
                 self._dirtyoffset = None
             finally:
                 f.close()
         except (IOError, OSError) as inst:
             repo.ui.log(
                 b'tagscache',
                 b"couldn't write cache/%s: %s\n"
                 % (_fnodescachefile, stringutil.forcebytestr(inst)),
             )
         finally:
             lock.release()

tests/test-tags.t

@@ -1,864 +1,892 b''
 setup
 
   $ cat >> $HGRCPATH << EOF
   > [extensions]
   > blackbox=
   > mock=$TESTDIR/mockblackbox.py
   > [blackbox]
   > track = command, commandfinish, tagscache
   > EOF
 
 Helper functions:
 
   $ cacheexists() {
   >   [ -f .hg/cache/tags2-visible ] && echo "tag cache exists" || echo "no tag cache"
   > }
 
   $ fnodescacheexists() {
   >   [ -f .hg/cache/hgtagsfnodes1 ] && echo "fnodes cache exists" || echo "no fnodes cache"
   > }
 
   $ dumptags() {
   >     rev=$1
   >     echo "rev $rev: .hgtags:"
   >     hg cat -r$rev .hgtags
   > }
 
 # XXX need to test that the tag cache works when we strip an old head
 # and add a new one rooted off non-tip: i.e. node and rev of tip are the
 # same, but stuff has changed behind tip.
 
 Setup:
 
   $ hg init t
   $ cd t
   $ cacheexists
   no tag cache
   $ fnodescacheexists
   no fnodes cache
   $ hg id
   000000000000 tip
   $ cacheexists
   no tag cache
   $ fnodescacheexists
   no fnodes cache
   $ echo a > a
   $ hg add a
   $ hg commit -m "test"
   $ hg co
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg identify
   acb14030fe0a tip
   $ hg identify -r 'wdir()'
   acb14030fe0a tip
   $ cacheexists
   tag cache exists
 No fnodes cache because .hgtags file doesn't exist
 (this is an implementation detail)
   $ fnodescacheexists
   no fnodes cache
 
 Try corrupting the cache
 
   $ printf 'a b' > .hg/cache/tags2-visible
   $ hg identify
   acb14030fe0a tip
   $ cacheexists
   tag cache exists
   $ fnodescacheexists
   no fnodes cache
   $ hg identify
   acb14030fe0a tip
 
 Create local tag with long name:
 
   $ T=`hg identify --debug --id`
   $ hg tag -l "This is a local tag with a really long name!"
   $ hg tags
   tip                                0:acb14030fe0a
   This is a local tag with a really long name!     0:acb14030fe0a
   $ rm .hg/localtags
 
 Create a tag behind hg's back:
 
   $ echo "$T first" > .hgtags
   $ cat .hgtags
   acb14030fe0a21b60322c440ad2d20cf7685a376 first
   $ hg add .hgtags
   $ hg commit -m "add tags"
   $ hg tags
   tip                                1:b9154636be93
   first                              0:acb14030fe0a
   $ hg identify
   b9154636be93 tip
 
 We should have a fnodes cache now that we have a real tag
 The cache should have an empty entry for rev 0 and a valid entry for rev 1.
 
 
   $ fnodescacheexists
   fnodes cache exists
   $ f --size --hexdump .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=48
   0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0010: ff ff ff ff ff ff ff ff b9 15 46 36 26 b7 b4 a7 |..........F6&...|
   0020: 73 e0 9e e3 c5 2f 51 0e 19 e0 5e 1f f9 66 d8 59 |s..../Q...^..f.Y|
   $ hg debugtagscache
-  0 acb14030fe0a21b60322c440ad2d20cf7685a376 missing/invalid
+  0 acb14030fe0a21b60322c440ad2d20cf7685a376 missing
   1 b9154636be938d3d431e75a7c906504a079bfe07 26b7b4a773e09ee3c52f510e19e05e1ff966d859
 
 Repeat with cold tag cache:
 
   $ rm -f .hg/cache/tags2-visible .hg/cache/hgtagsfnodes1
   $ hg identify
   b9154636be93 tip
 
   $ fnodescacheexists
   fnodes cache exists
   $ f --size --hexdump .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=48
   0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0010: ff ff ff ff ff ff ff ff b9 15 46 36 26 b7 b4 a7 |..........F6&...|
   0020: 73 e0 9e e3 c5 2f 51 0e 19 e0 5e 1f f9 66 d8 59 |s..../Q...^..f.Y|
 
 And again, but now unable to write tag cache or lock file:
 
 #if unix-permissions no-fsmonitor
 
   $ rm -f .hg/cache/tags2-visible .hg/cache/hgtagsfnodes1
   $ chmod 555 .hg/cache
   $ hg identify
   b9154636be93 tip
   $ chmod 755 .hg/cache
 
 (this block should be protected by no-fsmonitor, because "chmod 555 .hg"
 makes watchman fail at accessing to files under .hg)
 
   $ chmod 555 .hg
   $ hg identify
   b9154636be93 tip
   $ chmod 755 .hg
 #endif
 
 Tag cache debug info written to blackbox log
 
   $ rm -f .hg/cache/tags2-visible .hg/cache/hgtagsfnodes1
   $ hg identify
   b9154636be93 tip
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> identify
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> writing 48 bytes to cache/hgtagsfnodes1
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> 0/2 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> identify exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> blackbox -l 6
 
 Failure to acquire lock results in no write
 
   $ rm -f .hg/cache/tags2-visible .hg/cache/hgtagsfnodes1
   $ echo 'foo:1' > .hg/store/lock
   $ hg identify
   b9154636be93 tip
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> identify
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> not writing .hg/cache/hgtagsfnodes1 because lock cannot be acquired
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> 0/2 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> identify exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @b9154636be938d3d431e75a7c906504a079bfe07 (5000)> blackbox -l 6
 
   $ fnodescacheexists
   no fnodes cache
 
   $ rm .hg/store/lock
 
   $ rm -f .hg/cache/tags2-visible .hg/cache/hgtagsfnodes1
   $ hg identify
   b9154636be93 tip
 
 Create a branch:
 
   $ echo bb > a
   $ hg status
   M a
   $ hg identify
   b9154636be93+ tip
   $ hg co first
   0 files updated, 0 files merged, 1 files removed, 0 files unresolved
   $ hg id
   acb14030fe0a+ first
   $ hg id -r 'wdir()'
   acb14030fe0a+ first
   $ hg -v id
   acb14030fe0a+ first
   $ hg status
   M a
   $ echo 1 > b
   $ hg add b
   $ hg commit -m "branch"
   created new head
 
 Creating a new commit shouldn't append the .hgtags fnodes cache until
 tags info is accessed
 
   $ f --size --hexdump .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=48
   0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0010: ff ff ff ff ff ff ff ff b9 15 46 36 26 b7 b4 a7 |..........F6&...|
   0020: 73 e0 9e e3 c5 2f 51 0e 19 e0 5e 1f f9 66 d8 59 |s..../Q...^..f.Y|
 
   $ hg id
   c8edf04160c7 tip
 
 First 4 bytes of record 3 are changeset fragment
 
   $ f --size --hexdump .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=72
   0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0010: ff ff ff ff ff ff ff ff b9 15 46 36 26 b7 b4 a7 |..........F6&...|
   0020: 73 e0 9e e3 c5 2f 51 0e 19 e0 5e 1f f9 66 d8 59 |s..../Q...^..f.Y|
   0030: c8 ed f0 41 00 00 00 00 00 00 00 00 00 00 00 00 |...A............|
   0040: 00 00 00 00 00 00 00 00                         |........|
 
 Merge the two heads:
 
   $ hg merge 1
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   (branch merge, don't forget to commit)
   $ hg blackbox -l3
   1970/01/01 00:00:00 bob @c8edf04160c7f731e4589d66ab3ab3486a64ac28 (5000)> merge 1
   1970/01/01 00:00:00 bob @c8edf04160c7f731e4589d66ab3ab3486a64ac28+b9154636be938d3d431e75a7c906504a079bfe07 (5000)> merge 1 exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @c8edf04160c7f731e4589d66ab3ab3486a64ac28+b9154636be938d3d431e75a7c906504a079bfe07 (5000)> blackbox -l3
   $ hg id
   c8edf04160c7+b9154636be93+ tip
   $ hg status
   M .hgtags
   $ hg commit -m "merge"
 
 Create a fake head, make sure tag not visible afterwards:
 
   $ cp .hgtags tags
   $ hg tag last
   $ hg rm .hgtags
   $ hg commit -m "remove"
 
   $ mv tags .hgtags
   $ hg add .hgtags
   $ hg commit -m "readd"
   $ 
   $ hg tags
   tip                                6:35ff301afafe
   first                              0:acb14030fe0a
 
 Add invalid tags:
 
   $ echo "spam" >> .hgtags
   $ echo >> .hgtags
   $ echo "foo bar" >> .hgtags
   $ echo "a5a5 invalid" >> .hg/localtags
   $ cat .hgtags
   acb14030fe0a21b60322c440ad2d20cf7685a376 first
   spam
   
   foo bar
   $ hg commit -m "tags"
 
 Report tag parse error on other head:
 
   $ hg up 3
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ echo 'x y' >> .hgtags
   $ hg commit -m "head"
   created new head
 
   $ hg tags --debug
   .hgtags@75d9f02dfe28, line 2: cannot parse entry
   .hgtags@75d9f02dfe28, line 4: node 'foo' is not well formed
   .hgtags@c4be69a18c11, line 2: node 'x' is not well formed
   tip                                8:c4be69a18c11e8bc3a5fdbb576017c25f7d84663
   first                              0:acb14030fe0a21b60322c440ad2d20cf7685a376
   $ hg tip
   changeset:   8:c4be69a18c11
   tag:         tip
   parent:      3:ac5e980c4dc0
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     head
   
 
 Test tag precedence rules:
 
   $ cd ..
   $ hg init t2
   $ cd t2
   $ echo foo > foo
   $ hg add foo
   $ hg ci -m 'add foo'      # rev 0
   $ hg tag bar              # rev 1
   $ echo >> foo
   $ hg ci -m 'change foo 1' # rev 2
   $ hg up -C 1
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg tag -r 1 -f bar      # rev 3
   $ hg up -C 1
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ echo >> foo
   $ hg ci -m 'change foo 2' # rev 4
   created new head
   $ hg tags
   tip                                4:0c192d7d5e6b
   bar                                1:78391a272241
 
 Repeat in case of cache effects:
 
   $ hg tags
   tip                                4:0c192d7d5e6b
   bar                                1:78391a272241
 
 Detailed dump of tag info:
 
   $ hg heads -q             # expect 4, 3, 2
   4:0c192d7d5e6b
   3:6fa450212aeb
   2:7a94127795a3
   $ dumptags 2
   rev 2: .hgtags:
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   $ dumptags 3
   rev 3: .hgtags:
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   78391a272241d70354aa14c874552cad6b51bb42 bar
   $ dumptags 4
   rev 4: .hgtags:
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
 
 Dump cache:
 
   $ cat .hg/cache/tags2-visible
   4 0c192d7d5e6b78a714de54a2e9627952a877e25a
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   78391a272241d70354aa14c874552cad6b51bb42 bar
 
   $ f --size --hexdump .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=120
   0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   0030: 7a 94 12 77 0c 04 f2 a8 af 31 de 17 fa b7 42 28 |z..w.....1....B(|
   0040: 78 ee 5a 2d ad bc 94 3d 6f a4 50 21 7d 3b 71 8c |x.Z-...=o.P!};q.|
   0050: 96 4e f3 7b 89 e5 50 eb da fd 57 89 e7 6c e1 b0 |.N.{..P...W..l..|
   0060: 0c 19 2d 7d 0c 04 f2 a8 af 31 de 17 fa b7 42 28 |..-}.....1....B(|
   0070: 78 ee 5a 2d ad bc 94 3d                         |x.Z-...=|
 
 Corrupt the .hgtags fnodes cache
 Extra junk data at the end should get overwritten on next cache update
 
   $ echo extra >> .hg/cache/hgtagsfnodes1
   $ echo dummy1 > foo
   $ hg commit -m throwaway1
 
   $ hg tags
   tip                                5:8dbfe60eff30
   bar                                1:78391a272241
 
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> tags
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> writing 24 bytes to cache/hgtagsfnodes1
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> 3/4 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> tags exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @8dbfe60eff306a54259cfe007db9e330e7ecf866 (5000)> blackbox -l 6
 
 On junk data + missing cache entries, hg also overwrites the junk.
 
   $ rm -f .hg/cache/tags2-visible
   >>> import os
   >>> with open(".hg/cache/hgtagsfnodes1", "ab+") as fp:
   ...     fp.seek(-10, os.SEEK_END) and None
   ...     fp.truncate() and None
 
   $ hg debugtagscache | tail -2
   4 0c192d7d5e6b78a714de54a2e9627952a877e25a 0c04f2a8af31de17fab7422878ee5a2dadbc943d
-  5 8dbfe60eff306a54259cfe007db9e330e7ecf866 missing/invalid
+  5 8dbfe60eff306a54259cfe007db9e330e7ecf866 missing
   $ hg tags
   tip                                5:8dbfe60eff30
   bar                                1:78391a272241
   $ hg debugtagscache | tail -2
   4 0c192d7d5e6b78a714de54a2e9627952a877e25a 0c04f2a8af31de17fab7422878ee5a2dadbc943d
   5 8dbfe60eff306a54259cfe007db9e330e7ecf866 0c04f2a8af31de17fab7422878ee5a2dadbc943d
 
+If the 4 bytes of node hash for a record don't match an existing node, the entry
+is flagged as invalid.
+
+  >>> import os
+  >>> with open(".hg/cache/hgtagsfnodes1", "rb+") as fp:
+  ...     fp.seek(-24, os.SEEK_END) and None
+  ...     fp.write(b'\xde\xad') and None
+
+  $ f --size --hexdump .hg/cache/hgtagsfnodes1
+  .hg/cache/hgtagsfnodes1: size=144
+  0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
+  0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
+  0020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
+  0030: 7a 94 12 77 0c 04 f2 a8 af 31 de 17 fa b7 42 28 |z..w.....1....B(|
+  0040: 78 ee 5a 2d ad bc 94 3d 6f a4 50 21 7d 3b 71 8c |x.Z-...=o.P!};q.|
+  0050: 96 4e f3 7b 89 e5 50 eb da fd 57 89 e7 6c e1 b0 |.N.{..P...W..l..|
+  0060: 0c 19 2d 7d 0c 04 f2 a8 af 31 de 17 fa b7 42 28 |..-}.....1....B(|
+  0070: 78 ee 5a 2d ad bc 94 3d de ad e6 0e 0c 04 f2 a8 |x.Z-...=........|
+  0080: af 31 de 17 fa b7 42 28 78 ee 5a 2d ad bc 94 3d |.1....B(x.Z-...=|
+
+  $ hg debugtagscache | tail -2
+  4 0c192d7d5e6b78a714de54a2e9627952a877e25a 0c04f2a8af31de17fab7422878ee5a2dadbc943d
+  5 8dbfe60eff306a54259cfe007db9e330e7ecf866 invalid
+
+  $ hg tags
+  tip                                5:8dbfe60eff30
+  bar                                1:78391a272241
+
 #if unix-permissions no-root
 Errors writing to .hgtags fnodes cache are silently ignored
 
   $ echo dummy2 > foo
   $ hg commit -m throwaway2
 
   $ chmod a-w .hg/cache/hgtagsfnodes1
   $ rm -f .hg/cache/tags2-visible
 
   $ hg tags
   tip                                6:b968051b5cf3
   bar                                1:78391a272241
 
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> tags
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> couldn't write cache/hgtagsfnodes1: [Errno *] * (glob)
-  1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> 3/4 cache hits/lookups in * seconds (glob)
+  1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> 2/4 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> tags exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> blackbox -l 6
 
   $ chmod a+w .hg/cache/hgtagsfnodes1
 
   $ rm -f .hg/cache/tags2-visible
   $ hg tags
   tip                                6:b968051b5cf3
   bar                                1:78391a272241
 
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> tags
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> writing 24 bytes to cache/hgtagsfnodes1
-  1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> 3/4 cache hits/lookups in * seconds (glob)
+  1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> 2/4 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> tags exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @b968051b5cf3f624b771779c6d5f84f1d4c3fb5d (5000)> blackbox -l 6
 
   $ f --size .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=168
 
   $ hg -q --config extensions.strip= strip -r 6 --no-backup
 #endif
 
 Stripping doesn't truncate the tags cache until new data is available
 
   $ rm -f .hg/cache/hgtagsfnodes1 .hg/cache/tags2-visible
   $ hg tags
   tip                                5:8dbfe60eff30
   bar                                1:78391a272241
 
   $ f --size .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=144
 
   $ hg -q --config extensions.strip= strip -r 5 --no-backup
   $ hg tags
   tip                                4:0c192d7d5e6b
   bar                                1:78391a272241
 
   $ hg blackbox -l 5
   1970/01/01 00:00:00 bob @0c192d7d5e6b78a714de54a2e9627952a877e25a (5000)> writing 24 bytes to cache/hgtagsfnodes1
   1970/01/01 00:00:00 bob @0c192d7d5e6b78a714de54a2e9627952a877e25a (5000)> 2/4 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @0c192d7d5e6b78a714de54a2e9627952a877e25a (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @0c192d7d5e6b78a714de54a2e9627952a877e25a (5000)> tags exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @0c192d7d5e6b78a714de54a2e9627952a877e25a (5000)> blackbox -l 5
 
   $ f --size .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=120
 
   $ echo dummy > foo
   $ hg commit -m throwaway3
 
   $ hg tags
   tip                                5:035f65efb448
   bar                                1:78391a272241
 
   $ hg blackbox -l 6
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> tags
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> writing 24 bytes to cache/hgtagsfnodes1
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> 3/4 cache hits/lookups in * seconds (glob)
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> writing .hg/cache/tags2-visible with 1 tags
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> tags exited 0 after * seconds (glob)
   1970/01/01 00:00:00 bob @035f65efb448350f4772141702a81ab1df48c465 (5000)> blackbox -l 6
   $ f --size .hg/cache/hgtagsfnodes1
   .hg/cache/hgtagsfnodes1: size=144
 
   $ hg -q --config extensions.strip= strip -r 5 --no-backup
 
 Test tag removal:
 
   $ hg tag --remove bar     # rev 5
   $ hg tip -vp
   changeset:   5:5f6e8655b1c7
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   files:       .hgtags
   description:
   Removed tag bar
   
   
   diff -r 0c192d7d5e6b -r 5f6e8655b1c7 .hgtags
   --- a/.hgtags	Thu Jan 01 00:00:00 1970 +0000
   +++ b/.hgtags	Thu Jan 01 00:00:00 1970 +0000
   @@ -1,1 +1,3 @@
    bbd179dfa0a71671c253b3ae0aa1513b60d199fa bar
   +78391a272241d70354aa14c874552cad6b51bb42 bar
   +0000000000000000000000000000000000000000 bar
   
   $ hg tags
   tip                                5:5f6e8655b1c7
   $ hg tags                 # again, try to expose cache bugs
   tip                                5:5f6e8655b1c7
 
 Remove nonexistent tag:
 
   $ hg tag --remove foobar
   abort: tag 'foobar' does not exist
   [10]
   $ hg tip
   changeset:   5:5f6e8655b1c7
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Removed tag bar
   
 
 Undo a tag with rollback:
 
   $ hg rollback             # destroy rev 5 (restore bar)
   repository tip rolled back to revision 4 (undo commit)
   working directory now based on revision 4
   $ hg tags
   tip                                4:0c192d7d5e6b
   bar                                1:78391a272241
   $ hg tags
   tip                                4:0c192d7d5e6b
   bar                                1:78391a272241
 
 Test tag rank:
 
   $ cd ..
   $ hg init t3
   $ cd t3
   $ echo foo > foo
   $ hg add foo
   $ hg ci -m 'add foo'       # rev 0
   $ hg tag -f bar            # rev 1 bar -> 0
   $ hg tag -f bar            # rev 2 bar -> 1
   $ hg tag -fr 0 bar         # rev 3 bar -> 0
   $ hg tag -fr 1 bar         # rev 4 bar -> 1
   $ hg tag -fr 0 bar         # rev 5 bar -> 0
   $ hg tags
   tip                                5:85f05169d91d
   bar                                0:bbd179dfa0a7
   $ hg co 3
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ echo barbar > foo
   $ hg ci -m 'change foo'    # rev 6
   created new head
   $ hg tags
   tip                                6:735c3ca72986
   bar                                0:bbd179dfa0a7
 
 Don't allow moving tag without -f:
 
   $ hg tag -r 3 bar
   abort: tag 'bar' already exists (use -f to force)
   [10]
   $ hg tags
   tip                                6:735c3ca72986
   bar                                0:bbd179dfa0a7
 
 Strip 1: expose an old head:
 
   $ hg --config extensions.mq= strip 5
   saved backup bundle to $TESTTMP/t3/.hg/strip-backup/*-backup.hg (glob)
   $ hg tags                  # partly stale cache
   tip                                5:735c3ca72986
   bar                                1:78391a272241
   $ hg tags                  # up-to-date cache
   tip                                5:735c3ca72986
   bar                                1:78391a272241
 
 Strip 2: destroy whole branch, no old head exposed
 
   $ hg --config extensions.mq= strip 4
   saved backup bundle to $TESTTMP/t3/.hg/strip-backup/*-backup.hg (glob)
   $ hg tags                  # partly stale
   tip                                4:735c3ca72986
   bar                                0:bbd179dfa0a7
   $ rm -f .hg/cache/tags2-visible
   $ hg tags                  # cold cache
   tip                                4:735c3ca72986
   bar                                0:bbd179dfa0a7
 
 Test tag rank with 3 heads:
 
   $ cd ..
   $ hg init t4
   $ cd t4
   $ echo foo > foo
   $ hg add
   adding foo
   $ hg ci -m 'add foo'                 # rev 0
   $ hg tag bar                         # rev 1 bar -> 0
   $ hg tag -f bar                      # rev 2 bar -> 1
   $ hg up -qC 0
   $ hg tag -fr 2 bar                   # rev 3 bar -> 2
   $ hg tags
   tip                                3:197c21bbbf2c
   bar                                2:6fa450212aeb
   $ hg up -qC 0
   $ hg tag -m 'retag rev 0' -fr 0 bar  # rev 4 bar -> 0, but bar stays at 2
 
 Bar should still point to rev 2:
 
   $ hg tags
   tip                                4:3b4b14ed0202
   bar                                2:6fa450212aeb
 
 Test that removing global/local tags does not get confused when trying
 to remove a tag of type X which actually only exists as a type Y:
 
   $ cd ..
   $ hg init t5
   $ cd t5
   $ echo foo > foo
   $ hg add
   adding foo
   $ hg ci -m 'add foo'                 # rev 0
 
   $ hg tag -r 0 -l localtag
   $ hg tag --remove localtag
   abort: tag 'localtag' is not a global tag
   [10]
   $ 
   $ hg tag -r 0 globaltag
   $ hg tag --remove -l globaltag
   abort: tag 'globaltag' is not a local tag
   [10]
   $ hg tags -v
   tip                                1:a0b6fe111088
   localtag                           0:bbd179dfa0a7 local
   globaltag                          0:bbd179dfa0a7
 
 Templated output:
 
  (immediate values)
 
   $ hg tags -T '{pad(tag, 9)} {rev}:{node} ({type})\n'
   tip       1:a0b6fe111088c8c29567d3876cc466aa02927cae ()
   localtag  0:bbd179dfa0a71671c253b3ae0aa1513b60d199fa (local)
   globaltag 0:bbd179dfa0a71671c253b3ae0aa1513b60d199fa ()
 
  (ctx/revcache dependent)
 
   $ hg tags -T '{pad(tag, 9)} {rev} {file_adds}\n'
   tip       1 .hgtags
   localtag  0 foo
   globaltag 0 foo
 
   $ hg tags -T '{pad(tag, 9)} {rev}:{node|shortest}\n'
   tip       1:a0b6
   localtag  0:bbd1
   globaltag 0:bbd1
 
 Test for issue3911
 
   $ hg tag -r 0 -l localtag2
   $ hg tag -l --remove localtag2
   $ hg tags -v
   tip                                1:a0b6fe111088
   localtag                           0:bbd179dfa0a7 local
   globaltag                          0:bbd179dfa0a7
 
   $ hg tag -r 1 -f localtag
   $ hg tags -v
   tip                                2:5c70a037bb37
   localtag                           1:a0b6fe111088
   globaltag                          0:bbd179dfa0a7
 
   $ hg tags -v
   tip                                2:5c70a037bb37
   localtag                           1:a0b6fe111088
   globaltag                          0:bbd179dfa0a7
 
   $ hg tag -r 1 localtag2
   $ hg tags -v
   tip                                3:bbfb8cd42be2
   localtag2                          1:a0b6fe111088
   localtag                           1:a0b6fe111088
   globaltag                          0:bbd179dfa0a7
 
   $ hg tags -v
   tip                                3:bbfb8cd42be2
   localtag2                          1:a0b6fe111088
   localtag                           1:a0b6fe111088
   globaltag                          0:bbd179dfa0a7
 
   $ cd ..
 
 Create a repository with tags data to test .hgtags fnodes transfer
 
   $ hg init tagsserver
   $ cd tagsserver
   $ touch foo
   $ hg -q commit -A -m initial
   $ hg tag -m 'tag 0.1' 0.1
   $ echo second > foo
   $ hg commit -m second
   $ hg tag -m 'tag 0.2' 0.2
   $ hg tags
   tip                                3:40f0358cb314
   0.2                                2:f63cc8fe54e4
   0.1                                0:96ee1d7354c4
   $ cd ..
 
 Cloning should pull down hgtags fnodes mappings and write the cache file
 
   $ hg clone --pull tagsserver tagsclient
   requesting all changes
   adding changesets
   adding manifests
   adding file changes
   added 4 changesets with 4 changes to 2 files
   new changesets 96ee1d7354c4:40f0358cb314
   updating to branch default
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
 Missing tags2* files means the cache wasn't written through the normal mechanism.
 
   $ ls tagsclient/.hg/cache
   branch2-base
   branch2-immutable
   branch2-served
   branch2-served.hidden
   branch2-visible
   branch2-visible-hidden
   hgtagsfnodes1
   rbc-names-v1
   rbc-revs-v1
   tags2
   tags2-served
 
 Cache should contain the head only, even though other nodes have tags data
 
   $ f --size --hexdump tagsclient/.hg/cache/hgtagsfnodes1
   tagsclient/.hg/cache/hgtagsfnodes1: size=96
   0000: 96 ee 1d 73 00 00 00 00 00 00 00 00 00 00 00 00 |...s............|
   0010: 00 00 00 00 00 00 00 00 c4 da b0 c2 94 65 e1 c6 |.............e..|
   0020: 0d f7 f0 dd 32 04 ea 57 78 c8 97 97 79 fc d5 95 |....2..Wx...y...|
   0030: f6 3c c8 fe 94 65 e1 c6 0d f7 f0 dd 32 04 ea 57 |.<...e......2..W|
   0040: 78 c8 97 97 79 fc d5 95 40 f0 35 8c 19 e0 a7 d3 |x...y...@.5.....|
   0050: 8a 5c 6a 82 4d cf fb a5 87 d0 2f a3 1e 4f 2f 8a |.\j.M...../..O/.|
 
 Running hg tags should produce tags2* file and not change cache
 
   $ hg -R tagsclient tags
   tip                                3:40f0358cb314
   0.2                                2:f63cc8fe54e4
   0.1                                0:96ee1d7354c4
 
   $ ls tagsclient/.hg/cache
   branch2-base
   branch2-immutable
   branch2-served
   branch2-served.hidden
   branch2-visible
   branch2-visible-hidden
   hgtagsfnodes1
   rbc-names-v1
   rbc-revs-v1
   tags2
   tags2-served
   tags2-visible
 
   $ f --size --hexdump tagsclient/.hg/cache/hgtagsfnodes1
   tagsclient/.hg/cache/hgtagsfnodes1: size=96
   0000: 96 ee 1d 73 00 00 00 00 00 00 00 00 00 00 00 00 |...s............|
   0010: 00 00 00 00 00 00 00 00 c4 da b0 c2 94 65 e1 c6 |.............e..|
   0020: 0d f7 f0 dd 32 04 ea 57 78 c8 97 97 79 fc d5 95 |....2..Wx...y...|
   0030: f6 3c c8 fe 94 65 e1 c6 0d f7 f0 dd 32 04 ea 57 |.<...e......2..W|
   0040: 78 c8 97 97 79 fc d5 95 40 f0 35 8c 19 e0 a7 d3 |x...y...@.5.....|
   0050: 8a 5c 6a 82 4d cf fb a5 87 d0 2f a3 1e 4f 2f 8a |.\j.M...../..O/.|
 
 Check that the bundle includes cache data
 
   $ hg -R tagsclient bundle --all ./test-cache-in-bundle-all-rev.hg
   4 changesets found
   $ hg debugbundle ./test-cache-in-bundle-all-rev.hg
   Stream params: {Compression: BZ}
   changegroup -- {nbchanges: 4, version: 02} (mandatory: True)
       96ee1d7354c4ad7372047672c36a1f561e3a6a4c
       c4dab0c2fd337eb9191f80c3024830a4889a8f34
       f63cc8fe54e4d326f8d692805d70e092f851ddb1
       40f0358cb314c824a5929ee527308d90e023bc10
   hgtagsfnodes -- {} (mandatory: True)
   cache:rev-branch-cache -- {} (mandatory: False)
 
 Check that local clone includes cache data
 
   $ hg clone tagsclient tags-local-clone
   updating to branch default
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ (cd tags-local-clone/.hg/cache/; ls -1 tag*)
   tags2
   tags2-served
   tags2-visible
 
 Avoid writing logs on trying to delete an already deleted tag
   $ hg init issue5752
   $ cd issue5752
   $ echo > a
   $ hg commit -Am 'add a'
   adding a
   $ hg tag a
   $ hg tags
   tip                                1:bd7ee4f3939b
   a                                  0:a8a82d372bb3
   $ hg log
   changeset:   1:bd7ee4f3939b
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Added tag a for changeset a8a82d372bb3
   
   changeset:   0:a8a82d372bb3
   tag:         a
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     add a
   
   $ hg tag --remove a
   $ hg log
   changeset:   2:e7feacc7ec9e
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Removed tag a
   
   changeset:   1:bd7ee4f3939b
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Added tag a for changeset a8a82d372bb3
   
   changeset:   0:a8a82d372bb3
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     add a
   
   $ hg tag --remove a
   abort: tag 'a' is already removed
   [10]
   $ hg log
   changeset:   2:e7feacc7ec9e
   tag:         tip
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Removed tag a
   
   changeset:   1:bd7ee4f3939b
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     Added tag a for changeset a8a82d372bb3
   
   changeset:   0:a8a82d372bb3
   user:        test
   date:        Thu Jan 01 00:00:00 1970 +0000
   summary:     add a
   
   $ cat .hgtags
   a8a82d372bb35b42ff736e74f07c23bcd99c371f a
   a8a82d372bb35b42ff736e74f07c23bcd99c371f a
   0000000000000000000000000000000000000000 a