clonebundles: add support for inline (streaming) clonebundles

The idea behind inline clonebundles is to send them through
the ssh or https connection to the Mercurial server.
We've been using this specifically for streaming clonebundles,
although it works for 'regular' clonebundles as well
(but is less relevant, since pullbundles exist).

We've had this enabled for around 9 months for a part
of our users.

A few benefits are:
- no need to secure an external system,
  since everything goes through the same Mercurial server
- easier scaling (in our case: no risk of inconsistencies
  between multiple mercurial-server mirrors and nginx clonebundles hosts)

Remaining topics/questions right now:
- The inline clonebundles don't work for https yet.
  This is because httppeer doesn't seem to support sending client
  capabilities.
  I didn't focus on that as my main goal was to get this working
  for ssh.

hgext/clonebundles.py

@@ -1,1022 +1,1034 b''
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 """advertise pre-generated bundles to seed clones
 
 "clonebundles" is a server-side extension used to advertise the existence
 of pre-generated, externally hosted bundle files to clients that are
 cloning so that cloning can be faster, more reliable, and require less
 resources on the server. "pullbundles" is a related feature for sending
 pre-generated bundle files to clients as part of pull operations.
 
 Cloning can be a CPU and I/O intensive operation on servers. Traditionally,
 the server, in response to a client's request to clone, dynamically generates
 a bundle containing the entire repository content and sends it to the client.
 There is no caching on the server and the server will have to redundantly
 generate the same outgoing bundle in response to each clone request. For
 servers with large repositories or with high clone volume, the load from
 clones can make scaling the server challenging and costly.
 
 This extension provides server operators the ability to offload
 potentially expensive clone load to an external service. Pre-generated
 bundles also allow using more CPU intensive compression, reducing the
 effective bandwidth requirements.
 
 Here's how clone bundles work:
 
 1. A server operator establishes a mechanism for making bundle files available
    on a hosting service where Mercurial clients can fetch them.
 2. A manifest file listing available bundle URLs and some optional metadata
    is added to the Mercurial repository on the server.
 3. A client initiates a clone against a clone bundles aware server.
 4. The client sees the server is advertising clone bundles and fetches the
    manifest listing available bundles.
 5. The client filters and sorts the available bundles based on what it
    supports and prefers.
 6. The client downloads and applies an available bundle from the
    server-specified URL.
 7. The client reconnects to the original server and performs the equivalent
    of :hg:`pull` to retrieve all repository data not in the bundle. (The
    repository could have been updated between when the bundle was created
    and when the client started the clone.) This may use "pullbundles".
 
 Instead of the server generating full repository bundles for every clone
 request, it generates full bundles once and they are subsequently reused to
 bootstrap new clones. The server may still transfer data at clone time.
 However, this is only data that has been added/changed since the bundle was
 created. For large, established repositories, this can reduce server load for
 clones to less than 1% of original.
 
 Here's how pullbundles work:
 
 1. A manifest file listing available bundles and describing the revisions
    is added to the Mercurial repository on the server.
 2. A new-enough client informs the server that it supports partial pulls
    and initiates a pull.
 3. If the server has pull bundles enabled and sees the client advertising
    partial pulls, it checks for a matching pull bundle in the manifest.
    A bundle matches if the format is supported by the client, the client
    has the required revisions already and needs something from the bundle.
 4. If there is at least one matching bundle, the server sends it to the client.
 5. The client applies the bundle and notices that the server reply was
    incomplete. It initiates another pull.
 
 To work, this extension requires the following of server operators:
 
 * Generating bundle files of repository content (typically periodically,
   such as once per day).
 * Clone bundles: A file server that clients have network access to and that
   Python knows how to talk to through its normal URL handling facility
   (typically an HTTP/HTTPS server).
 * A process for keeping the bundles manifest in sync with available bundle
   files.
 
 Strictly speaking, using a static file hosting server isn't required: a server
 operator could use a dynamic service for retrieving bundle data. However,
 static file hosting services are simple and scalable and should be sufficient
 for most needs.
 
 Bundle files can be generated with the :hg:`bundle` command. Typically
 :hg:`bundle --all` is used to produce a bundle of the entire repository.
 
 :hg:`debugcreatestreamclonebundle` can be used to produce a special
 *streaming clonebundle*. These are bundle files that are extremely efficient
 to produce and consume (read: fast). However, they are larger than
 traditional bundle formats and require that clients support the exact set
 of repository data store formats in use by the repository that created them.
 Typically, a newer server can serve data that is compatible with older clients.
 However, *streaming clone bundles* don't have this guarantee. **Server
 operators need to be aware that newer versions of Mercurial may produce
 streaming clone bundles incompatible with older Mercurial versions.**
 
 A server operator is responsible for creating a ``.hg/clonebundles.manifest``
 file containing the list of available bundle files suitable for seeding
 clones. If this file does not exist, the repository will not advertise the
 existence of clone bundles when clients connect. For pull bundles,
 ``.hg/pullbundles.manifest`` is used.
 
 The manifest file contains a newline (\\n) delimited list of entries.
 
 Each line in this file defines an available bundle. Lines have the format:
 
     <URL> [<key>=<value>[ <key>=<value>]]
 
 That is, a URL followed by an optional, space-delimited list of key=value
 pairs describing additional properties of this bundle. Both keys and values
 are URI encoded.
 
 For pull bundles, the URL is a path under the ``.hg`` directory of the
 repository.
 
 Keys in UPPERCASE are reserved for use by Mercurial and are defined below.
 All non-uppercase keys can be used by site installations. An example use
 for custom properties is to use the *datacenter* attribute to define which
 data center a file is hosted in. Clients could then prefer a server in the
 data center closest to them.
 
 The following reserved keys are currently defined:
 
 BUNDLESPEC
    A "bundle specification" string that describes the type of the bundle.
 
    These are string values that are accepted by the "--type" argument of
    :hg:`bundle`.
 
    The values are parsed in strict mode, which means they must be of the
    "<compression>-<type>" form. See
    mercurial.exchange.parsebundlespec() for more details.
 
    :hg:`debugbundle --spec` can be used to print the bundle specification
    string for a bundle file. The output of this command can be used verbatim
    for the value of ``BUNDLESPEC`` (it is already escaped).
 
    Clients will automatically filter out specifications that are unknown or
    unsupported so they won't attempt to download something that likely won't
    apply.
 
    The actual value doesn't impact client behavior beyond filtering:
    clients will still sniff the bundle type from the header of downloaded
    files.
 
    **Use of this key is highly recommended**, as it allows clients to
    easily skip unsupported bundles. If this key is not defined, an old
    client may attempt to apply a bundle that it is incapable of reading.
 
 REQUIRESNI
    Whether Server Name Indication (SNI) is required to connect to the URL.
    SNI allows servers to use multiple certificates on the same IP. It is
    somewhat common in CDNs and other hosting providers. Older Python
    versions do not support SNI. Defining this attribute enables clients
    with older Python versions to filter this entry without experiencing
    an opaque SSL failure at connection time.
 
    If this is defined, it is important to advertise a non-SNI fallback
    URL or clients running old Python releases may not be able to clone
    with the clonebundles facility.
 
    Value should be "true".
 
 REQUIREDRAM
    Value specifies expected memory requirements to decode the payload.
    Values can have suffixes for common bytes sizes. e.g. "64MB".
 
    This key is often used with zstd-compressed bundles using a high
    compression level / window size, which can require 100+ MB of memory
    to decode.
 
 heads
    Used for pull bundles. This contains the ``;`` separated changeset
    hashes of the heads of the bundle content.
 
 bases
    Used for pull bundles. This contains the ``;`` separated changeset
    hashes of the roots of the bundle content. This can be skipped if
    the bundle was created without ``--base``.
 
 Manifests can contain multiple entries. Assuming metadata is defined, clients
 will filter entries from the manifest that they don't support. The remaining
 entries are optionally sorted by client preferences
 (``ui.clonebundleprefers`` config option). The client then attempts
 to fetch the bundle at the first URL in the remaining list.
 
 **Errors when downloading a bundle will fail the entire clone operation:
 clients do not automatically fall back to a traditional clone.** The reason
 for this is that if a server is using clone bundles, it is probably doing so
 because the feature is necessary to help it scale. In other words, there
 is an assumption that clone load will be offloaded to another service and
 that the Mercurial server isn't responsible for serving this clone load.
 If that other service experiences issues and clients start mass falling back to
 the original Mercurial server, the added clone load could overwhelm the server
 due to unexpected load and effectively take it offline. Not having clients
 automatically fall back to cloning from the original server mitigates this
 scenario.
 
 Because there is no automatic Mercurial server fallback on failure of the
 bundle hosting service, it is important for server operators to view the bundle
 hosting service as an extension of the Mercurial server in terms of
 availability and service level agreements: if the bundle hosting service goes
 down, so does the ability for clients to clone. Note: clients will see a
 message informing them how to bypass the clone bundles facility when a failure
 occurs. So server operators should prepare for some people to follow these
 instructions when a failure occurs, thus driving more load to the original
 Mercurial server when the bundle hosting service fails.
 
 
+inline clonebundles
+-------------------
+
+It is possible to transmit clonebundles inline in case repositories are
+accessed over SSH. This avoids having to setup an external HTTPS server
+and results in the same access control as already present for the SSH setup.
+
+Inline clonebundles should be placed into the `.hg/bundle-cache` directory.
+A clonebundle at `.hg/bundle-cache/mybundle.bundle` is referred to
+in the `clonebundles.manifest` file as `peer-bundle-cache://mybundle.bundle`.
+
+
 auto-generation of clone bundles
 --------------------------------
 
 It is possible to set Mercurial to automatically re-generate clone bundles when
 enough new content is available.
 
 Mercurial will take care of the process asynchronously. The defined list of
 bundle-type will be generated, uploaded, and advertised. Older bundles will get
 decommissioned as newer ones replace them.
 
 Bundles Generation:
 ...................
 
 The extension can generate multiple variants of the clone bundle. Each
 different variant will be defined by the "bundle-spec" they use::
 
     [clone-bundles]
     auto-generate.formats= zstd-v2, gzip-v2
 
 See `hg help bundlespec` for details about available options.
 
 By default, new bundles are generated when 5% of the repository contents or at
 least 1000 revisions are not contained in the cached bundles. This option can
 be controlled by the `clone-bundles.trigger.below-bundled-ratio` option
 (default 0.95) and the `clone-bundles.trigger.revs` option (default 1000)::
 
     [clone-bundles]
     trigger.below-bundled-ratio=0.95
     trigger.revs=1000
 
 This logic can be manually triggered using the `admin::clone-bundles-refresh`
 command, or automatically on each repository change if
 `clone-bundles.auto-generate.on-change` is set to `yes`.
 
     [clone-bundles]
     auto-generate.on-change=yes
     auto-generate.formats= zstd-v2, gzip-v2
 
 Bundles Upload and Serving:
 ...........................
 
 The generated bundles need to be made available to users through a "public" URL.
 This should be donne through `clone-bundles.upload-command` configuration. The
 value of this command should be a shell command. It will have access to the
 bundle file path through the `$HGCB_BUNDLE_PATH` variable. And the expected
 basename in the "public" URL is accessible at::
 
   [clone-bundles]
   upload-command=sftp put $HGCB_BUNDLE_PATH \
       sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME
 
 If the file was already uploaded, the command must still succeed.
 
 After upload, the file should be available at an url defined by
 `clone-bundles.url-template`.
 
   [clone-bundles]
   url-template=https://bundles.host/cache/clone-bundles/{basename}
 
 Old bundles cleanup:
 ....................
 
 When new bundles are generated, the older ones are no longer necessary and can
 be removed from storage. This is done through the `clone-bundles.delete-command`
 configuration. The command is given the url of the artifact to delete through
 the `$HGCB_BUNDLE_URL` environment variable.
 
   [clone-bundles]
   delete-command=sftp rm sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME
 
 If the file was already deleted, the command must still succeed.
 """
 
 
 import os
 import weakref
 
 from mercurial.i18n import _
 
 from mercurial import (
     bundlecaches,
     commands,
     error,
     extensions,
     localrepo,
     lock,
     node,
     registrar,
     util,
     wireprotov1server,
 )
 
 
 from mercurial.utils import (
     procutil,
 )
 
 testedwith = b'ships-with-hg-core'
 
 
 def capabilities(orig, repo, proto):
     caps = orig(repo, proto)
 
     # Only advertise if a manifest exists. This does add some I/O to requests.
     # But this should be cheaper than a wasted network round trip due to
     # missing file.
     if repo.vfs.exists(bundlecaches.CB_MANIFEST_FILE):
         caps.append(b'clonebundles')
 
     return caps
 
 
 def extsetup(ui):
     extensions.wrapfunction(wireprotov1server, b'_capabilities', capabilities)
 
 
 # logic for bundle auto-generation
 
 
 configtable = {}
 configitem = registrar.configitem(configtable)
 
 cmdtable = {}
 command = registrar.command(cmdtable)
 
 configitem(b'clone-bundles', b'auto-generate.on-change', default=False)
 configitem(b'clone-bundles', b'auto-generate.formats', default=list)
 configitem(b'clone-bundles', b'trigger.below-bundled-ratio', default=0.95)
 configitem(b'clone-bundles', b'trigger.revs', default=1000)
 
 configitem(b'clone-bundles', b'upload-command', default=None)
 
 configitem(b'clone-bundles', b'delete-command', default=None)
 
 configitem(b'clone-bundles', b'url-template', default=None)
 
 configitem(b'devel', b'debug.clonebundles', default=False)
 
 
 # category for the post-close transaction hooks
 CAT_POSTCLOSE = b"clonebundles-autobundles"
 
 # template for bundle file names
 BUNDLE_MASK = (
     b"full-%(bundle_type)s-%(revs)d_revs-%(tip_short)s_tip-%(op_id)s.hg"
 )
 
 
 # file in .hg/ use to track clonebundles being auto-generated
 AUTO_GEN_FILE = b'clonebundles.auto-gen'
 
 
 class BundleBase(object):
     """represents the core of properties that matters for us in a bundle
 
     :bundle_type: the bundlespec (see hg help bundlespec)
     :revs:        the number of revisions in the repo at bundle creation time
     :tip_rev:     the rev-num of the tip revision
     :tip_node:    the node id of the tip-most revision in the bundle
 
     :ready:       True if the bundle is ready to be served
     """
 
     ready = False
 
     def __init__(self, bundle_type, revs, tip_rev, tip_node):
         self.bundle_type = bundle_type
         self.revs = revs
         self.tip_rev = tip_rev
         self.tip_node = tip_node
 
     def valid_for(self, repo):
         """is this bundle applicable to the current repository
 
         This is useful for detecting bundles made irrelevant by stripping.
         """
         tip_node = node.bin(self.tip_node)
         return repo.changelog.index.get_rev(tip_node) == self.tip_rev
 
     def __eq__(self, other):
         left = (self.ready, self.bundle_type, self.tip_rev, self.tip_node)
         right = (other.ready, other.bundle_type, other.tip_rev, other.tip_node)
         return left == right
 
     def __neq__(self, other):
         return not self == other
 
     def __cmp__(self, other):
         if self == other:
             return 0
         return -1
 
 
 class RequestedBundle(BundleBase):
     """A bundle that should be generated.
 
     Additional attributes compared to BundleBase
     :heads:       list of head revisions (as rev-num)
     :op_id:       a "unique" identifier for the operation triggering the change
     """
 
     def __init__(self, bundle_type, revs, tip_rev, tip_node, head_revs, op_id):
         self.head_revs = head_revs
         self.op_id = op_id
         super(RequestedBundle, self).__init__(
             bundle_type,
             revs,
             tip_rev,
             tip_node,
         )
 
     @property
     def suggested_filename(self):
         """A filename that can be used for the generated bundle"""
         data = {
             b'bundle_type': self.bundle_type,
             b'revs': self.revs,
             b'heads': self.head_revs,
             b'tip_rev': self.tip_rev,
             b'tip_node': self.tip_node,
             b'tip_short': self.tip_node[:12],
             b'op_id': self.op_id,
         }
         return BUNDLE_MASK % data
 
     def generate_bundle(self, repo, file_path):
         """generate the bundle at `filepath`"""
         commands.bundle(
             repo.ui,
             repo,
             file_path,
             base=[b"null"],
             rev=self.head_revs,
             type=self.bundle_type,
             quiet=True,
         )
 
     def generating(self, file_path, hostname=None, pid=None):
         """return a GeneratingBundle object from this object"""
         if pid is None:
             pid = os.getpid()
         if hostname is None:
             hostname = lock._getlockprefix()
         return GeneratingBundle(
             self.bundle_type,
             self.revs,
             self.tip_rev,
             self.tip_node,
             hostname,
             pid,
             file_path,
         )
 
 
 class GeneratingBundle(BundleBase):
     """A bundle being generated
 
     extra attributes compared to BundleBase:
 
     :hostname: the hostname of the machine generating the bundle
     :pid:      the pid of the process generating the bundle
     :filepath: the target filename of the bundle
 
     These attributes exist to help detect stalled generation processes.
     """
 
     ready = False
 
     def __init__(
         self, bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath
     ):
         self.hostname = hostname
         self.pid = pid
         self.filepath = filepath
         super(GeneratingBundle, self).__init__(
             bundle_type, revs, tip_rev, tip_node
         )
 
     @classmethod
     def from_line(cls, line):
         """create an object by deserializing a line from AUTO_GEN_FILE"""
         assert line.startswith(b'PENDING-v1 ')
         (
             __,
             bundle_type,
             revs,
             tip_rev,
             tip_node,
             hostname,
             pid,
             filepath,
         ) = line.split()
         hostname = util.urlreq.unquote(hostname)
         filepath = util.urlreq.unquote(filepath)
         revs = int(revs)
         tip_rev = int(tip_rev)
         pid = int(pid)
         return cls(
             bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath
         )
 
     def to_line(self):
         """serialize the object to include as a line in AUTO_GEN_FILE"""
         templ = b"PENDING-v1 %s %d %d %s %s %d %s"
         data = (
             self.bundle_type,
             self.revs,
             self.tip_rev,
             self.tip_node,
             util.urlreq.quote(self.hostname),
             self.pid,
             util.urlreq.quote(self.filepath),
         )
         return templ % data
 
     def __eq__(self, other):
         if not super(GeneratingBundle, self).__eq__(other):
             return False
         left = (self.hostname, self.pid, self.filepath)
         right = (other.hostname, other.pid, other.filepath)
         return left == right
 
     def uploaded(self, url, basename):
         """return a GeneratedBundle from this object"""
         return GeneratedBundle(
             self.bundle_type,
             self.revs,
             self.tip_rev,
             self.tip_node,
             url,
             basename,
         )
 
 
 class GeneratedBundle(BundleBase):
     """A bundle that is done being generated and can be served
 
     extra attributes compared to BundleBase:
 
     :file_url: the url where the bundle is available.
     :basename: the "basename" used to upload (useful for deletion)
 
     These attributes exist to generate a bundle manifest
     (.hg/pullbundles.manifest)
     """
 
     ready = True
 
     def __init__(
         self, bundle_type, revs, tip_rev, tip_node, file_url, basename
     ):
         self.file_url = file_url
         self.basename = basename
         super(GeneratedBundle, self).__init__(
             bundle_type, revs, tip_rev, tip_node
         )
 
     @classmethod
     def from_line(cls, line):
         """create an object by deserializing a line from AUTO_GEN_FILE"""
         assert line.startswith(b'DONE-v1 ')
         (
             __,
             bundle_type,
             revs,
             tip_rev,
             tip_node,
             file_url,
             basename,
         ) = line.split()
         revs = int(revs)
         tip_rev = int(tip_rev)
         file_url = util.urlreq.unquote(file_url)
         return cls(bundle_type, revs, tip_rev, tip_node, file_url, basename)
 
     def to_line(self):
         """serialize the object to include as a line in AUTO_GEN_FILE"""
         templ = b"DONE-v1 %s %d %d %s %s %s"
         data = (
             self.bundle_type,
             self.revs,
             self.tip_rev,
             self.tip_node,
             util.urlreq.quote(self.file_url),
             self.basename,
         )
         return templ % data
 
     def manifest_line(self):
         """serialize the object to include as a line in pullbundles.manifest"""
         templ = b"%s BUNDLESPEC=%s REQUIRESNI=true"
         return templ % (self.file_url, self.bundle_type)
 
     def __eq__(self, other):
         if not super(GeneratedBundle, self).__eq__(other):
             return False
         return self.file_url == other.file_url
 
 
 def parse_auto_gen(content):
     """parse the AUTO_GEN_FILE to return a list of Bundle object"""
     bundles = []
     for line in content.splitlines():
         if line.startswith(b'PENDING-v1 '):
             bundles.append(GeneratingBundle.from_line(line))
         elif line.startswith(b'DONE-v1 '):
             bundles.append(GeneratedBundle.from_line(line))
     return bundles
 
 
 def dumps_auto_gen(bundles):
     """serialize a list of Bundle as a AUTO_GEN_FILE content"""
     lines = []
     for b in bundles:
         lines.append(b"%s\n" % b.to_line())
     lines.sort()
     return b"".join(lines)
 
 
 def read_auto_gen(repo):
     """read the AUTO_GEN_FILE for the <repo> a list of Bundle object"""
     data = repo.vfs.tryread(AUTO_GEN_FILE)
     if not data:
         return []
     return parse_auto_gen(data)
 
 
 def write_auto_gen(repo, bundles):
     """write a list of Bundle objects into the repo's AUTO_GEN_FILE"""
     assert repo._cb_lock_ref is not None
     data = dumps_auto_gen(bundles)
     with repo.vfs(AUTO_GEN_FILE, mode=b'wb', atomictemp=True) as f:
         f.write(data)
 
 
 def generate_manifest(bundles):
     """write a list of Bundle objects into the repo's AUTO_GEN_FILE"""
     bundles = list(bundles)
     bundles.sort(key=lambda b: b.bundle_type)
     lines = []
     for b in bundles:
         lines.append(b"%s\n" % b.manifest_line())
     return b"".join(lines)
 
 
 def update_ondisk_manifest(repo):
     """update the clonebundle manifest with latest url"""
     with repo.clonebundles_lock():
         bundles = read_auto_gen(repo)
 
         per_types = {}
         for b in bundles:
             if not (b.ready and b.valid_for(repo)):
                 continue
             current = per_types.get(b.bundle_type)
             if current is not None and current.revs >= b.revs:
                 continue
             per_types[b.bundle_type] = b
         manifest = generate_manifest(per_types.values())
         with repo.vfs(
             bundlecaches.CB_MANIFEST_FILE, mode=b"wb", atomictemp=True
         ) as f:
             f.write(manifest)
 
 
 def update_bundle_list(repo, new_bundles=(), del_bundles=()):
     """modify the repo's AUTO_GEN_FILE
 
     This method also regenerates the clone bundle manifest when needed"""
     with repo.clonebundles_lock():
         bundles = read_auto_gen(repo)
         if del_bundles:
             bundles = [b for b in bundles if b not in del_bundles]
         new_bundles = [b for b in new_bundles if b not in bundles]
         bundles.extend(new_bundles)
         write_auto_gen(repo, bundles)
         all_changed = []
         all_changed.extend(new_bundles)
         all_changed.extend(del_bundles)
         if any(b.ready for b in all_changed):
             update_ondisk_manifest(repo)
 
 
 def cleanup_tmp_bundle(repo, target):
     """remove a GeneratingBundle file and entry"""
     assert not target.ready
     with repo.clonebundles_lock():
         repo.vfs.tryunlink(target.filepath)
         update_bundle_list(repo, del_bundles=[target])
 
 
 def finalize_one_bundle(repo, target):
     """upload a generated bundle and advertise it in the clonebundles.manifest"""
     with repo.clonebundles_lock():
         bundles = read_auto_gen(repo)
         if target in bundles and target.valid_for(repo):
             result = upload_bundle(repo, target)
             update_bundle_list(repo, new_bundles=[result])
     cleanup_tmp_bundle(repo, target)
 
 
 def find_outdated_bundles(repo, bundles):
     """finds outdated bundles"""
     olds = []
     per_types = {}
     for b in bundles:
         if not b.valid_for(repo):
             olds.append(b)
             continue
         l = per_types.setdefault(b.bundle_type, [])
         l.append(b)
     for key in sorted(per_types):
         all = per_types[key]
         if len(all) > 1:
             all.sort(key=lambda b: b.revs, reverse=True)
             olds.extend(all[1:])
     return olds
 
 
 def collect_garbage(repo):
     """finds outdated bundles and get them deleted"""
     with repo.clonebundles_lock():
         bundles = read_auto_gen(repo)
         olds = find_outdated_bundles(repo, bundles)
         for o in olds:
             delete_bundle(repo, o)
         update_bundle_list(repo, del_bundles=olds)
 
 
 def upload_bundle(repo, bundle):
     """upload the result of a GeneratingBundle and return a GeneratedBundle
 
     The upload is done using the `clone-bundles.upload-command`
     """
     cmd = repo.ui.config(b'clone-bundles', b'upload-command')
     url = repo.ui.config(b'clone-bundles', b'url-template')
     basename = repo.vfs.basename(bundle.filepath)
     filepath = procutil.shellquote(bundle.filepath)
     variables = {
         b'HGCB_BUNDLE_PATH': filepath,
         b'HGCB_BUNDLE_BASENAME': basename,
     }
     env = procutil.shellenviron(environ=variables)
     ret = repo.ui.system(cmd, environ=env)
     if ret:
         raise error.Abort(b"command returned status %d: %s" % (ret, cmd))
     url = (
         url.decode('utf8')
         .format(basename=basename.decode('utf8'))
         .encode('utf8')
     )
     return bundle.uploaded(url, basename)
 
 
 def delete_bundle(repo, bundle):
     """delete a bundle from storage"""
     assert bundle.ready
     msg = b'clone-bundles: deleting bundle %s\n'
     msg %= bundle.basename
     if repo.ui.configbool(b'devel', b'debug.clonebundles'):
         repo.ui.write(msg)
     else:
         repo.ui.debug(msg)
 
     cmd = repo.ui.config(b'clone-bundles', b'delete-command')
     variables = {
         b'HGCB_BUNDLE_URL': bundle.file_url,
         b'HGCB_BASENAME': bundle.basename,
     }
     env = procutil.shellenviron(environ=variables)
     ret = repo.ui.system(cmd, environ=env)
     if ret:
         raise error.Abort(b"command returned status %d: %s" % (ret, cmd))
 
 
 def auto_bundle_needed_actions(repo, bundles, op_id):
     """find the list of bundles that need action
 
     returns a list of RequestedBundle objects that need to be generated and
     uploaded."""
     create_bundles = []
     delete_bundles = []
     repo = repo.filtered(b"immutable")
     targets = repo.ui.configlist(b'clone-bundles', b'auto-generate.formats')
     ratio = float(
         repo.ui.config(b'clone-bundles', b'trigger.below-bundled-ratio')
     )
     abs_revs = repo.ui.configint(b'clone-bundles', b'trigger.revs')
     revs = len(repo.changelog)
     generic_data = {
         'revs': revs,
         'head_revs': repo.changelog.headrevs(),
         'tip_rev': repo.changelog.tiprev(),
         'tip_node': node.hex(repo.changelog.tip()),
         'op_id': op_id,
     }
     for t in targets:
         if new_bundle_needed(repo, bundles, ratio, abs_revs, t, revs):
             data = generic_data.copy()
             data['bundle_type'] = t
             b = RequestedBundle(**data)
             create_bundles.append(b)
     delete_bundles.extend(find_outdated_bundles(repo, bundles))
     return create_bundles, delete_bundles
 
 
 def new_bundle_needed(repo, bundles, ratio, abs_revs, bundle_type, revs):
     """consider the current cached content and trigger new bundles if needed"""
     threshold = max((revs * ratio), (revs - abs_revs))
     for b in bundles:
         if not b.valid_for(repo) or b.bundle_type != bundle_type:
             continue
         if b.revs > threshold:
             return False
     return True
 
 
 def start_one_bundle(repo, bundle):
     """start the generation of a single bundle file
 
     the `bundle` argument should be a RequestedBundle object.
 
     This data is passed to the `debugmakeclonebundles` "as is".
     """
     data = util.pickle.dumps(bundle)
     cmd = [procutil.hgexecutable(), b'--cwd', repo.path, INTERNAL_CMD]
     env = procutil.shellenviron()
     msg = b'clone-bundles: starting bundle generation: %s\n'
     stdout = None
     stderr = None
     waits = []
     record_wait = None
     if repo.ui.configbool(b'devel', b'debug.clonebundles'):
         stdout = procutil.stdout
         stderr = procutil.stderr
         repo.ui.write(msg % bundle.bundle_type)
         record_wait = waits.append
     else:
         repo.ui.debug(msg % bundle.bundle_type)
     bg = procutil.runbgcommand
     bg(
         cmd,
         env,
         stdin_bytes=data,
         stdout=stdout,
         stderr=stderr,
         record_wait=record_wait,
     )
     for f in waits:
         f()
 
 
 INTERNAL_CMD = b'debug::internal-make-clone-bundles'
 
 
 @command(INTERNAL_CMD, [], b'')
 def debugmakeclonebundles(ui, repo):
     """Internal command to auto-generate debug bundles"""
     requested_bundle = util.pickle.load(procutil.stdin)
     procutil.stdin.close()
 
     collect_garbage(repo)
 
     fname = requested_bundle.suggested_filename
     fpath = repo.vfs.makedirs(b'tmp-bundles')
     fpath = repo.vfs.join(b'tmp-bundles', fname)
     bundle = requested_bundle.generating(fpath)
     update_bundle_list(repo, new_bundles=[bundle])
 
     requested_bundle.generate_bundle(repo, fpath)
 
     repo.invalidate()
     finalize_one_bundle(repo, bundle)
 
 
 def make_auto_bundler(source_repo):
     reporef = weakref.ref(source_repo)
 
     def autobundle(tr):
         repo = reporef()
         assert repo is not None
         bundles = read_auto_gen(repo)
         new, __ = auto_bundle_needed_actions(repo, bundles, b"%d_txn" % id(tr))
         for data in new:
             start_one_bundle(repo, data)
         return None
 
     return autobundle
 
 
 def reposetup(ui, repo):
     """install the two pieces needed for automatic clonebundle generation
 
     - add a "post-close" hook that fires bundling when needed
     - introduce a clone-bundle lock to let multiple processes meddle with the
       state files.
     """
     if not repo.local():
         return
 
     class autobundlesrepo(repo.__class__):
         def transaction(self, *args, **kwargs):
             tr = super(autobundlesrepo, self).transaction(*args, **kwargs)
             enabled = repo.ui.configbool(
                 b'clone-bundles',
                 b'auto-generate.on-change',
             )
             targets = repo.ui.configlist(
                 b'clone-bundles', b'auto-generate.formats'
             )
             if enabled and targets:
                 tr.addpostclose(CAT_POSTCLOSE, make_auto_bundler(self))
             return tr
 
         @localrepo.unfilteredmethod
         def clonebundles_lock(self, wait=True):
             '''Lock the repository file related to clone bundles'''
             if not util.safehasattr(self, '_cb_lock_ref'):
                 self._cb_lock_ref = None
             l = self._currentlock(self._cb_lock_ref)
             if l is not None:
                 l.lock()
                 return l
 
             l = self._lock(
                 vfs=self.vfs,
                 lockname=b"clonebundleslock",
                 wait=wait,
                 releasefn=None,
                 acquirefn=None,
                 desc=_(b'repository %s') % self.origroot,
             )
             self._cb_lock_ref = weakref.ref(l)
             return l
 
     repo._wlockfreeprefix.add(AUTO_GEN_FILE)
     repo._wlockfreeprefix.add(bundlecaches.CB_MANIFEST_FILE)
     repo.__class__ = autobundlesrepo
 
 
 @command(
     b'admin::clone-bundles-refresh',
     [
         (
             b'',
             b'background',
             False,
             _(b'start bundle generation in the background'),
         ),
     ],
     b'',
 )
 def cmd_admin_clone_bundles_refresh(
     ui,
     repo: localrepo.localrepository,
     background=False,
 ):
     """generate clone bundles according to the configuration
 
     This runs the logic for automatic generation, removing outdated bundles and
     generating new ones if necessary. See :hg:`help -e clone-bundles` for
     details about how to configure this feature.
     """
     debug = repo.ui.configbool(b'devel', b'debug.clonebundles')
     bundles = read_auto_gen(repo)
     op_id = b"%d_acbr" % os.getpid()
     create, delete = auto_bundle_needed_actions(repo, bundles, op_id)
 
     # if some bundles are scheduled for creation in the background, they will
     # deal with garbage collection too, so no need to synchroniously do it.
     #
     # However if no bundles are scheduled for creation, we need to explicitly do
     # it here.
     if not (background and create):
         # we clean up outdated bundles before generating new ones to keep the
         # last two versions of the bundle around for a while and avoid having to
         # deal with clients that just got served a manifest.
         for o in delete:
             delete_bundle(repo, o)
         update_bundle_list(repo, del_bundles=delete)
 
     if create:
         fpath = repo.vfs.makedirs(b'tmp-bundles')
 
     if background:
         for requested_bundle in create:
             start_one_bundle(repo, requested_bundle)
     else:
         for requested_bundle in create:
             if debug:
                 msg = b'clone-bundles: starting bundle generation: %s\n'
                 repo.ui.write(msg % requested_bundle.bundle_type)
             fname = requested_bundle.suggested_filename
             fpath = repo.vfs.join(b'tmp-bundles', fname)
             generating_bundle = requested_bundle.generating(fpath)
             update_bundle_list(repo, new_bundles=[generating_bundle])
             requested_bundle.generate_bundle(repo, fpath)
             result = upload_bundle(repo, generating_bundle)
             update_bundle_list(repo, new_bundles=[result])
             update_ondisk_manifest(repo)
             cleanup_tmp_bundle(repo, generating_bundle)
 
 
 @command(b'admin::clone-bundles-clear', [], b'')
 def cmd_admin_clone_bundles_clear(ui, repo: localrepo.localrepository):
     """remove existing clone bundle caches
 
     See `hg help admin::clone-bundles-refresh` for details on how to regenerate
     them.
 
     This command will only affect bundles currently available, it will not
     affect bundles being asynchronously generated.
     """
     bundles = read_auto_gen(repo)
     delete = [b for b in bundles if b.ready]
     for o in delete:
         delete_bundle(repo, o)
     update_bundle_list(repo, del_bundles=delete)

mercurial/bundlecaches.py

@@ -1,538 +1,540 b''
 # bundlecaches.py - utility to deal with pre-computed bundle for servers
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 import collections
 
 from typing import (
     cast,
 )
 
 from .i18n import _
 
 from .thirdparty import attr
 
 from . import (
     error,
     requirements as requirementsmod,
     sslutil,
     util,
 )
 from .utils import stringutil
 
 urlreq = util.urlreq
 
+BUNDLE_CACHE_DIR = b'bundle-cache'
 CB_MANIFEST_FILE = b'clonebundles.manifest'
+CLONEBUNDLESCHEME = b"peer-bundle-cache://"
 
 
 def get_manifest(repo):
     """get the bundle manifest to be served to a client from a server"""
     raw_text = repo.vfs.tryread(CB_MANIFEST_FILE)
     entries = [e.split(b' ', 1) for e in raw_text.splitlines()]
 
     new_lines = []
     for e in entries:
         url = alter_bundle_url(repo, e[0])
         if len(e) == 1:
             line = url + b'\n'
         else:
             line = b"%s %s\n" % (url, e[1])
         new_lines.append(line)
     return b''.join(new_lines)
 
 
 def alter_bundle_url(repo, url):
     """a function that exist to help extension and hosting to alter the url
 
     This will typically be used to inject authentication information in the url
     of cached bundles."""
     return url
 
 
 @attr.s
 class bundlespec:
     compression = attr.ib()
     wirecompression = attr.ib()
     version = attr.ib()
     wireversion = attr.ib()
     # parameters explicitly overwritten by the config or the specification
     _explicit_params = attr.ib()
     # default parameter for the version
     #
     # Keeping it separated is useful to check what was actually overwritten.
     _default_opts = attr.ib()
 
     @property
     def params(self):
         return collections.ChainMap(self._explicit_params, self._default_opts)
 
     @property
     def contentopts(self):
         # kept for Backward Compatibility concerns.
         return self.params
 
     def set_param(self, key, value, overwrite=True):
         """Set a bundle parameter value.
 
         Will only overwrite if overwrite is true"""
         if overwrite or key not in self._explicit_params:
             self._explicit_params[key] = value
 
 
 # Maps bundle version human names to changegroup versions.
 _bundlespeccgversions = {
     b'v1': b'01',
     b'v2': b'02',
     b'v3': b'03',
     b'packed1': b's1',
     b'bundle2': b'02',  # legacy
 }
 
 # Maps bundle version with content opts to choose which part to bundle
 _bundlespeccontentopts = {
     b'v1': {
         b'changegroup': True,
         b'cg.version': b'01',
         b'obsolescence': False,
         b'phases': False,
         b'tagsfnodescache': False,
         b'revbranchcache': False,
     },
     b'v2': {
         b'changegroup': True,
         b'cg.version': b'02',
         b'obsolescence': False,
         b'phases': False,
         b'tagsfnodescache': True,
         b'revbranchcache': True,
     },
     b'v3': {
         b'changegroup': True,
         b'cg.version': b'03',
         b'obsolescence': False,
         b'phases': True,
         b'tagsfnodescache': True,
         b'revbranchcache': True,
     },
     b'streamv2': {
         b'changegroup': False,
         b'cg.version': b'02',
         b'obsolescence': False,
         b'phases': False,
         b"streamv2": True,
         b'tagsfnodescache': False,
         b'revbranchcache': False,
     },
     b'streamv3-exp': {
         b'changegroup': False,
         b'cg.version': b'03',
         b'obsolescence': False,
         b'phases': False,
         b"streamv3-exp": True,
         b'tagsfnodescache': False,
         b'revbranchcache': False,
     },
     b'packed1': {
         b'cg.version': b's1',
     },
     b'bundle2': {  # legacy
         b'cg.version': b'02',
     },
 }
 _bundlespeccontentopts[b'bundle2'] = _bundlespeccontentopts[b'v2']
 
 _bundlespecvariants = {b"streamv2": {}}
 
 # Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
 _bundlespecv1compengines = {b'gzip', b'bzip2', b'none'}
 
 
 def param_bool(key, value):
     """make a boolean out of a parameter value"""
     b = stringutil.parsebool(value)
     if b is None:
         msg = _(b"parameter %s should be a boolean ('%s')")
         msg %= (key, value)
         raise error.InvalidBundleSpecification(msg)
     return b
 
 
 # mapping of known parameter name need their value processed
 bundle_spec_param_processing = {
     b"obsolescence": param_bool,
     b"obsolescence-mandatory": param_bool,
     b"phases": param_bool,
 }
 
 
 def _parseparams(s):
     """parse bundlespec parameter section
 
     input: "comp-version;params" string
 
     return: (spec; {param_key: param_value})
     """
     if b';' not in s:
         return s, {}
 
     params = {}
     version, paramstr = s.split(b';', 1)
 
     err = _(b'invalid bundle specification: missing "=" in parameter: %s')
     for p in paramstr.split(b';'):
         if b'=' not in p:
             msg = err % p
             raise error.InvalidBundleSpecification(msg)
 
         key, value = p.split(b'=', 1)
         key = urlreq.unquote(key)
         value = urlreq.unquote(value)
         process = bundle_spec_param_processing.get(key)
         if process is not None:
             value = process(key, value)
         params[key] = value
 
     return version, params
 
 
 def parsebundlespec(repo, spec, strict=True):
     """Parse a bundle string specification into parts.
 
     Bundle specifications denote a well-defined bundle/exchange format.
     The content of a given specification should not change over time in
     order to ensure that bundles produced by a newer version of Mercurial are
     readable from an older version.
 
     The string currently has the form:
 
        <compression>-<type>[;<parameter0>[;<parameter1>]]
 
     Where <compression> is one of the supported compression formats
     and <type> is (currently) a version string. A ";" can follow the type and
     all text afterwards is interpreted as URI encoded, ";" delimited key=value
     pairs.
 
     If ``strict`` is True (the default) <compression> is required. Otherwise,
     it is optional.
 
     Returns a bundlespec object of (compression, version, parameters).
     Compression will be ``None`` if not in strict mode and a compression isn't
     defined.
 
     An ``InvalidBundleSpecification`` is raised when the specification is
     not syntactically well formed.
 
     An ``UnsupportedBundleSpecification`` is raised when the compression or
     bundle type/version is not recognized.
 
     Note: this function will likely eventually return a more complex data
     structure, including bundle2 part information.
     """
     if strict and b'-' not in spec:
         raise error.InvalidBundleSpecification(
             _(
                 b'invalid bundle specification; '
                 b'must be prefixed with compression: %s'
             )
             % spec
         )
 
     pre_args = spec.split(b';', 1)[0]
     if b'-' in pre_args:
         compression, version = spec.split(b'-', 1)
 
         if compression not in util.compengines.supportedbundlenames:
             raise error.UnsupportedBundleSpecification(
                 _(b'%s compression is not supported') % compression
             )
 
         version, params = _parseparams(version)
 
         if version not in _bundlespeccontentopts:
             raise error.UnsupportedBundleSpecification(
                 _(b'%s is not a recognized bundle version') % version
             )
     else:
         # Value could be just the compression or just the version, in which
         # case some defaults are assumed (but only when not in strict mode).
         assert not strict
 
         spec, params = _parseparams(spec)
 
         if spec in util.compengines.supportedbundlenames:
             compression = spec
             version = b'v1'
             # Generaldelta repos require v2.
             if requirementsmod.GENERALDELTA_REQUIREMENT in repo.requirements:
                 version = b'v2'
             elif requirementsmod.REVLOGV2_REQUIREMENT in repo.requirements:
                 version = b'v2'
             # Modern compression engines require v2.
             if compression not in _bundlespecv1compengines:
                 version = b'v2'
         elif spec in _bundlespeccontentopts:
             if spec == b'packed1':
                 compression = b'none'
             else:
                 compression = b'bzip2'
             version = spec
         else:
             raise error.UnsupportedBundleSpecification(
                 _(b'%s is not a recognized bundle specification') % spec
             )
 
     # Bundle version 1 only supports a known set of compression engines.
     if version == b'v1' and compression not in _bundlespecv1compengines:
         raise error.UnsupportedBundleSpecification(
             _(b'compression engine %s is not supported on v1 bundles')
             % compression
         )
 
     # The specification for packed1 can optionally declare the data formats
     # required to apply it. If we see this metadata, compare against what the
     # repo supports and error if the bundle isn't compatible.
     if version == b'packed1' and b'requirements' in params:
         requirements = set(cast(bytes, params[b'requirements']).split(b','))
         missingreqs = requirements - requirementsmod.STREAM_FIXED_REQUIREMENTS
         if missingreqs:
             raise error.UnsupportedBundleSpecification(
                 _(b'missing support for repository features: %s')
                 % b', '.join(sorted(missingreqs))
             )
 
     # Compute contentopts based on the version
     if b"stream" in params:
         # This case is fishy as this mostly derails the version selection
         # mechanism. `stream` bundles are quite specific and used differently
         # as "normal" bundles.
         #
         # (we should probably define a cleaner way to do this and raise a
         # warning when the old way is encountered)
         if params[b"stream"] == b"v2":
             version = b"streamv2"
         if params[b"stream"] == b"v3-exp":
             version = b"streamv3-exp"
     contentopts = _bundlespeccontentopts.get(version, {}).copy()
     if version == b"streamv2" or version == b"streamv3-exp":
         # streamv2 have been reported as "v2" for a while.
         version = b"v2"
 
     engine = util.compengines.forbundlename(compression)
     compression, wirecompression = engine.bundletype()
     wireversion = _bundlespeccontentopts[version][b'cg.version']
 
     return bundlespec(
         compression, wirecompression, version, wireversion, params, contentopts
     )
 
 
 def parseclonebundlesmanifest(repo, s):
     """Parses the raw text of a clone bundles manifest.
 
     Returns a list of dicts. The dicts have a ``URL`` key corresponding
     to the URL and other keys are the attributes for the entry.
     """
     m = []
     for line in s.splitlines():
         fields = line.split()
         if not fields:
             continue
         attrs = {b'URL': fields[0]}
         for rawattr in fields[1:]:
             key, value = rawattr.split(b'=', 1)
             key = util.urlreq.unquote(key)
             value = util.urlreq.unquote(value)
             attrs[key] = value
 
             # Parse BUNDLESPEC into components. This makes client-side
             # preferences easier to specify since you can prefer a single
             # component of the BUNDLESPEC.
             if key == b'BUNDLESPEC':
                 try:
                     bundlespec = parsebundlespec(repo, value)
                     attrs[b'COMPRESSION'] = bundlespec.compression
                     attrs[b'VERSION'] = bundlespec.version
                 except error.InvalidBundleSpecification:
                     pass
                 except error.UnsupportedBundleSpecification:
                     pass
 
         m.append(attrs)
 
     return m
 
 
 def isstreamclonespec(bundlespec):
     # Stream clone v1
     if bundlespec.wirecompression == b'UN' and bundlespec.wireversion == b's1':
         return True
 
     # Stream clone v2
     if (
         bundlespec.wirecompression == b'UN'
         and bundlespec.wireversion == b'02'
         and (
             bundlespec.contentopts.get(b'streamv2')
             or bundlespec.contentopts.get(b'streamv3-exp')
         )
     ):
         return True
 
     return False
 
 
 def filterclonebundleentries(repo, entries, streamclonerequested=False):
     """Remove incompatible clone bundle manifest entries.
 
     Accepts a list of entries parsed with ``parseclonebundlesmanifest``
     and returns a new list consisting of only the entries that this client
     should be able to apply.
 
     There is no guarantee we'll be able to apply all returned entries because
     the metadata we use to filter on may be missing or wrong.
     """
     newentries = []
     for entry in entries:
         spec = entry.get(b'BUNDLESPEC')
         if spec:
             try:
                 bundlespec = parsebundlespec(repo, spec, strict=True)
 
                 # If a stream clone was requested, filter out non-streamclone
                 # entries.
                 if streamclonerequested and not isstreamclonespec(bundlespec):
                     repo.ui.debug(
                         b'filtering %s because not a stream clone\n'
                         % entry[b'URL']
                     )
                     continue
 
             except error.InvalidBundleSpecification as e:
                 repo.ui.debug(stringutil.forcebytestr(e) + b'\n')
                 continue
             except error.UnsupportedBundleSpecification as e:
                 repo.ui.debug(
                     b'filtering %s because unsupported bundle '
                     b'spec: %s\n' % (entry[b'URL'], stringutil.forcebytestr(e))
                 )
                 continue
         # If we don't have a spec and requested a stream clone, we don't know
         # what the entry is so don't attempt to apply it.
         elif streamclonerequested:
             repo.ui.debug(
                 b'filtering %s because cannot determine if a stream '
                 b'clone bundle\n' % entry[b'URL']
             )
             continue
 
         if b'REQUIRESNI' in entry and not sslutil.hassni:
             repo.ui.debug(
                 b'filtering %s because SNI not supported\n' % entry[b'URL']
             )
             continue
 
         if b'REQUIREDRAM' in entry:
             try:
                 requiredram = util.sizetoint(entry[b'REQUIREDRAM'])
             except error.ParseError:
                 repo.ui.debug(
                     b'filtering %s due to a bad REQUIREDRAM attribute\n'
                     % entry[b'URL']
                 )
                 continue
             actualram = repo.ui.estimatememory()
             if actualram is not None and actualram * 0.66 < requiredram:
                 repo.ui.debug(
                     b'filtering %s as it needs more than 2/3 of system memory\n'
                     % entry[b'URL']
                 )
                 continue
 
         newentries.append(entry)
 
     return newentries
 
 
 class clonebundleentry:
     """Represents an item in a clone bundles manifest.
 
     This rich class is needed to support sorting since sorted() in Python 3
     doesn't support ``cmp`` and our comparison is complex enough that ``key=``
     won't work.
     """
 
     def __init__(self, value, prefers):
         self.value = value
         self.prefers = prefers
 
     def _cmp(self, other):
         for prefkey, prefvalue in self.prefers:
             avalue = self.value.get(prefkey)
             bvalue = other.value.get(prefkey)
 
             # Special case for b missing attribute and a matches exactly.
             if avalue is not None and bvalue is None and avalue == prefvalue:
                 return -1
 
             # Special case for a missing attribute and b matches exactly.
             if bvalue is not None and avalue is None and bvalue == prefvalue:
                 return 1
 
             # We can't compare unless attribute present on both.
             if avalue is None or bvalue is None:
                 continue
 
             # Same values should fall back to next attribute.
             if avalue == bvalue:
                 continue
 
             # Exact matches come first.
             if avalue == prefvalue:
                 return -1
             if bvalue == prefvalue:
                 return 1
 
             # Fall back to next attribute.
             continue
 
         # If we got here we couldn't sort by attributes and prefers. Fall
         # back to index order.
         return 0
 
     def __lt__(self, other):
         return self._cmp(other) < 0
 
     def __gt__(self, other):
         return self._cmp(other) > 0
 
     def __eq__(self, other):
         return self._cmp(other) == 0
 
     def __le__(self, other):
         return self._cmp(other) <= 0
 
     def __ge__(self, other):
         return self._cmp(other) >= 0
 
     def __ne__(self, other):
         return self._cmp(other) != 0
 
 
 def sortclonebundleentries(ui, entries):
     prefers = ui.configlist(b'ui', b'clonebundleprefers')
     if not prefers:
         return list(entries)
 
     def _split(p):
         if b'=' not in p:
             hint = _(b"each comma separated item should be key=value pairs")
             raise error.Abort(
                 _(b"invalid ui.clonebundleprefers item: %s") % p, hint=hint
             )
         return p.split(b'=', 1)
 
     prefers = [_split(p) for p in prefers]
 
     items = sorted(clonebundleentry(v, prefers) for v in entries)
     return [i.value for i in items]

mercurial/exchange.py

@@ -1,2881 +1,2892 b''
 # exchange.py - utility to exchange data between repos.
 #
 # Copyright 2005-2007 Olivia Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 
 import collections
 import weakref
 
 from .i18n import _
 from .node import (
     hex,
     nullrev,
 )
 from . import (
     bookmarks as bookmod,
     bundle2,
     bundlecaches,
     changegroup,
     discovery,
     error,
     lock as lockmod,
     logexchange,
     narrowspec,
     obsolete,
     obsutil,
     phases,
     pushkey,
     pycompat,
     requirements,
     scmutil,
     streamclone,
     url as urlmod,
     util,
     wireprototypes,
 )
 from .utils import (
     hashutil,
     stringutil,
     urlutil,
 )
 from .interfaces import repository
 
 urlerr = util.urlerr
 urlreq = util.urlreq
 
 _NARROWACL_SECTION = b'narrowacl'
 
 
 def readbundle(ui, fh, fname, vfs=None):
     header = changegroup.readexactly(fh, 4)
 
     alg = None
     if not fname:
         fname = b"stream"
         if not header.startswith(b'HG') and header.startswith(b'\0'):
             fh = changegroup.headerlessfixup(fh, header)
             header = b"HG10"
             alg = b'UN'
     elif vfs:
         fname = vfs.join(fname)
 
     magic, version = header[0:2], header[2:4]
 
     if magic != b'HG':
         raise error.Abort(_(b'%s: not a Mercurial bundle') % fname)
     if version == b'10':
         if alg is None:
             alg = changegroup.readexactly(fh, 2)
         return changegroup.cg1unpacker(fh, alg)
     elif version.startswith(b'2'):
         return bundle2.getunbundler(ui, fh, magicstring=magic + version)
     elif version == b'S1':
         return streamclone.streamcloneapplier(fh)
     else:
         raise error.Abort(
             _(b'%s: unknown bundle version %s') % (fname, version)
         )
 
 
 def _format_params(params):
     parts = []
     for key, value in sorted(params.items()):
         value = urlreq.quote(value)
         parts.append(b"%s=%s" % (key, value))
     return b';'.join(parts)
 
 
 def getbundlespec(ui, fh):
     """Infer the bundlespec from a bundle file handle.
 
     The input file handle is seeked and the original seek position is not
     restored.
     """
 
     def speccompression(alg):
         try:
             return util.compengines.forbundletype(alg).bundletype()[0]
         except KeyError:
             return None
 
     params = {}
 
     b = readbundle(ui, fh, None)
     if isinstance(b, changegroup.cg1unpacker):
         alg = b._type
         if alg == b'_truncatedBZ':
             alg = b'BZ'
         comp = speccompression(alg)
         if not comp:
             raise error.Abort(_(b'unknown compression algorithm: %s') % alg)
         return b'%s-v1' % comp
     elif isinstance(b, bundle2.unbundle20):
         if b'Compression' in b.params:
             comp = speccompression(b.params[b'Compression'])
             if not comp:
                 raise error.Abort(
                     _(b'unknown compression algorithm: %s') % comp
                 )
         else:
             comp = b'none'
 
         version = None
         for part in b.iterparts():
             if part.type == b'changegroup':
                 cgversion = part.params[b'version']
                 if cgversion in (b'01', b'02'):
                     version = b'v2'
                 elif cgversion in (b'03',):
                     version = b'v2'
                     params[b'cg.version'] = cgversion
                 else:
                     raise error.Abort(
                         _(
                             b'changegroup version %s does not have '
                             b'a known bundlespec'
                         )
                         % version,
                         hint=_(b'try upgrading your Mercurial client'),
                     )
             elif part.type == b'stream2' and version is None:
                 # A stream2 part requires to be part of a v2 bundle
                 requirements = urlreq.unquote(part.params[b'requirements'])
                 splitted = requirements.split()
                 params = bundle2._formatrequirementsparams(splitted)
                 return b'none-v2;stream=v2;%s' % params
             elif part.type == b'stream3-exp' and version is None:
                 # A stream3 part requires to be part of a v2 bundle
                 requirements = urlreq.unquote(part.params[b'requirements'])
                 splitted = requirements.split()
                 params = bundle2._formatrequirementsparams(splitted)
                 return b'none-v2;stream=v3-exp;%s' % params
             elif part.type == b'obsmarkers':
                 params[b'obsolescence'] = b'yes'
                 if not part.mandatory:
                     params[b'obsolescence-mandatory'] = b'no'
 
         if not version:
             raise error.Abort(
                 _(b'could not identify changegroup version in bundle')
             )
         spec = b'%s-%s' % (comp, version)
         if params:
             spec += b';'
             spec += _format_params(params)
         return spec
 
     elif isinstance(b, streamclone.streamcloneapplier):
         requirements = streamclone.readbundle1header(fh)[2]
         formatted = bundle2._formatrequirementsparams(requirements)
         return b'none-packed1;%s' % formatted
     else:
         raise error.Abort(_(b'unknown bundle type: %s') % b)
 
 
 def _computeoutgoing(repo, heads, common):
     """Computes which revs are outgoing given a set of common
     and a set of heads.
 
     This is a separate function so extensions can have access to
     the logic.
 
     Returns a discovery.outgoing object.
     """
     cl = repo.changelog
     if common:
         hasnode = cl.hasnode
         common = [n for n in common if hasnode(n)]
     else:
         common = [repo.nullid]
     if not heads:
         heads = cl.heads()
     return discovery.outgoing(repo, common, heads)
 
 
 def _checkpublish(pushop):
     repo = pushop.repo
     ui = repo.ui
     behavior = ui.config(b'experimental', b'auto-publish')
     if pushop.publish or behavior not in (b'warn', b'confirm', b'abort'):
         return
     remotephases = listkeys(pushop.remote, b'phases')
     if not remotephases.get(b'publishing', False):
         return
 
     if pushop.revs is None:
         published = repo.filtered(b'served').revs(b'not public()')
     else:
         published = repo.revs(b'::%ln - public()', pushop.revs)
         # we want to use pushop.revs in the revset even if they themselves are
         # secret, but we don't want to have anything that the server won't see
         # in the result of this expression
         published &= repo.filtered(b'served')
     if published:
         if behavior == b'warn':
             ui.warn(
                 _(b'%i changesets about to be published\n') % len(published)
             )
         elif behavior == b'confirm':
             if ui.promptchoice(
                 _(b'push and publish %i changesets (yn)?$$ &Yes $$ &No')
                 % len(published)
             ):
                 raise error.CanceledError(_(b'user quit'))
         elif behavior == b'abort':
             msg = _(b'push would publish %i changesets') % len(published)
             hint = _(
                 b"use --publish or adjust 'experimental.auto-publish'"
                 b" config"
             )
             raise error.Abort(msg, hint=hint)
 
 
 def _forcebundle1(op):
     """return true if a pull/push must use bundle1
 
     This function is used to allow testing of the older bundle version"""
     ui = op.repo.ui
     # The goal is this config is to allow developer to choose the bundle
     # version used during exchanged. This is especially handy during test.
     # Value is a list of bundle version to be picked from, highest version
     # should be used.
     #
     # developer config: devel.legacy.exchange
     exchange = ui.configlist(b'devel', b'legacy.exchange')
     forcebundle1 = b'bundle2' not in exchange and b'bundle1' in exchange
     return forcebundle1 or not op.remote.capable(b'bundle2')
 
 
 class pushoperation:
     """A object that represent a single push operation
 
     Its purpose is to carry push related state and very common operations.
 
     A new pushoperation should be created at the beginning of each push and
     discarded afterward.
     """
 
     def __init__(
         self,
         repo,
         remote,
         force=False,
         revs=None,
         newbranch=False,
         bookmarks=(),
         publish=False,
         pushvars=None,
     ):
         # repo we push from
         self.repo = repo
         self.ui = repo.ui
         # repo we push to
         self.remote = remote
         # force option provided
         self.force = force
         # revs to be pushed (None is "all")
         self.revs = revs
         # bookmark explicitly pushed
         self.bookmarks = bookmarks
         # allow push of new branch
         self.newbranch = newbranch
         # step already performed
         # (used to check what steps have been already performed through bundle2)
         self.stepsdone = set()
         # Integer version of the changegroup push result
         # - None means nothing to push
         # - 0 means HTTP error
         # - 1 means we pushed and remote head count is unchanged *or*
         #   we have outgoing changesets but refused to push
         # - other values as described by addchangegroup()
         self.cgresult = None
         # Boolean value for the bookmark push
         self.bkresult = None
         # discover.outgoing object (contains common and outgoing data)
         self.outgoing = None
         # all remote topological heads before the push
         self.remoteheads = None
         # Details of the remote branch pre and post push
         #
         # mapping: {'branch': ([remoteheads],
         #                      [newheads],
         #                      [unsyncedheads],
         #                      [discardedheads])}
         # - branch: the branch name
         # - remoteheads: the list of remote heads known locally
         #                None if the branch is new
         # - newheads: the new remote heads (known locally) with outgoing pushed
         # - unsyncedheads: the list of remote heads unknown locally.
         # - discardedheads: the list of remote heads made obsolete by the push
         self.pushbranchmap = None
         # testable as a boolean indicating if any nodes are missing locally.
         self.incoming = None
         # summary of the remote phase situation
         self.remotephases = None
         # phases changes that must be pushed along side the changesets
         self.outdatedphases = None
         # phases changes that must be pushed if changeset push fails
         self.fallbackoutdatedphases = None
         # outgoing obsmarkers
         self.outobsmarkers = set()
         # outgoing bookmarks, list of (bm, oldnode | '', newnode | '')
         self.outbookmarks = []
         # transaction manager
         self.trmanager = None
         # map { pushkey partid -> callback handling failure}
         # used to handle exception from mandatory pushkey part failure
         self.pkfailcb = {}
         # an iterable of pushvars or None
         self.pushvars = pushvars
         # publish pushed changesets
         self.publish = publish
 
     @util.propertycache
     def futureheads(self):
         """future remote heads if the changeset push succeeds"""
         return self.outgoing.ancestorsof
 
     @util.propertycache
     def fallbackheads(self):
         """future remote heads if the changeset push fails"""
         if self.revs is None:
             # not target to push, all common are relevant
             return self.outgoing.commonheads
         unfi = self.repo.unfiltered()
         # I want cheads = heads(::ancestorsof and ::commonheads)
         # (ancestorsof is revs with secret changeset filtered out)
         #
         # This can be expressed as:
         #     cheads = ( (ancestorsof and ::commonheads)
         #              + (commonheads and ::ancestorsof))"
         #              )
         #
         # while trying to push we already computed the following:
         #     common = (::commonheads)
         #     missing = ((commonheads::ancestorsof) - commonheads)
         #
         # We can pick:
         # * ancestorsof part of common (::commonheads)
         common = self.outgoing.common
         rev = self.repo.changelog.index.rev
         cheads = [node for node in self.revs if rev(node) in common]
         # and
         # * commonheads parents on missing
         revset = unfi.set(
             b'%ln and parents(roots(%ln))',
             self.outgoing.commonheads,
             self.outgoing.missing,
         )
         cheads.extend(c.node() for c in revset)
         return cheads
 
     @property
     def commonheads(self):
         """set of all common heads after changeset bundle push"""
         if self.cgresult:
             return self.futureheads
         else:
             return self.fallbackheads
 
 
 # mapping of message used when pushing bookmark
 bookmsgmap = {
     b'update': (
         _(b"updating bookmark %s\n"),
         _(b'updating bookmark %s failed\n'),
     ),
     b'export': (
         _(b"exporting bookmark %s\n"),
         _(b'exporting bookmark %s failed\n'),
     ),
     b'delete': (
         _(b"deleting remote bookmark %s\n"),
         _(b'deleting remote bookmark %s failed\n'),
     ),
 }
 
 
 def push(
     repo,
     remote,
     force=False,
     revs=None,
     newbranch=False,
     bookmarks=(),
     publish=False,
     opargs=None,
 ):
     """Push outgoing changesets (limited by revs) from a local
     repository to remote. Return an integer:
       - None means nothing to push
       - 0 means HTTP error
       - 1 means we pushed and remote head count is unchanged *or*
         we have outgoing changesets but refused to push
       - other values as described by addchangegroup()
     """
     if opargs is None:
         opargs = {}
     pushop = pushoperation(
         repo,
         remote,
         force,
         revs,
         newbranch,
         bookmarks,
         publish,
         **pycompat.strkwargs(opargs)
     )
     if pushop.remote.local():
         missing = (
             set(pushop.repo.requirements) - pushop.remote.local().supported
         )
         if missing:
             msg = _(
                 b"required features are not"
                 b" supported in the destination:"
                 b" %s"
             ) % (b', '.join(sorted(missing)))
             raise error.Abort(msg)
 
     if not pushop.remote.canpush():
         raise error.Abort(_(b"destination does not support push"))
 
     if not pushop.remote.capable(b'unbundle'):
         raise error.Abort(
             _(
                 b'cannot push: destination does not support the '
                 b'unbundle wire protocol command'
             )
         )
     for category in sorted(bundle2.read_remote_wanted_sidedata(pushop.remote)):
         # Check that a computer is registered for that category for at least
         # one revlog kind.
         for kind, computers in repo._sidedata_computers.items():
             if computers.get(category):
                 break
         else:
             raise error.Abort(
                 _(
                     b'cannot push: required sidedata category not supported'
                     b" by this client: '%s'"
                 )
                 % pycompat.bytestr(category)
             )
     # get lock as we might write phase data
     wlock = lock = None
     try:
         # bundle2 push may receive a reply bundle touching bookmarks
         # requiring the wlock. Take it now to ensure proper ordering.
         maypushback = pushop.ui.configbool(b'experimental', b'bundle2.pushback')
         if (
             (not _forcebundle1(pushop))
             and maypushback
             and not bookmod.bookmarksinstore(repo)
         ):
             wlock = pushop.repo.wlock()
         lock = pushop.repo.lock()
         pushop.trmanager = transactionmanager(
             pushop.repo, b'push-response', pushop.remote.url()
         )
     except error.LockUnavailable as err:
         # source repo cannot be locked.
         # We do not abort the push, but just disable the local phase
         # synchronisation.
         msg = b'cannot lock source repository: %s\n' % stringutil.forcebytestr(
             err
         )
         pushop.ui.debug(msg)
 
     with wlock or util.nullcontextmanager():
         with lock or util.nullcontextmanager():
             with pushop.trmanager or util.nullcontextmanager():
                 pushop.repo.checkpush(pushop)
                 _checkpublish(pushop)
                 _pushdiscovery(pushop)
                 if not pushop.force:
                     _checksubrepostate(pushop)
                 if not _forcebundle1(pushop):
                     _pushbundle2(pushop)
                 _pushchangeset(pushop)
                 _pushsyncphase(pushop)
                 _pushobsolete(pushop)
                 _pushbookmark(pushop)
 
     if repo.ui.configbool(b'experimental', b'remotenames'):
         logexchange.pullremotenames(repo, remote)
 
     return pushop
 
 
 # list of steps to perform discovery before push
 pushdiscoveryorder = []
 
 # Mapping between step name and function
 #
 # This exists to help extensions wrap steps if necessary
 pushdiscoverymapping = {}
 
 
 def pushdiscovery(stepname):
     """decorator for function performing discovery before push
 
     The function is added to the step -> function mapping and appended to the
     list of steps.  Beware that decorated function will be added in order (this
     may matter).
 
     You can only use this decorator for a new step, if you want to wrap a step
     from an extension, change the pushdiscovery dictionary directly."""
 
     def dec(func):
         assert stepname not in pushdiscoverymapping
         pushdiscoverymapping[stepname] = func
         pushdiscoveryorder.append(stepname)
         return func
 
     return dec
 
 
 def _pushdiscovery(pushop):
     """Run all discovery steps"""
     for stepname in pushdiscoveryorder:
         step = pushdiscoverymapping[stepname]
         step(pushop)
 
 
 def _checksubrepostate(pushop):
     """Ensure all outgoing referenced subrepo revisions are present locally"""
 
     repo = pushop.repo
 
     # If the repository does not use subrepos, skip the expensive
     # manifest checks.
     if not len(repo.file(b'.hgsub')) or not len(repo.file(b'.hgsubstate')):
         return
 
     for n in pushop.outgoing.missing:
         ctx = repo[n]
 
         if b'.hgsub' in ctx.manifest() and b'.hgsubstate' in ctx.files():
             for subpath in sorted(ctx.substate):
                 sub = ctx.sub(subpath)
                 sub.verify(onpush=True)
 
 
 @pushdiscovery(b'changeset')
 def _pushdiscoverychangeset(pushop):
     """discover the changeset that need to be pushed"""
     fci = discovery.findcommonincoming
     if pushop.revs:
         commoninc = fci(
             pushop.repo,
             pushop.remote,
             force=pushop.force,
             ancestorsof=pushop.revs,
         )
     else:
         commoninc = fci(pushop.repo, pushop.remote, force=pushop.force)
     common, inc, remoteheads = commoninc
     fco = discovery.findcommonoutgoing
     outgoing = fco(
         pushop.repo,
         pushop.remote,
         onlyheads=pushop.revs,
         commoninc=commoninc,
         force=pushop.force,
     )
     pushop.outgoing = outgoing
     pushop.remoteheads = remoteheads
     pushop.incoming = inc
 
 
 @pushdiscovery(b'phase')
 def _pushdiscoveryphase(pushop):
     """discover the phase that needs to be pushed
 
     (computed for both success and failure case for changesets push)"""
     outgoing = pushop.outgoing
     unfi = pushop.repo.unfiltered()
     remotephases = listkeys(pushop.remote, b'phases')
 
     if (
         pushop.ui.configbool(b'ui', b'_usedassubrepo')
         and remotephases  # server supports phases
         and not pushop.outgoing.missing  # no changesets to be pushed
         and remotephases.get(b'publishing', False)
     ):
         # When:
         # - this is a subrepo push
         # - and remote support phase
         # - and no changeset are to be pushed
         # - and remote is publishing
         # We may be in issue 3781 case!
         # We drop the possible phase synchronisation done by
         # courtesy to publish changesets possibly locally draft
         # on the remote.
         pushop.outdatedphases = []
         pushop.fallbackoutdatedphases = []
         return
 
     pushop.remotephases = phases.remotephasessummary(
         pushop.repo, pushop.fallbackheads, remotephases
     )
     droots = pushop.remotephases.draftroots
 
     extracond = b''
     if not pushop.remotephases.publishing:
         extracond = b' and public()'
     revset = b'heads((%%ln::%%ln) %s)' % extracond
     # Get the list of all revs draft on remote by public here.
     # XXX Beware that revset break if droots is not strictly
     # XXX root we may want to ensure it is but it is costly
     fallback = list(unfi.set(revset, droots, pushop.fallbackheads))
     if not pushop.remotephases.publishing and pushop.publish:
         future = list(
             unfi.set(
                 b'%ln and (not public() or %ln::)', pushop.futureheads, droots
             )
         )
     elif not outgoing.missing:
         future = fallback
     else:
         # adds changeset we are going to push as draft
         #
         # should not be necessary for publishing server, but because of an
         # issue fixed in xxxxx we have to do it anyway.
         fdroots = list(
             unfi.set(b'roots(%ln  + %ln::)', outgoing.missing, droots)
         )
         fdroots = [f.node() for f in fdroots]
         future = list(unfi.set(revset, fdroots, pushop.futureheads))
     pushop.outdatedphases = future
     pushop.fallbackoutdatedphases = fallback
 
 
 @pushdiscovery(b'obsmarker')
 def _pushdiscoveryobsmarkers(pushop):
     if not obsolete.isenabled(pushop.repo, obsolete.exchangeopt):
         return
 
     if not pushop.repo.obsstore:
         return
 
     if b'obsolete' not in listkeys(pushop.remote, b'namespaces'):
         return
 
     repo = pushop.repo
     # very naive computation, that can be quite expensive on big repo.
     # However: evolution is currently slow on them anyway.
     nodes = (c.node() for c in repo.set(b'::%ln', pushop.futureheads))
     pushop.outobsmarkers = pushop.repo.obsstore.relevantmarkers(nodes)
 
 
 @pushdiscovery(b'bookmarks')
 def _pushdiscoverybookmarks(pushop):
     ui = pushop.ui
     repo = pushop.repo.unfiltered()
     remote = pushop.remote
     ui.debug(b"checking for updated bookmarks\n")
     ancestors = ()
     if pushop.revs:
         revnums = pycompat.maplist(repo.changelog.rev, pushop.revs)
         ancestors = repo.changelog.ancestors(revnums, inclusive=True)
 
     remotebookmark = bookmod.unhexlifybookmarks(listkeys(remote, b'bookmarks'))
 
     explicit = {
         repo._bookmarks.expandname(bookmark) for bookmark in pushop.bookmarks
     }
 
     comp = bookmod.comparebookmarks(repo, repo._bookmarks, remotebookmark)
     return _processcompared(pushop, ancestors, explicit, remotebookmark, comp)
 
 
 def _processcompared(pushop, pushed, explicit, remotebms, comp):
     """take decision on bookmarks to push to the remote repo
 
     Exists to help extensions alter this behavior.
     """
     addsrc, adddst, advsrc, advdst, diverge, differ, invalid, same = comp
 
     repo = pushop.repo
 
     for b, scid, dcid in advsrc:
         if b in explicit:
             explicit.remove(b)
         if not pushed or repo[scid].rev() in pushed:
             pushop.outbookmarks.append((b, dcid, scid))
     # search added bookmark
     for b, scid, dcid in addsrc:
         if b in explicit:
             explicit.remove(b)
             if bookmod.isdivergent(b):
                 pushop.ui.warn(_(b'cannot push divergent bookmark %s!\n') % b)
                 pushop.bkresult = 2
             else:
                 pushop.outbookmarks.append((b, b'', scid))
     # search for overwritten bookmark
     for b, scid, dcid in list(advdst) + list(diverge) + list(differ):
         if b in explicit:
             explicit.remove(b)
             pushop.outbookmarks.append((b, dcid, scid))
     # search for bookmark to delete
     for b, scid, dcid in adddst:
         if b in explicit:
             explicit.remove(b)
             # treat as "deleted locally"
             pushop.outbookmarks.append((b, dcid, b''))
     # identical bookmarks shouldn't get reported
     for b, scid, dcid in same:
         if b in explicit:
             explicit.remove(b)
 
     if explicit:
         explicit = sorted(explicit)
         # we should probably list all of them
         pushop.ui.warn(
             _(
                 b'bookmark %s does not exist on the local '
                 b'or remote repository!\n'
             )
             % explicit[0]
         )
         pushop.bkresult = 2
 
     pushop.outbookmarks.sort()
 
 
 def _pushcheckoutgoing(pushop):
     outgoing = pushop.outgoing
     unfi = pushop.repo.unfiltered()
     if not outgoing.missing:
         # nothing to push
         scmutil.nochangesfound(unfi.ui, unfi, outgoing.excluded)
         return False
     # something to push
     if not pushop.force:
         # if repo.obsstore == False --> no obsolete
         # then, save the iteration
         if unfi.obsstore:
             # this message are here for 80 char limit reason
             mso = _(b"push includes obsolete changeset: %s!")
             mspd = _(b"push includes phase-divergent changeset: %s!")
             mscd = _(b"push includes content-divergent changeset: %s!")
             mst = {
                 b"orphan": _(b"push includes orphan changeset: %s!"),
                 b"phase-divergent": mspd,
                 b"content-divergent": mscd,
             }
             # If we are to push if there is at least one
             # obsolete or unstable changeset in missing, at
             # least one of the missinghead will be obsolete or
             # unstable. So checking heads only is ok
             for node in outgoing.ancestorsof:
                 ctx = unfi[node]
                 if ctx.obsolete():
                     raise error.Abort(mso % ctx)
                 elif ctx.isunstable():
                     # TODO print more than one instability in the abort
                     # message
                     raise error.Abort(mst[ctx.instabilities()[0]] % ctx)
 
         discovery.checkheads(pushop)
     return True
 
 
 # List of names of steps to perform for an outgoing bundle2, order matters.
 b2partsgenorder = []
 
 # Mapping between step name and function
 #
 # This exists to help extensions wrap steps if necessary
 b2partsgenmapping = {}
 
 
 def b2partsgenerator(stepname, idx=None):
     """decorator for function generating bundle2 part
 
     The function is added to the step -> function mapping and appended to the
     list of steps.  Beware that decorated functions will be added in order
     (this may matter).
 
     You can only use this decorator for new steps, if you want to wrap a step
     from an extension, attack the b2partsgenmapping dictionary directly."""
 
     def dec(func):
         assert stepname not in b2partsgenmapping
         b2partsgenmapping[stepname] = func
         if idx is None:
             b2partsgenorder.append(stepname)
         else:
             b2partsgenorder.insert(idx, stepname)
         return func
 
     return dec
 
 
 def _pushb2ctxcheckheads(pushop, bundler):
     """Generate race condition checking parts
 
     Exists as an independent function to aid extensions
     """
     # * 'force' do not check for push race,
     # * if we don't push anything, there are nothing to check.
     if not pushop.force and pushop.outgoing.ancestorsof:
         allowunrelated = b'related' in bundler.capabilities.get(
             b'checkheads', ()
         )
         emptyremote = pushop.pushbranchmap is None
         if not allowunrelated or emptyremote:
             bundler.newpart(b'check:heads', data=iter(pushop.remoteheads))
         else:
             affected = set()
             for branch, heads in pushop.pushbranchmap.items():
                 remoteheads, newheads, unsyncedheads, discardedheads = heads
                 if remoteheads is not None:
                     remote = set(remoteheads)
                     affected |= set(discardedheads) & remote
                     affected |= remote - set(newheads)
             if affected:
                 data = iter(sorted(affected))
                 bundler.newpart(b'check:updated-heads', data=data)
 
 
 def _pushing(pushop):
     """return True if we are pushing anything"""
     return bool(
         pushop.outgoing.missing
         or pushop.outdatedphases
         or pushop.outobsmarkers
         or pushop.outbookmarks
     )
 
 
 @b2partsgenerator(b'check-bookmarks')
 def _pushb2checkbookmarks(pushop, bundler):
     """insert bookmark move checking"""
     if not _pushing(pushop) or pushop.force:
         return
     b2caps = bundle2.bundle2caps(pushop.remote)
     hasbookmarkcheck = b'bookmarks' in b2caps
     if not (pushop.outbookmarks and hasbookmarkcheck):
         return
     data = []
     for book, old, new in pushop.outbookmarks:
         data.append((book, old))
     checkdata = bookmod.binaryencode(pushop.repo, data)
     bundler.newpart(b'check:bookmarks', data=checkdata)
 
 
 @b2partsgenerator(b'check-phases')
 def _pushb2checkphases(pushop, bundler):
     """insert phase move checking"""
     if not _pushing(pushop) or pushop.force:
         return
     b2caps = bundle2.bundle2caps(pushop.remote)
     hasphaseheads = b'heads' in b2caps.get(b'phases', ())
     if pushop.remotephases is not None and hasphaseheads:
         # check that the remote phase has not changed
         checks = {p: [] for p in phases.allphases}
         checks[phases.public].extend(pushop.remotephases.publicheads)
         checks[phases.draft].extend(pushop.remotephases.draftroots)
         if any(checks.values()):
             for phase in checks:
                 checks[phase].sort()
             checkdata = phases.binaryencode(checks)
             bundler.newpart(b'check:phases', data=checkdata)
 
 
 @b2partsgenerator(b'changeset')
 def _pushb2ctx(pushop, bundler):
     """handle changegroup push through bundle2
 
     addchangegroup result is stored in the ``pushop.cgresult`` attribute.
     """
     if b'changesets' in pushop.stepsdone:
         return
     pushop.stepsdone.add(b'changesets')
     # Send known heads to the server for race detection.
     if not _pushcheckoutgoing(pushop):
         return
     pushop.repo.prepushoutgoinghooks(pushop)
 
     _pushb2ctxcheckheads(pushop, bundler)
 
     b2caps = bundle2.bundle2caps(pushop.remote)
     version = b'01'
     cgversions = b2caps.get(b'changegroup')
     if cgversions:  # 3.1 and 3.2 ship with an empty value
         cgversions = [
             v
             for v in cgversions
             if v in changegroup.supportedoutgoingversions(pushop.repo)
         ]
         if not cgversions:
             raise error.Abort(_(b'no common changegroup version'))
         version = max(cgversions)
 
     remote_sidedata = bundle2.read_remote_wanted_sidedata(pushop.remote)
     cgstream = changegroup.makestream(
         pushop.repo,
         pushop.outgoing,
         version,
         b'push',
         bundlecaps=b2caps,
         remote_sidedata=remote_sidedata,
     )
     cgpart = bundler.newpart(b'changegroup', data=cgstream)
     if cgversions:
         cgpart.addparam(b'version', version)
     if scmutil.istreemanifest(pushop.repo):
         cgpart.addparam(b'treemanifest', b'1')
     if repository.REPO_FEATURE_SIDE_DATA in pushop.repo.features:
         cgpart.addparam(b'exp-sidedata', b'1')
 
     def handlereply(op):
         """extract addchangegroup returns from server reply"""
         cgreplies = op.records.getreplies(cgpart.id)
         assert len(cgreplies[b'changegroup']) == 1
         pushop.cgresult = cgreplies[b'changegroup'][0][b'return']
 
     return handlereply
 
 
 @b2partsgenerator(b'phase')
 def _pushb2phases(pushop, bundler):
     """handle phase push through bundle2"""
     if b'phases' in pushop.stepsdone:
         return
     b2caps = bundle2.bundle2caps(pushop.remote)
     ui = pushop.repo.ui
 
     legacyphase = b'phases' in ui.configlist(b'devel', b'legacy.exchange')
     haspushkey = b'pushkey' in b2caps
     hasphaseheads = b'heads' in b2caps.get(b'phases', ())
 
     if hasphaseheads and not legacyphase:
         return _pushb2phaseheads(pushop, bundler)
     elif haspushkey:
         return _pushb2phasespushkey(pushop, bundler)
 
 
 def _pushb2phaseheads(pushop, bundler):
     """push phase information through a bundle2 - binary part"""
     pushop.stepsdone.add(b'phases')
     if pushop.outdatedphases:
         updates = {p: [] for p in phases.allphases}
         updates[0].extend(h.node() for h in pushop.outdatedphases)
         phasedata = phases.binaryencode(updates)
         bundler.newpart(b'phase-heads', data=phasedata)
 
 
 def _pushb2phasespushkey(pushop, bundler):
     """push phase information through a bundle2 - pushkey part"""
     pushop.stepsdone.add(b'phases')
     part2node = []
 
     def handlefailure(pushop, exc):
         targetid = int(exc.partid)
         for partid, node in part2node:
             if partid == targetid:
                 raise error.Abort(_(b'updating %s to public failed') % node)
 
     enc = pushkey.encode
     for newremotehead in pushop.outdatedphases:
         part = bundler.newpart(b'pushkey')
         part.addparam(b'namespace', enc(b'phases'))
         part.addparam(b'key', enc(newremotehead.hex()))
         part.addparam(b'old', enc(b'%d' % phases.draft))
         part.addparam(b'new', enc(b'%d' % phases.public))
         part2node.append((part.id, newremotehead))
         pushop.pkfailcb[part.id] = handlefailure
 
     def handlereply(op):
         for partid, node in part2node:
             partrep = op.records.getreplies(partid)
             results = partrep[b'pushkey']
             assert len(results) <= 1
             msg = None
             if not results:
                 msg = _(b'server ignored update of %s to public!\n') % node
             elif not int(results[0][b'return']):
                 msg = _(b'updating %s to public failed!\n') % node
             if msg is not None:
                 pushop.ui.warn(msg)
 
     return handlereply
 
 
 @b2partsgenerator(b'obsmarkers')
 def _pushb2obsmarkers(pushop, bundler):
     if b'obsmarkers' in pushop.stepsdone:
         return
     remoteversions = bundle2.obsmarkersversion(bundler.capabilities)
     if obsolete.commonversion(remoteversions) is None:
         return
     pushop.stepsdone.add(b'obsmarkers')
     if pushop.outobsmarkers:
         markers = obsutil.sortedmarkers(pushop.outobsmarkers)
         bundle2.buildobsmarkerspart(bundler, markers)
 
 
 @b2partsgenerator(b'bookmarks')
 def _pushb2bookmarks(pushop, bundler):
     """handle bookmark push through bundle2"""
     if b'bookmarks' in pushop.stepsdone:
         return
     b2caps = bundle2.bundle2caps(pushop.remote)
 
     legacy = pushop.repo.ui.configlist(b'devel', b'legacy.exchange')
     legacybooks = b'bookmarks' in legacy
 
     if not legacybooks and b'bookmarks' in b2caps:
         return _pushb2bookmarkspart(pushop, bundler)
     elif b'pushkey' in b2caps:
         return _pushb2bookmarkspushkey(pushop, bundler)
 
 
 def _bmaction(old, new):
     """small utility for bookmark pushing"""
     if not old:
         return b'export'
     elif not new:
         return b'delete'
     return b'update'
 
 
 def _abortonsecretctx(pushop, node, b):
     """abort if a given bookmark points to a secret changeset"""
     if node and pushop.repo[node].phase() == phases.secret:
         raise error.Abort(
             _(b'cannot push bookmark %s as it points to a secret changeset') % b
         )
 
 
 def _pushb2bookmarkspart(pushop, bundler):
     pushop.stepsdone.add(b'bookmarks')
     if not pushop.outbookmarks:
         return
 
     allactions = []
     data = []
     for book, old, new in pushop.outbookmarks:
         _abortonsecretctx(pushop, new, book)
         data.append((book, new))
         allactions.append((book, _bmaction(old, new)))
     checkdata = bookmod.binaryencode(pushop.repo, data)
     bundler.newpart(b'bookmarks', data=checkdata)
 
     def handlereply(op):
         ui = pushop.ui
         # if success
         for book, action in allactions:
             ui.status(bookmsgmap[action][0] % book)
 
     return handlereply
 
 
 def _pushb2bookmarkspushkey(pushop, bundler):
     pushop.stepsdone.add(b'bookmarks')
     part2book = []
     enc = pushkey.encode
 
     def handlefailure(pushop, exc):
         targetid = int(exc.partid)
         for partid, book, action in part2book:
             if partid == targetid:
                 raise error.Abort(bookmsgmap[action][1].rstrip() % book)
         # we should not be called for part we did not generated
         assert False
 
     for book, old, new in pushop.outbookmarks:
         _abortonsecretctx(pushop, new, book)
         part = bundler.newpart(b'pushkey')
         part.addparam(b'namespace', enc(b'bookmarks'))
         part.addparam(b'key', enc(book))
         part.addparam(b'old', enc(hex(old)))
         part.addparam(b'new', enc(hex(new)))
         action = b'update'
         if not old:
             action = b'export'
         elif not new:
             action = b'delete'
         part2book.append((part.id, book, action))
         pushop.pkfailcb[part.id] = handlefailure
 
     def handlereply(op):
         ui = pushop.ui
         for partid, book, action in part2book:
             partrep = op.records.getreplies(partid)
             results = partrep[b'pushkey']
             assert len(results) <= 1
             if not results:
                 pushop.ui.warn(_(b'server ignored bookmark %s update\n') % book)
             else:
                 ret = int(results[0][b'return'])
                 if ret:
                     ui.status(bookmsgmap[action][0] % book)
                 else:
                     ui.warn(bookmsgmap[action][1] % book)
                     if pushop.bkresult is not None:
                         pushop.bkresult = 1
 
     return handlereply
 
 
 @b2partsgenerator(b'pushvars', idx=0)
 def _getbundlesendvars(pushop, bundler):
     '''send shellvars via bundle2'''
     pushvars = pushop.pushvars
     if pushvars:
         shellvars = {}
         for raw in pushvars:
             if b'=' not in raw:
                 msg = (
                     b"unable to parse variable '%s', should follow "
                     b"'KEY=VALUE' or 'KEY=' format"
                 )
                 raise error.Abort(msg % raw)
             k, v = raw.split(b'=', 1)
             shellvars[k] = v
 
         part = bundler.newpart(b'pushvars')
 
         for key, value in shellvars.items():
             part.addparam(key, value, mandatory=False)
 
 
 def _pushbundle2(pushop):
     """push data to the remote using bundle2
 
     The only currently supported type of data is changegroup but this will
     evolve in the future."""
     bundler = bundle2.bundle20(pushop.ui, bundle2.bundle2caps(pushop.remote))
     pushback = pushop.trmanager and pushop.ui.configbool(
         b'experimental', b'bundle2.pushback'
     )
 
     # create reply capability
     capsblob = bundle2.encodecaps(
         bundle2.getrepocaps(pushop.repo, allowpushback=pushback, role=b'client')
     )
     bundler.newpart(b'replycaps', data=capsblob)
     replyhandlers = []
     for partgenname in b2partsgenorder:
         partgen = b2partsgenmapping[partgenname]
         ret = partgen(pushop, bundler)
         if callable(ret):
             replyhandlers.append(ret)
     # do not push if nothing to push
     if bundler.nbparts <= 1:
         return
     stream = util.chunkbuffer(bundler.getchunks())
     try:
         try:
             with pushop.remote.commandexecutor() as e:
                 reply = e.callcommand(
                     b'unbundle',
                     {
                         b'bundle': stream,
                         b'heads': [b'force'],
                         b'url': pushop.remote.url(),
                     },
                 ).result()
         except error.BundleValueError as exc:
             raise error.RemoteError(_(b'missing support for %s') % exc)
         try:
             trgetter = None
             if pushback:
                 trgetter = pushop.trmanager.transaction
             op = bundle2.processbundle(
                 pushop.repo,
                 reply,
                 trgetter,
                 remote=pushop.remote,
             )
         except error.BundleValueError as exc:
             raise error.RemoteError(_(b'missing support for %s') % exc)
         except bundle2.AbortFromPart as exc:
             pushop.ui.error(_(b'remote: %s\n') % exc)
             if exc.hint is not None:
                 pushop.ui.error(_(b'remote: %s\n') % (b'(%s)' % exc.hint))
             raise error.RemoteError(_(b'push failed on remote'))
     except error.PushkeyFailed as exc:
         partid = int(exc.partid)
         if partid not in pushop.pkfailcb:
             raise
         pushop.pkfailcb[partid](pushop, exc)
     for rephand in replyhandlers:
         rephand(op)
 
 
 def _pushchangeset(pushop):
     """Make the actual push of changeset bundle to remote repo"""
     if b'changesets' in pushop.stepsdone:
         return
     pushop.stepsdone.add(b'changesets')
     if not _pushcheckoutgoing(pushop):
         return
 
     # Should have verified this in push().
     assert pushop.remote.capable(b'unbundle')
 
     pushop.repo.prepushoutgoinghooks(pushop)
     outgoing = pushop.outgoing
     # TODO: get bundlecaps from remote
     bundlecaps = None
     # create a changegroup from local
     if pushop.revs is None and not (
         outgoing.excluded or pushop.repo.changelog.filteredrevs
     ):
         # push everything,
         # use the fast path, no race possible on push
         cg = changegroup.makechangegroup(
             pushop.repo,
             outgoing,
             b'01',
             b'push',
             fastpath=True,
             bundlecaps=bundlecaps,
         )
     else:
         cg = changegroup.makechangegroup(
             pushop.repo, outgoing, b'01', b'push', bundlecaps=bundlecaps
         )
 
     # apply changegroup to remote
     # local repo finds heads on server, finds out what
     # revs it must push. once revs transferred, if server
     # finds it has different heads (someone else won
     # commit/push race), server aborts.
     if pushop.force:
         remoteheads = [b'force']
     else:
         remoteheads = pushop.remoteheads
     # ssh: return remote's addchangegroup()
     # http: return remote's addchangegroup() or 0 for error
     pushop.cgresult = pushop.remote.unbundle(cg, remoteheads, pushop.repo.url())
 
 
 def _pushsyncphase(pushop):
     """synchronise phase information locally and remotely"""
     cheads = pushop.commonheads
     # even when we don't push, exchanging phase data is useful
     remotephases = listkeys(pushop.remote, b'phases')
     if (
         pushop.ui.configbool(b'ui', b'_usedassubrepo')
         and remotephases  # server supports phases
         and pushop.cgresult is None  # nothing was pushed
         and remotephases.get(b'publishing', False)
     ):
         # When:
         # - this is a subrepo push
         # - and remote support phase
         # - and no changeset was pushed
         # - and remote is publishing
         # We may be in issue 3871 case!
         # We drop the possible phase synchronisation done by
         # courtesy to publish changesets possibly locally draft
         # on the remote.
         remotephases = {b'publishing': b'True'}
     if not remotephases:  # old server or public only reply from non-publishing
         _localphasemove(pushop, cheads)
         # don't push any phase data as there is nothing to push
     else:
         ana = phases.analyzeremotephases(pushop.repo, cheads, remotephases)
         pheads, droots = ana
         ### Apply remote phase on local
         if remotephases.get(b'publishing', False):
             _localphasemove(pushop, cheads)
         else:  # publish = False
             _localphasemove(pushop, pheads)
             _localphasemove(pushop, cheads, phases.draft)
         ### Apply local phase on remote
 
         if pushop.cgresult:
             if b'phases' in pushop.stepsdone:
                 # phases already pushed though bundle2
                 return
             outdated = pushop.outdatedphases
         else:
             outdated = pushop.fallbackoutdatedphases
 
         pushop.stepsdone.add(b'phases')
 
         # filter heads already turned public by the push
         outdated = [c for c in outdated if c.node() not in pheads]
         # fallback to independent pushkey command
         for newremotehead in outdated:
             with pushop.remote.commandexecutor() as e:
                 r = e.callcommand(
                     b'pushkey',
                     {
                         b'namespace': b'phases',
                         b'key': newremotehead.hex(),
                         b'old': b'%d' % phases.draft,
                         b'new': b'%d' % phases.public,
                     },
                 ).result()
 
             if not r:
                 pushop.ui.warn(
                     _(b'updating %s to public failed!\n') % newremotehead
                 )
 
 
 def _localphasemove(pushop, nodes, phase=phases.public):
     """move <nodes> to <phase> in the local source repo"""
     if pushop.trmanager:
         phases.advanceboundary(
             pushop.repo, pushop.trmanager.transaction(), phase, nodes
         )
     else:
         # repo is not locked, do not change any phases!
         # Informs the user that phases should have been moved when
         # applicable.
         actualmoves = [n for n in nodes if phase < pushop.repo[n].phase()]
         phasestr = phases.phasenames[phase]
         if actualmoves:
             pushop.ui.status(
                 _(
                     b'cannot lock source repo, skipping '
                     b'local %s phase update\n'
                 )
                 % phasestr
             )
 
 
 def _pushobsolete(pushop):
     """utility function to push obsolete markers to a remote"""
     if b'obsmarkers' in pushop.stepsdone:
         return
     repo = pushop.repo
     remote = pushop.remote
     pushop.stepsdone.add(b'obsmarkers')
     if pushop.outobsmarkers:
         pushop.ui.debug(b'try to push obsolete markers to remote\n')
         rslts = []
         markers = obsutil.sortedmarkers(pushop.outobsmarkers)
         remotedata = obsolete._pushkeyescape(markers)
         for key in sorted(remotedata, reverse=True):
             # reverse sort to ensure we end with dump0
             data = remotedata[key]
             rslts.append(remote.pushkey(b'obsolete', key, b'', data))
         if [r for r in rslts if not r]:
             msg = _(b'failed to push some obsolete markers!\n')
             repo.ui.warn(msg)
 
 
 def _pushbookmark(pushop):
     """Update bookmark position on remote"""
     if pushop.cgresult == 0 or b'bookmarks' in pushop.stepsdone:
         return
     pushop.stepsdone.add(b'bookmarks')
     ui = pushop.ui
     remote = pushop.remote
 
     for b, old, new in pushop.outbookmarks:
         action = b'update'
         if not old:
             action = b'export'
         elif not new:
             action = b'delete'
 
         with remote.commandexecutor() as e:
             r = e.callcommand(
                 b'pushkey',
                 {
                     b'namespace': b'bookmarks',
                     b'key': b,
                     b'old': hex(old),
                     b'new': hex(new),
                 },
             ).result()
 
         if r:
             ui.status(bookmsgmap[action][0] % b)
         else:
             ui.warn(bookmsgmap[action][1] % b)
             # discovery can have set the value form invalid entry
             if pushop.bkresult is not None:
                 pushop.bkresult = 1
 
 
 class pulloperation:
     """A object that represent a single pull operation
 
     It purpose is to carry pull related state and very common operation.
 
     A new should be created at the beginning of each pull and discarded
     afterward.
     """
 
     def __init__(
         self,
         repo,
         remote,
         heads=None,
         force=False,
         bookmarks=(),
         remotebookmarks=None,
         streamclonerequested=None,
         includepats=None,
         excludepats=None,
         depth=None,
         path=None,
     ):
         # repo we pull into
         self.repo = repo
         # repo we pull from
         self.remote = remote
         # path object used to build this remote
         #
         # Ideally, the remote peer would carry that directly.
         self.remote_path = path
         # revision we try to pull (None is "all")
         self.heads = heads
         # bookmark pulled explicitly
         self.explicitbookmarks = [
             repo._bookmarks.expandname(bookmark) for bookmark in bookmarks
         ]
         # do we force pull?
         self.force = force
         # whether a streaming clone was requested
         self.streamclonerequested = streamclonerequested
         # transaction manager
         self.trmanager = None
         # set of common changeset between local and remote before pull
         self.common = None
         # set of pulled head
         self.rheads = None
         # list of missing changeset to fetch remotely
         self.fetch = None
         # remote bookmarks data
         self.remotebookmarks = remotebookmarks
         # result of changegroup pulling (used as return code by pull)
         self.cgresult = None
         # list of step already done
         self.stepsdone = set()
         # Whether we attempted a clone from pre-generated bundles.
         self.clonebundleattempted = False
         # Set of file patterns to include.
         self.includepats = includepats
         # Set of file patterns to exclude.
         self.excludepats = excludepats
         # Number of ancestor changesets to pull from each pulled head.
         self.depth = depth
 
     @util.propertycache
     def pulledsubset(self):
         """heads of the set of changeset target by the pull"""
         # compute target subset
         if self.heads is None:
             # We pulled every thing possible
             # sync on everything common
             c = set(self.common)
             ret = list(self.common)
             for n in self.rheads:
                 if n not in c:
                     ret.append(n)
             return ret
         else:
             # We pulled a specific subset
             # sync on this subset
             return self.heads
 
     @util.propertycache
     def canusebundle2(self):
         return not _forcebundle1(self)
 
     @util.propertycache
     def remotebundle2caps(self):
         return bundle2.bundle2caps(self.remote)
 
     def gettransaction(self):
         # deprecated; talk to trmanager directly
         return self.trmanager.transaction()
 
 
 class transactionmanager(util.transactional):
     """An object to manage the life cycle of a transaction
 
     It creates the transaction on demand and calls the appropriate hooks when
     closing the transaction."""
 
     def __init__(self, repo, source, url):
         self.repo = repo
         self.source = source
         self.url = url
         self._tr = None
 
     def transaction(self):
         """Return an open transaction object, constructing if necessary"""
         if not self._tr:
             trname = b'%s\n%s' % (self.source, urlutil.hidepassword(self.url))
             self._tr = self.repo.transaction(trname)
             self._tr.hookargs[b'source'] = self.source
             self._tr.hookargs[b'url'] = self.url
         return self._tr
 
     def close(self):
         """close transaction if created"""
         if self._tr is not None:
             self._tr.close()
 
     def release(self):
         """release transaction if created"""
         if self._tr is not None:
             self._tr.release()
 
 
 def listkeys(remote, namespace):
     with remote.commandexecutor() as e:
         return e.callcommand(b'listkeys', {b'namespace': namespace}).result()
 
 
 def _fullpullbundle2(repo, pullop):
     # The server may send a partial reply, i.e. when inlining
     # pre-computed bundles. In that case, update the common
     # set based on the results and pull another bundle.
     #
     # There are two indicators that the process is finished:
     # - no changeset has been added, or
     # - all remote heads are known locally.
     # The head check must use the unfiltered view as obsoletion
     # markers can hide heads.
     unfi = repo.unfiltered()
     unficl = unfi.changelog
 
     def headsofdiff(h1, h2):
         """Returns heads(h1 % h2)"""
         res = unfi.set(b'heads(%ln %% %ln)', h1, h2)
         return {ctx.node() for ctx in res}
 
     def headsofunion(h1, h2):
         """Returns heads((h1 + h2) - null)"""
         res = unfi.set(b'heads((%ln + %ln - null))', h1, h2)
         return {ctx.node() for ctx in res}
 
     while True:
         old_heads = unficl.heads()
         clstart = len(unficl)
         _pullbundle2(pullop)
         if requirements.NARROW_REQUIREMENT in repo.requirements:
             # XXX narrow clones filter the heads on the server side during
             # XXX getbundle and result in partial replies as well.
             # XXX Disable pull bundles in this case as band aid to avoid
             # XXX extra round trips.
             break
         if clstart == len(unficl):
             break
         if all(unficl.hasnode(n) for n in pullop.rheads):
             break
         new_heads = headsofdiff(unficl.heads(), old_heads)
         pullop.common = headsofunion(new_heads, pullop.common)
         pullop.rheads = set(pullop.rheads) - pullop.common
 
 
 def add_confirm_callback(repo, pullop):
     """adds a finalize callback to transaction which can be used to show stats
     to user and confirm the pull before committing transaction"""
 
     tr = pullop.trmanager.transaction()
     scmutil.registersummarycallback(
         repo, tr, txnname=b'pull', as_validator=True
     )
     reporef = weakref.ref(repo.unfiltered())
 
     def prompt(tr):
         repo = reporef()
         cm = _(b'accept incoming changes (yn)?$$ &Yes $$ &No')
         if repo.ui.promptchoice(cm):
             raise error.Abort(b"user aborted")
 
     tr.addvalidator(b'900-pull-prompt', prompt)
 
 
 def pull(
     repo,
     remote,
     path=None,
     heads=None,
     force=False,
     bookmarks=(),
     opargs=None,
     streamclonerequested=None,
     includepats=None,
     excludepats=None,
     depth=None,
     confirm=None,
 ):
     """Fetch repository data from a remote.
 
     This is the main function used to retrieve data from a remote repository.
 
     ``repo`` is the local repository to clone into.
     ``remote`` is a peer instance.
     ``heads`` is an iterable of revisions we want to pull. ``None`` (the
     default) means to pull everything from the remote.
     ``bookmarks`` is an iterable of bookmarks requesting to be pulled. By
     default, all remote bookmarks are pulled.
     ``opargs`` are additional keyword arguments to pass to ``pulloperation``
     initialization.
     ``streamclonerequested`` is a boolean indicating whether a "streaming
     clone" is requested. A "streaming clone" is essentially a raw file copy
     of revlogs from the server. This only works when the local repository is
     empty. The default value of ``None`` means to respect the server
     configuration for preferring stream clones.
     ``includepats`` and ``excludepats`` define explicit file patterns to
     include and exclude in storage, respectively. If not defined, narrow
     patterns from the repo instance are used, if available.
     ``depth`` is an integer indicating the DAG depth of history we're
     interested in. If defined, for each revision specified in ``heads``, we
     will fetch up to this many of its ancestors and data associated with them.
     ``confirm`` is a boolean indicating whether the pull should be confirmed
     before committing the transaction. This overrides HGPLAIN.
 
     Returns the ``pulloperation`` created for this pull.
     """
     if opargs is None:
         opargs = {}
 
     # We allow the narrow patterns to be passed in explicitly to provide more
     # flexibility for API consumers.
     if includepats is not None or excludepats is not None:
         includepats = includepats or set()
         excludepats = excludepats or set()
     else:
         includepats, excludepats = repo.narrowpats
 
     narrowspec.validatepatterns(includepats)
     narrowspec.validatepatterns(excludepats)
 
     pullop = pulloperation(
         repo,
         remote,
         path=path,
         heads=heads,
         force=force,
         bookmarks=bookmarks,
         streamclonerequested=streamclonerequested,
         includepats=includepats,
         excludepats=excludepats,
         depth=depth,
         **pycompat.strkwargs(opargs)
     )
 
     peerlocal = pullop.remote.local()
     if peerlocal:
         missing = set(peerlocal.requirements) - pullop.repo.supported
         if missing:
             msg = _(
                 b"required features are not"
                 b" supported in the destination:"
                 b" %s"
             ) % (b', '.join(sorted(missing)))
             raise error.Abort(msg)
 
     for category in repo._wanted_sidedata:
         # Check that a computer is registered for that category for at least
         # one revlog kind.
         for kind, computers in repo._sidedata_computers.items():
             if computers.get(category):
                 break
         else:
             # This should never happen since repos are supposed to be able to
             # generate the sidedata they require.
             raise error.ProgrammingError(
                 _(
                     b'sidedata category requested by local side without local'
                     b"support: '%s'"
                 )
                 % pycompat.bytestr(category)
             )
 
     pullop.trmanager = transactionmanager(repo, b'pull', remote.url())
     wlock = util.nullcontextmanager()
     if not bookmod.bookmarksinstore(repo):
         wlock = repo.wlock()
     with wlock, repo.lock(), pullop.trmanager:
         if confirm or (
             repo.ui.configbool(b"pull", b"confirm") and not repo.ui.plain()
         ):
             add_confirm_callback(repo, pullop)
 
         # This should ideally be in _pullbundle2(). However, it needs to run
         # before discovery to avoid extra work.
         _maybeapplyclonebundle(pullop)
         streamclone.maybeperformlegacystreamclone(pullop)
         _pulldiscovery(pullop)
         if pullop.canusebundle2:
             _fullpullbundle2(repo, pullop)
         _pullchangeset(pullop)
         _pullphase(pullop)
         _pullbookmarks(pullop)
         _pullobsolete(pullop)
 
     # storing remotenames
     if repo.ui.configbool(b'experimental', b'remotenames'):
         logexchange.pullremotenames(repo, remote)
 
     return pullop
 
 
 # list of steps to perform discovery before pull
 pulldiscoveryorder = []
 
 # Mapping between step name and function
 #
 # This exists to help extensions wrap steps if necessary
 pulldiscoverymapping = {}
 
 
 def pulldiscovery(stepname):
     """decorator for function performing discovery before pull
 
     The function is added to the step -> function mapping and appended to the
     list of steps.  Beware that decorated function will be added in order (this
     may matter).
 
     You can only use this decorator for a new step, if you want to wrap a step
     from an extension, change the pulldiscovery dictionary directly."""
 
     def dec(func):
         assert stepname not in pulldiscoverymapping
         pulldiscoverymapping[stepname] = func
         pulldiscoveryorder.append(stepname)
         return func
 
     return dec
 
 
 def _pulldiscovery(pullop):
     """Run all discovery steps"""
     for stepname in pulldiscoveryorder:
         step = pulldiscoverymapping[stepname]
         step(pullop)
 
 
 @pulldiscovery(b'b1:bookmarks')
 def _pullbookmarkbundle1(pullop):
     """fetch bookmark data in bundle1 case
 
     If not using bundle2, we have to fetch bookmarks before changeset
     discovery to reduce the chance and impact of race conditions."""
     if pullop.remotebookmarks is not None:
         return
     if pullop.canusebundle2 and b'listkeys' in pullop.remotebundle2caps:
         # all known bundle2 servers now support listkeys, but lets be nice with
         # new implementation.
         return
     books = listkeys(pullop.remote, b'bookmarks')
     pullop.remotebookmarks = bookmod.unhexlifybookmarks(books)
 
 
 @pulldiscovery(b'changegroup')
 def _pulldiscoverychangegroup(pullop):
     """discovery phase for the pull
 
     Current handle changeset discovery only, will change handle all discovery
     at some point."""
     tmp = discovery.findcommonincoming(
         pullop.repo, pullop.remote, heads=pullop.heads, force=pullop.force
     )
     common, fetch, rheads = tmp
     has_node = pullop.repo.unfiltered().changelog.index.has_node
     if fetch and rheads:
         # If a remote heads is filtered locally, put in back in common.
         #
         # This is a hackish solution to catch most of "common but locally
         # hidden situation".  We do not performs discovery on unfiltered
         # repository because it end up doing a pathological amount of round
         # trip for w huge amount of changeset we do not care about.
         #
         # If a set of such "common but filtered" changeset exist on the server
         # but are not including a remote heads, we'll not be able to detect it,
         scommon = set(common)
         for n in rheads:
             if has_node(n):
                 if n not in scommon:
                     common.append(n)
         if set(rheads).issubset(set(common)):
             fetch = []
     pullop.common = common
     pullop.fetch = fetch
     pullop.rheads = rheads
 
 
 def _pullbundle2(pullop):
     """pull data using bundle2
 
     For now, the only supported data are changegroup."""
     kwargs = {b'bundlecaps': caps20to10(pullop.repo, role=b'client')}
 
     # make ui easier to access
     ui = pullop.repo.ui
 
     # At the moment we don't do stream clones over bundle2. If that is
     # implemented then here's where the check for that will go.
     streaming = streamclone.canperformstreamclone(pullop, bundle2=True)[0]
 
     # declare pull perimeters
     kwargs[b'common'] = pullop.common
     kwargs[b'heads'] = pullop.heads or pullop.rheads
 
     # check server supports narrow and then adding includepats and excludepats
     servernarrow = pullop.remote.capable(wireprototypes.NARROWCAP)
     if servernarrow and pullop.includepats:
         kwargs[b'includepats'] = pullop.includepats
     if servernarrow and pullop.excludepats:
         kwargs[b'excludepats'] = pullop.excludepats
 
     if streaming:
         kwargs[b'cg'] = False
         kwargs[b'stream'] = True
         pullop.stepsdone.add(b'changegroup')
         pullop.stepsdone.add(b'phases')
 
     else:
         # pulling changegroup
         pullop.stepsdone.add(b'changegroup')
 
         kwargs[b'cg'] = pullop.fetch
 
         legacyphase = b'phases' in ui.configlist(b'devel', b'legacy.exchange')
         hasbinaryphase = b'heads' in pullop.remotebundle2caps.get(b'phases', ())
         if not legacyphase and hasbinaryphase:
             kwargs[b'phases'] = True
             pullop.stepsdone.add(b'phases')
 
         if b'listkeys' in pullop.remotebundle2caps:
             if b'phases' not in pullop.stepsdone:
                 kwargs[b'listkeys'] = [b'phases']
 
     bookmarksrequested = False
     legacybookmark = b'bookmarks' in ui.configlist(b'devel', b'legacy.exchange')
     hasbinarybook = b'bookmarks' in pullop.remotebundle2caps
 
     if pullop.remotebookmarks is not None:
         pullop.stepsdone.add(b'request-bookmarks')
 
     if (
         b'request-bookmarks' not in pullop.stepsdone
         and pullop.remotebookmarks is None
         and not legacybookmark
         and hasbinarybook
     ):
         kwargs[b'bookmarks'] = True
         bookmarksrequested = True
 
     if b'listkeys' in pullop.remotebundle2caps:
         if b'request-bookmarks' not in pullop.stepsdone:
             # make sure to always includes bookmark data when migrating
             # `hg incoming --bundle` to using this function.
             pullop.stepsdone.add(b'request-bookmarks')
             kwargs.setdefault(b'listkeys', []).append(b'bookmarks')
 
     # If this is a full pull / clone and the server supports the clone bundles
     # feature, tell the server whether we attempted a clone bundle. The
     # presence of this flag indicates the client supports clone bundles. This
     # will enable the server to treat clients that support clone bundles
     # differently from those that don't.
     if (
         pullop.remote.capable(b'clonebundles')
         and pullop.heads is None
         and list(pullop.common) == [pullop.repo.nullid]
     ):
         kwargs[b'cbattempted'] = pullop.clonebundleattempted
 
     if streaming:
         pullop.repo.ui.status(_(b'streaming all changes\n'))
     elif not pullop.fetch:
         pullop.repo.ui.status(_(b"no changes found\n"))
         pullop.cgresult = 0
     else:
         if pullop.heads is None and list(pullop.common) == [pullop.repo.nullid]:
             pullop.repo.ui.status(_(b"requesting all changes\n"))
     if obsolete.isenabled(pullop.repo, obsolete.exchangeopt):
         remoteversions = bundle2.obsmarkersversion(pullop.remotebundle2caps)
         if obsolete.commonversion(remoteversions) is not None:
             kwargs[b'obsmarkers'] = True
             pullop.stepsdone.add(b'obsmarkers')
     _pullbundle2extraprepare(pullop, kwargs)
 
     remote_sidedata = bundle2.read_remote_wanted_sidedata(pullop.remote)
     if remote_sidedata:
         kwargs[b'remote_sidedata'] = remote_sidedata
 
     with pullop.remote.commandexecutor() as e:
         args = dict(kwargs)
         args[b'source'] = b'pull'
         bundle = e.callcommand(b'getbundle', args).result()
 
         try:
             op = bundle2.bundleoperation(
                 pullop.repo,
                 pullop.gettransaction,
                 source=b'pull',
                 remote=pullop.remote,
             )
             op.modes[b'bookmarks'] = b'records'
             bundle2.processbundle(
                 pullop.repo,
                 bundle,
                 op=op,
                 remote=pullop.remote,
             )
         except bundle2.AbortFromPart as exc:
             pullop.repo.ui.error(_(b'remote: abort: %s\n') % exc)
             raise error.RemoteError(_(b'pull failed on remote'), hint=exc.hint)
         except error.BundleValueError as exc:
             raise error.RemoteError(_(b'missing support for %s') % exc)
 
     if pullop.fetch:
         pullop.cgresult = bundle2.combinechangegroupresults(op)
 
     # processing phases change
     for namespace, value in op.records[b'listkeys']:
         if namespace == b'phases':
             _pullapplyphases(pullop, value)
 
     # processing bookmark update
     if bookmarksrequested:
         books = {}
         for record in op.records[b'bookmarks']:
             books[record[b'bookmark']] = record[b"node"]
         pullop.remotebookmarks = books
     else:
         for namespace, value in op.records[b'listkeys']:
             if namespace == b'bookmarks':
                 pullop.remotebookmarks = bookmod.unhexlifybookmarks(value)
 
     # bookmark data were either already there or pulled in the bundle
     if pullop.remotebookmarks is not None:
         _pullbookmarks(pullop)
 
 
 def _pullbundle2extraprepare(pullop, kwargs):
     """hook function so that extensions can extend the getbundle call"""
 
 
 def _pullchangeset(pullop):
     """pull changeset from unbundle into the local repo"""
     # We delay the open of the transaction as late as possible so we
     # don't open transaction for nothing or you break future useful
     # rollback call
     if b'changegroup' in pullop.stepsdone:
         return
     pullop.stepsdone.add(b'changegroup')
     if not pullop.fetch:
         pullop.repo.ui.status(_(b"no changes found\n"))
         pullop.cgresult = 0
         return
     tr = pullop.gettransaction()
     if pullop.heads is None and list(pullop.common) == [pullop.repo.nullid]:
         pullop.repo.ui.status(_(b"requesting all changes\n"))
     elif pullop.heads is None and pullop.remote.capable(b'changegroupsubset'):
         # issue1320, avoid a race if remote changed after discovery
         pullop.heads = pullop.rheads
 
     if pullop.remote.capable(b'getbundle'):
         # TODO: get bundlecaps from remote
         cg = pullop.remote.getbundle(
             b'pull', common=pullop.common, heads=pullop.heads or pullop.rheads
         )
     elif pullop.heads is None:
         with pullop.remote.commandexecutor() as e:
             cg = e.callcommand(
                 b'changegroup',
                 {
                     b'nodes': pullop.fetch,
                     b'source': b'pull',
                 },
             ).result()
 
     elif not pullop.remote.capable(b'changegroupsubset'):
         raise error.Abort(
             _(
                 b"partial pull cannot be done because "
                 b"other repository doesn't support "
                 b"changegroupsubset."
             )
         )
     else:
         with pullop.remote.commandexecutor() as e:
             cg = e.callcommand(
                 b'changegroupsubset',
                 {
                     b'bases': pullop.fetch,
                     b'heads': pullop.heads,
                     b'source': b'pull',
                 },
             ).result()
 
     bundleop = bundle2.applybundle(
         pullop.repo,
         cg,
         tr,
         b'pull',
         pullop.remote.url(),
         remote=pullop.remote,
     )
     pullop.cgresult = bundle2.combinechangegroupresults(bundleop)
 
 
 def _pullphase(pullop):
     # Get remote phases data from remote
     if b'phases' in pullop.stepsdone:
         return
     remotephases = listkeys(pullop.remote, b'phases')
     _pullapplyphases(pullop, remotephases)
 
 
 def _pullapplyphases(pullop, remotephases):
     """apply phase movement from observed remote state"""
     if b'phases' in pullop.stepsdone:
         return
     pullop.stepsdone.add(b'phases')
     publishing = bool(remotephases.get(b'publishing', False))
     if remotephases and not publishing:
         # remote is new and non-publishing
         pheads, _dr = phases.analyzeremotephases(
             pullop.repo, pullop.pulledsubset, remotephases
         )
         dheads = pullop.pulledsubset
     else:
         # Remote is old or publishing all common changesets
         # should be seen as public
         pheads = pullop.pulledsubset
         dheads = []
     unfi = pullop.repo.unfiltered()
     phase = unfi._phasecache.phase
     rev = unfi.changelog.index.get_rev
     public = phases.public
     draft = phases.draft
 
     # exclude changesets already public locally and update the others
     pheads = [pn for pn in pheads if phase(unfi, rev(pn)) > public]
     if pheads:
         tr = pullop.gettransaction()
         phases.advanceboundary(pullop.repo, tr, public, pheads)
 
     # exclude changesets already draft locally and update the others
     dheads = [pn for pn in dheads if phase(unfi, rev(pn)) > draft]
     if dheads:
         tr = pullop.gettransaction()
         phases.advanceboundary(pullop.repo, tr, draft, dheads)
 
 
 def _pullbookmarks(pullop):
     """process the remote bookmark information to update the local one"""
     if b'bookmarks' in pullop.stepsdone:
         return
     pullop.stepsdone.add(b'bookmarks')
     repo = pullop.repo
     remotebookmarks = pullop.remotebookmarks
     bookmarks_mode = None
     if pullop.remote_path is not None:
         bookmarks_mode = pullop.remote_path.bookmarks_mode
     bookmod.updatefromremote(
         repo.ui,
         repo,
         remotebookmarks,
         pullop.remote.url(),
         pullop.gettransaction,
         explicit=pullop.explicitbookmarks,
         mode=bookmarks_mode,
     )
 
 
 def _pullobsolete(pullop):
     """utility function to pull obsolete markers from a remote
 
     The `gettransaction` is function that return the pull transaction, creating
     one if necessary. We return the transaction to inform the calling code that
     a new transaction have been created (when applicable).
 
     Exists mostly to allow overriding for experimentation purpose"""
     if b'obsmarkers' in pullop.stepsdone:
         return
     pullop.stepsdone.add(b'obsmarkers')
     tr = None
     if obsolete.isenabled(pullop.repo, obsolete.exchangeopt):
         pullop.repo.ui.debug(b'fetching remote obsolete markers\n')
         remoteobs = listkeys(pullop.remote, b'obsolete')
         if b'dump0' in remoteobs:
             tr = pullop.gettransaction()
             markers = []
             for key in sorted(remoteobs, reverse=True):
                 if key.startswith(b'dump'):
                     data = util.b85decode(remoteobs[key])
                     version, newmarks = obsolete._readmarkers(data)
                     markers += newmarks
             if markers:
                 pullop.repo.obsstore.add(tr, markers)
             pullop.repo.invalidatevolatilesets()
     return tr
 
 
 def applynarrowacl(repo, kwargs):
     """Apply narrow fetch access control.
 
     This massages the named arguments for getbundle wire protocol commands
     so requested data is filtered through access control rules.
     """
     ui = repo.ui
     # TODO this assumes existence of HTTP and is a layering violation.
     username = ui.shortuser(ui.environ.get(b'REMOTE_USER') or ui.username())
     user_includes = ui.configlist(
         _NARROWACL_SECTION,
         username + b'.includes',
         ui.configlist(_NARROWACL_SECTION, b'default.includes'),
     )
     user_excludes = ui.configlist(
         _NARROWACL_SECTION,
         username + b'.excludes',
         ui.configlist(_NARROWACL_SECTION, b'default.excludes'),
     )
     if not user_includes:
         raise error.Abort(
             _(b"%s configuration for user %s is empty")
             % (_NARROWACL_SECTION, username)
         )
 
     user_includes = [
         b'path:.' if p == b'*' else b'path:' + p for p in user_includes
     ]
     user_excludes = [
         b'path:.' if p == b'*' else b'path:' + p for p in user_excludes
     ]
 
     req_includes = set(kwargs.get('includepats', []))
     req_excludes = set(kwargs.get('excludepats', []))
 
     req_includes, req_excludes, invalid_includes = narrowspec.restrictpatterns(
         req_includes, req_excludes, user_includes, user_excludes
     )
 
     if invalid_includes:
         raise error.Abort(
             _(b"The following includes are not accessible for %s: %s")
             % (username, stringutil.pprint(invalid_includes))
         )
 
     new_args = {}
     new_args.update(kwargs)
     new_args['narrow'] = True
     new_args['narrow_acl'] = True
     new_args['includepats'] = req_includes
     if req_excludes:
         new_args['excludepats'] = req_excludes
 
     return new_args
 
 
 def _computeellipsis(repo, common, heads, known, match, depth=None):
     """Compute the shape of a narrowed DAG.
 
     Args:
       repo: The repository we're transferring.
       common: The roots of the DAG range we're transferring.
               May be just [nullid], which means all ancestors of heads.
       heads: The heads of the DAG range we're transferring.
       match: The narrowmatcher that allows us to identify relevant changes.
       depth: If not None, only consider nodes to be full nodes if they are at
              most depth changesets away from one of heads.
 
     Returns:
       A tuple of (visitnodes, relevant_nodes, ellipsisroots) where:
 
         visitnodes: The list of nodes (either full or ellipsis) which
                     need to be sent to the client.
         relevant_nodes: The set of changelog nodes which change a file inside
                  the narrowspec. The client needs these as non-ellipsis nodes.
         ellipsisroots: A dict of {rev: parents} that is used in
                        narrowchangegroup to produce ellipsis nodes with the
                        correct parents.
     """
     cl = repo.changelog
     mfl = repo.manifestlog
 
     clrev = cl.rev
 
     commonrevs = {clrev(n) for n in common} | {nullrev}
     headsrevs = {clrev(n) for n in heads}
 
     if depth:
         revdepth = {h: 0 for h in headsrevs}
 
     ellipsisheads = collections.defaultdict(set)
     ellipsisroots = collections.defaultdict(set)
 
     def addroot(head, curchange):
         """Add a root to an ellipsis head, splitting heads with 3 roots."""
         ellipsisroots[head].add(curchange)
         # Recursively split ellipsis heads with 3 roots by finding the
         # roots' youngest common descendant which is an elided merge commit.
         # That descendant takes 2 of the 3 roots as its own, and becomes a
         # root of the head.
         while len(ellipsisroots[head]) > 2:
             child, roots = splithead(head)
             splitroots(head, child, roots)
             head = child  # Recurse in case we just added a 3rd root
 
     def splitroots(head, child, roots):
         ellipsisroots[head].difference_update(roots)
         ellipsisroots[head].add(child)
         ellipsisroots[child].update(roots)
         ellipsisroots[child].discard(child)
 
     def splithead(head):
         r1, r2, r3 = sorted(ellipsisroots[head])
         for nr1, nr2 in ((r2, r3), (r1, r3), (r1, r2)):
             mid = repo.revs(
                 b'sort(merge() & %d::%d & %d::%d, -rev)', nr1, head, nr2, head
             )
             for j in mid:
                 if j == nr2:
                     return nr2, (nr1, nr2)
                 if j not in ellipsisroots or len(ellipsisroots[j]) < 2:
                     return j, (nr1, nr2)
         raise error.Abort(
             _(
                 b'Failed to split up ellipsis node! head: %d, '
                 b'roots: %d %d %d'
             )
             % (head, r1, r2, r3)
         )
 
     missing = list(cl.findmissingrevs(common=commonrevs, heads=headsrevs))
     visit = reversed(missing)
     relevant_nodes = set()
     visitnodes = [cl.node(m) for m in missing]
     required = set(headsrevs) | known
     for rev in visit:
         clrev = cl.changelogrevision(rev)
         ps = [prev for prev in cl.parentrevs(rev) if prev != nullrev]
         if depth is not None:
             curdepth = revdepth[rev]
             for p in ps:
                 revdepth[p] = min(curdepth + 1, revdepth.get(p, depth + 1))
         needed = False
         shallow_enough = depth is None or revdepth[rev] <= depth
         if shallow_enough:
             curmf = mfl[clrev.manifest].read()
             if ps:
                 # We choose to not trust the changed files list in
                 # changesets because it's not always correct. TODO: could
                 # we trust it for the non-merge case?
                 p1mf = mfl[cl.changelogrevision(ps[0]).manifest].read()
                 needed = bool(curmf.diff(p1mf, match))
                 if not needed and len(ps) > 1:
                     # For merge changes, the list of changed files is not
                     # helpful, since we need to emit the merge if a file
                     # in the narrow spec has changed on either side of the
                     # merge. As a result, we do a manifest diff to check.
                     p2mf = mfl[cl.changelogrevision(ps[1]).manifest].read()
                     needed = bool(curmf.diff(p2mf, match))
             else:
                 # For a root node, we need to include the node if any
                 # files in the node match the narrowspec.
                 needed = any(curmf.walk(match))
 
         if needed:
             for head in ellipsisheads[rev]:
                 addroot(head, rev)
             for p in ps:
                 required.add(p)
             relevant_nodes.add(cl.node(rev))
         else:
             if not ps:
                 ps = [nullrev]
             if rev in required:
                 for head in ellipsisheads[rev]:
                     addroot(head, rev)
                 for p in ps:
                     ellipsisheads[p].add(rev)
             else:
                 for p in ps:
                     ellipsisheads[p] |= ellipsisheads[rev]
 
     # add common changesets as roots of their reachable ellipsis heads
     for c in commonrevs:
         for head in ellipsisheads[c]:
             addroot(head, c)
     return visitnodes, relevant_nodes, ellipsisroots
 
 
 def caps20to10(repo, role):
     """return a set with appropriate options to use bundle20 during getbundle"""
     caps = {b'HG20'}
     capsblob = bundle2.encodecaps(bundle2.getrepocaps(repo, role=role))
     caps.add(b'bundle2=' + urlreq.quote(capsblob))
     return caps
 
 
 # List of names of steps to perform for a bundle2 for getbundle, order matters.
 getbundle2partsorder = []
 
 # Mapping between step name and function
 #
 # This exists to help extensions wrap steps if necessary
 getbundle2partsmapping = {}
 
 
 def getbundle2partsgenerator(stepname, idx=None):
     """decorator for function generating bundle2 part for getbundle
 
     The function is added to the step -> function mapping and appended to the
     list of steps.  Beware that decorated functions will be added in order
     (this may matter).
 
     You can only use this decorator for new steps, if you want to wrap a step
     from an extension, attack the getbundle2partsmapping dictionary directly."""
 
     def dec(func):
         assert stepname not in getbundle2partsmapping
         getbundle2partsmapping[stepname] = func
         if idx is None:
             getbundle2partsorder.append(stepname)
         else:
             getbundle2partsorder.insert(idx, stepname)
         return func
 
     return dec
 
 
 def bundle2requested(bundlecaps):
     if bundlecaps is not None:
         return any(cap.startswith(b'HG2') for cap in bundlecaps)
     return False
 
 
 def getbundlechunks(
     repo,
     source,
     heads=None,
     common=None,
     bundlecaps=None,
     remote_sidedata=None,
     **kwargs
 ):
     """Return chunks constituting a bundle's raw data.
 
     Could be a bundle HG10 or a bundle HG20 depending on bundlecaps
     passed.
 
     Returns a 2-tuple of a dict with metadata about the generated bundle
     and an iterator over raw chunks (of varying sizes).
     """
     kwargs = pycompat.byteskwargs(kwargs)
     info = {}
     usebundle2 = bundle2requested(bundlecaps)
     # bundle10 case
     if not usebundle2:
         if bundlecaps and not kwargs.get(b'cg', True):
             raise ValueError(
                 _(b'request for bundle10 must include changegroup')
             )
 
         if kwargs:
             raise ValueError(
                 _(b'unsupported getbundle arguments: %s')
                 % b', '.join(sorted(kwargs.keys()))
             )
         outgoing = _computeoutgoing(repo, heads, common)
         info[b'bundleversion'] = 1
         return (
             info,
             changegroup.makestream(
                 repo,
                 outgoing,
                 b'01',
                 source,
                 bundlecaps=bundlecaps,
                 remote_sidedata=remote_sidedata,
             ),
         )
 
     # bundle20 case
     info[b'bundleversion'] = 2
     b2caps = {}
     for bcaps in bundlecaps:
         if bcaps.startswith(b'bundle2='):
             blob = urlreq.unquote(bcaps[len(b'bundle2=') :])
             b2caps.update(bundle2.decodecaps(blob))
     bundler = bundle2.bundle20(repo.ui, b2caps)
 
     kwargs[b'heads'] = heads
     kwargs[b'common'] = common
 
     for name in getbundle2partsorder:
         func = getbundle2partsmapping[name]
         func(
             bundler,
             repo,
             source,
             bundlecaps=bundlecaps,
             b2caps=b2caps,
             remote_sidedata=remote_sidedata,
             **pycompat.strkwargs(kwargs)
         )
 
     info[b'prefercompressed'] = bundler.prefercompressed
 
     return info, bundler.getchunks()
 
 
 @getbundle2partsgenerator(b'stream')
 def _getbundlestream2(bundler, repo, *args, **kwargs):
     return bundle2.addpartbundlestream2(bundler, repo, **kwargs)
 
 
 @getbundle2partsgenerator(b'changegroup')
 def _getbundlechangegrouppart(
     bundler,
     repo,
     source,
     bundlecaps=None,
     b2caps=None,
     heads=None,
     common=None,
     remote_sidedata=None,
     **kwargs
 ):
     """add a changegroup part to the requested bundle"""
     if not kwargs.get('cg', True) or not b2caps:
         return
 
     version = b'01'
     cgversions = b2caps.get(b'changegroup')
     if cgversions:  # 3.1 and 3.2 ship with an empty value
         cgversions = [
             v
             for v in cgversions
             if v in changegroup.supportedoutgoingversions(repo)
         ]
         if not cgversions:
             raise error.Abort(_(b'no common changegroup version'))
         version = max(cgversions)
 
     outgoing = _computeoutgoing(repo, heads, common)
     if not outgoing.missing:
         return
 
     if kwargs.get('narrow', False):
         include = sorted(filter(bool, kwargs.get('includepats', [])))
         exclude = sorted(filter(bool, kwargs.get('excludepats', [])))
         matcher = narrowspec.match(repo.root, include=include, exclude=exclude)
     else:
         matcher = None
 
     cgstream = changegroup.makestream(
         repo,
         outgoing,
         version,
         source,
         bundlecaps=bundlecaps,
         matcher=matcher,
         remote_sidedata=remote_sidedata,
     )
 
     part = bundler.newpart(b'changegroup', data=cgstream)
     if cgversions:
         part.addparam(b'version', version)
 
     part.addparam(b'nbchanges', b'%d' % len(outgoing.missing), mandatory=False)
 
     if scmutil.istreemanifest(repo):
         part.addparam(b'treemanifest', b'1')
 
     if repository.REPO_FEATURE_SIDE_DATA in repo.features:
         part.addparam(b'exp-sidedata', b'1')
         sidedata = bundle2.format_remote_wanted_sidedata(repo)
         part.addparam(b'exp-wanted-sidedata', sidedata)
 
     if (
         kwargs.get('narrow', False)
         and kwargs.get('narrow_acl', False)
         and (include or exclude)
     ):
         # this is mandatory because otherwise ACL clients won't work
         narrowspecpart = bundler.newpart(b'Narrow:responsespec')
         narrowspecpart.data = b'%s\0%s' % (
             b'\n'.join(include),
             b'\n'.join(exclude),
         )
 
 
 @getbundle2partsgenerator(b'bookmarks')
 def _getbundlebookmarkpart(
     bundler, repo, source, bundlecaps=None, b2caps=None, **kwargs
 ):
     """add a bookmark part to the requested bundle"""
     if not kwargs.get('bookmarks', False):
         return
     if not b2caps or b'bookmarks' not in b2caps:
         raise error.Abort(_(b'no common bookmarks exchange method'))
     books = bookmod.listbinbookmarks(repo)
     data = bookmod.binaryencode(repo, books)
     if data:
         bundler.newpart(b'bookmarks', data=data)
 
 
 @getbundle2partsgenerator(b'listkeys')
 def _getbundlelistkeysparts(
     bundler, repo, source, bundlecaps=None, b2caps=None, **kwargs
 ):
     """add parts containing listkeys namespaces to the requested bundle"""
     listkeys = kwargs.get('listkeys', ())
     for namespace in listkeys:
         part = bundler.newpart(b'listkeys')
         part.addparam(b'namespace', namespace)
         keys = repo.listkeys(namespace).items()
         part.data = pushkey.encodekeys(keys)
 
 
 @getbundle2partsgenerator(b'obsmarkers')
 def _getbundleobsmarkerpart(
     bundler, repo, source, bundlecaps=None, b2caps=None, heads=None, **kwargs
 ):
     """add an obsolescence markers part to the requested bundle"""
     if kwargs.get('obsmarkers', False):
         if heads is None:
             heads = repo.heads()
         subset = [c.node() for c in repo.set(b'::%ln', heads)]
         markers = repo.obsstore.relevantmarkers(subset)
         markers = obsutil.sortedmarkers(markers)
         bundle2.buildobsmarkerspart(bundler, markers)
 
 
 @getbundle2partsgenerator(b'phases')
 def _getbundlephasespart(
     bundler, repo, source, bundlecaps=None, b2caps=None, heads=None, **kwargs
 ):
     """add phase heads part to the requested bundle"""
     if kwargs.get('phases', False):
         if not b2caps or b'heads' not in b2caps.get(b'phases'):
             raise error.Abort(_(b'no common phases exchange method'))
         if heads is None:
             heads = repo.heads()
 
         headsbyphase = collections.defaultdict(set)
         if repo.publishing():
             headsbyphase[phases.public] = heads
         else:
             # find the appropriate heads to move
 
             phase = repo._phasecache.phase
             node = repo.changelog.node
             rev = repo.changelog.rev
             for h in heads:
                 headsbyphase[phase(repo, rev(h))].add(h)
             seenphases = list(headsbyphase.keys())
 
             # We do not handle anything but public and draft phase for now)
             if seenphases:
                 assert max(seenphases) <= phases.draft
 
             # if client is pulling non-public changesets, we need to find
             # intermediate public heads.
             draftheads = headsbyphase.get(phases.draft, set())
             if draftheads:
                 publicheads = headsbyphase.get(phases.public, set())
 
                 revset = b'heads(only(%ln, %ln) and public())'
                 extraheads = repo.revs(revset, draftheads, publicheads)
                 for r in extraheads:
                     headsbyphase[phases.public].add(node(r))
 
         # transform data in a format used by the encoding function
         phasemapping = {
             phase: sorted(headsbyphase[phase]) for phase in phases.allphases
         }
 
         # generate the actual part
         phasedata = phases.binaryencode(phasemapping)
         bundler.newpart(b'phase-heads', data=phasedata)
 
 
 @getbundle2partsgenerator(b'hgtagsfnodes')
 def _getbundletagsfnodes(
     bundler,
     repo,
     source,
     bundlecaps=None,
     b2caps=None,
     heads=None,
     common=None,
     **kwargs
 ):
     """Transfer the .hgtags filenodes mapping.
 
     Only values for heads in this bundle will be transferred.
 
     The part data consists of pairs of 20 byte changeset node and .hgtags
     filenodes raw values.
     """
     # Don't send unless:
     # - changeset are being exchanged,
     # - the client supports it.
     if not b2caps or not (kwargs.get('cg', True) and b'hgtagsfnodes' in b2caps):
         return
 
     outgoing = _computeoutgoing(repo, heads, common)
     bundle2.addparttagsfnodescache(repo, bundler, outgoing)
 
 
 @getbundle2partsgenerator(b'cache:rev-branch-cache')
 def _getbundlerevbranchcache(
     bundler,
     repo,
     source,
     bundlecaps=None,
     b2caps=None,
     heads=None,
     common=None,
     **kwargs
 ):
     """Transfer the rev-branch-cache mapping
 
     The payload is a series of data related to each branch
 
     1) branch name length
     2) number of open heads
     3) number of closed heads
     4) open heads nodes
     5) closed heads nodes
     """
     # Don't send unless:
     # - changeset are being exchanged,
     # - the client supports it.
     # - narrow bundle isn't in play (not currently compatible).
     if (
         not kwargs.get('cg', True)
         or not b2caps
         or b'rev-branch-cache' not in b2caps
         or kwargs.get('narrow', False)
         or repo.ui.has_section(_NARROWACL_SECTION)
     ):
         return
 
     outgoing = _computeoutgoing(repo, heads, common)
     bundle2.addpartrevbranchcache(repo, bundler, outgoing)
 
 
 def check_heads(repo, their_heads, context):
     """check if the heads of a repo have been modified
 
     Used by peer for unbundling.
     """
     heads = repo.heads()
     heads_hash = hashutil.sha1(b''.join(sorted(heads))).digest()
     if not (
         their_heads == [b'force']
         or their_heads == heads
         or their_heads == [b'hashed', heads_hash]
     ):
         # someone else committed/pushed/unbundled while we
         # were transferring data
         raise error.PushRaced(
             b'repository changed while %s - please try again' % context
         )
 
 
 def unbundle(repo, cg, heads, source, url):
     """Apply a bundle to a repo.
 
     this function makes sure the repo is locked during the application and have
     mechanism to check that no push race occurred between the creation of the
     bundle and its application.
 
     If the push was raced as PushRaced exception is raised."""
     r = 0
     # need a transaction when processing a bundle2 stream
     # [wlock, lock, tr] - needs to be an array so nested functions can modify it
     lockandtr = [None, None, None]
     recordout = None
     # quick fix for output mismatch with bundle2 in 3.4
     captureoutput = repo.ui.configbool(
         b'experimental', b'bundle2-output-capture'
     )
     if url.startswith(b'remote:http:') or url.startswith(b'remote:https:'):
         captureoutput = True
     try:
         # note: outside bundle1, 'heads' is expected to be empty and this
         # 'check_heads' call wil be a no-op
         check_heads(repo, heads, b'uploading changes')
         # push can proceed
         if not isinstance(cg, bundle2.unbundle20):
             # legacy case: bundle1 (changegroup 01)
             txnname = b"\n".join([source, urlutil.hidepassword(url)])
             with repo.lock(), repo.transaction(txnname) as tr:
                 op = bundle2.applybundle(repo, cg, tr, source, url)
                 r = bundle2.combinechangegroupresults(op)
         else:
             r = None
             try:
 
                 def gettransaction():
                     if not lockandtr[2]:
                         if not bookmod.bookmarksinstore(repo):
                             lockandtr[0] = repo.wlock()
                         lockandtr[1] = repo.lock()
                         lockandtr[2] = repo.transaction(source)
                         lockandtr[2].hookargs[b'source'] = source
                         lockandtr[2].hookargs[b'url'] = url
                         lockandtr[2].hookargs[b'bundle2'] = b'1'
                     return lockandtr[2]
 
                 # Do greedy locking by default until we're satisfied with lazy
                 # locking.
                 if not repo.ui.configbool(
                     b'experimental', b'bundle2lazylocking'
                 ):
                     gettransaction()
 
                 op = bundle2.bundleoperation(
                     repo,
                     gettransaction,
                     captureoutput=captureoutput,
                     source=b'push',
                 )
                 try:
                     op = bundle2.processbundle(repo, cg, op=op)
                 finally:
                     r = op.reply
                     if captureoutput and r is not None:
                         repo.ui.pushbuffer(error=True, subproc=True)
 
                         def recordout(output):
                             r.newpart(b'output', data=output, mandatory=False)
 
                 if lockandtr[2] is not None:
                     lockandtr[2].close()
             except BaseException as exc:
                 exc.duringunbundle2 = True
                 if captureoutput and r is not None:
                     parts = exc._bundle2salvagedoutput = r.salvageoutput()
 
                     def recordout(output):
                         part = bundle2.bundlepart(
                             b'output', data=output, mandatory=False
                         )
                         parts.append(part)
 
                 raise
     finally:
         lockmod.release(lockandtr[2], lockandtr[1], lockandtr[0])
         if recordout is not None:
             recordout(repo.ui.popbuffer())
     return r
 
 
 def _maybeapplyclonebundle(pullop):
     """Apply a clone bundle from a remote, if possible."""
 
     repo = pullop.repo
     remote = pullop.remote
 
     if not repo.ui.configbool(b'ui', b'clonebundles'):
         return
 
     # Only run if local repo is empty.
     if len(repo):
         return
 
     if pullop.heads:
         return
 
     if not remote.capable(b'clonebundles'):
         return
 
     with remote.commandexecutor() as e:
         res = e.callcommand(b'clonebundles', {}).result()
 
     # If we call the wire protocol command, that's good enough to record the
     # attempt.
     pullop.clonebundleattempted = True
 
     entries = bundlecaches.parseclonebundlesmanifest(repo, res)
     if not entries:
         repo.ui.note(
             _(
                 b'no clone bundles available on remote; '
                 b'falling back to regular clone\n'
             )
         )
         return
 
     entries = bundlecaches.filterclonebundleentries(
         repo, entries, streamclonerequested=pullop.streamclonerequested
     )
 
     if not entries:
         # There is a thundering herd concern here. However, if a server
         # operator doesn't advertise bundles appropriate for its clients,
         # they deserve what's coming. Furthermore, from a client's
         # perspective, no automatic fallback would mean not being able to
         # clone!
         repo.ui.warn(
             _(
                 b'no compatible clone bundles available on server; '
                 b'falling back to regular clone\n'
             )
         )
         repo.ui.warn(
             _(b'(you may want to report this to the server operator)\n')
         )
         return
 
     entries = bundlecaches.sortclonebundleentries(repo.ui, entries)
 
     url = entries[0][b'URL']
     repo.ui.status(_(b'applying clone bundle from %s\n') % url)
-    if trypullbundlefromurl(repo.ui, repo, url):
+    if trypullbundlefromurl(repo.ui, repo, url, remote):
         repo.ui.status(_(b'finished applying clone bundle\n'))
     # Bundle failed.
     #
     # We abort by default to avoid the thundering herd of
     # clients flooding a server that was expecting expensive
     # clone load to be offloaded.
     elif repo.ui.configbool(b'ui', b'clonebundlefallback'):
         repo.ui.warn(_(b'falling back to normal clone\n'))
     else:
         raise error.Abort(
             _(b'error applying bundle'),
             hint=_(
                 b'if this error persists, consider contacting '
                 b'the server operator or disable clone '
                 b'bundles via '
                 b'"--config ui.clonebundles=false"'
             ),
         )
 
 
-def trypullbundlefromurl(ui, repo, url):
+def inline_clone_bundle_open(ui, url, peer):
+    if not peer:
+        raise error.Abort(_(b'no remote repository supplied for %s' % url))
+    clonebundleid = url[len(bundlecaches.CLONEBUNDLESCHEME) :]
+    peerclonebundle = peer.get_inline_clone_bundle(clonebundleid)
+    return util.chunkbuffer(peerclonebundle)
+
+
+def trypullbundlefromurl(ui, repo, url, peer):
     """Attempt to apply a bundle from a URL."""
     with repo.lock(), repo.transaction(b'bundleurl') as tr:
         try:
-            fh = urlmod.open(ui, url)
+            if url.startswith(bundlecaches.CLONEBUNDLESCHEME):
+                fh = inline_clone_bundle_open(ui, url, peer)
+            else:
+                fh = urlmod.open(ui, url)
             cg = readbundle(ui, fh, b'stream')
 
             if isinstance(cg, streamclone.streamcloneapplier):
                 cg.apply(repo)
             else:
                 bundle2.applybundle(repo, cg, tr, b'clonebundles', url)
             return True
         except urlerr.httperror as e:
             ui.warn(
                 _(b'HTTP error fetching bundle: %s\n')
                 % stringutil.forcebytestr(e)
             )
         except urlerr.urlerror as e:
             ui.warn(
                 _(b'error fetching bundle: %s\n')
                 % stringutil.forcebytestr(e.reason)
             )
 
         return False

mercurial/helptext/config.txt

@@ -1,3383 +1,3389 b''
 The Mercurial system uses a set of configuration files to control
 aspects of its behavior.
 
 Troubleshooting
 ===============
 
 If you're having problems with your configuration,
 :hg:`config --source` can help you understand what is introducing
 a setting into your environment.
 
 See :hg:`help config.syntax` and :hg:`help config.files`
 for information about how and where to override things.
 
 Structure
 =========
 
 The configuration files use a simple ini-file format. A configuration
 file consists of sections, led by a ``[section]`` header and followed
 by ``name = value`` entries::
 
   [ui]
   username = Firstname Lastname <[e-mail]>
   verbose = True
 
 The above entries will be referred to as ``ui.username`` and
 ``ui.verbose``, respectively. See :hg:`help config.syntax`.
 
 Files
 =====
 
 Mercurial reads configuration data from several files, if they exist.
 These files do not exist by default and you will have to create the
 appropriate configuration files yourself:
 
 Local configuration is put into the per-repository ``<repo>/.hg/hgrc`` file.
 
 Global configuration like the username setting is typically put into:
 
 .. container:: windows
 
   - ``%USERPROFILE%\mercurial.ini`` (on Windows)
 
 .. container:: unix.plan9
 
   - ``$HOME/.hgrc`` (on Unix, Plan9)
 
 The names of these files depend on the system on which Mercurial is
 installed. ``*.rc`` files from a single directory are read in
 alphabetical order, later ones overriding earlier ones. Where multiple
 paths are given below, settings from earlier paths override later
 ones.
 
 .. container:: verbose.unix
 
   On Unix, the following files are consulted:
 
   - ``<repo>/.hg/hgrc-not-shared`` (per-repository)
   - ``<repo>/.hg/hgrc`` (per-repository)
   - ``$HOME/.hgrc`` (per-user)
   - ``${XDG_CONFIG_HOME:-$HOME/.config}/hg/hgrc`` (per-user)
   - ``<install-root>/etc/mercurial/hgrc`` (per-installation)
   - ``<install-root>/etc/mercurial/hgrc.d/*.rc`` (per-installation)
   - ``/etc/mercurial/hgrc`` (per-system)
   - ``/etc/mercurial/hgrc.d/*.rc`` (per-system)
   - ``<internal>/*.rc`` (defaults)
 
 .. container:: verbose.windows
 
   On Windows, the following files are consulted:
 
   - ``<repo>/.hg/hgrc-not-shared`` (per-repository)
   - ``<repo>/.hg/hgrc`` (per-repository)
   - ``%USERPROFILE%\.hgrc`` (per-user)
   - ``%USERPROFILE%\Mercurial.ini`` (per-user)
   - ``%HOME%\.hgrc`` (per-user)
   - ``%HOME%\Mercurial.ini`` (per-user)
   - ``HKEY_LOCAL_MACHINE\SOFTWARE\Mercurial`` (per-system)
   - ``<install-dir>\hgrc.d\*.rc`` (per-installation)
   - ``<install-dir>\Mercurial.ini`` (per-installation)
   - ``%PROGRAMDATA%\Mercurial\hgrc`` (per-system)
   - ``%PROGRAMDATA%\Mercurial\Mercurial.ini`` (per-system)
   - ``%PROGRAMDATA%\Mercurial\hgrc.d\*.rc`` (per-system)
   - ``<internal>/*.rc`` (defaults)
 
   .. note::
 
    The registry key ``HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Mercurial``
    is used when running 32-bit Python on 64-bit Windows.
 
 .. container:: verbose.plan9
 
   On Plan9, the following files are consulted:
 
   - ``<repo>/.hg/hgrc-not-shared`` (per-repository)
   - ``<repo>/.hg/hgrc`` (per-repository)
   - ``$home/lib/hgrc`` (per-user)
   - ``<install-root>/lib/mercurial/hgrc`` (per-installation)
   - ``<install-root>/lib/mercurial/hgrc.d/*.rc`` (per-installation)
   - ``/lib/mercurial/hgrc`` (per-system)
   - ``/lib/mercurial/hgrc.d/*.rc`` (per-system)
   - ``<internal>/*.rc`` (defaults)
 
 Per-repository configuration options only apply in a
 particular repository. This file is not version-controlled, and
 will not get transferred during a "clone" operation. Options in
 this file override options in all other configuration files.
 
 .. container:: unix.plan9
 
   On Plan 9 and Unix, most of this file will be ignored if it doesn't
   belong to a trusted user or to a trusted group. See
   :hg:`help config.trusted` for more details.
 
 Per-user configuration file(s) are for the user running Mercurial.  Options
 in these files apply to all Mercurial commands executed by this user in any
 directory. Options in these files override per-system and per-installation
 options.
 
 Per-installation configuration files are searched for in the
 directory where Mercurial is installed. ``<install-root>`` is the
 parent directory of the **hg** executable (or symlink) being run.
 
 .. container:: unix.plan9
 
   For example, if installed in ``/shared/tools/bin/hg``, Mercurial
   will look in ``/shared/tools/etc/mercurial/hgrc``. Options in these
   files apply to all Mercurial commands executed by any user in any
   directory.
 
 Per-installation configuration files are for the system on
 which Mercurial is running. Options in these files apply to all
 Mercurial commands executed by any user in any directory. Registry
 keys contain PATH-like strings, every part of which must reference
 a ``Mercurial.ini`` file or be a directory where ``*.rc`` files will
 be read.  Mercurial checks each of these locations in the specified
 order until one or more configuration files are detected.
 
 Per-system configuration files are for the system on which Mercurial
 is running. Options in these files apply to all Mercurial commands
 executed by any user in any directory. Options in these files
 override per-installation options.
 
 Mercurial comes with some default configuration. The default configuration
 files are installed with Mercurial and will be overwritten on upgrades. Default
 configuration files should never be edited by users or administrators but can
 be overridden in other configuration files. So far the directory only contains
 merge tool configuration but packagers can also put other default configuration
 there.
 
 On versions 5.7 and later, if share-safe functionality is enabled,
 shares will read config file of share source too.
 `<share-source/.hg/hgrc>` is read before reading `<repo/.hg/hgrc>`.
 
 For configs which should not be shared, `<repo/.hg/hgrc-not-shared>`
 should be used.
 
 Syntax
 ======
 
 A configuration file consists of sections, led by a ``[section]`` header
 and followed by ``name = value`` entries (sometimes called
 ``configuration keys``)::
 
     [spam]
     eggs=ham
     green=
        eggs
 
 Each line contains one entry. If the lines that follow are indented,
 they are treated as continuations of that entry. Leading whitespace is
 removed from values. Empty lines are skipped. Lines beginning with
 ``#`` or ``;`` are ignored and may be used to provide comments.
 
 Configuration keys can be set multiple times, in which case Mercurial
 will use the value that was configured last. As an example::
 
     [spam]
     eggs=large
     ham=serrano
     eggs=small
 
 This would set the configuration key named ``eggs`` to ``small``.
 
 It is also possible to define a section multiple times. A section can
 be redefined on the same and/or on different configuration files. For
 example::
 
     [foo]
     eggs=large
     ham=serrano
     eggs=small
 
     [bar]
     eggs=ham
     green=
        eggs
 
     [foo]
     ham=prosciutto
     eggs=medium
     bread=toasted
 
 This would set the ``eggs``, ``ham``, and ``bread`` configuration keys
 of the ``foo`` section to ``medium``, ``prosciutto``, and ``toasted``,
 respectively. As you can see there only thing that matters is the last
 value that was set for each of the configuration keys.
 
 If a configuration key is set multiple times in different
 configuration files the final value will depend on the order in which
 the different configuration files are read, with settings from earlier
 paths overriding later ones as described on the ``Files`` section
 above.
 
 A line of the form ``%include file`` will include ``file`` into the
 current configuration file. The inclusion is recursive, which means
 that included files can include other files. Filenames are relative to
 the configuration file in which the ``%include`` directive is found.
 Environment variables and ``~user`` constructs are expanded in
 ``file``. This lets you do something like::
 
   %include ~/.hgrc.d/$HOST.rc
 
 to include a different configuration file on each computer you use.
 
 A line with ``%unset name`` will remove ``name`` from the current
 section, if it has been set previously.
 
 The values are either free-form text strings, lists of text strings,
 or Boolean values. Boolean values can be set to true using any of "1",
 "yes", "true", or "on" and to false using "0", "no", "false", or "off"
 (all case insensitive).
 
 List values are separated by whitespace or comma, except when values are
 placed in double quotation marks::
 
   allow_read = "John Doe, PhD", brian, betty
 
 Quotation marks can be escaped by prefixing them with a backslash. Only
 quotation marks at the beginning of a word is counted as a quotation
 (e.g., ``foo"bar baz`` is the list of ``foo"bar`` and ``baz``).
 
 Sections
 ========
 
 This section describes the different sections that may appear in a
 Mercurial configuration file, the purpose of each section, its possible
 keys, and their possible values.
 
 ``alias``
 ---------
 
 Defines command aliases.
 
 Aliases allow you to define your own commands in terms of other
 commands (or aliases), optionally including arguments. Positional
 arguments in the form of ``$1``, ``$2``, etc. in the alias definition
 are expanded by Mercurial before execution. Positional arguments not
 already used by ``$N`` in the definition are put at the end of the
 command to be executed.
 
 Alias definitions consist of lines of the form::
 
     <alias> = <command> [<argument>]...
 
 For example, this definition::
 
     latest = log --limit 5
 
 creates a new command ``latest`` that shows only the five most recent
 changesets. You can define subsequent aliases using earlier ones::
 
     stable5 = latest -b stable
 
 .. note::
 
    It is possible to create aliases with the same names as
    existing commands, which will then override the original
    definitions. This is almost always a bad idea!
 
 An alias can start with an exclamation point (``!``) to make it a
 shell alias. A shell alias is executed with the shell and will let you
 run arbitrary commands. As an example, ::
 
    echo = !echo $@
 
 will let you do ``hg echo foo`` to have ``foo`` printed in your
 terminal. A better example might be::
 
    purge = !$HG status --no-status --unknown -0 re: | xargs -0 rm -f
 
 which will make ``hg purge`` delete all unknown files in the
 repository in the same manner as the purge extension.
 
 Positional arguments like ``$1``, ``$2``, etc. in the alias definition
 expand to the command arguments. Unmatched arguments are
 removed. ``$0`` expands to the alias name and ``$@`` expands to all
 arguments separated by a space. ``"$@"`` (with quotes) expands to all
 arguments quoted individually and separated by a space. These expansions
 happen before the command is passed to the shell.
 
 Shell aliases are executed in an environment where ``$HG`` expands to
 the path of the Mercurial that was used to execute the alias. This is
 useful when you want to call further Mercurial commands in a shell
 alias, as was done above for the purge alias. In addition,
 ``$HG_ARGS`` expands to the arguments given to Mercurial. In the ``hg
 echo foo`` call above, ``$HG_ARGS`` would expand to ``echo foo``.
 
 .. note::
 
    Some global configuration options such as ``-R`` are
    processed before shell aliases and will thus not be passed to
    aliases.
 
 
 ``annotate``
 ------------
 
 Settings used when displaying file annotations. All values are
 Booleans and default to False. See :hg:`help config.diff` for
 related options for the diff command.
 
 ``ignorews``
     Ignore white space when comparing lines.
 
 ``ignorewseol``
     Ignore white space at the end of a line when comparing lines.
 
 ``ignorewsamount``
     Ignore changes in the amount of white space.
 
 ``ignoreblanklines``
     Ignore changes whose lines are all blank.
 
 
 ``auth``
 --------
 
 Authentication credentials and other authentication-like configuration
 for HTTP connections. This section allows you to store usernames and
 passwords for use when logging *into* HTTP servers. See
 :hg:`help config.web` if you want to configure *who* can login to
 your HTTP server.
 
 The following options apply to all hosts.
 
 ``cookiefile``
     Path to a file containing HTTP cookie lines. Cookies matching a
     host will be sent automatically.
 
     The file format uses the Mozilla cookies.txt format, which defines cookies
     on their own lines. Each line contains 7 fields delimited by the tab
     character (domain, is_domain_cookie, path, is_secure, expires, name,
     value). For more info, do an Internet search for "Netscape cookies.txt
     format."
 
     Note: the cookies parser does not handle port numbers on domains. You
     will need to remove ports from the domain for the cookie to be recognized.
     This could result in a cookie being disclosed to an unwanted server.
 
     The cookies file is read-only.
 
 Other options in this section are grouped by name and have the following
 format::
 
     <name>.<argument> = <value>
 
 where ``<name>`` is used to group arguments into authentication
 entries. Example::
 
     foo.prefix = hg.intevation.de/mercurial
     foo.username = foo
     foo.password = bar
     foo.schemes = http https
 
     bar.prefix = secure.example.org
     bar.key = path/to/file.key
     bar.cert = path/to/file.cert
     bar.schemes = https
 
 Supported arguments:
 
 ``prefix``
     Either ``*`` or a URI prefix with or without the scheme part.
     The authentication entry with the longest matching prefix is used
     (where ``*`` matches everything and counts as a match of length
     1). If the prefix doesn't include a scheme, the match is performed
     against the URI with its scheme stripped as well, and the schemes
     argument, q.v., is then subsequently consulted.
 
 ``username``
     Optional. Username to authenticate with. If not given, and the
     remote site requires basic or digest authentication, the user will
     be prompted for it. Environment variables are expanded in the
     username letting you do ``foo.username = $USER``. If the URI
     includes a username, only ``[auth]`` entries with a matching
     username or without a username will be considered.
 
 ``password``
     Optional. Password to authenticate with. If not given, and the
     remote site requires basic or digest authentication, the user
     will be prompted for it.
 
 ``key``
     Optional. PEM encoded client certificate key file. Environment
     variables are expanded in the filename.
 
 ``cert``
     Optional. PEM encoded client certificate chain file. Environment
     variables are expanded in the filename.
 
 ``schemes``
     Optional. Space separated list of URI schemes to use this
     authentication entry with. Only used if the prefix doesn't include
     a scheme. Supported schemes are http and https. They will match
     static-http and static-https respectively, as well.
     (default: https)
 
 If no suitable authentication entry is found, the user is prompted
 for credentials as usual if required by the remote.
 
 ``cmdserver``
 -------------
 
 Controls command server settings. (ADVANCED)
 
 ``message-encodings``
     List of encodings for the ``m`` (message) channel. The first encoding
     supported by the server will be selected and advertised in the hello
     message. This is useful only when ``ui.message-output`` is set to
     ``channel``. Supported encodings are ``cbor``.
 
 ``shutdown-on-interrupt``
     If set to false, the server's main loop will continue running after
     SIGINT received. ``runcommand`` requests can still be interrupted by
     SIGINT. Close the write end of the pipe to shut down the server
     process gracefully.
     (default: True)
 
 ``color``
 ---------
 
 Configure the Mercurial color mode. For details about how to define your custom
 effect and style see :hg:`help color`.
 
 ``mode``
     String: control the method used to output color. One of ``auto``, ``ansi``,
     ``win32``, ``terminfo`` or ``debug``. In auto mode, Mercurial will
     use ANSI mode by default (or win32 mode prior to Windows 10) if it detects a
     terminal. Any invalid value will disable color.
 
 ``pagermode``
     String: optional override of ``color.mode`` used with pager.
 
     On some systems, terminfo mode may cause problems when using
     color with ``less -R`` as a pager program. less with the -R option
     will only display ECMA-48 color codes, and terminfo mode may sometimes
     emit codes that less doesn't understand. You can work around this by
     either using ansi mode (or auto mode), or by using less -r (which will
     pass through all terminal control codes, not just color control
     codes).
 
     On some systems (such as MSYS in Windows), the terminal may support
     a different color mode than the pager program.
 
 ``commands``
 ------------
 
 ``commit.post-status``
     Show status of files in the working directory after successful commit.
     (default: False)
 
 ``merge.require-rev``
     Require that the revision to merge the current commit with be specified on
     the command line. If this is enabled and a revision is not specified, the
     command aborts.
     (default: False)
 
 ``push.require-revs``
     Require revisions to push be specified using one or more mechanisms such as
     specifying them positionally on the command line, using ``-r``, ``-b``,
     and/or ``-B`` on the command line, or using ``paths.<path>:pushrev`` in the
     configuration. If this is enabled and revisions are not specified, the
     command aborts.
     (default: False)
 
 ``resolve.confirm``
     Confirm before performing action if no filename is passed.
     (default: False)
 
 ``resolve.explicit-re-merge``
     Require uses of ``hg resolve`` to specify which action it should perform,
     instead of re-merging files by default.
     (default: False)
 
 ``resolve.mark-check``
     Determines what level of checking :hg:`resolve --mark` will perform before
     marking files as resolved. Valid values are ``none`, ``warn``, and
     ``abort``. ``warn`` will output a warning listing the file(s) that still
     have conflict markers in them, but will still mark everything resolved.
     ``abort`` will output the same warning but will not mark things as resolved.
     If --all is passed and this is set to ``abort``, only a warning will be
     shown (an error will not be raised).
     (default: ``none``)
 
 ``status.relative``
     Make paths in :hg:`status` output relative to the current directory.
     (default: False)
 
 ``status.terse``
     Default value for the --terse flag, which condenses status output.
     (default: empty)
 
 ``update.check``
     Determines what level of checking :hg:`update` will perform before moving
     to a destination revision. Valid values are ``abort``, ``none``,
     ``linear``, and ``noconflict``.
 
     - ``abort`` always fails if the working directory has uncommitted changes.
 
     - ``none`` performs no checking, and may result in a merge with uncommitted changes.
 
     - ``linear`` allows any update as long as it follows a straight line in the
       revision history, and may trigger a merge with uncommitted changes.
 
     - ``noconflict`` will allow any update which would not trigger a merge with
       uncommitted changes, if any are present.
 
     (default: ``linear``)
 
 ``update.requiredest``
     Require that the user pass a destination when running :hg:`update`.
     For example, :hg:`update .::` will be allowed, but a plain :hg:`update`
     will be disallowed.
     (default: False)
 
 ``committemplate``
 ------------------
 
 ``changeset``
     String: configuration in this section is used as the template to
     customize the text shown in the editor when committing.
 
 In addition to pre-defined template keywords, commit log specific one
 below can be used for customization:
 
 ``extramsg``
     String: Extra message (typically 'Leave message empty to abort
     commit.'). This may be changed by some commands or extensions.
 
 For example, the template configuration below shows as same text as
 one shown by default::
 
     [committemplate]
     changeset = {desc}\n\n
         HG: Enter commit message.  Lines beginning with 'HG:' are removed.
         HG: {extramsg}
         HG: --
         HG: user: {author}\n{ifeq(p2rev, "-1", "",
        "HG: branch merge\n")
        }HG: branch '{branch}'\n{if(activebookmark,
        "HG: bookmark '{activebookmark}'\n")   }{subrepos %
        "HG: subrepo {subrepo}\n"              }{file_adds %
        "HG: added {file}\n"                   }{file_mods %
        "HG: changed {file}\n"                 }{file_dels %
        "HG: removed {file}\n"                 }{if(files, "",
        "HG: no files changed\n")}
 
 ``diff()``
     String: show the diff (see :hg:`help templates` for detail)
 
 Sometimes it is helpful to show the diff of the changeset in the editor without
 having to prefix 'HG: ' to each line so that highlighting works correctly. For
 this, Mercurial provides a special string which will ignore everything below
 it::
 
      HG: ------------------------ >8 ------------------------
 
 For example, the template configuration below will show the diff below the
 extra message::
 
     [committemplate]
     changeset = {desc}\n\n
         HG: Enter commit message.  Lines beginning with 'HG:' are removed.
         HG: {extramsg}
         HG: ------------------------ >8 ------------------------
         HG: Do not touch the line above.
         HG: Everything below will be removed.
         {diff()}
 
 .. note::
 
    For some problematic encodings (see :hg:`help win32mbcs` for
    detail), this customization should be configured carefully, to
    avoid showing broken characters.
 
    For example, if a multibyte character ending with backslash (0x5c) is
    followed by the ASCII character 'n' in the customized template,
    the sequence of backslash and 'n' is treated as line-feed unexpectedly
    (and the multibyte character is broken, too).
 
 Customized template is used for commands below (``--edit`` may be
 required):
 
 - :hg:`backout`
 - :hg:`commit`
 - :hg:`fetch` (for merge commit only)
 - :hg:`graft`
 - :hg:`histedit`
 - :hg:`import`
 - :hg:`qfold`, :hg:`qnew` and :hg:`qrefresh`
 - :hg:`rebase`
 - :hg:`shelve`
 - :hg:`sign`
 - :hg:`tag`
 - :hg:`transplant`
 
 Configuring items below instead of ``changeset`` allows showing
 customized message only for specific actions, or showing different
 messages for each action.
 
 - ``changeset.backout`` for :hg:`backout`
 - ``changeset.commit.amend.merge`` for :hg:`commit --amend` on merges
 - ``changeset.commit.amend.normal`` for :hg:`commit --amend` on other
 - ``changeset.commit.normal.merge`` for :hg:`commit` on merges
 - ``changeset.commit.normal.normal`` for :hg:`commit` on other
 - ``changeset.fetch`` for :hg:`fetch` (impling merge commit)
 - ``changeset.gpg.sign`` for :hg:`sign`
 - ``changeset.graft`` for :hg:`graft`
 - ``changeset.histedit.edit`` for ``edit`` of :hg:`histedit`
 - ``changeset.histedit.fold`` for ``fold`` of :hg:`histedit`
 - ``changeset.histedit.mess`` for ``mess`` of :hg:`histedit`
 - ``changeset.histedit.pick`` for ``pick`` of :hg:`histedit`
 - ``changeset.import.bypass`` for :hg:`import --bypass`
 - ``changeset.import.normal.merge`` for :hg:`import` on merges
 - ``changeset.import.normal.normal`` for :hg:`import` on other
 - ``changeset.mq.qnew`` for :hg:`qnew`
 - ``changeset.mq.qfold`` for :hg:`qfold`
 - ``changeset.mq.qrefresh`` for :hg:`qrefresh`
 - ``changeset.rebase.collapse`` for :hg:`rebase --collapse`
 - ``changeset.rebase.merge`` for :hg:`rebase` on merges
 - ``changeset.rebase.normal`` for :hg:`rebase` on other
 - ``changeset.shelve.shelve`` for :hg:`shelve`
 - ``changeset.tag.add`` for :hg:`tag` without ``--remove``
 - ``changeset.tag.remove`` for :hg:`tag --remove`
 - ``changeset.transplant.merge`` for :hg:`transplant` on merges
 - ``changeset.transplant.normal`` for :hg:`transplant` on other
 
 These dot-separated lists of names are treated as hierarchical ones.
 For example, ``changeset.tag.remove`` customizes the commit message
 only for :hg:`tag --remove`, but ``changeset.tag`` customizes the
 commit message for :hg:`tag` regardless of ``--remove`` option.
 
 When the external editor is invoked for a commit, the corresponding
 dot-separated list of names without the ``changeset.`` prefix
 (e.g. ``commit.normal.normal``) is in the ``HGEDITFORM`` environment
 variable.
 
 In this section, items other than ``changeset`` can be referred from
 others. For example, the configuration to list committed files up
 below can be referred as ``{listupfiles}``::
 
     [committemplate]
     listupfiles = {file_adds %
        "HG: added {file}\n"     }{file_mods %
        "HG: changed {file}\n"   }{file_dels %
        "HG: removed {file}\n"   }{if(files, "",
        "HG: no files changed\n")}
 
 ``decode/encode``
 -----------------
 
 Filters for transforming files on checkout/checkin. This would
 typically be used for newline processing or other
 localization/canonicalization of files.
 
 Filters consist of a filter pattern followed by a filter command.
 Filter patterns are globs by default, rooted at the repository root.
 For example, to match any file ending in ``.txt`` in the root
 directory only, use the pattern ``*.txt``. To match any file ending
 in ``.c`` anywhere in the repository, use the pattern ``**.c``.
 For each file only the first matching filter applies.
 
 The filter command can start with a specifier, either ``pipe:`` or
 ``tempfile:``. If no specifier is given, ``pipe:`` is used by default.
 
 A ``pipe:`` command must accept data on stdin and return the transformed
 data on stdout.
 
 Pipe example::
 
   [encode]
   # uncompress gzip files on checkin to improve delta compression
   # note: not necessarily a good idea, just an example
   *.gz = pipe: gunzip
 
   [decode]
   # recompress gzip files when writing them to the working dir (we
   # can safely omit "pipe:", because it's the default)
   *.gz = gzip
 
 A ``tempfile:`` command is a template. The string ``INFILE`` is replaced
 with the name of a temporary file that contains the data to be
 filtered by the command. The string ``OUTFILE`` is replaced with the name
 of an empty temporary file, where the filtered data must be written by
 the command.
 
 .. container:: windows
 
    .. note::
 
      The tempfile mechanism is recommended for Windows systems,
      where the standard shell I/O redirection operators often have
      strange effects and may corrupt the contents of your files.
 
 This filter mechanism is used internally by the ``eol`` extension to
 translate line ending characters between Windows (CRLF) and Unix (LF)
 format. We suggest you use the ``eol`` extension for convenience.
 
 
 ``defaults``
 ------------
 
 (defaults are deprecated. Don't use them. Use aliases instead.)
 
 Use the ``[defaults]`` section to define command defaults, i.e. the
 default options/arguments to pass to the specified commands.
 
 The following example makes :hg:`log` run in verbose mode, and
 :hg:`status` show only the modified files, by default::
 
   [defaults]
   log = -v
   status = -m
 
 The actual commands, instead of their aliases, must be used when
 defining command defaults. The command defaults will also be applied
 to the aliases of the commands defined.
 
 
 ``diff``
 --------
 
 Settings used when displaying diffs. Everything except for ``unified``
 is a Boolean and defaults to False. See :hg:`help config.annotate`
 for related options for the annotate command.
 
 ``git``
     Use git extended diff format.
 
 ``nobinary``
     Omit git binary patches.
 
 ``nodates``
     Don't include dates in diff headers.
 
 ``noprefix``
     Omit 'a/' and 'b/' prefixes from filenames. Ignored in plain mode.
 
 ``showfunc``
     Show which function each change is in.
 
 ``ignorews``
     Ignore white space when comparing lines.
 
 ``ignorewsamount``
     Ignore changes in the amount of white space.
 
 ``ignoreblanklines``
     Ignore changes whose lines are all blank.
 
 ``unified``
     Number of lines of context to show.
 
 ``word-diff``
     Highlight changed words.
 
 ``email``
 ---------
 
 Settings for extensions that send email messages.
 
 ``from``
     Optional. Email address to use in "From" header and SMTP envelope
     of outgoing messages.
 
 ``to``
     Optional. Comma-separated list of recipients' email addresses.
 
 ``cc``
     Optional. Comma-separated list of carbon copy recipients'
     email addresses.
 
 ``bcc``
     Optional. Comma-separated list of blind carbon copy recipients'
     email addresses.
 
 ``method``
     Optional. Method to use to send email messages. If value is ``smtp``
     (default), use SMTP (see the ``[smtp]`` section for configuration).
     Otherwise, use as name of program to run that acts like sendmail
     (takes ``-f`` option for sender, list of recipients on command line,
     message on stdin). Normally, setting this to ``sendmail`` or
     ``/usr/sbin/sendmail`` is enough to use sendmail to send messages.
 
 ``charsets``
     Optional. Comma-separated list of character sets considered
     convenient for recipients. Addresses, headers, and parts not
     containing patches of outgoing messages will be encoded in the
     first character set to which conversion from local encoding
     (``$HGENCODING``, ``ui.fallbackencoding``) succeeds. If correct
     conversion fails, the text in question is sent as is.
     (default: '')
 
     Order of outgoing email character sets:
 
     1. ``us-ascii``: always first, regardless of settings
     2. ``email.charsets``: in order given by user
     3. ``ui.fallbackencoding``: if not in email.charsets
     4. ``$HGENCODING``: if not in email.charsets
     5. ``utf-8``: always last, regardless of settings
 
 Email example::
 
   [email]
   from = Joseph User <[e-mail]>
   method = /usr/sbin/sendmail
   # charsets for western Europeans
   # us-ascii, utf-8 omitted, as they are tried first and last
   charsets = iso-8859-1, iso-8859-15, windows-1252
 
 
 ``extensions``
 --------------
 
 Mercurial has an extension mechanism for adding new features. To
 enable an extension, create an entry for it in this section.
 
 If you know that the extension is already in Python's search path,
 you can give the name of the module, followed by ``=``, with nothing
 after the ``=``.
 
 Otherwise, give a name that you choose, followed by ``=``, followed by
 the path to the ``.py`` file (including the file name extension) that
 defines the extension.
 
 To explicitly disable an extension that is enabled in an hgrc of
 broader scope, prepend its path with ``!``, as in ``foo = !/ext/path``
 or ``foo = !`` when path is not supplied.
 
 Example for ``~/.hgrc``::
 
   [extensions]
   # (the churn extension will get loaded from Mercurial's path)
   churn =
   # (this extension will get loaded from the file specified)
   myfeature = ~/.hgext/myfeature.py
 
 If an extension fails to load, a warning will be issued, and Mercurial will
 proceed. To enforce that an extension must be loaded, one can set the `required`
 suboption in the config::
 
   [extensions]
   myfeature = ~/.hgext/myfeature.py
   myfeature:required = yes
 
 To debug extension loading issue, one can add `--traceback` to their mercurial
 invocation.
 
 A default setting can we set using the special `*` extension key::
 
   [extensions]
   *:required = yes
   myfeature = ~/.hgext/myfeature.py
   rebase=
 
 
 ``format``
 ----------
 
 Configuration that controls the repository format. Newer format options are more
 powerful, but incompatible with some older versions of Mercurial. Format options
 are considered at repository initialization only. You need to make a new clone
 for config changes to be taken into account.
 
 For more details about repository format and version compatibility, see
 https://www.mercurial-scm.org/wiki/MissingRequirement
 
 ``usegeneraldelta``
     Enable or disable the "generaldelta" repository format which improves
     repository compression by allowing "revlog" to store deltas against
     arbitrary revisions instead of the previously stored one. This provides
     significant improvement for repositories with branches.
 
     Repositories with this on-disk format require Mercurial version 1.9.
 
     Enabled by default.
 
 ``dotencode``
     Enable or disable the "dotencode" repository format which enhances
     the "fncache" repository format (which has to be enabled to use
     dotencode) to avoid issues with filenames starting with "._" on
     Mac OS X and spaces on Windows.
 
     Repositories with this on-disk format require Mercurial version 1.7.
 
     Enabled by default.
 
 ``usefncache``
     Enable or disable the "fncache" repository format which enhances
     the "store" repository format (which has to be enabled to use
     fncache) to allow longer filenames and avoids using Windows
     reserved names, e.g. "nul".
 
     Repositories with this on-disk format require Mercurial version 1.1.
 
     Enabled by default.
 
 ``use-dirstate-v2``
     Enable or disable the experimental "dirstate-v2" feature. The dirstate
     functionality is shared by all commands interacting with the working copy.
     The new version is more robust, faster and stores more information.
 
     The performance-improving version of this feature is currently only
     implemented in Rust (see :hg:`help rust`), so people not using a version of
     Mercurial compiled with the Rust parts might actually suffer some slowdown.
     For this reason, such versions will by default refuse to access repositories
     with "dirstate-v2" enabled.
 
     This behavior can be adjusted via configuration: check
     :hg:`help config.storage.dirstate-v2.slow-path` for details.
 
     Repositories with this on-disk format require Mercurial 6.0 or above.
 
     By default this format variant is disabled if the fast implementation is not
     available, and enabled by default if the fast implementation is available.
 
     To accomodate installations of Mercurial without the fast implementation,
     you can downgrade your repository. To do so run the following command:
 
     $ hg debugupgraderepo \
           --run \
           --config format.use-dirstate-v2=False \
           --config storage.dirstate-v2.slow-path=allow
 
     For a more comprehensive guide, see :hg:`help internals.dirstate-v2`.
 
 ``use-dirstate-v2.automatic-upgrade-of-mismatching-repositories``
    When enabled, an automatic upgrade will be triggered when a repository format
    does not match its `use-dirstate-v2` config.
 
    This is an advanced behavior that most users will not need. We recommend you
    don't use this unless you are a seasoned administrator of a Mercurial install
    base.
 
    Automatic upgrade means that any process accessing the repository will
    upgrade the repository format to use `dirstate-v2`. This only triggers if a
    change is needed. This also applies to operations that would have been
    read-only (like hg status).
 
    If the repository cannot be locked, the automatic-upgrade operation will be
    skipped. The next operation will attempt it again.
 
    This configuration will apply for moves in any direction, either adding the
    `dirstate-v2` format if `format.use-dirstate-v2=yes` or removing the
    `dirstate-v2` requirement if `format.use-dirstate-v2=no`. So we recommend
    setting both this value and `format.use-dirstate-v2` at the same time.
 
 ``use-dirstate-v2.automatic-upgrade-of-mismatching-repositories:quiet``
     Hide message when performing such automatic upgrade.
 
 ``use-dirstate-tracked-hint``
     Enable or disable the writing of "tracked key" file alongside the dirstate.
     (default to disabled)
 
     That "tracked-hint" can help external automations to detect changes to the
     set of tracked files. (i.e the result of `hg files` or `hg status -macd`)
 
     The tracked-hint is written in a new `.hg/dirstate-tracked-hint`. That file
     contains two lines:
     - the first line is the file version (currently: 1),
     - the second line contains the "tracked-hint".
     That file is written right after the dirstate is written.
 
     The tracked-hint changes whenever the set of file tracked in the dirstate
     changes. The general idea is:
     - if the hint is identical, the set of tracked file SHOULD be identical,
     - if the hint is different, the set of tracked file MIGHT be different.
 
     The "hint is identical" case uses `SHOULD` as the dirstate and the hint file
     are two distinct files and therefore that cannot be read or written to in an
     atomic way. If the key is identical, nothing garantees that the dirstate is
     not updated right after the hint file. This is considered a negligible
     limitation for the intended usecase. It is actually possible to prevent this
     race by taking the repository lock during read operations.
 
     They are two "ways" to use this feature:
 
     1) monitoring changes to the `.hg/dirstate-tracked-hint`, if the file
     changes, the tracked set might have changed.
 
     2) storing the value and comparing it to a later value.
 
 
 ``use-dirstate-tracked-hint.automatic-upgrade-of-mismatching-repositories``
    When enabled, an automatic upgrade will be triggered when a repository format
    does not match its `use-dirstate-tracked-hint` config.
 
    This is an advanced behavior that most users will not need. We recommend you
    don't use this unless you are a seasoned administrator of a Mercurial install
    base.
 
    Automatic upgrade means that any process accessing the repository will
    upgrade the repository format to use `dirstate-tracked-hint`. This only
    triggers if a change is needed. This also applies to operations that would
    have been read-only (like hg status).
 
    If the repository cannot be locked, the automatic-upgrade operation will be
    skipped. The next operation will attempt it again.
 
    This configuration will apply for moves in any direction, either adding the
    `dirstate-tracked-hint` format if `format.use-dirstate-tracked-hint=yes` or
    removing the `dirstate-tracked-hint` requirement if
    `format.use-dirstate-tracked-hint=no`. So we recommend setting both this
    value and `format.use-dirstate-tracked-hint` at the same time.
 
 
 ``use-dirstate-tracked-hint.automatic-upgrade-of-mismatching-repositories:quiet``
     Hide message when performing such automatic upgrade.
 
 
 ``use-persistent-nodemap``
     Enable or disable the "persistent-nodemap" feature which improves
     performance if the Rust extensions are available.
 
     The "persistent-nodemap" persist the "node -> rev" on disk removing the
     need to dynamically build that mapping for each Mercurial invocation. This
     significantly reduces the startup cost of various local and server-side
     operation for larger repositories.
 
     The performance-improving version of this feature is currently only
     implemented in Rust (see :hg:`help rust`), so people not using a version of
     Mercurial compiled with the Rust parts might actually suffer some slowdown.
     For this reason, such versions will by default refuse to access repositories
     with "persistent-nodemap".
 
     This behavior can be adjusted via configuration: check
     :hg:`help config.storage.revlog.persistent-nodemap.slow-path` for details.
 
     Repositories with this on-disk format require Mercurial 5.4 or above.
 
     By default this format variant is disabled if the fast implementation is not
     available, and enabled by default if the fast implementation is available.
 
     To accomodate installations of Mercurial without the fast implementation,
     you can downgrade your repository. To do so run the following command:
 
     $ hg debugupgraderepo \
           --run \
           --config format.use-persistent-nodemap=False \
           --config storage.revlog.persistent-nodemap.slow-path=allow
 
 ``use-share-safe``
     Enforce "safe" behaviors for all "shares" that access this repository.
 
     With this feature, "shares" using this repository as a source will:
 
     * read the source repository's configuration (`<source>/.hg/hgrc`).
     * read and use the source repository's "requirements"
       (except the working copy specific one).
 
     Without this feature, "shares" using this repository as a source will:
 
     * keep tracking the repository "requirements" in the share only, ignoring
       the source "requirements", possibly diverging from them.
     * ignore source repository config. This can create problems, like silently
       ignoring important hooks.
 
     Beware that existing shares will not be upgraded/downgraded, and by
     default, Mercurial will refuse to interact with them until the mismatch
     is resolved. See :hg:`help config.share.safe-mismatch.source-safe` and
     :hg:`help config.share.safe-mismatch.source-not-safe` for details.
 
     Introduced in Mercurial 5.7.
 
     Enabled by default in Mercurial 6.1.
 
 ``use-share-safe.automatic-upgrade-of-mismatching-repositories``
    When enabled, an automatic upgrade will be triggered when a repository format
    does not match its `use-share-safe` config.
 
    This is an advanced behavior that most users will not need. We recommend you
    don't use this unless you are a seasoned administrator of a Mercurial install
    base.
 
    Automatic upgrade means that any process accessing the repository will
    upgrade the repository format to use `share-safe`. This only triggers if a
    change is needed. This also applies to operation that would have been
    read-only (like hg status).
 
    If the repository cannot be locked, the automatic-upgrade operation will be
    skipped. The next operation will attempt it again.
 
    This configuration will apply for moves in any direction, either adding the
    `share-safe` format if `format.use-share-safe=yes` or removing the
    `share-safe` requirement if `format.use-share-safe=no`. So we recommend
    setting both this value and `format.use-share-safe` at the same time.
 
 ``use-share-safe.automatic-upgrade-of-mismatching-repositories:quiet``
     Hide message when performing such automatic upgrade.
 
 ``usestore``
     Enable or disable the "store" repository format which improves
     compatibility with systems that fold case or otherwise mangle
     filenames. Disabling this option will allow you to store longer filenames
     in some situations at the expense of compatibility.
 
     Repositories with this on-disk format require Mercurial version 0.9.4.
 
     Enabled by default.
 
 ``sparse-revlog``
     Enable or disable the ``sparse-revlog`` delta strategy. This format improves
     delta re-use inside revlog. For very branchy repositories, it results in a
     smaller store. For repositories with many revisions, it also helps
     performance (by using shortened delta chains.)
 
     Repositories with this on-disk format require Mercurial version 4.7
 
     Enabled by default.
 
 ``revlog-compression``
     Compression algorithm used by revlog. Supported values are `zlib` and
     `zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is
     a newer format that is usually a net win over `zlib`, operating faster at
     better compression rates. Use `zstd` to reduce CPU usage. Multiple values
     can be specified, the first available one will be used.
 
     On some systems, the Mercurial installation may lack `zstd` support.
 
     Default is `zstd` if available, `zlib` otherwise.
 
 ``bookmarks-in-store``
     Store bookmarks in .hg/store/. This means that bookmarks are shared when
     using `hg share` regardless of the `-B` option.
 
     Repositories with this on-disk format require Mercurial version 5.1.
 
     Disabled by default.
 
 
 ``graph``
 ---------
 
 Web graph view configuration. This section let you change graph
 elements display properties by branches, for instance to make the
 ``default`` branch stand out.
 
 Each line has the following format::
 
     <branch>.<argument> = <value>
 
 where ``<branch>`` is the name of the branch being
 customized. Example::
 
     [graph]
     # 2px width
     default.width = 2
     # red color
     default.color = FF0000
 
 Supported arguments:
 
 ``width``
     Set branch edges width in pixels.
 
 ``color``
     Set branch edges color in hexadecimal RGB notation.
 
 ``hooks``
 ---------
 
 Commands or Python functions that get automatically executed by
 various actions such as starting or finishing a commit. Multiple
 hooks can be run for the same action by appending a suffix to the
 action. Overriding a site-wide hook can be done by changing its
 value or setting it to an empty string.  Hooks can be prioritized
 by adding a prefix of ``priority.`` to the hook name on a new line
 and setting the priority. The default priority is 0.
 
 Example ``.hg/hgrc``::
 
   [hooks]
   # update working directory after adding changesets
   changegroup.update = hg update
   # do not use the site-wide hook
   incoming =
   incoming.email = /my/email/hook
   incoming.autobuild = /my/build/hook
   # force autobuild hook to run before other incoming hooks
   priority.incoming.autobuild = 1
   ###  control HGPLAIN setting when running autobuild hook
   # HGPLAIN always set (default from Mercurial 5.7)
   incoming.autobuild:run-with-plain = yes
   # HGPLAIN never set
   incoming.autobuild:run-with-plain = no
   # HGPLAIN inherited from environment (default before Mercurial 5.7)
   incoming.autobuild:run-with-plain = auto
 
 Most hooks are run with environment variables set that give useful
 additional information. For each hook below, the environment variables
 it is passed are listed with names in the form ``$HG_foo``. The
 ``$HG_HOOKTYPE`` and ``$HG_HOOKNAME`` variables are set for all hooks.
 They contain the type of hook which triggered the run and the full name
 of the hook in the config, respectively. In the example above, this will
 be ``$HG_HOOKTYPE=incoming`` and ``$HG_HOOKNAME=incoming.email``.
 
 .. container:: windows
 
   Some basic Unix syntax can be enabled for portability, including ``$VAR``
   and ``${VAR}`` style variables.  A ``~`` followed by ``\`` or ``/`` will
   be expanded to ``%USERPROFILE%`` to simulate a subset of tilde expansion
   on Unix.  To use a literal ``$`` or ``~``, it must be escaped with a back
   slash or inside of a strong quote.  Strong quotes will be replaced by
   double quotes after processing.
 
   This feature is enabled by adding a prefix of ``tonative.`` to the hook
   name on a new line, and setting it to ``True``.  For example::
 
     [hooks]
     incoming.autobuild = /my/build/hook
     # enable translation to cmd.exe syntax for autobuild hook
     tonative.incoming.autobuild = True
 
 ``changegroup``
   Run after a changegroup has been added via push, pull or unbundle.  The ID of
   the first new changeset is in ``$HG_NODE`` and last is in ``$HG_NODE_LAST``.
   The URL from which changes came is in ``$HG_URL``.
 
 ``commit``
   Run after a changeset has been created in the local repository. The ID
   of the newly created changeset is in ``$HG_NODE``. Parent changeset
   IDs are in ``$HG_PARENT1`` and ``$HG_PARENT2``.
 
 ``incoming``
   Run after a changeset has been pulled, pushed, or unbundled into
   the local repository. The ID of the newly arrived changeset is in
   ``$HG_NODE``. The URL that was source of the changes is in ``$HG_URL``.
 
 ``outgoing``
   Run after sending changes from the local repository to another. The ID of
   first changeset sent is in ``$HG_NODE``. The source of operation is in
   ``$HG_SOURCE``. Also see :hg:`help config.hooks.preoutgoing`.
 
 ``post-<command>``
   Run after successful invocations of the associated command. The
   contents of the command line are passed as ``$HG_ARGS`` and the result
   code in ``$HG_RESULT``. Parsed command line arguments are passed as
   ``$HG_PATS`` and ``$HG_OPTS``. These contain string representations of
   the python data internally passed to <command>. ``$HG_OPTS`` is a
   dictionary of options (with unspecified options set to their defaults).
   ``$HG_PATS`` is a list of arguments. Hook failure is ignored.
 
 ``fail-<command>``
   Run after a failed invocation of an associated command. The contents
   of the command line are passed as ``$HG_ARGS``. Parsed command line
   arguments are passed as ``$HG_PATS`` and ``$HG_OPTS``. These contain
   string representations of the python data internally passed to
   <command>. ``$HG_OPTS`` is a dictionary of options (with unspecified
   options set to their defaults). ``$HG_PATS`` is a list of arguments.
   Hook failure is ignored.
 
 ``pre-<command>``
   Run before executing the associated command. The contents of the
   command line are passed as ``$HG_ARGS``. Parsed command line arguments
   are passed as ``$HG_PATS`` and ``$HG_OPTS``. These contain string
   representations of the data internally passed to <command>. ``$HG_OPTS``
   is a dictionary of options (with unspecified options set to their
   defaults). ``$HG_PATS`` is a list of arguments. If the hook returns
   failure, the command doesn't execute and Mercurial returns the failure
   code.
 
 ``prechangegroup``
   Run before a changegroup is added via push, pull or unbundle. Exit
   status 0 allows the changegroup to proceed. A non-zero status will
   cause the push, pull or unbundle to fail. The URL from which changes
   will come is in ``$HG_URL``.
 
 ``precommit``
   Run before starting a local commit. Exit status 0 allows the
   commit to proceed. A non-zero status will cause the commit to fail.
   Parent changeset IDs are in ``$HG_PARENT1`` and ``$HG_PARENT2``.
 
 ``prelistkeys``
   Run before listing pushkeys (like bookmarks) in the
   repository. A non-zero status will cause failure. The key namespace is
   in ``$HG_NAMESPACE``.
 
 ``preoutgoing``
   Run before collecting changes to send from the local repository to
   another. A non-zero status will cause failure. This lets you prevent
   pull over HTTP or SSH. It can also prevent propagating commits (via
   local pull, push (outbound) or bundle commands), but not completely,
   since you can just copy files instead. The source of operation is in
   ``$HG_SOURCE``. If "serve", the operation is happening on behalf of a remote
   SSH or HTTP repository. If "push", "pull" or "bundle", the operation
   is happening on behalf of a repository on same system.
 
 ``prepushkey``
   Run before a pushkey (like a bookmark) is added to the
   repository. A non-zero status will cause the key to be rejected. The
   key namespace is in ``$HG_NAMESPACE``, the key is in ``$HG_KEY``,
   the old value (if any) is in ``$HG_OLD``, and the new value is in
   ``$HG_NEW``.
 
 ``pretag``
   Run before creating a tag. Exit status 0 allows the tag to be
   created. A non-zero status will cause the tag to fail. The ID of the
   changeset to tag is in ``$HG_NODE``. The name of tag is in ``$HG_TAG``. The
   tag is local if ``$HG_LOCAL=1``, or in the repository if ``$HG_LOCAL=0``.
 
+``pretransmit-inline-clone-bundle``
+  Run before transferring an inline clonebundle to the peer.
+  If the exit status is 0, the inline clonebundle will be allowed to be
+  transferred. A non-zero status will cause the transfer to fail.
+  The path of the inline clonebundle is in ``$HG_CLONEBUNDLEPATH``.
+
 ``pretxnopen``
   Run before any new repository transaction is open. The reason for the
   transaction will be in ``$HG_TXNNAME``, and a unique identifier for the
   transaction will be in ``$HG_TXNID``. A non-zero status will prevent the
   transaction from being opened.
 
 ``pretxnclose``
   Run right before the transaction is actually finalized. Any repository change
   will be visible to the hook program. This lets you validate the transaction
   content or change it. Exit status 0 allows the commit to proceed. A non-zero
   status will cause the transaction to be rolled back. The reason for the
   transaction opening will be in ``$HG_TXNNAME``, and a unique identifier for
   the transaction will be in ``$HG_TXNID``. The rest of the available data will
   vary according the transaction type.  Changes unbundled to the repository will
   add ``$HG_URL`` and ``$HG_SOURCE``.  New changesets will add ``$HG_NODE`` (the
   ID of the first added changeset), ``$HG_NODE_LAST`` (the ID of the last added
   changeset).  Bookmark and phase changes will set ``$HG_BOOKMARK_MOVED`` and
   ``$HG_PHASES_MOVED`` to ``1`` respectively.  The number of new obsmarkers, if
   any, will be in ``$HG_NEW_OBSMARKERS``, etc.
 
 ``pretxnclose-bookmark``
   Run right before a bookmark change is actually finalized. Any repository
   change will be visible to the hook program. This lets you validate the
   transaction content or change it. Exit status 0 allows the commit to
   proceed. A non-zero status will cause the transaction to be rolled back.
   The name of the bookmark will be available in ``$HG_BOOKMARK``, the new
   bookmark location will be available in ``$HG_NODE`` while the previous
   location will be available in ``$HG_OLDNODE``. In case of a bookmark
   creation ``$HG_OLDNODE`` will be empty. In case of deletion ``$HG_NODE``
   will be empty.
   In addition, the reason for the transaction opening will be in
   ``$HG_TXNNAME``, and a unique identifier for the transaction will be in
   ``$HG_TXNID``.
 
 ``pretxnclose-phase``
   Run right before a phase change is actually finalized. Any repository change
   will be visible to the hook program. This lets you validate the transaction
   content or change it. Exit status 0 allows the commit to proceed.  A non-zero
   status will cause the transaction to be rolled back. The hook is called
   multiple times, once for each revision affected by a phase change.
   The affected node is available in ``$HG_NODE``, the phase in ``$HG_PHASE``
   while the previous ``$HG_OLDPHASE``. In case of new node, ``$HG_OLDPHASE``
   will be empty.  In addition, the reason for the transaction opening will be in
   ``$HG_TXNNAME``, and a unique identifier for the transaction will be in
   ``$HG_TXNID``. The hook is also run for newly added revisions. In this case
   the ``$HG_OLDPHASE`` entry will be empty.
 
 ``txnclose``
   Run after any repository transaction has been committed. At this
   point, the transaction can no longer be rolled back. The hook will run
   after the lock is released. See :hg:`help config.hooks.pretxnclose` for
   details about available variables.
 
 ``txnclose-bookmark``
   Run after any bookmark change has been committed. At this point, the
   transaction can no longer be rolled back. The hook will run after the lock
   is released. See :hg:`help config.hooks.pretxnclose-bookmark` for details
   about available variables.
 
 ``txnclose-phase``
   Run after any phase change has been committed. At this point, the
   transaction can no longer be rolled back. The hook will run after the lock
   is released. See :hg:`help config.hooks.pretxnclose-phase` for details about
   available variables.
 
 ``txnabort``
   Run when a transaction is aborted. See :hg:`help config.hooks.pretxnclose`
   for details about available variables.
 
 ``pretxnchangegroup``
   Run after a changegroup has been added via push, pull or unbundle, but before
   the transaction has been committed. The changegroup is visible to the hook
   program. This allows validation of incoming changes before accepting them.
   The ID of the first new changeset is in ``$HG_NODE`` and last is in
   ``$HG_NODE_LAST``. Exit status 0 allows the transaction to commit. A non-zero
   status will cause the transaction to be rolled back, and the push, pull or
   unbundle will fail. The URL that was the source of changes is in ``$HG_URL``.
 
 ``pretxncommit``
   Run after a changeset has been created, but before the transaction is
   committed. The changeset is visible to the hook program. This allows
   validation of the commit message and changes. Exit status 0 allows the
   commit to proceed. A non-zero status will cause the transaction to
   be rolled back. The ID of the new changeset is in ``$HG_NODE``. The parent
   changeset IDs are in ``$HG_PARENT1`` and ``$HG_PARENT2``.
 
 ``preupdate``
   Run before updating the working directory. Exit status 0 allows
   the update to proceed. A non-zero status will prevent the update.
   The changeset ID of first new parent is in ``$HG_PARENT1``. If updating to a
   merge, the ID of second new parent is in ``$HG_PARENT2``.
 
 ``listkeys``
   Run after listing pushkeys (like bookmarks) in the repository. The
   key namespace is in ``$HG_NAMESPACE``. ``$HG_VALUES`` is a
   dictionary containing the keys and values.
 
 ``pushkey``
   Run after a pushkey (like a bookmark) is added to the
   repository. The key namespace is in ``$HG_NAMESPACE``, the key is in
   ``$HG_KEY``, the old value (if any) is in ``$HG_OLD``, and the new
   value is in ``$HG_NEW``.
 
 ``tag``
   Run after a tag is created. The ID of the tagged changeset is in ``$HG_NODE``.
   The name of tag is in ``$HG_TAG``. The tag is local if ``$HG_LOCAL=1``, or in
   the repository if ``$HG_LOCAL=0``.
 
 ``update``
   Run after updating the working directory. The changeset ID of first
   new parent is in ``$HG_PARENT1``. If updating to a merge, the ID of second new
   parent is in ``$HG_PARENT2``. If the update succeeded, ``$HG_ERROR=0``. If the
   update failed (e.g. because conflicts were not resolved), ``$HG_ERROR=1``.
 
 .. note::
 
    It is generally better to use standard hooks rather than the
    generic pre- and post- command hooks, as they are guaranteed to be
    called in the appropriate contexts for influencing transactions.
    Also, hooks like "commit" will be called in all contexts that
    generate a commit (e.g. tag) and not just the commit command.
 
 .. note::
 
    Environment variables with empty values may not be passed to
    hooks on platforms such as Windows. As an example, ``$HG_PARENT2``
    will have an empty value under Unix-like platforms for non-merge
    changesets, while it will not be available at all under Windows.
 
 The syntax for Python hooks is as follows::
 
   hookname = python:modulename.submodule.callable
   hookname = python:/path/to/python/module.py:callable
 
 Python hooks are run within the Mercurial process. Each hook is
 called with at least three keyword arguments: a ui object (keyword
 ``ui``), a repository object (keyword ``repo``), and a ``hooktype``
 keyword that tells what kind of hook is used. Arguments listed as
 environment variables above are passed as keyword arguments, with no
 ``HG_`` prefix, and names in lower case.
 
 If a Python hook returns a "true" value or raises an exception, this
 is treated as a failure.
 
 
 ``hostfingerprints``
 --------------------
 
 (Deprecated. Use ``[hostsecurity]``'s ``fingerprints`` options instead.)
 
 Fingerprints of the certificates of known HTTPS servers.
 
 A HTTPS connection to a server with a fingerprint configured here will
 only succeed if the servers certificate matches the fingerprint.
 This is very similar to how ssh known hosts works.
 
 The fingerprint is the SHA-1 hash value of the DER encoded certificate.
 Multiple values can be specified (separated by spaces or commas). This can
 be used to define both old and new fingerprints while a host transitions
 to a new certificate.
 
 The CA chain and web.cacerts is not used for servers with a fingerprint.
 
 For example::
 
     [hostfingerprints]
     hg.intevation.de = fc:e2:8d:d9:51:cd:cb:c1:4d:18:6b:b7:44:8d:49:72:57:e6:cd:33
     hg.intevation.org = fc:e2:8d:d9:51:cd:cb:c1:4d:18:6b:b7:44:8d:49:72:57:e6:cd:33
 
 ``hostsecurity``
 ----------------
 
 Used to specify global and per-host security settings for connecting to
 other machines.
 
 The following options control default behavior for all hosts.
 
 ``ciphers``
     Defines the cryptographic ciphers to use for connections.
 
     Value must be a valid OpenSSL Cipher List Format as documented at
     https://www.openssl.org/docs/manmaster/apps/ciphers.html#CIPHER-LIST-FORMAT.
 
     This setting is for advanced users only. Setting to incorrect values
     can significantly lower connection security or decrease performance.
     You have been warned.
 
     This option requires Python 2.7.
 
 ``minimumprotocol``
     Defines the minimum channel encryption protocol to use.
 
     By default, the highest version of TLS supported by both client and server
     is used.
 
     Allowed values are: ``tls1.0``, ``tls1.1``, ``tls1.2``.
 
     When running on an old Python version, only ``tls1.0`` is allowed since
     old versions of Python only support up to TLS 1.0.
 
     When running a Python that supports modern TLS versions, the default is
     ``tls1.1``. ``tls1.0`` can still be used to allow TLS 1.0. However, this
     weakens security and should only be used as a feature of last resort if
     a server does not support TLS 1.1+.
 
 Options in the ``[hostsecurity]`` section can have the form
 ``hostname``:``setting``. This allows multiple settings to be defined on a
 per-host basis.
 
 The following per-host settings can be defined.
 
 ``ciphers``
     This behaves like ``ciphers`` as described above except it only applies
     to the host on which it is defined.
 
 ``fingerprints``
     A list of hashes of the DER encoded peer/remote certificate. Values have
     the form ``algorithm``:``fingerprint``. e.g.
     ``sha256:c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2``.
     In addition, colons (``:``) can appear in the fingerprint part.
 
     The following algorithms/prefixes are supported: ``sha1``, ``sha256``,
     ``sha512``.
 
     Use of ``sha256`` or ``sha512`` is preferred.
 
     If a fingerprint is specified, the CA chain is not validated for this
     host and Mercurial will require the remote certificate to match one
     of the fingerprints specified. This means if the server updates its
     certificate, Mercurial will abort until a new fingerprint is defined.
     This can provide stronger security than traditional CA-based validation
     at the expense of convenience.
 
     This option takes precedence over ``verifycertsfile``.
 
 ``minimumprotocol``
     This behaves like ``minimumprotocol`` as described above except it
     only applies to the host on which it is defined.
 
 ``verifycertsfile``
     Path to file a containing a list of PEM encoded certificates used to
     verify the server certificate. Environment variables and ``~user``
     constructs are expanded in the filename.
 
     The server certificate or the certificate's certificate authority (CA)
     must match a certificate from this file or certificate verification
     will fail and connections to the server will be refused.
 
     If defined, only certificates provided by this file will be used:
     ``web.cacerts`` and any system/default certificates will not be
     used.
 
     This option has no effect if the per-host ``fingerprints`` option
     is set.
 
     The format of the file is as follows::
 
         -----BEGIN CERTIFICATE-----
         ... (certificate in base64 PEM encoding) ...
         -----END CERTIFICATE-----
         -----BEGIN CERTIFICATE-----
         ... (certificate in base64 PEM encoding) ...
         -----END CERTIFICATE-----
 
 For example::
 
     [hostsecurity]
     hg.example.com:fingerprints = sha256:c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2
     hg2.example.com:fingerprints = sha1:914f1aff87249c09b6859b88b1906d30756491ca, sha1:fc:e2:8d:d9:51:cd:cb:c1:4d:18:6b:b7:44:8d:49:72:57:e6:cd:33
     hg3.example.com:fingerprints = sha256:9a:b0:dc:e2:75:ad:8a:b7:84:58:e5:1f:07:32:f1:87:e6:bd:24:22:af:b7:ce:8e:9c:b4:10:cf:b9:f4:0e:d2
     foo.example.com:verifycertsfile = /etc/ssl/trusted-ca-certs.pem
 
 To change the default minimum protocol version to TLS 1.2 but to allow TLS 1.1
 when connecting to ``hg.example.com``::
 
     [hostsecurity]
     minimumprotocol = tls1.2
     hg.example.com:minimumprotocol = tls1.1
 
 ``http_proxy``
 --------------
 
 Used to access web-based Mercurial repositories through a HTTP
 proxy.
 
 ``host``
     Host name and (optional) port of the proxy server, for example
     "myproxy:8000".
 
 ``no``
     Optional. Comma-separated list of host names that should bypass
     the proxy.
 
 ``passwd``
     Optional. Password to authenticate with at the proxy server.
 
 ``user``
     Optional. User name to authenticate with at the proxy server.
 
 ``always``
     Optional. Always use the proxy, even for localhost and any entries
     in ``http_proxy.no``. (default: False)
 
 ``http``
 --------
 
 Used to configure access to Mercurial repositories via HTTP.
 
 ``timeout``
     If set, blocking operations will timeout after that many seconds.
     (default: None)
 
 ``merge``
 ---------
 
 This section specifies behavior during merges and updates.
 
 ``checkignored``
    Controls behavior when an ignored file on disk has the same name as a tracked
    file in the changeset being merged or updated to, and has different
    contents. Options are ``abort``, ``warn`` and ``ignore``. With ``abort``,
    abort on such files. With ``warn``, warn on such files and back them up as
    ``.orig``. With ``ignore``, don't print a warning and back them up as
    ``.orig``. (default: ``abort``)
 
 ``checkunknown``
    Controls behavior when an unknown file that isn't ignored has the same name
    as a tracked file in the changeset being merged or updated to, and has
    different contents. Similar to ``merge.checkignored``, except for files that
    are not ignored. (default: ``abort``)
 
 ``on-failure``
    When set to ``continue`` (the default), the merge process attempts to
    merge all unresolved files using the merge chosen tool, regardless of
    whether previous file merge attempts during the process succeeded or not.
    Setting this to ``prompt`` will prompt after any merge failure continue
    or halt the merge process. Setting this to ``halt`` will automatically
    halt the merge process on any merge tool failure. The merge process
    can be restarted by using the ``resolve`` command. When a merge is
    halted, the repository is left in a normal ``unresolved`` merge state.
    (default: ``continue``)
 
 ``strict-capability-check``
    Whether capabilities of internal merge tools are checked strictly
    or not, while examining rules to decide merge tool to be used.
    (default: False)
 
 ``merge-patterns``
 ------------------
 
 This section specifies merge tools to associate with particular file
 patterns. Tools matched here will take precedence over the default
 merge tool. Patterns are globs by default, rooted at the repository
 root.
 
 Example::
 
   [merge-patterns]
   **.c = kdiff3
   **.jpg = myimgmerge
 
 ``merge-tools``
 ---------------
 
 This section configures external merge tools to use for file-level
 merges. This section has likely been preconfigured at install time.
 Use :hg:`config merge-tools` to check the existing configuration.
 Also see :hg:`help merge-tools` for more details.
 
 Example ``~/.hgrc``::
 
   [merge-tools]
   # Override stock tool location
   kdiff3.executable = ~/bin/kdiff3
   # Specify command line
   kdiff3.args = $base $local $other -o $output
   # Give higher priority
   kdiff3.priority = 1
 
   # Changing the priority of preconfigured tool
   meld.priority = 0
 
   # Disable a preconfigured tool
   vimdiff.disabled = yes
 
   # Define new tool
   myHtmlTool.args = -m $local $other $base $output
   myHtmlTool.regkey = Software\FooSoftware\HtmlMerge
   myHtmlTool.priority = 1
 
 Supported arguments:
 
 ``priority``
   The priority in which to evaluate this tool.
   (default: 0)
 
 ``executable``
   Either just the name of the executable or its pathname.
 
   .. container:: windows
 
     On Windows, the path can use environment variables with ${ProgramFiles}
     syntax.
 
   (default: the tool name)
 
 ``args``
   The arguments to pass to the tool executable. You can refer to the
   files being merged as well as the output file through these
   variables: ``$base``, ``$local``, ``$other``, ``$output``.
 
   The meaning of ``$local`` and ``$other`` can vary depending on which action is
   being performed. During an update or merge, ``$local`` represents the original
   state of the file, while ``$other`` represents the commit you are updating to or
   the commit you are merging with. During a rebase, ``$local`` represents the
   destination of the rebase, and ``$other`` represents the commit being rebased.
 
   Some operations define custom labels to assist with identifying the revisions,
   accessible via ``$labellocal``, ``$labelother``, and ``$labelbase``. If custom
   labels are not available, these will be ``local``, ``other``, and ``base``,
   respectively.
   (default: ``$local $base $other``)
 
 ``premerge``
   Attempt to run internal non-interactive 3-way merge tool before
   launching external tool.  Options are ``true``, ``false``, ``keep``,
   ``keep-merge3``, or ``keep-mergediff`` (experimental). The ``keep`` option
   will leave markers in the file if the premerge fails. The ``keep-merge3``
   will do the same but include information about the base of the merge in the
   marker (see internal :merge3 in :hg:`help merge-tools`). The
   ``keep-mergediff`` option is similar but uses a different marker style
   (see internal :merge3 in :hg:`help merge-tools`). (default: True)
 
 ``binary``
   This tool can merge binary files. (default: False, unless tool
   was selected by file pattern match)
 
 ``symlink``
   This tool can merge symlinks. (default: False)
 
 ``check``
   A list of merge success-checking options:
 
   ``changed``
     Ask whether merge was successful when the merged file shows no changes.
   ``conflicts``
     Check whether there are conflicts even though the tool reported success.
   ``prompt``
     Always prompt for merge success, regardless of success reported by tool.
 
 ``fixeol``
   Attempt to fix up EOL changes caused by the merge tool.
   (default: False)
 
 ``gui``
   This tool requires a graphical interface to run. (default: False)
 
 ``mergemarkers``
   Controls whether the labels passed via ``$labellocal``, ``$labelother``, and
   ``$labelbase`` are ``detailed`` (respecting ``mergemarkertemplate``) or
   ``basic``. If ``premerge`` is ``keep`` or ``keep-merge3``, the conflict
   markers generated during premerge will be ``detailed`` if either this option or
   the corresponding option in the ``[ui]`` section is ``detailed``.
   (default: ``basic``)
 
 ``mergemarkertemplate``
   This setting can be used to override ``mergemarker`` from the
   ``[command-templates]`` section on a per-tool basis; this applies to the
   ``$label``-prefixed variables and to the conflict markers that are generated
   if ``premerge`` is ``keep` or ``keep-merge3``. See the corresponding variable
   in ``[ui]`` for more information.
 
 .. container:: windows
 
   ``regkey``
     Windows registry key which describes install location of this
     tool. Mercurial will search for this key first under
     ``HKEY_CURRENT_USER`` and then under ``HKEY_LOCAL_MACHINE``.
     (default: None)
 
   ``regkeyalt``
     An alternate Windows registry key to try if the first key is not
     found.  The alternate key uses the same ``regname`` and ``regappend``
     semantics of the primary key.  The most common use for this key
     is to search for 32bit applications on 64bit operating systems.
     (default: None)
 
   ``regname``
     Name of value to read from specified registry key.
     (default: the unnamed (default) value)
 
   ``regappend``
     String to append to the value read from the registry, typically
     the executable name of the tool.
     (default: None)
 
 ``pager``
 ---------
 
 Setting used to control when to paginate and with what external tool. See
 :hg:`help pager` for details.
 
 ``pager``
     Define the external tool used as pager.
 
     If no pager is set, Mercurial uses the environment variable $PAGER.
     If neither pager.pager, nor $PAGER is set, a default pager will be
     used, typically `less` on Unix and `more` on Windows. Example::
 
       [pager]
       pager = less -FRX
 
 ``ignore``
     List of commands to disable the pager for. Example::
 
       [pager]
       ignore = version, help, update
 
 ``patch``
 ---------
 
 Settings used when applying patches, for instance through the 'import'
 command or with Mercurial Queues extension.
 
 ``eol``
     When set to 'strict' patch content and patched files end of lines
     are preserved. When set to ``lf`` or ``crlf``, both files end of
     lines are ignored when patching and the result line endings are
     normalized to either LF (Unix) or CRLF (Windows). When set to
     ``auto``, end of lines are again ignored while patching but line
     endings in patched files are normalized to their original setting
     on a per-file basis. If target file does not exist or has no end
     of line, patch line endings are preserved.
     (default: strict)
 
 ``fuzz``
     The number of lines of 'fuzz' to allow when applying patches. This
     controls how much context the patcher is allowed to ignore when
     trying to apply a patch.
     (default: 2)
 
 ``paths``
 ---------
 
 Assigns symbolic names and behavior to repositories.
 
 Options are symbolic names defining the URL or directory that is the
 location of the repository. Example::
 
     [paths]
     my_server = https://example.com/my_repo
     local_path = /home/me/repo
 
 These symbolic names can be used from the command line. To pull
 from ``my_server``: :hg:`pull my_server`. To push to ``local_path``:
 :hg:`push local_path`. You can check :hg:`help urls` for details about
 valid URLs.
 
 Options containing colons (``:``) denote sub-options that can influence
 behavior for that specific path. Example::
 
     [paths]
     my_server = https://example.com/my_path
     my_server:pushurl = ssh://example.com/my_path
 
 Paths using the `path://otherpath` scheme will inherit the sub-options value from
 the path they point to.
 
 The following sub-options can be defined:
 
 ``multi-urls``
    A boolean option. When enabled the value of the `[paths]` entry will be
    parsed as a list and the alias will resolve to multiple destination. If some
    of the list entry use the `path://` syntax, the suboption will be inherited
    individually.
 
 ``pushurl``
    The URL to use for push operations. If not defined, the location
    defined by the path's main entry is used.
 
 ``pushrev``
    A revset defining which revisions to push by default.
 
    When :hg:`push` is executed without a ``-r`` argument, the revset
    defined by this sub-option is evaluated to determine what to push.
 
    For example, a value of ``.`` will push the working directory's
    revision by default.
 
    Revsets specifying bookmarks will not result in the bookmark being
    pushed.
 
 ``bookmarks.mode``
   How bookmark will be dealt during the exchange. It support the following value
 
   - ``default``: the default behavior, local and remote bookmarks are "merged"
     on push/pull.
 
   - ``mirror``: when pulling, replace local bookmarks by remote bookmarks. This
     is useful to replicate a repository, or as an optimization.
 
   - ``ignore``: ignore bookmarks during exchange.
     (This currently only affect pulling)
 
 .. container:: verbose
 
   ``pulled-delta-reuse-policy``
   Control the policy regarding deltas sent by the remote during pulls.
 
   This is an advanced option that non-admin users should not need to understand
   or set. This option can be used to speed up pulls from trusted central
   servers, or to fix-up deltas from older servers.
 
   It supports the following values:
 
   - ``default``: use the policy defined by
     `storage.revlog.reuse-external-delta-parent`,
 
   - ``no-reuse``: start a new optimal delta search for each new revision we add
     to the repository. The deltas from the server will be reused when the base
     it applies to is tested (this can be frequent if that base is the one and
     unique parent of that revision). This can significantly slowdown pulls but
     will result in an optimized storage space if the remote peer is sending poor
     quality deltas.
 
   - ``try-base``: try to reuse the deltas from the remote peer as long as they
     create a valid delta-chain in the local repository. This speeds up the
     unbundling process, but can result in sub-optimal storage space if the
     remote peer is sending poor quality deltas.
 
   - ``forced``: the deltas from the peer will be reused in all cases, even if
     the resulting delta-chain is "invalid". This setting will ensure the bundle
     is applied at minimal CPU cost, but it can result in longer delta chains
     being created on the client, making revisions potentially slower to access
     in the future. If you think you need this option, you should make sure you
     are also talking to the Mercurial developer community to get confirmation.
 
   See `hg help config.storage.revlog.reuse-external-delta-parent` for a similar
   global option. That option defines the behavior of `default`.
 
 The following special named paths exist:
 
 ``default``
    The URL or directory to use when no source or remote is specified.
 
    :hg:`clone` will automatically define this path to the location the
    repository was cloned from.
 
 ``default-push``
    (deprecated) The URL or directory for the default :hg:`push` location.
    ``default:pushurl`` should be used instead.
 
 ``phases``
 ----------
 
 Specifies default handling of phases. See :hg:`help phases` for more
 information about working with phases.
 
 ``publish``
     Controls draft phase behavior when working as a server. When true,
     pushed changesets are set to public in both client and server and
     pulled or cloned changesets are set to public in the client.
     (default: True)
 
 ``new-commit``
     Phase of newly-created commits.
     (default: draft)
 
 ``checksubrepos``
     Check the phase of the current revision of each subrepository. Allowed
     values are "ignore", "follow" and "abort". For settings other than
     "ignore", the phase of the current revision of each subrepository is
     checked before committing the parent repository. If any of those phases is
     greater than the phase of the parent repository (e.g. if a subrepo is in a
     "secret" phase while the parent repo is in "draft" phase), the commit is
     either aborted (if checksubrepos is set to "abort") or the higher phase is
     used for the parent repository commit (if set to "follow").
     (default: follow)
 
 
 ``profiling``
 -------------
 
 Specifies profiling type, format, and file output. Two profilers are
 supported: an instrumenting profiler (named ``ls``), and a sampling
 profiler (named ``stat``).
 
 In this section description, 'profiling data' stands for the raw data
 collected during profiling, while 'profiling report' stands for a
 statistical text report generated from the profiling data.
 
 ``enabled``
     Enable the profiler.
     (default: false)
 
     This is equivalent to passing ``--profile`` on the command line.
 
 ``type``
     The type of profiler to use.
     (default: stat)
 
     ``ls``
       Use Python's built-in instrumenting profiler. This profiler
       works on all platforms, but each line number it reports is the
       first line of a function. This restriction makes it difficult to
       identify the expensive parts of a non-trivial function.
     ``stat``
       Use a statistical profiler, statprof. This profiler is most
       useful for profiling commands that run for longer than about 0.1
       seconds.
 
 ``format``
     Profiling format.  Specific to the ``ls`` instrumenting profiler.
     (default: text)
 
     ``text``
       Generate a profiling report. When saving to a file, it should be
       noted that only the report is saved, and the profiling data is
       not kept.
     ``kcachegrind``
       Format profiling data for kcachegrind use: when saving to a
       file, the generated file can directly be loaded into
       kcachegrind.
 
 ``statformat``
     Profiling format for the ``stat`` profiler.
     (default: hotpath)
 
     ``hotpath``
       Show a tree-based display containing the hot path of execution (where
       most time was spent).
     ``bymethod``
       Show a table of methods ordered by how frequently they are active.
     ``byline``
       Show a table of lines in files ordered by how frequently they are active.
     ``json``
       Render profiling data as JSON.
 
 ``freq``
     Sampling frequency.  Specific to the ``stat`` sampling profiler.
     (default: 1000)
 
 ``output``
     File path where profiling data or report should be saved. If the
     file exists, it is replaced. (default: None, data is printed on
     stderr)
 
 ``sort``
     Sort field.  Specific to the ``ls`` instrumenting profiler.
     One of ``callcount``, ``reccallcount``, ``totaltime`` and
     ``inlinetime``.
     (default: inlinetime)
 
 ``time-track``
     Control if the stat profiler track ``cpu`` or ``real`` time.
     (default: ``cpu`` on Windows, otherwise ``real``)
 
 ``limit``
     Number of lines to show. Specific to the ``ls`` instrumenting profiler.
     (default: 30)
 
 ``nested``
     Show at most this number of lines of drill-down info after each main entry.
     This can help explain the difference between Total and Inline.
     Specific to the ``ls`` instrumenting profiler.
     (default: 0)
 
 ``showmin``
     Minimum fraction of samples an entry must have for it to be displayed.
     Can be specified as a float between ``0.0`` and ``1.0`` or can have a
     ``%`` afterwards to allow values up to ``100``. e.g. ``5%``.
 
     Only used by the ``stat`` profiler.
 
     For the ``hotpath`` format, default is ``0.05``.
     For the ``chrome`` format, default is ``0.005``.
 
     The option is unused on other formats.
 
 ``showmax``
     Maximum fraction of samples an entry can have before it is ignored in
     display. Values format is the same as ``showmin``.
 
     Only used by the ``stat`` profiler.
 
     For the ``chrome`` format, default is ``0.999``.
 
     The option is unused on other formats.
 
 ``showtime``
     Show time taken as absolute durations, in addition to percentages.
     Only used by the ``hotpath`` format.
     (default: true)
 
 ``progress``
 ------------
 
 Mercurial commands can draw progress bars that are as informative as
 possible. Some progress bars only offer indeterminate information, while others
 have a definite end point.
 
 ``debug``
     Whether to print debug info when updating the progress bar. (default: False)
 
 ``delay``
     Number of seconds (float) before showing the progress bar. (default: 3)
 
 ``changedelay``
     Minimum delay before showing a new topic. When set to less than 3 * refresh,
     that value will be used instead. (default: 1)
 
 ``estimateinterval``
     Maximum sampling interval in seconds for speed and estimated time
     calculation. (default: 60)
 
 ``refresh``
     Time in seconds between refreshes of the progress bar. (default: 0.1)
 
 ``format``
     Format of the progress bar.
 
     Valid entries for the format field are ``topic``, ``bar``, ``number``,
     ``unit``, ``estimate``, ``speed``, and ``item``. ``item`` defaults to the
     last 20 characters of the item, but this can be changed by adding either
     ``-<num>`` which would take the last num characters, or ``+<num>`` for the
     first num characters.
 
     (default: topic bar number estimate)
 
 ``width``
     If set, the maximum width of the progress information (that is, min(width,
     term width) will be used).
 
 ``clear-complete``
     Clear the progress bar after it's done. (default: True)
 
 ``disable``
     If true, don't show a progress bar.
 
 ``assume-tty``
     If true, ALWAYS show a progress bar, unless disable is given.
 
 ``rebase``
 ----------
 
 ``evolution.allowdivergence``
     Default to False, when True allow creating divergence when performing
     rebase of obsolete changesets.
 
 ``revsetalias``
 ---------------
 
 Alias definitions for revsets. See :hg:`help revsets` for details.
 
 ``rewrite``
 -----------
 
 ``backup-bundle``
     Whether to save stripped changesets to a bundle file. (default: True)
 
 ``update-timestamp``
     If true, updates the date and time of the changeset to current. It is only
     applicable for `hg amend`, `hg commit --amend` and `hg uncommit` in the
     current version.
 
 ``empty-successor``
 
     Control what happens with empty successors that are the result of rewrite
     operations. If set to ``skip``, the successor is not created. If set to
     ``keep``, the empty successor is created and kept.
 
     Currently, only the rebase and absorb commands consider this configuration.
     (EXPERIMENTAL)
 
 ``rhg``
 -------
 
 The pure Rust fast-path for Mercurial. See `rust/README.rst` in the Mercurial repository.
 
 ``fallback-executable``
     Path to the executable to run in a sub-process when falling back to
     another implementation of Mercurial.
 
 ``fallback-immediately``
     Fall back to ``fallback-executable`` as soon as possible, regardless of
     the `rhg.on-unsupported` configuration. Useful for debugging, for example to
     bypass `rhg` if the deault `hg` points to `rhg`.
 
     Note that because this requires loading the configuration, it is possible
     that `rhg` error out before being able to fall back.
 
 ``ignored-extensions``
     Controls which extensions should be ignored by `rhg`. By default, `rhg`
     triggers the `rhg.on-unsupported` behavior any unsupported extensions.
     Users can disable that behavior when they know that a given extension
     does not need support from `rhg`.
 
     Expects a list of extension names, or ``*`` to ignore all extensions.
 
     Note: ``*:<suboption>`` is also a valid extension name for this
     configuration option.
     As of this writing, the only valid "global" suboption is ``required``.
 
 ``on-unsupported``
     Controls the behavior of `rhg` when detecting unsupported features.
 
     Possible values are `abort` (default), `abort-silent` and `fallback`.
 
     ``abort``
         Print an error message describing what feature is not supported,
         and exit with code 252
 
     ``abort-silent``
         Silently exit with code 252
 
     ``fallback``
         Try running the fallback executable with the same parameters
         (and trace the fallback reason, use `RUST_LOG=trace` to see).
 
 ``share``
 ---------
 
 ``safe-mismatch.source-safe``
     Controls what happens when the shared repository does not use the
     share-safe mechanism but its source repository does.
 
     Possible values are `abort` (default), `allow`, `upgrade-abort` and
     `upgrade-allow`.
 
     ``abort``
         Disallows running any command and aborts
     ``allow``
         Respects the feature presence in the share source
     ``upgrade-abort``
         Tries to upgrade the share to use share-safe; if it fails, aborts
     ``upgrade-allow``
         Tries to upgrade the share; if it fails, continue by
         respecting the share source setting
 
     Check :hg:`help config.format.use-share-safe` for details about the
     share-safe feature.
 
 ``safe-mismatch.source-safe:verbose-upgrade``
     Display a message when upgrading, (default: True)
 
 ``safe-mismatch.source-safe.warn``
     Shows a warning on operations if the shared repository does not use
     share-safe, but the source repository does.
     (default: True)
 
 ``safe-mismatch.source-not-safe``
     Controls what happens when the shared repository uses the share-safe
     mechanism but its source does not.
 
     Possible values are `abort` (default), `allow`, `downgrade-abort` and
     `downgrade-allow`.
 
     ``abort``
         Disallows running any command and aborts
     ``allow``
         Respects the feature presence in the share source
     ``downgrade-abort``
         Tries to downgrade the share to not use share-safe; if it fails, aborts
     ``downgrade-allow``
         Tries to downgrade the share to not use share-safe;
         if it fails, continue by respecting the shared source setting
 
     Check :hg:`help config.format.use-share-safe` for details about the
     share-safe feature.
 
 ``safe-mismatch.source-not-safe:verbose-upgrade``
     Display a message when upgrading, (default: True)
 
 ``safe-mismatch.source-not-safe.warn``
     Shows a warning on operations if the shared repository uses share-safe,
     but the source repository does not.
     (default: True)
 
 ``storage``
 -----------
 
 Control the strategy Mercurial uses internally to store history. Options in this
 category impact performance and repository size.
 
 ``revlog.issue6528.fix-incoming``
     Version 5.8 of Mercurial had a bug leading to altering the parent of file
     revision with copy information (or any other metadata) on exchange. This
     leads to the copy metadata to be overlooked by various internal logic. The
     issue was fixed in Mercurial 5.8.1.
     (See https://bz.mercurial-scm.org/show_bug.cgi?id=6528 for details)
 
     As a result Mercurial is now checking and fixing incoming file revisions to
     make sure there parents are in the right order. This behavior can be
     disabled by setting this option to `no`. This apply to revisions added
     through push, pull, clone and unbundle.
 
     To fix affected revisions that already exist within the repository, one can
     use :hg:`debug-repair-issue-6528`.
 
 .. container:: verbose
 
     ``revlog.delta-parent-search.candidate-group-chunk-size``
         Tune the number of delta bases the storage will consider in the
         same "round" of search. In some very rare cases, using a smaller value
         might result in faster processing at the possible expense of storage
         space, while using larger values might result in slower processing at the
         possible benefit of storage space. A value of "0" means no limitation.
 
         default: no limitation
 
         This is unlikely that you'll have to tune this configuration. If you think
         you do, consider talking with the mercurial developer community about your
         repositories.
 
 ``revlog.optimize-delta-parent-choice``
     When storing a merge revision, both parents will be equally considered as
     a possible delta base. This results in better delta selection and improved
     revlog compression. This option is enabled by default.
 
     Turning this option off can result in large increase of repository size for
     repository with many merges.
 
 ``revlog.persistent-nodemap.mmap``
     Whether to use the Operating System "memory mapping" feature (when
     possible) to access the persistent nodemap data. This improve performance
     and reduce memory pressure.
 
     Default to True.
 
     For details on the "persistent-nodemap" feature, see:
     :hg:`help config.format.use-persistent-nodemap`.
 
 ``revlog.persistent-nodemap.slow-path``
     Control the behavior of Merucrial when using a repository with "persistent"
     nodemap with an installation of Mercurial without a fast implementation for
     the feature:
 
     ``allow``: Silently use the slower implementation to access the repository.
     ``warn``: Warn, but use the slower implementation to access the repository.
     ``abort``: Prevent access to such repositories. (This is the default)
 
     For details on the "persistent-nodemap" feature, see:
     :hg:`help config.format.use-persistent-nodemap`.
 
 ``revlog.reuse-external-delta-parent``
     Control the order in which delta parents are considered when adding new
     revisions from an external source.
     (typically: apply bundle from `hg pull` or `hg push`).
 
     New revisions are usually provided as a delta against other revisions. By
     default, Mercurial will try to reuse this delta first, therefore using the
     same "delta parent" as the source. Directly using delta's from the source
     reduces CPU usage and usually speeds up operation. However, in some case,
     the source might have sub-optimal delta bases and forcing their reevaluation
     is useful. For example, pushes from an old client could have sub-optimal
     delta's parent that the server want to optimize. (lack of general delta, bad
     parents, choice, lack of sparse-revlog, etc).
 
     This option is enabled by default. Turning it off will ensure bad delta
     parent choices from older client do not propagate to this repository, at
     the cost of a small increase in CPU consumption.
 
     Note: this option only control the order in which delta parents are
     considered.  Even when disabled, the existing delta from the source will be
     reused if the same delta parent is selected.
 
 ``revlog.reuse-external-delta``
     Control the reuse of delta from external source.
     (typically: apply bundle from `hg pull` or `hg push`).
 
     New revisions are usually provided as a delta against another revision. By
     default, Mercurial will not recompute the same delta again, trusting
     externally provided deltas. There have been rare cases of small adjustment
     to the diffing algorithm in the past. So in some rare case, recomputing
     delta provided by ancient clients can provides better results. Disabling
     this option means going through a full delta recomputation for all incoming
     revisions. It means a large increase in CPU usage and will slow operations
     down.
 
     This option is enabled by default. When disabled, it also disables the
     related ``storage.revlog.reuse-external-delta-parent`` option.
 
 ``revlog.zlib.level``
     Zlib compression level used when storing data into the repository. Accepted
     Value range from 1 (lowest compression) to 9 (highest compression). Zlib
     default value is 6.
 
 
 ``revlog.zstd.level``
     zstd compression level used when storing data into the repository. Accepted
     Value range from 1 (lowest compression) to 22 (highest compression).
     (default 3)
 
 ``server``
 ----------
 
 Controls generic server settings.
 
 ``bookmarks-pushkey-compat``
     Trigger pushkey hook when being pushed bookmark updates. This config exist
     for compatibility purpose (default to True)
 
     If you use ``pushkey`` and ``pre-pushkey`` hooks to control bookmark
     movement we recommend you migrate them to ``txnclose-bookmark`` and
     ``pretxnclose-bookmark``.
 
 ``compressionengines``
     List of compression engines and their relative priority to advertise
     to clients.
 
     The order of compression engines determines their priority, the first
     having the highest priority. If a compression engine is not listed
     here, it won't be advertised to clients.
 
     If not set (the default), built-in defaults are used. Run
     :hg:`debuginstall` to list available compression engines and their
     default wire protocol priority.
 
     Older Mercurial clients only support zlib compression and this setting
     has no effect for legacy clients.
 
 ``uncompressed``
     Whether to allow clients to clone a repository using the
     uncompressed streaming protocol. This transfers about 40% more
     data than a regular clone, but uses less memory and CPU on both
     server and client. Over a LAN (100 Mbps or better) or a very fast
     WAN, an uncompressed streaming clone is a lot faster (~10x) than a
     regular clone. Over most WAN connections (anything slower than
     about 6 Mbps), uncompressed streaming is slower, because of the
     extra data transfer overhead. This mode will also temporarily hold
     the write lock while determining what data to transfer.
     (default: True)
 
 ``uncompressedallowsecret``
     Whether to allow stream clones when the repository contains secret
     changesets. (default: False)
 
 ``preferuncompressed``
     When set, clients will try to use the uncompressed streaming
     protocol. (default: False)
 
 ``disablefullbundle``
     When set, servers will refuse attempts to do pull-based clones.
     If this option is set, ``preferuncompressed`` and/or clone bundles
     are highly recommended. Partial clones will still be allowed.
     (default: False)
 
 ``streamunbundle``
     When set, servers will apply data sent from the client directly,
     otherwise it will be written to a temporary file first. This option
     effectively prevents concurrent pushes.
 
 ``pullbundle``
     When set, the server will check pullbundles.manifest for bundles
     covering the requested heads and common nodes. The first matching
     entry will be streamed to the client.
 
     For HTTP transport, the stream will still use zlib compression
     for older clients.
 
 ``concurrent-push-mode``
     Level of allowed race condition between two pushing clients.
 
     - 'strict': push is abort if another client touched the repository
       while the push was preparing.
     - 'check-related': push is only aborted if it affects head that got also
       affected while the push was preparing. (default since 5.4)
 
     'check-related' only takes effect for compatible clients (version
     4.3 and later). Older clients will use 'strict'.
 
 ``validate``
     Whether to validate the completeness of pushed changesets by
     checking that all new file revisions specified in manifests are
     present. (default: False)
 
 ``maxhttpheaderlen``
     Instruct HTTP clients not to send request headers longer than this
     many bytes. (default: 1024)
 
 ``bundle1``
     Whether to allow clients to push and pull using the legacy bundle1
     exchange format. (default: True)
 
 ``bundle1gd``
     Like ``bundle1`` but only used if the repository is using the
     *generaldelta* storage format. (default: True)
 
 ``bundle1.push``
     Whether to allow clients to push using the legacy bundle1 exchange
     format. (default: True)
 
 ``bundle1gd.push``
     Like ``bundle1.push`` but only used if the repository is using the
     *generaldelta* storage format. (default: True)
 
 ``bundle1.pull``
     Whether to allow clients to pull using the legacy bundle1 exchange
     format. (default: True)
 
 ``bundle1gd.pull``
     Like ``bundle1.pull`` but only used if the repository is using the
     *generaldelta* storage format. (default: True)
 
     Large repositories using the *generaldelta* storage format should
     consider setting this option because converting *generaldelta*
     repositories to the exchange format required by the bundle1 data
     format can consume a lot of CPU.
 
 ``bundle2.stream``
     Whether to allow clients to pull using the bundle2 streaming protocol.
     (default: True)
 
 ``zliblevel``
     Integer between ``-1`` and ``9`` that controls the zlib compression level
     for wire protocol commands that send zlib compressed output (notably the
     commands that send repository history data).
 
     The default (``-1``) uses the default zlib compression level, which is
     likely equivalent to ``6``. ``0`` means no compression. ``9`` means
     maximum compression.
 
     Setting this option allows server operators to make trade-offs between
     bandwidth and CPU used. Lowering the compression lowers CPU utilization
     but sends more bytes to clients.
 
     This option only impacts the HTTP server.
 
 ``zstdlevel``
     Integer between ``1`` and ``22`` that controls the zstd compression level
     for wire protocol commands. ``1`` is the minimal amount of compression and
     ``22`` is the highest amount of compression.
 
     The default (``3``) should be significantly faster than zlib while likely
     delivering better compression ratios.
 
     This option only impacts the HTTP server.
 
     See also ``server.zliblevel``.
 
 ``view``
     Repository filter used when exchanging revisions with the peer.
 
     The default view (``served``) excludes secret and hidden changesets.
     Another useful value is ``immutable`` (no draft, secret or hidden
     changesets). (EXPERIMENTAL)
 
 ``smtp``
 --------
 
 Configuration for extensions that need to send email messages.
 
 ``host``
     Host name of mail server, e.g. "mail.example.com".
 
 ``port``
     Optional. Port to connect to on mail server. (default: 465 if
     ``tls`` is smtps; 25 otherwise)
 
 ``tls``
     Optional. Method to enable TLS when connecting to mail server: starttls,
     smtps or none. (default: none)
 
 ``username``
     Optional. User name for authenticating with the SMTP server.
     (default: None)
 
 ``password``
     Optional. Password for authenticating with the SMTP server. If not
     specified, interactive sessions will prompt the user for a
     password; non-interactive sessions will fail. (default: None)
 
 ``local_hostname``
     Optional. The hostname that the sender can use to identify
     itself to the MTA.
 
 
 ``subpaths``
 ------------
 
 Subrepository source URLs can go stale if a remote server changes name
 or becomes temporarily unavailable. This section lets you define
 rewrite rules of the form::
 
     <pattern> = <replacement>
 
 where ``pattern`` is a regular expression matching a subrepository
 source URL and ``replacement`` is the replacement string used to
 rewrite it. Groups can be matched in ``pattern`` and referenced in
 ``replacements``. For instance::
 
     http://server/(.*)-hg/ = http://hg.server/\1/
 
 rewrites ``http://server/foo-hg/`` into ``http://hg.server/foo/``.
 
 Relative subrepository paths are first made absolute, and the
 rewrite rules are then applied on the full (absolute) path. If ``pattern``
 doesn't match the full path, an attempt is made to apply it on the
 relative path alone. The rules are applied in definition order.
 
 ``subrepos``
 ------------
 
 This section contains options that control the behavior of the
 subrepositories feature. See also :hg:`help subrepos`.
 
 Security note: auditing in Mercurial is known to be insufficient to
 prevent clone-time code execution with carefully constructed Git
 subrepos. It is unknown if a similar detect is present in Subversion
 subrepos. Both Git and Subversion subrepos are disabled by default
 out of security concerns. These subrepo types can be enabled using
 the respective options below.
 
 ``allowed``
     Whether subrepositories are allowed in the working directory.
 
     When false, commands involving subrepositories (like :hg:`update`)
     will fail for all subrepository types.
     (default: true)
 
 ``hg:allowed``
     Whether Mercurial subrepositories are allowed in the working
     directory. This option only has an effect if ``subrepos.allowed``
     is true.
     (default: true)
 
 ``git:allowed``
     Whether Git subrepositories are allowed in the working directory.
     This option only has an effect if ``subrepos.allowed`` is true.
 
     See the security note above before enabling Git subrepos.
     (default: false)
 
 ``svn:allowed``
     Whether Subversion subrepositories are allowed in the working
     directory. This option only has an effect if ``subrepos.allowed``
     is true.
 
     See the security note above before enabling Subversion subrepos.
     (default: false)
 
 ``templatealias``
 -----------------
 
 Alias definitions for templates. See :hg:`help templates` for details.
 
 ``templates``
 -------------
 
 Use the ``[templates]`` section to define template strings.
 See :hg:`help templates` for details.
 
 ``trusted``
 -----------
 
 Mercurial will not use the settings in the
 ``.hg/hgrc`` file from a repository if it doesn't belong to a trusted
 user or to a trusted group, as various hgrc features allow arbitrary
 commands to be run. This issue is often encountered when configuring
 hooks or extensions for shared repositories or servers. However,
 the web interface will use some safe settings from the ``[web]``
 section.
 
 This section specifies what users and groups are trusted. The
 current user is always trusted. To trust everybody, list a user or a
 group with name ``*``. These settings must be placed in an
 *already-trusted file* to take effect, such as ``$HOME/.hgrc`` of the
 user or service running Mercurial.
 
 ``users``
   Comma-separated list of trusted users.
 
 ``groups``
   Comma-separated list of trusted groups.
 
 
 ``ui``
 ------
 
 User interface controls.
 
 ``archivemeta``
     Whether to include the .hg_archival.txt file containing meta data
     (hashes for the repository base and for tip) in archives created
     by the :hg:`archive` command or downloaded via hgweb.
     (default: True)
 
 ``askusername``
     Whether to prompt for a username when committing. If True, and
     neither ``$HGUSER`` nor ``$EMAIL`` has been specified, then the user will
     be prompted to enter a username. If no username is entered, the
     default ``USER@HOST`` is used instead.
     (default: False)
 
 ``clonebundles``
     Whether the "clone bundles" feature is enabled.
 
     When enabled, :hg:`clone` may download and apply a server-advertised
     bundle file from a URL instead of using the normal exchange mechanism.
 
     This can likely result in faster and more reliable clones.
 
     (default: True)
 
 ``clonebundlefallback``
     Whether failure to apply an advertised "clone bundle" from a server
     should result in fallback to a regular clone.
 
     This is disabled by default because servers advertising "clone
     bundles" often do so to reduce server load. If advertised bundles
     start mass failing and clients automatically fall back to a regular
     clone, this would add significant and unexpected load to the server
     since the server is expecting clone operations to be offloaded to
     pre-generated bundles. Failing fast (the default behavior) ensures
     clients don't overwhelm the server when "clone bundle" application
     fails.
 
     (default: False)
 
 ``clonebundleprefers``
     Defines preferences for which "clone bundles" to use.
 
     Servers advertising "clone bundles" may advertise multiple available
     bundles. Each bundle may have different attributes, such as the bundle
     type and compression format. This option is used to prefer a particular
     bundle over another.
 
     The following keys are defined by Mercurial:
 
     BUNDLESPEC
        A bundle type specifier. These are strings passed to :hg:`bundle -t`.
        e.g. ``gzip-v2`` or ``bzip2-v1``.
 
     COMPRESSION
        The compression format of the bundle. e.g. ``gzip`` and ``bzip2``.
 
     Server operators may define custom keys.
 
     Example values: ``COMPRESSION=bzip2``,
     ``BUNDLESPEC=gzip-v2, COMPRESSION=gzip``.
 
     By default, the first bundle advertised by the server is used.
 
 ``color``
     When to colorize output. Possible value are Boolean ("yes" or "no"), or
     "debug", or "always". (default: "yes"). "yes" will use color whenever it
     seems possible. See :hg:`help color` for details.
 
 ``commitsubrepos``
     Whether to commit modified subrepositories when committing the
     parent repository. If False and one subrepository has uncommitted
     changes, abort the commit.
     (default: False)
 
 ``debug``
     Print debugging information. (default: False)
 
 ``editor``
     The editor to use during a commit. (default: ``$EDITOR`` or ``vi``)
 
 ``fallbackencoding``
     Encoding to try if it's not possible to decode the changelog using
     UTF-8. (default: ISO-8859-1)
 
 ``graphnodetemplate``
     (DEPRECATED) Use ``command-templates.graphnode`` instead.
 
 ``ignore``
     A file to read per-user ignore patterns from. This file should be
     in the same format as a repository-wide .hgignore file. Filenames
     are relative to the repository root. This option supports hook syntax,
     so if you want to specify multiple ignore files, you can do so by
     setting something like ``ignore.other = ~/.hgignore2``. For details
     of the ignore file format, see the ``hgignore(5)`` man page.
 
 ``interactive``
     Allow to prompt the user. (default: True)
 
 ``interface``
     Select the default interface for interactive features (default: text).
     Possible values are 'text' and 'curses'.
 
 ``interface.chunkselector``
     Select the interface for change recording (e.g. :hg:`commit -i`).
     Possible values are 'text' and 'curses'.
     This config overrides the interface specified by ui.interface.
 
 ``large-file-limit``
     Largest file size that gives no memory use warning.
     Possible values are integers or 0 to disable the check.
     Value is expressed in bytes by default, one can use standard units for
     convenience (e.g. 10MB, 0.1GB, etc) (default: 10MB)
 
 ``logtemplate``
     (DEPRECATED) Use ``command-templates.log`` instead.
 
 ``merge``
     The conflict resolution program to use during a manual merge.
     For more information on merge tools see :hg:`help merge-tools`.
     For configuring merge tools see the ``[merge-tools]`` section.
 
 ``mergemarkers``
     Sets the merge conflict marker label styling. The ``detailed`` style
     uses the ``command-templates.mergemarker`` setting to style the labels.
     The ``basic`` style just uses 'local' and 'other' as the marker label.
     One of ``basic`` or ``detailed``.
     (default: ``basic``)
 
 ``mergemarkertemplate``
     (DEPRECATED) Use ``command-templates.mergemarker`` instead.
 
 ``message-output``
     Where to write status and error messages. (default: ``stdio``)
 
     ``channel``
       Use separate channel for structured output. (Command-server only)
     ``stderr``
       Everything to stderr.
     ``stdio``
       Status to stdout, and error to stderr.
 
 ``origbackuppath``
     The path to a directory used to store generated .orig files. If the path is
     not a directory, one will be created.  If set, files stored in this
     directory have the same name as the original file and do not have a .orig
     suffix.
 
 ``paginate``
   Control the pagination of command output (default: True). See :hg:`help pager`
   for details.
 
 ``patch``
     An optional external tool that ``hg import`` and some extensions
     will use for applying patches. By default Mercurial uses an
     internal patch utility. The external tool must work as the common
     Unix ``patch`` program. In particular, it must accept a ``-p``
     argument to strip patch headers, a ``-d`` argument to specify the
     current directory, a file name to patch, and a patch file to take
     from stdin.
 
     It is possible to specify a patch tool together with extra
     arguments. For example, setting this option to ``patch --merge``
     will use the ``patch`` program with its 2-way merge option.
 
 ``portablefilenames``
     Check for portable filenames. Can be ``warn``, ``ignore`` or ``abort``.
     (default: ``warn``)
 
     ``warn``
       Print a warning message on POSIX platforms, if a file with a non-portable
       filename is added (e.g. a file with a name that can't be created on
       Windows because it contains reserved parts like ``AUX``, reserved
       characters like ``:``, or would cause a case collision with an existing
       file).
 
     ``ignore``
       Don't print a warning.
 
     ``abort``
       The command is aborted.
 
     ``true``
       Alias for ``warn``.
 
     ``false``
       Alias for ``ignore``.
 
     .. container:: windows
 
       On Windows, this configuration option is ignored and the command aborted.
 
 ``pre-merge-tool-output-template``
     (DEPRECATED) Use ``command-template.pre-merge-tool-output`` instead.
 
 ``quiet``
     Reduce the amount of output printed.
     (default: False)
 
 ``relative-paths``
     Prefer relative paths in the UI.
 
 ``remotecmd``
     Remote command to use for clone/push/pull operations.
     (default: ``hg``)
 
 ``report_untrusted``
     Warn if a ``.hg/hgrc`` file is ignored due to not being owned by a
     trusted user or group.
     (default: True)
 
 ``slash``
     (Deprecated. Use ``slashpath`` template filter instead.)
 
     Display paths using a slash (``/``) as the path separator. This
     only makes a difference on systems where the default path
     separator is not the slash character (e.g. Windows uses the
     backslash character (``\``)).
     (default: False)
 
 ``statuscopies``
     Display copies in the status command.
 
 ``ssh``
     Command to use for SSH connections. (default: ``ssh``)
 
 ``ssherrorhint``
     A hint shown to the user in the case of SSH error (e.g.
     ``Please see http://company/internalwiki/ssh.html``)
 
 ``strict``
     Require exact command names, instead of allowing unambiguous
     abbreviations. (default: False)
 
 ``style``
     Name of style to use for command output.
 
 ``supportcontact``
     A URL where users should report a Mercurial traceback. Use this if you are a
     large organisation with its own Mercurial deployment process and crash
     reports should be addressed to your internal support.
 
 ``textwidth``
     Maximum width of help text. A longer line generated by ``hg help`` or
     ``hg subcommand --help`` will be broken after white space to get this
     width or the terminal width, whichever comes first.
     A non-positive value will disable this and the terminal width will be
     used. (default: 78)
 
 ``timeout``
     The timeout used when a lock is held (in seconds), a negative value
     means no timeout. (default: 600)
 
 ``timeout.warn``
     Time (in seconds) before a warning is printed about held lock. A negative
     value means no warning. (default: 0)
 
 ``traceback``
     Mercurial always prints a traceback when an unknown exception
     occurs. Setting this to True will make Mercurial print a traceback
     on all exceptions, even those recognized by Mercurial (such as
     IOError or MemoryError). (default: False)
 
 ``tweakdefaults``
 
     By default Mercurial's behavior changes very little from release
     to release, but over time the recommended config settings
     shift. Enable this config to opt in to get automatic tweaks to
     Mercurial's behavior over time. This config setting will have no
     effect if ``HGPLAIN`` is set or ``HGPLAINEXCEPT`` is set and does
     not include ``tweakdefaults``. (default: False)
 
     It currently means::
 
       .. tweakdefaultsmarker
 
 ``username``
     The committer of a changeset created when running "commit".
     Typically a person's name and email address, e.g. ``Fred Widget
     <[e-mail]>``. Environment variables in the
     username are expanded.
 
     (default: ``$EMAIL`` or ``username@hostname``. If the username in
     hgrc is empty, e.g. if the system admin set ``username =`` in the
     system hgrc, it has to be specified manually or in a different
     hgrc file)
 
 ``verbose``
     Increase the amount of output printed. (default: False)
 
 
 ``command-templates``
 ---------------------
 
 Templates used for customizing the output of commands.
 
 ``graphnode``
     The template used to print changeset nodes in an ASCII revision graph.
     (default: ``{graphnode}``)
 
 ``log``
     Template string for commands that print changesets.
 
 ``mergemarker``
     The template used to print the commit description next to each conflict
     marker during merge conflicts. See :hg:`help templates` for the template
     format.
 
     Defaults to showing the hash, tags, branches, bookmarks, author, and
     the first line of the commit description.
 
     If you use non-ASCII characters in names for tags, branches, bookmarks,
     authors, and/or commit descriptions, you must pay attention to encodings of
     managed files. At template expansion, non-ASCII characters use the encoding
     specified by the ``--encoding`` global option, ``HGENCODING`` or other
     environment variables that govern your locale. If the encoding of the merge
     markers is different from the encoding of the merged files,
     serious problems may occur.
 
     Can be overridden per-merge-tool, see the ``[merge-tools]`` section.
 
 ``oneline-summary``
     A template used by `hg rebase` and other commands for showing a one-line
     summary of a commit. If the template configured here is longer than one
     line, then only the first line is used.
 
     The template can be overridden per command by defining a template in
     `oneline-summary.<command>`, where `<command>` can be e.g. "rebase".
 
 ``pre-merge-tool-output``
     A template that is printed before executing an external merge tool. This can
     be used to print out additional context that might be useful to have during
     the conflict resolution, such as the description of the various commits
     involved or bookmarks/tags.
 
     Additional information is available in the ``local`, ``base``, and ``other``
     dicts. For example: ``{local.label}``, ``{base.name}``, or
     ``{other.islink}``.
 
 
 ``web``
 -------
 
 Web interface configuration. The settings in this section apply to
 both the builtin webserver (started by :hg:`serve`) and the script you
 run through a webserver (``hgweb.cgi`` and the derivatives for FastCGI
 and WSGI).
 
 The Mercurial webserver does no authentication (it does not prompt for
 usernames and passwords to validate *who* users are), but it does do
 authorization (it grants or denies access for *authenticated users*
 based on settings in this section). You must either configure your
 webserver to do authentication for you, or disable the authorization
 checks.
 
 For a quick setup in a trusted environment, e.g., a private LAN, where
 you want it to accept pushes from anybody, you can use the following
 command line::
 
     $ hg --config web.allow-push=* --config web.push_ssl=False serve
 
 Note that this will allow anybody to push anything to the server and
 that this should not be used for public servers.
 
 The full set of options is:
 
 ``accesslog``
     Where to output the access log. (default: stdout)
 
 ``address``
     Interface address to bind to. (default: all)
 
 ``allow-archive``
     List of archive format (bz2, gz, zip) allowed for downloading.
     (default: empty)
 
 ``allowbz2``
     (DEPRECATED) Whether to allow .tar.bz2 downloading of repository
     revisions.
     (default: False)
 
 ``allowgz``
     (DEPRECATED) Whether to allow .tar.gz downloading of repository
     revisions.
     (default: False)
 
 ``allow-pull``
     Whether to allow pulling from the repository. (default: True)
 
 ``allow-push``
     Whether to allow pushing to the repository. If empty or not set,
     pushing is not allowed. If the special value ``*``, any remote
     user can push, including unauthenticated users. Otherwise, the
     remote user must have been authenticated, and the authenticated
     user name must be present in this list. The contents of the
     allow-push list are examined after the deny_push list.
 
 ``allow_read``
     If the user has not already been denied repository access due to
     the contents of deny_read, this list determines whether to grant
     repository access to the user. If this list is not empty, and the
     user is unauthenticated or not present in the list, then access is
     denied for the user. If the list is empty or not set, then access
     is permitted to all users by default. Setting allow_read to the
     special value ``*`` is equivalent to it not being set (i.e. access
     is permitted to all users). The contents of the allow_read list are
     examined after the deny_read list.
 
 ``allowzip``
     (DEPRECATED) Whether to allow .zip downloading of repository
     revisions. This feature creates temporary files.
     (default: False)
 
 ``archivesubrepos``
     Whether to recurse into subrepositories when archiving.
     (default: False)
 
 ``baseurl``
     Base URL to use when publishing URLs in other locations, so
     third-party tools like email notification hooks can construct
     URLs. Example: ``http://hgserver/repos/``.
 
 ``cacerts``
     Path to file containing a list of PEM encoded certificate
     authority certificates. Environment variables and ``~user``
     constructs are expanded in the filename. If specified on the
     client, then it will verify the identity of remote HTTPS servers
     with these certificates.
 
     To disable SSL verification temporarily, specify ``--insecure`` from
     command line.
 
     You can use OpenSSL's CA certificate file if your platform has
     one. On most Linux systems this will be
     ``/etc/ssl/certs/ca-certificates.crt``. Otherwise you will have to
     generate this file manually. The form must be as follows::
 
         -----BEGIN CERTIFICATE-----
         ... (certificate in base64 PEM encoding) ...
         -----END CERTIFICATE-----
         -----BEGIN CERTIFICATE-----
         ... (certificate in base64 PEM encoding) ...
         -----END CERTIFICATE-----
 
 ``cache``
     Whether to support caching in hgweb. (default: True)
 
 ``certificate``
     Certificate to use when running :hg:`serve`.
 
 ``collapse``
     With ``descend`` enabled, repositories in subdirectories are shown at
     a single level alongside repositories in the current path. With
     ``collapse`` also enabled, repositories residing at a deeper level than
     the current path are grouped behind navigable directory entries that
     lead to the locations of these repositories. In effect, this setting
     collapses each collection of repositories found within a subdirectory
     into a single entry for that subdirectory. (default: False)
 
 ``comparisoncontext``
     Number of lines of context to show in side-by-side file comparison. If
     negative or the value ``full``, whole files are shown. (default: 5)
 
     This setting can be overridden by a ``context`` request parameter to the
     ``comparison`` command, taking the same values.
 
 ``contact``
     Name or email address of the person in charge of the repository.
     (default: ui.username or ``$EMAIL`` or "unknown" if unset or empty)
 
 ``csp``
     Send a ``Content-Security-Policy`` HTTP header with this value.
 
     The value may contain a special string ``%nonce%``, which will be replaced
     by a randomly-generated one-time use value. If the value contains
     ``%nonce%``, ``web.cache`` will be disabled, as caching undermines the
     one-time property of the nonce. This nonce will also be inserted into
     ``<script>`` elements containing inline JavaScript.
 
     Note: lots of HTML content sent by the server is derived from repository
     data. Please consider the potential for malicious repository data to
     "inject" itself into generated HTML content as part of your security
     threat model.
 
 ``deny_push``
     Whether to deny pushing to the repository. If empty or not set,
     push is not denied. If the special value ``*``, all remote users are
     denied push. Otherwise, unauthenticated users are all denied, and
     any authenticated user name present in this list is also denied. The
     contents of the deny_push list are examined before the allow-push list.
 
 ``deny_read``
     Whether to deny reading/viewing of the repository. If this list is
     not empty, unauthenticated users are all denied, and any
     authenticated user name present in this list is also denied access to
     the repository. If set to the special value ``*``, all remote users
     are denied access (rarely needed ;). If deny_read is empty or not set,
     the determination of repository access depends on the presence and
     content of the allow_read list (see description). If both
     deny_read and allow_read are empty or not set, then access is
     permitted to all users by default. If the repository is being
     served via hgwebdir, denied users will not be able to see it in
     the list of repositories. The contents of the deny_read list have
     priority over (are examined before) the contents of the allow_read
     list.
 
 ``descend``
     hgwebdir indexes will not descend into subdirectories. Only repositories
     directly in the current path will be shown (other repositories are still
     available from the index corresponding to their containing path).
 
 ``description``
     Textual description of the repository's purpose or contents.
     (default: "unknown")
 
 ``encoding``
     Character encoding name. (default: the current locale charset)
     Example: "UTF-8".
 
 ``errorlog``
     Where to output the error log. (default: stderr)
 
 ``guessmime``
     Control MIME types for raw download of file content.
     Set to True to let hgweb guess the content type from the file
     extension. This will serve HTML files as ``text/html`` and might
     allow cross-site scripting attacks when serving untrusted
     repositories. (default: False)
 
 ``hidden``
     Whether to hide the repository in the hgwebdir index.
     (default: False)
 
 ``ipv6``
     Whether to use IPv6. (default: False)
 
 ``labels``
     List of string *labels* associated with the repository.
 
     Labels are exposed as a template keyword and can be used to customize
     output. e.g. the ``index`` template can group or filter repositories
     by labels and the ``summary`` template can display additional content
     if a specific label is present.
 
 ``logoimg``
     File name of the logo image that some templates display on each page.
     The file name is relative to ``staticurl``. That is, the full path to
     the logo image is "staticurl/logoimg".
     If unset, ``hglogo.png`` will be used.
 
 ``logourl``
     Base URL to use for logos. If unset, ``https://mercurial-scm.org/``
     will be used.
 
 ``maxchanges``
     Maximum number of changes to list on the changelog. (default: 10)
 
 ``maxfiles``
     Maximum number of files to list per changeset. (default: 10)
 
 ``maxshortchanges``
     Maximum number of changes to list on the shortlog, graph or filelog
     pages. (default: 60)
 
 ``name``
     Repository name to use in the web interface.
     (default: current working directory)
 
 ``port``
     Port to listen on. (default: 8000)
 
 ``prefix``
     Prefix path to serve from. (default: '' (server root))
 
 ``push_ssl``
     Whether to require that inbound pushes be transported over SSL to
     prevent password sniffing. (default: True)
 
 ``refreshinterval``
     How frequently directory listings re-scan the filesystem for new
     repositories, in seconds. This is relevant when wildcards are used
     to define paths. Depending on how much filesystem traversal is
     required, refreshing may negatively impact performance.
 
     Values less than or equal to 0 always refresh.
     (default: 20)
 
 ``server-header``
     Value for HTTP ``Server`` response header.
 
 ``static``
     Directory where static files are served from.
 
 ``staticurl``
     Base URL to use for static files. If unset, static files (e.g. the
     hgicon.png favicon) will be served by the CGI script itself. Use
     this setting to serve them directly with the HTTP server.
     Example: ``http://hgserver/static/``.
 
 ``stripes``
     How many lines a "zebra stripe" should span in multi-line output.
     Set to 0 to disable. (default: 1)
 
 ``style``
     Which template map style to use. The available options are the names of
     subdirectories in the HTML templates path. (default: ``paper``)
     Example: ``monoblue``.
 
 ``templates``
     Where to find the HTML templates. The default path to the HTML templates
     can be obtained from ``hg debuginstall``.
 
 ``websub``
 ----------
 
 Web substitution filter definition. You can use this section to
 define a set of regular expression substitution patterns which
 let you automatically modify the hgweb server output.
 
 The default hgweb templates only apply these substitution patterns
 on the revision description fields. You can apply them anywhere
 you want when you create your own templates by adding calls to the
 "websub" filter (usually after calling the "escape" filter).
 
 This can be used, for example, to convert issue references to links
 to your issue tracker, or to convert "markdown-like" syntax into
 HTML (see the examples below).
 
 Each entry in this section names a substitution filter.
 The value of each entry defines the substitution expression itself.
 The websub expressions follow the old interhg extension syntax,
 which in turn imitates the Unix sed replacement syntax::
 
     patternname = s/SEARCH_REGEX/REPLACE_EXPRESSION/[i]
 
 You can use any separator other than "/". The final "i" is optional
 and indicates that the search must be case insensitive.
 
 Examples::
 
     [websub]
     issues = s|issue(\d+)|<a href="http://bts.example.org/issue\1">issue\1</a>|i
     italic = s/\b_(\S+)_\b/<i>\1<\/i>/
     bold = s/\*\b(\S+)\b\*/<b>\1<\/b>/
 
 ``worker``
 ----------
 
 Parallel master/worker configuration. We currently perform working
 directory updates in parallel on Unix-like systems, which greatly
 helps performance.
 
 ``enabled``
     Whether to enable workers code to be used.
     (default: true)
 
 ``numcpus``
     Number of CPUs to use for parallel operations. A zero or
     negative value is treated as ``use the default``.
     (default: 4 or the number of CPUs on the system, whichever is larger)
 
 ``backgroundclose``
     Whether to enable closing file handles on background threads during certain
     operations. Some platforms aren't very efficient at closing file
     handles that have been written or appended to. By performing file closing
     on background threads, file write rate can increase substantially.
     (default: true on Windows, false elsewhere)
 
 ``backgroundcloseminfilecount``
     Minimum number of files required to trigger background file closing.
     Operations not writing this many files won't start background close
     threads.
     (default: 2048)
 
 ``backgroundclosemaxqueue``
     The maximum number of opened file handles waiting to be closed in the
     background. This option only has an effect if ``backgroundclose`` is
     enabled.
     (default: 384)
 
 ``backgroundclosethreadcount``
     Number of threads to process background file closes. Only relevant if
     ``backgroundclose`` is enabled.
     (default: 4)

mercurial/httppeer.py

@@ -1,663 +1,670 b''
 # httppeer.py - HTTP repository proxy classes for mercurial
 #
 # Copyright 2005, 2006 Olivia Mackall <[e-mail]>
 # Copyright 2006 Vadim Gelfer <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 
 import errno
 import io
 import os
 import socket
 import struct
 
 from concurrent import futures
 from .i18n import _
 from .pycompat import getattr
 from . import (
     bundle2,
     error,
     httpconnection,
     pycompat,
     statichttprepo,
     url as urlmod,
     util,
     wireprotov1peer,
 )
 from .utils import urlutil
 
 httplib = util.httplib
 urlerr = util.urlerr
 urlreq = util.urlreq
 
 
 def encodevalueinheaders(value, header, limit):
     """Encode a string value into multiple HTTP headers.
 
     ``value`` will be encoded into 1 or more HTTP headers with the names
     ``header-<N>`` where ``<N>`` is an integer starting at 1. Each header
     name + value will be at most ``limit`` bytes long.
 
     Returns an iterable of 2-tuples consisting of header names and
     values as native strings.
     """
     # HTTP Headers are ASCII. Python 3 requires them to be unicodes,
     # not bytes. This function always takes bytes in as arguments.
     fmt = pycompat.strurl(header) + r'-%s'
     # Note: it is *NOT* a bug that the last bit here is a bytestring
     # and not a unicode: we're just getting the encoded length anyway,
     # and using an r-string to make it portable between Python 2 and 3
     # doesn't work because then the \r is a literal backslash-r
     # instead of a carriage return.
     valuelen = limit - len(fmt % '000') - len(b': \r\n')
     result = []
 
     n = 0
     for i in range(0, len(value), valuelen):
         n += 1
         result.append((fmt % str(n), pycompat.strurl(value[i : i + valuelen])))
 
     return result
 
 
 class _multifile:
     def __init__(self, *fileobjs):
         for f in fileobjs:
             if not util.safehasattr(f, 'length'):
                 raise ValueError(
                     b'_multifile only supports file objects that '
                     b'have a length but this one does not:',
                     type(f),
                     f,
                 )
         self._fileobjs = fileobjs
         self._index = 0
 
     @property
     def length(self):
         return sum(f.length for f in self._fileobjs)
 
     def read(self, amt=None):
         if amt <= 0:
             return b''.join(f.read() for f in self._fileobjs)
         parts = []
         while amt and self._index < len(self._fileobjs):
             parts.append(self._fileobjs[self._index].read(amt))
             got = len(parts[-1])
             if got < amt:
                 self._index += 1
             amt -= got
         return b''.join(parts)
 
     def seek(self, offset, whence=os.SEEK_SET):
         if whence != os.SEEK_SET:
             raise NotImplementedError(
                 b'_multifile does not support anything other'
                 b' than os.SEEK_SET for whence on seek()'
             )
         if offset != 0:
             raise NotImplementedError(
                 b'_multifile only supports seeking to start, but that '
                 b'could be fixed if you need it'
             )
         for f in self._fileobjs:
             f.seek(0)
         self._index = 0
 
 
 def makev1commandrequest(
     ui,
     requestbuilder,
     caps,
     capablefn,
     repobaseurl,
     cmd,
     args,
     remotehidden=False,
 ):
     """Make an HTTP request to run a command for a version 1 client.
 
     ``caps`` is a set of known server capabilities. The value may be
     None if capabilities are not yet known.
 
     ``capablefn`` is a function to evaluate a capability.
 
     ``cmd``, ``args``, and ``data`` define the command, its arguments, and
     raw data to pass to it.
     """
     if cmd == b'pushkey':
         args[b'data'] = b''
     data = args.pop(b'data', None)
     headers = args.pop(b'headers', {})
 
     ui.debug(b"sending %s command\n" % cmd)
     q = [(b'cmd', cmd)]
     if remotehidden:
         q.append(('access-hidden', '1'))
     headersize = 0
     # Important: don't use self.capable() here or else you end up
     # with infinite recursion when trying to look up capabilities
     # for the first time.
     postargsok = caps is not None and b'httppostargs' in caps
 
     # Send arguments via POST.
     if postargsok and args:
         strargs = urlreq.urlencode(sorted(args.items()))
         if not data:
             data = strargs
         else:
             if isinstance(data, bytes):
                 i = io.BytesIO(data)
                 i.length = len(data)
                 data = i
             argsio = io.BytesIO(strargs)
             argsio.length = len(strargs)
             data = _multifile(argsio, data)
         headers['X-HgArgs-Post'] = len(strargs)
     elif args:
         # Calling self.capable() can infinite loop if we are calling
         # "capabilities". But that command should never accept wire
         # protocol arguments. So this should never happen.
         assert cmd != b'capabilities'
         httpheader = capablefn(b'httpheader')
         if httpheader:
             headersize = int(httpheader.split(b',', 1)[0])
 
         # Send arguments via HTTP headers.
         if headersize > 0:
             # The headers can typically carry more data than the URL.
             encoded_args = urlreq.urlencode(sorted(args.items()))
             for header, value in encodevalueinheaders(
                 encoded_args, b'X-HgArg', headersize
             ):
                 headers[header] = value
         # Send arguments via query string (Mercurial <1.9).
         else:
             q += sorted(args.items())
 
     qs = b'?%s' % urlreq.urlencode(q)
     cu = b"%s%s" % (repobaseurl, qs)
     size = 0
     if util.safehasattr(data, 'length'):
         size = data.length
     elif data is not None:
         size = len(data)
     if data is not None and 'Content-Type' not in headers:
         headers['Content-Type'] = 'application/mercurial-0.1'
 
     # Tell the server we accept application/mercurial-0.2 and multiple
     # compression formats if the server is capable of emitting those
     # payloads.
     # Note: Keep this set empty by default, as client advertisement of
     # protocol parameters should only occur after the handshake.
     protoparams = set()
 
     mediatypes = set()
     if caps is not None:
         mt = capablefn(b'httpmediatype')
         if mt:
             protoparams.add(b'0.1')
             mediatypes = set(mt.split(b','))
 
         protoparams.add(b'partial-pull')
 
     if b'0.2tx' in mediatypes:
         protoparams.add(b'0.2')
 
     if b'0.2tx' in mediatypes and capablefn(b'compression'):
         # We /could/ compare supported compression formats and prune
         # non-mutually supported or error if nothing is mutually supported.
         # For now, send the full list to the server and have it error.
         comps = [
             e.wireprotosupport().name
             for e in util.compengines.supportedwireengines(util.CLIENTROLE)
         ]
         protoparams.add(b'comp=%s' % b','.join(comps))
 
     if protoparams:
         protoheaders = encodevalueinheaders(
             b' '.join(sorted(protoparams)), b'X-HgProto', headersize or 1024
         )
         for header, value in protoheaders:
             headers[header] = value
 
     varyheaders = []
     for header in headers:
         if header.lower().startswith('x-hg'):
             varyheaders.append(header)
 
     if varyheaders:
         headers['Vary'] = ','.join(sorted(varyheaders))
 
     req = requestbuilder(pycompat.strurl(cu), data, headers)
 
     if data is not None:
         ui.debug(b"sending %d bytes\n" % size)
         req.add_unredirected_header('Content-Length', '%d' % size)
 
     return req, cu, qs
 
 
 def sendrequest(ui, opener, req):
     """Send a prepared HTTP request.
 
     Returns the response object.
     """
     dbg = ui.debug
     if ui.debugflag and ui.configbool(b'devel', b'debug.peer-request'):
         line = b'devel-peer-request: %s\n'
         dbg(
             line
             % b'%s %s'
             % (
                 pycompat.bytesurl(req.get_method()),
                 pycompat.bytesurl(req.get_full_url()),
             )
         )
         hgargssize = None
 
         for header, value in sorted(req.header_items()):
             header = pycompat.bytesurl(header)
             value = pycompat.bytesurl(value)
             if header.startswith(b'X-hgarg-'):
                 if hgargssize is None:
                     hgargssize = 0
                 hgargssize += len(value)
             else:
                 dbg(line % b'  %s %s' % (header, value))
 
         if hgargssize is not None:
             dbg(
                 line
                 % b'  %d bytes of commands arguments in headers'
                 % hgargssize
             )
         data = req.data
         if data is not None:
             length = getattr(data, 'length', None)
             if length is None:
                 length = len(data)
             dbg(line % b'  %d bytes of data' % length)
 
         start = util.timer()
 
     res = None
     try:
         res = opener.open(req)
     except urlerr.httperror as inst:
         if inst.code == 401:
             raise error.Abort(_(b'authorization failed'))
         raise
     except httplib.HTTPException as inst:
         ui.debug(
             b'http error requesting %s\n'
             % urlutil.hidepassword(req.get_full_url())
         )
         ui.traceback()
         raise IOError(None, inst)
     finally:
         if ui.debugflag and ui.configbool(b'devel', b'debug.peer-request'):
             code = res.code if res else -1
             dbg(
                 line
                 % b'  finished in %.4f seconds (%d)'
                 % (util.timer() - start, code)
             )
 
     # Insert error handlers for common I/O failures.
     urlmod.wrapresponse(res)
 
     return res
 
 
 class RedirectedRepoError(error.RepoError):
     def __init__(self, msg, respurl):
         super(RedirectedRepoError, self).__init__(msg)
         self.respurl = respurl
 
 
 def parsev1commandresponse(ui, baseurl, requrl, qs, resp, compressible):
     # record the url we got redirected to
     redirected = False
     respurl = pycompat.bytesurl(resp.geturl())
     if respurl.endswith(qs):
         respurl = respurl[: -len(qs)]
         qsdropped = False
     else:
         qsdropped = True
 
     if baseurl.rstrip(b'/') != respurl.rstrip(b'/'):
         redirected = True
         if not ui.quiet:
             ui.warn(_(b'real URL is %s\n') % respurl)
 
     try:
         proto = pycompat.bytesurl(resp.getheader('content-type', ''))
     except AttributeError:
         proto = pycompat.bytesurl(resp.headers.get('content-type', ''))
 
     safeurl = urlutil.hidepassword(baseurl)
     if proto.startswith(b'application/hg-error'):
         raise error.OutOfBandError(resp.read())
 
     # Pre 1.0 versions of Mercurial used text/plain and
     # application/hg-changegroup. We don't support such old servers.
     if not proto.startswith(b'application/mercurial-'):
         ui.debug(b"requested URL: '%s'\n" % urlutil.hidepassword(requrl))
         msg = _(
             b"'%s' does not appear to be an hg repository:\n"
             b"---%%<--- (%s)\n%s\n---%%<---\n"
         ) % (safeurl, proto or b'no content-type', resp.read(1024))
 
         # Some servers may strip the query string from the redirect. We
         # raise a special error type so callers can react to this specially.
         if redirected and qsdropped:
             raise RedirectedRepoError(msg, respurl)
         else:
             raise error.RepoError(msg)
 
     try:
         subtype = proto.split(b'-', 1)[1]
 
         version_info = tuple([int(n) for n in subtype.split(b'.')])
     except ValueError:
         raise error.RepoError(
             _(b"'%s' sent a broken Content-Type header (%s)") % (safeurl, proto)
         )
 
     # TODO consider switching to a decompression reader that uses
     # generators.
     if version_info == (0, 1):
         if compressible:
             resp = util.compengines[b'zlib'].decompressorreader(resp)
 
     elif version_info == (0, 2):
         # application/mercurial-0.2 always identifies the compression
         # engine in the payload header.
         elen = struct.unpack(b'B', util.readexactly(resp, 1))[0]
         ename = util.readexactly(resp, elen)
         engine = util.compengines.forwiretype(ename)
 
         resp = engine.decompressorreader(resp)
     else:
         raise error.RepoError(
             _(b"'%s' uses newer protocol %s") % (safeurl, subtype)
         )
 
     return respurl, proto, resp
 
 
 class httppeer(wireprotov1peer.wirepeer):
     def __init__(
         self, ui, path, url, opener, requestbuilder, caps, remotehidden=False
     ):
         super().__init__(ui, path=path, remotehidden=remotehidden)
         self._url = url
         self._caps = caps
         self.limitedarguments = caps is not None and b'httppostargs' not in caps
         self._urlopener = opener
         self._requestbuilder = requestbuilder
         self._remotehidden = remotehidden
 
     def __del__(self):
         for h in self._urlopener.handlers:
             h.close()
             getattr(h, "close_all", lambda: None)()
 
     # Begin of ipeerconnection interface.
 
     def url(self):
         return self.path.loc
 
     def local(self):
         return None
 
     def canpush(self):
         return True
 
     def close(self):
         try:
             reqs, sent, recv = (
                 self._urlopener.requestscount,
                 self._urlopener.sentbytescount,
                 self._urlopener.receivedbytescount,
             )
         except AttributeError:
             return
         self.ui.note(
             _(
                 b'(sent %d HTTP requests and %d bytes; '
                 b'received %d bytes in responses)\n'
             )
             % (reqs, sent, recv)
         )
 
     # End of ipeerconnection interface.
 
     # Begin of ipeercommands interface.
 
     def capabilities(self):
         return self._caps
 
+    def _finish_inline_clone_bundle(self, stream):
+        # HTTP streams must hit the end to process the last empty
+        # chunk of Chunked-Encoding so the connection can be reused.
+        chunk = stream.read(1)
+        if chunk:
+            self._abort(error.ResponseError(_(b"unexpected response:"), chunk))
+
     # End of ipeercommands interface.
 
     def _callstream(self, cmd, _compressible=False, **args):
         args = pycompat.byteskwargs(args)
 
         req, cu, qs = makev1commandrequest(
             self.ui,
             self._requestbuilder,
             self._caps,
             self.capable,
             self._url,
             cmd,
             args,
             self._remotehidden,
         )
 
         resp = sendrequest(self.ui, self._urlopener, req)
 
         self._url, ct, resp = parsev1commandresponse(
             self.ui, self._url, cu, qs, resp, _compressible
         )
 
         return resp
 
     def _call(self, cmd, **args):
         fp = self._callstream(cmd, **args)
         try:
             return fp.read()
         finally:
             # if using keepalive, allow connection to be reused
             fp.close()
 
     def _callpush(self, cmd, cg, **args):
         # have to stream bundle to a temp file because we do not have
         # http 1.1 chunked transfer.
 
         types = self.capable(b'unbundle')
         try:
             types = types.split(b',')
         except AttributeError:
             # servers older than d1b16a746db6 will send 'unbundle' as a
             # boolean capability. They only support headerless/uncompressed
             # bundles.
             types = [b""]
         for x in types:
             if x in bundle2.bundletypes:
                 type = x
                 break
 
         tempname = bundle2.writebundle(self.ui, cg, None, type)
         fp = httpconnection.httpsendfile(self.ui, tempname, b"rb")
         headers = {'Content-Type': 'application/mercurial-0.1'}
 
         try:
             r = self._call(cmd, data=fp, headers=headers, **args)
             vals = r.split(b'\n', 1)
             if len(vals) < 2:
                 raise error.ResponseError(_(b"unexpected response:"), r)
             return vals
         except urlerr.httperror:
             # Catch and re-raise these so we don't try and treat them
             # like generic socket errors. They lack any values in
             # .args on Python 3 which breaks our socket.error block.
             raise
         except socket.error as err:
             if err.args[0] in (errno.ECONNRESET, errno.EPIPE):
                 raise error.Abort(_(b'push failed: %s') % err.args[1])
             raise error.Abort(err.args[1])
         finally:
             fp.close()
             os.unlink(tempname)
 
     def _calltwowaystream(self, cmd, fp, **args):
         filename = None
         try:
             # dump bundle to disk
             fd, filename = pycompat.mkstemp(prefix=b"hg-bundle-", suffix=b".hg")
             with os.fdopen(fd, "wb") as fh:
                 d = fp.read(4096)
                 while d:
                     fh.write(d)
                     d = fp.read(4096)
             # start http push
             with httpconnection.httpsendfile(self.ui, filename, b"rb") as fp_:
                 headers = {'Content-Type': 'application/mercurial-0.1'}
                 return self._callstream(cmd, data=fp_, headers=headers, **args)
         finally:
             if filename is not None:
                 os.unlink(filename)
 
     def _callcompressable(self, cmd, **args):
         return self._callstream(cmd, _compressible=True, **args)
 
     def _abort(self, exception):
         raise exception
 
 
 class queuedcommandfuture(futures.Future):
     """Wraps result() on command futures to trigger submission on call."""
 
     def result(self, timeout=None):
         if self.done():
             return futures.Future.result(self, timeout)
 
         self._peerexecutor.sendcommands()
 
         # sendcommands() will restore the original __class__ and self.result
         # will resolve to Future.result.
         return self.result(timeout)
 
 
 def performhandshake(ui, url, opener, requestbuilder):
     # The handshake is a request to the capabilities command.
 
     caps = None
 
     def capable(x):
         raise error.ProgrammingError(b'should not be called')
 
     args = {}
 
     req, requrl, qs = makev1commandrequest(
         ui, requestbuilder, caps, capable, url, b'capabilities', args
     )
     resp = sendrequest(ui, opener, req)
 
     # The server may redirect us to the repo root, stripping the
     # ?cmd=capabilities query string from the URL. The server would likely
     # return HTML in this case and ``parsev1commandresponse()`` would raise.
     # We catch this special case and re-issue the capabilities request against
     # the new URL.
     #
     # We should ideally not do this, as a redirect that drops the query
     # string from the URL is arguably a server bug. (Garbage in, garbage out).
     # However,  Mercurial clients for several years appeared to handle this
     # issue without behavior degradation. And according to issue 5860, it may
     # be a longstanding bug in some server implementations. So we allow a
     # redirect that drops the query string to "just work."
     try:
         respurl, ct, resp = parsev1commandresponse(
             ui, url, requrl, qs, resp, compressible=False
         )
     except RedirectedRepoError as e:
         req, requrl, qs = makev1commandrequest(
             ui, requestbuilder, caps, capable, e.respurl, b'capabilities', args
         )
         resp = sendrequest(ui, opener, req)
         respurl, ct, resp = parsev1commandresponse(
             ui, url, requrl, qs, resp, compressible=False
         )
 
     try:
         rawdata = resp.read()
     finally:
         resp.close()
 
     if not ct.startswith(b'application/mercurial-'):
         raise error.ProgrammingError(b'unexpected content-type: %s' % ct)
 
     info = {b'v1capabilities': set(rawdata.split())}
 
     return respurl, info
 
 
 def _make_peer(
     ui, path, opener=None, requestbuilder=urlreq.request, remotehidden=False
 ):
     """Construct an appropriate HTTP peer instance.
 
     ``opener`` is an ``url.opener`` that should be used to establish
     connections, perform HTTP requests.
 
     ``requestbuilder`` is the type used for constructing HTTP requests.
     It exists as an argument so extensions can override the default.
     """
     if path.url.query or path.url.fragment:
         msg = _(b'unsupported URL component: "%s"')
         msg %= path.url.query or path.url.fragment
         raise error.Abort(msg)
 
     # urllib cannot handle URLs with embedded user or passwd.
     url, authinfo = path.url.authinfo()
     ui.debug(b'using %s\n' % url)
 
     opener = opener or urlmod.opener(ui, authinfo)
 
     respurl, info = performhandshake(ui, url, opener, requestbuilder)
 
     return httppeer(
         ui,
         path,
         respurl,
         opener,
         requestbuilder,
         info[b'v1capabilities'],
         remotehidden=remotehidden,
     )
 
 
 def make_peer(
     ui, path, create, intents=None, createopts=None, remotehidden=False
 ):
     if create:
         raise error.Abort(_(b'cannot create new http repository'))
     try:
         if path.url.scheme == b'https' and not urlmod.has_https:
             raise error.Abort(
                 _(b'Python support for SSL and HTTPS is not installed')
             )
 
         inst = _make_peer(ui, path, remotehidden=remotehidden)
 
         return inst
     except error.RepoError as httpexception:
         try:
             r = statichttprepo.make_peer(ui, b"static-" + path.loc, create)
             ui.note(_(b'(falling back to static-http)\n'))
             return r
         except error.RepoError:
             raise httpexception  # use the original http RepoError instead

mercurial/interfaces/repository.py

@@ -1,2073 +1,2079 b''
 # repository.py - Interfaces and base classes for repositories and peers.
 # coding: utf-8
 #
 # Copyright 2017 Gregory Szorc <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 
 from ..i18n import _
 from .. import error
 from . import util as interfaceutil
 
 # Local repository feature string.
 
 # Revlogs are being used for file storage.
 REPO_FEATURE_REVLOG_FILE_STORAGE = b'revlogfilestorage'
 # The storage part of the repository is shared from an external source.
 REPO_FEATURE_SHARED_STORAGE = b'sharedstore'
 # LFS supported for backing file storage.
 REPO_FEATURE_LFS = b'lfs'
 # Repository supports being stream cloned.
 REPO_FEATURE_STREAM_CLONE = b'streamclone'
 # Repository supports (at least) some sidedata to be stored
 REPO_FEATURE_SIDE_DATA = b'side-data'
 # Files storage may lack data for all ancestors.
 REPO_FEATURE_SHALLOW_FILE_STORAGE = b'shallowfilestorage'
 
 REVISION_FLAG_CENSORED = 1 << 15
 REVISION_FLAG_ELLIPSIS = 1 << 14
 REVISION_FLAG_EXTSTORED = 1 << 13
 REVISION_FLAG_HASCOPIESINFO = 1 << 12
 
 REVISION_FLAGS_KNOWN = (
     REVISION_FLAG_CENSORED
     | REVISION_FLAG_ELLIPSIS
     | REVISION_FLAG_EXTSTORED
     | REVISION_FLAG_HASCOPIESINFO
 )
 
 CG_DELTAMODE_STD = b'default'
 CG_DELTAMODE_PREV = b'previous'
 CG_DELTAMODE_FULL = b'fulltext'
 CG_DELTAMODE_P1 = b'p1'
 
 
 ## Cache related constants:
 #
 # Used to control which cache should be warmed in a repo.updatecaches(…) call.
 
 # Warm branchmaps of all known repoview's filter-level
 CACHE_BRANCHMAP_ALL = b"branchmap-all"
 # Warm branchmaps of repoview's filter-level used by server
 CACHE_BRANCHMAP_SERVED = b"branchmap-served"
 # Warm internal changelog cache (eg: persistent nodemap)
 CACHE_CHANGELOG_CACHE = b"changelog-cache"
 # Warm full manifest cache
 CACHE_FULL_MANIFEST = b"full-manifest"
 # Warm file-node-tags cache
 CACHE_FILE_NODE_TAGS = b"file-node-tags"
 # Warm internal manifestlog cache (eg: persistent nodemap)
 CACHE_MANIFESTLOG_CACHE = b"manifestlog-cache"
 # Warn rev branch cache
 CACHE_REV_BRANCH = b"rev-branch-cache"
 # Warm tags' cache for default repoview'
 CACHE_TAGS_DEFAULT = b"tags-default"
 # Warm tags' cache for  repoview's filter-level used by server
 CACHE_TAGS_SERVED = b"tags-served"
 
 # the cache to warm by default after a simple transaction
 # (this is a mutable set to let extension update it)
 CACHES_DEFAULT = {
     CACHE_BRANCHMAP_SERVED,
 }
 
 # the caches to warm when warming all of them
 # (this is a mutable set to let extension update it)
 CACHES_ALL = {
     CACHE_BRANCHMAP_SERVED,
     CACHE_BRANCHMAP_ALL,
     CACHE_CHANGELOG_CACHE,
     CACHE_FILE_NODE_TAGS,
     CACHE_FULL_MANIFEST,
     CACHE_MANIFESTLOG_CACHE,
     CACHE_TAGS_DEFAULT,
     CACHE_TAGS_SERVED,
 }
 
 # the cache to warm by default on simple call
 # (this is a mutable set to let extension update it)
 CACHES_POST_CLONE = CACHES_ALL.copy()
 CACHES_POST_CLONE.discard(CACHE_FILE_NODE_TAGS)
 
 
 class ipeerconnection(interfaceutil.Interface):
     """Represents a "connection" to a repository.
 
     This is the base interface for representing a connection to a repository.
     It holds basic properties and methods applicable to all peer types.
 
     This is not a complete interface definition and should not be used
     outside of this module.
     """
 
     ui = interfaceutil.Attribute("""ui.ui instance""")
     path = interfaceutil.Attribute("""a urlutil.path instance or None""")
 
     def url():
         """Returns a URL string representing this peer.
 
         Currently, implementations expose the raw URL used to construct the
         instance. It may contain credentials as part of the URL. The
         expectations of the value aren't well-defined and this could lead to
         data leakage.
 
         TODO audit/clean consumers and more clearly define the contents of this
         value.
         """
 
     def local():
         """Returns a local repository instance.
 
         If the peer represents a local repository, returns an object that
         can be used to interface with it. Otherwise returns ``None``.
         """
 
     def canpush():
         """Returns a boolean indicating if this peer can be pushed to."""
 
     def close():
         """Close the connection to this peer.
 
         This is called when the peer will no longer be used. Resources
         associated with the peer should be cleaned up.
         """
 
 
 class ipeercapabilities(interfaceutil.Interface):
     """Peer sub-interface related to capabilities."""
 
     def capable(name):
         """Determine support for a named capability.
 
         Returns ``False`` if capability not supported.
 
         Returns ``True`` if boolean capability is supported. Returns a string
         if capability support is non-boolean.
 
         Capability strings may or may not map to wire protocol capabilities.
         """
 
     def requirecap(name, purpose):
         """Require a capability to be present.
 
         Raises a ``CapabilityError`` if the capability isn't present.
         """
 
 
 class ipeercommands(interfaceutil.Interface):
     """Client-side interface for communicating over the wire protocol.
 
     This interface is used as a gateway to the Mercurial wire protocol.
     methods commonly call wire protocol commands of the same name.
     """
 
     def branchmap():
         """Obtain heads in named branches.
 
         Returns a dict mapping branch name to an iterable of nodes that are
         heads on that branch.
         """
 
     def capabilities():
         """Obtain capabilities of the peer.
 
         Returns a set of string capabilities.
         """
 
+    def get_inline_clone_bundle(path):
+        """Retrieve clonebundle across the wire.
+
+        Returns a chunkbuffer
+        """
+
     def clonebundles():
         """Obtains the clone bundles manifest for the repo.
 
         Returns the manifest as unparsed bytes.
         """
 
     def debugwireargs(one, two, three=None, four=None, five=None):
         """Used to facilitate debugging of arguments passed over the wire."""
 
     def getbundle(source, **kwargs):
         """Obtain remote repository data as a bundle.
 
         This command is how the bulk of repository data is transferred from
         the peer to the local repository
 
         Returns a generator of bundle data.
         """
 
     def heads():
         """Determine all known head revisions in the peer.
 
         Returns an iterable of binary nodes.
         """
 
     def known(nodes):
         """Determine whether multiple nodes are known.
 
         Accepts an iterable of nodes whose presence to check for.
 
         Returns an iterable of booleans indicating of the corresponding node
         at that index is known to the peer.
         """
 
     def listkeys(namespace):
         """Obtain all keys in a pushkey namespace.
 
         Returns an iterable of key names.
         """
 
     def lookup(key):
         """Resolve a value to a known revision.
 
         Returns a binary node of the resolved revision on success.
         """
 
     def pushkey(namespace, key, old, new):
         """Set a value using the ``pushkey`` protocol.
 
         Arguments correspond to the pushkey namespace and key to operate on and
         the old and new values for that key.
 
         Returns a string with the peer result. The value inside varies by the
         namespace.
         """
 
     def stream_out():
         """Obtain streaming clone data.
 
         Successful result should be a generator of data chunks.
         """
 
     def unbundle(bundle, heads, url):
         """Transfer repository data to the peer.
 
         This is how the bulk of data during a push is transferred.
 
         Returns the integer number of heads added to the peer.
         """
 
 
 class ipeerlegacycommands(interfaceutil.Interface):
     """Interface for implementing support for legacy wire protocol commands.
 
     Wire protocol commands transition to legacy status when they are no longer
     used by modern clients. To facilitate identifying which commands are
     legacy, the interfaces are split.
     """
 
     def between(pairs):
         """Obtain nodes between pairs of nodes.
 
         ``pairs`` is an iterable of node pairs.
 
         Returns an iterable of iterables of nodes corresponding to each
         requested pair.
         """
 
     def branches(nodes):
         """Obtain ancestor changesets of specific nodes back to a branch point.
 
         For each requested node, the peer finds the first ancestor node that is
         a DAG root or is a merge.
 
         Returns an iterable of iterables with the resolved values for each node.
         """
 
     def changegroup(nodes, source):
         """Obtain a changegroup with data for descendants of specified nodes."""
 
     def changegroupsubset(bases, heads, source):
         pass
 
 
 class ipeercommandexecutor(interfaceutil.Interface):
     """Represents a mechanism to execute remote commands.
 
     This is the primary interface for requesting that wire protocol commands
     be executed. Instances of this interface are active in a context manager
     and have a well-defined lifetime. When the context manager exits, all
     outstanding requests are waited on.
     """
 
     def callcommand(name, args):
         """Request that a named command be executed.
 
         Receives the command name and a dictionary of command arguments.
 
         Returns a ``concurrent.futures.Future`` that will resolve to the
         result of that command request. That exact value is left up to
         the implementation and possibly varies by command.
 
         Not all commands can coexist with other commands in an executor
         instance: it depends on the underlying wire protocol transport being
         used and the command itself.
 
         Implementations MAY call ``sendcommands()`` automatically if the
         requested command can not coexist with other commands in this executor.
 
         Implementations MAY call ``sendcommands()`` automatically when the
         future's ``result()`` is called. So, consumers using multiple
         commands with an executor MUST ensure that ``result()`` is not called
         until all command requests have been issued.
         """
 
     def sendcommands():
         """Trigger submission of queued command requests.
 
         Not all transports submit commands as soon as they are requested to
         run. When called, this method forces queued command requests to be
         issued. It will no-op if all commands have already been sent.
 
         When called, no more new commands may be issued with this executor.
         """
 
     def close():
         """Signal that this command request is finished.
 
         When called, no more new commands may be issued. All outstanding
         commands that have previously been issued are waited on before
         returning. This not only includes waiting for the futures to resolve,
         but also waiting for all response data to arrive. In other words,
         calling this waits for all on-wire state for issued command requests
         to finish.
 
         When used as a context manager, this method is called when exiting the
         context manager.
 
         This method may call ``sendcommands()`` if there are buffered commands.
         """
 
 
 class ipeerrequests(interfaceutil.Interface):
     """Interface for executing commands on a peer."""
 
     limitedarguments = interfaceutil.Attribute(
         """True if the peer cannot receive large argument value for commands."""
     )
 
     def commandexecutor():
         """A context manager that resolves to an ipeercommandexecutor.
 
         The object this resolves to can be used to issue command requests
         to the peer.
 
         Callers should call its ``callcommand`` method to issue command
         requests.
 
         A new executor should be obtained for each distinct set of commands
         (possibly just a single command) that the consumer wants to execute
         as part of a single operation or round trip. This is because some
         peers are half-duplex and/or don't support persistent connections.
         e.g. in the case of HTTP peers, commands sent to an executor represent
         a single HTTP request. While some peers may support multiple command
         sends over the wire per executor, consumers need to code to the least
         capable peer. So it should be assumed that command executors buffer
         called commands until they are told to send them and that each
         command executor could result in a new connection or wire-level request
         being issued.
         """
 
 
 class ipeerbase(ipeerconnection, ipeercapabilities, ipeerrequests):
     """Unified interface for peer repositories.
 
     All peer instances must conform to this interface.
     """
 
 
 class ipeerv2(ipeerconnection, ipeercapabilities, ipeerrequests):
     """Unified peer interface for wire protocol version 2 peers."""
 
     apidescriptor = interfaceutil.Attribute(
         """Data structure holding description of server API."""
     )
 
 
 @interfaceutil.implementer(ipeerbase)
 class peer:
     """Base class for peer repositories."""
 
     limitedarguments = False
 
     def __init__(self, ui, path=None, remotehidden=False):
         self.ui = ui
         self.path = path
 
     def capable(self, name):
         caps = self.capabilities()
         if name in caps:
             return True
 
         name = b'%s=' % name
         for cap in caps:
             if cap.startswith(name):
                 return cap[len(name) :]
 
         return False
 
     def requirecap(self, name, purpose):
         if self.capable(name):
             return
 
         raise error.CapabilityError(
             _(
                 b'cannot %s; remote repository does not support the '
                 b'\'%s\' capability'
             )
             % (purpose, name)
         )
 
 
 class iverifyproblem(interfaceutil.Interface):
     """Represents a problem with the integrity of the repository.
 
     Instances of this interface are emitted to describe an integrity issue
     with a repository (e.g. corrupt storage, missing data, etc).
 
     Instances are essentially messages associated with severity.
     """
 
     warning = interfaceutil.Attribute(
         """Message indicating a non-fatal problem."""
     )
 
     error = interfaceutil.Attribute("""Message indicating a fatal problem.""")
 
     node = interfaceutil.Attribute(
         """Revision encountering the problem.
 
         ``None`` means the problem doesn't apply to a single revision.
         """
     )
 
 
 class irevisiondelta(interfaceutil.Interface):
     """Represents a delta between one revision and another.
 
     Instances convey enough information to allow a revision to be exchanged
     with another repository.
 
     Instances represent the fulltext revision data or a delta against
     another revision. Therefore the ``revision`` and ``delta`` attributes
     are mutually exclusive.
 
     Typically used for changegroup generation.
     """
 
     node = interfaceutil.Attribute("""20 byte node of this revision.""")
 
     p1node = interfaceutil.Attribute(
         """20 byte node of 1st parent of this revision."""
     )
 
     p2node = interfaceutil.Attribute(
         """20 byte node of 2nd parent of this revision."""
     )
 
     linknode = interfaceutil.Attribute(
         """20 byte node of the changelog revision this node is linked to."""
     )
 
     flags = interfaceutil.Attribute(
         """2 bytes of integer flags that apply to this revision.
 
         This is a bitwise composition of the ``REVISION_FLAG_*`` constants.
         """
     )
 
     basenode = interfaceutil.Attribute(
         """20 byte node of the revision this data is a delta against.
 
         ``nullid`` indicates that the revision is a full revision and not
         a delta.
         """
     )
 
     baserevisionsize = interfaceutil.Attribute(
         """Size of base revision this delta is against.
 
         May be ``None`` if ``basenode`` is ``nullid``.
         """
     )
 
     revision = interfaceutil.Attribute(
         """Raw fulltext of revision data for this node."""
     )
 
     delta = interfaceutil.Attribute(
         """Delta between ``basenode`` and ``node``.
 
         Stored in the bdiff delta format.
         """
     )
 
     sidedata = interfaceutil.Attribute(
         """Raw sidedata bytes for the given revision."""
     )
 
     protocol_flags = interfaceutil.Attribute(
         """Single byte of integer flags that can influence the protocol.
 
         This is a bitwise composition of the ``storageutil.CG_FLAG*`` constants.
         """
     )
 
 
 class ifilerevisionssequence(interfaceutil.Interface):
     """Contains index data for all revisions of a file.
 
     Types implementing this behave like lists of tuples. The index
     in the list corresponds to the revision number. The values contain
     index metadata.
 
     The *null* revision (revision number -1) is always the last item
     in the index.
     """
 
     def __len__():
         """The total number of revisions."""
 
     def __getitem__(rev):
         """Returns the object having a specific revision number.
 
         Returns an 8-tuple with the following fields:
 
         offset+flags
            Contains the offset and flags for the revision. 64-bit unsigned
            integer where first 6 bytes are the offset and the next 2 bytes
            are flags. The offset can be 0 if it is not used by the store.
         compressed size
             Size of the revision data in the store. It can be 0 if it isn't
             needed by the store.
         uncompressed size
             Fulltext size. It can be 0 if it isn't needed by the store.
         base revision
             Revision number of revision the delta for storage is encoded
             against. -1 indicates not encoded against a base revision.
         link revision
             Revision number of changelog revision this entry is related to.
         p1 revision
             Revision number of 1st parent. -1 if no 1st parent.
         p2 revision
             Revision number of 2nd parent. -1 if no 1st parent.
         node
             Binary node value for this revision number.
 
         Negative values should index off the end of the sequence. ``-1``
         should return the null revision. ``-2`` should return the most
         recent revision.
         """
 
     def __contains__(rev):
         """Whether a revision number exists."""
 
     def insert(self, i, entry):
         """Add an item to the index at specific revision."""
 
 
 class ifileindex(interfaceutil.Interface):
     """Storage interface for index data of a single file.
 
     File storage data is divided into index metadata and data storage.
     This interface defines the index portion of the interface.
 
     The index logically consists of:
 
     * A mapping between revision numbers and nodes.
     * DAG data (storing and querying the relationship between nodes).
     * Metadata to facilitate storage.
     """
 
     nullid = interfaceutil.Attribute(
         """node for the null revision for use as delta base."""
     )
 
     def __len__():
         """Obtain the number of revisions stored for this file."""
 
     def __iter__():
         """Iterate over revision numbers for this file."""
 
     def hasnode(node):
         """Returns a bool indicating if a node is known to this store.
 
         Implementations must only return True for full, binary node values:
         hex nodes, revision numbers, and partial node matches must be
         rejected.
 
         The null node is never present.
         """
 
     def revs(start=0, stop=None):
         """Iterate over revision numbers for this file, with control."""
 
     def parents(node):
         """Returns a 2-tuple of parent nodes for a revision.
 
         Values will be ``nullid`` if the parent is empty.
         """
 
     def parentrevs(rev):
         """Like parents() but operates on revision numbers."""
 
     def rev(node):
         """Obtain the revision number given a node.
 
         Raises ``error.LookupError`` if the node is not known.
         """
 
     def node(rev):
         """Obtain the node value given a revision number.
 
         Raises ``IndexError`` if the node is not known.
         """
 
     def lookup(node):
         """Attempt to resolve a value to a node.
 
         Value can be a binary node, hex node, revision number, or a string
         that can be converted to an integer.
 
         Raises ``error.LookupError`` if a node could not be resolved.
         """
 
     def linkrev(rev):
         """Obtain the changeset revision number a revision is linked to."""
 
     def iscensored(rev):
         """Return whether a revision's content has been censored."""
 
     def commonancestorsheads(node1, node2):
         """Obtain an iterable of nodes containing heads of common ancestors.
 
         See ``ancestor.commonancestorsheads()``.
         """
 
     def descendants(revs):
         """Obtain descendant revision numbers for a set of revision numbers.
 
         If ``nullrev`` is in the set, this is equivalent to ``revs()``.
         """
 
     def heads(start=None, stop=None):
         """Obtain a list of nodes that are DAG heads, with control.
 
         The set of revisions examined can be limited by specifying
         ``start`` and ``stop``. ``start`` is a node. ``stop`` is an
         iterable of nodes. DAG traversal starts at earlier revision
         ``start`` and iterates forward until any node in ``stop`` is
         encountered.
         """
 
     def children(node):
         """Obtain nodes that are children of a node.
 
         Returns a list of nodes.
         """
 
 
 class ifiledata(interfaceutil.Interface):
     """Storage interface for data storage of a specific file.
 
     This complements ``ifileindex`` and provides an interface for accessing
     data for a tracked file.
     """
 
     def size(rev):
         """Obtain the fulltext size of file data.
 
         Any metadata is excluded from size measurements.
         """
 
     def revision(node, raw=False):
         """Obtain fulltext data for a node.
 
         By default, any storage transformations are applied before the data
         is returned. If ``raw`` is True, non-raw storage transformations
         are not applied.
 
         The fulltext data may contain a header containing metadata. Most
         consumers should use ``read()`` to obtain the actual file data.
         """
 
     def rawdata(node):
         """Obtain raw data for a node."""
 
     def read(node):
         """Resolve file fulltext data.
 
         This is similar to ``revision()`` except any metadata in the data
         headers is stripped.
         """
 
     def renamed(node):
         """Obtain copy metadata for a node.
 
         Returns ``False`` if no copy metadata is stored or a 2-tuple of
         (path, node) from which this revision was copied.
         """
 
     def cmp(node, fulltext):
         """Compare fulltext to another revision.
 
         Returns True if the fulltext is different from what is stored.
 
         This takes copy metadata into account.
 
         TODO better document the copy metadata and censoring logic.
         """
 
     def emitrevisions(
         nodes,
         nodesorder=None,
         revisiondata=False,
         assumehaveparentrevisions=False,
         deltamode=CG_DELTAMODE_STD,
     ):
         """Produce ``irevisiondelta`` for revisions.
 
         Given an iterable of nodes, emits objects conforming to the
         ``irevisiondelta`` interface that describe revisions in storage.
 
         This method is a generator.
 
         The input nodes may be unordered. Implementations must ensure that a
         node's parents are emitted before the node itself. Transitively, this
         means that a node may only be emitted once all its ancestors in
         ``nodes`` have also been emitted.
 
         By default, emits "index" data (the ``node``, ``p1node``, and
         ``p2node`` attributes). If ``revisiondata`` is set, revision data
         will also be present on the emitted objects.
 
         With default argument values, implementations can choose to emit
         either fulltext revision data or a delta. When emitting deltas,
         implementations must consider whether the delta's base revision
         fulltext is available to the receiver.
 
         The base revision fulltext is guaranteed to be available if any of
         the following are met:
 
         * Its fulltext revision was emitted by this method call.
         * A delta for that revision was emitted by this method call.
         * ``assumehaveparentrevisions`` is True and the base revision is a
           parent of the node.
 
         ``nodesorder`` can be used to control the order that revisions are
         emitted. By default, revisions can be reordered as long as they are
         in DAG topological order (see above). If the value is ``nodes``,
         the iteration order from ``nodes`` should be used. If the value is
         ``storage``, then the native order from the backing storage layer
         is used. (Not all storage layers will have strong ordering and behavior
         of this mode is storage-dependent.) ``nodes`` ordering can force
         revisions to be emitted before their ancestors, so consumers should
         use it with care.
 
         The ``linknode`` attribute on the returned ``irevisiondelta`` may not
         be set and it is the caller's responsibility to resolve it, if needed.
 
         If ``deltamode`` is CG_DELTAMODE_PREV and revision data is requested,
         all revision data should be emitted as deltas against the revision
         emitted just prior. The initial revision should be a delta against its
         1st parent.
         """
 
 
 class ifilemutation(interfaceutil.Interface):
     """Storage interface for mutation events of a tracked file."""
 
     def add(filedata, meta, transaction, linkrev, p1, p2):
         """Add a new revision to the store.
 
         Takes file data, dictionary of metadata, a transaction, linkrev,
         and parent nodes.
 
         Returns the node that was added.
 
         May no-op if a revision matching the supplied data is already stored.
         """
 
     def addrevision(
         revisiondata,
         transaction,
         linkrev,
         p1,
         p2,
         node=None,
         flags=0,
         cachedelta=None,
     ):
         """Add a new revision to the store and return its number.
 
         This is similar to ``add()`` except it operates at a lower level.
 
         The data passed in already contains a metadata header, if any.
 
         ``node`` and ``flags`` can be used to define the expected node and
         the flags to use with storage. ``flags`` is a bitwise value composed
         of the various ``REVISION_FLAG_*`` constants.
 
         ``add()`` is usually called when adding files from e.g. the working
         directory. ``addrevision()`` is often called by ``add()`` and for
         scenarios where revision data has already been computed, such as when
         applying raw data from a peer repo.
         """
 
     def addgroup(
         deltas,
         linkmapper,
         transaction,
         addrevisioncb=None,
         duplicaterevisioncb=None,
         maybemissingparents=False,
     ):
         """Process a series of deltas for storage.
 
         ``deltas`` is an iterable of 7-tuples of
         (node, p1, p2, linknode, deltabase, delta, flags) defining revisions
         to add.
 
         The ``delta`` field contains ``mpatch`` data to apply to a base
         revision, identified by ``deltabase``. The base node can be
         ``nullid``, in which case the header from the delta can be ignored
         and the delta used as the fulltext.
 
         ``alwayscache`` instructs the lower layers to cache the content of the
         newly added revision, even if it needs to be explicitly computed.
         This used to be the default when ``addrevisioncb`` was provided up to
         Mercurial 5.8.
 
         ``addrevisioncb`` should be called for each new rev as it is committed.
         ``duplicaterevisioncb`` should be called for all revs with a
         pre-existing node.
 
         ``maybemissingparents`` is a bool indicating whether the incoming
         data may reference parents/ancestor revisions that aren't present.
         This flag is set when receiving data into a "shallow" store that
         doesn't hold all history.
 
         Returns a list of nodes that were processed. A node will be in the list
         even if it existed in the store previously.
         """
 
     def censorrevision(tr, node, tombstone=b''):
         """Remove the content of a single revision.
 
         The specified ``node`` will have its content purged from storage.
         Future attempts to access the revision data for this node will
         result in failure.
 
         A ``tombstone`` message can optionally be stored. This message may be
         displayed to users when they attempt to access the missing revision
         data.
 
         Storage backends may have stored deltas against the previous content
         in this revision. As part of censoring a revision, these storage
         backends are expected to rewrite any internally stored deltas such
         that they no longer reference the deleted content.
         """
 
     def getstrippoint(minlink):
         """Find the minimum revision that must be stripped to strip a linkrev.
 
         Returns a 2-tuple containing the minimum revision number and a set
         of all revisions numbers that would be broken by this strip.
 
         TODO this is highly revlog centric and should be abstracted into
         a higher-level deletion API. ``repair.strip()`` relies on this.
         """
 
     def strip(minlink, transaction):
         """Remove storage of items starting at a linkrev.
 
         This uses ``getstrippoint()`` to determine the first node to remove.
         Then it effectively truncates storage for all revisions after that.
 
         TODO this is highly revlog centric and should be abstracted into a
         higher-level deletion API.
         """
 
 
 class ifilestorage(ifileindex, ifiledata, ifilemutation):
     """Complete storage interface for a single tracked file."""
 
     def files():
         """Obtain paths that are backing storage for this file.
 
         TODO this is used heavily by verify code and there should probably
         be a better API for that.
         """
 
     def storageinfo(
         exclusivefiles=False,
         sharedfiles=False,
         revisionscount=False,
         trackedsize=False,
         storedsize=False,
     ):
         """Obtain information about storage for this file's data.
 
         Returns a dict describing storage for this tracked path. The keys
         in the dict map to arguments of the same. The arguments are bools
         indicating whether to calculate and obtain that data.
 
         exclusivefiles
            Iterable of (vfs, path) describing files that are exclusively
            used to back storage for this tracked path.
 
         sharedfiles
            Iterable of (vfs, path) describing files that are used to back
            storage for this tracked path. Those files may also provide storage
            for other stored entities.
 
         revisionscount
            Number of revisions available for retrieval.
 
         trackedsize
            Total size in bytes of all tracked revisions. This is a sum of the
            length of the fulltext of all revisions.
 
         storedsize
            Total size in bytes used to store data for all tracked revisions.
            This is commonly less than ``trackedsize`` due to internal usage
            of deltas rather than fulltext revisions.
 
         Not all storage backends may support all queries are have a reasonable
         value to use. In that case, the value should be set to ``None`` and
         callers are expected to handle this special value.
         """
 
     def verifyintegrity(state):
         """Verifies the integrity of file storage.
 
         ``state`` is a dict holding state of the verifier process. It can be
         used to communicate data between invocations of multiple storage
         primitives.
 
         If individual revisions cannot have their revision content resolved,
         the method is expected to set the ``skipread`` key to a set of nodes
         that encountered problems.  If set, the method can also add the node(s)
         to ``safe_renamed`` in order to indicate nodes that may perform the
         rename checks with currently accessible data.
 
         The method yields objects conforming to the ``iverifyproblem``
         interface.
         """
 
 
 class idirs(interfaceutil.Interface):
     """Interface representing a collection of directories from paths.
 
     This interface is essentially a derived data structure representing
     directories from a collection of paths.
     """
 
     def addpath(path):
         """Add a path to the collection.
 
         All directories in the path will be added to the collection.
         """
 
     def delpath(path):
         """Remove a path from the collection.
 
         If the removal was the last path in a particular directory, the
         directory is removed from the collection.
         """
 
     def __iter__():
         """Iterate over the directories in this collection of paths."""
 
     def __contains__(path):
         """Whether a specific directory is in this collection."""
 
 
 class imanifestdict(interfaceutil.Interface):
     """Interface representing a manifest data structure.
 
     A manifest is effectively a dict mapping paths to entries. Each entry
     consists of a binary node and extra flags affecting that entry.
     """
 
     def __getitem__(path):
         """Returns the binary node value for a path in the manifest.
 
         Raises ``KeyError`` if the path does not exist in the manifest.
 
         Equivalent to ``self.find(path)[0]``.
         """
 
     def find(path):
         """Returns the entry for a path in the manifest.
 
         Returns a 2-tuple of (node, flags).
 
         Raises ``KeyError`` if the path does not exist in the manifest.
         """
 
     def __len__():
         """Return the number of entries in the manifest."""
 
     def __nonzero__():
         """Returns True if the manifest has entries, False otherwise."""
 
     __bool__ = __nonzero__
 
     def __setitem__(path, node):
         """Define the node value for a path in the manifest.
 
         If the path is already in the manifest, its flags will be copied to
         the new entry.
         """
 
     def __contains__(path):
         """Whether a path exists in the manifest."""
 
     def __delitem__(path):
         """Remove a path from the manifest.
 
         Raises ``KeyError`` if the path is not in the manifest.
         """
 
     def __iter__():
         """Iterate over paths in the manifest."""
 
     def iterkeys():
         """Iterate over paths in the manifest."""
 
     def keys():
         """Obtain a list of paths in the manifest."""
 
     def filesnotin(other, match=None):
         """Obtain the set of paths in this manifest but not in another.
 
         ``match`` is an optional matcher function to be applied to both
         manifests.
 
         Returns a set of paths.
         """
 
     def dirs():
         """Returns an object implementing the ``idirs`` interface."""
 
     def hasdir(dir):
         """Returns a bool indicating if a directory is in this manifest."""
 
     def walk(match):
         """Generator of paths in manifest satisfying a matcher.
 
         If the matcher has explicit files listed and they don't exist in
         the manifest, ``match.bad()`` is called for each missing file.
         """
 
     def diff(other, match=None, clean=False):
         """Find differences between this manifest and another.
 
         This manifest is compared to ``other``.
 
         If ``match`` is provided, the two manifests are filtered against this
         matcher and only entries satisfying the matcher are compared.
 
         If ``clean`` is True, unchanged files are included in the returned
         object.
 
         Returns a dict with paths as keys and values of 2-tuples of 2-tuples of
         the form ``((node1, flag1), (node2, flag2))`` where ``(node1, flag1)``
         represents the node and flags for this manifest and ``(node2, flag2)``
         are the same for the other manifest.
         """
 
     def setflag(path, flag):
         """Set the flag value for a given path.
 
         Raises ``KeyError`` if the path is not already in the manifest.
         """
 
     def get(path, default=None):
         """Obtain the node value for a path or a default value if missing."""
 
     def flags(path):
         """Return the flags value for a path (default: empty bytestring)."""
 
     def copy():
         """Return a copy of this manifest."""
 
     def items():
         """Returns an iterable of (path, node) for items in this manifest."""
 
     def iteritems():
         """Identical to items()."""
 
     def iterentries():
         """Returns an iterable of (path, node, flags) for this manifest.
 
         Similar to ``iteritems()`` except items are a 3-tuple and include
         flags.
         """
 
     def text():
         """Obtain the raw data representation for this manifest.
 
         Result is used to create a manifest revision.
         """
 
     def fastdelta(base, changes):
         """Obtain a delta between this manifest and another given changes.
 
         ``base`` in the raw data representation for another manifest.
 
         ``changes`` is an iterable of ``(path, to_delete)``.
 
         Returns a 2-tuple containing ``bytearray(self.text())`` and the
         delta between ``base`` and this manifest.
 
         If this manifest implementation can't support ``fastdelta()``,
         raise ``mercurial.manifest.FastdeltaUnavailable``.
         """
 
 
 class imanifestrevisionbase(interfaceutil.Interface):
     """Base interface representing a single revision of a manifest.
 
     Should not be used as a primary interface: should always be inherited
     as part of a larger interface.
     """
 
     def copy():
         """Obtain a copy of this manifest instance.
 
         Returns an object conforming to the ``imanifestrevisionwritable``
         interface. The instance will be associated with the same
         ``imanifestlog`` collection as this instance.
         """
 
     def read():
         """Obtain the parsed manifest data structure.
 
         The returned object conforms to the ``imanifestdict`` interface.
         """
 
 
 class imanifestrevisionstored(imanifestrevisionbase):
     """Interface representing a manifest revision committed to storage."""
 
     def node():
         """The binary node for this manifest."""
 
     parents = interfaceutil.Attribute(
         """List of binary nodes that are parents for this manifest revision."""
     )
 
     def readdelta(shallow=False):
         """Obtain the manifest data structure representing changes from parent.
 
         This manifest is compared to its 1st parent. A new manifest representing
         those differences is constructed.
 
         The returned object conforms to the ``imanifestdict`` interface.
         """
 
     def readfast(shallow=False):
         """Calls either ``read()`` or ``readdelta()``.
 
         The faster of the two options is called.
         """
 
     def find(key):
         """Calls self.read().find(key)``.
 
         Returns a 2-tuple of ``(node, flags)`` or raises ``KeyError``.
         """
 
 
 class imanifestrevisionwritable(imanifestrevisionbase):
     """Interface representing a manifest revision that can be committed."""
 
     def write(transaction, linkrev, p1node, p2node, added, removed, match=None):
         """Add this revision to storage.
 
         Takes a transaction object, the changeset revision number it will
         be associated with, its parent nodes, and lists of added and
         removed paths.
 
         If match is provided, storage can choose not to inspect or write out
         items that do not match. Storage is still required to be able to provide
         the full manifest in the future for any directories written (these
         manifests should not be "narrowed on disk").
 
         Returns the binary node of the created revision.
         """
 
 
 class imanifeststorage(interfaceutil.Interface):
     """Storage interface for manifest data."""
 
     nodeconstants = interfaceutil.Attribute(
         """nodeconstants used by the current repository."""
     )
 
     tree = interfaceutil.Attribute(
         """The path to the directory this manifest tracks.
 
         The empty bytestring represents the root manifest.
         """
     )
 
     index = interfaceutil.Attribute(
         """An ``ifilerevisionssequence`` instance."""
     )
 
     opener = interfaceutil.Attribute(
         """VFS opener to use to access underlying files used for storage.
 
         TODO this is revlog specific and should not be exposed.
         """
     )
 
     _generaldelta = interfaceutil.Attribute(
         """Whether generaldelta storage is being used.
 
         TODO this is revlog specific and should not be exposed.
         """
     )
 
     fulltextcache = interfaceutil.Attribute(
         """Dict with cache of fulltexts.
 
         TODO this doesn't feel appropriate for the storage interface.
         """
     )
 
     def __len__():
         """Obtain the number of revisions stored for this manifest."""
 
     def __iter__():
         """Iterate over revision numbers for this manifest."""
 
     def rev(node):
         """Obtain the revision number given a binary node.
 
         Raises ``error.LookupError`` if the node is not known.
         """
 
     def node(rev):
         """Obtain the node value given a revision number.
 
         Raises ``error.LookupError`` if the revision is not known.
         """
 
     def lookup(value):
         """Attempt to resolve a value to a node.
 
         Value can be a binary node, hex node, revision number, or a bytes
         that can be converted to an integer.
 
         Raises ``error.LookupError`` if a ndoe could not be resolved.
         """
 
     def parents(node):
         """Returns a 2-tuple of parent nodes for a node.
 
         Values will be ``nullid`` if the parent is empty.
         """
 
     def parentrevs(rev):
         """Like parents() but operates on revision numbers."""
 
     def linkrev(rev):
         """Obtain the changeset revision number a revision is linked to."""
 
     def revision(node, _df=None):
         """Obtain fulltext data for a node."""
 
     def rawdata(node, _df=None):
         """Obtain raw data for a node."""
 
     def revdiff(rev1, rev2):
         """Obtain a delta between two revision numbers.
 
         The returned data is the result of ``bdiff.bdiff()`` on the raw
         revision data.
         """
 
     def cmp(node, fulltext):
         """Compare fulltext to another revision.
 
         Returns True if the fulltext is different from what is stored.
         """
 
     def emitrevisions(
         nodes,
         nodesorder=None,
         revisiondata=False,
         assumehaveparentrevisions=False,
     ):
         """Produce ``irevisiondelta`` describing revisions.
 
         See the documentation for ``ifiledata`` for more.
         """
 
     def addgroup(
         deltas,
         linkmapper,
         transaction,
         addrevisioncb=None,
         duplicaterevisioncb=None,
     ):
         """Process a series of deltas for storage.
 
         See the documentation in ``ifilemutation`` for more.
         """
 
     def rawsize(rev):
         """Obtain the size of tracked data.
 
         Is equivalent to ``len(m.rawdata(node))``.
 
         TODO this method is only used by upgrade code and may be removed.
         """
 
     def getstrippoint(minlink):
         """Find minimum revision that must be stripped to strip a linkrev.
 
         See the documentation in ``ifilemutation`` for more.
         """
 
     def strip(minlink, transaction):
         """Remove storage of items starting at a linkrev.
 
         See the documentation in ``ifilemutation`` for more.
         """
 
     def checksize():
         """Obtain the expected sizes of backing files.
 
         TODO this is used by verify and it should not be part of the interface.
         """
 
     def files():
         """Obtain paths that are backing storage for this manifest.
 
         TODO this is used by verify and there should probably be a better API
         for this functionality.
         """
 
     def deltaparent(rev):
         """Obtain the revision that a revision is delta'd against.
 
         TODO delta encoding is an implementation detail of storage and should
         not be exposed to the storage interface.
         """
 
     def clone(tr, dest, **kwargs):
         """Clone this instance to another."""
 
     def clearcaches(clear_persisted_data=False):
         """Clear any caches associated with this instance."""
 
     def dirlog(d):
         """Obtain a manifest storage instance for a tree."""
 
     def add(
         m, transaction, link, p1, p2, added, removed, readtree=None, match=None
     ):
         """Add a revision to storage.
 
         ``m`` is an object conforming to ``imanifestdict``.
 
         ``link`` is the linkrev revision number.
 
         ``p1`` and ``p2`` are the parent revision numbers.
 
         ``added`` and ``removed`` are iterables of added and removed paths,
         respectively.
 
         ``readtree`` is a function that can be used to read the child tree(s)
         when recursively writing the full tree structure when using
         treemanifets.
 
         ``match`` is a matcher that can be used to hint to storage that not all
         paths must be inspected; this is an optimization and can be safely
         ignored. Note that the storage must still be able to reproduce a full
         manifest including files that did not match.
         """
 
     def storageinfo(
         exclusivefiles=False,
         sharedfiles=False,
         revisionscount=False,
         trackedsize=False,
         storedsize=False,
     ):
         """Obtain information about storage for this manifest's data.
 
         See ``ifilestorage.storageinfo()`` for a description of this method.
         This one behaves the same way, except for manifest data.
         """
 
     def get_revlog():
         """return an actual revlog instance if any
 
         This exist because a lot of code leverage the fact the underlying
         storage is a revlog for optimization, so giving simple way to access
         the revlog instance helps such code.
         """
 
 
 class imanifestlog(interfaceutil.Interface):
     """Interface representing a collection of manifest snapshots.
 
     Represents the root manifest in a repository.
 
     Also serves as a means to access nested tree manifests and to cache
     tree manifests.
     """
 
     nodeconstants = interfaceutil.Attribute(
         """nodeconstants used by the current repository."""
     )
 
     def __getitem__(node):
         """Obtain a manifest instance for a given binary node.
 
         Equivalent to calling ``self.get('', node)``.
 
         The returned object conforms to the ``imanifestrevisionstored``
         interface.
         """
 
     def get(tree, node, verify=True):
         """Retrieve the manifest instance for a given directory and binary node.
 
         ``node`` always refers to the node of the root manifest (which will be
         the only manifest if flat manifests are being used).
 
         If ``tree`` is the empty string, the root manifest is returned.
         Otherwise the manifest for the specified directory will be returned
         (requires tree manifests).
 
         If ``verify`` is True, ``LookupError`` is raised if the node is not
         known.
 
         The returned object conforms to the ``imanifestrevisionstored``
         interface.
         """
 
     def getstorage(tree):
         """Retrieve an interface to storage for a particular tree.
 
         If ``tree`` is the empty bytestring, storage for the root manifest will
         be returned. Otherwise storage for a tree manifest is returned.
 
         TODO formalize interface for returned object.
         """
 
     def clearcaches():
         """Clear caches associated with this collection."""
 
     def rev(node):
         """Obtain the revision number for a binary node.
 
         Raises ``error.LookupError`` if the node is not known.
         """
 
     def update_caches(transaction):
         """update whatever cache are relevant for the used storage."""
 
 
 class ilocalrepositoryfilestorage(interfaceutil.Interface):
     """Local repository sub-interface providing access to tracked file storage.
 
     This interface defines how a repository accesses storage for a single
     tracked file path.
     """
 
     def file(f):
         """Obtain a filelog for a tracked path.
 
         The returned type conforms to the ``ifilestorage`` interface.
         """
 
 
 class ilocalrepositorymain(interfaceutil.Interface):
     """Main interface for local repositories.
 
     This currently captures the reality of things - not how things should be.
     """
 
     nodeconstants = interfaceutil.Attribute(
         """Constant nodes matching the hash function used by the repository."""
     )
     nullid = interfaceutil.Attribute(
         """null revision for the hash function used by the repository."""
     )
 
     supported = interfaceutil.Attribute(
         """Set of requirements that this repo is capable of opening."""
     )
 
     requirements = interfaceutil.Attribute(
         """Set of requirements this repo uses."""
     )
 
     features = interfaceutil.Attribute(
         """Set of "features" this repository supports.
 
         A "feature" is a loosely-defined term. It can refer to a feature
         in the classical sense or can describe an implementation detail
         of the repository. For example, a ``readonly`` feature may denote
         the repository as read-only. Or a ``revlogfilestore`` feature may
         denote that the repository is using revlogs for file storage.
 
         The intent of features is to provide a machine-queryable mechanism
         for repo consumers to test for various repository characteristics.
 
         Features are similar to ``requirements``. The main difference is that
         requirements are stored on-disk and represent requirements to open the
         repository. Features are more run-time capabilities of the repository
         and more granular capabilities (which may be derived from requirements).
         """
     )
 
     filtername = interfaceutil.Attribute(
         """Name of the repoview that is active on this repo."""
     )
 
     vfs_map = interfaceutil.Attribute(
         """a bytes-key → vfs mapping used by transaction and others"""
     )
 
     wvfs = interfaceutil.Attribute(
         """VFS used to access the working directory."""
     )
 
     vfs = interfaceutil.Attribute(
         """VFS rooted at the .hg directory.
 
         Used to access repository data not in the store.
         """
     )
 
     svfs = interfaceutil.Attribute(
         """VFS rooted at the store.
 
         Used to access repository data in the store. Typically .hg/store.
         But can point elsewhere if the store is shared.
         """
     )
 
     root = interfaceutil.Attribute(
         """Path to the root of the working directory."""
     )
 
     path = interfaceutil.Attribute("""Path to the .hg directory.""")
 
     origroot = interfaceutil.Attribute(
         """The filesystem path that was used to construct the repo."""
     )
 
     auditor = interfaceutil.Attribute(
         """A pathauditor for the working directory.
 
         This checks if a path refers to a nested repository.
 
         Operates on the filesystem.
         """
     )
 
     nofsauditor = interfaceutil.Attribute(
         """A pathauditor for the working directory.
 
         This is like ``auditor`` except it doesn't do filesystem checks.
         """
     )
 
     baseui = interfaceutil.Attribute(
         """Original ui instance passed into constructor."""
     )
 
     ui = interfaceutil.Attribute("""Main ui instance for this instance.""")
 
     sharedpath = interfaceutil.Attribute(
         """Path to the .hg directory of the repo this repo was shared from."""
     )
 
     store = interfaceutil.Attribute("""A store instance.""")
 
     spath = interfaceutil.Attribute("""Path to the store.""")
 
     sjoin = interfaceutil.Attribute("""Alias to self.store.join.""")
 
     cachevfs = interfaceutil.Attribute(
         """A VFS used to access the cache directory.
 
         Typically .hg/cache.
         """
     )
 
     wcachevfs = interfaceutil.Attribute(
         """A VFS used to access the cache directory dedicated to working copy
 
         Typically .hg/wcache.
         """
     )
 
     filteredrevcache = interfaceutil.Attribute(
         """Holds sets of revisions to be filtered."""
     )
 
     names = interfaceutil.Attribute("""A ``namespaces`` instance.""")
 
     filecopiesmode = interfaceutil.Attribute(
         """The way files copies should be dealt with in this repo."""
     )
 
     def close():
         """Close the handle on this repository."""
 
     def peer(path=None):
         """Obtain an object conforming to the ``peer`` interface."""
 
     def unfiltered():
         """Obtain an unfiltered/raw view of this repo."""
 
     def filtered(name, visibilityexceptions=None):
         """Obtain a named view of this repository."""
 
     obsstore = interfaceutil.Attribute("""A store of obsolescence data.""")
 
     changelog = interfaceutil.Attribute("""A handle on the changelog revlog.""")
 
     manifestlog = interfaceutil.Attribute(
         """An instance conforming to the ``imanifestlog`` interface.
 
         Provides access to manifests for the repository.
         """
     )
 
     dirstate = interfaceutil.Attribute("""Working directory state.""")
 
     narrowpats = interfaceutil.Attribute(
         """Matcher patterns for this repository's narrowspec."""
     )
 
     def narrowmatch(match=None, includeexact=False):
         """Obtain a matcher for the narrowspec."""
 
     def setnarrowpats(newincludes, newexcludes):
         """Define the narrowspec for this repository."""
 
     def __getitem__(changeid):
         """Try to resolve a changectx."""
 
     def __contains__(changeid):
         """Whether a changeset exists."""
 
     def __nonzero__():
         """Always returns True."""
         return True
 
     __bool__ = __nonzero__
 
     def __len__():
         """Returns the number of changesets in the repo."""
 
     def __iter__():
         """Iterate over revisions in the changelog."""
 
     def revs(expr, *args):
         """Evaluate a revset.
 
         Emits revisions.
         """
 
     def set(expr, *args):
         """Evaluate a revset.
 
         Emits changectx instances.
         """
 
     def anyrevs(specs, user=False, localalias=None):
         """Find revisions matching one of the given revsets."""
 
     def url():
         """Returns a string representing the location of this repo."""
 
     def hook(name, throw=False, **args):
         """Call a hook."""
 
     def tags():
         """Return a mapping of tag to node."""
 
     def tagtype(tagname):
         """Return the type of a given tag."""
 
     def tagslist():
         """Return a list of tags ordered by revision."""
 
     def nodetags(node):
         """Return the tags associated with a node."""
 
     def nodebookmarks(node):
         """Return the list of bookmarks pointing to the specified node."""
 
     def branchmap():
         """Return a mapping of branch to heads in that branch."""
 
     def revbranchcache():
         pass
 
     def register_changeset(rev, changelogrevision):
         """Extension point for caches for new nodes.
 
         Multiple consumers are expected to need parts of the changelogrevision,
         so it is provided as optimization to avoid duplicate lookups. A simple
         cache would be fragile when other revisions are accessed, too."""
         pass
 
     def branchtip(branchtip, ignoremissing=False):
         """Return the tip node for a given branch."""
 
     def lookup(key):
         """Resolve the node for a revision."""
 
     def lookupbranch(key):
         """Look up the branch name of the given revision or branch name."""
 
     def known(nodes):
         """Determine whether a series of nodes is known.
 
         Returns a list of bools.
         """
 
     def local():
         """Whether the repository is local."""
         return True
 
     def publishing():
         """Whether the repository is a publishing repository."""
 
     def cancopy():
         pass
 
     def shared():
         """The type of shared repository or None."""
 
     def wjoin(f, *insidef):
         """Calls self.vfs.reljoin(self.root, f, *insidef)"""
 
     def setparents(p1, p2):
         """Set the parent nodes of the working directory."""
 
     def filectx(path, changeid=None, fileid=None):
         """Obtain a filectx for the given file revision."""
 
     def getcwd():
         """Obtain the current working directory from the dirstate."""
 
     def pathto(f, cwd=None):
         """Obtain the relative path to a file."""
 
     def adddatafilter(name, fltr):
         pass
 
     def wread(filename):
         """Read a file from wvfs, using data filters."""
 
     def wwrite(filename, data, flags, backgroundclose=False, **kwargs):
         """Write data to a file in the wvfs, using data filters."""
 
     def wwritedata(filename, data):
         """Resolve data for writing to the wvfs, using data filters."""
 
     def currenttransaction():
         """Obtain the current transaction instance or None."""
 
     def transaction(desc, report=None):
         """Open a new transaction to write to the repository."""
 
     def undofiles():
         """Returns a list of (vfs, path) for files to undo transactions."""
 
     def recover():
         """Roll back an interrupted transaction."""
 
     def rollback(dryrun=False, force=False):
         """Undo the last transaction.
 
         DANGEROUS.
         """
 
     def updatecaches(tr=None, full=False, caches=None):
         """Warm repo caches."""
 
     def invalidatecaches():
         """Invalidate cached data due to the repository mutating."""
 
     def invalidatevolatilesets():
         pass
 
     def invalidatedirstate():
         """Invalidate the dirstate."""
 
     def invalidate(clearfilecache=False):
         pass
 
     def invalidateall():
         pass
 
     def lock(wait=True):
         """Lock the repository store and return a lock instance."""
 
     def currentlock():
         """Return the lock if it's held or None."""
 
     def wlock(wait=True):
         """Lock the non-store parts of the repository."""
 
     def currentwlock():
         """Return the wlock if it's held or None."""
 
     def checkcommitpatterns(wctx, match, status, fail):
         pass
 
     def commit(
         text=b'',
         user=None,
         date=None,
         match=None,
         force=False,
         editor=False,
         extra=None,
     ):
         """Add a new revision to the repository."""
 
     def commitctx(ctx, error=False, origctx=None):
         """Commit a commitctx instance to the repository."""
 
     def destroying():
         """Inform the repository that nodes are about to be destroyed."""
 
     def destroyed():
         """Inform the repository that nodes have been destroyed."""
 
     def status(
         node1=b'.',
         node2=None,
         match=None,
         ignored=False,
         clean=False,
         unknown=False,
         listsubrepos=False,
     ):
         """Convenience method to call repo[x].status()."""
 
     def addpostdsstatus(ps):
         pass
 
     def postdsstatus():
         pass
 
     def clearpostdsstatus():
         pass
 
     def heads(start=None):
         """Obtain list of nodes that are DAG heads."""
 
     def branchheads(branch=None, start=None, closed=False):
         pass
 
     def branches(nodes):
         pass
 
     def between(pairs):
         pass
 
     def checkpush(pushop):
         pass
 
     prepushoutgoinghooks = interfaceutil.Attribute("""util.hooks instance.""")
 
     def pushkey(namespace, key, old, new):
         pass
 
     def listkeys(namespace):
         pass
 
     def debugwireargs(one, two, three=None, four=None, five=None):
         pass
 
     def savecommitmessage(text):
         pass
 
     def register_sidedata_computer(
         kind, category, keys, computer, flags, replace=False
     ):
         pass
 
     def register_wanted_sidedata(category):
         pass
 
 
 class completelocalrepository(
     ilocalrepositorymain, ilocalrepositoryfilestorage
 ):
     """Complete interface for a local repository."""
 
 
 class iwireprotocolcommandcacher(interfaceutil.Interface):
     """Represents a caching backend for wire protocol commands.
 
     Wire protocol version 2 supports transparent caching of many commands.
     To leverage this caching, servers can activate objects that cache
     command responses. Objects handle both cache writing and reading.
     This interface defines how that response caching mechanism works.
 
     Wire protocol version 2 commands emit a series of objects that are
     serialized and sent to the client. The caching layer exists between
     the invocation of the command function and the sending of its output
     objects to an output layer.
 
     Instances of this interface represent a binding to a cache that
     can serve a response (in place of calling a command function) and/or
     write responses to a cache for subsequent use.
 
     When a command request arrives, the following happens with regards
     to this interface:
 
     1. The server determines whether the command request is cacheable.
     2. If it is, an instance of this interface is spawned.
     3. The cacher is activated in a context manager (``__enter__`` is called).
     4. A cache *key* for that request is derived. This will call the
        instance's ``adjustcachekeystate()`` method so the derivation
        can be influenced.
     5. The cacher is informed of the derived cache key via a call to
        ``setcachekey()``.
     6. The cacher's ``lookup()`` method is called to test for presence of
        the derived key in the cache.
     7. If ``lookup()`` returns a hit, that cached result is used in place
        of invoking the command function. ``__exit__`` is called and the instance
        is discarded.
     8. The command function is invoked.
     9. ``onobject()`` is called for each object emitted by the command
        function.
     10. After the final object is seen, ``onfinished()`` is called.
     11. ``__exit__`` is called to signal the end of use of the instance.
 
     Cache *key* derivation can be influenced by the instance.
 
     Cache keys are initially derived by a deterministic representation of
     the command request. This includes the command name, arguments, protocol
     version, etc. This initial key derivation is performed by CBOR-encoding a
     data structure and feeding that output into a hasher.
 
     Instances of this interface can influence this initial key derivation
     via ``adjustcachekeystate()``.
 
     The instance is informed of the derived cache key via a call to
     ``setcachekey()``. The instance must store the key locally so it can
     be consulted on subsequent operations that may require it.
 
     When constructed, the instance has access to a callable that can be used
     for encoding response objects. This callable receives as its single
     argument an object emitted by a command function. It returns an iterable
     of bytes chunks representing the encoded object. Unless the cacher is
     caching native Python objects in memory or has a way of reconstructing
     the original Python objects, implementations typically call this function
     to produce bytes from the output objects and then store those bytes in
     the cache. When it comes time to re-emit those bytes, they are wrapped
     in a ``wireprototypes.encodedresponse`` instance to tell the output
     layer that they are pre-encoded.
 
     When receiving the objects emitted by the command function, instances
     can choose what to do with those objects. The simplest thing to do is
     re-emit the original objects. They will be forwarded to the output
     layer and will be processed as if the cacher did not exist.
 
     Implementations could also choose to not emit objects - instead locally
     buffering objects or their encoded representation. They could then emit
     a single "coalesced" object when ``onfinished()`` is called. In
     this way, the implementation would function as a filtering layer of
     sorts.
 
     When caching objects, typically the encoded form of the object will
     be stored. Keep in mind that if the original object is forwarded to
     the output layer, it will need to be encoded there as well. For large
     output, this redundant encoding could add overhead. Implementations
     could wrap the encoded object data in ``wireprototypes.encodedresponse``
     instances to avoid this overhead.
     """
 
     def __enter__():
         """Marks the instance as active.
 
         Should return self.
         """
 
     def __exit__(exctype, excvalue, exctb):
         """Called when cacher is no longer used.
 
         This can be used by implementations to perform cleanup actions (e.g.
         disconnecting network sockets, aborting a partially cached response.
         """
 
     def adjustcachekeystate(state):
         """Influences cache key derivation by adjusting state to derive key.
 
         A dict defining the state used to derive the cache key is passed.
 
         Implementations can modify this dict to record additional state that
         is wanted to influence key derivation.
 
         Implementations are *highly* encouraged to not modify or delete
         existing keys.
         """
 
     def setcachekey(key):
         """Record the derived cache key for this request.
 
         Instances may mutate the key for internal usage, as desired. e.g.
         instances may wish to prepend the repo name, introduce path
         components for filesystem or URL addressing, etc. Behavior is up to
         the cache.
 
         Returns a bool indicating if the request is cacheable by this
         instance.
         """
 
     def lookup():
         """Attempt to resolve an entry in the cache.
 
         The instance is instructed to look for the cache key that it was
         informed about via the call to ``setcachekey()``.
 
         If there's no cache hit or the cacher doesn't wish to use the cached
         entry, ``None`` should be returned.
 
         Else, a dict defining the cached result should be returned. The
         dict may have the following keys:
 
         objs
            An iterable of objects that should be sent to the client. That
            iterable of objects is expected to be what the command function
            would return if invoked or an equivalent representation thereof.
         """
 
     def onobject(obj):
         """Called when a new object is emitted from the command function.
 
         Receives as its argument the object that was emitted from the
         command function.
 
         This method returns an iterator of objects to forward to the output
         layer. The easiest implementation is a generator that just
         ``yield obj``.
         """
 
     def onfinished():
         """Called after all objects have been emitted from the command function.
 
         Implementations should return an iterator of objects to forward to
         the output layer.
 
         This method can be a generator.
         """