wireprotov2: expose rich arguments metadata

Now that we internally store rich metadata about arguments, it makes
sense to make that metadata available to the client. This will allow
clients to validate outgoing command requests before they are sent
over the wire.

Strictly speaking, we should bump the wire protocol version for this
change since it is backwards incompatible. But no client-side code
touches the arguments map and I don't want to incur the work.

Differential Revision: https://phab.mercurial-scm.org/D4618

mercurial/help/internals/wireprotocolv2.txt

@@ -1,445 +1,457 b''
 **Experimental and under active development**
 
 This section documents the wire protocol commands exposed to transports
 using the frame-based protocol. The set of commands exposed through
 these transports is distinct from the set of commands exposed to legacy
 transports.
 
 The frame-based protocol uses CBOR to encode command execution requests.
 All command arguments must be mapped to a specific or set of CBOR data
 types.
 
 The response to many commands is also CBOR. There is no common response
 format: each command defines its own response format.
 
 TODOs
 =====
 
 * Add "node namespace" support to each command. In order to support
   SHA-1 hash transition, we want servers to be able to expose different
   "node namespaces" for the same data. Every command operating on nodes
   should specify which "node namespace" it is operating on and responses
   should encode the "node namespace" accordingly.
 
 Commands
 ========
 
 The sections below detail all commands available to wire protocol version
 2.
 
 branchmap
 ---------
 
 Obtain heads in named branches.
 
 Receives no arguments.
 
 The response is a map with bytestring keys defining the branch name.
 Values are arrays of bytestring defining raw changeset nodes.
 
 capabilities
 ------------
 
 Obtain the server's capabilities.
 
 Receives no arguments.
 
 This command is typically called only as part of the handshake during
 initial connection establishment.
 
 The response is a map with bytestring keys defining server information.
 
 The defined keys are:
 
 commands
    A map defining available wire protocol commands on this server.
 
    Keys in the map are the names of commands that can be invoked. Values
    are maps defining information about that command. The bytestring keys
    are:
 
       args
-         A map of argument names and their expected types.
+         (map) Describes arguments accepted by the command.
+
+         Keys are bytestrings denoting the argument name.
+
+         Values are maps describing the argument. The map has the following
+         bytestring keys:
 
-         Types are defined as a representative value for the expected type.
-         e.g. an argument expecting a boolean type will have its value
-         set to true. An integer type will have its value set to 42. The
-         actual values are arbitrary and may not have meaning.
+         default
+            (varied) The default value for this argument if not specified. Only
+            present if ``required`` is not true.
+
+         required
+            (boolean) Whether the argument must be specified. Failure to send
+            required arguments will result in an error executing the command.
+
+         type
+            (bytestring) The type of the argument. e.g. ``bytes`` or ``bool``.
+
       permissions
          An array of permissions required to execute this command.
 
 compression
    An array of maps defining available compression format support.
 
    The array is sorted from most preferred to least preferred.
 
    Each entry has the following bytestring keys:
 
       name
          Name of the compression engine. e.g. ``zstd`` or ``zlib``.
 
 framingmediatypes
    An array of bytestrings defining the supported framing protocol
    media types. Servers will not accept media types not in this list.
 
 pathfilterprefixes
    (set of bytestring) Matcher prefixes that are recognized when performing
    path filtering. Specifying a path filter whose type/prefix does not
    match one in this set will likely be rejected by the server.
 
 rawrepoformats
    An array of storage formats the repository is using. This set of
    requirements can be used to determine whether a client can read a
    *raw* copy of file data available.
 
 changesetdata
 -------------
 
 Obtain various data related to changesets.
 
 The command accepts the following arguments:
 
 noderange
    (array of arrays of bytestrings) An array of 2 elements, each being an
    array of node bytestrings. The first array denotes the changelog revisions
    that are already known to the client. The second array denotes the changelog
    revision DAG heads to fetch. The argument essentially defines a DAG range
    bounded by root and head nodes to fetch.
 
    The roots array may be empty. The heads array must be defined.
 
 nodes
    (array of bytestrings) Changelog revisions to request explicitly.
 
 fields
    (set of bytestring) Which data associated with changelog revisions to
    fetch. The following values are recognized:
 
    bookmarks
       Bookmarks associated with a revision.
 
    parents
       Parent revisions.
 
    phase
       The phase state of a revision.
 
    revision
       The raw, revision data for the changelog entry. The hash of this data
       will match the revision's node value.
 
 The server resolves the set of revisions relevant to the request by taking
 the union of the ``noderange`` and ``nodes`` arguments. At least one of these
 arguments must be defined.
 
 The response bytestream starts with a CBOR map describing the data that follows.
 This map has the following bytestring keys:
 
 totalitems
    (unsigned integer) Total number of changelog revisions whose data is being
    transferred. This maps to the set of revisions in the requested node
    range, not the total number of records that follow (see below for why).
 
 Following the map header is a series of 0 or more CBOR values. If values
 are present, the first value will always be a map describing a single changeset
 revision. If revision data is requested, the raw revision data (encoded as
 a CBOR bytestring) will follow the map describing it. Otherwise, another CBOR
 map describing the next changeset revision will occur.
 
 Each map has the following bytestring keys:
 
 node
    (bytestring) The node value for this revision. This is the SHA-1 hash of
    the raw revision data.
 
 bookmarks (optional)
    (array of bytestrings) Bookmarks attached to this revision. Only present
    if ``bookmarks`` data is being requested and the revision has bookmarks
    attached.
 
 parents (optional)
    (array of bytestrings) The nodes representing the parent revisions of this
    revision. Only present if ``parents`` data is being requested.
 
 phase (optional)
    (bytestring) The phase that a revision is in. Recognized values are
    ``secret``, ``draft``, and ``public``. Only present if ``phase`` data
    is being requested.
 
 revisionsize (optional)
    (unsigned integer) Indicates the size of raw revision data that follows this
    map. The following data contains a serialized form of the changeset data,
    including the author, date, commit message, set of changed files, manifest
    node, and other metadata.
 
    Only present if ``revision`` data was requested and the data follows this
    map.
 
 If nodes are requested via ``noderange``, they will be emitted in DAG order,
 parents always before children.
 
 If nodes are requested via ``nodes``, they will be emitted in requested order.
 
 Nodes from ``nodes`` are emitted before nodes from ``noderange``.
 
 The set of changeset revisions emitted may not match the exact set of
 changesets requested. Furthermore, the set of keys present on each
 map may vary. This is to facilitate emitting changeset updates as well
 as new revisions.
 
 For example, if the request wants ``phase`` and ``revision`` data,
 the response may contain entries for each changeset in the common nodes
 set with the ``phase`` key and without the ``revision`` key in order
 to reflect a phase-only update.
 
 TODO support different revision selection mechanisms (e.g. non-public, specific
 revisions)
 TODO support different hash "namespaces" for revisions (e.g. sha-1 versus other)
 TODO support emitting obsolescence data
 TODO support filtering based on relevant paths (narrow clone)
 TODO support depth limiting
 TODO support hgtagsfnodes cache / tags data
 TODO support branch heads cache
 
 filedata
 --------
 
 Obtain various data related to an individual tracked file.
 
 The command accepts the following arguments:
 
 fields
    (set of bytestring) Which data associated with a file to fetch.
    The following values are recognized:
 
    parents
       Parent nodes for the revision.
 
    revision
       The raw revision data for a file.
 
 haveparents
    (bool) Whether the client has the parent revisions of all requested
    nodes. If set, the server may emit revision data as deltas against
    any parent revision. If not set, the server MUST only emit deltas for
    revisions previously emitted by this command.
 
    False is assumed in the absence of any value.
 
 nodes
    (array of bytestrings) File nodes whose data to retrieve.
 
 path
    (bytestring) Path of the tracked file whose data to retrieve.
 
 TODO allow specifying revisions via alternate means (such as from
 changeset revisions or ranges)
 
 The response bytestream starts with a CBOR map describing the data that
 follows. It has the following bytestream keys:
 
 totalitems
    (unsigned integer) Total number of file revisions whose data is
    being returned.
 
 Following the header map is a series of 0 or more CBOR values. The first
 value is always a map describing a file revision. If this map has the
 ``deltasize`` or ``revisionsize`` keys, a bytestring containing the delta
 or revision, respectively, will immediately follow the map. Otherwise
 the next value will be a map describing the next file revision.
 
 Each map has the following bytestring keys:
 
 node
    (bytestring) The node of the file revision whose data is represented.
 
 deltabasenode
    (bytestring) Node of the file revision the following delta is against.
 
    Only present if the ``revision`` field is requested and delta data
    follows this map.
 
 deltasize
    (unsigned integer) The size of the delta data that follows this map.
 
    Only present if the ``revision`` field is requested and delta data
    follows this map.
 
 parents
    (array of bytestring) The nodes of the parents of this file revision.
 
    Only present if the ``parents`` field is requested.
 
 revisionsize
    (unsigned integer) The size of the fulltext revision data that follows
    this map.
 
    Only present if the ``revision`` field is requested and fulltext revision
    data follows this map.
 
 When ``revision`` data is requested, the server chooses to emit either fulltext
 revision data or a delta. What the server decides can be inferred by looking
 for the presence of the ``deltasize`` or ``revisionsize`` keys in the map.
 Servers MUST NOT define both keys.
 
 heads
 -----
 
 Obtain DAG heads in the repository.
 
 The command accepts the following arguments:
 
 publiconly (optional)
    (boolean) If set, operate on the DAG for public phase changesets only.
    Non-public (i.e. draft) phase DAG heads will not be returned.
 
 The response is a CBOR array of bytestrings defining changeset nodes
 of DAG heads. The array can be empty if the repository is empty or no
 changesets satisfied the request.
 
 TODO consider exposing phase of heads in response
 
 known
 -----
 
 Determine whether a series of changeset nodes is known to the server.
 
 The command accepts the following arguments:
 
 nodes
    (array of bytestrings) List of changeset nodes whose presence to
    query.
 
 The response is a bytestring where each byte contains a 0 or 1 for the
 corresponding requested node at the same index.
 
 TODO use a bit array for even more compact response
 
 listkeys
 --------
 
 List values in a specified ``pushkey`` namespace.
 
 The command receives the following arguments:
 
 namespace
    (bytestring) Pushkey namespace to query.
 
 The response is a map with bytestring keys and values.
 
 TODO consider using binary to represent nodes in certain pushkey namespaces.
 
 lookup
 ------
 
 Try to resolve a value to a changeset revision.
 
 Unlike ``known`` which operates on changeset nodes, lookup operates on
 node fragments and other names that a user may use.
 
 The command receives the following arguments:
 
 key
    (bytestring) Value to try to resolve.
 
 On success, returns a bytestring containing the resolved node.
 
 manifestdata
 ------------
 
 Obtain various data related to manifests (which are lists of files in
 a revision).
 
 The command accepts the following arguments:
 
 fields
    (set of bytestring) Which data associated with manifests to fetch.
    The following values are recognized:
 
    parents
       Parent nodes for the manifest.
 
    revision
       The raw revision data for the manifest.
 
 haveparents
    (bool) Whether the client has the parent revisions of all requested
    nodes. If set, the server may emit revision data as deltas against
    any parent revision. If not set, the server MUST only emit deltas for
    revisions previously emitted by this command.
 
    False is assumed in the absence of any value.
 
 nodes
    (array of bytestring) Manifest nodes whose data to retrieve.
 
 tree
    (bytestring) Path to manifest to retrieve. The empty bytestring represents
    the root manifest. All other values represent directories/trees within
    the repository.
 
 TODO allow specifying revisions via alternate means (such as from changeset
 revisions or ranges)
 TODO consider recursive expansion of manifests (with path filtering for
 narrow use cases)
 
 The response bytestream starts with a CBOR map describing the data that
 follows. It has the following bytestring keys:
 
 totalitems
    (unsigned integer) Total number of manifest revisions whose data is
    being returned.
 
 Following the header map is a series of 0 or more CBOR values. The first
 value is always a map describing a manifest revision. If this map has the
 ``deltasize`` or ``revisionsize`` keys, a bytestring containing the delta
 or revision, respectively, will immediately follow the map. Otherwise
 the next value will be a map describing the next manifest revision.
 
 Each map has the following bytestring keys:
 
 node
    (bytestring) The node of the manifest revision whose data is represented.
 
 deltabasenode
    (bytestring) The node that the delta representation of this revision is
    computed against. Only present if the ``revision`` field is requested and
    a delta is being emitted.
 
 deltasize
    (unsigned integer) The size of the delta data that follows this map.
    Only present if the ``revision`` field is requested and a delta is
    being emitted.
 
 parents
    (array of bytestring) The nodes of the parents of this manifest revision.
    Only present if the ``parents`` field is requested.
 
 revisionsize
    (unsigned integer) The size of the fulltext revision data that follows
    this map. Only present if the ``revision`` field is requested and a fulltext
    revision is being emitted.
 
 When ``revision`` data is requested, the server chooses to emit either fulltext
 revision data or a delta. What the server decides can be inferred by looking
 for the presence of the ``deltasize`` or ``revisionsize`` keys in the map.
 Servers MUST NOT define both keys.
 
 pushkey
 -------
 
 Set a value using the ``pushkey`` protocol.
 
 The command receives the following arguments:
 
 namespace
    (bytestring) Pushkey namespace to operate on.
 key
    (bytestring) The pushkey key to set.
 old
    (bytestring) Old value for this key.
 new
    (bytestring) New value for this key.
 
 TODO consider using binary to represent nodes is certain pushkey namespaces.
 TODO better define response type and meaning.

mercurial/wireprotov2server.py

@@ -1,1060 +1,1071 b''
 # Copyright 21 May 2005 - (c) 2005 Jake Edge <[e-mail]>
 # Copyright 2005-2007 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 from __future__ import absolute_import
 
 import contextlib
 
 from .i18n import _
 from .node import (
     hex,
     nullid,
     nullrev,
 )
 from . import (
     changegroup,
     dagop,
     discovery,
     encoding,
     error,
     narrowspec,
     pycompat,
     streamclone,
     util,
     wireprotoframing,
     wireprototypes,
 )
 from .utils import (
     interfaceutil,
 )
 
 FRAMINGTYPE = b'application/mercurial-exp-framing-0005'
 
 HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
 
 COMMANDS = wireprototypes.commanddict()
 
 def handlehttpv2request(rctx, req, res, checkperm, urlparts):
     from .hgweb import common as hgwebcommon
 
     # URL space looks like: <permissions>/<command>, where <permission> can
     # be ``ro`` or ``rw`` to signal read-only or read-write, respectively.
 
     # Root URL does nothing meaningful... yet.
     if not urlparts:
         res.status = b'200 OK'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('HTTP version 2 API handler'))
         return
 
     if len(urlparts) == 1:
         res.status = b'404 Not Found'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('do not know how to process %s\n') %
                          req.dispatchpath)
         return
 
     permission, command = urlparts[0:2]
 
     if permission not in (b'ro', b'rw'):
         res.status = b'404 Not Found'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('unknown permission: %s') % permission)
         return
 
     if req.method != 'POST':
         res.status = b'405 Method Not Allowed'
         res.headers[b'Allow'] = b'POST'
         res.setbodybytes(_('commands require POST requests'))
         return
 
     # At some point we'll want to use our own API instead of recycling the
     # behavior of version 1 of the wire protocol...
     # TODO return reasonable responses - not responses that overload the
     # HTTP status line message for error reporting.
     try:
         checkperm(rctx, req, 'pull' if permission == b'ro' else 'push')
     except hgwebcommon.ErrorResponse as e:
         res.status = hgwebcommon.statusmessage(e.code, pycompat.bytestr(e))
         for k, v in e.headers:
             res.headers[k] = v
         res.setbodybytes('permission denied')
         return
 
     # We have a special endpoint to reflect the request back at the client.
     if command == b'debugreflect':
         _processhttpv2reflectrequest(rctx.repo.ui, rctx.repo, req, res)
         return
 
     # Extra commands that we handle that aren't really wire protocol
     # commands. Think extra hard before making this hackery available to
     # extension.
     extracommands = {'multirequest'}
 
     if command not in COMMANDS and command not in extracommands:
         res.status = b'404 Not Found'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('unknown wire protocol command: %s\n') % command)
         return
 
     repo = rctx.repo
     ui = repo.ui
 
     proto = httpv2protocolhandler(req, ui)
 
     if (not COMMANDS.commandavailable(command, proto)
         and command not in extracommands):
         res.status = b'404 Not Found'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('invalid wire protocol command: %s') % command)
         return
 
     # TODO consider cases where proxies may add additional Accept headers.
     if req.headers.get(b'Accept') != FRAMINGTYPE:
         res.status = b'406 Not Acceptable'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('client MUST specify Accept header with value: %s\n')
                            % FRAMINGTYPE)
         return
 
     if req.headers.get(b'Content-Type') != FRAMINGTYPE:
         res.status = b'415 Unsupported Media Type'
         # TODO we should send a response with appropriate media type,
         # since client does Accept it.
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('client MUST send Content-Type header with '
                            'value: %s\n') % FRAMINGTYPE)
         return
 
     _processhttpv2request(ui, repo, req, res, permission, command, proto)
 
 def _processhttpv2reflectrequest(ui, repo, req, res):
     """Reads unified frame protocol request and dumps out state to client.
 
     This special endpoint can be used to help debug the wire protocol.
 
     Instead of routing the request through the normal dispatch mechanism,
     we instead read all frames, decode them, and feed them into our state
     tracker. We then dump the log of all that activity back out to the
     client.
     """
     import json
 
     # Reflection APIs have a history of being abused, accidentally disclosing
     # sensitive data, etc. So we have a config knob.
     if not ui.configbool('experimental', 'web.api.debugreflect'):
         res.status = b'404 Not Found'
         res.headers[b'Content-Type'] = b'text/plain'
         res.setbodybytes(_('debugreflect service not available'))
         return
 
     # We assume we have a unified framing protocol request body.
 
     reactor = wireprotoframing.serverreactor()
     states = []
 
     while True:
         frame = wireprotoframing.readframe(req.bodyfh)
 
         if not frame:
             states.append(b'received: <no frame>')
             break
 
         states.append(b'received: %d %d %d %s' % (frame.typeid, frame.flags,
                                                   frame.requestid,
                                                   frame.payload))
 
         action, meta = reactor.onframerecv(frame)
         states.append(json.dumps((action, meta), sort_keys=True,
                                  separators=(', ', ': ')))
 
     action, meta = reactor.oninputeof()
     meta['action'] = action
     states.append(json.dumps(meta, sort_keys=True, separators=(', ',': ')))
 
     res.status = b'200 OK'
     res.headers[b'Content-Type'] = b'text/plain'
     res.setbodybytes(b'\n'.join(states))
 
 def _processhttpv2request(ui, repo, req, res, authedperm, reqcommand, proto):
     """Post-validation handler for HTTPv2 requests.
 
     Called when the HTTP request contains unified frame-based protocol
     frames for evaluation.
     """
     # TODO Some HTTP clients are full duplex and can receive data before
     # the entire request is transmitted. Figure out a way to indicate support
     # for that so we can opt into full duplex mode.
     reactor = wireprotoframing.serverreactor(deferoutput=True)
     seencommand = False
 
     outstream = reactor.makeoutputstream()
 
     while True:
         frame = wireprotoframing.readframe(req.bodyfh)
         if not frame:
             break
 
         action, meta = reactor.onframerecv(frame)
 
         if action == 'wantframe':
             # Need more data before we can do anything.
             continue
         elif action == 'runcommand':
             sentoutput = _httpv2runcommand(ui, repo, req, res, authedperm,
                                            reqcommand, reactor, outstream,
                                            meta, issubsequent=seencommand)
 
             if sentoutput:
                 return
 
             seencommand = True
 
         elif action == 'error':
             # TODO define proper error mechanism.
             res.status = b'200 OK'
             res.headers[b'Content-Type'] = b'text/plain'
             res.setbodybytes(meta['message'] + b'\n')
             return
         else:
             raise error.ProgrammingError(
                 'unhandled action from frame processor: %s' % action)
 
     action, meta = reactor.oninputeof()
     if action == 'sendframes':
         # We assume we haven't started sending the response yet. If we're
         # wrong, the response type will raise an exception.
         res.status = b'200 OK'
         res.headers[b'Content-Type'] = FRAMINGTYPE
         res.setbodygen(meta['framegen'])
     elif action == 'noop':
         pass
     else:
         raise error.ProgrammingError('unhandled action from frame processor: %s'
                                      % action)
 
 def _httpv2runcommand(ui, repo, req, res, authedperm, reqcommand, reactor,
                       outstream, command, issubsequent):
     """Dispatch a wire protocol command made from HTTPv2 requests.
 
     The authenticated permission (``authedperm``) along with the original
     command from the URL (``reqcommand``) are passed in.
     """
     # We already validated that the session has permissions to perform the
     # actions in ``authedperm``. In the unified frame protocol, the canonical
     # command to run is expressed in a frame. However, the URL also requested
     # to run a specific command. We need to be careful that the command we
     # run doesn't have permissions requirements greater than what was granted
     # by ``authedperm``.
     #
     # Our rule for this is we only allow one command per HTTP request and
     # that command must match the command in the URL. However, we make
     # an exception for the ``multirequest`` URL. This URL is allowed to
     # execute multiple commands. We double check permissions of each command
     # as it is invoked to ensure there is no privilege escalation.
     # TODO consider allowing multiple commands to regular command URLs
     # iff each command is the same.
 
     proto = httpv2protocolhandler(req, ui, args=command['args'])
 
     if reqcommand == b'multirequest':
         if not COMMANDS.commandavailable(command['command'], proto):
             # TODO proper error mechanism
             res.status = b'200 OK'
             res.headers[b'Content-Type'] = b'text/plain'
             res.setbodybytes(_('wire protocol command not available: %s') %
                              command['command'])
             return True
 
         # TODO don't use assert here, since it may be elided by -O.
         assert authedperm in (b'ro', b'rw')
         wirecommand = COMMANDS[command['command']]
         assert wirecommand.permission in ('push', 'pull')
 
         if authedperm == b'ro' and wirecommand.permission != 'pull':
             # TODO proper error mechanism
             res.status = b'403 Forbidden'
             res.headers[b'Content-Type'] = b'text/plain'
             res.setbodybytes(_('insufficient permissions to execute '
                                'command: %s') % command['command'])
             return True
 
         # TODO should we also call checkperm() here? Maybe not if we're going
         # to overhaul that API. The granted scope from the URL check should
         # be good enough.
 
     else:
         # Don't allow multiple commands outside of ``multirequest`` URL.
         if issubsequent:
             # TODO proper error mechanism
             res.status = b'200 OK'
             res.headers[b'Content-Type'] = b'text/plain'
             res.setbodybytes(_('multiple commands cannot be issued to this '
                                'URL'))
             return True
 
         if reqcommand != command['command']:
             # TODO define proper error mechanism
             res.status = b'200 OK'
             res.headers[b'Content-Type'] = b'text/plain'
             res.setbodybytes(_('command in frame must match command in URL'))
             return True
 
     res.status = b'200 OK'
     res.headers[b'Content-Type'] = FRAMINGTYPE
 
     try:
         objs = dispatch(repo, proto, command['command'])
 
         action, meta = reactor.oncommandresponsereadyobjects(
             outstream, command['requestid'], objs)
 
     except error.WireprotoCommandError as e:
         action, meta = reactor.oncommanderror(
             outstream, command['requestid'], e.message, e.messageargs)
 
     except Exception as e:
         action, meta = reactor.onservererror(
             outstream, command['requestid'],
             _('exception when invoking command: %s') % e)
 
     if action == 'sendframes':
         res.setbodygen(meta['framegen'])
         return True
     elif action == 'noop':
         return False
     else:
         raise error.ProgrammingError('unhandled event from reactor: %s' %
                                      action)
 
 def getdispatchrepo(repo, proto, command):
     return repo.filtered('served')
 
 def dispatch(repo, proto, command):
     repo = getdispatchrepo(repo, proto, command)
 
     func, spec = COMMANDS[command]
     args = proto.getargs(spec)
 
     return func(repo, proto, **args)
 
 @interfaceutil.implementer(wireprototypes.baseprotocolhandler)
 class httpv2protocolhandler(object):
     def __init__(self, req, ui, args=None):
         self._req = req
         self._ui = ui
         self._args = args
 
     @property
     def name(self):
         return HTTP_WIREPROTO_V2
 
     def getargs(self, args):
         # First look for args that were passed but aren't registered on this
         # command.
         extra = set(self._args) - set(args)
         if extra:
             raise error.WireprotoCommandError(
                 'unsupported argument to command: %s' %
                 ', '.join(sorted(extra)))
 
         # And look for required arguments that are missing.
         missing = {a for a in args if args[a]['required']} - set(self._args)
 
         if missing:
             raise error.WireprotoCommandError(
                 'missing required arguments: %s' % ', '.join(sorted(missing)))
 
         # Now derive the arguments to pass to the command, taking into
         # account the arguments specified by the client.
         data = {}
         for k, meta in sorted(args.items()):
             # This argument wasn't passed by the client.
             if k not in self._args:
                 data[k] = meta['default']()
                 continue
 
             v = self._args[k]
 
             # Sets may be expressed as lists. Silently normalize.
             if meta['type'] == 'set' and isinstance(v, list):
                 v = set(v)
 
             # TODO consider more/stronger type validation.
 
             data[k] = v
 
         return data
 
     def getprotocaps(self):
         # Protocol capabilities are currently not implemented for HTTP V2.
         return set()
 
     def getpayload(self):
         raise NotImplementedError
 
     @contextlib.contextmanager
     def mayberedirectstdio(self):
         raise NotImplementedError
 
     def client(self):
         raise NotImplementedError
 
     def addcapabilities(self, repo, caps):
         return caps
 
     def checkperm(self, perm):
         raise NotImplementedError
 
 def httpv2apidescriptor(req, repo):
     proto = httpv2protocolhandler(req, repo.ui)
 
     return _capabilitiesv2(repo, proto)
 
 def _capabilitiesv2(repo, proto):
     """Obtain the set of capabilities for version 2 transports.
 
     These capabilities are distinct from the capabilities for version 1
     transports.
     """
     compression = []
     for engine in wireprototypes.supportedcompengines(repo.ui, util.SERVERROLE):
         compression.append({
             b'name': engine.wireprotosupport().name,
         })
 
     caps = {
         'commands': {},
         'compression': compression,
         'framingmediatypes': [FRAMINGTYPE],
         'pathfilterprefixes': set(narrowspec.VALID_PREFIXES),
     }
 
     # TODO expose available changesetdata fields.
 
     for command, entry in COMMANDS.items():
-        args = {arg: meta['example'] for arg, meta in entry.args.items()}
+        args = {}
+
+        for arg, meta in entry.args.items():
+            args[arg] = {
+                # TODO should this be a normalized type using CBOR's
+                # terminology?
+                b'type': meta['type'],
+                b'required': meta['required'],
+            }
+
+            if not meta['required']:
+                args[arg][b'default'] = meta['default']()
 
         caps['commands'][command] = {
             'args': args,
             'permissions': [entry.permission],
         }
 
     if streamclone.allowservergeneration(repo):
         caps['rawrepoformats'] = sorted(repo.requirements &
                                         repo.supportedformats)
 
     return proto.addcapabilities(repo, caps)
 
 def builddeltarequests(store, nodes, haveparents):
     """Build a series of revision delta requests against a backend store.
 
     Returns a list of revision numbers in the order they should be sent
     and a list of ``irevisiondeltarequest`` instances to be made against
     the backend store.
     """
     # We sort and send nodes in DAG order because this is optimal for
     # storage emission.
     # TODO we may want a better storage API here - one where we can throw
     # a list of nodes and delta preconditions over a figurative wall and
     # have the storage backend figure it out for us.
     revs = dagop.linearize({store.rev(n) for n in nodes}, store.parentrevs)
 
     requests = []
     seenrevs = set()
 
     for rev in revs:
         node = store.node(rev)
         parentnodes = store.parents(node)
         parentrevs = [store.rev(n) for n in parentnodes]
         deltabaserev = store.deltaparent(rev)
         deltabasenode = store.node(deltabaserev)
 
         # The choice of whether to send a fulltext revision or a delta and
         # what delta to send is governed by a few factors.
         #
         # To send a delta, we need to ensure the receiver is capable of
         # decoding it. And that requires the receiver to have the base
         # revision the delta is against.
         #
         # We can only guarantee the receiver has the base revision if
         # a) we've already sent the revision as part of this group
         # b) the receiver has indicated they already have the revision.
         # And the mechanism for "b" is the client indicating they have
         # parent revisions. So this means we can only send the delta if
         # it is sent before or it is against a delta and the receiver says
         # they have a parent.
 
         # We can send storage delta if it is against a revision we've sent
         # in this group.
         if deltabaserev != nullrev and deltabaserev in seenrevs:
             basenode = deltabasenode
 
         # We can send storage delta if it is against a parent revision and
         # the receiver indicates they have the parents.
         elif (deltabaserev != nullrev and deltabaserev in parentrevs
               and haveparents):
             basenode = deltabasenode
 
         # Otherwise the storage delta isn't appropriate. Fall back to
         # using another delta, if possible.
 
         # Use p1 if we've emitted it or receiver says they have it.
         elif parentrevs[0] != nullrev and (
             parentrevs[0] in seenrevs or haveparents):
             basenode = parentnodes[0]
 
         # Use p2 if we've emitted it or receiver says they have it.
         elif parentrevs[1] != nullrev and (
             parentrevs[1] in seenrevs or haveparents):
             basenode = parentnodes[1]
 
         # Nothing appropriate to delta against. Send the full revision.
         else:
             basenode = nullid
 
         requests.append(changegroup.revisiondeltarequest(
             node=node,
             p1node=parentnodes[0],
             p2node=parentnodes[1],
             # Receiver deals with linknode resolution.
             linknode=nullid,
             basenode=basenode,
         ))
 
         seenrevs.add(rev)
 
     return revs, requests
 
 def wireprotocommand(name, args=None, permission='push'):
     """Decorator to declare a wire protocol command.
 
     ``name`` is the name of the wire protocol command being provided.
 
     ``args`` is a dict defining arguments accepted by the command. Keys are
     the argument name. Values are dicts with the following keys:
 
        ``type``
           The argument data type. Must be one of the following string
           literals: ``bytes``, ``int``, ``list``, ``dict``, ``set``,
           or ``bool``.
 
        ``default``
           A callable returning the default value for this argument. If not
           specified, ``None`` will be the default value.
 
        ``required``
           Bool indicating whether the argument is required.
 
        ``example``
           An example value for this argument.
 
     ``permission`` defines the permission type needed to run this command.
     Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
     respectively. Default is to assume command requires ``push`` permissions
     because otherwise commands not declaring their permissions could modify
     a repository that is supposed to be read-only.
 
     Wire protocol commands are generators of objects to be serialized and
     sent to the client.
 
     If a command raises an uncaught exception, this will be translated into
     a command error.
     """
     transports = {k for k, v in wireprototypes.TRANSPORTS.items()
                   if v['version'] == 2}
 
     if permission not in ('push', 'pull'):
         raise error.ProgrammingError('invalid wire protocol permission; '
                                      'got %s; expected "push" or "pull"' %
                                      permission)
 
     if args is None:
         args = {}
 
     if not isinstance(args, dict):
         raise error.ProgrammingError('arguments for version 2 commands '
                                      'must be declared as dicts')
 
     for arg, meta in args.items():
         if arg == '*':
             raise error.ProgrammingError('* argument name not allowed on '
                                          'version 2 commands')
 
         if not isinstance(meta, dict):
             raise error.ProgrammingError('arguments for version 2 commands '
                                          'must declare metadata as a dict')
 
         if 'type' not in meta:
             raise error.ProgrammingError('%s argument for command %s does not '
                                          'declare type field' % (arg, name))
 
         if meta['type'] not in ('bytes', 'int', 'list', 'dict', 'set', 'bool'):
             raise error.ProgrammingError('%s argument for command %s has '
                                          'illegal type: %s' % (arg, name,
                                                                meta['type']))
 
         if 'example' not in meta:
             raise error.ProgrammingError('%s argument for command %s does not '
                                          'declare example field' % (arg, name))
 
         if 'default' in meta and meta.get('required'):
             raise error.ProgrammingError('%s argument for command %s is marked '
                                          'as required but has a default value' %
                                          (arg, name))
 
         meta.setdefault('default', lambda: None)
         meta.setdefault('required', False)
 
     def register(func):
         if name in COMMANDS:
             raise error.ProgrammingError('%s command already registered '
                                          'for version 2' % name)
 
         COMMANDS[name] = wireprototypes.commandentry(
             func, args=args, transports=transports, permission=permission)
 
         return func
 
     return register
 
 @wireprotocommand('branchmap', permission='pull')
 def branchmapv2(repo, proto):
     yield {encoding.fromlocal(k): v
            for k, v in repo.branchmap().iteritems()}
 
 @wireprotocommand('capabilities', permission='pull')
 def capabilitiesv2(repo, proto):
     yield _capabilitiesv2(repo, proto)
 
 @wireprotocommand(
     'changesetdata',
     args={
         'noderange': {
             'type': 'list',
             'example': [[b'0123456...'], [b'abcdef...']],
         },
         'nodes': {
             'type': 'list',
             'example': [b'0123456...'],
         },
         'fields': {
             'type': 'set',
             'default': set,
             'example': {b'parents', b'revision'},
         },
     },
     permission='pull')
 def changesetdata(repo, proto, noderange, nodes, fields):
     # TODO look for unknown fields and abort when they can't be serviced.
 
     if noderange is None and nodes is None:
         raise error.WireprotoCommandError(
             'noderange or nodes must be defined')
 
     if noderange is not None:
         if len(noderange) != 2:
             raise error.WireprotoCommandError(
                 'noderange must consist of 2 elements')
 
         if not noderange[1]:
             raise error.WireprotoCommandError(
                 'heads in noderange request cannot be empty')
 
     cl = repo.changelog
     hasnode = cl.hasnode
 
     seen = set()
     outgoing = []
 
     if nodes is not None:
         outgoing.extend(n for n in nodes if hasnode(n))
         seen |= set(outgoing)
 
     if noderange is not None:
         if noderange[0]:
             common = [n for n in noderange[0] if hasnode(n)]
         else:
             common = [nullid]
 
         for n in discovery.outgoing(repo, common, noderange[1]).missing:
             if n not in seen:
                 outgoing.append(n)
             # Don't need to add to seen here because this is the final
             # source of nodes and there should be no duplicates in this
             # list.
 
     seen.clear()
     publishing = repo.publishing()
 
     if outgoing:
         repo.hook('preoutgoing', throw=True, source='serve')
 
     yield {
         b'totalitems': len(outgoing),
     }
 
     # The phases of nodes already transferred to the client may have changed
     # since the client last requested data. We send phase-only records
     # for these revisions, if requested.
     if b'phase' in fields and noderange is not None:
         # TODO skip nodes whose phase will be reflected by a node in the
         # outgoing set. This is purely an optimization to reduce data
         # size.
         for node in noderange[0]:
             yield {
                 b'node': node,
                 b'phase': b'public' if publishing else repo[node].phasestr()
             }
 
     nodebookmarks = {}
     for mark, node in repo._bookmarks.items():
         nodebookmarks.setdefault(node, set()).add(mark)
 
     # It is already topologically sorted by revision number.
     for node in outgoing:
         d = {
             b'node': node,
         }
 
         if b'parents' in fields:
             d[b'parents'] = cl.parents(node)
 
         if b'phase' in fields:
             if publishing:
                 d[b'phase'] = b'public'
             else:
                 ctx = repo[node]
                 d[b'phase'] = ctx.phasestr()
 
         if b'bookmarks' in fields and node in nodebookmarks:
             d[b'bookmarks'] = sorted(nodebookmarks[node])
             del nodebookmarks[node]
 
         revisiondata = None
 
         if b'revision' in fields:
             revisiondata = cl.revision(node, raw=True)
             d[b'revisionsize'] = len(revisiondata)
 
         # TODO make it possible for extensions to wrap a function or register
         # a handler to service custom fields.
 
         yield d
 
         if revisiondata is not None:
             yield revisiondata
 
     # If requested, send bookmarks from nodes that didn't have revision
     # data sent so receiver is aware of any bookmark updates.
     if b'bookmarks' in fields:
         for node, marks in sorted(nodebookmarks.iteritems()):
             yield {
                 b'node': node,
                 b'bookmarks': sorted(marks),
             }
 
 class FileAccessError(Exception):
     """Represents an error accessing a specific file."""
 
     def __init__(self, path, msg, args):
         self.path = path
         self.msg = msg
         self.args = args
 
 def getfilestore(repo, proto, path):
     """Obtain a file storage object for use with wire protocol.
 
     Exists as a standalone function so extensions can monkeypatch to add
     access control.
     """
     # This seems to work even if the file doesn't exist. So catch
     # "empty" files and return an error.
     fl = repo.file(path)
 
     if not len(fl):
         raise FileAccessError(path, 'unknown file: %s', (path,))
 
     return fl
 
 @wireprotocommand(
     'filedata',
     args={
         'haveparents': {
             'type': 'bool',
             'default': lambda: False,
             'example': True,
         },
         'nodes': {
             'type': 'list',
             'required': True,
             'example': [b'0123456...'],
         },
         'fields': {
             'type': 'set',
             'default': set,
             'example': {b'parents', b'revision'},
         },
         'path': {
             'type': 'bytes',
             'required': True,
             'example': b'foo.txt',
         }
     },
     permission='pull')
 def filedata(repo, proto, haveparents, nodes, fields, path):
     try:
         # Extensions may wish to access the protocol handler.
         store = getfilestore(repo, proto, path)
     except FileAccessError as e:
         raise error.WireprotoCommandError(e.msg, e.args)
 
     # Validate requested nodes.
     for node in nodes:
         try:
             store.rev(node)
         except error.LookupError:
             raise error.WireprotoCommandError('unknown file node: %s',
                                               (hex(node),))
 
     revs, requests = builddeltarequests(store, nodes, haveparents)
 
     yield {
         b'totalitems': len(revs),
     }
 
     if b'revision' in fields:
         deltas = store.emitrevisiondeltas(requests)
     else:
         deltas = None
 
     for rev in revs:
         node = store.node(rev)
 
         if deltas is not None:
             delta = next(deltas)
         else:
             delta = None
 
         d = {
             b'node': node,
         }
 
         if b'parents' in fields:
             d[b'parents'] = store.parents(node)
 
         if b'revision' in fields:
             assert delta is not None
             assert delta.flags == 0
             assert d[b'node'] == delta.node
 
             if delta.revision is not None:
                 revisiondata = delta.revision
                 d[b'revisionsize'] = len(revisiondata)
             else:
                 d[b'deltabasenode'] = delta.basenode
                 revisiondata = delta.delta
                 d[b'deltasize'] = len(revisiondata)
         else:
             revisiondata = None
 
         yield d
 
         if revisiondata is not None:
             yield revisiondata
 
     if deltas is not None:
         try:
             next(deltas)
             raise error.ProgrammingError('should not have more deltas')
         except GeneratorExit:
             pass
 
 @wireprotocommand(
     'heads',
     args={
         'publiconly': {
             'type': 'bool',
             'default': lambda: False,
             'example': False,
         },
     },
     permission='pull')
 def headsv2(repo, proto, publiconly):
     if publiconly:
         repo = repo.filtered('immutable')
 
     yield repo.heads()
 
 @wireprotocommand(
     'known',
     args={
         'nodes': {
             'type': 'list',
             'default': list,
             'example': [b'deadbeef'],
         },
     },
     permission='pull')
 def knownv2(repo, proto, nodes):
     result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
     yield result
 
 @wireprotocommand(
     'listkeys',
     args={
         'namespace': {
             'type': 'bytes',
             'required': True,
             'example': b'ns',
         },
     },
     permission='pull')
 def listkeysv2(repo, proto, namespace):
     keys = repo.listkeys(encoding.tolocal(namespace))
     keys = {encoding.fromlocal(k): encoding.fromlocal(v)
             for k, v in keys.iteritems()}
 
     yield keys
 
 @wireprotocommand(
     'lookup',
     args={
         'key': {
             'type': 'bytes',
             'required': True,
             'example': b'foo',
         },
     },
     permission='pull')
 def lookupv2(repo, proto, key):
     key = encoding.tolocal(key)
 
     # TODO handle exception.
     node = repo.lookup(key)
 
     yield node
 
 @wireprotocommand(
     'manifestdata',
     args={
         'nodes': {
             'type': 'list',
             'required': True,
             'example': [b'0123456...'],
         },
         'haveparents': {
             'type': 'bool',
             'default': lambda: False,
             'example': True,
         },
         'fields': {
             'type': 'set',
             'default': set,
             'example': {b'parents', b'revision'},
         },
         'tree': {
             'type': 'bytes',
             'required': True,
             'example': b'',
         },
     },
     permission='pull')
 def manifestdata(repo, proto, haveparents, nodes, fields, tree):
     store = repo.manifestlog.getstorage(tree)
 
     # Validate the node is known and abort on unknown revisions.
     for node in nodes:
         try:
             store.rev(node)
         except error.LookupError:
             raise error.WireprotoCommandError(
                 'unknown node: %s', (node,))
 
     revs, requests = builddeltarequests(store, nodes, haveparents)
 
     yield {
         b'totalitems': len(revs),
     }
 
     if b'revision' in fields:
         deltas = store.emitrevisiondeltas(requests)
     else:
         deltas = None
 
     for rev in revs:
         node = store.node(rev)
 
         if deltas is not None:
             delta = next(deltas)
         else:
             delta = None
 
         d = {
             b'node': node,
         }
 
         if b'parents' in fields:
             d[b'parents'] = store.parents(node)
 
         if b'revision' in fields:
             assert delta is not None
             assert delta.flags == 0
             assert d[b'node'] == delta.node
 
             if delta.revision is not None:
                 revisiondata = delta.revision
                 d[b'revisionsize'] = len(revisiondata)
             else:
                 d[b'deltabasenode'] = delta.basenode
                 revisiondata = delta.delta
                 d[b'deltasize'] = len(revisiondata)
         else:
             revisiondata = None
 
         yield d
 
         if revisiondata is not None:
             yield revisiondata
 
     if deltas is not None:
         try:
             next(deltas)
             raise error.ProgrammingError('should not have more deltas')
         except GeneratorExit:
             pass
 
 @wireprotocommand(
     'pushkey',
     args={
         'namespace': {
             'type': 'bytes',
             'required': True,
             'example': b'ns',
         },
         'key': {
             'type': 'bytes',
             'required': True,
             'example': b'key',
         },
         'old': {
             'type': 'bytes',
             'required': True,
             'example': b'old',
         },
         'new': {
             'type': 'bytes',
             'required': True,
             'example': 'new',
         },
     },
     permission='push')
 def pushkeyv2(repo, proto, namespace, key, old, new):
     # TODO handle ui output redirection
     yield repo.pushkey(encoding.tolocal(namespace),
                        encoding.tolocal(key),
                        encoding.tolocal(old),
                        encoding.tolocal(new))

tests/test-http-protocol.t

@@ -1,749 +1,749 b''
 #require no-chg
 
   $ . $TESTDIR/wireprotohelpers.sh
 
   $ cat >> $HGRCPATH << EOF
   > [web]
   > push_ssl = false
   > allow_push = *
   > EOF
 
   $ hg init server
   $ cd server
   $ touch a
   $ hg -q commit -A -m initial
   $ cd ..
 
   $ hg serve -R server -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid >> $DAEMON_PIDS
 
 compression formats are advertised in compression capability
 
 #if zstd
   $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=capabilities' | tr ' ' '\n' | grep '^compression=zstd,zlib$' > /dev/null
 #else
   $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=capabilities' | tr ' ' '\n' | grep '^compression=zlib$' > /dev/null
 #endif
 
   $ killdaemons.py
 
 server.compressionengines can replace engines list wholesale
 
   $ hg serve --config server.compressionengines=none -R server -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
   $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=capabilities' | tr ' ' '\n' | grep '^compression=none$' > /dev/null
 
   $ killdaemons.py
 
 Order of engines can also change
 
   $ hg serve --config server.compressionengines=none,zlib -R server -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
   $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=capabilities' | tr ' ' '\n' | grep '^compression=none,zlib$' > /dev/null
 
   $ killdaemons.py
 
 Start a default server again
 
   $ hg serve -R server -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
 
 Server should send application/mercurial-0.1 to clients if no Accept is used
 
   $ get-with-headers.py --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' -
   200 Script output follows
   content-type: application/mercurial-0.1
   date: $HTTP_DATE$
   server: testing stub value
   transfer-encoding: chunked
 
 Server should send application/mercurial-0.1 when client says it wants it
 
   $ get-with-headers.py --hgproto '0.1' --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' -
   200 Script output follows
   content-type: application/mercurial-0.1
   date: $HTTP_DATE$
   server: testing stub value
   transfer-encoding: chunked
 
 Server should send application/mercurial-0.2 when client says it wants it
 
   $ get-with-headers.py --hgproto '0.2' --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' -
   200 Script output follows
   content-type: application/mercurial-0.2
   date: $HTTP_DATE$
   server: testing stub value
   transfer-encoding: chunked
 
   $ get-with-headers.py --hgproto '0.1 0.2' --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' -
   200 Script output follows
   content-type: application/mercurial-0.2
   date: $HTTP_DATE$
   server: testing stub value
   transfer-encoding: chunked
 
 Requesting a compression format that server doesn't support results will fall back to 0.1
 
   $ get-with-headers.py --hgproto '0.2 comp=aa' --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' -
   200 Script output follows
   content-type: application/mercurial-0.1
   date: $HTTP_DATE$
   server: testing stub value
   transfer-encoding: chunked
 
 #if zstd
 zstd is used if available
 
   $ get-with-headers.py --hgproto '0.2 comp=zstd' $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' > resp
   $ f --size --hexdump --bytes 36 --sha1 resp
   resp: size=248, sha1=4d8d8f87fb82bd542ce52881fdc94f850748
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 04 7a 73 74 64 |t follows...zstd|
   0020: 28 b5 2f fd                                     |(./.|
 
 #endif
 
 application/mercurial-0.2 is not yet used on non-streaming responses
 
   $ get-with-headers.py --hgproto '0.2' $LOCALIP:$HGPORT '?cmd=heads' -
   200 Script output follows
   content-length: 41
   content-type: application/mercurial-0.1
   date: $HTTP_DATE$
   server: testing stub value
   
   e93700bd72895c5addab234c56d4024b487a362f
 
 Now test protocol preference usage
 
   $ killdaemons.py
   $ hg serve --config server.compressionengines=none,zlib -R server -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
 
 No Accept will send 0.1+zlib, even though "none" is preferred b/c "none" isn't supported on 0.1
 
   $ get-with-headers.py --headeronly $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000' Content-Type
   200 Script output follows
   content-type: application/mercurial-0.1
 
   $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000'  > resp
   $ f --size --hexdump --bytes 28 --sha1 resp
   resp: size=227, sha1=35a4c074da74f32f5440da3cbf04
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 78             |t follows..x|
 
 Explicit 0.1 will send zlib because "none" isn't supported on 0.1
 
   $ get-with-headers.py --hgproto '0.1' $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000'  > resp
   $ f --size --hexdump --bytes 28 --sha1 resp
   resp: size=227, sha1=35a4c074da74f32f5440da3cbf04
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 78             |t follows..x|
 
 0.2 with no compression will get "none" because that is server's preference
 (spec says ZL and UN are implicitly supported)
 
   $ get-with-headers.py --hgproto '0.2' $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000'  > resp
   $ f --size --hexdump --bytes 32 --sha1 resp
   resp: size=432, sha1=ac931b412ec185a02e0e5bcff98dac83
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 04 6e 6f 6e 65 |t follows...none|
 
 Client receives server preference even if local order doesn't match
 
   $ get-with-headers.py --hgproto '0.2 comp=zlib,none' $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000'  > resp
   $ f --size --hexdump --bytes 32 --sha1 resp
   resp: size=432, sha1=ac931b412ec185a02e0e5bcff98dac83
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 04 6e 6f 6e 65 |t follows...none|
 
 Client receives only supported format even if not server preferred format
 
   $ get-with-headers.py --hgproto '0.2 comp=zlib' $LOCALIP:$HGPORT '?cmd=getbundle&heads=e93700bd72895c5addab234c56d4024b487a362f&common=0000000000000000000000000000000000000000'  > resp
   $ f --size --hexdump --bytes 33 --sha1 resp
   resp: size=232, sha1=a1c727f0c9693ca15742a75c30419bc36
   0000: 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 04 7a 6c 69 62 |t follows...zlib|
   0020: 78                                              |x|
 
   $ killdaemons.py
   $ cd ..
 
 Test listkeys for listing namespaces
 
   $ hg init empty
   $ hg -R empty serve -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg --verbose debugwireproto http://$LOCALIP:$HGPORT << EOF
   > command listkeys
   >     namespace namespaces
   > EOF
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending listkeys command
   s>     GET /?cmd=listkeys HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgArg-1,X-HgProto-1\r\n
   s>     x-hgarg-1: namespace=namespaces\r\n
   s>     x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 30\r\n
   s>     \r\n
   s>     bookmarks\t\n
   s>     namespaces\t\n
   s>     phases\t
   response: {
     b'bookmarks': b'',
     b'namespaces': b'',
     b'phases': b''
   }
 
 Same thing, but with "httprequest" command
 
   $ hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT << EOF
   > httprequest GET ?cmd=listkeys
   >     user-agent: test
   >     x-hgarg-1: namespace=namespaces
   > EOF
   using raw connection to peer
   s>     GET /?cmd=listkeys HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgarg-1: namespace=namespaces\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 30\r\n
   s>     \r\n
   s>     bookmarks\t\n
   s>     namespaces\t\n
   s>     phases\t
 
 Client with HTTPv2 enabled advertises that and gets old capabilities response from old server
 
   $ hg --config experimental.httppeer.advertise-v2=true --verbose debugwireproto http://$LOCALIP:$HGPORT << EOF
   > command heads
   > EOF
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1,X-HgUpgrade-1\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: exp-http-v2-0001\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending heads command
   s>     GET /?cmd=heads HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1\r\n
   s>     x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 41\r\n
   s>     \r\n
   s>     0000000000000000000000000000000000000000\n
   response: [
     b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
   ]
 
   $ killdaemons.py
   $ enablehttpv2 empty
   $ hg --config server.compressionengines=zlib -R empty serve -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
 
 Client with HTTPv2 enabled automatically upgrades if the server supports it
 
   $ hg --config experimental.httppeer.advertise-v2=true --verbose debugwireproto http://$LOCALIP:$HGPORT << EOF
   > command heads
   > EOF
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1,X-HgUpgrade-1\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: exp-http-v2-0001\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-cbor\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
-  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xd9\x01\x02\x82GparentsHrevisionInoderange\x82\x81J0123456...\x81Iabcdef...Enodes\x81J0123456...Kpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...DpathGfoo.txtKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...Dtree@Kpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
+  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetInoderange\xa3Gdefault\xf6Hrequired\xf4DtypeDlistEnodes\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullGpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending heads command
   s>     POST /api/exp-http-v2-0001/ro/heads HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-exp-framing-0005\r\n
   s>     content-type: application/mercurial-exp-framing-0005\r\n
   s>     content-length: 20\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s>     \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEheads
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-exp-framing-0005\r\n
   s>     Transfer-Encoding: chunked\r\n
   s>     \r\n
   s>     13\r\n
   s>     \x0b\x00\x00\x01\x00\x02\x011
   s>     \xa1FstatusBok
   s>     \r\n
   received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
   s>     1e\r\n
   s>     \x16\x00\x00\x01\x00\x02\x001
   s>     \x81T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
   s>     \r\n
   received frame(size=22; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
   s>     8\r\n
   s>     \x00\x00\x00\x01\x00\x02\x002
   s>     \r\n
   s>     0\r\n
   s>     \r\n
   received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
   response: [
     b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
   ]
 
   $ killdaemons.py
 
 HTTP client follows HTTP redirect on handshake to new repo
 
   $ cd $TESTTMP
 
   $ hg init redirector
   $ hg init redirected
   $ cd redirected
   $ touch foo
   $ hg -q commit -A -m initial
   $ cd ..
 
   $ cat > paths.conf << EOF
   > [paths]
   > / = $TESTTMP/*
   > EOF
 
   $ cat > redirectext.py << EOF
   > from mercurial import extensions, wireprotoserver
   > def wrappedcallhttp(orig, repo, req, res, proto, cmd):
   >     path = req.advertisedurl[len(req.advertisedbaseurl):]
   >     if not path.startswith(b'/redirector'):
   >         return orig(repo, req, res, proto, cmd)
   >     relpath = path[len(b'/redirector'):]
   >     res.status = b'301 Redirect'
   >     newurl = b'%s/redirected%s' % (req.baseurl, relpath)
   >     if not repo.ui.configbool('testing', 'redirectqs', True) and b'?' in newurl:
   >         newurl = newurl[0:newurl.index(b'?')]
   >     res.headers[b'Location'] = newurl
   >     res.headers[b'Content-Type'] = b'text/plain'
   >     res.setbodybytes(b'redirected')
   >     return True
   > 
   > extensions.wrapfunction(wireprotoserver, '_callhttp', wrappedcallhttp)
   > EOF
 
   $ hg --config extensions.redirect=$TESTTMP/redirectext.py \
   >    --config server.compressionengines=zlib \
   >     serve --web-conf paths.conf --pid-file hg.pid -p $HGPORT -d
   $ cat hg.pid > $DAEMON_PIDS
 
 Verify our HTTP 301 is served properly
 
   $ hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT << EOF
   > httprequest GET /redirector?cmd=capabilities
   >     user-agent: test
   > EOF
   using raw connection to peer
   s>     GET /redirector?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 301 Redirect\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Location: http://$LOCALIP:$HGPORT/redirected?cmd=capabilities\r\n (glob)
   s>     Content-Type: text/plain\r\n
   s>     Content-Length: 10\r\n
   s>     \r\n
   s>     redirected
   s>     GET /redirected?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 467\r\n
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
 
 Test with the HTTP peer
 
   $ hg --verbose debugwireproto http://$LOCALIP:$HGPORT/redirector << EOF
   > command heads
   > EOF
   s>     GET /redirector?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 301 Redirect\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Location: http://$LOCALIP:$HGPORT/redirected?cmd=capabilities\r\n (glob)
   s>     Content-Type: text/plain\r\n
   s>     Content-Length: 10\r\n
   s>     \r\n
   s>     redirected
   s>     GET /redirected?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 467\r\n
   s>     \r\n
   real URL is http://$LOCALIP:$HGPORT/redirected (glob)
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending heads command
   s>     GET /redirected?cmd=heads HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1\r\n
   s>     x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 41\r\n
   s>     \r\n
   s>     96ee1d7354c4ad7372047672c36a1f561e3a6a4c\n
   response: [
     b'\x96\xee\x1dsT\xc4\xadsr\x04vr\xc3j\x1fV\x1e:jL'
   ]
 
   $ killdaemons.py
 
 Now test a variation where we strip the query string from the redirect URL.
 (SCM Manager apparently did this and clients would recover from it)
 
   $ hg --config extensions.redirect=$TESTTMP/redirectext.py \
   >    --config server.compressionengines=zlib \
   >    --config testing.redirectqs=false \
   >     serve --web-conf paths.conf --pid-file hg.pid -p $HGPORT -d
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT << EOF
   > httprequest GET /redirector?cmd=capabilities
   >     user-agent: test
   > EOF
   using raw connection to peer
   s>     GET /redirector?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 301 Redirect\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Location: http://$LOCALIP:$HGPORT/redirected\r\n (glob)
   s>     Content-Type: text/plain\r\n
   s>     Content-Length: 10\r\n
   s>     \r\n
   s>     redirected
   s>     GET /redirected HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     ETag: W/"*"\r\n (glob)
   s>     Content-Type: text/html; charset=ascii\r\n
   s>     Transfer-Encoding: chunked\r\n
   s>     \r\n
   s>     414\r\n
   s>     <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">\n
   s>     <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US">\n
   s>     <head>\n
   s>     <link rel="icon" href="/redirected/static/hgicon.png" type="image/png" />\n
   s>     <meta name="robots" content="index, nofollow" />\n
   s>     <link rel="stylesheet" href="/redirected/static/style-paper.css" type="text/css" />\n
   s>     <script type="text/javascript" src="/redirected/static/mercurial.js"></script>\n
   s>     \n
   s>     <title>redirected: log</title>\n
   s>     <link rel="alternate" type="application/atom+xml"\n
   s>        href="/redirected/atom-log" title="Atom feed for redirected" />\n
   s>     <link rel="alternate" type="application/rss+xml"\n
   s>        href="/redirected/rss-log" title="RSS feed for redirected" />\n
   s>     </head>\n
   s>     <body>\n
   s>     \n
   s>     <div class="container">\n
   s>     <div class="menu">\n
   s>     <div class="logo">\n
   s>     <a href="https://mercurial-scm.org/">\n
   s>     <img src="/redirected/static/hglogo.png" alt="mercurial" /></a>\n
   s>     </div>\n
   s>     <ul>\n
   s>     <li class="active">log</li>\n
   s>     <li><a href="/redirected/graph/tip">graph</a></li>\n
   s>     <li><a href="/redirected/tags">tags</a></li>\n
   s>     <li><a href="
   s>     \r\n
   s>     810\r\n
   s>     /redirected/bookmarks">bookmarks</a></li>\n
   s>     <li><a href="/redirected/branches">branches</a></li>\n
   s>     </ul>\n
   s>     <ul>\n
   s>     <li><a href="/redirected/rev/tip">changeset</a></li>\n
   s>     <li><a href="/redirected/file/tip">browse</a></li>\n
   s>     </ul>\n
   s>     <ul>\n
   s>     \n
   s>     </ul>\n
   s>     <ul>\n
   s>      <li><a href="/redirected/help">help</a></li>\n
   s>     </ul>\n
   s>     <div class="atom-logo">\n
   s>     <a href="/redirected/atom-log" title="subscribe to atom feed">\n
   s>     <img class="atom-logo" src="/redirected/static/feed-icon-14x14.png" alt="atom feed" />\n
   s>     </a>\n
   s>     </div>\n
   s>     </div>\n
   s>     \n
   s>     <div class="main">\n
   s>     <h2 class="breadcrumb"><a href="/">Mercurial</a> &gt; <a href="/redirected">redirected</a> </h2>\n
   s>     <h3>log</h3>\n
   s>     \n
   s>     \n
   s>     <form class="search" action="/redirected/log">\n
   s>     \n
   s>     <p><input name="rev" id="search1" type="text" size="30" value="" /></p>\n
   s>     <div id="hint">Find changesets by keywords (author, files, the commit message), revision\n
   s>     number or hash, or <a href="/redirected/help/revsets">revset expression</a>.</div>\n
   s>     </form>\n
   s>     \n
   s>     <div class="navigate">\n
   s>     <a href="/redirected/shortlog/tip?revcount=30">less</a>\n
   s>     <a href="/redirected/shortlog/tip?revcount=120">more</a>\n
   s>     | rev 0: <a href="/redirected/shortlog/96ee1d7354c4">(0)</a> <a href="/redirected/shortlog/tip">tip</a> \n
   s>     </div>\n
   s>     \n
   s>     <table class="bigtable">\n
   s>     <thead>\n
   s>      <tr>\n
   s>       <th class="age">age</th>\n
   s>       <th class="author">author</th>\n
   s>       <th class="description">description</th>\n
   s>      </tr>\n
   s>     </thead>\n
   s>     <tbody class="stripes2">\n
   s>      <tr>\n
   s>       <td class="age">Thu, 01 Jan 1970 00:00:00 +0000</td>\n
   s>       <td class="author">test</td>\n
   s>       <td class="description">\n
   s>        <a href="/redirected/rev/96ee1d7354c4">initial</a>\n
   s>        <span class="phase">draft</span> <span class="branchhead">default</span> <span class="tag">tip</span> \n
   s>       </td>\n
   s>      </tr>\n
   s>     \n
   s>     </tbody>\n
   s>     </table>\n
   s>     \n
   s>     <div class="navigate">\n
   s>     <a href="/redirected/shortlog/tip?revcount=30">less</a>\n
   s>     <a href="/redirected/shortlog/tip?revcount=120">more</a>\n
   s>     | rev 0: <a href="/redirected/shortlog/96ee1d7354c4">(0)</a> <a href="/redirected/shortlog/tip">tip</a> \n
   s>     </div>\n
   s>     \n
   s>     <script type="text/javascript">\n
   s>         ajaxScrollInit(\n
   s>                 \'/redirected/shortlog/%next%\',\n
   s>                 \'\', <!-- NEXTHASH\n
   s>                 function (htmlText) {
   s>     \r\n
   s>     14a\r\n
   s>     \n
   s>                     var m = htmlText.match(/\'(\\w+)\', <!-- NEXTHASH/);\n
   s>                     return m ? m[1] : null;\n
   s>                 },\n
   s>                 \'.bigtable > tbody\',\n
   s>                 \'<tr class="%class%">\\\n
   s>                 <td colspan="3" style="text-align: center;">%text%</td>\\\n
   s>                 </tr>\'\n
   s>         );\n
   s>     </script>\n
   s>     \n
   s>     </div>\n
   s>     </div>\n
   s>     \n
   s>     \n
   s>     \n
   s>     </body>\n
   s>     </html>\n
   s>     \n
   s>     \r\n
   s>     0\r\n
   s>     \r\n
 
   $ hg --verbose debugwireproto http://$LOCALIP:$HGPORT/redirector << EOF
   > command heads
   > EOF
   s>     GET /redirector?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 301 Redirect\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Location: http://$LOCALIP:$HGPORT/redirected\r\n (glob)
   s>     Content-Type: text/plain\r\n
   s>     Content-Length: 10\r\n
   s>     \r\n
   s>     redirected
   s>     GET /redirected HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     ETag: W/"*"\r\n (glob)
   s>     Content-Type: text/html; charset=ascii\r\n
   s>     Transfer-Encoding: chunked\r\n
   s>     \r\n
   real URL is http://$LOCALIP:$HGPORT/redirected (glob)
   s>     414\r\n
   s>     <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">\n
   s>     <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US">\n
   s>     <head>\n
   s>     <link rel="icon" href="/redirected/static/hgicon.png" type="image/png" />\n
   s>     <meta name="robots" content="index, nofollow" />\n
   s>     <link rel="stylesheet" href="/redirected/static/style-paper.css" type="text/css" />\n
   s>     <script type="text/javascript" src="/redirected/static/mercurial.js"></script>\n
   s>     \n
   s>     <title>redirected: log</title>\n
   s>     <link rel="alternate" type="application/atom+xml"\n
   s>        href="/redirected/atom-log" title="Atom feed for redirected" />\n
   s>     <link rel="alternate" type="application/rss+xml"\n
   s>        href="/redirected/rss-log" title="RSS feed for redirected" />\n
   s>     </head>\n
   s>     <body>\n
   s>     \n
   s>     <div class="container">\n
   s>     <div class="menu">\n
   s>     <div class="logo">\n
   s>     <a href="https://mercurial-scm.org/">\n
   s>     <img src="/redirected/static/hglogo.png" alt="mercurial" /></a>\n
   s>     </div>\n
   s>     <ul>\n
   s>     <li class="active">log</li>\n
   s>     <li><a href="/redirected/graph/tip">graph</a></li>\n
   s>     <li><a href="/redirected/tags">tags</a
   s>     GET /redirected?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 467\r\n
   s>     \r\n
   real URL is http://$LOCALIP:$HGPORT/redirected (glob)
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending heads command
   s>     GET /redirected?cmd=heads HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1\r\n
   s>     x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: 41\r\n
   s>     \r\n
   s>     96ee1d7354c4ad7372047672c36a1f561e3a6a4c\n
   response: [
     b'\x96\xee\x1dsT\xc4\xadsr\x04vr\xc3j\x1fV\x1e:jL'
   ]

tests/test-wireproto-command-capabilities.t

@@ -1,544 +1,628 b''
 #require no-chg
 
   $ . $TESTDIR/wireprotohelpers.sh
 
   $ hg init server
 
 zstd isn't present in plain builds. Make tests easier by removing
 zstd from the equation.
 
   $ cat >> server/.hg/hgrc << EOF
   > [server]
   > compressionengines = zlib
   > EOF
 
   $ hg -R server serve -p $HGPORT -d --pid-file hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 A normal capabilities request is serviced for version 1
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >     user-agent: test
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
 
 A proper request without the API server enabled returns the legacy response
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: foo
   >    x-hgproto-1: cbor
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: foo\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
 
 Restart with just API server enabled. This enables serving the new format.
 
   $ killdaemons.py
   $ cat error.log
 
   $ cat >> server/.hg/hgrc << EOF
   > [experimental]
   > web.apiserver = true
   > EOF
 
   $ hg -R server serve -p $HGPORT -d --pid-file hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 X-HgUpgrade-<N> without CBOR advertisement uses legacy response
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: foo bar
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgupgrade-1: foo bar\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
 
 X-HgUpgrade-<N> without known serialization in X-HgProto-<N> uses legacy response
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: foo bar
   >    x-hgproto-1: some value
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgproto-1: some value\r\n
   s>     x-hgupgrade-1: foo bar\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 Script output follows\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-0.1\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
 
 X-HgUpgrade-<N> + X-HgProto-<N> headers trigger new response format
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: foo bar
   >    x-hgproto-1: cbor
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: foo bar\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-cbor\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     \xa3GapibaseDapi/Dapis\xa0Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   cbor> {
     b'apibase': b'api/',
     b'apis': {},
     b'v1capabilities': b'batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash'
   }
 
 Restart server to enable HTTPv2
 
   $ killdaemons.py
   $ enablehttpv2 server
   $ hg -R server serve -p $HGPORT -d --pid-file hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 Only requested API services are returned
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: foo bar
   >    x-hgproto-1: cbor
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: foo bar\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-cbor\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
   s>     \xa3GapibaseDapi/Dapis\xa0Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   cbor> {
     b'apibase': b'api/',
     b'apis': {},
     b'v1capabilities': b'batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash'
   }
 
 Request for HTTPv2 service returns information about it
 
   $ sendhttpraw << EOF
   > httprequest GET ?cmd=capabilities
   >    user-agent: test
   >    x-hgupgrade-1: exp-http-v2-0001 foo bar
   >    x-hgproto-1: cbor
   > EOF
   using raw connection to peer
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     user-agent: test\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: exp-http-v2-0001 foo bar\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-cbor\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
-  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xd9\x01\x02\x82GparentsHrevisionInoderange\x82\x81J0123456...\x81Iabcdef...Enodes\x81J0123456...Kpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...DpathGfoo.txtKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...Dtree@Kpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
+  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetInoderange\xa3Gdefault\xf6Hrequired\xf4DtypeDlistEnodes\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullGpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   cbor> {
     b'apibase': b'api/',
     b'apis': {
       b'exp-http-v2-0001': {
         b'commands': {
           b'branchmap': {
             b'args': {},
             b'permissions': [
               b'pull'
             ]
           },
           b'capabilities': {
             b'args': {},
             b'permissions': [
               b'pull'
             ]
           },
           b'changesetdata': {
             b'args': {
-              b'fields': set([
-                b'parents',
-                b'revision'
-              ]),
-              b'noderange': [
-                [
-                  b'0123456...'
-                ],
-                [
-                  b'abcdef...'
-                ]
-              ],
-              b'nodes': [
-                b'0123456...'
-              ]
+              b'fields': {
+                b'default': set([]),
+                b'required': False,
+                b'type': b'set'
+              },
+              b'noderange': {
+                b'default': None,
+                b'required': False,
+                b'type': b'list'
+              },
+              b'nodes': {
+                b'default': None,
+                b'required': False,
+                b'type': b'list'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'filedata': {
             b'args': {
-              b'fields': set([
-                b'parents',
-                b'revision'
-              ]),
-              b'haveparents': True,
-              b'nodes': [
-                b'0123456...'
-              ],
-              b'path': b'foo.txt'
+              b'fields': {
+                b'default': set([]),
+                b'required': False,
+                b'type': b'set'
+              },
+              b'haveparents': {
+                b'default': False,
+                b'required': False,
+                b'type': b'bool'
+              },
+              b'nodes': {
+                b'required': True,
+                b'type': b'list'
+              },
+              b'path': {
+                b'required': True,
+                b'type': b'bytes'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'heads': {
             b'args': {
-              b'publiconly': False
+              b'publiconly': {
+                b'default': False,
+                b'required': False,
+                b'type': b'bool'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'known': {
             b'args': {
-              b'nodes': [
-                b'deadbeef'
-              ]
+              b'nodes': {
+                b'default': [],
+                b'required': False,
+                b'type': b'list'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'listkeys': {
             b'args': {
-              b'namespace': b'ns'
+              b'namespace': {
+                b'required': True,
+                b'type': b'bytes'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'lookup': {
             b'args': {
-              b'key': b'foo'
+              b'key': {
+                b'required': True,
+                b'type': b'bytes'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'manifestdata': {
             b'args': {
-              b'fields': set([
-                b'parents',
-                b'revision'
-              ]),
-              b'haveparents': True,
-              b'nodes': [
-                b'0123456...'
-              ],
-              b'tree': b''
+              b'fields': {
+                b'default': set([]),
+                b'required': False,
+                b'type': b'set'
+              },
+              b'haveparents': {
+                b'default': False,
+                b'required': False,
+                b'type': b'bool'
+              },
+              b'nodes': {
+                b'required': True,
+                b'type': b'list'
+              },
+              b'tree': {
+                b'required': True,
+                b'type': b'bytes'
+              }
             },
             b'permissions': [
               b'pull'
             ]
           },
           b'pushkey': {
             b'args': {
-              b'key': b'key',
-              b'namespace': b'ns',
-              b'new': b'new',
-              b'old': b'old'
+              b'key': {
+                b'required': True,
+                b'type': b'bytes'
+              },
+              b'namespace': {
+                b'required': True,
+                b'type': b'bytes'
+              },
+              b'new': {
+                b'required': True,
+                b'type': b'bytes'
+              },
+              b'old': {
+                b'required': True,
+                b'type': b'bytes'
+              }
             },
             b'permissions': [
               b'push'
             ]
           }
         },
         b'compression': [
           {
             b'name': b'zlib'
           }
         ],
         b'framingmediatypes': [
           b'application/mercurial-exp-framing-0005'
         ],
         b'pathfilterprefixes': set([
           b'path:',
           b'rootfilesin:'
         ]),
         b'rawrepoformats': [
           b'generaldelta',
           b'revlogv1'
         ]
       }
     },
     b'v1capabilities': b'batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash'
   }
 
 capabilities command returns expected info
 
   $ sendhttpv2peerhandshake << EOF
   > command capabilities
   > EOF
   creating http peer for wire protocol version 2
   s>     GET /?cmd=capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     vary: X-HgProto-1,X-HgUpgrade-1\r\n
   s>     x-hgproto-1: cbor\r\n
   s>     x-hgupgrade-1: exp-http-v2-0001\r\n
   s>     accept: application/mercurial-0.1\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-cbor\r\n
   s>     Content-Length: *\r\n (glob)
   s>     \r\n
-  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xd9\x01\x02\x82GparentsHrevisionInoderange\x82\x81J0123456...\x81Iabcdef...Enodes\x81J0123456...Kpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...DpathGfoo.txtKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...Dtree@Kpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
+  s>     \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0001\xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetInoderange\xa3Gdefault\xf6Hrequired\xf4DtypeDlistEnodes\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullGpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Nv1capabilitiesY\x01\xd3batch branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
   sending capabilities command
   s>     POST /api/exp-http-v2-0001/ro/capabilities HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
   s>     accept: application/mercurial-exp-framing-0005\r\n
   s>     content-type: application/mercurial-exp-framing-0005\r\n
   s>     content-length: 27\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     user-agent: Mercurial debugwireproto\r\n
   s>     \r\n
   s>     \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities
   s> makefile('rb', None)
   s>     HTTP/1.1 200 OK\r\n
   s>     Server: testing stub value\r\n
   s>     Date: $HTTP_DATE$\r\n
   s>     Content-Type: application/mercurial-exp-framing-0005\r\n
   s>     Transfer-Encoding: chunked\r\n
   s>     \r\n
   s>     13\r\n
   s>     \x0b\x00\x00\x01\x00\x02\x011
   s>     \xa1FstatusBok
   s>     \r\n
   received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
-  s>     33e\r\n
-  s>     6\x03\x00\x01\x00\x02\x001
-  s>     \xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xd9\x01\x02\x82GparentsHrevisionInoderange\x82\x81J0123456...\x81Iabcdef...Enodes\x81J0123456...Kpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...DpathGfoo.txtKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xf5Enodes\x81J0123456...Dtree@Kpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1
+  s>     485\r\n
+  s>     }\x04\x00\x01\x00\x02\x001
+  s>     \xa5Hcommands\xaaIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa3Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetInoderange\xa3Gdefault\xf6Hrequired\xf4DtypeDlistEnodes\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa2Dargs\xa4Ffields\xa3Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullGpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1
   s>     \r\n
-  received frame(size=822; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
+  received frame(size=1149; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
   s>     8\r\n
   s>     \x00\x00\x00\x01\x00\x02\x002
   s>     \r\n
   s>     0\r\n
   s>     \r\n
   received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
   response: gen[
     {
       b'commands': {
         b'branchmap': {
           b'args': {},
           b'permissions': [
             b'pull'
           ]
         },
         b'capabilities': {
           b'args': {},
           b'permissions': [
             b'pull'
           ]
         },
         b'changesetdata': {
           b'args': {
-            b'fields': set([
-              b'parents',
-              b'revision'
-            ]),
-            b'noderange': [
-              [
-                b'0123456...'
-              ],
-              [
-                b'abcdef...'
-              ]
-            ],
-            b'nodes': [
-              b'0123456...'
-            ]
+            b'fields': {
+              b'default': set([]),
+              b'required': False,
+              b'type': b'set'
+            },
+            b'noderange': {
+              b'default': None,
+              b'required': False,
+              b'type': b'list'
+            },
+            b'nodes': {
+              b'default': None,
+              b'required': False,
+              b'type': b'list'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'filedata': {
           b'args': {
-            b'fields': set([
-              b'parents',
-              b'revision'
-            ]),
-            b'haveparents': True,
-            b'nodes': [
-              b'0123456...'
-            ],
-            b'path': b'foo.txt'
+            b'fields': {
+              b'default': set([]),
+              b'required': False,
+              b'type': b'set'
+            },
+            b'haveparents': {
+              b'default': False,
+              b'required': False,
+              b'type': b'bool'
+            },
+            b'nodes': {
+              b'required': True,
+              b'type': b'list'
+            },
+            b'path': {
+              b'required': True,
+              b'type': b'bytes'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'heads': {
           b'args': {
-            b'publiconly': False
+            b'publiconly': {
+              b'default': False,
+              b'required': False,
+              b'type': b'bool'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'known': {
           b'args': {
-            b'nodes': [
-              b'deadbeef'
-            ]
+            b'nodes': {
+              b'default': [],
+              b'required': False,
+              b'type': b'list'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'listkeys': {
           b'args': {
-            b'namespace': b'ns'
+            b'namespace': {
+              b'required': True,
+              b'type': b'bytes'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'lookup': {
           b'args': {
-            b'key': b'foo'
+            b'key': {
+              b'required': True,
+              b'type': b'bytes'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'manifestdata': {
           b'args': {
-            b'fields': set([
-              b'parents',
-              b'revision'
-            ]),
-            b'haveparents': True,
-            b'nodes': [
-              b'0123456...'
-            ],
-            b'tree': b''
+            b'fields': {
+              b'default': set([]),
+              b'required': False,
+              b'type': b'set'
+            },
+            b'haveparents': {
+              b'default': False,
+              b'required': False,
+              b'type': b'bool'
+            },
+            b'nodes': {
+              b'required': True,
+              b'type': b'list'
+            },
+            b'tree': {
+              b'required': True,
+              b'type': b'bytes'
+            }
           },
           b'permissions': [
             b'pull'
           ]
         },
         b'pushkey': {
           b'args': {
-            b'key': b'key',
-            b'namespace': b'ns',
-            b'new': b'new',
-            b'old': b'old'
+            b'key': {
+              b'required': True,
+              b'type': b'bytes'
+            },
+            b'namespace': {
+              b'required': True,
+              b'type': b'bytes'
+            },
+            b'new': {
+              b'required': True,
+              b'type': b'bytes'
+            },
+            b'old': {
+              b'required': True,
+              b'type': b'bytes'
+            }
           },
           b'permissions': [
             b'push'
           ]
         }
       },
       b'compression': [
         {
           b'name': b'zlib'
         }
       ],
       b'framingmediatypes': [
         b'application/mercurial-exp-framing-0005'
       ],
       b'pathfilterprefixes': set([
         b'path:',
         b'rootfilesin:'
       ]),
       b'rawrepoformats': [
         b'generaldelta',
         b'revlogv1'
       ]
     }
   ]
 
   $ cat error.log