wireprotov2: add bookmarks to "changesetdata" command

Like we did for phases, we want to emit bookmarks data attached
to each changeset.

The approach here is very similar to phases: we emit bookmarks
data inline with requested revision data. But we emit
records for nodes that weren't requested as well so consumers have
access to the full set of defined bookmarks.

Differential Revision: https://phab.mercurial-scm.org/D4485

mercurial/help/internals/wireprotocolv2.txt

@@ -110,6 +110,9 b' fields'
    (set of bytestring) Which data associated with changelog revisions to
    fetch. The following values are recognized:
 
+   bookmarks
+      Bookmarks associated with a revision.
+
    parents
       Parent revisions.
 
@@ -144,6 +147,11 b' node'
    (bytestring) The node value for this revision. This is the SHA-1 hash of
    the raw revision data.
 
+bookmarks (optional)
+   (array of bytestrings) Bookmarks attached to this revision. Only present
+   if ``bookmarks`` data is being requested and the revision has bookmarks
+   attached.
+
 parents (optional)
    (array of bytestrings) The nodes representing the parent revisions of this
    revision. Only present if ``parents`` data is being requested.
@@ -182,7 +190,6 b' to reflect a phase-only update.'
 TODO support different revision selection mechanisms (e.g. non-public, specific
 revisions)
 TODO support different hash "namespaces" for revisions (e.g. sha-1 versus other)
-TODO support emitting bookmarks data
 TODO support emitting obsolescence data
 TODO support filtering based on relevant paths (narrow clone)
 TODO support depth limiting

mercurial/wireprotov2server.py

@@ -533,6 +533,10 b' def changesetdata(repo, proto, noderange'
                 b'phase': b'public' if publishing else repo[node].phasestr()
             }
 
+    nodebookmarks = {}
+    for mark, node in repo._bookmarks.items():
+        nodebookmarks.setdefault(node, set()).add(mark)
+
     # It is already topologically sorted by revision number.
     for node in outgoing:
         d = {
@@ -549,6 +553,10 b' def changesetdata(repo, proto, noderange'
                 ctx = repo[node]
                 d[b'phase'] = ctx.phasestr()
 
+        if b'bookmarks' in fields and node in nodebookmarks:
+            d[b'bookmarks'] = sorted(nodebookmarks[node])
+            del nodebookmarks[node]
+
         revisiondata = None
 
         if b'revision' in fields:
@@ -560,6 +568,15 b' def changesetdata(repo, proto, noderange'
         if revisiondata is not None:
             yield revisiondata
 
+    # If requested, send bookmarks from nodes that didn't have revision
+    # data sent so receiver is aware of any bookmark updates.
+    if b'bookmarks' in fields:
+        for node, marks in sorted(nodebookmarks.iteritems()):
+            yield {
+                b'node': node,
+                b'bookmarks': sorted(marks),
+            }
+
 @wireprotocommand('heads',
                   args={
                       'publiconly': False,

tests/test-wireproto-command-changesetdata.t

@@ -478,6 +478,213 b' Revision data is transferred upon reques'
     b'1b74476799ec8318045db759b1b4bcc9b839d0aa\ntest\n0 0\na\n\ncommit 3'
   ]
 
+Bookmarks key isn't present if no bookmarks data
+
+  $ sendhttpv2peer << EOF
+  > command changesetdata
+  >     fields eval:[b'bookmarks']
+  >     noderange eval:[[], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
+  > EOF
+  creating http peer for wire protocol version 2
+  sending changesetdata command
+  s>     POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-exp-framing-0005\r\n
+  s>     content-type: application/mercurial-exp-framing-0005\r\n
+  s>     content-length: 107\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     user-agent: Mercurial debugwireproto\r\n
+  s>     \r\n
+  s>     c\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x81IbookmarksInoderange\x82\x80\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
+  s> makefile('rb', None)
+  s>     HTTP/1.1 200 OK\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: application/mercurial-exp-framing-0005\r\n
+  s>     Transfer-Encoding: chunked\r\n
+  s>     \r\n
+  s>     13\r\n
+  s>     \x0b\x00\x00\x01\x00\x02\x011
+  s>     \xa1FstatusBok
+  s>     \r\n
+  received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
+  s>     81\r\n
+  s>     y\x00\x00\x01\x00\x02\x001
+  s>     \xa1Jtotalitems\x04\xa1DnodeT3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:\xa1DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1\xa1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd\xa1DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
+  s>     \r\n
+  received frame(size=121; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
+  s>     8\r\n
+  s>     \x00\x00\x00\x01\x00\x02\x002
+  s>     \r\n
+  s>     0\r\n
+  s>     \r\n
+  received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
+  response: gen[
+    {
+      b'totalitems': 4
+    },
+    {
+      b'node': b'3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:'
+    },
+    {
+      b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1'
+    },
+    {
+      b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd'
+    },
+    {
+      b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
+    }
+  ]
+
+Bookmarks are sent when requested
+
+  $ hg -R ../server bookmark -r 0bb8ad894a15b15380b2a2a5b183e20f2a4b28dd book-1
+  $ hg -R ../server bookmark -r eae5f82c2e622368d27daecb76b7e393d0f24211 book-2
+  $ hg -R ../server bookmark -r eae5f82c2e622368d27daecb76b7e393d0f24211 book-3
+
+  $ sendhttpv2peer << EOF
+  > command changesetdata
+  >     fields eval:[b'bookmarks']
+  >     noderange eval:[[], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
+  > EOF
+  creating http peer for wire protocol version 2
+  sending changesetdata command
+  s>     POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-exp-framing-0005\r\n
+  s>     content-type: application/mercurial-exp-framing-0005\r\n
+  s>     content-length: 107\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     user-agent: Mercurial debugwireproto\r\n
+  s>     \r\n
+  s>     c\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x81IbookmarksInoderange\x82\x80\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
+  s> makefile('rb', None)
+  s>     HTTP/1.1 200 OK\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: application/mercurial-exp-framing-0005\r\n
+  s>     Transfer-Encoding: chunked\r\n
+  s>     \r\n
+  s>     13\r\n
+  s>     \x0b\x00\x00\x01\x00\x02\x011
+  s>     \xa1FstatusBok
+  s>     \r\n
+  received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
+  s>     ac\r\n
+  s>     \xa4\x00\x00\x01\x00\x02\x001
+  s>     \xa1Jtotalitems\x04\xa1DnodeT3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:\xa1DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1\xa2Ibookmarks\x81Fbook-1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd\xa2Ibookmarks\x82Fbook-2Fbook-3DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
+  s>     \r\n
+  received frame(size=164; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
+  s>     8\r\n
+  s>     \x00\x00\x00\x01\x00\x02\x002
+  s>     \r\n
+  s>     0\r\n
+  s>     \r\n
+  received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
+  response: gen[
+    {
+      b'totalitems': 4
+    },
+    {
+      b'node': b'3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:'
+    },
+    {
+      b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1'
+    },
+    {
+      b'bookmarks': [
+        b'book-1'
+      ],
+      b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd'
+    },
+    {
+      b'bookmarks': [
+        b'book-2',
+        b'book-3'
+      ],
+      b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
+    }
+  ]
+
+Bookmarks are sent when we make a no-new-revisions request
+
+  $ sendhttpv2peer << EOF
+  > command changesetdata
+  >     fields eval:[b'bookmarks', b'revision']
+  >     noderange eval:[[b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11'], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
+  > EOF
+  creating http peer for wire protocol version 2
+  sending changesetdata command
+  s>     POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-exp-framing-0005\r\n
+  s>     content-type: application/mercurial-exp-framing-0005\r\n
+  s>     content-length: 137\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     user-agent: Mercurial debugwireproto\r\n
+  s>     \r\n
+  s>     \x81\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x82IbookmarksHrevisionInoderange\x82\x81T\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
+  s> makefile('rb', None)
+  s>     HTTP/1.1 200 OK\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: application/mercurial-exp-framing-0005\r\n
+  s>     Transfer-Encoding: chunked\r\n
+  s>     \r\n
+  s>     13\r\n
+  s>     \x0b\x00\x00\x01\x00\x02\x011
+  s>     \xa1FstatusBok
+  s>     \r\n
+  received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
+  s>     12f\r\n
+  s>     \'\x01\x00\x01\x00\x02\x001
+  s>     \xa1Jtotalitems\x02\xa2DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1Lrevisionsize\x18?X?7f144aea0ba742713887b564d57e9d12f12ff382\n
+  s>     test\n
+  s>     0 0\n
+  s>     a\n
+  s>     b\n
+  s>     \n
+  s>     commit 1\xa3Ibookmarks\x81Fbook-1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddLrevisionsize\x18=X=37f0a2d1c28ffe4b879109a7d1bbf8f07b3c763b\n
+  s>     test\n
+  s>     0 0\n
+  s>     b\n
+  s>     \n
+  s>     commit 2\xa2Ibookmarks\x82Fbook-2Fbook-3DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
+  s>     \r\n
+  received frame(size=295; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
+  s>     8\r\n
+  s>     \x00\x00\x00\x01\x00\x02\x002
+  s>     \r\n
+  s>     0\r\n
+  s>     \r\n
+  received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
+  response: gen[
+    {
+      b'totalitems': 2
+    },
+    {
+      b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1',
+      b'revisionsize': 63
+    },
+    b'7f144aea0ba742713887b564d57e9d12f12ff382\ntest\n0 0\na\nb\n\ncommit 1',
+    {
+      b'bookmarks': [
+        b'book-1'
+      ],
+      b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd',
+      b'revisionsize': 61
+    },
+    b'37f0a2d1c28ffe4b879109a7d1bbf8f07b3c763b\ntest\n0 0\nb\n\ncommit 2',
+    {
+      b'bookmarks': [
+        b'book-2',
+        b'book-3'
+      ],
+      b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
+    }
+  ]
+
 Multiple fields can be transferred
 
   $ sendhttpv2peer << EOF