bundle2: a very first version of bundle2 unbundler

This changeset introduce an unbundler class to match the bundle2 bundler. It is
currently able to unbundle an empty bundle2 only and will gain more feature at
the same pace than the bundler.

It also comes with its special extension command in test.

mercurial/bundle2.py

@@ -1,84 +1,133 b''
 # bundle2.py - generic container format to transmit arbitrary data.
 #
 # Copyright 2013 Facebook, Inc.
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 """Handling of the new bundle2 format
 
 The goal of bundle2 is to act as an atomically packet to transmit a set of
 payloads in an application agnostic way. It consist in a sequence of "parts"
 that will be handed to and processed by the application layer.
 
 
 General format architecture
 ===========================
 
 The format is architectured as follow
 
  - magic string
  - stream level parameters
  - payload parts (any number)
  - end of stream marker.
 
 The current implementation is limited to empty bundle.
 
 Details on the Binary format
 ============================
 
 All numbers are unsigned and big endian.
 
 stream level parameters
 ------------------------
 
 Binary format is as follow
 
 :params size: (16 bits integer)
 
   The total number of Bytes used by the parameters
 
   Currently force to 0.
 
 :params value: arbitrary number of Bytes
 
   A blob of `params size` containing the serialized version of all stream level
   parameters.
 
   Currently always empty.
 
 
 Payload part
 ------------------------
 
 Binary format is as follow
 
 :header size: (16 bits inter)
 
   The total number of Bytes used by the part headers. When the header is empty
   (size = 0) this is interpreted as the end of stream marker.
 
   Currently forced to 0 in the current state of the implementation
 """
 
+import util
+import changegroup
+
+
 _magicstring = 'HG20'
 
 class bundle20(object):
     """represent an outgoing bundle2 container
 
     People will eventually be able to add param and parts to this object and
     generated a stream from it."""
 
     def __init__(self):
         self._params = []
         self._parts = []
 
     def getchunks(self):
         yield _magicstring
         # no support for any param yet
         # to be obviously fixed soon.
         assert not self._params
         yield '\0\0'
         # no support for parts
         # to be obviously fixed soon.
         assert not self._parts
         yield '\0\0'
+
+class unbundle20(object):
+    """interpret a bundle2 stream
+
+    (this will eventually yield parts)"""
+
+    def __init__(self, fp):
+        # assume the magic string is ok and drop it
+        # to be obviously fixed soon.
+        self._fp = fp
+        self._readexact(4)
+
+    def _unpack(self, format):
+        """unpack this struct format from the stream"""
+        data = self._readexact(struct.calcsize(format))
+        return _unpack(format, data)
+
+    def _readexact(self, size):
+        """read exactly <size> bytes from the stream"""
+        return changegroup.readexactly(self._fp, size)
+
+    @util.propertycache
+    def params(self):
+        """dictionnary of stream level parameters"""
+        paramsize = self._readexact(2)
+        assert paramsize == '\0\0'
+        return {}
+
+    def __iter__(self):
+        """yield all parts contained in the stream"""
+        # make sure param have been loaded
+        self.params
+        part = self._readpart()
+        while part is not None:
+            yield part
+            part = self._readpart()
+
+    def _readpart(self):
+        """return None when an end of stream markers is reach"""
+        headersize = self._readexact(2)
+        assert headersize == '\0\0'
+        return None
+
+
+

tests/test-bundle2.t

@@ -1,36 +1,51 b''
 
 Create an extension to test bundle2 API
 
   $ cat > bundle2.py << EOF
   > """A small extension to test bundle2 implementation
   > 
   > Current bundle2 implementation is far too limited to be used in any core
   > code. We still need to be able to test it while it grow up.
   > """
   > 
+  > import sys
   > from mercurial import cmdutil
   > from mercurial import bundle2
   > cmdtable = {}
   > command = cmdutil.command(cmdtable)
   > 
   > @command('bundle2', [], '')
   > def cmdbundle2(ui, repo):
   >     """write a bundle2 container on standard ouput"""
   >     bundle = bundle2.bundle20()
   >     for chunk in bundle.getchunks():
   >         ui.write(chunk)
+  > 
+  > @command('unbundle2', [], '')
+  > def cmdunbundle2(ui, repo):
+  >     """read a bundle2 container from standard input"""
+  >     unbundler = bundle2.unbundle20(sys.stdin)
+  >     ui.write('options count: %i\n' % len(unbundler.params))
+  >     parts = list(unbundler)
+  >     ui.write('parts count:   %i\n' % len(parts))
   > EOF
   $ cat >> $HGRCPATH << EOF
   > [extensions]
   > bundle2=$TESTTMP/bundle2.py
   > EOF
 
 The extension requires a repo (currently unused)
 
   $ hg init main
   $ cd main
 
 Test simple generation of empty bundle
 
   $ hg bundle2
   HG20\x00\x00\x00\x00 (no-eol) (esc)
+
+Test parsing of an empty bundle
+
+  $ hg bundle2 | hg unbundle2
+  options count: 0
+  parts count:   0