changegroups.txt
188 lines
| 8.2 KiB
| text/plain
|
TextLexer
|
r27372 | Changegroups are representations of repository revlog data, specifically | ||
|
r31213 | the changelog data, root/flat manifest data, treemanifest data, and | ||
filelogs. | ||||
|
r27372 | |||
|
r27434 | There are 3 versions of changegroups: ``1``, ``2``, and ``3``. From a | ||
|
r31213 | high-level, versions ``1`` and ``2`` are almost exactly the same, with the | ||
only difference being an additional item in the *delta header*. Version | ||||
``3`` adds support for revlog flags in the *delta header* and optionally | ||||
exchanging treemanifests (enabled by setting an option on the | ||||
``changegroup`` part in the bundle2). | ||||
|
r27372 | |||
|
r31213 | Changegroups when not exchanging treemanifests consist of 3 logical | ||
segments:: | ||||
|
r27372 | |||
+---------------------------------+ | ||||
| | | | | ||||
| changeset | manifest | filelogs | | ||||
| | | | | ||||
|
r31213 | | | | | | ||
|
r27372 | +---------------------------------+ | ||
|
r31213 | When exchanging treemanifests, there are 4 logical segments:: | ||
+-------------------------------------------------+ | ||||
| | | | | | ||||
| changeset | root | treemanifests | filelogs | | ||||
| | manifest | | | | ||||
| | | | | | ||||
+-------------------------------------------------+ | ||||
|
r27372 | The principle building block of each segment is a *chunk*. A *chunk* | ||
is a framed piece of data:: | ||||
+---------------------------------------+ | ||||
| | | | ||||
| length | data | | ||||
|
r31213 | | (4 bytes) | (<length - 4> bytes) | | ||
|
r27372 | | | | | ||
+---------------------------------------+ | ||||
|
r31213 | All integers are big-endian signed integers. Each chunk starts with a 32-bit | ||
integer indicating the length of the entire chunk (including the length field | ||||
itself). | ||||
|
r27372 | |||
|
r31213 | There is a special case chunk that has a value of 0 for the length | ||
(``0x00000000``). We call this an *empty chunk*. | ||||
|
r27372 | |||
Delta Groups | ||||
|
r29747 | ============ | ||
|
r27372 | |||
A *delta group* expresses the content of a revlog as a series of deltas, | ||||
or patches against previous revisions. | ||||
Delta groups consist of 0 or more *chunks* followed by the *empty chunk* | ||||
to signal the end of the delta group:: | ||||
+------------------------------------------------------------------------+ | ||||
| | | | | | | ||||
| chunk0 length | chunk0 data | chunk1 length | chunk1 data | 0x0 | | ||||
|
r31213 | | (4 bytes) | (various) | (4 bytes) | (various) | (4 bytes) | | ||
|
r27372 | | | | | | | | ||
|
r31213 | +------------------------------------------------------------------------+ | ||
|
r27372 | |||
Each *chunk*'s data consists of the following:: | ||||
|
r31213 | +---------------------------------------+ | ||
| | | | ||||
| delta header | delta data | | ||||
| (various by version) | (various) | | ||||
| | | | ||||
+---------------------------------------+ | ||||
|
r27372 | |||
|
r31213 | The *delta data* is a series of *delta*s that describe a diff from an existing | ||
entry (either that the recipient already has, or previously specified in the | ||||
bundlei/changegroup). | ||||
|
r27372 | |||
|
r27434 | The *delta header* is different between versions ``1``, ``2``, and | ||
``3`` of the changegroup format. | ||||
|
r27372 | |||
|
r31213 | Version 1 (headerlen=80):: | ||
|
r27372 | |||
+------------------------------------------------------+ | ||||
| | | | | | ||||
| node | p1 node | p2 node | link node | | ||||
| (20 bytes) | (20 bytes) | (20 bytes) | (20 bytes) | | ||||
| | | | | | ||||
+------------------------------------------------------+ | ||||
|
r31213 | Version 2 (headerlen=100):: | ||
|
r27372 | |||
+------------------------------------------------------------------+ | ||||
| | | | | | | ||||
| node | p1 node | p2 node | base node | link node | | ||||
| (20 bytes) | (20 bytes) | (20 bytes) | (20 bytes) | (20 bytes) | | ||||
| | | | | | | ||||
+------------------------------------------------------------------+ | ||||
|
r31213 | Version 3 (headerlen=102):: | ||
|
r27434 | |||
+------------------------------------------------------------------------------+ | ||||
| | | | | | | | ||||
|
r31213 | | node | p1 node | p2 node | base node | link node | flags | | ||
|
r27434 | | (20 bytes) | (20 bytes) | (20 bytes) | (20 bytes) | (20 bytes) | (2 bytes) | | ||
| | | | | | | | ||||
+------------------------------------------------------------------------------+ | ||||
|
r31213 | The *delta data* consists of ``chunklen - 4 - headerlen`` bytes, which contain a | ||
series of *delta*s, densely packed (no separators). These deltas describe a diff | ||||
from an existing entry (either that the recipient already has, or previously | ||||
specified in the bundle/changegroup). The format is described more fully in | ||||
|
r31287 | ``hg help internals.bdiff``, but briefly:: | ||
|
r27372 | |||
|
r31213 | +---------------------------------------------------------------+ | ||
| | | | | | ||||
| start offset | end offset | new length | content | | ||||
| (4 bytes) | (4 bytes) | (4 bytes) | (<new length> bytes) | | ||||
| | | | | | ||||
+---------------------------------------------------------------+ | ||||
Please note that the length field in the delta data does *not* include itself. | ||||
|
r27372 | |||
In version 1, the delta is always applied against the previous node from | ||||
the changegroup or the first parent if this is the first entry in the | ||||
changegroup. | ||||
|
r31213 | In version 2 and up, the delta base node is encoded in the entry in the | ||
|
r27372 | changegroup. This allows the delta to be expressed against any parent, | ||
which can result in smaller deltas and more efficient encoding of data. | ||||
Changeset Segment | ||||
|
r29747 | ================= | ||
|
r27372 | |||
The *changeset segment* consists of a single *delta group* holding | ||||
|
r31213 | changelog data. The *empty chunk* at the end of the *delta group* denotes | ||
the boundary to the *manifest segment*. | ||||
|
r27372 | |||
Manifest Segment | ||||
|
r29747 | ================ | ||
|
r27372 | |||
|
r31213 | The *manifest segment* consists of a single *delta group* holding manifest | ||
data. If treemanifests are in use, it contains only the manifest for the | ||||
root directory of the repository. Otherwise, it contains the entire | ||||
manifest data. The *empty chunk* at the end of the *delta group* denotes | ||||
the boundary to the next segment (either the *treemanifests segment* or the | ||||
*filelogs segment*, depending on version and the request options). | ||||
Treemanifests Segment | ||||
--------------------- | ||||
The *treemanifests segment* only exists in changegroup version ``3``, and | ||||
only if the 'treemanifest' param is part of the bundle2 changegroup part | ||||
(it is not possible to use changegroup version 3 outside of bundle2). | ||||
Aside from the filenames in the *treemanifests segment* containing a | ||||
trailing ``/`` character, it behaves identically to the *filelogs segment* | ||||
(see below). The final sub-segment is followed by an *empty chunk* (logically, | ||||
a sub-segment with filename size 0). This denotes the boundary to the | ||||
*filelogs segment*. | ||||
|
r27372 | |||
Filelogs Segment | ||||
|
r29747 | ================ | ||
|
r27372 | |||
|
r31213 | The *filelogs segment* consists of multiple sub-segments, each | ||
|
r27372 | corresponding to an individual file whose data is being described:: | ||
|
r31213 | +--------------------------------------------------+ | ||
| | | | | | | ||||
| filelog0 | filelog1 | filelog2 | ... | 0x0 | | ||||
| | | | | (4 bytes) | | ||||
| | | | | | | ||||
+--------------------------------------------------+ | ||||
|
r27372 | |||
|
r31213 | The final filelog sub-segment is followed by an *empty chunk* (logically, | ||
a sub-segment with filename size 0). This denotes the end of the segment | ||||
and of the overall changegroup. | ||||
|
r27372 | |||
Each filelog sub-segment consists of the following:: | ||||
|
r31213 | +------------------------------------------------------+ | ||
| | | | | ||||
| filename length | filename | delta group | | ||||
| (4 bytes) | (<length - 4> bytes) | (various) | | ||||
| | | | | ||||
+------------------------------------------------------+ | ||||
|
r27372 | |||
That is, a *chunk* consisting of the filename (not terminated or padded) | ||||
|
r31213 | followed by N chunks constituting the *delta group* for this file. The | ||
*empty chunk* at the end of each *delta group* denotes the boundary to the | ||||
next filelog sub-segment. | ||||