Show More
@@ -1,466 +1,486 b'' | |||||
1 | # bundle2.py - generic container format to transmit arbitrary data. |
|
1 | # bundle2.py - generic container format to transmit arbitrary data. | |
2 | # |
|
2 | # | |
3 | # Copyright 2013 Facebook, Inc. |
|
3 | # Copyright 2013 Facebook, Inc. | |
4 | # |
|
4 | # | |
5 | # This software may be used and distributed according to the terms of the |
|
5 | # This software may be used and distributed according to the terms of the | |
6 | # GNU General Public License version 2 or any later version. |
|
6 | # GNU General Public License version 2 or any later version. | |
7 | """Handling of the new bundle2 format |
|
7 | """Handling of the new bundle2 format | |
8 |
|
8 | |||
9 | The goal of bundle2 is to act as an atomically packet to transmit a set of |
|
9 | The goal of bundle2 is to act as an atomically packet to transmit a set of | |
10 | payloads in an application agnostic way. It consist in a sequence of "parts" |
|
10 | payloads in an application agnostic way. It consist in a sequence of "parts" | |
11 | that will be handed to and processed by the application layer. |
|
11 | that will be handed to and processed by the application layer. | |
12 |
|
12 | |||
13 |
|
13 | |||
14 | General format architecture |
|
14 | General format architecture | |
15 | =========================== |
|
15 | =========================== | |
16 |
|
16 | |||
17 | The format is architectured as follow |
|
17 | The format is architectured as follow | |
18 |
|
18 | |||
19 | - magic string |
|
19 | - magic string | |
20 | - stream level parameters |
|
20 | - stream level parameters | |
21 | - payload parts (any number) |
|
21 | - payload parts (any number) | |
22 | - end of stream marker. |
|
22 | - end of stream marker. | |
23 |
|
23 | |||
24 | the Binary format |
|
24 | the Binary format | |
25 | ============================ |
|
25 | ============================ | |
26 |
|
26 | |||
27 | All numbers are unsigned and big endian. |
|
27 | All numbers are unsigned and big endian. | |
28 |
|
28 | |||
29 | stream level parameters |
|
29 | stream level parameters | |
30 | ------------------------ |
|
30 | ------------------------ | |
31 |
|
31 | |||
32 | Binary format is as follow |
|
32 | Binary format is as follow | |
33 |
|
33 | |||
34 | :params size: (16 bits integer) |
|
34 | :params size: (16 bits integer) | |
35 |
|
35 | |||
36 | The total number of Bytes used by the parameters |
|
36 | The total number of Bytes used by the parameters | |
37 |
|
37 | |||
38 | :params value: arbitrary number of Bytes |
|
38 | :params value: arbitrary number of Bytes | |
39 |
|
39 | |||
40 | A blob of `params size` containing the serialized version of all stream level |
|
40 | A blob of `params size` containing the serialized version of all stream level | |
41 | parameters. |
|
41 | parameters. | |
42 |
|
42 | |||
43 | The blob contains a space separated list of parameters. parameter with value |
|
43 | The blob contains a space separated list of parameters. parameter with value | |
44 | are stored in the form `<name>=<value>`. Both name and value are urlquoted. |
|
44 | are stored in the form `<name>=<value>`. Both name and value are urlquoted. | |
45 |
|
45 | |||
46 | Empty name are obviously forbidden. |
|
46 | Empty name are obviously forbidden. | |
47 |
|
47 | |||
48 | Name MUST start with a letter. If this first letter is lower case, the |
|
48 | Name MUST start with a letter. If this first letter is lower case, the | |
49 | parameter is advisory and can be safefly ignored. However when the first |
|
49 | parameter is advisory and can be safefly ignored. However when the first | |
50 | letter is capital, the parameter is mandatory and the bundling process MUST |
|
50 | letter is capital, the parameter is mandatory and the bundling process MUST | |
51 | stop if he is not able to proceed it. |
|
51 | stop if he is not able to proceed it. | |
52 |
|
52 | |||
53 | Stream parameters use a simple textual format for two main reasons: |
|
53 | Stream parameters use a simple textual format for two main reasons: | |
54 |
|
54 | |||
55 | - Stream level parameters should remains simple and we want to discourage any |
|
55 | - Stream level parameters should remains simple and we want to discourage any | |
56 | crazy usage. |
|
56 | crazy usage. | |
57 | - Textual data allow easy human inspection of a the bundle2 header in case of |
|
57 | - Textual data allow easy human inspection of a the bundle2 header in case of | |
58 | troubles. |
|
58 | troubles. | |
59 |
|
59 | |||
60 | Any Applicative level options MUST go into a bundle2 part instead. |
|
60 | Any Applicative level options MUST go into a bundle2 part instead. | |
61 |
|
61 | |||
62 | Payload part |
|
62 | Payload part | |
63 | ------------------------ |
|
63 | ------------------------ | |
64 |
|
64 | |||
65 | Binary format is as follow |
|
65 | Binary format is as follow | |
66 |
|
66 | |||
67 | :header size: (16 bits inter) |
|
67 | :header size: (16 bits inter) | |
68 |
|
68 | |||
69 | The total number of Bytes used by the part headers. When the header is empty |
|
69 | The total number of Bytes used by the part headers. When the header is empty | |
70 | (size = 0) this is interpreted as the end of stream marker. |
|
70 | (size = 0) this is interpreted as the end of stream marker. | |
71 |
|
71 | |||
72 | :header: |
|
72 | :header: | |
73 |
|
73 | |||
74 | The header defines how to interpret the part. It contains two piece of |
|
74 | The header defines how to interpret the part. It contains two piece of | |
75 | data: the part type, and the part parameters. |
|
75 | data: the part type, and the part parameters. | |
76 |
|
76 | |||
77 | The part type is used to route an application level handler, that can |
|
77 | The part type is used to route an application level handler, that can | |
78 | interpret payload. |
|
78 | interpret payload. | |
79 |
|
79 | |||
80 | Part parameters are passed to the application level handler. They are |
|
80 | Part parameters are passed to the application level handler. They are | |
81 | meant to convey information that will help the application level object to |
|
81 | meant to convey information that will help the application level object to | |
82 | interpret the part payload. |
|
82 | interpret the part payload. | |
83 |
|
83 | |||
84 | The binary format of the header is has follow |
|
84 | The binary format of the header is has follow | |
85 |
|
85 | |||
86 | :typesize: (one byte) |
|
86 | :typesize: (one byte) | |
87 |
|
87 | |||
88 | :typename: alphanumerical part name |
|
88 | :typename: alphanumerical part name | |
89 |
|
89 | |||
90 | :parameters: |
|
90 | :parameters: | |
91 |
|
91 | |||
92 | Part's parameter may have arbitraty content, the binary structure is:: |
|
92 | Part's parameter may have arbitraty content, the binary structure is:: | |
93 |
|
93 | |||
94 | <mandatory-count><advisory-count><param-sizes><param-data> |
|
94 | <mandatory-count><advisory-count><param-sizes><param-data> | |
95 |
|
95 | |||
96 | :mandatory-count: 1 byte, number of mandatory parameters |
|
96 | :mandatory-count: 1 byte, number of mandatory parameters | |
97 |
|
97 | |||
98 | :advisory-count: 1 byte, number of advisory parameters |
|
98 | :advisory-count: 1 byte, number of advisory parameters | |
99 |
|
99 | |||
100 | :param-sizes: |
|
100 | :param-sizes: | |
101 |
|
101 | |||
102 | N couple of bytes, where N is the total number of parameters. Each |
|
102 | N couple of bytes, where N is the total number of parameters. Each | |
103 | couple contains (<size-of-key>, <size-of-value) for one parameter. |
|
103 | couple contains (<size-of-key>, <size-of-value) for one parameter. | |
104 |
|
104 | |||
105 | :param-data: |
|
105 | :param-data: | |
106 |
|
106 | |||
107 | A blob of bytes from which each parameter key and value can be |
|
107 | A blob of bytes from which each parameter key and value can be | |
108 | retrieved using the list of size couples stored in the previous |
|
108 | retrieved using the list of size couples stored in the previous | |
109 | field. |
|
109 | field. | |
110 |
|
110 | |||
111 | Mandatory parameters comes first, then the advisory ones. |
|
111 | Mandatory parameters comes first, then the advisory ones. | |
112 |
|
112 | |||
113 | :payload: |
|
113 | :payload: | |
114 |
|
114 | |||
115 | payload is a series of `<chunksize><chunkdata>`. |
|
115 | payload is a series of `<chunksize><chunkdata>`. | |
116 |
|
116 | |||
117 | `chunksize` is a 32 bits integer, `chunkdata` are plain bytes (as much as |
|
117 | `chunksize` is a 32 bits integer, `chunkdata` are plain bytes (as much as | |
118 | `chunksize` says)` The payload part is concluded by a zero size chunk. |
|
118 | `chunksize` says)` The payload part is concluded by a zero size chunk. | |
119 |
|
119 | |||
120 | The current implementation always produces either zero or one chunk. |
|
120 | The current implementation always produces either zero or one chunk. | |
121 | This is an implementation limitation that will ultimatly be lifted. |
|
121 | This is an implementation limitation that will ultimatly be lifted. | |
122 |
|
122 | |||
123 | Bundle processing |
|
123 | Bundle processing | |
124 | ============================ |
|
124 | ============================ | |
125 |
|
125 | |||
126 | Each part is processed in order using a "part handler". Handler are registered |
|
126 | Each part is processed in order using a "part handler". Handler are registered | |
127 | for a certain part type. |
|
127 | for a certain part type. | |
128 |
|
128 | |||
129 | The matching of a part to its handler is case insensitive. The case of the |
|
129 | The matching of a part to its handler is case insensitive. The case of the | |
130 | part type is used to know if a part is mandatory or advisory. If the Part type |
|
130 | part type is used to know if a part is mandatory or advisory. If the Part type | |
131 | contains any uppercase char it is considered mandatory. When no handler is |
|
131 | contains any uppercase char it is considered mandatory. When no handler is | |
132 | known for a Mandatory part, the process is aborted and an exception is raised. |
|
132 | known for a Mandatory part, the process is aborted and an exception is raised. | |
133 | If the part is advisory and no handler is known, the part is ignored. When the |
|
133 | If the part is advisory and no handler is known, the part is ignored. When the | |
134 | process is aborted, the full bundle is still read from the stream to keep the |
|
134 | process is aborted, the full bundle is still read from the stream to keep the | |
135 | channel usable. But none of the part read from an abort are processed. In the |
|
135 | channel usable. But none of the part read from an abort are processed. In the | |
136 | future, dropping the stream may become an option for channel we do not care to |
|
136 | future, dropping the stream may become an option for channel we do not care to | |
137 | preserve. |
|
137 | preserve. | |
138 | """ |
|
138 | """ | |
139 |
|
139 | |||
140 | import util |
|
140 | import util | |
141 | import struct |
|
141 | import struct | |
142 | import urllib |
|
142 | import urllib | |
143 | import string |
|
143 | import string | |
144 |
|
144 | |||
145 | import changegroup |
|
145 | import changegroup | |
146 | from i18n import _ |
|
146 | from i18n import _ | |
147 |
|
147 | |||
148 | _pack = struct.pack |
|
148 | _pack = struct.pack | |
149 | _unpack = struct.unpack |
|
149 | _unpack = struct.unpack | |
150 |
|
150 | |||
151 | _magicstring = 'HG20' |
|
151 | _magicstring = 'HG20' | |
152 |
|
152 | |||
153 | _fstreamparamsize = '>H' |
|
153 | _fstreamparamsize = '>H' | |
154 | _fpartheadersize = '>H' |
|
154 | _fpartheadersize = '>H' | |
155 | _fparttypesize = '>B' |
|
155 | _fparttypesize = '>B' | |
156 | _fpayloadsize = '>I' |
|
156 | _fpayloadsize = '>I' | |
157 | _fpartparamcount = '>BB' |
|
157 | _fpartparamcount = '>BB' | |
158 |
|
158 | |||
159 | def _makefpartparamsizes(nbparams): |
|
159 | def _makefpartparamsizes(nbparams): | |
160 | """return a struct format to read part parameter sizes |
|
160 | """return a struct format to read part parameter sizes | |
161 |
|
161 | |||
162 | The number parameters is variable so we need to build that format |
|
162 | The number parameters is variable so we need to build that format | |
163 | dynamically. |
|
163 | dynamically. | |
164 | """ |
|
164 | """ | |
165 | return '>'+('BB'*nbparams) |
|
165 | return '>'+('BB'*nbparams) | |
166 |
|
166 | |||
167 | parthandlermapping = {} |
|
167 | parthandlermapping = {} | |
168 |
|
168 | |||
169 | def parthandler(parttype): |
|
169 | def parthandler(parttype): | |
170 | """decorator that register a function as a bundle2 part handler |
|
170 | """decorator that register a function as a bundle2 part handler | |
171 |
|
171 | |||
172 | eg:: |
|
172 | eg:: | |
173 |
|
173 | |||
174 | @parthandler('myparttype') |
|
174 | @parthandler('myparttype') | |
175 | def myparttypehandler(...): |
|
175 | def myparttypehandler(...): | |
176 | '''process a part of type "my part".''' |
|
176 | '''process a part of type "my part".''' | |
177 | ... |
|
177 | ... | |
178 | """ |
|
178 | """ | |
179 | def _decorator(func): |
|
179 | def _decorator(func): | |
180 | lparttype = parttype.lower() # enforce lower case matching. |
|
180 | lparttype = parttype.lower() # enforce lower case matching. | |
181 | assert lparttype not in parthandlermapping |
|
181 | assert lparttype not in parthandlermapping | |
182 | parthandlermapping[lparttype] = func |
|
182 | parthandlermapping[lparttype] = func | |
183 | return func |
|
183 | return func | |
184 | return _decorator |
|
184 | return _decorator | |
185 |
|
185 | |||
|
186 | class bundleoperation(object): | |||
|
187 | """an object that represents a single bundling process | |||
|
188 | ||||
|
189 | Its purpose is to carry unbundle-related objects and states. | |||
|
190 | ||||
|
191 | A new object should be created at the beginning of each bundle processing. | |||
|
192 | The object is to be returned by the processing function. | |||
|
193 | ||||
|
194 | The object has very little content now it will ultimately contain: | |||
|
195 | * an access to the repo the bundle is applied to, | |||
|
196 | * a ui object, | |||
|
197 | * a way to retrieve a transaction to add changes to the repo, | |||
|
198 | * a way to record the result of processing each part, | |||
|
199 | * a way to construct a bundle response when applicable. | |||
|
200 | """ | |||
|
201 | ||||
|
202 | def __init__(self, repo): | |||
|
203 | self.repo = repo | |||
|
204 | self.ui = repo.ui | |||
|
205 | ||||
186 | def processbundle(repo, unbundler): |
|
206 | def processbundle(repo, unbundler): | |
187 | """This function process a bundle, apply effect to/from a repo |
|
207 | """This function process a bundle, apply effect to/from a repo | |
188 |
|
208 | |||
189 | It iterates over each part then searches for and uses the proper handling |
|
209 | It iterates over each part then searches for and uses the proper handling | |
190 | code to process the part. Parts are processed in order. |
|
210 | code to process the part. Parts are processed in order. | |
191 |
|
211 | |||
192 | This is very early version of this function that will be strongly reworked |
|
212 | This is very early version of this function that will be strongly reworked | |
193 | before final usage. |
|
213 | before final usage. | |
194 |
|
214 | |||
195 | Unknown Mandatory part will abort the process. |
|
215 | Unknown Mandatory part will abort the process. | |
196 | """ |
|
216 | """ | |
197 | ui = repo.ui |
|
217 | op = bundleoperation(repo) | |
198 | # todo: |
|
218 | # todo: | |
199 | # - replace this is a init function soon. |
|
219 | # - replace this is a init function soon. | |
200 | # - exception catching |
|
220 | # - exception catching | |
201 | unbundler.params |
|
221 | unbundler.params | |
202 | iterparts = iter(unbundler) |
|
222 | iterparts = iter(unbundler) | |
203 | try: |
|
223 | try: | |
204 | for part in iterparts: |
|
224 | for part in iterparts: | |
205 | parttype = part.type |
|
225 | parttype = part.type | |
206 | # part key are matched lower case |
|
226 | # part key are matched lower case | |
207 | key = parttype.lower() |
|
227 | key = parttype.lower() | |
208 | try: |
|
228 | try: | |
209 | handler = parthandlermapping[key] |
|
229 | handler = parthandlermapping[key] | |
210 |
ui.debug('found a |
|
230 | op.ui.debug('found a handler for part %r\n' % parttype) | |
211 | except KeyError: |
|
231 | except KeyError: | |
212 | if key != parttype: # mandatory parts |
|
232 | if key != parttype: # mandatory parts | |
213 | # todo: |
|
233 | # todo: | |
214 | # - use a more precise exception |
|
234 | # - use a more precise exception | |
215 | raise |
|
235 | raise | |
216 | ui.debug('ignoring unknown advisory part %r\n' % key) |
|
236 | op.ui.debug('ignoring unknown advisory part %r\n' % key) | |
217 | # todo: |
|
237 | # todo: | |
218 | # - consume the part once we use streaming |
|
238 | # - consume the part once we use streaming | |
219 | continue |
|
239 | continue | |
220 |
handler( |
|
240 | handler(op, part) | |
221 | except Exception: |
|
241 | except Exception: | |
222 | for part in iterparts: |
|
242 | for part in iterparts: | |
223 | pass # consume the bundle content |
|
243 | pass # consume the bundle content | |
224 | raise |
|
244 | raise | |
225 |
|
245 | |||
226 | class bundle20(object): |
|
246 | class bundle20(object): | |
227 | """represent an outgoing bundle2 container |
|
247 | """represent an outgoing bundle2 container | |
228 |
|
248 | |||
229 | Use the `addparam` method to add stream level parameter. and `addpart` to |
|
249 | Use the `addparam` method to add stream level parameter. and `addpart` to | |
230 | populate it. Then call `getchunks` to retrieve all the binary chunks of |
|
250 | populate it. Then call `getchunks` to retrieve all the binary chunks of | |
231 | datathat compose the bundle2 container.""" |
|
251 | datathat compose the bundle2 container.""" | |
232 |
|
252 | |||
233 | def __init__(self, ui): |
|
253 | def __init__(self, ui): | |
234 | self.ui = ui |
|
254 | self.ui = ui | |
235 | self._params = [] |
|
255 | self._params = [] | |
236 | self._parts = [] |
|
256 | self._parts = [] | |
237 |
|
257 | |||
238 | def addparam(self, name, value=None): |
|
258 | def addparam(self, name, value=None): | |
239 | """add a stream level parameter""" |
|
259 | """add a stream level parameter""" | |
240 | if not name: |
|
260 | if not name: | |
241 | raise ValueError('empty parameter name') |
|
261 | raise ValueError('empty parameter name') | |
242 | if name[0] not in string.letters: |
|
262 | if name[0] not in string.letters: | |
243 | raise ValueError('non letter first character: %r' % name) |
|
263 | raise ValueError('non letter first character: %r' % name) | |
244 | self._params.append((name, value)) |
|
264 | self._params.append((name, value)) | |
245 |
|
265 | |||
246 | def addpart(self, part): |
|
266 | def addpart(self, part): | |
247 | """add a new part to the bundle2 container |
|
267 | """add a new part to the bundle2 container | |
248 |
|
268 | |||
249 | Parts contains the actuall applicative payload.""" |
|
269 | Parts contains the actuall applicative payload.""" | |
250 | self._parts.append(part) |
|
270 | self._parts.append(part) | |
251 |
|
271 | |||
252 | def getchunks(self): |
|
272 | def getchunks(self): | |
253 | self.ui.debug('start emission of %s stream\n' % _magicstring) |
|
273 | self.ui.debug('start emission of %s stream\n' % _magicstring) | |
254 | yield _magicstring |
|
274 | yield _magicstring | |
255 | param = self._paramchunk() |
|
275 | param = self._paramchunk() | |
256 | self.ui.debug('bundle parameter: %s\n' % param) |
|
276 | self.ui.debug('bundle parameter: %s\n' % param) | |
257 | yield _pack(_fstreamparamsize, len(param)) |
|
277 | yield _pack(_fstreamparamsize, len(param)) | |
258 | if param: |
|
278 | if param: | |
259 | yield param |
|
279 | yield param | |
260 |
|
280 | |||
261 | self.ui.debug('start of parts\n') |
|
281 | self.ui.debug('start of parts\n') | |
262 | for part in self._parts: |
|
282 | for part in self._parts: | |
263 | self.ui.debug('bundle part: "%s"\n' % part.type) |
|
283 | self.ui.debug('bundle part: "%s"\n' % part.type) | |
264 | for chunk in part.getchunks(): |
|
284 | for chunk in part.getchunks(): | |
265 | yield chunk |
|
285 | yield chunk | |
266 | self.ui.debug('end of bundle\n') |
|
286 | self.ui.debug('end of bundle\n') | |
267 | yield '\0\0' |
|
287 | yield '\0\0' | |
268 |
|
288 | |||
269 | def _paramchunk(self): |
|
289 | def _paramchunk(self): | |
270 | """return a encoded version of all stream parameters""" |
|
290 | """return a encoded version of all stream parameters""" | |
271 | blocks = [] |
|
291 | blocks = [] | |
272 | for par, value in self._params: |
|
292 | for par, value in self._params: | |
273 | par = urllib.quote(par) |
|
293 | par = urllib.quote(par) | |
274 | if value is not None: |
|
294 | if value is not None: | |
275 | value = urllib.quote(value) |
|
295 | value = urllib.quote(value) | |
276 | par = '%s=%s' % (par, value) |
|
296 | par = '%s=%s' % (par, value) | |
277 | blocks.append(par) |
|
297 | blocks.append(par) | |
278 | return ' '.join(blocks) |
|
298 | return ' '.join(blocks) | |
279 |
|
299 | |||
280 | class unbundle20(object): |
|
300 | class unbundle20(object): | |
281 | """interpret a bundle2 stream |
|
301 | """interpret a bundle2 stream | |
282 |
|
302 | |||
283 | (this will eventually yield parts)""" |
|
303 | (this will eventually yield parts)""" | |
284 |
|
304 | |||
285 | def __init__(self, ui, fp): |
|
305 | def __init__(self, ui, fp): | |
286 | self.ui = ui |
|
306 | self.ui = ui | |
287 | self._fp = fp |
|
307 | self._fp = fp | |
288 | header = self._readexact(4) |
|
308 | header = self._readexact(4) | |
289 | magic, version = header[0:2], header[2:4] |
|
309 | magic, version = header[0:2], header[2:4] | |
290 | if magic != 'HG': |
|
310 | if magic != 'HG': | |
291 | raise util.Abort(_('not a Mercurial bundle')) |
|
311 | raise util.Abort(_('not a Mercurial bundle')) | |
292 | if version != '20': |
|
312 | if version != '20': | |
293 | raise util.Abort(_('unknown bundle version %s') % version) |
|
313 | raise util.Abort(_('unknown bundle version %s') % version) | |
294 | self.ui.debug('start processing of %s stream\n' % header) |
|
314 | self.ui.debug('start processing of %s stream\n' % header) | |
295 |
|
315 | |||
296 | def _unpack(self, format): |
|
316 | def _unpack(self, format): | |
297 | """unpack this struct format from the stream""" |
|
317 | """unpack this struct format from the stream""" | |
298 | data = self._readexact(struct.calcsize(format)) |
|
318 | data = self._readexact(struct.calcsize(format)) | |
299 | return _unpack(format, data) |
|
319 | return _unpack(format, data) | |
300 |
|
320 | |||
301 | def _readexact(self, size): |
|
321 | def _readexact(self, size): | |
302 | """read exactly <size> bytes from the stream""" |
|
322 | """read exactly <size> bytes from the stream""" | |
303 | return changegroup.readexactly(self._fp, size) |
|
323 | return changegroup.readexactly(self._fp, size) | |
304 |
|
324 | |||
305 | @util.propertycache |
|
325 | @util.propertycache | |
306 | def params(self): |
|
326 | def params(self): | |
307 | """dictionnary of stream level parameters""" |
|
327 | """dictionnary of stream level parameters""" | |
308 | self.ui.debug('reading bundle2 stream parameters\n') |
|
328 | self.ui.debug('reading bundle2 stream parameters\n') | |
309 | params = {} |
|
329 | params = {} | |
310 | paramssize = self._unpack(_fstreamparamsize)[0] |
|
330 | paramssize = self._unpack(_fstreamparamsize)[0] | |
311 | if paramssize: |
|
331 | if paramssize: | |
312 | for p in self._readexact(paramssize).split(' '): |
|
332 | for p in self._readexact(paramssize).split(' '): | |
313 | p = p.split('=', 1) |
|
333 | p = p.split('=', 1) | |
314 | p = [urllib.unquote(i) for i in p] |
|
334 | p = [urllib.unquote(i) for i in p] | |
315 | if len(p) < 2: |
|
335 | if len(p) < 2: | |
316 | p.append(None) |
|
336 | p.append(None) | |
317 | self._processparam(*p) |
|
337 | self._processparam(*p) | |
318 | params[p[0]] = p[1] |
|
338 | params[p[0]] = p[1] | |
319 | return params |
|
339 | return params | |
320 |
|
340 | |||
321 | def _processparam(self, name, value): |
|
341 | def _processparam(self, name, value): | |
322 | """process a parameter, applying its effect if needed |
|
342 | """process a parameter, applying its effect if needed | |
323 |
|
343 | |||
324 | Parameter starting with a lower case letter are advisory and will be |
|
344 | Parameter starting with a lower case letter are advisory and will be | |
325 | ignored when unknown. Those starting with an upper case letter are |
|
345 | ignored when unknown. Those starting with an upper case letter are | |
326 | mandatory and will this function will raise a KeyError when unknown. |
|
346 | mandatory and will this function will raise a KeyError when unknown. | |
327 |
|
347 | |||
328 | Note: no option are currently supported. Any input will be either |
|
348 | Note: no option are currently supported. Any input will be either | |
329 | ignored or failing. |
|
349 | ignored or failing. | |
330 | """ |
|
350 | """ | |
331 | if not name: |
|
351 | if not name: | |
332 | raise ValueError('empty parameter name') |
|
352 | raise ValueError('empty parameter name') | |
333 | if name[0] not in string.letters: |
|
353 | if name[0] not in string.letters: | |
334 | raise ValueError('non letter first character: %r' % name) |
|
354 | raise ValueError('non letter first character: %r' % name) | |
335 | # Some logic will be later added here to try to process the option for |
|
355 | # Some logic will be later added here to try to process the option for | |
336 | # a dict of known parameter. |
|
356 | # a dict of known parameter. | |
337 | if name[0].islower(): |
|
357 | if name[0].islower(): | |
338 | self.ui.debug("ignoring unknown parameter %r\n" % name) |
|
358 | self.ui.debug("ignoring unknown parameter %r\n" % name) | |
339 | else: |
|
359 | else: | |
340 | raise KeyError(name) |
|
360 | raise KeyError(name) | |
341 |
|
361 | |||
342 |
|
362 | |||
343 | def __iter__(self): |
|
363 | def __iter__(self): | |
344 | """yield all parts contained in the stream""" |
|
364 | """yield all parts contained in the stream""" | |
345 | # make sure param have been loaded |
|
365 | # make sure param have been loaded | |
346 | self.params |
|
366 | self.params | |
347 | self.ui.debug('start extraction of bundle2 parts\n') |
|
367 | self.ui.debug('start extraction of bundle2 parts\n') | |
348 | part = self._readpart() |
|
368 | part = self._readpart() | |
349 | while part is not None: |
|
369 | while part is not None: | |
350 | yield part |
|
370 | yield part | |
351 | part = self._readpart() |
|
371 | part = self._readpart() | |
352 | self.ui.debug('end of bundle2 stream\n') |
|
372 | self.ui.debug('end of bundle2 stream\n') | |
353 |
|
373 | |||
354 | def _readpart(self): |
|
374 | def _readpart(self): | |
355 | """return None when an end of stream markers is reach""" |
|
375 | """return None when an end of stream markers is reach""" | |
356 |
|
376 | |||
357 | headersize = self._unpack(_fpartheadersize)[0] |
|
377 | headersize = self._unpack(_fpartheadersize)[0] | |
358 | self.ui.debug('part header size: %i\n' % headersize) |
|
378 | self.ui.debug('part header size: %i\n' % headersize) | |
359 | if not headersize: |
|
379 | if not headersize: | |
360 | return None |
|
380 | return None | |
361 | headerblock = self._readexact(headersize) |
|
381 | headerblock = self._readexact(headersize) | |
362 | # some utility to help reading from the header block |
|
382 | # some utility to help reading from the header block | |
363 | self._offset = 0 # layer violation to have something easy to understand |
|
383 | self._offset = 0 # layer violation to have something easy to understand | |
364 | def fromheader(size): |
|
384 | def fromheader(size): | |
365 | """return the next <size> byte from the header""" |
|
385 | """return the next <size> byte from the header""" | |
366 | offset = self._offset |
|
386 | offset = self._offset | |
367 | data = headerblock[offset:(offset + size)] |
|
387 | data = headerblock[offset:(offset + size)] | |
368 | self._offset = offset + size |
|
388 | self._offset = offset + size | |
369 | return data |
|
389 | return data | |
370 | def unpackheader(format): |
|
390 | def unpackheader(format): | |
371 | """read given format from header |
|
391 | """read given format from header | |
372 |
|
392 | |||
373 | This automatically compute the size of the format to read.""" |
|
393 | This automatically compute the size of the format to read.""" | |
374 | data = fromheader(struct.calcsize(format)) |
|
394 | data = fromheader(struct.calcsize(format)) | |
375 | return _unpack(format, data) |
|
395 | return _unpack(format, data) | |
376 |
|
396 | |||
377 | typesize = unpackheader(_fparttypesize)[0] |
|
397 | typesize = unpackheader(_fparttypesize)[0] | |
378 | parttype = fromheader(typesize) |
|
398 | parttype = fromheader(typesize) | |
379 | self.ui.debug('part type: "%s"\n' % parttype) |
|
399 | self.ui.debug('part type: "%s"\n' % parttype) | |
380 | ## reading parameters |
|
400 | ## reading parameters | |
381 | # param count |
|
401 | # param count | |
382 | mancount, advcount = unpackheader(_fpartparamcount) |
|
402 | mancount, advcount = unpackheader(_fpartparamcount) | |
383 | self.ui.debug('part parameters: %i\n' % (mancount + advcount)) |
|
403 | self.ui.debug('part parameters: %i\n' % (mancount + advcount)) | |
384 | # param size |
|
404 | # param size | |
385 | paramsizes = unpackheader(_makefpartparamsizes(mancount + advcount)) |
|
405 | paramsizes = unpackheader(_makefpartparamsizes(mancount + advcount)) | |
386 | # make it a list of couple again |
|
406 | # make it a list of couple again | |
387 | paramsizes = zip(paramsizes[::2], paramsizes[1::2]) |
|
407 | paramsizes = zip(paramsizes[::2], paramsizes[1::2]) | |
388 | # split mandatory from advisory |
|
408 | # split mandatory from advisory | |
389 | mansizes = paramsizes[:mancount] |
|
409 | mansizes = paramsizes[:mancount] | |
390 | advsizes = paramsizes[mancount:] |
|
410 | advsizes = paramsizes[mancount:] | |
391 | # retrive param value |
|
411 | # retrive param value | |
392 | manparams = [] |
|
412 | manparams = [] | |
393 | for key, value in mansizes: |
|
413 | for key, value in mansizes: | |
394 | manparams.append((fromheader(key), fromheader(value))) |
|
414 | manparams.append((fromheader(key), fromheader(value))) | |
395 | advparams = [] |
|
415 | advparams = [] | |
396 | for key, value in advsizes: |
|
416 | for key, value in advsizes: | |
397 | advparams.append((fromheader(key), fromheader(value))) |
|
417 | advparams.append((fromheader(key), fromheader(value))) | |
398 | del self._offset # clean up layer, nobody saw anything. |
|
418 | del self._offset # clean up layer, nobody saw anything. | |
399 | ## part payload |
|
419 | ## part payload | |
400 | payload = [] |
|
420 | payload = [] | |
401 | payloadsize = self._unpack(_fpayloadsize)[0] |
|
421 | payloadsize = self._unpack(_fpayloadsize)[0] | |
402 | self.ui.debug('payload chunk size: %i\n' % payloadsize) |
|
422 | self.ui.debug('payload chunk size: %i\n' % payloadsize) | |
403 | while payloadsize: |
|
423 | while payloadsize: | |
404 | payload.append(self._readexact(payloadsize)) |
|
424 | payload.append(self._readexact(payloadsize)) | |
405 | payloadsize = self._unpack(_fpayloadsize)[0] |
|
425 | payloadsize = self._unpack(_fpayloadsize)[0] | |
406 | self.ui.debug('payload chunk size: %i\n' % payloadsize) |
|
426 | self.ui.debug('payload chunk size: %i\n' % payloadsize) | |
407 | payload = ''.join(payload) |
|
427 | payload = ''.join(payload) | |
408 | current = part(parttype, manparams, advparams, data=payload) |
|
428 | current = part(parttype, manparams, advparams, data=payload) | |
409 | return current |
|
429 | return current | |
410 |
|
430 | |||
411 |
|
431 | |||
412 | class part(object): |
|
432 | class part(object): | |
413 | """A bundle2 part contains application level payload |
|
433 | """A bundle2 part contains application level payload | |
414 |
|
434 | |||
415 | The part `type` is used to route the part to the application level |
|
435 | The part `type` is used to route the part to the application level | |
416 | handler. |
|
436 | handler. | |
417 | """ |
|
437 | """ | |
418 |
|
438 | |||
419 | def __init__(self, parttype, mandatoryparams=(), advisoryparams=(), |
|
439 | def __init__(self, parttype, mandatoryparams=(), advisoryparams=(), | |
420 | data=''): |
|
440 | data=''): | |
421 | self.type = parttype |
|
441 | self.type = parttype | |
422 | self.data = data |
|
442 | self.data = data | |
423 | self.mandatoryparams = mandatoryparams |
|
443 | self.mandatoryparams = mandatoryparams | |
424 | self.advisoryparams = advisoryparams |
|
444 | self.advisoryparams = advisoryparams | |
425 |
|
445 | |||
426 | def getchunks(self): |
|
446 | def getchunks(self): | |
427 | #### header |
|
447 | #### header | |
428 | ## parttype |
|
448 | ## parttype | |
429 | header = [_pack(_fparttypesize, len(self.type)), |
|
449 | header = [_pack(_fparttypesize, len(self.type)), | |
430 | self.type, |
|
450 | self.type, | |
431 | ] |
|
451 | ] | |
432 | ## parameters |
|
452 | ## parameters | |
433 | # count |
|
453 | # count | |
434 | manpar = self.mandatoryparams |
|
454 | manpar = self.mandatoryparams | |
435 | advpar = self.advisoryparams |
|
455 | advpar = self.advisoryparams | |
436 | header.append(_pack(_fpartparamcount, len(manpar), len(advpar))) |
|
456 | header.append(_pack(_fpartparamcount, len(manpar), len(advpar))) | |
437 | # size |
|
457 | # size | |
438 | parsizes = [] |
|
458 | parsizes = [] | |
439 | for key, value in manpar: |
|
459 | for key, value in manpar: | |
440 | parsizes.append(len(key)) |
|
460 | parsizes.append(len(key)) | |
441 | parsizes.append(len(value)) |
|
461 | parsizes.append(len(value)) | |
442 | for key, value in advpar: |
|
462 | for key, value in advpar: | |
443 | parsizes.append(len(key)) |
|
463 | parsizes.append(len(key)) | |
444 | parsizes.append(len(value)) |
|
464 | parsizes.append(len(value)) | |
445 | paramsizes = _pack(_makefpartparamsizes(len(parsizes) / 2), *parsizes) |
|
465 | paramsizes = _pack(_makefpartparamsizes(len(parsizes) / 2), *parsizes) | |
446 | header.append(paramsizes) |
|
466 | header.append(paramsizes) | |
447 | # key, value |
|
467 | # key, value | |
448 | for key, value in manpar: |
|
468 | for key, value in manpar: | |
449 | header.append(key) |
|
469 | header.append(key) | |
450 | header.append(value) |
|
470 | header.append(value) | |
451 | for key, value in advpar: |
|
471 | for key, value in advpar: | |
452 | header.append(key) |
|
472 | header.append(key) | |
453 | header.append(value) |
|
473 | header.append(value) | |
454 | ## finalize header |
|
474 | ## finalize header | |
455 | headerchunk = ''.join(header) |
|
475 | headerchunk = ''.join(header) | |
456 | yield _pack(_fpartheadersize, len(headerchunk)) |
|
476 | yield _pack(_fpartheadersize, len(headerchunk)) | |
457 | yield headerchunk |
|
477 | yield headerchunk | |
458 | ## payload |
|
478 | ## payload | |
459 | # we only support fixed size data now. |
|
479 | # we only support fixed size data now. | |
460 | # This will be improved in the future. |
|
480 | # This will be improved in the future. | |
461 | if len(self.data): |
|
481 | if len(self.data): | |
462 | yield _pack(_fpayloadsize, len(self.data)) |
|
482 | yield _pack(_fpayloadsize, len(self.data)) | |
463 | yield self.data |
|
483 | yield self.data | |
464 | # end of payload |
|
484 | # end of payload | |
465 | yield _pack(_fpayloadsize, 0) |
|
485 | yield _pack(_fpayloadsize, 0) | |
466 |
|
486 |
@@ -1,407 +1,407 b'' | |||||
1 |
|
1 | |||
2 | Create an extension to test bundle2 API |
|
2 | Create an extension to test bundle2 API | |
3 |
|
3 | |||
4 | $ cat > bundle2.py << EOF |
|
4 | $ cat > bundle2.py << EOF | |
5 | > """A small extension to test bundle2 implementation |
|
5 | > """A small extension to test bundle2 implementation | |
6 | > |
|
6 | > | |
7 | > Current bundle2 implementation is far too limited to be used in any core |
|
7 | > Current bundle2 implementation is far too limited to be used in any core | |
8 | > code. We still need to be able to test it while it grow up. |
|
8 | > code. We still need to be able to test it while it grow up. | |
9 | > """ |
|
9 | > """ | |
10 | > |
|
10 | > | |
11 | > import sys |
|
11 | > import sys | |
12 | > from mercurial import cmdutil |
|
12 | > from mercurial import cmdutil | |
13 | > from mercurial import util |
|
13 | > from mercurial import util | |
14 | > from mercurial import bundle2 |
|
14 | > from mercurial import bundle2 | |
15 | > cmdtable = {} |
|
15 | > cmdtable = {} | |
16 | > command = cmdutil.command(cmdtable) |
|
16 | > command = cmdutil.command(cmdtable) | |
17 | > |
|
17 | > | |
18 | > ELEPHANTSSONG = """Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko |
|
18 | > ELEPHANTSSONG = """Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko | |
19 | > Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko |
|
19 | > Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko | |
20 | > Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko.""" |
|
20 | > Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko.""" | |
21 | > assert len(ELEPHANTSSONG) == 178 # future test say 178 bytes, trust it. |
|
21 | > assert len(ELEPHANTSSONG) == 178 # future test say 178 bytes, trust it. | |
22 | > |
|
22 | > | |
23 | > @bundle2.parthandler('test:song') |
|
23 | > @bundle2.parthandler('test:song') | |
24 |
> def songhandler( |
|
24 | > def songhandler(op, part): | |
25 | > """handle a "test:song" bundle2 part, printing the lyrics on stdin""" |
|
25 | > """handle a "test:song" bundle2 part, printing the lyrics on stdin""" | |
26 |
> |
|
26 | > op.ui.write('The choir starts singing:\n') | |
27 | > for line in part.data.split('\n'): |
|
27 | > for line in part.data.split('\n'): | |
28 |
> |
|
28 | > op.ui.write(' %s\n' % line) | |
29 | > |
|
29 | > | |
30 | > @command('bundle2', |
|
30 | > @command('bundle2', | |
31 | > [('', 'param', [], 'stream level parameter'), |
|
31 | > [('', 'param', [], 'stream level parameter'), | |
32 | > ('', 'unknown', False, 'include an unknown mandatory part in the bundle'), |
|
32 | > ('', 'unknown', False, 'include an unknown mandatory part in the bundle'), | |
33 | > ('', 'parts', False, 'include some arbitrary parts to the bundle'),], |
|
33 | > ('', 'parts', False, 'include some arbitrary parts to the bundle'),], | |
34 | > '[OUTPUTFILE]') |
|
34 | > '[OUTPUTFILE]') | |
35 | > def cmdbundle2(ui, repo, path=None, **opts): |
|
35 | > def cmdbundle2(ui, repo, path=None, **opts): | |
36 | > """write a bundle2 container on standard ouput""" |
|
36 | > """write a bundle2 container on standard ouput""" | |
37 | > bundler = bundle2.bundle20(ui) |
|
37 | > bundler = bundle2.bundle20(ui) | |
38 | > for p in opts['param']: |
|
38 | > for p in opts['param']: | |
39 | > p = p.split('=', 1) |
|
39 | > p = p.split('=', 1) | |
40 | > try: |
|
40 | > try: | |
41 | > bundler.addparam(*p) |
|
41 | > bundler.addparam(*p) | |
42 | > except ValueError, exc: |
|
42 | > except ValueError, exc: | |
43 | > raise util.Abort('%s' % exc) |
|
43 | > raise util.Abort('%s' % exc) | |
44 | > |
|
44 | > | |
45 | > if opts['parts']: |
|
45 | > if opts['parts']: | |
46 | > part = bundle2.part('test:empty') |
|
46 | > part = bundle2.part('test:empty') | |
47 | > bundler.addpart(part) |
|
47 | > bundler.addpart(part) | |
48 | > # add a second one to make sure we handle multiple parts |
|
48 | > # add a second one to make sure we handle multiple parts | |
49 | > part = bundle2.part('test:empty') |
|
49 | > part = bundle2.part('test:empty') | |
50 | > bundler.addpart(part) |
|
50 | > bundler.addpart(part) | |
51 | > part = bundle2.part('test:song', data=ELEPHANTSSONG) |
|
51 | > part = bundle2.part('test:song', data=ELEPHANTSSONG) | |
52 | > bundler.addpart(part) |
|
52 | > bundler.addpart(part) | |
53 | > part = bundle2.part('test:math', |
|
53 | > part = bundle2.part('test:math', | |
54 | > [('pi', '3.14'), ('e', '2.72')], |
|
54 | > [('pi', '3.14'), ('e', '2.72')], | |
55 | > [('cooking', 'raw')], |
|
55 | > [('cooking', 'raw')], | |
56 | > '42') |
|
56 | > '42') | |
57 | > bundler.addpart(part) |
|
57 | > bundler.addpart(part) | |
58 | > if opts['unknown']: |
|
58 | > if opts['unknown']: | |
59 | > part = bundle2.part('test:UNKNOWN', |
|
59 | > part = bundle2.part('test:UNKNOWN', | |
60 | > data='some random content') |
|
60 | > data='some random content') | |
61 | > bundler.addpart(part) |
|
61 | > bundler.addpart(part) | |
62 | > |
|
62 | > | |
63 | > if path is None: |
|
63 | > if path is None: | |
64 | > file = sys.stdout |
|
64 | > file = sys.stdout | |
65 | > else: |
|
65 | > else: | |
66 | > file = open(path, 'w') |
|
66 | > file = open(path, 'w') | |
67 | > |
|
67 | > | |
68 | > for chunk in bundler.getchunks(): |
|
68 | > for chunk in bundler.getchunks(): | |
69 | > file.write(chunk) |
|
69 | > file.write(chunk) | |
70 | > |
|
70 | > | |
71 | > @command('unbundle2', [], '') |
|
71 | > @command('unbundle2', [], '') | |
72 | > def cmdunbundle2(ui, repo): |
|
72 | > def cmdunbundle2(ui, repo): | |
73 | > """process a bundle2 stream from stdin on the current repo""" |
|
73 | > """process a bundle2 stream from stdin on the current repo""" | |
74 | > try: |
|
74 | > try: | |
75 | > lock = repo.lock() |
|
75 | > lock = repo.lock() | |
76 | > try: |
|
76 | > try: | |
77 | > unbundler = bundle2.unbundle20(ui, sys.stdin) |
|
77 | > unbundler = bundle2.unbundle20(ui, sys.stdin) | |
78 | > bundle2.processbundle(repo, unbundler) |
|
78 | > bundle2.processbundle(repo, unbundler) | |
79 | > except KeyError, exc: |
|
79 | > except KeyError, exc: | |
80 | > raise util.Abort('missing support for %s' % exc) |
|
80 | > raise util.Abort('missing support for %s' % exc) | |
81 | > finally: |
|
81 | > finally: | |
82 | > lock.release() |
|
82 | > lock.release() | |
83 | > remains = sys.stdin.read() |
|
83 | > remains = sys.stdin.read() | |
84 | > ui.write('%i unread bytes\n' % len(remains)) |
|
84 | > ui.write('%i unread bytes\n' % len(remains)) | |
85 | > |
|
85 | > | |
86 | > @command('statbundle2', [], '') |
|
86 | > @command('statbundle2', [], '') | |
87 | > def cmdstatbundle2(ui, repo): |
|
87 | > def cmdstatbundle2(ui, repo): | |
88 | > """print statistic on the bundle2 container read from stdin""" |
|
88 | > """print statistic on the bundle2 container read from stdin""" | |
89 | > unbundler = bundle2.unbundle20(ui, sys.stdin) |
|
89 | > unbundler = bundle2.unbundle20(ui, sys.stdin) | |
90 | > try: |
|
90 | > try: | |
91 | > params = unbundler.params |
|
91 | > params = unbundler.params | |
92 | > except KeyError, exc: |
|
92 | > except KeyError, exc: | |
93 | > raise util.Abort('unknown parameters: %s' % exc) |
|
93 | > raise util.Abort('unknown parameters: %s' % exc) | |
94 | > ui.write('options count: %i\n' % len(params)) |
|
94 | > ui.write('options count: %i\n' % len(params)) | |
95 | > for key in sorted(params): |
|
95 | > for key in sorted(params): | |
96 | > ui.write('- %s\n' % key) |
|
96 | > ui.write('- %s\n' % key) | |
97 | > value = params[key] |
|
97 | > value = params[key] | |
98 | > if value is not None: |
|
98 | > if value is not None: | |
99 | > ui.write(' %s\n' % value) |
|
99 | > ui.write(' %s\n' % value) | |
100 | > parts = list(unbundler) |
|
100 | > parts = list(unbundler) | |
101 | > ui.write('parts count: %i\n' % len(parts)) |
|
101 | > ui.write('parts count: %i\n' % len(parts)) | |
102 | > for p in parts: |
|
102 | > for p in parts: | |
103 | > ui.write(' :%s:\n' % p.type) |
|
103 | > ui.write(' :%s:\n' % p.type) | |
104 | > ui.write(' mandatory: %i\n' % len(p.mandatoryparams)) |
|
104 | > ui.write(' mandatory: %i\n' % len(p.mandatoryparams)) | |
105 | > ui.write(' advisory: %i\n' % len(p.advisoryparams)) |
|
105 | > ui.write(' advisory: %i\n' % len(p.advisoryparams)) | |
106 | > ui.write(' payload: %i bytes\n' % len(p.data)) |
|
106 | > ui.write(' payload: %i bytes\n' % len(p.data)) | |
107 | > EOF |
|
107 | > EOF | |
108 | $ cat >> $HGRCPATH << EOF |
|
108 | $ cat >> $HGRCPATH << EOF | |
109 | > [extensions] |
|
109 | > [extensions] | |
110 | > bundle2=$TESTTMP/bundle2.py |
|
110 | > bundle2=$TESTTMP/bundle2.py | |
111 | > EOF |
|
111 | > EOF | |
112 |
|
112 | |||
113 | The extension requires a repo (currently unused) |
|
113 | The extension requires a repo (currently unused) | |
114 |
|
114 | |||
115 | $ hg init main |
|
115 | $ hg init main | |
116 | $ cd main |
|
116 | $ cd main | |
117 | $ touch a |
|
117 | $ touch a | |
118 | $ hg add a |
|
118 | $ hg add a | |
119 | $ hg commit -m 'a' |
|
119 | $ hg commit -m 'a' | |
120 |
|
120 | |||
121 |
|
121 | |||
122 | Empty bundle |
|
122 | Empty bundle | |
123 | ================= |
|
123 | ================= | |
124 |
|
124 | |||
125 | - no option |
|
125 | - no option | |
126 | - no parts |
|
126 | - no parts | |
127 |
|
127 | |||
128 | Test bundling |
|
128 | Test bundling | |
129 |
|
129 | |||
130 | $ hg bundle2 |
|
130 | $ hg bundle2 | |
131 | HG20\x00\x00\x00\x00 (no-eol) (esc) |
|
131 | HG20\x00\x00\x00\x00 (no-eol) (esc) | |
132 |
|
132 | |||
133 | Test unbundling |
|
133 | Test unbundling | |
134 |
|
134 | |||
135 | $ hg bundle2 | hg statbundle2 |
|
135 | $ hg bundle2 | hg statbundle2 | |
136 | options count: 0 |
|
136 | options count: 0 | |
137 | parts count: 0 |
|
137 | parts count: 0 | |
138 |
|
138 | |||
139 | Test old style bundle are detected and refused |
|
139 | Test old style bundle are detected and refused | |
140 |
|
140 | |||
141 | $ hg bundle --all ../bundle.hg |
|
141 | $ hg bundle --all ../bundle.hg | |
142 | 1 changesets found |
|
142 | 1 changesets found | |
143 | $ hg statbundle2 < ../bundle.hg |
|
143 | $ hg statbundle2 < ../bundle.hg | |
144 | abort: unknown bundle version 10 |
|
144 | abort: unknown bundle version 10 | |
145 | [255] |
|
145 | [255] | |
146 |
|
146 | |||
147 | Test parameters |
|
147 | Test parameters | |
148 | ================= |
|
148 | ================= | |
149 |
|
149 | |||
150 | - some options |
|
150 | - some options | |
151 | - no parts |
|
151 | - no parts | |
152 |
|
152 | |||
153 | advisory parameters, no value |
|
153 | advisory parameters, no value | |
154 | ------------------------------- |
|
154 | ------------------------------- | |
155 |
|
155 | |||
156 | Simplest possible parameters form |
|
156 | Simplest possible parameters form | |
157 |
|
157 | |||
158 | Test generation simple option |
|
158 | Test generation simple option | |
159 |
|
159 | |||
160 | $ hg bundle2 --param 'caution' |
|
160 | $ hg bundle2 --param 'caution' | |
161 | HG20\x00\x07caution\x00\x00 (no-eol) (esc) |
|
161 | HG20\x00\x07caution\x00\x00 (no-eol) (esc) | |
162 |
|
162 | |||
163 | Test unbundling |
|
163 | Test unbundling | |
164 |
|
164 | |||
165 | $ hg bundle2 --param 'caution' | hg statbundle2 |
|
165 | $ hg bundle2 --param 'caution' | hg statbundle2 | |
166 | options count: 1 |
|
166 | options count: 1 | |
167 | - caution |
|
167 | - caution | |
168 | parts count: 0 |
|
168 | parts count: 0 | |
169 |
|
169 | |||
170 | Test generation multiple option |
|
170 | Test generation multiple option | |
171 |
|
171 | |||
172 | $ hg bundle2 --param 'caution' --param 'meal' |
|
172 | $ hg bundle2 --param 'caution' --param 'meal' | |
173 | HG20\x00\x0ccaution meal\x00\x00 (no-eol) (esc) |
|
173 | HG20\x00\x0ccaution meal\x00\x00 (no-eol) (esc) | |
174 |
|
174 | |||
175 | Test unbundling |
|
175 | Test unbundling | |
176 |
|
176 | |||
177 | $ hg bundle2 --param 'caution' --param 'meal' | hg statbundle2 |
|
177 | $ hg bundle2 --param 'caution' --param 'meal' | hg statbundle2 | |
178 | options count: 2 |
|
178 | options count: 2 | |
179 | - caution |
|
179 | - caution | |
180 | - meal |
|
180 | - meal | |
181 | parts count: 0 |
|
181 | parts count: 0 | |
182 |
|
182 | |||
183 | advisory parameters, with value |
|
183 | advisory parameters, with value | |
184 | ------------------------------- |
|
184 | ------------------------------- | |
185 |
|
185 | |||
186 | Test generation |
|
186 | Test generation | |
187 |
|
187 | |||
188 | $ hg bundle2 --param 'caution' --param 'meal=vegan' --param 'elephants' |
|
188 | $ hg bundle2 --param 'caution' --param 'meal=vegan' --param 'elephants' | |
189 | HG20\x00\x1ccaution meal=vegan elephants\x00\x00 (no-eol) (esc) |
|
189 | HG20\x00\x1ccaution meal=vegan elephants\x00\x00 (no-eol) (esc) | |
190 |
|
190 | |||
191 | Test unbundling |
|
191 | Test unbundling | |
192 |
|
192 | |||
193 | $ hg bundle2 --param 'caution' --param 'meal=vegan' --param 'elephants' | hg statbundle2 |
|
193 | $ hg bundle2 --param 'caution' --param 'meal=vegan' --param 'elephants' | hg statbundle2 | |
194 | options count: 3 |
|
194 | options count: 3 | |
195 | - caution |
|
195 | - caution | |
196 | - elephants |
|
196 | - elephants | |
197 | - meal |
|
197 | - meal | |
198 | vegan |
|
198 | vegan | |
199 | parts count: 0 |
|
199 | parts count: 0 | |
200 |
|
200 | |||
201 | parameter with special char in value |
|
201 | parameter with special char in value | |
202 | --------------------------------------------------- |
|
202 | --------------------------------------------------- | |
203 |
|
203 | |||
204 | Test generation |
|
204 | Test generation | |
205 |
|
205 | |||
206 | $ hg bundle2 --param 'e|! 7/=babar%#==tutu' --param simple |
|
206 | $ hg bundle2 --param 'e|! 7/=babar%#==tutu' --param simple | |
207 | HG20\x00)e%7C%21%207/=babar%25%23%3D%3Dtutu simple\x00\x00 (no-eol) (esc) |
|
207 | HG20\x00)e%7C%21%207/=babar%25%23%3D%3Dtutu simple\x00\x00 (no-eol) (esc) | |
208 |
|
208 | |||
209 | Test unbundling |
|
209 | Test unbundling | |
210 |
|
210 | |||
211 | $ hg bundle2 --param 'e|! 7/=babar%#==tutu' --param simple | hg statbundle2 |
|
211 | $ hg bundle2 --param 'e|! 7/=babar%#==tutu' --param simple | hg statbundle2 | |
212 | options count: 2 |
|
212 | options count: 2 | |
213 | - e|! 7/ |
|
213 | - e|! 7/ | |
214 | babar%#==tutu |
|
214 | babar%#==tutu | |
215 | - simple |
|
215 | - simple | |
216 | parts count: 0 |
|
216 | parts count: 0 | |
217 |
|
217 | |||
218 | Test unknown mandatory option |
|
218 | Test unknown mandatory option | |
219 | --------------------------------------------------- |
|
219 | --------------------------------------------------- | |
220 |
|
220 | |||
221 | $ hg bundle2 --param 'Gravity' | hg statbundle2 |
|
221 | $ hg bundle2 --param 'Gravity' | hg statbundle2 | |
222 | abort: unknown parameters: 'Gravity' |
|
222 | abort: unknown parameters: 'Gravity' | |
223 | [255] |
|
223 | [255] | |
224 |
|
224 | |||
225 | Test debug output |
|
225 | Test debug output | |
226 | --------------------------------------------------- |
|
226 | --------------------------------------------------- | |
227 |
|
227 | |||
228 | bundling debug |
|
228 | bundling debug | |
229 |
|
229 | |||
230 | $ hg bundle2 --debug --param 'e|! 7/=babar%#==tutu' --param simple ../out.hg2 |
|
230 | $ hg bundle2 --debug --param 'e|! 7/=babar%#==tutu' --param simple ../out.hg2 | |
231 | start emission of HG20 stream |
|
231 | start emission of HG20 stream | |
232 | bundle parameter: e%7C%21%207/=babar%25%23%3D%3Dtutu simple |
|
232 | bundle parameter: e%7C%21%207/=babar%25%23%3D%3Dtutu simple | |
233 | start of parts |
|
233 | start of parts | |
234 | end of bundle |
|
234 | end of bundle | |
235 |
|
235 | |||
236 | file content is ok |
|
236 | file content is ok | |
237 |
|
237 | |||
238 | $ cat ../out.hg2 |
|
238 | $ cat ../out.hg2 | |
239 | HG20\x00)e%7C%21%207/=babar%25%23%3D%3Dtutu simple\x00\x00 (no-eol) (esc) |
|
239 | HG20\x00)e%7C%21%207/=babar%25%23%3D%3Dtutu simple\x00\x00 (no-eol) (esc) | |
240 |
|
240 | |||
241 | unbundling debug |
|
241 | unbundling debug | |
242 |
|
242 | |||
243 | $ hg statbundle2 --debug < ../out.hg2 |
|
243 | $ hg statbundle2 --debug < ../out.hg2 | |
244 | start processing of HG20 stream |
|
244 | start processing of HG20 stream | |
245 | reading bundle2 stream parameters |
|
245 | reading bundle2 stream parameters | |
246 | ignoring unknown parameter 'e|! 7/' |
|
246 | ignoring unknown parameter 'e|! 7/' | |
247 | ignoring unknown parameter 'simple' |
|
247 | ignoring unknown parameter 'simple' | |
248 | options count: 2 |
|
248 | options count: 2 | |
249 | - e|! 7/ |
|
249 | - e|! 7/ | |
250 | babar%#==tutu |
|
250 | babar%#==tutu | |
251 | - simple |
|
251 | - simple | |
252 | start extraction of bundle2 parts |
|
252 | start extraction of bundle2 parts | |
253 | part header size: 0 |
|
253 | part header size: 0 | |
254 | end of bundle2 stream |
|
254 | end of bundle2 stream | |
255 | parts count: 0 |
|
255 | parts count: 0 | |
256 |
|
256 | |||
257 |
|
257 | |||
258 | Test buggy input |
|
258 | Test buggy input | |
259 | --------------------------------------------------- |
|
259 | --------------------------------------------------- | |
260 |
|
260 | |||
261 | empty parameter name |
|
261 | empty parameter name | |
262 |
|
262 | |||
263 | $ hg bundle2 --param '' --quiet |
|
263 | $ hg bundle2 --param '' --quiet | |
264 | abort: empty parameter name |
|
264 | abort: empty parameter name | |
265 | [255] |
|
265 | [255] | |
266 |
|
266 | |||
267 | bad parameter name |
|
267 | bad parameter name | |
268 |
|
268 | |||
269 | $ hg bundle2 --param 42babar |
|
269 | $ hg bundle2 --param 42babar | |
270 | abort: non letter first character: '42babar' |
|
270 | abort: non letter first character: '42babar' | |
271 | [255] |
|
271 | [255] | |
272 |
|
272 | |||
273 |
|
273 | |||
274 | Test part |
|
274 | Test part | |
275 | ================= |
|
275 | ================= | |
276 |
|
276 | |||
277 | $ hg bundle2 --parts ../parts.hg2 --debug |
|
277 | $ hg bundle2 --parts ../parts.hg2 --debug | |
278 | start emission of HG20 stream |
|
278 | start emission of HG20 stream | |
279 | bundle parameter: |
|
279 | bundle parameter: | |
280 | start of parts |
|
280 | start of parts | |
281 | bundle part: "test:empty" |
|
281 | bundle part: "test:empty" | |
282 | bundle part: "test:empty" |
|
282 | bundle part: "test:empty" | |
283 | bundle part: "test:song" |
|
283 | bundle part: "test:song" | |
284 | bundle part: "test:math" |
|
284 | bundle part: "test:math" | |
285 | end of bundle |
|
285 | end of bundle | |
286 |
|
286 | |||
287 | $ cat ../parts.hg2 |
|
287 | $ cat ../parts.hg2 | |
288 | HG20\x00\x00\x00\r (esc) |
|
288 | HG20\x00\x00\x00\r (esc) | |
289 | test:empty\x00\x00\x00\x00\x00\x00\x00\r (esc) |
|
289 | test:empty\x00\x00\x00\x00\x00\x00\x00\r (esc) | |
290 | test:empty\x00\x00\x00\x00\x00\x00\x00\x0c test:song\x00\x00\x00\x00\x00\xb2Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko (esc) |
|
290 | test:empty\x00\x00\x00\x00\x00\x00\x00\x0c test:song\x00\x00\x00\x00\x00\xb2Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko (esc) | |
291 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko |
|
291 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko | |
292 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko.\x00\x00\x00\x00\x00' test:math\x02\x01\x02\x04\x01\x04\x07\x03pi3.14e2.72cookingraw\x00\x00\x00\x0242\x00\x00\x00\x00\x00\x00 (no-eol) (esc) |
|
292 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko.\x00\x00\x00\x00\x00' test:math\x02\x01\x02\x04\x01\x04\x07\x03pi3.14e2.72cookingraw\x00\x00\x00\x0242\x00\x00\x00\x00\x00\x00 (no-eol) (esc) | |
293 |
|
293 | |||
294 |
|
294 | |||
295 | $ hg statbundle2 < ../parts.hg2 |
|
295 | $ hg statbundle2 < ../parts.hg2 | |
296 | options count: 0 |
|
296 | options count: 0 | |
297 | parts count: 4 |
|
297 | parts count: 4 | |
298 | :test:empty: |
|
298 | :test:empty: | |
299 | mandatory: 0 |
|
299 | mandatory: 0 | |
300 | advisory: 0 |
|
300 | advisory: 0 | |
301 | payload: 0 bytes |
|
301 | payload: 0 bytes | |
302 | :test:empty: |
|
302 | :test:empty: | |
303 | mandatory: 0 |
|
303 | mandatory: 0 | |
304 | advisory: 0 |
|
304 | advisory: 0 | |
305 | payload: 0 bytes |
|
305 | payload: 0 bytes | |
306 | :test:song: |
|
306 | :test:song: | |
307 | mandatory: 0 |
|
307 | mandatory: 0 | |
308 | advisory: 0 |
|
308 | advisory: 0 | |
309 | payload: 178 bytes |
|
309 | payload: 178 bytes | |
310 | :test:math: |
|
310 | :test:math: | |
311 | mandatory: 2 |
|
311 | mandatory: 2 | |
312 | advisory: 1 |
|
312 | advisory: 1 | |
313 | payload: 2 bytes |
|
313 | payload: 2 bytes | |
314 |
|
314 | |||
315 | $ hg statbundle2 --debug < ../parts.hg2 |
|
315 | $ hg statbundle2 --debug < ../parts.hg2 | |
316 | start processing of HG20 stream |
|
316 | start processing of HG20 stream | |
317 | reading bundle2 stream parameters |
|
317 | reading bundle2 stream parameters | |
318 | options count: 0 |
|
318 | options count: 0 | |
319 | start extraction of bundle2 parts |
|
319 | start extraction of bundle2 parts | |
320 | part header size: 13 |
|
320 | part header size: 13 | |
321 | part type: "test:empty" |
|
321 | part type: "test:empty" | |
322 | part parameters: 0 |
|
322 | part parameters: 0 | |
323 | payload chunk size: 0 |
|
323 | payload chunk size: 0 | |
324 | part header size: 13 |
|
324 | part header size: 13 | |
325 | part type: "test:empty" |
|
325 | part type: "test:empty" | |
326 | part parameters: 0 |
|
326 | part parameters: 0 | |
327 | payload chunk size: 0 |
|
327 | payload chunk size: 0 | |
328 | part header size: 12 |
|
328 | part header size: 12 | |
329 | part type: "test:song" |
|
329 | part type: "test:song" | |
330 | part parameters: 0 |
|
330 | part parameters: 0 | |
331 | payload chunk size: 178 |
|
331 | payload chunk size: 178 | |
332 | payload chunk size: 0 |
|
332 | payload chunk size: 0 | |
333 | part header size: 39 |
|
333 | part header size: 39 | |
334 | part type: "test:math" |
|
334 | part type: "test:math" | |
335 | part parameters: 3 |
|
335 | part parameters: 3 | |
336 | payload chunk size: 2 |
|
336 | payload chunk size: 2 | |
337 | payload chunk size: 0 |
|
337 | payload chunk size: 0 | |
338 | part header size: 0 |
|
338 | part header size: 0 | |
339 | end of bundle2 stream |
|
339 | end of bundle2 stream | |
340 | parts count: 4 |
|
340 | parts count: 4 | |
341 | :test:empty: |
|
341 | :test:empty: | |
342 | mandatory: 0 |
|
342 | mandatory: 0 | |
343 | advisory: 0 |
|
343 | advisory: 0 | |
344 | payload: 0 bytes |
|
344 | payload: 0 bytes | |
345 | :test:empty: |
|
345 | :test:empty: | |
346 | mandatory: 0 |
|
346 | mandatory: 0 | |
347 | advisory: 0 |
|
347 | advisory: 0 | |
348 | payload: 0 bytes |
|
348 | payload: 0 bytes | |
349 | :test:song: |
|
349 | :test:song: | |
350 | mandatory: 0 |
|
350 | mandatory: 0 | |
351 | advisory: 0 |
|
351 | advisory: 0 | |
352 | payload: 178 bytes |
|
352 | payload: 178 bytes | |
353 | :test:math: |
|
353 | :test:math: | |
354 | mandatory: 2 |
|
354 | mandatory: 2 | |
355 | advisory: 1 |
|
355 | advisory: 1 | |
356 | payload: 2 bytes |
|
356 | payload: 2 bytes | |
357 |
|
357 | |||
358 | Test actual unbundling |
|
358 | Test actual unbundling | |
359 | ======================== |
|
359 | ======================== | |
360 |
|
360 | |||
361 | Process the bundle |
|
361 | Process the bundle | |
362 |
|
362 | |||
363 | $ hg unbundle2 --debug < ../parts.hg2 |
|
363 | $ hg unbundle2 --debug < ../parts.hg2 | |
364 | start processing of HG20 stream |
|
364 | start processing of HG20 stream | |
365 | reading bundle2 stream parameters |
|
365 | reading bundle2 stream parameters | |
366 | start extraction of bundle2 parts |
|
366 | start extraction of bundle2 parts | |
367 | part header size: 13 |
|
367 | part header size: 13 | |
368 | part type: "test:empty" |
|
368 | part type: "test:empty" | |
369 | part parameters: 0 |
|
369 | part parameters: 0 | |
370 | payload chunk size: 0 |
|
370 | payload chunk size: 0 | |
371 | ignoring unknown advisory part 'test:empty' |
|
371 | ignoring unknown advisory part 'test:empty' | |
372 | part header size: 13 |
|
372 | part header size: 13 | |
373 | part type: "test:empty" |
|
373 | part type: "test:empty" | |
374 | part parameters: 0 |
|
374 | part parameters: 0 | |
375 | payload chunk size: 0 |
|
375 | payload chunk size: 0 | |
376 | ignoring unknown advisory part 'test:empty' |
|
376 | ignoring unknown advisory part 'test:empty' | |
377 | part header size: 12 |
|
377 | part header size: 12 | |
378 | part type: "test:song" |
|
378 | part type: "test:song" | |
379 | part parameters: 0 |
|
379 | part parameters: 0 | |
380 | payload chunk size: 178 |
|
380 | payload chunk size: 178 | |
381 | payload chunk size: 0 |
|
381 | payload chunk size: 0 | |
382 | found an handler for part 'test:song' |
|
382 | found an handler for part 'test:song' | |
383 | The choir start singing: |
|
383 | The choir start singing: | |
384 | Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko |
|
384 | Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko | |
385 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko |
|
385 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko | |
386 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko. |
|
386 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko. | |
387 | part header size: 39 |
|
387 | part header size: 39 | |
388 | part type: "test:math" |
|
388 | part type: "test:math" | |
389 | part parameters: 3 |
|
389 | part parameters: 3 | |
390 | payload chunk size: 2 |
|
390 | payload chunk size: 2 | |
391 | payload chunk size: 0 |
|
391 | payload chunk size: 0 | |
392 | ignoring unknown advisory part 'test:math' |
|
392 | ignoring unknown advisory part 'test:math' | |
393 | part header size: 0 |
|
393 | part header size: 0 | |
394 | end of bundle2 stream |
|
394 | end of bundle2 stream | |
395 | 0 unread bytes |
|
395 | 0 unread bytes | |
396 |
|
396 | |||
397 |
|
397 | |||
398 | $ hg bundle2 --parts --unknown ../unknown.hg2 |
|
398 | $ hg bundle2 --parts --unknown ../unknown.hg2 | |
399 |
|
399 | |||
400 | $ hg unbundle2 < ../unknown.hg2 |
|
400 | $ hg unbundle2 < ../unknown.hg2 | |
401 | The choir start singing: |
|
401 | The choir start singing: | |
402 | Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko |
|
402 | Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko | |
403 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko |
|
403 | Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko | |
404 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko. |
|
404 | Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko. | |
405 | 0 unread bytes |
|
405 | 0 unread bytes | |
406 | abort: missing support for 'test:unknown' |
|
406 | abort: missing support for 'test:unknown' | |
407 | [255] |
|
407 | [255] |
General Comments 0
You need to be logged in to leave comments.
Login now