##// END OF EJS Templates
obsolete: ensure that `getrevs` always return a set...
Pierre-Yves David -
r22507:5c00c529 default
parent child Browse files
Show More
@@ -1,1004 +1,1004 b''
1 # obsolete.py - obsolete markers handling
1 # obsolete.py - obsolete markers handling
2 #
2 #
3 # Copyright 2012 Pierre-Yves David <pierre-yves.david@ens-lyon.org>
3 # Copyright 2012 Pierre-Yves David <pierre-yves.david@ens-lyon.org>
4 # Logilab SA <contact@logilab.fr>
4 # Logilab SA <contact@logilab.fr>
5 #
5 #
6 # This software may be used and distributed according to the terms of the
6 # This software may be used and distributed according to the terms of the
7 # GNU General Public License version 2 or any later version.
7 # GNU General Public License version 2 or any later version.
8
8
9 """Obsolete marker handling
9 """Obsolete marker handling
10
10
11 An obsolete marker maps an old changeset to a list of new
11 An obsolete marker maps an old changeset to a list of new
12 changesets. If the list of new changesets is empty, the old changeset
12 changesets. If the list of new changesets is empty, the old changeset
13 is said to be "killed". Otherwise, the old changeset is being
13 is said to be "killed". Otherwise, the old changeset is being
14 "replaced" by the new changesets.
14 "replaced" by the new changesets.
15
15
16 Obsolete markers can be used to record and distribute changeset graph
16 Obsolete markers can be used to record and distribute changeset graph
17 transformations performed by history rewrite operations, and help
17 transformations performed by history rewrite operations, and help
18 building new tools to reconcile conflicting rewrite actions. To
18 building new tools to reconcile conflicting rewrite actions. To
19 facilitate conflict resolution, markers include various annotations
19 facilitate conflict resolution, markers include various annotations
20 besides old and news changeset identifiers, such as creation date or
20 besides old and news changeset identifiers, such as creation date or
21 author name.
21 author name.
22
22
23 The old obsoleted changeset is called a "precursor" and possible
23 The old obsoleted changeset is called a "precursor" and possible
24 replacements are called "successors". Markers that used changeset X as
24 replacements are called "successors". Markers that used changeset X as
25 a precursor are called "successor markers of X" because they hold
25 a precursor are called "successor markers of X" because they hold
26 information about the successors of X. Markers that use changeset Y as
26 information about the successors of X. Markers that use changeset Y as
27 a successors are call "precursor markers of Y" because they hold
27 a successors are call "precursor markers of Y" because they hold
28 information about the precursors of Y.
28 information about the precursors of Y.
29
29
30 Examples:
30 Examples:
31
31
32 - When changeset A is replaced by changeset A', one marker is stored:
32 - When changeset A is replaced by changeset A', one marker is stored:
33
33
34 (A, (A',))
34 (A, (A',))
35
35
36 - When changesets A and B are folded into a new changeset C, two markers are
36 - When changesets A and B are folded into a new changeset C, two markers are
37 stored:
37 stored:
38
38
39 (A, (C,)) and (B, (C,))
39 (A, (C,)) and (B, (C,))
40
40
41 - When changeset A is simply "pruned" from the graph, a marker is created:
41 - When changeset A is simply "pruned" from the graph, a marker is created:
42
42
43 (A, ())
43 (A, ())
44
44
45 - When changeset A is split into B and C, a single marker are used:
45 - When changeset A is split into B and C, a single marker are used:
46
46
47 (A, (C, C))
47 (A, (C, C))
48
48
49 We use a single marker to distinguish the "split" case from the "divergence"
49 We use a single marker to distinguish the "split" case from the "divergence"
50 case. If two independent operations rewrite the same changeset A in to A' and
50 case. If two independent operations rewrite the same changeset A in to A' and
51 A'', we have an error case: divergent rewriting. We can detect it because
51 A'', we have an error case: divergent rewriting. We can detect it because
52 two markers will be created independently:
52 two markers will be created independently:
53
53
54 (A, (B,)) and (A, (C,))
54 (A, (B,)) and (A, (C,))
55
55
56 Format
56 Format
57 ------
57 ------
58
58
59 Markers are stored in an append-only file stored in
59 Markers are stored in an append-only file stored in
60 '.hg/store/obsstore'.
60 '.hg/store/obsstore'.
61
61
62 The file starts with a version header:
62 The file starts with a version header:
63
63
64 - 1 unsigned byte: version number, starting at zero.
64 - 1 unsigned byte: version number, starting at zero.
65
65
66
66
67 The header is followed by the markers. Each marker is made of:
67 The header is followed by the markers. Each marker is made of:
68
68
69 - 1 unsigned byte: number of new changesets "N", can be zero.
69 - 1 unsigned byte: number of new changesets "N", can be zero.
70
70
71 - 1 unsigned 32-bits integer: metadata size "M" in bytes.
71 - 1 unsigned 32-bits integer: metadata size "M" in bytes.
72
72
73 - 1 byte: a bit field. It is reserved for flags used in common
73 - 1 byte: a bit field. It is reserved for flags used in common
74 obsolete marker operations, to avoid repeated decoding of metadata
74 obsolete marker operations, to avoid repeated decoding of metadata
75 entries.
75 entries.
76
76
77 - 20 bytes: obsoleted changeset identifier.
77 - 20 bytes: obsoleted changeset identifier.
78
78
79 - N*20 bytes: new changesets identifiers.
79 - N*20 bytes: new changesets identifiers.
80
80
81 - M bytes: metadata as a sequence of nul-terminated strings. Each
81 - M bytes: metadata as a sequence of nul-terminated strings. Each
82 string contains a key and a value, separated by a colon ':', without
82 string contains a key and a value, separated by a colon ':', without
83 additional encoding. Keys cannot contain '\0' or ':' and values
83 additional encoding. Keys cannot contain '\0' or ':' and values
84 cannot contain '\0'.
84 cannot contain '\0'.
85
85
86 """
86 """
87 import struct
87 import struct
88 import util, base85, node
88 import util, base85, node
89 import phases
89 import phases
90 from i18n import _
90 from i18n import _
91
91
92 _pack = struct.pack
92 _pack = struct.pack
93 _unpack = struct.unpack
93 _unpack = struct.unpack
94
94
95 _SEEK_END = 2 # os.SEEK_END was introduced in Python 2.5
95 _SEEK_END = 2 # os.SEEK_END was introduced in Python 2.5
96
96
97 # the obsolete feature is not mature enough to be enabled by default.
97 # the obsolete feature is not mature enough to be enabled by default.
98 # you have to rely on third party extension extension to enable this.
98 # you have to rely on third party extension extension to enable this.
99 _enabled = False
99 _enabled = False
100
100
101 # data used for parsing and writing
101 # data used for parsing and writing
102 _fm0version = 0
102 _fm0version = 0
103 _fm0fixed = '>BIB20s'
103 _fm0fixed = '>BIB20s'
104 _fm0node = '20s'
104 _fm0node = '20s'
105 _fm0fsize = struct.calcsize(_fm0fixed)
105 _fm0fsize = struct.calcsize(_fm0fixed)
106 _fm0fnodesize = struct.calcsize(_fm0node)
106 _fm0fnodesize = struct.calcsize(_fm0node)
107
107
108 ### obsolescence marker flag
108 ### obsolescence marker flag
109
109
110 ## bumpedfix flag
110 ## bumpedfix flag
111 #
111 #
112 # When a changeset A' succeed to a changeset A which became public, we call A'
112 # When a changeset A' succeed to a changeset A which became public, we call A'
113 # "bumped" because it's a successors of a public changesets
113 # "bumped" because it's a successors of a public changesets
114 #
114 #
115 # o A' (bumped)
115 # o A' (bumped)
116 # |`:
116 # |`:
117 # | o A
117 # | o A
118 # |/
118 # |/
119 # o Z
119 # o Z
120 #
120 #
121 # The way to solve this situation is to create a new changeset Ad as children
121 # The way to solve this situation is to create a new changeset Ad as children
122 # of A. This changeset have the same content than A'. So the diff from A to A'
122 # of A. This changeset have the same content than A'. So the diff from A to A'
123 # is the same than the diff from A to Ad. Ad is marked as a successors of A'
123 # is the same than the diff from A to Ad. Ad is marked as a successors of A'
124 #
124 #
125 # o Ad
125 # o Ad
126 # |`:
126 # |`:
127 # | x A'
127 # | x A'
128 # |'|
128 # |'|
129 # o | A
129 # o | A
130 # |/
130 # |/
131 # o Z
131 # o Z
132 #
132 #
133 # But by transitivity Ad is also a successors of A. To avoid having Ad marked
133 # But by transitivity Ad is also a successors of A. To avoid having Ad marked
134 # as bumped too, we add the `bumpedfix` flag to the marker. <A', (Ad,)>.
134 # as bumped too, we add the `bumpedfix` flag to the marker. <A', (Ad,)>.
135 # This flag mean that the successors express the changes between the public and
135 # This flag mean that the successors express the changes between the public and
136 # bumped version and fix the situation, breaking the transitivity of
136 # bumped version and fix the situation, breaking the transitivity of
137 # "bumped" here.
137 # "bumped" here.
138 bumpedfix = 1
138 bumpedfix = 1
139
139
140 def _readmarkers(data):
140 def _readmarkers(data):
141 """Read and enumerate markers from raw data"""
141 """Read and enumerate markers from raw data"""
142 off = 0
142 off = 0
143 diskversion = _unpack('>B', data[off:off + 1])[0]
143 diskversion = _unpack('>B', data[off:off + 1])[0]
144 off += 1
144 off += 1
145 if diskversion not in formats:
145 if diskversion not in formats:
146 raise util.Abort(_('parsing obsolete marker: unknown version %r')
146 raise util.Abort(_('parsing obsolete marker: unknown version %r')
147 % diskversion)
147 % diskversion)
148 return diskversion, formats[diskversion][0](data, off)
148 return diskversion, formats[diskversion][0](data, off)
149
149
150 def encodemarkers(markers, addheader=False, version=_fm0version):
150 def encodemarkers(markers, addheader=False, version=_fm0version):
151 # Kept separate from flushmarkers(), it will be reused for
151 # Kept separate from flushmarkers(), it will be reused for
152 # markers exchange.
152 # markers exchange.
153 encodeone = formats[version][1]
153 encodeone = formats[version][1]
154 if addheader:
154 if addheader:
155 yield _pack('>B', _fm0version)
155 yield _pack('>B', _fm0version)
156 for marker in markers:
156 for marker in markers:
157 yield encodeone(marker)
157 yield encodeone(marker)
158
158
159 def _fm0readmarkers(data, off=0):
159 def _fm0readmarkers(data, off=0):
160 # Loop on markers
160 # Loop on markers
161 l = len(data)
161 l = len(data)
162 while off + _fm0fsize <= l:
162 while off + _fm0fsize <= l:
163 # read fixed part
163 # read fixed part
164 cur = data[off:off + _fm0fsize]
164 cur = data[off:off + _fm0fsize]
165 off += _fm0fsize
165 off += _fm0fsize
166 nbsuc, mdsize, flags, pre = _unpack(_fm0fixed, cur)
166 nbsuc, mdsize, flags, pre = _unpack(_fm0fixed, cur)
167 # read replacement
167 # read replacement
168 sucs = ()
168 sucs = ()
169 if nbsuc:
169 if nbsuc:
170 s = (_fm0fnodesize * nbsuc)
170 s = (_fm0fnodesize * nbsuc)
171 cur = data[off:off + s]
171 cur = data[off:off + s]
172 sucs = _unpack(_fm0node * nbsuc, cur)
172 sucs = _unpack(_fm0node * nbsuc, cur)
173 off += s
173 off += s
174 # read metadata
174 # read metadata
175 # (metadata will be decoded on demand)
175 # (metadata will be decoded on demand)
176 metadata = data[off:off + mdsize]
176 metadata = data[off:off + mdsize]
177 if len(metadata) != mdsize:
177 if len(metadata) != mdsize:
178 raise util.Abort(_('parsing obsolete marker: metadata is too '
178 raise util.Abort(_('parsing obsolete marker: metadata is too '
179 'short, %d bytes expected, got %d')
179 'short, %d bytes expected, got %d')
180 % (mdsize, len(metadata)))
180 % (mdsize, len(metadata)))
181 off += mdsize
181 off += mdsize
182 meta = decodemeta(metadata)
182 meta = decodemeta(metadata)
183 try:
183 try:
184 when, offset = meta.pop('date', '0 0').split(' ')
184 when, offset = meta.pop('date', '0 0').split(' ')
185 date = float(when), int(offset)
185 date = float(when), int(offset)
186 except ValueError:
186 except ValueError:
187 date = (0., 0)
187 date = (0., 0)
188 parents = None
188 parents = None
189 if 'p2' in meta:
189 if 'p2' in meta:
190 parents = (meta.pop('p1', None), meta.pop('p2', None))
190 parents = (meta.pop('p1', None), meta.pop('p2', None))
191 elif 'p1' in meta:
191 elif 'p1' in meta:
192 parents = (meta.pop('p1', None),)
192 parents = (meta.pop('p1', None),)
193 elif 'p0' in meta:
193 elif 'p0' in meta:
194 parents = ()
194 parents = ()
195 if parents is not None:
195 if parents is not None:
196 try:
196 try:
197 parents = tuple(node.bin(p) for p in parents)
197 parents = tuple(node.bin(p) for p in parents)
198 # if parent content is not a nodeid, drop the data
198 # if parent content is not a nodeid, drop the data
199 for p in parents:
199 for p in parents:
200 if len(p) != 20:
200 if len(p) != 20:
201 parents = None
201 parents = None
202 break
202 break
203 except TypeError:
203 except TypeError:
204 # if content cannot be translated to nodeid drop the data.
204 # if content cannot be translated to nodeid drop the data.
205 parents = None
205 parents = None
206
206
207 metadata = encodemeta(meta)
207 metadata = encodemeta(meta)
208
208
209 yield (pre, sucs, flags, metadata, date, parents)
209 yield (pre, sucs, flags, metadata, date, parents)
210
210
211 def _fm0encodeonemarker(marker):
211 def _fm0encodeonemarker(marker):
212 pre, sucs, flags, metadata, date, parents = marker
212 pre, sucs, flags, metadata, date, parents = marker
213 metadata = decodemeta(metadata)
213 metadata = decodemeta(metadata)
214 metadata['date'] = '%d %i' % date
214 metadata['date'] = '%d %i' % date
215 if parents is not None:
215 if parents is not None:
216 if not parents:
216 if not parents:
217 # mark that we explicitly recorded no parents
217 # mark that we explicitly recorded no parents
218 metadata['p0'] = ''
218 metadata['p0'] = ''
219 for i, p in enumerate(parents):
219 for i, p in enumerate(parents):
220 metadata['p%i' % (i + 1)] = node.hex(p)
220 metadata['p%i' % (i + 1)] = node.hex(p)
221 metadata = encodemeta(metadata)
221 metadata = encodemeta(metadata)
222 nbsuc = len(sucs)
222 nbsuc = len(sucs)
223 format = _fm0fixed + (_fm0node * nbsuc)
223 format = _fm0fixed + (_fm0node * nbsuc)
224 data = [nbsuc, len(metadata), flags, pre]
224 data = [nbsuc, len(metadata), flags, pre]
225 data.extend(sucs)
225 data.extend(sucs)
226 return _pack(format, *data) + metadata
226 return _pack(format, *data) + metadata
227
227
228 # mapping to read/write various marker formats
228 # mapping to read/write various marker formats
229 # <version> -> (decoder, encoder)
229 # <version> -> (decoder, encoder)
230 formats = {0: (_fm0readmarkers, _fm0encodeonemarker)}
230 formats = {0: (_fm0readmarkers, _fm0encodeonemarker)}
231
231
232 def encodemeta(meta):
232 def encodemeta(meta):
233 """Return encoded metadata string to string mapping.
233 """Return encoded metadata string to string mapping.
234
234
235 Assume no ':' in key and no '\0' in both key and value."""
235 Assume no ':' in key and no '\0' in both key and value."""
236 for key, value in meta.iteritems():
236 for key, value in meta.iteritems():
237 if ':' in key or '\0' in key:
237 if ':' in key or '\0' in key:
238 raise ValueError("':' and '\0' are forbidden in metadata key'")
238 raise ValueError("':' and '\0' are forbidden in metadata key'")
239 if '\0' in value:
239 if '\0' in value:
240 raise ValueError("':' is forbidden in metadata value'")
240 raise ValueError("':' is forbidden in metadata value'")
241 return '\0'.join(['%s:%s' % (k, meta[k]) for k in sorted(meta)])
241 return '\0'.join(['%s:%s' % (k, meta[k]) for k in sorted(meta)])
242
242
243 def decodemeta(data):
243 def decodemeta(data):
244 """Return string to string dictionary from encoded version."""
244 """Return string to string dictionary from encoded version."""
245 d = {}
245 d = {}
246 for l in data.split('\0'):
246 for l in data.split('\0'):
247 if l:
247 if l:
248 key, value = l.split(':')
248 key, value = l.split(':')
249 d[key] = value
249 d[key] = value
250 return d
250 return d
251
251
252 class marker(object):
252 class marker(object):
253 """Wrap obsolete marker raw data"""
253 """Wrap obsolete marker raw data"""
254
254
255 def __init__(self, repo, data):
255 def __init__(self, repo, data):
256 # the repo argument will be used to create changectx in later version
256 # the repo argument will be used to create changectx in later version
257 self._repo = repo
257 self._repo = repo
258 self._data = data
258 self._data = data
259 self._decodedmeta = None
259 self._decodedmeta = None
260
260
261 def __hash__(self):
261 def __hash__(self):
262 return hash(self._data)
262 return hash(self._data)
263
263
264 def __eq__(self, other):
264 def __eq__(self, other):
265 if type(other) != type(self):
265 if type(other) != type(self):
266 return False
266 return False
267 return self._data == other._data
267 return self._data == other._data
268
268
269 def precnode(self):
269 def precnode(self):
270 """Precursor changeset node identifier"""
270 """Precursor changeset node identifier"""
271 return self._data[0]
271 return self._data[0]
272
272
273 def succnodes(self):
273 def succnodes(self):
274 """List of successor changesets node identifiers"""
274 """List of successor changesets node identifiers"""
275 return self._data[1]
275 return self._data[1]
276
276
277 def parentnodes(self):
277 def parentnodes(self):
278 """Parents of the precursors (None if not recorded)"""
278 """Parents of the precursors (None if not recorded)"""
279 return self._data[5]
279 return self._data[5]
280
280
281 def metadata(self):
281 def metadata(self):
282 """Decoded metadata dictionary"""
282 """Decoded metadata dictionary"""
283 if self._decodedmeta is None:
283 if self._decodedmeta is None:
284 self._decodedmeta = decodemeta(self._data[3])
284 self._decodedmeta = decodemeta(self._data[3])
285 return self._decodedmeta
285 return self._decodedmeta
286
286
287 def date(self):
287 def date(self):
288 """Creation date as (unixtime, offset)"""
288 """Creation date as (unixtime, offset)"""
289 return self._data[4]
289 return self._data[4]
290
290
291 def flags(self):
291 def flags(self):
292 """The flags field of the marker"""
292 """The flags field of the marker"""
293 return self._data[2]
293 return self._data[2]
294
294
295 class obsstore(object):
295 class obsstore(object):
296 """Store obsolete markers
296 """Store obsolete markers
297
297
298 Markers can be accessed with two mappings:
298 Markers can be accessed with two mappings:
299 - precursors[x] -> set(markers on precursors edges of x)
299 - precursors[x] -> set(markers on precursors edges of x)
300 - successors[x] -> set(markers on successors edges of x)
300 - successors[x] -> set(markers on successors edges of x)
301 - children[x] -> set(markers on precursors edges of children(x)
301 - children[x] -> set(markers on precursors edges of children(x)
302 """
302 """
303
303
304 fields = ('prec', 'succs', 'flag', 'meta', 'date', 'parents')
304 fields = ('prec', 'succs', 'flag', 'meta', 'date', 'parents')
305 # prec: nodeid, precursor changesets
305 # prec: nodeid, precursor changesets
306 # succs: tuple of nodeid, successor changesets (0-N length)
306 # succs: tuple of nodeid, successor changesets (0-N length)
307 # flag: integer, flag field carrying modifier for the markers (see doc)
307 # flag: integer, flag field carrying modifier for the markers (see doc)
308 # meta: binary blob, encoded metadata dictionary
308 # meta: binary blob, encoded metadata dictionary
309 # date: (float, int) tuple, date of marker creation
309 # date: (float, int) tuple, date of marker creation
310 # parents: (tuple of nodeid) or None, parents of precursors
310 # parents: (tuple of nodeid) or None, parents of precursors
311 # None is used when no data has been recorded
311 # None is used when no data has been recorded
312
312
313 def __init__(self, sopener):
313 def __init__(self, sopener):
314 # caches for various obsolescence related cache
314 # caches for various obsolescence related cache
315 self.caches = {}
315 self.caches = {}
316 self._all = []
316 self._all = []
317 self.precursors = {}
317 self.precursors = {}
318 self.successors = {}
318 self.successors = {}
319 self.children = {}
319 self.children = {}
320 self.sopener = sopener
320 self.sopener = sopener
321 data = sopener.tryread('obsstore')
321 data = sopener.tryread('obsstore')
322 self._version = _fm0version
322 self._version = _fm0version
323 if data:
323 if data:
324 self._version, markers = _readmarkers(data)
324 self._version, markers = _readmarkers(data)
325 self._load(markers)
325 self._load(markers)
326
326
327 def __iter__(self):
327 def __iter__(self):
328 return iter(self._all)
328 return iter(self._all)
329
329
330 def __len__(self):
330 def __len__(self):
331 return len(self._all)
331 return len(self._all)
332
332
333 def __nonzero__(self):
333 def __nonzero__(self):
334 return bool(self._all)
334 return bool(self._all)
335
335
336 def create(self, transaction, prec, succs=(), flag=0, parents=None,
336 def create(self, transaction, prec, succs=(), flag=0, parents=None,
337 date=None, metadata=None):
337 date=None, metadata=None):
338 """obsolete: add a new obsolete marker
338 """obsolete: add a new obsolete marker
339
339
340 * ensuring it is hashable
340 * ensuring it is hashable
341 * check mandatory metadata
341 * check mandatory metadata
342 * encode metadata
342 * encode metadata
343
343
344 If you are a human writing code creating marker you want to use the
344 If you are a human writing code creating marker you want to use the
345 `createmarkers` function in this module instead.
345 `createmarkers` function in this module instead.
346
346
347 return True if a new marker have been added, False if the markers
347 return True if a new marker have been added, False if the markers
348 already existed (no op).
348 already existed (no op).
349 """
349 """
350 if metadata is None:
350 if metadata is None:
351 metadata = {}
351 metadata = {}
352 if date is None:
352 if date is None:
353 if 'date' in metadata:
353 if 'date' in metadata:
354 # as a courtesy for out-of-tree extensions
354 # as a courtesy for out-of-tree extensions
355 date = util.parsedate(metadata.pop('date'))
355 date = util.parsedate(metadata.pop('date'))
356 else:
356 else:
357 date = util.makedate()
357 date = util.makedate()
358 if len(prec) != 20:
358 if len(prec) != 20:
359 raise ValueError(prec)
359 raise ValueError(prec)
360 for succ in succs:
360 for succ in succs:
361 if len(succ) != 20:
361 if len(succ) != 20:
362 raise ValueError(succ)
362 raise ValueError(succ)
363 if prec in succs:
363 if prec in succs:
364 raise ValueError(_('in-marker cycle with %s') % node.hex(prec))
364 raise ValueError(_('in-marker cycle with %s') % node.hex(prec))
365 marker = (str(prec), tuple(succs), int(flag), encodemeta(metadata),
365 marker = (str(prec), tuple(succs), int(flag), encodemeta(metadata),
366 date, parents)
366 date, parents)
367 return bool(self.add(transaction, [marker]))
367 return bool(self.add(transaction, [marker]))
368
368
369 def add(self, transaction, markers):
369 def add(self, transaction, markers):
370 """Add new markers to the store
370 """Add new markers to the store
371
371
372 Take care of filtering duplicate.
372 Take care of filtering duplicate.
373 Return the number of new marker."""
373 Return the number of new marker."""
374 if not _enabled:
374 if not _enabled:
375 raise util.Abort('obsolete feature is not enabled on this repo')
375 raise util.Abort('obsolete feature is not enabled on this repo')
376 known = set(self._all)
376 known = set(self._all)
377 new = []
377 new = []
378 for m in markers:
378 for m in markers:
379 if m not in known:
379 if m not in known:
380 known.add(m)
380 known.add(m)
381 new.append(m)
381 new.append(m)
382 if new:
382 if new:
383 f = self.sopener('obsstore', 'ab')
383 f = self.sopener('obsstore', 'ab')
384 try:
384 try:
385 # Whether the file's current position is at the begin or at
385 # Whether the file's current position is at the begin or at
386 # the end after opening a file for appending is implementation
386 # the end after opening a file for appending is implementation
387 # defined. So we must seek to the end before calling tell(),
387 # defined. So we must seek to the end before calling tell(),
388 # or we may get a zero offset for non-zero sized files on
388 # or we may get a zero offset for non-zero sized files on
389 # some platforms (issue3543).
389 # some platforms (issue3543).
390 f.seek(0, _SEEK_END)
390 f.seek(0, _SEEK_END)
391 offset = f.tell()
391 offset = f.tell()
392 transaction.add('obsstore', offset)
392 transaction.add('obsstore', offset)
393 # offset == 0: new file - add the version header
393 # offset == 0: new file - add the version header
394 for bytes in encodemarkers(new, offset == 0, self._version):
394 for bytes in encodemarkers(new, offset == 0, self._version):
395 f.write(bytes)
395 f.write(bytes)
396 finally:
396 finally:
397 # XXX: f.close() == filecache invalidation == obsstore rebuilt.
397 # XXX: f.close() == filecache invalidation == obsstore rebuilt.
398 # call 'filecacheentry.refresh()' here
398 # call 'filecacheentry.refresh()' here
399 f.close()
399 f.close()
400 self._load(new)
400 self._load(new)
401 # new marker *may* have changed several set. invalidate the cache.
401 # new marker *may* have changed several set. invalidate the cache.
402 self.caches.clear()
402 self.caches.clear()
403 # records the number of new markers for the transaction hooks
403 # records the number of new markers for the transaction hooks
404 previous = int(transaction.hookargs.get('new_obsmarkers', '0'))
404 previous = int(transaction.hookargs.get('new_obsmarkers', '0'))
405 transaction.hookargs['new_obsmarkers'] = str(previous + len(new))
405 transaction.hookargs['new_obsmarkers'] = str(previous + len(new))
406 return len(new)
406 return len(new)
407
407
408 def mergemarkers(self, transaction, data):
408 def mergemarkers(self, transaction, data):
409 """merge a binary stream of markers inside the obsstore
409 """merge a binary stream of markers inside the obsstore
410
410
411 Returns the number of new markers added."""
411 Returns the number of new markers added."""
412 version, markers = _readmarkers(data)
412 version, markers = _readmarkers(data)
413 return self.add(transaction, markers)
413 return self.add(transaction, markers)
414
414
415 def _load(self, markers):
415 def _load(self, markers):
416 for mark in markers:
416 for mark in markers:
417 self._all.append(mark)
417 self._all.append(mark)
418 pre, sucs = mark[:2]
418 pre, sucs = mark[:2]
419 self.successors.setdefault(pre, set()).add(mark)
419 self.successors.setdefault(pre, set()).add(mark)
420 for suc in sucs:
420 for suc in sucs:
421 self.precursors.setdefault(suc, set()).add(mark)
421 self.precursors.setdefault(suc, set()).add(mark)
422 parents = mark[5]
422 parents = mark[5]
423 if parents is not None:
423 if parents is not None:
424 for p in parents:
424 for p in parents:
425 self.children.setdefault(p, set()).add(mark)
425 self.children.setdefault(p, set()).add(mark)
426 if node.nullid in self.precursors:
426 if node.nullid in self.precursors:
427 raise util.Abort(_('bad obsolescence marker detected: '
427 raise util.Abort(_('bad obsolescence marker detected: '
428 'invalid successors nullid'))
428 'invalid successors nullid'))
429 def relevantmarkers(self, nodes):
429 def relevantmarkers(self, nodes):
430 """return a set of all obsolescence markers relevant to a set of nodes.
430 """return a set of all obsolescence markers relevant to a set of nodes.
431
431
432 "relevant" to a set of nodes mean:
432 "relevant" to a set of nodes mean:
433
433
434 - marker that use this changeset as successor
434 - marker that use this changeset as successor
435 - prune marker of direct children on this changeset
435 - prune marker of direct children on this changeset
436 - recursive application of the two rules on precursors of these markers
436 - recursive application of the two rules on precursors of these markers
437
437
438 It is a set so you cannot rely on order."""
438 It is a set so you cannot rely on order."""
439
439
440 pendingnodes = set(nodes)
440 pendingnodes = set(nodes)
441 seenmarkers = set()
441 seenmarkers = set()
442 seennodes = set(pendingnodes)
442 seennodes = set(pendingnodes)
443 precursorsmarkers = self.precursors
443 precursorsmarkers = self.precursors
444 children = self.children
444 children = self.children
445 while pendingnodes:
445 while pendingnodes:
446 direct = set()
446 direct = set()
447 for current in pendingnodes:
447 for current in pendingnodes:
448 direct.update(precursorsmarkers.get(current, ()))
448 direct.update(precursorsmarkers.get(current, ()))
449 pruned = [m for m in children.get(current, ()) if not m[1]]
449 pruned = [m for m in children.get(current, ()) if not m[1]]
450 direct.update(pruned)
450 direct.update(pruned)
451 direct -= seenmarkers
451 direct -= seenmarkers
452 pendingnodes = set([m[0] for m in direct])
452 pendingnodes = set([m[0] for m in direct])
453 seenmarkers |= direct
453 seenmarkers |= direct
454 pendingnodes -= seennodes
454 pendingnodes -= seennodes
455 seennodes |= pendingnodes
455 seennodes |= pendingnodes
456 return seenmarkers
456 return seenmarkers
457
457
458 def commonversion(versions):
458 def commonversion(versions):
459 """Return the newest version listed in both versions and our local formats.
459 """Return the newest version listed in both versions and our local formats.
460
460
461 Returns None if no common version exists.
461 Returns None if no common version exists.
462 """
462 """
463 versions.sort(reverse=True)
463 versions.sort(reverse=True)
464 # search for highest version known on both side
464 # search for highest version known on both side
465 for v in versions:
465 for v in versions:
466 if v in formats:
466 if v in formats:
467 return v
467 return v
468 return None
468 return None
469
469
470 # arbitrary picked to fit into 8K limit from HTTP server
470 # arbitrary picked to fit into 8K limit from HTTP server
471 # you have to take in account:
471 # you have to take in account:
472 # - the version header
472 # - the version header
473 # - the base85 encoding
473 # - the base85 encoding
474 _maxpayload = 5300
474 _maxpayload = 5300
475
475
476 def _pushkeyescape(markers):
476 def _pushkeyescape(markers):
477 """encode markers into a dict suitable for pushkey exchange
477 """encode markers into a dict suitable for pushkey exchange
478
478
479 - binary data is base85 encoded
479 - binary data is base85 encoded
480 - split in chunks smaller than 5300 bytes"""
480 - split in chunks smaller than 5300 bytes"""
481 keys = {}
481 keys = {}
482 parts = []
482 parts = []
483 currentlen = _maxpayload * 2 # ensure we create a new part
483 currentlen = _maxpayload * 2 # ensure we create a new part
484 for marker in markers:
484 for marker in markers:
485 nextdata = _fm0encodeonemarker(marker)
485 nextdata = _fm0encodeonemarker(marker)
486 if (len(nextdata) + currentlen > _maxpayload):
486 if (len(nextdata) + currentlen > _maxpayload):
487 currentpart = []
487 currentpart = []
488 currentlen = 0
488 currentlen = 0
489 parts.append(currentpart)
489 parts.append(currentpart)
490 currentpart.append(nextdata)
490 currentpart.append(nextdata)
491 currentlen += len(nextdata)
491 currentlen += len(nextdata)
492 for idx, part in enumerate(reversed(parts)):
492 for idx, part in enumerate(reversed(parts)):
493 data = ''.join([_pack('>B', _fm0version)] + part)
493 data = ''.join([_pack('>B', _fm0version)] + part)
494 keys['dump%i' % idx] = base85.b85encode(data)
494 keys['dump%i' % idx] = base85.b85encode(data)
495 return keys
495 return keys
496
496
497 def listmarkers(repo):
497 def listmarkers(repo):
498 """List markers over pushkey"""
498 """List markers over pushkey"""
499 if not repo.obsstore:
499 if not repo.obsstore:
500 return {}
500 return {}
501 return _pushkeyescape(repo.obsstore)
501 return _pushkeyescape(repo.obsstore)
502
502
503 def pushmarker(repo, key, old, new):
503 def pushmarker(repo, key, old, new):
504 """Push markers over pushkey"""
504 """Push markers over pushkey"""
505 if not key.startswith('dump'):
505 if not key.startswith('dump'):
506 repo.ui.warn(_('unknown key: %r') % key)
506 repo.ui.warn(_('unknown key: %r') % key)
507 return 0
507 return 0
508 if old:
508 if old:
509 repo.ui.warn(_('unexpected old value for %r') % key)
509 repo.ui.warn(_('unexpected old value for %r') % key)
510 return 0
510 return 0
511 data = base85.b85decode(new)
511 data = base85.b85decode(new)
512 lock = repo.lock()
512 lock = repo.lock()
513 try:
513 try:
514 tr = repo.transaction('pushkey: obsolete markers')
514 tr = repo.transaction('pushkey: obsolete markers')
515 try:
515 try:
516 repo.obsstore.mergemarkers(tr, data)
516 repo.obsstore.mergemarkers(tr, data)
517 tr.close()
517 tr.close()
518 return 1
518 return 1
519 finally:
519 finally:
520 tr.release()
520 tr.release()
521 finally:
521 finally:
522 lock.release()
522 lock.release()
523
523
524 def getmarkers(repo, nodes=None):
524 def getmarkers(repo, nodes=None):
525 """returns markers known in a repository
525 """returns markers known in a repository
526
526
527 If <nodes> is specified, only markers "relevant" to those nodes are are
527 If <nodes> is specified, only markers "relevant" to those nodes are are
528 returned"""
528 returned"""
529 if nodes is None:
529 if nodes is None:
530 rawmarkers = repo.obsstore
530 rawmarkers = repo.obsstore
531 else:
531 else:
532 rawmarkers = repo.obsstore.relevantmarkers(nodes)
532 rawmarkers = repo.obsstore.relevantmarkers(nodes)
533
533
534 for markerdata in rawmarkers:
534 for markerdata in rawmarkers:
535 yield marker(repo, markerdata)
535 yield marker(repo, markerdata)
536
536
537 def relevantmarkers(repo, node):
537 def relevantmarkers(repo, node):
538 """all obsolete markers relevant to some revision"""
538 """all obsolete markers relevant to some revision"""
539 for markerdata in repo.obsstore.relevantmarkers(node):
539 for markerdata in repo.obsstore.relevantmarkers(node):
540 yield marker(repo, markerdata)
540 yield marker(repo, markerdata)
541
541
542
542
543 def precursormarkers(ctx):
543 def precursormarkers(ctx):
544 """obsolete marker marking this changeset as a successors"""
544 """obsolete marker marking this changeset as a successors"""
545 for data in ctx._repo.obsstore.precursors.get(ctx.node(), ()):
545 for data in ctx._repo.obsstore.precursors.get(ctx.node(), ()):
546 yield marker(ctx._repo, data)
546 yield marker(ctx._repo, data)
547
547
548 def successormarkers(ctx):
548 def successormarkers(ctx):
549 """obsolete marker making this changeset obsolete"""
549 """obsolete marker making this changeset obsolete"""
550 for data in ctx._repo.obsstore.successors.get(ctx.node(), ()):
550 for data in ctx._repo.obsstore.successors.get(ctx.node(), ()):
551 yield marker(ctx._repo, data)
551 yield marker(ctx._repo, data)
552
552
553 def allsuccessors(obsstore, nodes, ignoreflags=0):
553 def allsuccessors(obsstore, nodes, ignoreflags=0):
554 """Yield node for every successor of <nodes>.
554 """Yield node for every successor of <nodes>.
555
555
556 Some successors may be unknown locally.
556 Some successors may be unknown locally.
557
557
558 This is a linear yield unsuited to detecting split changesets. It includes
558 This is a linear yield unsuited to detecting split changesets. It includes
559 initial nodes too."""
559 initial nodes too."""
560 remaining = set(nodes)
560 remaining = set(nodes)
561 seen = set(remaining)
561 seen = set(remaining)
562 while remaining:
562 while remaining:
563 current = remaining.pop()
563 current = remaining.pop()
564 yield current
564 yield current
565 for mark in obsstore.successors.get(current, ()):
565 for mark in obsstore.successors.get(current, ()):
566 # ignore marker flagged with specified flag
566 # ignore marker flagged with specified flag
567 if mark[2] & ignoreflags:
567 if mark[2] & ignoreflags:
568 continue
568 continue
569 for suc in mark[1]:
569 for suc in mark[1]:
570 if suc not in seen:
570 if suc not in seen:
571 seen.add(suc)
571 seen.add(suc)
572 remaining.add(suc)
572 remaining.add(suc)
573
573
574 def allprecursors(obsstore, nodes, ignoreflags=0):
574 def allprecursors(obsstore, nodes, ignoreflags=0):
575 """Yield node for every precursors of <nodes>.
575 """Yield node for every precursors of <nodes>.
576
576
577 Some precursors may be unknown locally.
577 Some precursors may be unknown locally.
578
578
579 This is a linear yield unsuited to detecting folded changesets. It includes
579 This is a linear yield unsuited to detecting folded changesets. It includes
580 initial nodes too."""
580 initial nodes too."""
581
581
582 remaining = set(nodes)
582 remaining = set(nodes)
583 seen = set(remaining)
583 seen = set(remaining)
584 while remaining:
584 while remaining:
585 current = remaining.pop()
585 current = remaining.pop()
586 yield current
586 yield current
587 for mark in obsstore.precursors.get(current, ()):
587 for mark in obsstore.precursors.get(current, ()):
588 # ignore marker flagged with specified flag
588 # ignore marker flagged with specified flag
589 if mark[2] & ignoreflags:
589 if mark[2] & ignoreflags:
590 continue
590 continue
591 suc = mark[0]
591 suc = mark[0]
592 if suc not in seen:
592 if suc not in seen:
593 seen.add(suc)
593 seen.add(suc)
594 remaining.add(suc)
594 remaining.add(suc)
595
595
596 def foreground(repo, nodes):
596 def foreground(repo, nodes):
597 """return all nodes in the "foreground" of other node
597 """return all nodes in the "foreground" of other node
598
598
599 The foreground of a revision is anything reachable using parent -> children
599 The foreground of a revision is anything reachable using parent -> children
600 or precursor -> successor relation. It is very similar to "descendant" but
600 or precursor -> successor relation. It is very similar to "descendant" but
601 augmented with obsolescence information.
601 augmented with obsolescence information.
602
602
603 Beware that possible obsolescence cycle may result if complex situation.
603 Beware that possible obsolescence cycle may result if complex situation.
604 """
604 """
605 repo = repo.unfiltered()
605 repo = repo.unfiltered()
606 foreground = set(repo.set('%ln::', nodes))
606 foreground = set(repo.set('%ln::', nodes))
607 if repo.obsstore:
607 if repo.obsstore:
608 # We only need this complicated logic if there is obsolescence
608 # We only need this complicated logic if there is obsolescence
609 # XXX will probably deserve an optimised revset.
609 # XXX will probably deserve an optimised revset.
610 nm = repo.changelog.nodemap
610 nm = repo.changelog.nodemap
611 plen = -1
611 plen = -1
612 # compute the whole set of successors or descendants
612 # compute the whole set of successors or descendants
613 while len(foreground) != plen:
613 while len(foreground) != plen:
614 plen = len(foreground)
614 plen = len(foreground)
615 succs = set(c.node() for c in foreground)
615 succs = set(c.node() for c in foreground)
616 mutable = [c.node() for c in foreground if c.mutable()]
616 mutable = [c.node() for c in foreground if c.mutable()]
617 succs.update(allsuccessors(repo.obsstore, mutable))
617 succs.update(allsuccessors(repo.obsstore, mutable))
618 known = (n for n in succs if n in nm)
618 known = (n for n in succs if n in nm)
619 foreground = set(repo.set('%ln::', known))
619 foreground = set(repo.set('%ln::', known))
620 return set(c.node() for c in foreground)
620 return set(c.node() for c in foreground)
621
621
622
622
623 def successorssets(repo, initialnode, cache=None):
623 def successorssets(repo, initialnode, cache=None):
624 """Return all set of successors of initial nodes
624 """Return all set of successors of initial nodes
625
625
626 The successors set of a changeset A are a group of revisions that succeed
626 The successors set of a changeset A are a group of revisions that succeed
627 A. It succeeds A as a consistent whole, each revision being only a partial
627 A. It succeeds A as a consistent whole, each revision being only a partial
628 replacement. The successors set contains non-obsolete changesets only.
628 replacement. The successors set contains non-obsolete changesets only.
629
629
630 This function returns the full list of successor sets which is why it
630 This function returns the full list of successor sets which is why it
631 returns a list of tuples and not just a single tuple. Each tuple is a valid
631 returns a list of tuples and not just a single tuple. Each tuple is a valid
632 successors set. Not that (A,) may be a valid successors set for changeset A
632 successors set. Not that (A,) may be a valid successors set for changeset A
633 (see below).
633 (see below).
634
634
635 In most cases, a changeset A will have a single element (e.g. the changeset
635 In most cases, a changeset A will have a single element (e.g. the changeset
636 A is replaced by A') in its successors set. Though, it is also common for a
636 A is replaced by A') in its successors set. Though, it is also common for a
637 changeset A to have no elements in its successor set (e.g. the changeset
637 changeset A to have no elements in its successor set (e.g. the changeset
638 has been pruned). Therefore, the returned list of successors sets will be
638 has been pruned). Therefore, the returned list of successors sets will be
639 [(A',)] or [], respectively.
639 [(A',)] or [], respectively.
640
640
641 When a changeset A is split into A' and B', however, it will result in a
641 When a changeset A is split into A' and B', however, it will result in a
642 successors set containing more than a single element, i.e. [(A',B')].
642 successors set containing more than a single element, i.e. [(A',B')].
643 Divergent changesets will result in multiple successors sets, i.e. [(A',),
643 Divergent changesets will result in multiple successors sets, i.e. [(A',),
644 (A'')].
644 (A'')].
645
645
646 If a changeset A is not obsolete, then it will conceptually have no
646 If a changeset A is not obsolete, then it will conceptually have no
647 successors set. To distinguish this from a pruned changeset, the successor
647 successors set. To distinguish this from a pruned changeset, the successor
648 set will only contain itself, i.e. [(A,)].
648 set will only contain itself, i.e. [(A,)].
649
649
650 Finally, successors unknown locally are considered to be pruned (obsoleted
650 Finally, successors unknown locally are considered to be pruned (obsoleted
651 without any successors).
651 without any successors).
652
652
653 The optional `cache` parameter is a dictionary that may contain precomputed
653 The optional `cache` parameter is a dictionary that may contain precomputed
654 successors sets. It is meant to reuse the computation of a previous call to
654 successors sets. It is meant to reuse the computation of a previous call to
655 `successorssets` when multiple calls are made at the same time. The cache
655 `successorssets` when multiple calls are made at the same time. The cache
656 dictionary is updated in place. The caller is responsible for its live
656 dictionary is updated in place. The caller is responsible for its live
657 spawn. Code that makes multiple calls to `successorssets` *must* use this
657 spawn. Code that makes multiple calls to `successorssets` *must* use this
658 cache mechanism or suffer terrible performances.
658 cache mechanism or suffer terrible performances.
659
659
660 """
660 """
661
661
662 succmarkers = repo.obsstore.successors
662 succmarkers = repo.obsstore.successors
663
663
664 # Stack of nodes we search successors sets for
664 # Stack of nodes we search successors sets for
665 toproceed = [initialnode]
665 toproceed = [initialnode]
666 # set version of above list for fast loop detection
666 # set version of above list for fast loop detection
667 # element added to "toproceed" must be added here
667 # element added to "toproceed" must be added here
668 stackedset = set(toproceed)
668 stackedset = set(toproceed)
669 if cache is None:
669 if cache is None:
670 cache = {}
670 cache = {}
671
671
672 # This while loop is the flattened version of a recursive search for
672 # This while loop is the flattened version of a recursive search for
673 # successors sets
673 # successors sets
674 #
674 #
675 # def successorssets(x):
675 # def successorssets(x):
676 # successors = directsuccessors(x)
676 # successors = directsuccessors(x)
677 # ss = [[]]
677 # ss = [[]]
678 # for succ in directsuccessors(x):
678 # for succ in directsuccessors(x):
679 # # product as in itertools cartesian product
679 # # product as in itertools cartesian product
680 # ss = product(ss, successorssets(succ))
680 # ss = product(ss, successorssets(succ))
681 # return ss
681 # return ss
682 #
682 #
683 # But we can not use plain recursive calls here:
683 # But we can not use plain recursive calls here:
684 # - that would blow the python call stack
684 # - that would blow the python call stack
685 # - obsolescence markers may have cycles, we need to handle them.
685 # - obsolescence markers may have cycles, we need to handle them.
686 #
686 #
687 # The `toproceed` list act as our call stack. Every node we search
687 # The `toproceed` list act as our call stack. Every node we search
688 # successors set for are stacked there.
688 # successors set for are stacked there.
689 #
689 #
690 # The `stackedset` is set version of this stack used to check if a node is
690 # The `stackedset` is set version of this stack used to check if a node is
691 # already stacked. This check is used to detect cycles and prevent infinite
691 # already stacked. This check is used to detect cycles and prevent infinite
692 # loop.
692 # loop.
693 #
693 #
694 # successors set of all nodes are stored in the `cache` dictionary.
694 # successors set of all nodes are stored in the `cache` dictionary.
695 #
695 #
696 # After this while loop ends we use the cache to return the successors sets
696 # After this while loop ends we use the cache to return the successors sets
697 # for the node requested by the caller.
697 # for the node requested by the caller.
698 while toproceed:
698 while toproceed:
699 # Every iteration tries to compute the successors sets of the topmost
699 # Every iteration tries to compute the successors sets of the topmost
700 # node of the stack: CURRENT.
700 # node of the stack: CURRENT.
701 #
701 #
702 # There are four possible outcomes:
702 # There are four possible outcomes:
703 #
703 #
704 # 1) We already know the successors sets of CURRENT:
704 # 1) We already know the successors sets of CURRENT:
705 # -> mission accomplished, pop it from the stack.
705 # -> mission accomplished, pop it from the stack.
706 # 2) Node is not obsolete:
706 # 2) Node is not obsolete:
707 # -> the node is its own successors sets. Add it to the cache.
707 # -> the node is its own successors sets. Add it to the cache.
708 # 3) We do not know successors set of direct successors of CURRENT:
708 # 3) We do not know successors set of direct successors of CURRENT:
709 # -> We add those successors to the stack.
709 # -> We add those successors to the stack.
710 # 4) We know successors sets of all direct successors of CURRENT:
710 # 4) We know successors sets of all direct successors of CURRENT:
711 # -> We can compute CURRENT successors set and add it to the
711 # -> We can compute CURRENT successors set and add it to the
712 # cache.
712 # cache.
713 #
713 #
714 current = toproceed[-1]
714 current = toproceed[-1]
715 if current in cache:
715 if current in cache:
716 # case (1): We already know the successors sets
716 # case (1): We already know the successors sets
717 stackedset.remove(toproceed.pop())
717 stackedset.remove(toproceed.pop())
718 elif current not in succmarkers:
718 elif current not in succmarkers:
719 # case (2): The node is not obsolete.
719 # case (2): The node is not obsolete.
720 if current in repo:
720 if current in repo:
721 # We have a valid last successors.
721 # We have a valid last successors.
722 cache[current] = [(current,)]
722 cache[current] = [(current,)]
723 else:
723 else:
724 # Final obsolete version is unknown locally.
724 # Final obsolete version is unknown locally.
725 # Do not count that as a valid successors
725 # Do not count that as a valid successors
726 cache[current] = []
726 cache[current] = []
727 else:
727 else:
728 # cases (3) and (4)
728 # cases (3) and (4)
729 #
729 #
730 # We proceed in two phases. Phase 1 aims to distinguish case (3)
730 # We proceed in two phases. Phase 1 aims to distinguish case (3)
731 # from case (4):
731 # from case (4):
732 #
732 #
733 # For each direct successors of CURRENT, we check whether its
733 # For each direct successors of CURRENT, we check whether its
734 # successors sets are known. If they are not, we stack the
734 # successors sets are known. If they are not, we stack the
735 # unknown node and proceed to the next iteration of the while
735 # unknown node and proceed to the next iteration of the while
736 # loop. (case 3)
736 # loop. (case 3)
737 #
737 #
738 # During this step, we may detect obsolescence cycles: a node
738 # During this step, we may detect obsolescence cycles: a node
739 # with unknown successors sets but already in the call stack.
739 # with unknown successors sets but already in the call stack.
740 # In such a situation, we arbitrary set the successors sets of
740 # In such a situation, we arbitrary set the successors sets of
741 # the node to nothing (node pruned) to break the cycle.
741 # the node to nothing (node pruned) to break the cycle.
742 #
742 #
743 # If no break was encountered we proceed to phase 2.
743 # If no break was encountered we proceed to phase 2.
744 #
744 #
745 # Phase 2 computes successors sets of CURRENT (case 4); see details
745 # Phase 2 computes successors sets of CURRENT (case 4); see details
746 # in phase 2 itself.
746 # in phase 2 itself.
747 #
747 #
748 # Note the two levels of iteration in each phase.
748 # Note the two levels of iteration in each phase.
749 # - The first one handles obsolescence markers using CURRENT as
749 # - The first one handles obsolescence markers using CURRENT as
750 # precursor (successors markers of CURRENT).
750 # precursor (successors markers of CURRENT).
751 #
751 #
752 # Having multiple entry here means divergence.
752 # Having multiple entry here means divergence.
753 #
753 #
754 # - The second one handles successors defined in each marker.
754 # - The second one handles successors defined in each marker.
755 #
755 #
756 # Having none means pruned node, multiple successors means split,
756 # Having none means pruned node, multiple successors means split,
757 # single successors are standard replacement.
757 # single successors are standard replacement.
758 #
758 #
759 for mark in sorted(succmarkers[current]):
759 for mark in sorted(succmarkers[current]):
760 for suc in mark[1]:
760 for suc in mark[1]:
761 if suc not in cache:
761 if suc not in cache:
762 if suc in stackedset:
762 if suc in stackedset:
763 # cycle breaking
763 # cycle breaking
764 cache[suc] = []
764 cache[suc] = []
765 else:
765 else:
766 # case (3) If we have not computed successors sets
766 # case (3) If we have not computed successors sets
767 # of one of those successors we add it to the
767 # of one of those successors we add it to the
768 # `toproceed` stack and stop all work for this
768 # `toproceed` stack and stop all work for this
769 # iteration.
769 # iteration.
770 toproceed.append(suc)
770 toproceed.append(suc)
771 stackedset.add(suc)
771 stackedset.add(suc)
772 break
772 break
773 else:
773 else:
774 continue
774 continue
775 break
775 break
776 else:
776 else:
777 # case (4): we know all successors sets of all direct
777 # case (4): we know all successors sets of all direct
778 # successors
778 # successors
779 #
779 #
780 # Successors set contributed by each marker depends on the
780 # Successors set contributed by each marker depends on the
781 # successors sets of all its "successors" node.
781 # successors sets of all its "successors" node.
782 #
782 #
783 # Each different marker is a divergence in the obsolescence
783 # Each different marker is a divergence in the obsolescence
784 # history. It contributes successors sets distinct from other
784 # history. It contributes successors sets distinct from other
785 # markers.
785 # markers.
786 #
786 #
787 # Within a marker, a successor may have divergent successors
787 # Within a marker, a successor may have divergent successors
788 # sets. In such a case, the marker will contribute multiple
788 # sets. In such a case, the marker will contribute multiple
789 # divergent successors sets. If multiple successors have
789 # divergent successors sets. If multiple successors have
790 # divergent successors sets, a Cartesian product is used.
790 # divergent successors sets, a Cartesian product is used.
791 #
791 #
792 # At the end we post-process successors sets to remove
792 # At the end we post-process successors sets to remove
793 # duplicated entry and successors set that are strict subset of
793 # duplicated entry and successors set that are strict subset of
794 # another one.
794 # another one.
795 succssets = []
795 succssets = []
796 for mark in sorted(succmarkers[current]):
796 for mark in sorted(succmarkers[current]):
797 # successors sets contributed by this marker
797 # successors sets contributed by this marker
798 markss = [[]]
798 markss = [[]]
799 for suc in mark[1]:
799 for suc in mark[1]:
800 # cardinal product with previous successors
800 # cardinal product with previous successors
801 productresult = []
801 productresult = []
802 for prefix in markss:
802 for prefix in markss:
803 for suffix in cache[suc]:
803 for suffix in cache[suc]:
804 newss = list(prefix)
804 newss = list(prefix)
805 for part in suffix:
805 for part in suffix:
806 # do not duplicated entry in successors set
806 # do not duplicated entry in successors set
807 # first entry wins.
807 # first entry wins.
808 if part not in newss:
808 if part not in newss:
809 newss.append(part)
809 newss.append(part)
810 productresult.append(newss)
810 productresult.append(newss)
811 markss = productresult
811 markss = productresult
812 succssets.extend(markss)
812 succssets.extend(markss)
813 # remove duplicated and subset
813 # remove duplicated and subset
814 seen = []
814 seen = []
815 final = []
815 final = []
816 candidate = sorted(((set(s), s) for s in succssets if s),
816 candidate = sorted(((set(s), s) for s in succssets if s),
817 key=lambda x: len(x[1]), reverse=True)
817 key=lambda x: len(x[1]), reverse=True)
818 for setversion, listversion in candidate:
818 for setversion, listversion in candidate:
819 for seenset in seen:
819 for seenset in seen:
820 if setversion.issubset(seenset):
820 if setversion.issubset(seenset):
821 break
821 break
822 else:
822 else:
823 final.append(listversion)
823 final.append(listversion)
824 seen.append(setversion)
824 seen.append(setversion)
825 final.reverse() # put small successors set first
825 final.reverse() # put small successors set first
826 cache[current] = final
826 cache[current] = final
827 return cache[initialnode]
827 return cache[initialnode]
828
828
829 def _knownrevs(repo, nodes):
829 def _knownrevs(repo, nodes):
830 """yield revision numbers of known nodes passed in parameters
830 """yield revision numbers of known nodes passed in parameters
831
831
832 Unknown revisions are silently ignored."""
832 Unknown revisions are silently ignored."""
833 torev = repo.changelog.nodemap.get
833 torev = repo.changelog.nodemap.get
834 for n in nodes:
834 for n in nodes:
835 rev = torev(n)
835 rev = torev(n)
836 if rev is not None:
836 if rev is not None:
837 yield rev
837 yield rev
838
838
839 # mapping of 'set-name' -> <function to compute this set>
839 # mapping of 'set-name' -> <function to compute this set>
840 cachefuncs = {}
840 cachefuncs = {}
841 def cachefor(name):
841 def cachefor(name):
842 """Decorator to register a function as computing the cache for a set"""
842 """Decorator to register a function as computing the cache for a set"""
843 def decorator(func):
843 def decorator(func):
844 assert name not in cachefuncs
844 assert name not in cachefuncs
845 cachefuncs[name] = func
845 cachefuncs[name] = func
846 return func
846 return func
847 return decorator
847 return decorator
848
848
849 def getrevs(repo, name):
849 def getrevs(repo, name):
850 """Return the set of revision that belong to the <name> set
850 """Return the set of revision that belong to the <name> set
851
851
852 Such access may compute the set and cache it for future use"""
852 Such access may compute the set and cache it for future use"""
853 repo = repo.unfiltered()
853 repo = repo.unfiltered()
854 if not repo.obsstore:
854 if not repo.obsstore:
855 return ()
855 return frozenset()
856 if name not in repo.obsstore.caches:
856 if name not in repo.obsstore.caches:
857 repo.obsstore.caches[name] = cachefuncs[name](repo)
857 repo.obsstore.caches[name] = cachefuncs[name](repo)
858 return repo.obsstore.caches[name]
858 return repo.obsstore.caches[name]
859
859
860 # To be simple we need to invalidate obsolescence cache when:
860 # To be simple we need to invalidate obsolescence cache when:
861 #
861 #
862 # - new changeset is added:
862 # - new changeset is added:
863 # - public phase is changed
863 # - public phase is changed
864 # - obsolescence marker are added
864 # - obsolescence marker are added
865 # - strip is used a repo
865 # - strip is used a repo
866 def clearobscaches(repo):
866 def clearobscaches(repo):
867 """Remove all obsolescence related cache from a repo
867 """Remove all obsolescence related cache from a repo
868
868
869 This remove all cache in obsstore is the obsstore already exist on the
869 This remove all cache in obsstore is the obsstore already exist on the
870 repo.
870 repo.
871
871
872 (We could be smarter here given the exact event that trigger the cache
872 (We could be smarter here given the exact event that trigger the cache
873 clearing)"""
873 clearing)"""
874 # only clear cache is there is obsstore data in this repo
874 # only clear cache is there is obsstore data in this repo
875 if 'obsstore' in repo._filecache:
875 if 'obsstore' in repo._filecache:
876 repo.obsstore.caches.clear()
876 repo.obsstore.caches.clear()
877
877
878 @cachefor('obsolete')
878 @cachefor('obsolete')
879 def _computeobsoleteset(repo):
879 def _computeobsoleteset(repo):
880 """the set of obsolete revisions"""
880 """the set of obsolete revisions"""
881 obs = set()
881 obs = set()
882 getrev = repo.changelog.nodemap.get
882 getrev = repo.changelog.nodemap.get
883 getphase = repo._phasecache.phase
883 getphase = repo._phasecache.phase
884 for n in repo.obsstore.successors:
884 for n in repo.obsstore.successors:
885 rev = getrev(n)
885 rev = getrev(n)
886 if rev is not None and getphase(repo, rev):
886 if rev is not None and getphase(repo, rev):
887 obs.add(rev)
887 obs.add(rev)
888 return obs
888 return obs
889
889
890 @cachefor('unstable')
890 @cachefor('unstable')
891 def _computeunstableset(repo):
891 def _computeunstableset(repo):
892 """the set of non obsolete revisions with obsolete parents"""
892 """the set of non obsolete revisions with obsolete parents"""
893 # revset is not efficient enough here
893 # revset is not efficient enough here
894 # we do (obsolete()::) - obsolete() by hand
894 # we do (obsolete()::) - obsolete() by hand
895 obs = getrevs(repo, 'obsolete')
895 obs = getrevs(repo, 'obsolete')
896 if not obs:
896 if not obs:
897 return set()
897 return set()
898 cl = repo.changelog
898 cl = repo.changelog
899 return set(r for r in cl.descendants(obs) if r not in obs)
899 return set(r for r in cl.descendants(obs) if r not in obs)
900
900
901 @cachefor('suspended')
901 @cachefor('suspended')
902 def _computesuspendedset(repo):
902 def _computesuspendedset(repo):
903 """the set of obsolete parents with non obsolete descendants"""
903 """the set of obsolete parents with non obsolete descendants"""
904 suspended = repo.changelog.ancestors(getrevs(repo, 'unstable'))
904 suspended = repo.changelog.ancestors(getrevs(repo, 'unstable'))
905 return set(r for r in getrevs(repo, 'obsolete') if r in suspended)
905 return set(r for r in getrevs(repo, 'obsolete') if r in suspended)
906
906
907 @cachefor('extinct')
907 @cachefor('extinct')
908 def _computeextinctset(repo):
908 def _computeextinctset(repo):
909 """the set of obsolete parents without non obsolete descendants"""
909 """the set of obsolete parents without non obsolete descendants"""
910 return getrevs(repo, 'obsolete') - getrevs(repo, 'suspended')
910 return getrevs(repo, 'obsolete') - getrevs(repo, 'suspended')
911
911
912
912
913 @cachefor('bumped')
913 @cachefor('bumped')
914 def _computebumpedset(repo):
914 def _computebumpedset(repo):
915 """the set of revs trying to obsolete public revisions"""
915 """the set of revs trying to obsolete public revisions"""
916 bumped = set()
916 bumped = set()
917 # util function (avoid attribute lookup in the loop)
917 # util function (avoid attribute lookup in the loop)
918 phase = repo._phasecache.phase # would be faster to grab the full list
918 phase = repo._phasecache.phase # would be faster to grab the full list
919 public = phases.public
919 public = phases.public
920 cl = repo.changelog
920 cl = repo.changelog
921 torev = cl.nodemap.get
921 torev = cl.nodemap.get
922 obs = getrevs(repo, 'obsolete')
922 obs = getrevs(repo, 'obsolete')
923 for rev in repo:
923 for rev in repo:
924 # We only evaluate mutable, non-obsolete revision
924 # We only evaluate mutable, non-obsolete revision
925 if (public < phase(repo, rev)) and (rev not in obs):
925 if (public < phase(repo, rev)) and (rev not in obs):
926 node = cl.node(rev)
926 node = cl.node(rev)
927 # (future) A cache of precursors may worth if split is very common
927 # (future) A cache of precursors may worth if split is very common
928 for pnode in allprecursors(repo.obsstore, [node],
928 for pnode in allprecursors(repo.obsstore, [node],
929 ignoreflags=bumpedfix):
929 ignoreflags=bumpedfix):
930 prev = torev(pnode) # unfiltered! but so is phasecache
930 prev = torev(pnode) # unfiltered! but so is phasecache
931 if (prev is not None) and (phase(repo, prev) <= public):
931 if (prev is not None) and (phase(repo, prev) <= public):
932 # we have a public precursors
932 # we have a public precursors
933 bumped.add(rev)
933 bumped.add(rev)
934 break # Next draft!
934 break # Next draft!
935 return bumped
935 return bumped
936
936
937 @cachefor('divergent')
937 @cachefor('divergent')
938 def _computedivergentset(repo):
938 def _computedivergentset(repo):
939 """the set of rev that compete to be the final successors of some revision.
939 """the set of rev that compete to be the final successors of some revision.
940 """
940 """
941 divergent = set()
941 divergent = set()
942 obsstore = repo.obsstore
942 obsstore = repo.obsstore
943 newermap = {}
943 newermap = {}
944 for ctx in repo.set('(not public()) - obsolete()'):
944 for ctx in repo.set('(not public()) - obsolete()'):
945 mark = obsstore.precursors.get(ctx.node(), ())
945 mark = obsstore.precursors.get(ctx.node(), ())
946 toprocess = set(mark)
946 toprocess = set(mark)
947 while toprocess:
947 while toprocess:
948 prec = toprocess.pop()[0]
948 prec = toprocess.pop()[0]
949 if prec not in newermap:
949 if prec not in newermap:
950 successorssets(repo, prec, newermap)
950 successorssets(repo, prec, newermap)
951 newer = [n for n in newermap[prec] if n]
951 newer = [n for n in newermap[prec] if n]
952 if len(newer) > 1:
952 if len(newer) > 1:
953 divergent.add(ctx.rev())
953 divergent.add(ctx.rev())
954 break
954 break
955 toprocess.update(obsstore.precursors.get(prec, ()))
955 toprocess.update(obsstore.precursors.get(prec, ()))
956 return divergent
956 return divergent
957
957
958
958
959 def createmarkers(repo, relations, flag=0, date=None, metadata=None):
959 def createmarkers(repo, relations, flag=0, date=None, metadata=None):
960 """Add obsolete markers between changesets in a repo
960 """Add obsolete markers between changesets in a repo
961
961
962 <relations> must be an iterable of (<old>, (<new>, ...)[,{metadata}])
962 <relations> must be an iterable of (<old>, (<new>, ...)[,{metadata}])
963 tuple. `old` and `news` are changectx. metadata is an optional dictionary
963 tuple. `old` and `news` are changectx. metadata is an optional dictionary
964 containing metadata for this marker only. It is merged with the global
964 containing metadata for this marker only. It is merged with the global
965 metadata specified through the `metadata` argument of this function,
965 metadata specified through the `metadata` argument of this function,
966
966
967 Trying to obsolete a public changeset will raise an exception.
967 Trying to obsolete a public changeset will raise an exception.
968
968
969 Current user and date are used except if specified otherwise in the
969 Current user and date are used except if specified otherwise in the
970 metadata attribute.
970 metadata attribute.
971
971
972 This function operates within a transaction of its own, but does
972 This function operates within a transaction of its own, but does
973 not take any lock on the repo.
973 not take any lock on the repo.
974 """
974 """
975 # prepare metadata
975 # prepare metadata
976 if metadata is None:
976 if metadata is None:
977 metadata = {}
977 metadata = {}
978 if 'user' not in metadata:
978 if 'user' not in metadata:
979 metadata['user'] = repo.ui.username()
979 metadata['user'] = repo.ui.username()
980 tr = repo.transaction('add-obsolescence-marker')
980 tr = repo.transaction('add-obsolescence-marker')
981 try:
981 try:
982 for rel in relations:
982 for rel in relations:
983 prec = rel[0]
983 prec = rel[0]
984 sucs = rel[1]
984 sucs = rel[1]
985 localmetadata = metadata.copy()
985 localmetadata = metadata.copy()
986 if 2 < len(rel):
986 if 2 < len(rel):
987 localmetadata.update(rel[2])
987 localmetadata.update(rel[2])
988
988
989 if not prec.mutable():
989 if not prec.mutable():
990 raise util.Abort("cannot obsolete immutable changeset: %s"
990 raise util.Abort("cannot obsolete immutable changeset: %s"
991 % prec)
991 % prec)
992 nprec = prec.node()
992 nprec = prec.node()
993 nsucs = tuple(s.node() for s in sucs)
993 nsucs = tuple(s.node() for s in sucs)
994 npare = None
994 npare = None
995 if not nsucs:
995 if not nsucs:
996 npare = tuple(p.node() for p in prec.parents())
996 npare = tuple(p.node() for p in prec.parents())
997 if nprec in nsucs:
997 if nprec in nsucs:
998 raise util.Abort("changeset %s cannot obsolete itself" % prec)
998 raise util.Abort("changeset %s cannot obsolete itself" % prec)
999 repo.obsstore.create(tr, nprec, nsucs, flag, parents=npare,
999 repo.obsstore.create(tr, nprec, nsucs, flag, parents=npare,
1000 date=date, metadata=localmetadata)
1000 date=date, metadata=localmetadata)
1001 repo.filteredrevcache.clear()
1001 repo.filteredrevcache.clear()
1002 tr.close()
1002 tr.close()
1003 finally:
1003 finally:
1004 tr.release()
1004 tr.release()
General Comments 0
You need to be logged in to leave comments. Login now