##// END OF EJS Templates
fix: make Fixer initialization more explicit for clarity...
Martin von Zweigbergk -
r43493:0101db49 default
parent child Browse files
Show More
@@ -1,870 +1,874 b''
1 1 # fix - rewrite file content in changesets and working copy
2 2 #
3 3 # Copyright 2018 Google LLC.
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2 or any later version.
7 7 """rewrite file content in changesets or working copy (EXPERIMENTAL)
8 8
9 9 Provides a command that runs configured tools on the contents of modified files,
10 10 writing back any fixes to the working copy or replacing changesets.
11 11
12 12 Here is an example configuration that causes :hg:`fix` to apply automatic
13 13 formatting fixes to modified lines in C++ code::
14 14
15 15 [fix]
16 16 clang-format:command=clang-format --assume-filename={rootpath}
17 17 clang-format:linerange=--lines={first}:{last}
18 18 clang-format:pattern=set:**.cpp or **.hpp
19 19
20 20 The :command suboption forms the first part of the shell command that will be
21 21 used to fix a file. The content of the file is passed on standard input, and the
22 22 fixed file content is expected on standard output. Any output on standard error
23 23 will be displayed as a warning. If the exit status is not zero, the file will
24 24 not be affected. A placeholder warning is displayed if there is a non-zero exit
25 25 status but no standard error output. Some values may be substituted into the
26 26 command::
27 27
28 28 {rootpath} The path of the file being fixed, relative to the repo root
29 29 {basename} The name of the file being fixed, without the directory path
30 30
31 31 If the :linerange suboption is set, the tool will only be run if there are
32 32 changed lines in a file. The value of this suboption is appended to the shell
33 33 command once for every range of changed lines in the file. Some values may be
34 34 substituted into the command::
35 35
36 36 {first} The 1-based line number of the first line in the modified range
37 37 {last} The 1-based line number of the last line in the modified range
38 38
39 39 Deleted sections of a file will be ignored by :linerange, because there is no
40 40 corresponding line range in the version being fixed.
41 41
42 42 By default, tools that set :linerange will only be executed if there is at least
43 43 one changed line range. This is meant to prevent accidents like running a code
44 44 formatter in such a way that it unexpectedly reformats the whole file. If such a
45 45 tool needs to operate on unchanged files, it should set the :skipclean suboption
46 46 to false.
47 47
48 48 The :pattern suboption determines which files will be passed through each
49 49 configured tool. See :hg:`help patterns` for possible values. If there are file
50 50 arguments to :hg:`fix`, the intersection of these patterns is used.
51 51
52 52 There is also a configurable limit for the maximum size of file that will be
53 53 processed by :hg:`fix`::
54 54
55 55 [fix]
56 56 maxfilesize = 2MB
57 57
58 58 Normally, execution of configured tools will continue after a failure (indicated
59 59 by a non-zero exit status). It can also be configured to abort after the first
60 60 such failure, so that no files will be affected if any tool fails. This abort
61 61 will also cause :hg:`fix` to exit with a non-zero status::
62 62
63 63 [fix]
64 64 failure = abort
65 65
66 66 When multiple tools are configured to affect a file, they execute in an order
67 67 defined by the :priority suboption. The priority suboption has a default value
68 68 of zero for each tool. Tools are executed in order of descending priority. The
69 69 execution order of tools with equal priority is unspecified. For example, you
70 70 could use the 'sort' and 'head' utilities to keep only the 10 smallest numbers
71 71 in a text file by ensuring that 'sort' runs before 'head'::
72 72
73 73 [fix]
74 74 sort:command = sort -n
75 75 head:command = head -n 10
76 76 sort:pattern = numbers.txt
77 77 head:pattern = numbers.txt
78 78 sort:priority = 2
79 79 head:priority = 1
80 80
81 81 To account for changes made by each tool, the line numbers used for incremental
82 82 formatting are recomputed before executing the next tool. So, each tool may see
83 83 different values for the arguments added by the :linerange suboption.
84 84
85 85 Each fixer tool is allowed to return some metadata in addition to the fixed file
86 86 content. The metadata must be placed before the file content on stdout,
87 87 separated from the file content by a zero byte. The metadata is parsed as a JSON
88 88 value (so, it should be UTF-8 encoded and contain no zero bytes). A fixer tool
89 89 is expected to produce this metadata encoding if and only if the :metadata
90 90 suboption is true::
91 91
92 92 [fix]
93 93 tool:command = tool --prepend-json-metadata
94 94 tool:metadata = true
95 95
96 96 The metadata values are passed to hooks, which can be used to print summaries or
97 97 perform other post-fixing work. The supported hooks are::
98 98
99 99 "postfixfile"
100 100 Run once for each file in each revision where any fixer tools made changes
101 101 to the file content. Provides "$HG_REV" and "$HG_PATH" to identify the file,
102 102 and "$HG_METADATA" with a map of fixer names to metadata values from fixer
103 103 tools that affected the file. Fixer tools that didn't affect the file have a
104 104 valueof None. Only fixer tools that executed are present in the metadata.
105 105
106 106 "postfix"
107 107 Run once after all files and revisions have been handled. Provides
108 108 "$HG_REPLACEMENTS" with information about what revisions were created and
109 109 made obsolete. Provides a boolean "$HG_WDIRWRITTEN" to indicate whether any
110 110 files in the working copy were updated. Provides a list "$HG_METADATA"
111 111 mapping fixer tool names to lists of metadata values returned from
112 112 executions that modified a file. This aggregates the same metadata
113 113 previously passed to the "postfixfile" hook.
114 114
115 115 Fixer tools are run the in repository's root directory. This allows them to read
116 116 configuration files from the working copy, or even write to the working copy.
117 117 The working copy is not updated to match the revision being fixed. In fact,
118 118 several revisions may be fixed in parallel. Writes to the working copy are not
119 119 amended into the revision being fixed; fixer tools should always write fixed
120 120 file content back to stdout as documented above.
121 121 """
122 122
123 123 from __future__ import absolute_import
124 124
125 125 import collections
126 126 import itertools
127 127 import json
128 128 import os
129 129 import re
130 130 import subprocess
131 131
132 132 from mercurial.i18n import _
133 133 from mercurial.node import nullrev
134 134 from mercurial.node import wdirrev
135 from mercurial.pycompat import setattr
136 135
137 from mercurial.utils import (
138 procutil,
139 stringutil,
140 )
136 from mercurial.utils import procutil
141 137
142 138 from mercurial import (
143 139 cmdutil,
144 140 context,
145 141 copies,
146 142 error,
147 143 mdiff,
148 144 merge,
149 145 obsolete,
150 146 pycompat,
151 147 registrar,
152 148 scmutil,
153 149 util,
154 150 worker,
155 151 )
156 152
157 153 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
158 154 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
159 155 # be specifying the version(s) of Mercurial they are tested with, or
160 156 # leave the attribute unspecified.
161 157 testedwith = b'ships-with-hg-core'
162 158
163 159 cmdtable = {}
164 160 command = registrar.command(cmdtable)
165 161
166 162 configtable = {}
167 163 configitem = registrar.configitem(configtable)
168 164
169 165 # Register the suboptions allowed for each configured fixer, and default values.
170 166 FIXER_ATTRS = {
171 167 b'command': None,
172 168 b'linerange': None,
173 169 b'pattern': None,
174 170 b'priority': 0,
175 b'metadata': b'false',
176 b'skipclean': b'true',
177 b'enabled': b'true',
171 b'metadata': False,
172 b'skipclean': True,
173 b'enabled': True,
178 174 }
179 175
180 176 for key, default in FIXER_ATTRS.items():
181 177 configitem(b'fix', b'.*:%s$' % key, default=default, generic=True)
182 178
183 179 # A good default size allows most source code files to be fixed, but avoids
184 180 # letting fixer tools choke on huge inputs, which could be surprising to the
185 181 # user.
186 182 configitem(b'fix', b'maxfilesize', default=b'2MB')
187 183
188 184 # Allow fix commands to exit non-zero if an executed fixer tool exits non-zero.
189 185 # This helps users do shell scripts that stop when a fixer tool signals a
190 186 # problem.
191 187 configitem(b'fix', b'failure', default=b'continue')
192 188
193 189
194 190 def checktoolfailureaction(ui, message, hint=None):
195 191 """Abort with 'message' if fix.failure=abort"""
196 192 action = ui.config(b'fix', b'failure')
197 193 if action not in (b'continue', b'abort'):
198 194 raise error.Abort(
199 195 _(b'unknown fix.failure action: %s') % (action,),
200 196 hint=_(b'use "continue" or "abort"'),
201 197 )
202 198 if action == b'abort':
203 199 raise error.Abort(message, hint=hint)
204 200
205 201
206 202 allopt = (b'', b'all', False, _(b'fix all non-public non-obsolete revisions'))
207 203 baseopt = (
208 204 b'',
209 205 b'base',
210 206 [],
211 207 _(
212 208 b'revisions to diff against (overrides automatic '
213 209 b'selection, and applies to every revision being '
214 210 b'fixed)'
215 211 ),
216 212 _(b'REV'),
217 213 )
218 214 revopt = (b'r', b'rev', [], _(b'revisions to fix'), _(b'REV'))
219 215 wdiropt = (b'w', b'working-dir', False, _(b'fix the working directory'))
220 216 wholeopt = (b'', b'whole', False, _(b'always fix every line of a file'))
221 217 usage = _(b'[OPTION]... [FILE]...')
222 218
223 219
224 220 @command(
225 221 b'fix',
226 222 [allopt, baseopt, revopt, wdiropt, wholeopt],
227 223 usage,
228 224 helpcategory=command.CATEGORY_FILE_CONTENTS,
229 225 )
230 226 def fix(ui, repo, *pats, **opts):
231 227 """rewrite file content in changesets or working directory
232 228
233 229 Runs any configured tools to fix the content of files. Only affects files
234 230 with changes, unless file arguments are provided. Only affects changed lines
235 231 of files, unless the --whole flag is used. Some tools may always affect the
236 232 whole file regardless of --whole.
237 233
238 234 If revisions are specified with --rev, those revisions will be checked, and
239 235 they may be replaced with new revisions that have fixed file content. It is
240 236 desirable to specify all descendants of each specified revision, so that the
241 237 fixes propagate to the descendants. If all descendants are fixed at the same
242 238 time, no merging, rebasing, or evolution will be required.
243 239
244 240 If --working-dir is used, files with uncommitted changes in the working copy
245 241 will be fixed. If the checked-out revision is also fixed, the working
246 242 directory will update to the replacement revision.
247 243
248 244 When determining what lines of each file to fix at each revision, the whole
249 245 set of revisions being fixed is considered, so that fixes to earlier
250 246 revisions are not forgotten in later ones. The --base flag can be used to
251 247 override this default behavior, though it is not usually desirable to do so.
252 248 """
253 249 opts = pycompat.byteskwargs(opts)
254 250 if opts[b'all']:
255 251 if opts[b'rev']:
256 252 raise error.Abort(_(b'cannot specify both "--rev" and "--all"'))
257 253 opts[b'rev'] = [b'not public() and not obsolete()']
258 254 opts[b'working_dir'] = True
259 255 with repo.wlock(), repo.lock(), repo.transaction(b'fix'):
260 256 revstofix = getrevstofix(ui, repo, opts)
261 257 basectxs = getbasectxs(repo, opts, revstofix)
262 258 workqueue, numitems = getworkqueue(
263 259 ui, repo, pats, opts, revstofix, basectxs
264 260 )
265 261 fixers = getfixers(ui)
266 262
267 263 # There are no data dependencies between the workers fixing each file
268 264 # revision, so we can use all available parallelism.
269 265 def getfixes(items):
270 266 for rev, path in items:
271 267 ctx = repo[rev]
272 268 olddata = ctx[path].data()
273 269 metadata, newdata = fixfile(
274 270 ui, repo, opts, fixers, ctx, path, basectxs[rev]
275 271 )
276 272 # Don't waste memory/time passing unchanged content back, but
277 273 # produce one result per item either way.
278 274 yield (
279 275 rev,
280 276 path,
281 277 metadata,
282 278 newdata if newdata != olddata else None,
283 279 )
284 280
285 281 results = worker.worker(
286 282 ui, 1.0, getfixes, tuple(), workqueue, threadsafe=False
287 283 )
288 284
289 285 # We have to hold on to the data for each successor revision in memory
290 286 # until all its parents are committed. We ensure this by committing and
291 287 # freeing memory for the revisions in some topological order. This
292 288 # leaves a little bit of memory efficiency on the table, but also makes
293 289 # the tests deterministic. It might also be considered a feature since
294 290 # it makes the results more easily reproducible.
295 291 filedata = collections.defaultdict(dict)
296 292 aggregatemetadata = collections.defaultdict(list)
297 293 replacements = {}
298 294 wdirwritten = False
299 295 commitorder = sorted(revstofix, reverse=True)
300 296 with ui.makeprogress(
301 297 topic=_(b'fixing'), unit=_(b'files'), total=sum(numitems.values())
302 298 ) as progress:
303 299 for rev, path, filerevmetadata, newdata in results:
304 300 progress.increment(item=path)
305 301 for fixername, fixermetadata in filerevmetadata.items():
306 302 aggregatemetadata[fixername].append(fixermetadata)
307 303 if newdata is not None:
308 304 filedata[rev][path] = newdata
309 305 hookargs = {
310 306 b'rev': rev,
311 307 b'path': path,
312 308 b'metadata': filerevmetadata,
313 309 }
314 310 repo.hook(
315 311 b'postfixfile',
316 312 throw=False,
317 313 **pycompat.strkwargs(hookargs)
318 314 )
319 315 numitems[rev] -= 1
320 316 # Apply the fixes for this and any other revisions that are
321 317 # ready and sitting at the front of the queue. Using a loop here
322 318 # prevents the queue from being blocked by the first revision to
323 319 # be ready out of order.
324 320 while commitorder and not numitems[commitorder[-1]]:
325 321 rev = commitorder.pop()
326 322 ctx = repo[rev]
327 323 if rev == wdirrev:
328 324 writeworkingdir(repo, ctx, filedata[rev], replacements)
329 325 wdirwritten = bool(filedata[rev])
330 326 else:
331 327 replacerev(ui, repo, ctx, filedata[rev], replacements)
332 328 del filedata[rev]
333 329
334 330 cleanup(repo, replacements, wdirwritten)
335 331 hookargs = {
336 332 b'replacements': replacements,
337 333 b'wdirwritten': wdirwritten,
338 334 b'metadata': aggregatemetadata,
339 335 }
340 336 repo.hook(b'postfix', throw=True, **pycompat.strkwargs(hookargs))
341 337
342 338
343 339 def cleanup(repo, replacements, wdirwritten):
344 340 """Calls scmutil.cleanupnodes() with the given replacements.
345 341
346 342 "replacements" is a dict from nodeid to nodeid, with one key and one value
347 343 for every revision that was affected by fixing. This is slightly different
348 344 from cleanupnodes().
349 345
350 346 "wdirwritten" is a bool which tells whether the working copy was affected by
351 347 fixing, since it has no entry in "replacements".
352 348
353 349 Useful as a hook point for extending "hg fix" with output summarizing the
354 350 effects of the command, though we choose not to output anything here.
355 351 """
356 352 replacements = {
357 353 prec: [succ] for prec, succ in pycompat.iteritems(replacements)
358 354 }
359 355 scmutil.cleanupnodes(repo, replacements, b'fix', fixphase=True)
360 356
361 357
362 358 def getworkqueue(ui, repo, pats, opts, revstofix, basectxs):
363 359 """"Constructs the list of files to be fixed at specific revisions
364 360
365 361 It is up to the caller how to consume the work items, and the only
366 362 dependence between them is that replacement revisions must be committed in
367 363 topological order. Each work item represents a file in the working copy or
368 364 in some revision that should be fixed and written back to the working copy
369 365 or into a replacement revision.
370 366
371 367 Work items for the same revision are grouped together, so that a worker
372 368 pool starting with the first N items in parallel is likely to finish the
373 369 first revision's work before other revisions. This can allow us to write
374 370 the result to disk and reduce memory footprint. At time of writing, the
375 371 partition strategy in worker.py seems favorable to this. We also sort the
376 372 items by ascending revision number to match the order in which we commit
377 373 the fixes later.
378 374 """
379 375 workqueue = []
380 376 numitems = collections.defaultdict(int)
381 377 maxfilesize = ui.configbytes(b'fix', b'maxfilesize')
382 378 for rev in sorted(revstofix):
383 379 fixctx = repo[rev]
384 380 match = scmutil.match(fixctx, pats, opts)
385 381 for path in sorted(
386 382 pathstofix(ui, repo, pats, opts, match, basectxs[rev], fixctx)
387 383 ):
388 384 fctx = fixctx[path]
389 385 if fctx.islink():
390 386 continue
391 387 if fctx.size() > maxfilesize:
392 388 ui.warn(
393 389 _(b'ignoring file larger than %s: %s\n')
394 390 % (util.bytecount(maxfilesize), path)
395 391 )
396 392 continue
397 393 workqueue.append((rev, path))
398 394 numitems[rev] += 1
399 395 return workqueue, numitems
400 396
401 397
402 398 def getrevstofix(ui, repo, opts):
403 399 """Returns the set of revision numbers that should be fixed"""
404 400 revs = set(scmutil.revrange(repo, opts[b'rev']))
405 401 for rev in revs:
406 402 checkfixablectx(ui, repo, repo[rev])
407 403 if revs:
408 404 cmdutil.checkunfinished(repo)
409 405 checknodescendants(repo, revs)
410 406 if opts.get(b'working_dir'):
411 407 revs.add(wdirrev)
412 408 if list(merge.mergestate.read(repo).unresolved()):
413 409 raise error.Abort(b'unresolved conflicts', hint=b"use 'hg resolve'")
414 410 if not revs:
415 411 raise error.Abort(
416 412 b'no changesets specified', hint=b'use --rev or --working-dir'
417 413 )
418 414 return revs
419 415
420 416
421 417 def checknodescendants(repo, revs):
422 418 if not obsolete.isenabled(repo, obsolete.allowunstableopt) and repo.revs(
423 419 b'(%ld::) - (%ld)', revs, revs
424 420 ):
425 421 raise error.Abort(
426 422 _(b'can only fix a changeset together with all its descendants')
427 423 )
428 424
429 425
430 426 def checkfixablectx(ui, repo, ctx):
431 427 """Aborts if the revision shouldn't be replaced with a fixed one."""
432 428 if not ctx.mutable():
433 429 raise error.Abort(
434 430 b'can\'t fix immutable changeset %s'
435 431 % (scmutil.formatchangeid(ctx),)
436 432 )
437 433 if ctx.obsolete():
438 434 # It would be better to actually check if the revision has a successor.
439 435 allowdivergence = ui.configbool(
440 436 b'experimental', b'evolution.allowdivergence'
441 437 )
442 438 if not allowdivergence:
443 439 raise error.Abort(
444 440 b'fixing obsolete revision could cause divergence'
445 441 )
446 442
447 443
448 444 def pathstofix(ui, repo, pats, opts, match, basectxs, fixctx):
449 445 """Returns the set of files that should be fixed in a context
450 446
451 447 The result depends on the base contexts; we include any file that has
452 448 changed relative to any of the base contexts. Base contexts should be
453 449 ancestors of the context being fixed.
454 450 """
455 451 files = set()
456 452 for basectx in basectxs:
457 453 stat = basectx.status(
458 454 fixctx, match=match, listclean=bool(pats), listunknown=bool(pats)
459 455 )
460 456 files.update(
461 457 set(
462 458 itertools.chain(
463 459 stat.added, stat.modified, stat.clean, stat.unknown
464 460 )
465 461 )
466 462 )
467 463 return files
468 464
469 465
470 466 def lineranges(opts, path, basectxs, fixctx, content2):
471 467 """Returns the set of line ranges that should be fixed in a file
472 468
473 469 Of the form [(10, 20), (30, 40)].
474 470
475 471 This depends on the given base contexts; we must consider lines that have
476 472 changed versus any of the base contexts, and whether the file has been
477 473 renamed versus any of them.
478 474
479 475 Another way to understand this is that we exclude line ranges that are
480 476 common to the file in all base contexts.
481 477 """
482 478 if opts.get(b'whole'):
483 479 # Return a range containing all lines. Rely on the diff implementation's
484 480 # idea of how many lines are in the file, instead of reimplementing it.
485 481 return difflineranges(b'', content2)
486 482
487 483 rangeslist = []
488 484 for basectx in basectxs:
489 485 basepath = copies.pathcopies(basectx, fixctx).get(path, path)
490 486 if basepath in basectx:
491 487 content1 = basectx[basepath].data()
492 488 else:
493 489 content1 = b''
494 490 rangeslist.extend(difflineranges(content1, content2))
495 491 return unionranges(rangeslist)
496 492
497 493
498 494 def unionranges(rangeslist):
499 495 """Return the union of some closed intervals
500 496
501 497 >>> unionranges([])
502 498 []
503 499 >>> unionranges([(1, 100)])
504 500 [(1, 100)]
505 501 >>> unionranges([(1, 100), (1, 100)])
506 502 [(1, 100)]
507 503 >>> unionranges([(1, 100), (2, 100)])
508 504 [(1, 100)]
509 505 >>> unionranges([(1, 99), (1, 100)])
510 506 [(1, 100)]
511 507 >>> unionranges([(1, 100), (40, 60)])
512 508 [(1, 100)]
513 509 >>> unionranges([(1, 49), (50, 100)])
514 510 [(1, 100)]
515 511 >>> unionranges([(1, 48), (50, 100)])
516 512 [(1, 48), (50, 100)]
517 513 >>> unionranges([(1, 2), (3, 4), (5, 6)])
518 514 [(1, 6)]
519 515 """
520 516 rangeslist = sorted(set(rangeslist))
521 517 unioned = []
522 518 if rangeslist:
523 519 unioned, rangeslist = [rangeslist[0]], rangeslist[1:]
524 520 for a, b in rangeslist:
525 521 c, d = unioned[-1]
526 522 if a > d + 1:
527 523 unioned.append((a, b))
528 524 else:
529 525 unioned[-1] = (c, max(b, d))
530 526 return unioned
531 527
532 528
533 529 def difflineranges(content1, content2):
534 530 """Return list of line number ranges in content2 that differ from content1.
535 531
536 532 Line numbers are 1-based. The numbers are the first and last line contained
537 533 in the range. Single-line ranges have the same line number for the first and
538 534 last line. Excludes any empty ranges that result from lines that are only
539 535 present in content1. Relies on mdiff's idea of where the line endings are in
540 536 the string.
541 537
542 538 >>> from mercurial import pycompat
543 539 >>> lines = lambda s: b'\\n'.join([c for c in pycompat.iterbytestr(s)])
544 540 >>> difflineranges2 = lambda a, b: difflineranges(lines(a), lines(b))
545 541 >>> difflineranges2(b'', b'')
546 542 []
547 543 >>> difflineranges2(b'a', b'')
548 544 []
549 545 >>> difflineranges2(b'', b'A')
550 546 [(1, 1)]
551 547 >>> difflineranges2(b'a', b'a')
552 548 []
553 549 >>> difflineranges2(b'a', b'A')
554 550 [(1, 1)]
555 551 >>> difflineranges2(b'ab', b'')
556 552 []
557 553 >>> difflineranges2(b'', b'AB')
558 554 [(1, 2)]
559 555 >>> difflineranges2(b'abc', b'ac')
560 556 []
561 557 >>> difflineranges2(b'ab', b'aCb')
562 558 [(2, 2)]
563 559 >>> difflineranges2(b'abc', b'aBc')
564 560 [(2, 2)]
565 561 >>> difflineranges2(b'ab', b'AB')
566 562 [(1, 2)]
567 563 >>> difflineranges2(b'abcde', b'aBcDe')
568 564 [(2, 2), (4, 4)]
569 565 >>> difflineranges2(b'abcde', b'aBCDe')
570 566 [(2, 4)]
571 567 """
572 568 ranges = []
573 569 for lines, kind in mdiff.allblocks(content1, content2):
574 570 firstline, lastline = lines[2:4]
575 571 if kind == b'!' and firstline != lastline:
576 572 ranges.append((firstline + 1, lastline))
577 573 return ranges
578 574
579 575
580 576 def getbasectxs(repo, opts, revstofix):
581 577 """Returns a map of the base contexts for each revision
582 578
583 579 The base contexts determine which lines are considered modified when we
584 580 attempt to fix just the modified lines in a file. It also determines which
585 581 files we attempt to fix, so it is important to compute this even when
586 582 --whole is used.
587 583 """
588 584 # The --base flag overrides the usual logic, and we give every revision
589 585 # exactly the set of baserevs that the user specified.
590 586 if opts.get(b'base'):
591 587 baserevs = set(scmutil.revrange(repo, opts.get(b'base')))
592 588 if not baserevs:
593 589 baserevs = {nullrev}
594 590 basectxs = {repo[rev] for rev in baserevs}
595 591 return {rev: basectxs for rev in revstofix}
596 592
597 593 # Proceed in topological order so that we can easily determine each
598 594 # revision's baserevs by looking at its parents and their baserevs.
599 595 basectxs = collections.defaultdict(set)
600 596 for rev in sorted(revstofix):
601 597 ctx = repo[rev]
602 598 for pctx in ctx.parents():
603 599 if pctx.rev() in basectxs:
604 600 basectxs[rev].update(basectxs[pctx.rev()])
605 601 else:
606 602 basectxs[rev].add(pctx)
607 603 return basectxs
608 604
609 605
610 606 def fixfile(ui, repo, opts, fixers, fixctx, path, basectxs):
611 607 """Run any configured fixers that should affect the file in this context
612 608
613 609 Returns the file content that results from applying the fixers in some order
614 610 starting with the file's content in the fixctx. Fixers that support line
615 611 ranges will affect lines that have changed relative to any of the basectxs
616 612 (i.e. they will only avoid lines that are common to all basectxs).
617 613
618 614 A fixer tool's stdout will become the file's new content if and only if it
619 615 exits with code zero. The fixer tool's working directory is the repository's
620 616 root.
621 617 """
622 618 metadata = {}
623 619 newdata = fixctx[path].data()
624 620 for fixername, fixer in pycompat.iteritems(fixers):
625 621 if fixer.affects(opts, fixctx, path):
626 622 ranges = lineranges(opts, path, basectxs, fixctx, newdata)
627 623 command = fixer.command(ui, path, ranges)
628 624 if command is None:
629 625 continue
630 626 ui.debug(b'subprocess: %s\n' % (command,))
631 627 proc = subprocess.Popen(
632 628 procutil.tonativestr(command),
633 629 shell=True,
634 630 cwd=procutil.tonativestr(repo.root),
635 631 stdin=subprocess.PIPE,
636 632 stdout=subprocess.PIPE,
637 633 stderr=subprocess.PIPE,
638 634 )
639 635 stdout, stderr = proc.communicate(newdata)
640 636 if stderr:
641 637 showstderr(ui, fixctx.rev(), fixername, stderr)
642 638 newerdata = stdout
643 639 if fixer.shouldoutputmetadata():
644 640 try:
645 641 metadatajson, newerdata = stdout.split(b'\0', 1)
646 642 metadata[fixername] = json.loads(metadatajson)
647 643 except ValueError:
648 644 ui.warn(
649 645 _(b'ignored invalid output from fixer tool: %s\n')
650 646 % (fixername,)
651 647 )
652 648 continue
653 649 else:
654 650 metadata[fixername] = None
655 651 if proc.returncode == 0:
656 652 newdata = newerdata
657 653 else:
658 654 if not stderr:
659 655 message = _(b'exited with status %d\n') % (proc.returncode,)
660 656 showstderr(ui, fixctx.rev(), fixername, message)
661 657 checktoolfailureaction(
662 658 ui,
663 659 _(b'no fixes will be applied'),
664 660 hint=_(
665 661 b'use --config fix.failure=continue to apply any '
666 662 b'successful fixes anyway'
667 663 ),
668 664 )
669 665 return metadata, newdata
670 666
671 667
672 668 def showstderr(ui, rev, fixername, stderr):
673 669 """Writes the lines of the stderr string as warnings on the ui
674 670
675 671 Uses the revision number and fixername to give more context to each line of
676 672 the error message. Doesn't include file names, since those take up a lot of
677 673 space and would tend to be included in the error message if they were
678 674 relevant.
679 675 """
680 676 for line in re.split(b'[\r\n]+', stderr):
681 677 if line:
682 678 ui.warn(b'[')
683 679 if rev is None:
684 680 ui.warn(_(b'wdir'), label=b'evolve.rev')
685 681 else:
686 682 ui.warn((str(rev)), label=b'evolve.rev')
687 683 ui.warn(b'] %s: %s\n' % (fixername, line))
688 684
689 685
690 686 def writeworkingdir(repo, ctx, filedata, replacements):
691 687 """Write new content to the working copy and check out the new p1 if any
692 688
693 689 We check out a new revision if and only if we fixed something in both the
694 690 working directory and its parent revision. This avoids the need for a full
695 691 update/merge, and means that the working directory simply isn't affected
696 692 unless the --working-dir flag is given.
697 693
698 694 Directly updates the dirstate for the affected files.
699 695 """
700 696 for path, data in pycompat.iteritems(filedata):
701 697 fctx = ctx[path]
702 698 fctx.write(data, fctx.flags())
703 699 if repo.dirstate[path] == b'n':
704 700 repo.dirstate.normallookup(path)
705 701
706 702 oldparentnodes = repo.dirstate.parents()
707 703 newparentnodes = [replacements.get(n, n) for n in oldparentnodes]
708 704 if newparentnodes != oldparentnodes:
709 705 repo.setparents(*newparentnodes)
710 706
711 707
712 708 def replacerev(ui, repo, ctx, filedata, replacements):
713 709 """Commit a new revision like the given one, but with file content changes
714 710
715 711 "ctx" is the original revision to be replaced by a modified one.
716 712
717 713 "filedata" is a dict that maps paths to their new file content. All other
718 714 paths will be recreated from the original revision without changes.
719 715 "filedata" may contain paths that didn't exist in the original revision;
720 716 they will be added.
721 717
722 718 "replacements" is a dict that maps a single node to a single node, and it is
723 719 updated to indicate the original revision is replaced by the newly created
724 720 one. No entry is added if the replacement's node already exists.
725 721
726 722 The new revision has the same parents as the old one, unless those parents
727 723 have already been replaced, in which case those replacements are the parents
728 724 of this new revision. Thus, if revisions are replaced in topological order,
729 725 there is no need to rebase them into the original topology later.
730 726 """
731 727
732 728 p1rev, p2rev = repo.changelog.parentrevs(ctx.rev())
733 729 p1ctx, p2ctx = repo[p1rev], repo[p2rev]
734 730 newp1node = replacements.get(p1ctx.node(), p1ctx.node())
735 731 newp2node = replacements.get(p2ctx.node(), p2ctx.node())
736 732
737 733 # We don't want to create a revision that has no changes from the original,
738 734 # but we should if the original revision's parent has been replaced.
739 735 # Otherwise, we would produce an orphan that needs no actual human
740 736 # intervention to evolve. We can't rely on commit() to avoid creating the
741 737 # un-needed revision because the extra field added below produces a new hash
742 738 # regardless of file content changes.
743 739 if (
744 740 not filedata
745 741 and p1ctx.node() not in replacements
746 742 and p2ctx.node() not in replacements
747 743 ):
748 744 return
749 745
750 746 def filectxfn(repo, memctx, path):
751 747 if path not in ctx:
752 748 return None
753 749 fctx = ctx[path]
754 750 copysource = fctx.copysource()
755 751 return context.memfilectx(
756 752 repo,
757 753 memctx,
758 754 path=fctx.path(),
759 755 data=filedata.get(path, fctx.data()),
760 756 islink=fctx.islink(),
761 757 isexec=fctx.isexec(),
762 758 copysource=copysource,
763 759 )
764 760
765 761 extra = ctx.extra().copy()
766 762 extra[b'fix_source'] = ctx.hex()
767 763
768 764 memctx = context.memctx(
769 765 repo,
770 766 parents=(newp1node, newp2node),
771 767 text=ctx.description(),
772 768 files=set(ctx.files()) | set(filedata.keys()),
773 769 filectxfn=filectxfn,
774 770 user=ctx.user(),
775 771 date=ctx.date(),
776 772 extra=extra,
777 773 branch=ctx.branch(),
778 774 editor=None,
779 775 )
780 776 sucnode = memctx.commit()
781 777 prenode = ctx.node()
782 778 if prenode == sucnode:
783 779 ui.debug(b'node %s already existed\n' % (ctx.hex()))
784 780 else:
785 781 replacements[ctx.node()] = sucnode
786 782
787 783
788 784 def getfixers(ui):
789 785 """Returns a map of configured fixer tools indexed by their names
790 786
791 787 Each value is a Fixer object with methods that implement the behavior of the
792 788 fixer's config suboptions. Does not validate the config values.
793 789 """
794 790 fixers = {}
795 791 for name in fixernames(ui):
796 fixers[name] = Fixer()
797 for key in FIXER_ATTRS:
798 setattr(
799 fixers[name],
800 pycompat.sysstr(b'_' + key),
801 ui.config(b'fix', name + b':' + key),
802 )
803 fixers[name]._priority = int(fixers[name]._priority)
804 fixers[name]._metadata = stringutil.parsebool(fixers[name]._metadata)
805 fixers[name]._skipclean = stringutil.parsebool(fixers[name]._skipclean)
806 fixers[name]._enabled = stringutil.parsebool(fixers[name]._enabled)
792 enabled = ui.configbool(b'fix', name + b':enabled')
793 command = ui.config(b'fix', name + b':command')
794 pattern = ui.config(b'fix', name + b':pattern')
795 linerange = ui.config(b'fix', name + b':linerange')
796 priority = ui.configint(b'fix', name + b':priority')
797 metadata = ui.configbool(b'fix', name + b':metadata')
798 skipclean = ui.configbool(b'fix', name + b':skipclean')
807 799 # Don't use a fixer if it has no pattern configured. It would be
808 800 # dangerous to let it affect all files. It would be pointless to let it
809 801 # affect no files. There is no reasonable subset of files to use as the
810 802 # default.
811 if fixers[name]._pattern is None:
803 if pattern is None:
812 804 ui.warn(
813 805 _(b'fixer tool has no pattern configuration: %s\n') % (name,)
814 806 )
815 del fixers[name]
816 elif not fixers[name]._enabled:
807 elif not enabled:
817 808 ui.debug(b'ignoring disabled fixer tool: %s\n' % (name,))
818 del fixers[name]
809 else:
810 fixers[name] = Fixer(
811 command, pattern, linerange, priority, metadata, skipclean
812 )
819 813 return collections.OrderedDict(
820 814 sorted(fixers.items(), key=lambda item: item[1]._priority, reverse=True)
821 815 )
822 816
823 817
824 818 def fixernames(ui):
825 819 """Returns the names of [fix] config options that have suboptions"""
826 820 names = set()
827 821 for k, v in ui.configitems(b'fix'):
828 822 if b':' in k:
829 823 names.add(k.split(b':', 1)[0])
830 824 return names
831 825
832 826
833 827 class Fixer(object):
834 828 """Wraps the raw config values for a fixer with methods"""
835 829
830 def __init__(
831 self, command, pattern, linerange, priority, metadata, skipclean
832 ):
833 self._command = command
834 self._pattern = pattern
835 self._linerange = linerange
836 self._priority = priority
837 self._metadata = metadata
838 self._skipclean = skipclean
839
836 840 def affects(self, opts, fixctx, path):
837 841 """Should this fixer run on the file at the given path and context?"""
838 842 return self._pattern is not None and scmutil.match(
839 843 fixctx, [self._pattern], opts
840 844 )(path)
841 845
842 846 def shouldoutputmetadata(self):
843 847 """Should the stdout of this fixer start with JSON and a null byte?"""
844 848 return self._metadata
845 849
846 850 def command(self, ui, path, ranges):
847 851 """A shell command to use to invoke this fixer on the given file/lines
848 852
849 853 May return None if there is no appropriate command to run for the given
850 854 parameters.
851 855 """
852 856 expand = cmdutil.rendercommandtemplate
853 857 parts = [
854 858 expand(
855 859 ui,
856 860 self._command,
857 861 {b'rootpath': path, b'basename': os.path.basename(path)},
858 862 )
859 863 ]
860 864 if self._linerange:
861 865 if self._skipclean and not ranges:
862 866 # No line ranges to fix, so don't run the fixer.
863 867 return None
864 868 for first, last in ranges:
865 869 parts.append(
866 870 expand(
867 871 ui, self._linerange, {b'first': first, b'last': last}
868 872 )
869 873 )
870 874 return b' '.join(parts)
General Comments 0
You need to be logged in to leave comments. Login now