fix.py
882 lines
| 33.1 KiB
| text/x-python
|
PythonLexer
/ hgext / fix.py
|
r37200 | # fix - rewrite file content in changesets and working copy | ||
# | ||||
# Copyright 2018 Google LLC. | ||||
# | ||||
# This software may be used and distributed according to the terms of the | ||||
# GNU General Public License version 2 or any later version. | ||||
"""rewrite file content in changesets or working copy (EXPERIMENTAL) | ||||
Provides a command that runs configured tools on the contents of modified files, | ||||
writing back any fixes to the working copy or replacing changesets. | ||||
Here is an example configuration that causes :hg:`fix` to apply automatic | ||||
formatting fixes to modified lines in C++ code:: | ||||
[fix] | ||||
clang-format:command=clang-format --assume-filename={rootpath} | ||||
clang-format:linerange=--lines={first}:{last} | ||||
|
r40569 | clang-format:pattern=set:**.cpp or **.hpp | ||
|
r37200 | |||
The :command suboption forms the first part of the shell command that will be | ||||
used to fix a file. The content of the file is passed on standard input, and the | ||||
|
r40568 | fixed file content is expected on standard output. Any output on standard error | ||
will be displayed as a warning. If the exit status is not zero, the file will | ||||
not be affected. A placeholder warning is displayed if there is a non-zero exit | ||||
status but no standard error output. Some values may be substituted into the | ||||
command:: | ||||
|
r37200 | |||
{rootpath} The path of the file being fixed, relative to the repo root | ||||
{basename} The name of the file being fixed, without the directory path | ||||
If the :linerange suboption is set, the tool will only be run if there are | ||||
changed lines in a file. The value of this suboption is appended to the shell | ||||
command once for every range of changed lines in the file. Some values may be | ||||
substituted into the command:: | ||||
{first} The 1-based line number of the first line in the modified range | ||||
{last} The 1-based line number of the last line in the modified range | ||||
|
r43001 | Deleted sections of a file will be ignored by :linerange, because there is no | ||
corresponding line range in the version being fixed. | ||||
By default, tools that set :linerange will only be executed if there is at least | ||||
one changed line range. This is meant to prevent accidents like running a code | ||||
formatter in such a way that it unexpectedly reformats the whole file. If such a | ||||
tool needs to operate on unchanged files, it should set the :skipclean suboption | ||||
to false. | ||||
|
r40569 | The :pattern suboption determines which files will be passed through each | ||
|
r43501 | configured tool. See :hg:`help patterns` for possible values. However, all | ||
patterns are relative to the repo root, even if that text says they are relative | ||||
to the current working directory. If there are file arguments to :hg:`fix`, the | ||||
intersection of these patterns is used. | ||||
|
r37200 | |||
There is also a configurable limit for the maximum size of file that will be | ||||
processed by :hg:`fix`:: | ||||
[fix] | ||||
|
r40568 | maxfilesize = 2MB | ||
Normally, execution of configured tools will continue after a failure (indicated | ||||
by a non-zero exit status). It can also be configured to abort after the first | ||||
such failure, so that no files will be affected if any tool fails. This abort | ||||
will also cause :hg:`fix` to exit with a non-zero status:: | ||||
[fix] | ||||
failure = abort | ||||
|
r37200 | |||
|
r40599 | When multiple tools are configured to affect a file, they execute in an order | ||
defined by the :priority suboption. The priority suboption has a default value | ||||
of zero for each tool. Tools are executed in order of descending priority. The | ||||
execution order of tools with equal priority is unspecified. For example, you | ||||
could use the 'sort' and 'head' utilities to keep only the 10 smallest numbers | ||||
in a text file by ensuring that 'sort' runs before 'head':: | ||||
[fix] | ||||
|
r41162 | sort:command = sort -n | ||
head:command = head -n 10 | ||||
|
r40599 | sort:pattern = numbers.txt | ||
head:pattern = numbers.txt | ||||
sort:priority = 2 | ||||
head:priority = 1 | ||||
To account for changes made by each tool, the line numbers used for incremental | ||||
formatting are recomputed before executing the next tool. So, each tool may see | ||||
different values for the arguments added by the :linerange suboption. | ||||
|
r42372 | |||
Each fixer tool is allowed to return some metadata in addition to the fixed file | ||||
content. The metadata must be placed before the file content on stdout, | ||||
separated from the file content by a zero byte. The metadata is parsed as a JSON | ||||
value (so, it should be UTF-8 encoded and contain no zero bytes). A fixer tool | ||||
is expected to produce this metadata encoding if and only if the :metadata | ||||
suboption is true:: | ||||
[fix] | ||||
tool:command = tool --prepend-json-metadata | ||||
tool:metadata = true | ||||
The metadata values are passed to hooks, which can be used to print summaries or | ||||
perform other post-fixing work. The supported hooks are:: | ||||
"postfixfile" | ||||
Run once for each file in each revision where any fixer tools made changes | ||||
to the file content. Provides "$HG_REV" and "$HG_PATH" to identify the file, | ||||
and "$HG_METADATA" with a map of fixer names to metadata values from fixer | ||||
tools that affected the file. Fixer tools that didn't affect the file have a | ||||
|
r44500 | value of None. Only fixer tools that executed are present in the metadata. | ||
|
r42372 | |||
"postfix" | ||||
Run once after all files and revisions have been handled. Provides | ||||
"$HG_REPLACEMENTS" with information about what revisions were created and | ||||
made obsolete. Provides a boolean "$HG_WDIRWRITTEN" to indicate whether any | ||||
files in the working copy were updated. Provides a list "$HG_METADATA" | ||||
mapping fixer tool names to lists of metadata values returned from | ||||
executions that modified a file. This aggregates the same metadata | ||||
previously passed to the "postfixfile" hook. | ||||
|
r42900 | |||
|
r44500 | Fixer tools are run in the repository's root directory. This allows them to read | ||
|
r42900 | configuration files from the working copy, or even write to the working copy. | ||
The working copy is not updated to match the revision being fixed. In fact, | ||||
several revisions may be fixed in parallel. Writes to the working copy are not | ||||
amended into the revision being fixed; fixer tools should always write fixed | ||||
file content back to stdout as documented above. | ||||
|
r37200 | """ | ||
from __future__ import absolute_import | ||||
import collections | ||||
import itertools | ||||
import os | ||||
import re | ||||
import subprocess | ||||
from mercurial.i18n import _ | ||||
from mercurial.node import nullrev | ||||
from mercurial.node import wdirrev | ||||
|
r43493 | from mercurial.utils import procutil | ||
|
r39851 | |||
|
r37200 | from mercurial import ( | ||
cmdutil, | ||||
context, | ||||
copies, | ||||
error, | ||||
|
r43501 | match as matchmod, | ||
|
r37200 | mdiff, | ||
merge, | ||||
obsolete, | ||||
|
r37636 | pycompat, | ||
|
r37200 | registrar, | ||
scmutil, | ||||
util, | ||||
|
r38554 | worker, | ||
|
r37200 | ) | ||
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for | ||||
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should | ||||
# be specifying the version(s) of Mercurial they are tested with, or | ||||
# leave the attribute unspecified. | ||||
|
r43347 | testedwith = b'ships-with-hg-core' | ||
|
r37200 | |||
cmdtable = {} | ||||
command = registrar.command(cmdtable) | ||||
configtable = {} | ||||
configitem = registrar.configitem(configtable) | ||||
|
r42372 | # Register the suboptions allowed for each configured fixer, and default values. | ||
|
r40599 | FIXER_ATTRS = { | ||
|
r43347 | b'command': None, | ||
b'linerange': None, | ||||
b'pattern': None, | ||||
b'priority': 0, | ||||
|
r43493 | b'metadata': False, | ||
b'skipclean': True, | ||||
b'enabled': True, | ||||
|
r40599 | } | ||
|
r37200 | |||
|
r40599 | for key, default in FIXER_ATTRS.items(): | ||
|
r43488 | configitem(b'fix', b'.*:%s$' % key, default=default, generic=True) | ||
|
r37200 | |||
# A good default size allows most source code files to be fixed, but avoids | ||||
# letting fixer tools choke on huge inputs, which could be surprising to the | ||||
# user. | ||||
|
r43347 | configitem(b'fix', b'maxfilesize', default=b'2MB') | ||
|
r37200 | |||
|
r40568 | # Allow fix commands to exit non-zero if an executed fixer tool exits non-zero. | ||
# This helps users do shell scripts that stop when a fixer tool signals a | ||||
# problem. | ||||
|
r43347 | configitem(b'fix', b'failure', default=b'continue') | ||
|
r40568 | |||
|
r43346 | |||
|
r40568 | def checktoolfailureaction(ui, message, hint=None): | ||
"""Abort with 'message' if fix.failure=abort""" | ||||
|
r43347 | action = ui.config(b'fix', b'failure') | ||
if action not in (b'continue', b'abort'): | ||||
|
r43346 | raise error.Abort( | ||
|
r43347 | _(b'unknown fix.failure action: %s') % (action,), | ||
hint=_(b'use "continue" or "abort"'), | ||||
|
r43346 | ) | ||
|
r43347 | if action == b'abort': | ||
|
r40568 | raise error.Abort(message, hint=hint) | ||
|
r43346 | |||
|
r43347 | allopt = (b'', b'all', False, _(b'fix all non-public non-obsolete revisions')) | ||
|
r43346 | baseopt = ( | ||
|
r43347 | b'', | ||
b'base', | ||||
|
r43346 | [], | ||
_( | ||||
|
r43347 | b'revisions to diff against (overrides automatic ' | ||
b'selection, and applies to every revision being ' | ||||
b'fixed)' | ||||
|
r43346 | ), | ||
|
r43347 | _(b'REV'), | ||
|
r43346 | ) | ||
|
r43347 | revopt = (b'r', b'rev', [], _(b'revisions to fix'), _(b'REV')) | ||
wdiropt = (b'w', b'working-dir', False, _(b'fix the working directory')) | ||||
wholeopt = (b'', b'whole', False, _(b'always fix every line of a file')) | ||||
usage = _(b'[OPTION]... [FILE]...') | ||||
|
r38984 | |||
|
r43346 | |||
@command( | ||||
|
r43347 | b'fix', | ||
|
r43346 | [allopt, baseopt, revopt, wdiropt, wholeopt], | ||
usage, | ||||
helpcategory=command.CATEGORY_FILE_CONTENTS, | ||||
) | ||||
|
r37200 | def fix(ui, repo, *pats, **opts): | ||
"""rewrite file content in changesets or working directory | ||||
Runs any configured tools to fix the content of files. Only affects files | ||||
with changes, unless file arguments are provided. Only affects changed lines | ||||
of files, unless the --whole flag is used. Some tools may always affect the | ||||
whole file regardless of --whole. | ||||
If revisions are specified with --rev, those revisions will be checked, and | ||||
they may be replaced with new revisions that have fixed file content. It is | ||||
desirable to specify all descendants of each specified revision, so that the | ||||
fixes propagate to the descendants. If all descendants are fixed at the same | ||||
time, no merging, rebasing, or evolution will be required. | ||||
If --working-dir is used, files with uncommitted changes in the working copy | ||||
will be fixed. If the checked-out revision is also fixed, the working | ||||
directory will update to the replacement revision. | ||||
When determining what lines of each file to fix at each revision, the whole | ||||
set of revisions being fixed is considered, so that fixes to earlier | ||||
revisions are not forgotten in later ones. The --base flag can be used to | ||||
override this default behavior, though it is not usually desirable to do so. | ||||
""" | ||||
|
r37636 | opts = pycompat.byteskwargs(opts) | ||
|
r43347 | if opts[b'all']: | ||
if opts[b'rev']: | ||||
raise error.Abort(_(b'cannot specify both "--rev" and "--all"')) | ||||
opts[b'rev'] = [b'not public() and not obsolete()'] | ||||
opts[b'working_dir'] = True | ||||
with repo.wlock(), repo.lock(), repo.transaction(b'fix'): | ||||
|
r37200 | revstofix = getrevstofix(ui, repo, opts) | ||
basectxs = getbasectxs(repo, opts, revstofix) | ||||
|
r43346 | workqueue, numitems = getworkqueue( | ||
ui, repo, pats, opts, revstofix, basectxs | ||||
) | ||||
|
r38554 | fixers = getfixers(ui) | ||
# There are no data dependencies between the workers fixing each file | ||||
# revision, so we can use all available parallelism. | ||||
def getfixes(items): | ||||
for rev, path in items: | ||||
ctx = repo[rev] | ||||
olddata = ctx[path].data() | ||||
|
r43346 | metadata, newdata = fixfile( | ||
ui, repo, opts, fixers, ctx, path, basectxs[rev] | ||||
) | ||||
|
r38554 | # Don't waste memory/time passing unchanged content back, but | ||
# produce one result per item either way. | ||||
|
r43346 | yield ( | ||
rev, | ||||
path, | ||||
metadata, | ||||
newdata if newdata != olddata else None, | ||||
) | ||||
results = worker.worker( | ||||
ui, 1.0, getfixes, tuple(), workqueue, threadsafe=False | ||||
) | ||||
|
r38554 | |||
# We have to hold on to the data for each successor revision in memory | ||||
# until all its parents are committed. We ensure this by committing and | ||||
# freeing memory for the revisions in some topological order. This | ||||
# leaves a little bit of memory efficiency on the table, but also makes | ||||
# the tests deterministic. It might also be considered a feature since | ||||
# it makes the results more easily reproducible. | ||||
|
r37200 | filedata = collections.defaultdict(dict) | ||
|
r42372 | aggregatemetadata = collections.defaultdict(list) | ||
|
r37200 | replacements = {} | ||
|
r38985 | wdirwritten = False | ||
|
r38554 | commitorder = sorted(revstofix, reverse=True) | ||
|
r43346 | with ui.makeprogress( | ||
|
r43347 | topic=_(b'fixing'), unit=_(b'files'), total=sum(numitems.values()) | ||
|
r43346 | ) as progress: | ||
|
r42372 | for rev, path, filerevmetadata, newdata in results: | ||
|
r38555 | progress.increment(item=path) | ||
|
r42372 | for fixername, fixermetadata in filerevmetadata.items(): | ||
aggregatemetadata[fixername].append(fixermetadata) | ||||
|
r38555 | if newdata is not None: | ||
filedata[rev][path] = newdata | ||||
|
r42372 | hookargs = { | ||
|
r43347 | b'rev': rev, | ||
b'path': path, | ||||
b'metadata': filerevmetadata, | ||||
|
r42372 | } | ||
|
r43346 | repo.hook( | ||
|
r43347 | b'postfixfile', | ||
|
r43346 | throw=False, | ||
**pycompat.strkwargs(hookargs) | ||||
) | ||||
|
r38555 | numitems[rev] -= 1 | ||
# Apply the fixes for this and any other revisions that are | ||||
# ready and sitting at the front of the queue. Using a loop here | ||||
# prevents the queue from being blocked by the first revision to | ||||
# be ready out of order. | ||||
while commitorder and not numitems[commitorder[-1]]: | ||||
rev = commitorder.pop() | ||||
ctx = repo[rev] | ||||
if rev == wdirrev: | ||||
writeworkingdir(repo, ctx, filedata[rev], replacements) | ||||
|
r38985 | wdirwritten = bool(filedata[rev]) | ||
|
r38555 | else: | ||
replacerev(ui, repo, ctx, filedata[rev], replacements) | ||||
del filedata[rev] | ||||
|
r37200 | |||
|
r38985 | cleanup(repo, replacements, wdirwritten) | ||
|
r42372 | hookargs = { | ||
|
r43347 | b'replacements': replacements, | ||
b'wdirwritten': wdirwritten, | ||||
b'metadata': aggregatemetadata, | ||||
|
r42372 | } | ||
|
r43347 | repo.hook(b'postfix', throw=True, **pycompat.strkwargs(hookargs)) | ||
|
r38847 | |||
|
r43346 | |||
|
r38847 | def cleanup(repo, replacements, wdirwritten): | ||
"""Calls scmutil.cleanupnodes() with the given replacements. | ||||
"replacements" is a dict from nodeid to nodeid, with one key and one value | ||||
for every revision that was affected by fixing. This is slightly different | ||||
from cleanupnodes(). | ||||
"wdirwritten" is a bool which tells whether the working copy was affected by | ||||
fixing, since it has no entry in "replacements". | ||||
Useful as a hook point for extending "hg fix" with output summarizing the | ||||
effects of the command, though we choose not to output anything here. | ||||
""" | ||||
|
r43375 | replacements = { | ||
prec: [succ] for prec, succ in pycompat.iteritems(replacements) | ||||
} | ||||
|
r43347 | scmutil.cleanupnodes(repo, replacements, b'fix', fixphase=True) | ||
|
r37200 | |||
|
r43346 | |||
|
r37200 | def getworkqueue(ui, repo, pats, opts, revstofix, basectxs): | ||
""""Constructs the list of files to be fixed at specific revisions | ||||
It is up to the caller how to consume the work items, and the only | ||||
dependence between them is that replacement revisions must be committed in | ||||
topological order. Each work item represents a file in the working copy or | ||||
in some revision that should be fixed and written back to the working copy | ||||
or into a replacement revision. | ||||
|
r38554 | |||
Work items for the same revision are grouped together, so that a worker | ||||
pool starting with the first N items in parallel is likely to finish the | ||||
first revision's work before other revisions. This can allow us to write | ||||
the result to disk and reduce memory footprint. At time of writing, the | ||||
partition strategy in worker.py seems favorable to this. We also sort the | ||||
items by ascending revision number to match the order in which we commit | ||||
the fixes later. | ||||
|
r37200 | """ | ||
workqueue = [] | ||||
numitems = collections.defaultdict(int) | ||||
|
r43347 | maxfilesize = ui.configbytes(b'fix', b'maxfilesize') | ||
|
r38554 | for rev in sorted(revstofix): | ||
|
r37200 | fixctx = repo[rev] | ||
match = scmutil.match(fixctx, pats, opts) | ||||
|
r43346 | for path in sorted( | ||
pathstofix(ui, repo, pats, opts, match, basectxs[rev], fixctx) | ||||
): | ||||
|
r37200 | fctx = fixctx[path] | ||
if fctx.islink(): | ||||
continue | ||||
if fctx.size() > maxfilesize: | ||||
|
r43346 | ui.warn( | ||
|
r43347 | _(b'ignoring file larger than %s: %s\n') | ||
|
r43346 | % (util.bytecount(maxfilesize), path) | ||
) | ||||
|
r37200 | continue | ||
workqueue.append((rev, path)) | ||||
numitems[rev] += 1 | ||||
return workqueue, numitems | ||||
|
r43346 | |||
|
r37200 | def getrevstofix(ui, repo, opts): | ||
"""Returns the set of revision numbers that should be fixed""" | ||||
|
r43347 | revs = set(scmutil.revrange(repo, opts[b'rev'])) | ||
|
r37200 | for rev in revs: | ||
checkfixablectx(ui, repo, repo[rev]) | ||||
if revs: | ||||
cmdutil.checkunfinished(repo) | ||||
checknodescendants(repo, revs) | ||||
|
r43347 | if opts.get(b'working_dir'): | ||
|
r37200 | revs.add(wdirrev) | ||
if list(merge.mergestate.read(repo).unresolved()): | ||||
|
r43347 | raise error.Abort(b'unresolved conflicts', hint=b"use 'hg resolve'") | ||
|
r37200 | if not revs: | ||
raise error.Abort( | ||||
|
r43347 | b'no changesets specified', hint=b'use --rev or --working-dir' | ||
|
r43346 | ) | ||
|
r37200 | return revs | ||
|
r43346 | |||
|
r37200 | def checknodescendants(repo, revs): | ||
|
r43346 | if not obsolete.isenabled(repo, obsolete.allowunstableopt) and repo.revs( | ||
|
r43347 | b'(%ld::) - (%ld)', revs, revs | ||
|
r43346 | ): | ||
raise error.Abort( | ||||
|
r43387 | _(b'can only fix a changeset together with all its descendants') | ||
|
r43346 | ) | ||
|
r37200 | |||
def checkfixablectx(ui, repo, ctx): | ||||
"""Aborts if the revision shouldn't be replaced with a fixed one.""" | ||||
if not ctx.mutable(): | ||||
|
r43346 | raise error.Abort( | ||
|
r43347 | b'can\'t fix immutable changeset %s' | ||
% (scmutil.formatchangeid(ctx),) | ||||
|
r43346 | ) | ||
|
r37200 | if ctx.obsolete(): | ||
# It would be better to actually check if the revision has a successor. | ||||
|
r43346 | allowdivergence = ui.configbool( | ||
|
r43347 | b'experimental', b'evolution.allowdivergence' | ||
|
r43346 | ) | ||
|
r37200 | if not allowdivergence: | ||
|
r43347 | raise error.Abort( | ||
b'fixing obsolete revision could cause divergence' | ||||
) | ||||
|
r37200 | |||
|
r43346 | |||
|
r37200 | def pathstofix(ui, repo, pats, opts, match, basectxs, fixctx): | ||
"""Returns the set of files that should be fixed in a context | ||||
The result depends on the base contexts; we include any file that has | ||||
changed relative to any of the base contexts. Base contexts should be | ||||
ancestors of the context being fixed. | ||||
""" | ||||
files = set() | ||||
for basectx in basectxs: | ||||
|
r43346 | stat = basectx.status( | ||
fixctx, match=match, listclean=bool(pats), listunknown=bool(pats) | ||||
) | ||||
|
r37200 | files.update( | ||
|
r43346 | set( | ||
itertools.chain( | ||||
stat.added, stat.modified, stat.clean, stat.unknown | ||||
) | ||||
) | ||||
) | ||||
|
r37200 | return files | ||
|
r43346 | |||
|
r37200 | def lineranges(opts, path, basectxs, fixctx, content2): | ||
"""Returns the set of line ranges that should be fixed in a file | ||||
Of the form [(10, 20), (30, 40)]. | ||||
This depends on the given base contexts; we must consider lines that have | ||||
changed versus any of the base contexts, and whether the file has been | ||||
renamed versus any of them. | ||||
Another way to understand this is that we exclude line ranges that are | ||||
common to the file in all base contexts. | ||||
""" | ||||
|
r43347 | if opts.get(b'whole'): | ||
|
r37200 | # Return a range containing all lines. Rely on the diff implementation's | ||
# idea of how many lines are in the file, instead of reimplementing it. | ||||
|
r43347 | return difflineranges(b'', content2) | ||
|
r37200 | |||
rangeslist = [] | ||||
for basectx in basectxs: | ||||
basepath = copies.pathcopies(basectx, fixctx).get(path, path) | ||||
if basepath in basectx: | ||||
content1 = basectx[basepath].data() | ||||
else: | ||||
|
r43347 | content1 = b'' | ||
|
r37200 | rangeslist.extend(difflineranges(content1, content2)) | ||
return unionranges(rangeslist) | ||||
|
r43346 | |||
|
r37200 | def unionranges(rangeslist): | ||
"""Return the union of some closed intervals | ||||
>>> unionranges([]) | ||||
[] | ||||
>>> unionranges([(1, 100)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 100), (1, 100)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 100), (2, 100)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 99), (1, 100)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 100), (40, 60)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 49), (50, 100)]) | ||||
[(1, 100)] | ||||
>>> unionranges([(1, 48), (50, 100)]) | ||||
[(1, 48), (50, 100)] | ||||
>>> unionranges([(1, 2), (3, 4), (5, 6)]) | ||||
[(1, 6)] | ||||
""" | ||||
rangeslist = sorted(set(rangeslist)) | ||||
unioned = [] | ||||
if rangeslist: | ||||
unioned, rangeslist = [rangeslist[0]], rangeslist[1:] | ||||
for a, b in rangeslist: | ||||
c, d = unioned[-1] | ||||
if a > d + 1: | ||||
unioned.append((a, b)) | ||||
else: | ||||
unioned[-1] = (c, max(b, d)) | ||||
return unioned | ||||
|
r43346 | |||
|
r37200 | def difflineranges(content1, content2): | ||
"""Return list of line number ranges in content2 that differ from content1. | ||||
Line numbers are 1-based. The numbers are the first and last line contained | ||||
in the range. Single-line ranges have the same line number for the first and | ||||
last line. Excludes any empty ranges that result from lines that are only | ||||
present in content1. Relies on mdiff's idea of where the line endings are in | ||||
the string. | ||||
|
r37230 | >>> from mercurial import pycompat | ||
>>> lines = lambda s: b'\\n'.join([c for c in pycompat.iterbytestr(s)]) | ||||
|
r37200 | >>> difflineranges2 = lambda a, b: difflineranges(lines(a), lines(b)) | ||
|
r37230 | >>> difflineranges2(b'', b'') | ||
|
r37200 | [] | ||
|
r37230 | >>> difflineranges2(b'a', b'') | ||
|
r37200 | [] | ||
|
r37230 | >>> difflineranges2(b'', b'A') | ||
|
r37200 | [(1, 1)] | ||
|
r37230 | >>> difflineranges2(b'a', b'a') | ||
|
r37200 | [] | ||
|
r37230 | >>> difflineranges2(b'a', b'A') | ||
[(1, 1)] | ||||
>>> difflineranges2(b'ab', b'') | ||||
[] | ||||
>>> difflineranges2(b'', b'AB') | ||||
|
r37200 | [(1, 2)] | ||
|
r37230 | >>> difflineranges2(b'abc', b'ac') | ||
|
r37200 | [] | ||
|
r37230 | >>> difflineranges2(b'ab', b'aCb') | ||
|
r37200 | [(2, 2)] | ||
|
r37230 | >>> difflineranges2(b'abc', b'aBc') | ||
|
r37200 | [(2, 2)] | ||
|
r37230 | >>> difflineranges2(b'ab', b'AB') | ||
|
r37200 | [(1, 2)] | ||
|
r37230 | >>> difflineranges2(b'abcde', b'aBcDe') | ||
|
r37200 | [(2, 2), (4, 4)] | ||
|
r37230 | >>> difflineranges2(b'abcde', b'aBCDe') | ||
|
r37200 | [(2, 4)] | ||
""" | ||||
ranges = [] | ||||
for lines, kind in mdiff.allblocks(content1, content2): | ||||
firstline, lastline = lines[2:4] | ||||
|
r43347 | if kind == b'!' and firstline != lastline: | ||
|
r37200 | ranges.append((firstline + 1, lastline)) | ||
return ranges | ||||
|
r43346 | |||
|
r37200 | def getbasectxs(repo, opts, revstofix): | ||
"""Returns a map of the base contexts for each revision | ||||
The base contexts determine which lines are considered modified when we | ||||
|
r38609 | attempt to fix just the modified lines in a file. It also determines which | ||
files we attempt to fix, so it is important to compute this even when | ||||
--whole is used. | ||||
|
r37200 | """ | ||
# The --base flag overrides the usual logic, and we give every revision | ||||
# exactly the set of baserevs that the user specified. | ||||
|
r43347 | if opts.get(b'base'): | ||
baserevs = set(scmutil.revrange(repo, opts.get(b'base'))) | ||||
|
r37200 | if not baserevs: | ||
baserevs = {nullrev} | ||||
basectxs = {repo[rev] for rev in baserevs} | ||||
return {rev: basectxs for rev in revstofix} | ||||
# Proceed in topological order so that we can easily determine each | ||||
# revision's baserevs by looking at its parents and their baserevs. | ||||
basectxs = collections.defaultdict(set) | ||||
for rev in sorted(revstofix): | ||||
ctx = repo[rev] | ||||
for pctx in ctx.parents(): | ||||
if pctx.rev() in basectxs: | ||||
basectxs[rev].update(basectxs[pctx.rev()]) | ||||
else: | ||||
basectxs[rev].add(pctx) | ||||
return basectxs | ||||
|
r43346 | |||
|
r42900 | def fixfile(ui, repo, opts, fixers, fixctx, path, basectxs): | ||
|
r37200 | """Run any configured fixers that should affect the file in this context | ||
Returns the file content that results from applying the fixers in some order | ||||
starting with the file's content in the fixctx. Fixers that support line | ||||
ranges will affect lines that have changed relative to any of the basectxs | ||||
(i.e. they will only avoid lines that are common to all basectxs). | ||||
|
r39003 | |||
A fixer tool's stdout will become the file's new content if and only if it | ||||
|
r42900 | exits with code zero. The fixer tool's working directory is the repository's | ||
root. | ||||
|
r37200 | """ | ||
|
r42372 | metadata = {} | ||
|
r37200 | newdata = fixctx[path].data() | ||
|
r43375 | for fixername, fixer in pycompat.iteritems(fixers): | ||
|
r37200 | if fixer.affects(opts, fixctx, path): | ||
|
r43003 | ranges = lineranges(opts, path, basectxs, fixctx, newdata) | ||
command = fixer.command(ui, path, ranges) | ||||
|
r37200 | if command is None: | ||
continue | ||||
|
r43347 | ui.debug(b'subprocess: %s\n' % (command,)) | ||
|
r37200 | proc = subprocess.Popen( | ||
|
r39868 | procutil.tonativestr(command), | ||
|
r37200 | shell=True, | ||
|
r43463 | cwd=procutil.tonativestr(repo.root), | ||
|
r37200 | stdin=subprocess.PIPE, | ||
stdout=subprocess.PIPE, | ||||
|
r43346 | stderr=subprocess.PIPE, | ||
) | ||||
|
r42372 | stdout, stderr = proc.communicate(newdata) | ||
|
r37200 | if stderr: | ||
showstderr(ui, fixctx.rev(), fixername, stderr) | ||||
|
r42372 | newerdata = stdout | ||
if fixer.shouldoutputmetadata(): | ||||
try: | ||||
|
r43347 | metadatajson, newerdata = stdout.split(b'\0', 1) | ||
|
r43697 | metadata[fixername] = pycompat.json_loads(metadatajson) | ||
|
r42372 | except ValueError: | ||
|
r43346 | ui.warn( | ||
|
r43347 | _(b'ignored invalid output from fixer tool: %s\n') | ||
|
r43346 | % (fixername,) | ||
) | ||||
|
r42372 | continue | ||
else: | ||||
metadata[fixername] = None | ||||
|
r39003 | if proc.returncode == 0: | ||
|
r37200 | newdata = newerdata | ||
|
r40568 | else: | ||
if not stderr: | ||||
|
r43347 | message = _(b'exited with status %d\n') % (proc.returncode,) | ||
|
r40568 | showstderr(ui, fixctx.rev(), fixername, message) | ||
checktoolfailureaction( | ||||
|
r43346 | ui, | ||
|
r43347 | _(b'no fixes will be applied'), | ||
|
r43346 | hint=_( | ||
|
r43347 | b'use --config fix.failure=continue to apply any ' | ||
b'successful fixes anyway' | ||||
|
r43346 | ), | ||
) | ||||
|
r42372 | return metadata, newdata | ||
|
r37200 | |||
|
r43346 | |||
|
r37200 | def showstderr(ui, rev, fixername, stderr): | ||
"""Writes the lines of the stderr string as warnings on the ui | ||||
Uses the revision number and fixername to give more context to each line of | ||||
the error message. Doesn't include file names, since those take up a lot of | ||||
space and would tend to be included in the error message if they were | ||||
relevant. | ||||
""" | ||||
|
r43347 | for line in re.split(b'[\r\n]+', stderr): | ||
|
r37200 | if line: | ||
|
r43347 | ui.warn(b'[') | ||
|
r37200 | if rev is None: | ||
|
r43347 | ui.warn(_(b'wdir'), label=b'evolve.rev') | ||
|
r37200 | else: | ||
|
r43347 | ui.warn((str(rev)), label=b'evolve.rev') | ||
ui.warn(b'] %s: %s\n' % (fixername, line)) | ||||
|
r43346 | |||
|
r37200 | |||
def writeworkingdir(repo, ctx, filedata, replacements): | ||||
"""Write new content to the working copy and check out the new p1 if any | ||||
We check out a new revision if and only if we fixed something in both the | ||||
working directory and its parent revision. This avoids the need for a full | ||||
update/merge, and means that the working directory simply isn't affected | ||||
unless the --working-dir flag is given. | ||||
Directly updates the dirstate for the affected files. | ||||
""" | ||||
|
r43375 | for path, data in pycompat.iteritems(filedata): | ||
|
r37200 | fctx = ctx[path] | ||
fctx.write(data, fctx.flags()) | ||||
|
r43347 | if repo.dirstate[path] == b'n': | ||
|
r37200 | repo.dirstate.normallookup(path) | ||
oldparentnodes = repo.dirstate.parents() | ||||
newparentnodes = [replacements.get(n, n) for n in oldparentnodes] | ||||
if newparentnodes != oldparentnodes: | ||||
repo.setparents(*newparentnodes) | ||||
|
r43346 | |||
|
r37200 | def replacerev(ui, repo, ctx, filedata, replacements): | ||
"""Commit a new revision like the given one, but with file content changes | ||||
"ctx" is the original revision to be replaced by a modified one. | ||||
"filedata" is a dict that maps paths to their new file content. All other | ||||
paths will be recreated from the original revision without changes. | ||||
"filedata" may contain paths that didn't exist in the original revision; | ||||
they will be added. | ||||
"replacements" is a dict that maps a single node to a single node, and it is | ||||
updated to indicate the original revision is replaced by the newly created | ||||
one. No entry is added if the replacement's node already exists. | ||||
The new revision has the same parents as the old one, unless those parents | ||||
have already been replaced, in which case those replacements are the parents | ||||
of this new revision. Thus, if revisions are replaced in topological order, | ||||
there is no need to rebase them into the original topology later. | ||||
""" | ||||
p1rev, p2rev = repo.changelog.parentrevs(ctx.rev()) | ||||
p1ctx, p2ctx = repo[p1rev], repo[p2rev] | ||||
newp1node = replacements.get(p1ctx.node(), p1ctx.node()) | ||||
newp2node = replacements.get(p2ctx.node(), p2ctx.node()) | ||||
|
r40604 | # We don't want to create a revision that has no changes from the original, | ||
# but we should if the original revision's parent has been replaced. | ||||
# Otherwise, we would produce an orphan that needs no actual human | ||||
# intervention to evolve. We can't rely on commit() to avoid creating the | ||||
# un-needed revision because the extra field added below produces a new hash | ||||
# regardless of file content changes. | ||||
|
r43346 | if ( | ||
not filedata | ||||
and p1ctx.node() not in replacements | ||||
and p2ctx.node() not in replacements | ||||
): | ||||
|
r40604 | return | ||
|
r37200 | def filectxfn(repo, memctx, path): | ||
if path not in ctx: | ||||
return None | ||||
fctx = ctx[path] | ||||
|
r42161 | copysource = fctx.copysource() | ||
|
r37200 | return context.memfilectx( | ||
repo, | ||||
memctx, | ||||
path=fctx.path(), | ||||
data=filedata.get(path, fctx.data()), | ||||
islink=fctx.islink(), | ||||
isexec=fctx.isexec(), | ||||
|
r43346 | copysource=copysource, | ||
) | ||||
|
r37200 | |||
|
r40604 | extra = ctx.extra().copy() | ||
|
r43347 | extra[b'fix_source'] = ctx.hex() | ||
|
r40604 | |||
|
r38442 | memctx = context.memctx( | ||
repo, | ||||
parents=(newp1node, newp2node), | ||||
text=ctx.description(), | ||||
files=set(ctx.files()) | set(filedata.keys()), | ||||
filectxfn=filectxfn, | ||||
user=ctx.user(), | ||||
date=ctx.date(), | ||||
|
r40604 | extra=extra, | ||
|
r38442 | branch=ctx.branch(), | ||
|
r43346 | editor=None, | ||
) | ||||
|
r38442 | sucnode = memctx.commit() | ||
prenode = ctx.node() | ||||
if prenode == sucnode: | ||||
|
r43347 | ui.debug(b'node %s already existed\n' % (ctx.hex())) | ||
|
r38442 | else: | ||
replacements[ctx.node()] = sucnode | ||||
|
r37200 | |||
|
r43346 | |||
|
r37200 | def getfixers(ui): | ||
"""Returns a map of configured fixer tools indexed by their names | ||||
Each value is a Fixer object with methods that implement the behavior of the | ||||
fixer's config suboptions. Does not validate the config values. | ||||
""" | ||||
|
r40599 | fixers = {} | ||
|
r37200 | for name in fixernames(ui): | ||
|
r43493 | enabled = ui.configbool(b'fix', name + b':enabled') | ||
command = ui.config(b'fix', name + b':command') | ||||
pattern = ui.config(b'fix', name + b':pattern') | ||||
linerange = ui.config(b'fix', name + b':linerange') | ||||
priority = ui.configint(b'fix', name + b':priority') | ||||
metadata = ui.configbool(b'fix', name + b':metadata') | ||||
skipclean = ui.configbool(b'fix', name + b':skipclean') | ||||
|
r42882 | # Don't use a fixer if it has no pattern configured. It would be | ||
# dangerous to let it affect all files. It would be pointless to let it | ||||
# affect no files. There is no reasonable subset of files to use as the | ||||
# default. | ||||
|
r43494 | if command is None: | ||
ui.warn( | ||||
_(b'fixer tool has no command configuration: %s\n') % (name,) | ||||
) | ||||
elif pattern is None: | ||||
|
r42882 | ui.warn( | ||
|
r43347 | _(b'fixer tool has no pattern configuration: %s\n') % (name,) | ||
|
r43346 | ) | ||
|
r43493 | elif not enabled: | ||
|
r43347 | ui.debug(b'ignoring disabled fixer tool: %s\n' % (name,)) | ||
|
r43493 | else: | ||
fixers[name] = Fixer( | ||||
command, pattern, linerange, priority, metadata, skipclean | ||||
) | ||||
|
r40599 | return collections.OrderedDict( | ||
|
r43346 | sorted(fixers.items(), key=lambda item: item[1]._priority, reverse=True) | ||
) | ||||
|
r37200 | |||
def fixernames(ui): | ||||
"""Returns the names of [fix] config options that have suboptions""" | ||||
names = set() | ||||
|
r43347 | for k, v in ui.configitems(b'fix'): | ||
if b':' in k: | ||||
names.add(k.split(b':', 1)[0]) | ||||
|
r37200 | return names | ||
|
r43346 | |||
|
r37200 | class Fixer(object): | ||
"""Wraps the raw config values for a fixer with methods""" | ||||
|
r43493 | def __init__( | ||
self, command, pattern, linerange, priority, metadata, skipclean | ||||
): | ||||
self._command = command | ||||
self._pattern = pattern | ||||
self._linerange = linerange | ||||
self._priority = priority | ||||
self._metadata = metadata | ||||
self._skipclean = skipclean | ||||
|
r37200 | def affects(self, opts, fixctx, path): | ||
"""Should this fixer run on the file at the given path and context?""" | ||||
|
r43501 | repo = fixctx.repo() | ||
matcher = matchmod.match( | ||||
repo.root, repo.root, [self._pattern], ctx=fixctx | ||||
) | ||||
return matcher(path) | ||||
|
r37200 | |||
|
r42372 | def shouldoutputmetadata(self): | ||
"""Should the stdout of this fixer start with JSON and a null byte?""" | ||||
return self._metadata | ||||
|
r43003 | def command(self, ui, path, ranges): | ||
|
r37200 | """A shell command to use to invoke this fixer on the given file/lines | ||
May return None if there is no appropriate command to run for the given | ||||
parameters. | ||||
""" | ||||
|
r37792 | expand = cmdutil.rendercommandtemplate | ||
|
r43346 | parts = [ | ||
expand( | ||||
ui, | ||||
self._command, | ||||
|
r43347 | {b'rootpath': path, b'basename': os.path.basename(path)}, | ||
|
r43346 | ) | ||
] | ||||
|
r37200 | if self._linerange: | ||
|
r43001 | if self._skipclean and not ranges: | ||
|
r37200 | # No line ranges to fix, so don't run the fixer. | ||
return None | ||||
for first, last in ranges: | ||||
|
r43346 | parts.append( | ||
|
r43347 | expand( | ||
ui, self._linerange, {b'first': first, b'last': last} | ||||
) | ||||
|
r43346 | ) | ||
|
r43347 | return b' '.join(parts) | ||