filemerge: add support for partial conflict resolution by external tool

A common class of merge conflicts is in imports/#includes/etc. It's
relatively easy to write a tool that can resolve these conflicts,
perhaps by naively just unioning the statements and leaving any
cleanup to other tools to do later [1]. Such specialized tools cannot
generally resolve all conflicts in a file, of course. Let's therefore
call them "partial merge tools". Note that the internal simplemerge
algorithm is such a partial merge tool - one that only resolves
trivial "conflicts" where one side is unchanged or both sides change
in the same way.

One can also imagine having smarter language-aware partial tools that
merge the AST. It may be useful for such tools to interactively let
the user resolve any conflicts it can't resolve itself. However,
having the option of implementing it as a partial merge tool means
that the developer doesn't *need* to create a UI for it. Instead, the
user can resolve any remaining conflicts with their regular merge tool
(e.g. `:merge3` or `meld).

We don't currently have a way to let the user define such partial
merge tools. That's what this patch addresses. It lets the user
configure partial merge tools to run. Each tool can be configured to
run only on files matching certain patterns (e.g. "*.py"). The tool
takes three inputs (local, base, other) and resolves conflicts by
updating these in place. For example, let's say the inputs are these:


base:
```
import sys

def main():
  print('Hello')
```

local:
```
import os
import sys

def main():
  print('Hi')
```

other:
```
import re
import sys

def main():
  print('Howdy')
```

A partial merge tool could now resolve the conflicting imports by
replacing the import statements in *all* files by the following
snippet, while leaving the remainder of the files unchanged.

```
import os
import re
import sys
```

As a result, simplemerge and any regular merge tool that runs after
the partial merge tool(s) will consider the imports to be
non-conflicting and will only present the conflict in `main()` to the
user.

Differential Revision: https://phab.mercurial-scm.org/D12356

tests/test-merge-partial-tool.t

@@ -0,0 +1,209 b''
+Test support for partial-resolution tools
+
+Create a tool that resolves conflicts after line 5 by simply dropping those
+lines (even if there are no conflicts there)
+  $ cat >> "$TESTTMP/head.sh" <<'EOF'
+  > #!/bin/sh
+  > for f in "$@"; do
+  >   head -5 $f > tmp
+  >   mv -f tmp $f
+  > done
+  > EOF
+  $ chmod +x "$TESTTMP/head.sh"
+...and another tool that keeps only the last 5 lines instead of the first 5.
+  $ cat >> "$TESTTMP/tail.sh" <<'EOF'
+  > #!/bin/sh
+  > for f in "$@"; do
+  >   tail -5 $f > tmp
+  >   mv -f tmp $f
+  > done
+  > EOF
+  $ chmod +x "$TESTTMP/tail.sh"
+
+Set up both tools to run on all patterns (the default), and let the `tail` tool
+run after the `head` tool, which means it will have no effect (we'll override it
+to test order later)
+  $ cat >> "$HGRCPATH" <<EOF
+  > [partial-merge-tools]
+  > head.executable=$TESTTMP/head.sh
+  > tail.executable=$TESTTMP/tail.sh
+  > tail.order=1
+  > EOF
+
+  $ make_commit() {
+  >   echo "$@" | xargs -n1 > file
+  >   hg add file 2> /dev/null
+  >   hg ci -m "$*"
+  > }
+
+
+Let a partial-resolution tool resolve some conflicts and leave other conflicts
+for the regular merge tool (:merge3 here)
+
+  $ hg init repo
+  $ cd repo
+  $ make_commit a b c d e f
+  $ make_commit a b2 c d e f2
+  $ hg up 0
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ make_commit a b3 c d e f3
+  created new head
+  $ hg merge 1 -t :merge3
+  merging file
+  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon
+  [1]
+  $ cat file
+  a
+  <<<<<<< working copy:    e11a49d4b620 - test: a b3 c d e f3
+  b3
+  ||||||| common ancestor: 8ae8bb9cc43a - test: a b c d e f
+  b
+  =======
+  b2
+  >>>>>>> merge rev:       fbc096a40cc5 - test: a b2 c d e f2
+  c
+  d
+  e
+
+
+With premerge=keep, the partial-resolution tools runs before and doesn't see
+the conflict markers
+
+  $ hg up -C 2
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cat >> .hg/hgrc <<EOF
+  > [merge-tools]
+  > my-local.executable = cat
+  > my-local.args = $local
+  > my-local.premerge = keep-merge3
+  > EOF
+  $ hg merge 1 -t my-local
+  merging file
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ cat file
+  a
+  <<<<<<< working copy:    e11a49d4b620 - test: a b3 c d e f3
+  b3
+  ||||||| common ancestor: 8ae8bb9cc43a - test: a b c d e f
+  b
+  =======
+  b2
+  >>>>>>> merge rev:       fbc096a40cc5 - test: a b2 c d e f2
+  c
+  d
+  e
+
+
+When a partial-resolution tool resolves all conflicts, the resolution should
+be recorded and the regular merge tool should not be invoked for the file.
+
+  $ hg up -C 0
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ make_commit a b c d e f2
+  created new head
+  $ hg up 0
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ make_commit a b c d e f3
+  created new head
+  $ hg merge 3 -t false
+  merging file
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ cat file
+  a
+  b
+  c
+  d
+  e
+
+
+Only tools whose patterns match are run. We make `head` not match here, so
+only `tail` should run
+
+  $ hg up -C 4
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg merge 3 -t :merge3 --config partial-merge-tools.head.patterns=other
+  merging file
+  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon
+  [1]
+  $ cat file
+  b
+  c
+  d
+  e
+  <<<<<<< working copy:    d57edaa6e21a - test: a b c d e f3
+  f3
+  ||||||| common ancestor: 8ae8bb9cc43a - test: a b c d e f
+  f
+  =======
+  f2
+  >>>>>>> merge rev:       8c217da987be - test: a b c d e f2
+
+
+If there are several matching tools, they are run in requested order. We move
+`head` after `tail` in order here so it has no effect (the conflict in "f" thus
+remains).
+
+  $ hg up -C 4
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg merge 3 -t :merge3 --config partial-merge-tools.head.order=2
+  merging file
+  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon
+  [1]
+  $ cat file
+  b
+  c
+  d
+  e
+  <<<<<<< working copy:    d57edaa6e21a - test: a b c d e f3
+  f3
+  ||||||| common ancestor: 8ae8bb9cc43a - test: a b c d e f
+  f
+  =======
+  f2
+  >>>>>>> merge rev:       8c217da987be - test: a b c d e f2
+
+
+When using "nomerge" tools (e.g. `:other`), the partial-resolution tools
+should not be run.
+
+  $ hg up -C 4
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg merge 3 -t :other
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ cat file
+  a
+  b
+  c
+  d
+  e
+  f2
+
+
+If a partial-resolution tool resolved some conflict and simplemerge can
+merge the rest, then the regular merge tool should not be used. Here we merge
+"a b c d e3 f3" with "a b2 c d e f2". The `head` tool resolves the conflict in
+"f" and the internal simplemerge merges the remaining changes in "b" and "e".
+
+  $ hg up -C 0
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ make_commit a b c d e3 f3
+  created new head
+  $ hg merge 1 -t false
+  merging file
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ cat file
+  a
+  b2
+  c
+  d
+  e3

mercurial/configitems.py

@@ -1570,6 +1570,37 b' coreconfigitem('
     default=False,
 )
 coreconfigitem(
+    b'partial-merge-tools',
+    b'.*',
+    default=None,
+    generic=True,
+    experimental=True,
+)
+coreconfigitem(
+    b'partial-merge-tools',
+    br'.*\.patterns',
+    default=dynamicdefault,
+    generic=True,
+    priority=-1,
+    experimental=True,
+)
+coreconfigitem(
+    b'partial-merge-tools',
+    br'.*\.executable$',
+    default=dynamicdefault,
+    generic=True,
+    priority=-1,
+    experimental=True,
+)
+coreconfigitem(
+    b'partial-merge-tools',
+    br'.*\.order',
+    default=0,
+    generic=True,
+    priority=-1,
+    experimental=True,
+)
+coreconfigitem(
     b'merge-tools',
     b'.*',
     default=None,

mercurial/filemerge.py

@@ -1051,6 +1051,7 b' def filemerge(repo, wctx, mynode, orig, '
             markerstyle = internalmarkerstyle
 
         if mergetype == fullmerge:
+            _run_partial_resolution_tools(repo, local, other, base)
             # conflict markers generated by premerge will use 'detailed'
             # settings if either ui.mergemarkers or the tool's mergemarkers
             # setting is 'detailed'. This way tools can have basic labels in
@@ -1115,6 +1116,62 b' def filemerge(repo, wctx, mynode, orig, '
             backup.remove()
 
 
+def _run_partial_resolution_tools(repo, local, other, base):
+    """Runs partial-resolution tools on the three inputs and updates them."""
+    ui = repo.ui
+    # Tuples of (order, name, executable path)
+    tools = []
+    seen = set()
+    section = b"partial-merge-tools"
+    for k, v in ui.configitems(section):
+        name = k.split(b'.')[0]
+        if name in seen:
+            continue
+        patterns = ui.configlist(section, b'%s.patterns' % name, [])
+        is_match = True
+        if patterns:
+            m = match.match(repo.root, b'', patterns)
+            is_match = m(local.fctx.path())
+        if is_match:
+            order = ui.configint(section, b'%s.order' % name, 0)
+            executable = ui.config(section, b'%s.executable' % name, name)
+            tools.append((order, name, executable))
+
+    if not tools:
+        return
+    # Sort in configured order (first in tuple)
+    tools.sort()
+
+    files = [
+        (b"local", local.fctx.path(), local.text()),
+        (b"base", base.fctx.path(), base.text()),
+        (b"other", other.fctx.path(), other.text()),
+    ]
+
+    with _maketempfiles(files) as temppaths:
+        localpath, basepath, otherpath = temppaths
+
+        for order, name, executable in tools:
+            cmd = procutil.shellquote(executable)
+            # TODO: Allow the user to configure the command line using
+            # $local, $base, $other.
+            cmd = b'%s %s %s %s' % (cmd, localpath, basepath, otherpath)
+            r = ui.system(cmd, cwd=repo.root, blockedtag=b'partial-mergetool')
+            if r:
+                raise error.StateError(
+                    b'partial merge tool %s exited with code %d' % (name, r)
+                )
+            local_text = util.readfile(localpath)
+            other_text = util.readfile(otherpath)
+            if local_text == other_text:
+                # No need to run other tools if all conflicts have been resolved
+                break
+
+        local.set_text(local_text)
+        base.set_text(util.readfile(basepath))
+        other.set_text(other_text)
+
+
 def _haltmerge():
     msg = _(b'merge halted after failed merge (see hg resolve)')
     raise error.InterventionRequired(msg)

mercurial/simplemerge.py

@@ -490,6 +490,9 b' class MergeInput:'
             self._text = self.fctx.decodeddata()
         return self._text
 
+    def set_text(self, text):
+        self._text = text
+
 
 def simplemerge(
     local,