merge default into stable for 5.5rc0

mercurial/helptext/internals/bid-merge.txt

@@ -0,0 +1,115 b''
+Bid merge is a feature introduced in Mercurial 3.0, a merge algorithm for
+dealing with complicated merges.
+
+Bid merge is controled  by the `merge.preferancestor` configuration option. The
+default is set to `merge.preferancetors=*` and enable bid merge. Mercurial will
+perform a bid merge in the cases where a merge otherwise would emit a note:
+using X as ancestor of X and X message.
+
+Problem it is solving
+=====================
+
+Mercurial's core merge algorithm is the traditional "three-way merge". This
+algorithm combines all the changes in two changesets relative to a common
+ancestor. But with complex DAGs, it is often possible to have more than one
+"best" common ancestor, with no easy way to distinguish between them.
+
+For example, C and D has 2 common ancestors in the following graph::
+
+  C   D
+  |\ /|
+  | x |
+  |/ \|
+  A   B
+   \ /
+    R
+
+Mercurial used to arbitrarily chooses the first of these, which can result in
+various issues:
+
+* unexpected hard 3-way merges that would have been completely trivial if
+  another ancestor had been used
+
+* conflicts that have already been resolved may reappear
+
+* changes that have been reversed can silently oscillate
+
+One common problem is a merge which with the "right" ancestor would be trivial
+to resolve because only one side changed. Using another ancestor where the same
+lines are different, it will give an annoying 3-way merge.
+
+Other systems like Git have attacked some of these problems with a so-called
+"recursive" merge strategy, that internally merges all the possible ancestors
+to produce a single "virtual" ancestor to merge against. This is awkward as the
+internal merge itself may involve conflicts (and possibly even multiple levels
+of recursion), which either requires choosing a conflict disposition (e.g.
+always choose the local version) or exposing the user to extremely confusing
+merge prompts for old revisions. Generating the virtual merge also potentially
+involves invoking filters and extensions.
+
+Concept
+=======
+
+(Bid merge is pretty much the same as Consensus merge.)
+
+Bid merge is a strategy that attempts to sensibly combine the results of the
+multiple possible three-way merges directly without producing a virtual
+ancestor. The basic idea is that for each ancestor, we perform a top-level
+manifest merge and generate a list of proposed actions, which we consider
+"bids". We then make an "auction" among all the bids for each file and pick the
+most favourable. Some files might be trivial to merge with one ancestor, other
+files with another ancestor.
+
+The most obvious advantage of considering multiple ancestors is the case where
+some of the bids for a file is a "real" (interactive) merge but where one or
+more bids just take on of the parent revisions. A bid for just taking an
+existing revision is very simple and low risk and is an obvious winner.
+
+The auction algorithm for merging the bids is so far very simple:
+
+* If there is consensus from all the ancestors, there is no doubt what to do. A
+  clever result will be indistinguishable from just picking a random bid. The
+  consensus case is thus not only trivial, it is also already handled
+  perfectly.
+
+* If "keep local" or "get from other" actions is an option (and there is only
+  one such option), just do it.
+
+* If the auction doesn't have a single clear winner, pick one of the bids
+  "randomly" - just as it would have done if only one ancestor was considered.
+
+This meta merge algorithm has room for future improvements, especially for
+doing better than picking a random bid.
+
+Some observations
+=================
+
+Experience with bid merge shows that many merges that actually have a very
+simple solution (because only one side changed) only can be solved efficiently
+when we start looking at file content in filemerge ... and it thus also
+requires all ancestors passed to filemerge. That is because Mercurial includes
+the history in filelog hashes. A file with changes that ends up not changing
+the content (could be change + backout or graft + merge or criss cross merges)
+still shows up as a changed file to manifestmerge. (The git data model has an
+advantage here when it uses hashes of content without history.) One way to
+handle that would be to refactor manifestmerge, mergestate/resolve and
+filemerge so they become more of the same thing.
+
+There is also cases where different conflicting chunks could benefit from using
+multiple ancestors in filemerge - but that will require merge tools with fancy
+support for using multiple ancestors in 3+-way merge. That is left as an
+exercise for another day. That seems to be a case where "recursive merge" has
+an advantage.
+
+The current manifest merge actions are very low level imperative and not
+symmetrical. They do not only describe how two manifests should be merged, they
+also describe a strategy for changing a context from a state where it is one of
+the parents to the state where it is the result of the merge with the other
+parent. I can imagine that manifestmerge could be simplified (and made more
+suitable for in memory merges) by separating the abstract merge actions from
+the actual file system operation actions. A more clever wcontext could perhaps
+also take care of some of the branchmerge special cases.
+
+We assume that the definition of Mercurial manifest merge will make sure that
+exactly the same files will be produced, no matter which ancestor is used. That
+assumption might be wrong in very rare cases that really not is a problem.

mercurial/metadata.py

@@ -0,0 +1,327 b''
+# metadata.py -- code related to various metadata computation and access.
+#
+# Copyright 2019 Google, Inc <[e-mail]>
+# Copyright 2020 Pierre-Yves David <[e-mail]>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+from __future__ import absolute_import, print_function
+
+import multiprocessing
+
+from . import (
+    error,
+    node,
+    pycompat,
+    util,
+)
+
+from .revlogutils import (
+    flagutil as sidedataflag,
+    sidedata as sidedatamod,
+)
+
+
+def computechangesetfilesadded(ctx):
+    """return the list of files added in a changeset
+    """
+    added = []
+    for f in ctx.files():
+        if not any(f in p for p in ctx.parents()):
+            added.append(f)
+    return added
+
+
+def get_removal_filter(ctx, x=None):
+    """return a function to detect files "wrongly" detected as `removed`
+
+    When a file is removed relative to p1 in a merge, this
+    function determines whether the absence is due to a
+    deletion from a parent, or whether the merge commit
+    itself deletes the file. We decide this by doing a
+    simplified three way merge of the manifest entry for
+    the file. There are two ways we decide the merge
+    itself didn't delete a file:
+    - neither parent (nor the merge) contain the file
+    - exactly one parent contains the file, and that
+      parent has the same filelog entry as the merge
+      ancestor (or all of them if there two). In other
+      words, that parent left the file unchanged while the
+      other one deleted it.
+    One way to think about this is that deleting a file is
+    similar to emptying it, so the list of changed files
+    should be similar either way. The computation
+    described above is not done directly in _filecommit
+    when creating the list of changed files, however
+    it does something very similar by comparing filelog
+    nodes.
+    """
+
+    if x is not None:
+        p1, p2, m1, m2 = x
+    else:
+        p1 = ctx.p1()
+        p2 = ctx.p2()
+        m1 = p1.manifest()
+        m2 = p2.manifest()
+
+    @util.cachefunc
+    def mas():
+        p1n = p1.node()
+        p2n = p2.node()
+        cahs = ctx.repo().changelog.commonancestorsheads(p1n, p2n)
+        if not cahs:
+            cahs = [node.nullrev]
+        return [ctx.repo()[r].manifest() for r in cahs]
+
+    def deletionfromparent(f):
+        if f in m1:
+            return f not in m2 and all(
+                f in ma and ma.find(f) == m1.find(f) for ma in mas()
+            )
+        elif f in m2:
+            return all(f in ma and ma.find(f) == m2.find(f) for ma in mas())
+        else:
+            return True
+
+    return deletionfromparent
+
+
+def computechangesetfilesremoved(ctx):
+    """return the list of files removed in a changeset
+    """
+    removed = []
+    for f in ctx.files():
+        if f not in ctx:
+            removed.append(f)
+    if removed:
+        rf = get_removal_filter(ctx)
+        removed = [r for r in removed if not rf(r)]
+    return removed
+
+
+def computechangesetcopies(ctx):
+    """return the copies data for a changeset
+
+    The copies data are returned as a pair of dictionnary (p1copies, p2copies).
+
+    Each dictionnary are in the form: `{newname: oldname}`
+    """
+    p1copies = {}
+    p2copies = {}
+    p1 = ctx.p1()
+    p2 = ctx.p2()
+    narrowmatch = ctx._repo.narrowmatch()
+    for dst in ctx.files():
+        if not narrowmatch(dst) or dst not in ctx:
+            continue
+        copied = ctx[dst].renamed()
+        if not copied:
+            continue
+        src, srcnode = copied
+        if src in p1 and p1[src].filenode() == srcnode:
+            p1copies[dst] = src
+        elif src in p2 and p2[src].filenode() == srcnode:
+            p2copies[dst] = src
+    return p1copies, p2copies
+
+
+def encodecopies(files, copies):
+    items = []
+    for i, dst in enumerate(files):
+        if dst in copies:
+            items.append(b'%d\0%s' % (i, copies[dst]))
+    if len(items) != len(copies):
+        raise error.ProgrammingError(
+            b'some copy targets missing from file list'
+        )
+    return b"\n".join(items)
+
+
+def decodecopies(files, data):
+    try:
+        copies = {}
+        if not data:
+            return copies
+        for l in data.split(b'\n'):
+            strindex, src = l.split(b'\0')
+            i = int(strindex)
+            dst = files[i]
+            copies[dst] = src
+        return copies
+    except (ValueError, IndexError):
+        # Perhaps someone had chosen the same key name (e.g. "p1copies") and
+        # used different syntax for the value.
+        return None
+
+
+def encodefileindices(files, subset):
+    subset = set(subset)
+    indices = []
+    for i, f in enumerate(files):
+        if f in subset:
+            indices.append(b'%d' % i)
+    return b'\n'.join(indices)
+
+
+def decodefileindices(files, data):
+    try:
+        subset = []
+        if not data:
+            return subset
+        for strindex in data.split(b'\n'):
+            i = int(strindex)
+            if i < 0 or i >= len(files):
+                return None
+            subset.append(files[i])
+        return subset
+    except (ValueError, IndexError):
+        # Perhaps someone had chosen the same key name (e.g. "added") and
+        # used different syntax for the value.
+        return None
+
+
+def _getsidedata(srcrepo, rev):
+    ctx = srcrepo[rev]
+    filescopies = computechangesetcopies(ctx)
+    filesadded = computechangesetfilesadded(ctx)
+    filesremoved = computechangesetfilesremoved(ctx)
+    sidedata = {}
+    if any([filescopies, filesadded, filesremoved]):
+        sortedfiles = sorted(ctx.files())
+        p1copies, p2copies = filescopies
+        p1copies = encodecopies(sortedfiles, p1copies)
+        p2copies = encodecopies(sortedfiles, p2copies)
+        filesadded = encodefileindices(sortedfiles, filesadded)
+        filesremoved = encodefileindices(sortedfiles, filesremoved)
+        if p1copies:
+            sidedata[sidedatamod.SD_P1COPIES] = p1copies
+        if p2copies:
+            sidedata[sidedatamod.SD_P2COPIES] = p2copies
+        if filesadded:
+            sidedata[sidedatamod.SD_FILESADDED] = filesadded
+        if filesremoved:
+            sidedata[sidedatamod.SD_FILESREMOVED] = filesremoved
+    return sidedata
+
+
+def getsidedataadder(srcrepo, destrepo):
+    use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
+    if pycompat.iswindows or not use_w:
+        return _get_simple_sidedata_adder(srcrepo, destrepo)
+    else:
+        return _get_worker_sidedata_adder(srcrepo, destrepo)
+
+
+def _sidedata_worker(srcrepo, revs_queue, sidedata_queue, tokens):
+    """The function used by worker precomputing sidedata
+
+    It read an input queue containing revision numbers
+    It write in an output queue containing (rev, <sidedata-map>)
+
+    The `None` input value is used as a stop signal.
+
+    The `tokens` semaphore is user to avoid having too many unprocessed
+    entries. The workers needs to acquire one token before fetching a task.
+    They will be released by the consumer of the produced data.
+    """
+    tokens.acquire()
+    rev = revs_queue.get()
+    while rev is not None:
+        data = _getsidedata(srcrepo, rev)
+        sidedata_queue.put((rev, data))
+        tokens.acquire()
+        rev = revs_queue.get()
+    # processing of `None` is completed, release the token.
+    tokens.release()
+
+
+BUFF_PER_WORKER = 50
+
+
+def _get_worker_sidedata_adder(srcrepo, destrepo):
+    """The parallel version of the sidedata computation
+
+    This code spawn a pool of worker that precompute a buffer of sidedata
+    before we actually need them"""
+    # avoid circular import copies -> scmutil -> worker -> copies
+    from . import worker
+
+    nbworkers = worker._numworkers(srcrepo.ui)
+
+    tokens = multiprocessing.BoundedSemaphore(nbworkers * BUFF_PER_WORKER)
+    revsq = multiprocessing.Queue()
+    sidedataq = multiprocessing.Queue()
+
+    assert srcrepo.filtername is None
+    # queue all tasks beforehand, revision numbers are small and it make
+    # synchronisation simpler
+    #
+    # Since the computation for each node can be quite expensive, the overhead
+    # of using a single queue is not revelant. In practice, most computation
+    # are fast but some are very expensive and dominate all the other smaller
+    # cost.
+    for r in srcrepo.changelog.revs():
+        revsq.put(r)
+    # queue the "no more tasks" markers
+    for i in range(nbworkers):
+        revsq.put(None)
+
+    allworkers = []
+    for i in range(nbworkers):
+        args = (srcrepo, revsq, sidedataq, tokens)
+        w = multiprocessing.Process(target=_sidedata_worker, args=args)
+        allworkers.append(w)
+        w.start()
+
+    # dictionnary to store results for revision higher than we one we are
+    # looking for. For example, if we need the sidedatamap for 42, and 43 is
+    # received, when shelve 43 for later use.
+    staging = {}
+
+    def sidedata_companion(revlog, rev):
+        sidedata = {}
+        if util.safehasattr(revlog, b'filteredrevs'):  # this is a changelog
+            # Is the data previously shelved ?
+            sidedata = staging.pop(rev, None)
+            if sidedata is None:
+                # look at the queued result until we find the one we are lookig
+                # for (shelve the other ones)
+                r, sidedata = sidedataq.get()
+                while r != rev:
+                    staging[r] = sidedata
+                    r, sidedata = sidedataq.get()
+            tokens.release()
+        return False, (), sidedata
+
+    return sidedata_companion
+
+
+def _get_simple_sidedata_adder(srcrepo, destrepo):
+    """The simple version of the sidedata computation
+
+    It just compute it in the same thread on request"""
+
+    def sidedatacompanion(revlog, rev):
+        sidedata = {}
+        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
+            sidedata = _getsidedata(srcrepo, rev)
+        return False, (), sidedata
+
+    return sidedatacompanion
+
+
+def getsidedataremover(srcrepo, destrepo):
+    def sidedatacompanion(revlog, rev):
+        f = ()
+        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
+            if revlog.flags(rev) & sidedataflag.REVIDX_SIDEDATA:
+                f = (
+                    sidedatamod.SD_P1COPIES,
+                    sidedatamod.SD_P2COPIES,
+                    sidedatamod.SD_FILESADDED,
+                    sidedatamod.SD_FILESREMOVED,
+                )
+        return False, f, {}
+
+    return sidedatacompanion

.hgignore

@@ -52,6 +52,7 b' cscope.*'
 .idea/*
 .asv/*
 .pytype/*
+.mypy_cache
 i18n/hg.pot
 locale/*/LC_MESSAGES/hg.mo
 hgext/__index__.py

contrib/chg/chg.c

@@ -232,7 +232,7 b' static void execcmdserver(const struct c'
 			abortmsgerrno("failed to putenv CHG_CLEAR_LC_CTYPE");
 	} else {
 		if (setenv("CHGORIG_LC_CTYPE", lc_ctype_env, 1) != 0) {
-			abortmsgerrno("failed to setenv CHGORIG_LC_CTYYPE");
+			abortmsgerrno("failed to setenv CHGORIG_LC_CTYPE");
 		}
 	}
 

contrib/dumprevlog

@@ -28,7 +28,7 b' binopen.options = {}'
 
 def printb(data, end=b'\n'):
     sys.stdout.flush()
-    pycompat.stdout.write(data + end)
+    procutil.stdout.write(data + end)
 
 
 for f in sys.argv[1:]:

contrib/fuzz/Makefile

@@ -11,6 +11,7 b' CXX = clang++'
 LIB_FUZZING_ENGINE ?= standalone_fuzz_target_runner.o
 
 PYTHON_CONFIG ?= $$OUT/sanpy/bin/python-config
+PYTHON_CONFIG_FLAGS ?= --ldflags
 
 CXXFLAGS += -Wno-deprecated-register
 
@@ -67,7 +68,7 b' dirs_fuzzer: dirs.cc pyutil.o $(PARSERS_'
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial dirs.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/dirs_fuzzer
 
 fncache_fuzzer: fncache.cc 
@@ -75,7 +76,7 b' fncache_fuzzer: fncache.cc'
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial fncache.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/fncache_fuzzer
 
 jsonescapeu8fast_fuzzer: jsonescapeu8fast.cc pyutil.o $(PARSERS_OBJS)
@@ -83,7 +84,7 b' jsonescapeu8fast_fuzzer: jsonescapeu8fas'
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial jsonescapeu8fast.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/jsonescapeu8fast_fuzzer
 
 manifest_fuzzer: manifest.cc pyutil.o $(PARSERS_OBJS) $$OUT/manifest_fuzzer_seed_corpus.zip
@@ -91,7 +92,7 b' manifest_fuzzer: manifest.cc pyutil.o $('
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial manifest.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/manifest_fuzzer
 
 revlog_fuzzer: revlog.cc pyutil.o $(PARSERS_OBJS) $$OUT/revlog_fuzzer_seed_corpus.zip
@@ -99,7 +100,7 b' revlog_fuzzer: revlog.cc pyutil.o $(PARS'
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial revlog.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/revlog_fuzzer
 
 dirstate_fuzzer: dirstate.cc pyutil.o $(PARSERS_OBJS) $$OUT/dirstate_fuzzer_seed_corpus.zip
@@ -107,7 +108,7 b' dirstate_fuzzer: dirstate.cc pyutil.o $('
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial dirstate.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/dirstate_fuzzer
 
 fm1readmarkers_fuzzer: fm1readmarkers.cc pyutil.o $(PARSERS_OBJS) $$OUT/fm1readmarkers_fuzzer_seed_corpus.zip
@@ -115,7 +116,7 b' fm1readmarkers_fuzzer: fm1readmarkers.cc'
 	  -Wno-register -Wno-macro-redefined \
 	  -I../../mercurial fm1readmarkers.cc \
 	  pyutil.o $(PARSERS_OBJS) \
-	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) --ldflags` \
+	  $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
 	  -o $$OUT/fm1readmarkers_fuzzer
 
 clean:

contrib/fuzz/manifest.cc

@@ -3,6 +3,7 b''
 #include <stdlib.h>
 #include <unistd.h>
 
+#include "FuzzedDataProvider.h"
 #include "pyutil.h"
 
 #include <string>
@@ -24,7 +25,7 b' try:'
       lm[e]
       e in lm
       (e + 'nope') in lm
-  lm[b'xyzzy'] = (b'\0' * 20, 'x')
+  lm[b'xyzzy'] = (b'\0' * nlen, 'x')
   # do an insert, text should change
   assert lm.text() != mdata, "insert should change text and didn't: %r %r" % (lm.text(), mdata)
   cloned = lm.filtercopy(lambda x: x != 'xyzzy')
@@ -51,10 +52,14 b' int LLVMFuzzerTestOneInput(const uint8_t'
 	if (Size > 100000) {
 		return 0;
 	}
+	FuzzedDataProvider provider(Data, Size);
+	Py_ssize_t nodelength = provider.ConsumeBool() ? 20 : 32;
+	PyObject *nlen = PyLong_FromSsize_t(nodelength);
 	PyObject *mtext =
 	    PyBytes_FromStringAndSize((const char *)Data, (Py_ssize_t)Size);
 	PyObject *locals = PyDict_New();
 	PyDict_SetItemString(locals, "mdata", mtext);
+	PyDict_SetItemString(locals, "nlen", nlen);
 	PyObject *res = PyEval_EvalCode(code, contrib::pyglobals(), locals);
 	if (!res) {
 		PyErr_Print();

contrib/fuzz/manifest_corpus.py

@@ -10,7 +10,7 b' args = ap.parse_args()'
 with zipfile.ZipFile(args.out[0], "w", zipfile.ZIP_STORED) as zf:
     zf.writestr(
         "manifest_zero",
-        '''PKG-INFO\09b3ed8f2b81095a13064402e930565f083346e9a
+        '''\0PKG-INFO\09b3ed8f2b81095a13064402e930565f083346e9a
 README\080b6e76643dcb44d4bc729e932fc464b3e36dbe3
 hg\0b6444347c629cc058d478023905cfb83b7f5bb9d
 mercurial/__init__.py\0b80de5d138758541c5f05265ad144ab9fa86d1db
@@ -25,9 +25,14 b' setup.py\\0ccf3f6daf0f13101ca73631f7a1769'
 tkmerge\03c922edb43a9c143682f7bc7b00f98b3c756ebe7
 ''',
     )
-    zf.writestr("badmanifest_shorthashes", "narf\0aa\nnarf2\0aaa\n")
+    zf.writestr("badmanifest_shorthashes", "\0narf\0aa\nnarf2\0aaa\n")
     zf.writestr(
         "badmanifest_nonull",
-        "narf\0cccccccccccccccccccccccccccccccccccccccc\n"
+        "\0narf\0cccccccccccccccccccccccccccccccccccccccc\n"
         "narf2aaaaaaaaaaaaaaaaaaaa\n",
     )
+
+    zf.writestr(
+        "manifest_long_nodes",
+        "\1a\0ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff\n",
+    )

contrib/fuzz/pyutil.cc

@@ -21,7 +21,7 b' static PyObject *globals;'
 void initpy(const char *cselfpath)
 {
 #ifdef HG_FUZZER_PY3
-	const std::string subdir = "/sanpy/lib/python3.7";
+	const std::string subdir = "/sanpy/lib/python3.8";
 #else
 	const std::string subdir = "/sanpy/lib/python2.7";
 #endif

contrib/heptapod-ci.yml

@@ -5,6 +5,8 b' image: octobus/ci-mercurial-core'
 before_script:
     - hg clone . /tmp/mercurial-ci/ --noupdate
     - hg -R /tmp/mercurial-ci/ update `hg log --rev '.' --template '{node}'`
+    - cd /tmp/mercurial-ci/rust/rhg
+    - cargo build
     - cd /tmp/mercurial-ci/
     - ls -1 tests/test-check-*.* > /tmp/check-tests.txt
 
@@ -79,3 +81,9 b' test-py3-rust:'
         RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"
         PYTHON: python3
         TEST_HGMODULEPOLICY: "rust+c"
+
+test-py2-chg:
+    <<: *runtests
+    variables:
+        RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt --chg"
+        TEST_HGMODULEPOLICY: "c"

contrib/packaging/debian/rules

@@ -2,14 +2,51 b''
 # Uncomment this to turn on verbose mode.
 # export DH_VERBOSE=1
 
+# By default we build a .deb where the native components are built with the
+# current "default" version of py3 on the build machine. If you wish to build a
+# .deb that has native components built for multiple versions of py3:
+#
+#   1. install python3.x and python3.x-dev for each version you want
+#   2. set DEB_HG_MULTI_VERSION=1 or DEB_HG_PYTHON_VERSIONS in your environment
+#      (if both are set, DEB_HG_PYTHON_VERSIONS has precedence)
+#
+# If you choose `DEB_HG_MULTI_VERSION=1`, it will build for every "supported"
+# version of py3 that's installed on the build machine. This may not be equal to
+# the actual versions that are installed, see the comment above where we set
+# DEB_HG_PYTHON_VERSIONS below. If you choose to set `DEB_HG_PYTHON_VERSIONS`
+# yourself, set it to a space-separated string of python version numbers, like:
+#   DEB_HG_PYTHON_VERSIONS="3.7 3.8" make deb
+DEB_HG_MULTI_VERSION?=0
+
 CPUS=$(shell cat /proc/cpuinfo | grep -E ^processor | wc -l)
 
+# By default, only build for the version of python3 that the system considers
+# the 'default' (which should be the one invoked by just running 'python3'
+# without a minor version). If DEB_HG_PYTHON_VERSIONS is set, this is ignored.
+ifeq ($(DEB_HG_MULTI_VERSION), 1)
+	# If we're building for multiple versions, use all of the "supported" versions
+	# on the build machine. Note: the mechanism in use here (`py3versions`) is the
+	# recommended one, but it relies on a file written by the python3-minimal
+	# package, and this file is not dynamic and does not account for manual
+	# installations, just the ones that would be installed by `python3-all`. This
+	# includes the `-i` flag, which claims it's to list all "installed" versions,
+	# but it doesn't. This was quite confusing, hence this tale of woe. :)
+	DEB_HG_PYTHON_VERSIONS?=$(shell py3versions -vs)
+else
+	# If we're building for only one version, identify the "default" version on
+	# the build machine and use that when building; this is just so that we don't
+	# have to duplicate the rules below for multi-version vs. single-version. The
+	# shebang line will still be /usr/bin/python3 (no minor version).
+	DEB_HG_PYTHON_VERSIONS?=$(shell py3versions -vd)
+endif
+
 export HGPYTHON3=1
 export PYTHON=python3
 
 %:
 	dh $@ --with python3
 
+# Note: testing can be disabled using the standard `DEB_BUILD_OPTIONS=nocheck`
 override_dh_auto_test:
 	http_proxy='' dh_auto_test -- TESTFLAGS="-j$(CPUS)"
 
@@ -24,8 +61,15 b' override_dh_auto_build:'
 	$(MAKE) all
 	$(MAKE) -C contrib/chg all
 
-override_dh_auto_install:
-	python3 setup.py install --root "$(CURDIR)"/debian/mercurial --install-layout=deb
+# Build the native extensions for a specfic python3 version (which must be
+# installed on the build machine).
+install-python%:
+	python$* setup.py install --root "$(CURDIR)"/debian/mercurial --install-layout=deb
+
+# Build the final package. This rule has a dependencies section that causes the
+# native extensions to be compiled for every version of python3 listed in
+# DEB_HG_PYTHON_VERSIONS.
+override_dh_auto_install: $(DEB_HG_PYTHON_VERSIONS:%=install-python%)
 	# chg
 	make -C contrib/chg \
 		DESTDIR="$(CURDIR)"/debian/mercurial \

contrib/perf.py

@@ -3794,19 +3794,47 b' def perflrucache('
         fm.end()
 
 
-@command(b'perfwrite', formatteropts)
+@command(
+    b'perfwrite',
+    formatteropts
+    + [
+        (b'', b'write-method', b'write', b'ui write method'),
+        (b'', b'nlines', 100, b'number of lines'),
+        (b'', b'nitems', 100, b'number of items (per line)'),
+        (b'', b'item', b'x', b'item that is written'),
+        (b'', b'batch-line', None, b'pass whole line to write method at once'),
+        (b'', b'flush-line', None, b'flush after each line'),
+    ],
+)
 def perfwrite(ui, repo, **opts):
-    """microbenchmark ui.write
+    """microbenchmark ui.write (and others)
     """
     opts = _byteskwargs(opts)
 
+    write = getattr(ui, _sysstr(opts[b'write_method']))
+    nlines = int(opts[b'nlines'])
+    nitems = int(opts[b'nitems'])
+    item = opts[b'item']
+    batch_line = opts.get(b'batch_line')
+    flush_line = opts.get(b'flush_line')
+
+    if batch_line:
+        line = item * nitems + b'\n'
+
+    def benchmark():
+        for i in pycompat.xrange(nlines):
+            if batch_line:
+                write(line)
+            else:
+                for i in pycompat.xrange(nitems):
+                    write(item)
+                write(b'\n')
+            if flush_line:
+                ui.flush()
+        ui.flush()
+
     timer, fm = gettimer(ui, opts)
-
-    def write():
-        for i in range(100000):
-            ui.writenoi18n(b'Testing write performance\n')
-
-    timer(write)
+    timer(benchmark)
     fm.end()
 
 

contrib/simplemerge

@@ -45,8 +45,8 b' class ParseError(Exception):'
 
 
 def showhelp():
-    pycompat.stdout.write(usage)
-    pycompat.stdout.write(b'\noptions:\n')
+    procutil.stdout.write(usage)
+    procutil.stdout.write(b'\noptions:\n')
 
     out_opts = []
     for shortopt, longopt, default, desc in options:
@@ -62,11 +62,11 b' def showhelp():'
         )
     opts_len = max([len(opt[0]) for opt in out_opts])
     for first, second in out_opts:
-        pycompat.stdout.write(b' %-*s  %s\n' % (opts_len, first, second))
+        procutil.stdout.write(b' %-*s  %s\n' % (opts_len, first, second))
 
 
 try:
-    for fp in (sys.stdin, pycompat.stdout, sys.stderr):
+    for fp in (sys.stdin, procutil.stdout, sys.stderr):
         procutil.setbinary(fp)
 
     opts = {}
@@ -92,11 +92,11 b' try:'
     )
 except ParseError as e:
     e = stringutil.forcebytestr(e)
-    pycompat.stdout.write(b"%s: %s\n" % (sys.argv[0].encode('utf8'), e))
+    procutil.stdout.write(b"%s: %s\n" % (sys.argv[0].encode('utf8'), e))
     showhelp()
     sys.exit(1)
 except error.Abort as e:
-    pycompat.stderr.write(b"abort: %s\n" % e)
+    procutil.stderr.write(b"abort: %s\n" % e)
     sys.exit(255)
 except KeyboardInterrupt:
     sys.exit(255)

contrib/undumprevlog

@@ -9,7 +9,6 b' import sys'
 from mercurial import (
     encoding,
     node,
-    pycompat,
     revlog,
     transaction,
     vfs as vfsmod,
@@ -30,7 +29,7 b' while True:'
     if l.startswith("file:"):
         f = encoding.strtolocal(l[6:-1])
         r = revlog.revlog(opener, f)
-        pycompat.stdout.write(b'%s\n' % f)
+        procutil.stdout.write(b'%s\n' % f)
     elif l.startswith("node:"):
         n = node.bin(l[6:-1])
     elif l.startswith("linkrev:"):

hgext/absorb.py

@@ -50,6 +50,7 b' from mercurial import ('
     phases,
     pycompat,
     registrar,
+    rewriteutil,
     scmutil,
     util,
 )
@@ -782,7 +783,10 b' class fixupstate(object):'
                 # nothing changed, nothing commited
                 nextp1 = ctx
                 continue
-            if self._willbecomenoop(memworkingcopy, ctx, nextp1):
+            willbecomenoop = ctx.files() and self._willbecomenoop(
+                memworkingcopy, ctx, nextp1
+            )
+            if self.skip_empty_successor and willbecomenoop:
                 # changeset is no longer necessary
                 self.replacemap[ctx.node()] = None
                 msg = _(b'became empty and was dropped')
@@ -793,7 +797,11 b' class fixupstate(object):'
                 nextp1 = lastcommitted
                 self.replacemap[ctx.node()] = lastcommitted.node()
                 if memworkingcopy:
-                    msg = _(b'%d file(s) changed, became %s') % (
+                    if willbecomenoop:
+                        msg = _(b'%d file(s) changed, became empty as %s')
+                    else:
+                        msg = _(b'%d file(s) changed, became %s')
+                    msg = msg % (
                         len(memworkingcopy),
                         self._ctx2str(lastcommitted),
                     )
@@ -887,6 +895,10 b' class fixupstate(object):'
             if len(parents) != 1:
                 return False
             pctx = parents[0]
+        if ctx.branch() != pctx.branch():
+            return False
+        if ctx.extra().get(b'close'):
+            return False
         # ctx changes more files (not a subset of memworkingcopy)
         if not set(ctx.files()).issubset(set(memworkingcopy)):
             return False
@@ -929,6 +941,10 b' class fixupstate(object):'
                 self.repo, replacements, operation=b'absorb', fixphase=True
             )
 
+    @util.propertycache
+    def skip_empty_successor(self):
+        return rewriteutil.skip_empty_successor(self.ui, b'absorb')
+
 
 def _parsechunk(hunk):
     """(crecord.uihunk or patch.recordhunk) -> (path, (a1, a2, [bline]))"""
@@ -1045,7 +1061,7 b' def absorb(ui, repo, stack=None, targetc'
             not opts.get(b'apply_changes')
             and state.ctxaffected
             and ui.promptchoice(
-                b"apply changes (yn)? $$ &Yes $$ &No", default=1
+                b"apply changes (y/N)? $$ &Yes $$ &No", default=1
             )
         ):
             raise error.Abort(_(b'absorb cancelled\n'))

hgext/convert/cvs.py

@@ -226,8 +226,7 b' class convert_cvs(converter_source):'
                     cmd = [rsh, host] + cmd
 
             # popen2 does not support argument lists under Windows
-            cmd = [procutil.shellquote(arg) for arg in cmd]
-            cmd = procutil.quotecommand(b' '.join(cmd))
+            cmd = b' '.join(procutil.shellquote(arg) for arg in cmd)
             self.writep, self.readp = procutil.popen2(cmd)
 
         self.realroot = root

hgext/convert/gnuarch.py

@@ -217,7 +217,7 b' class gnuarch_source(common.converter_so'
         cmdline = [procutil.shellquote(arg) for arg in cmdline]
         bdevnull = pycompat.bytestr(os.devnull)
         cmdline += [b'>', bdevnull, b'2>', bdevnull]
-        cmdline = procutil.quotecommand(b' '.join(cmdline))
+        cmdline = b' '.join(cmdline)
         self.ui.debug(cmdline, b'\n')
         return os.system(pycompat.rapply(procutil.tonativestr, cmdline))
 

hgext/convert/subversion.py

@@ -1366,7 +1366,7 b' class svn_source(converter_source):'
         arg = encodeargs(args)
         hgexe = procutil.hgexecutable()
         cmd = b'%s debugsvnlog' % procutil.shellquote(hgexe)
-        stdin, stdout = procutil.popen2(procutil.quotecommand(cmd))
+        stdin, stdout = procutil.popen2(cmd)
         stdin.write(arg)
         try:
             stdin.close()

hgext/extdiff.py

@@ -233,7 +233,6 b' def _systembackground(cmd, environ=None,'
     ''' like 'procutil.system', but returns the Popen object directly
         so we don't have to wait on it.
     '''
-    cmd = procutil.quotecommand(cmd)
     env = procutil.shellenviron(environ)
     proc = subprocess.Popen(
         procutil.tonativestr(cmd),
@@ -351,6 +350,187 b' def _runperfilediff('
                 proc.wait()
 
 
+def diffpatch(ui, repo, node1, node2, tmproot, matcher, cmdline):
+    template = b'hg-%h.patch'
+    # write patches to temporary files
+    with formatter.nullformatter(ui, b'extdiff', {}) as fm:
+        cmdutil.export(
+            repo,
+            [repo[node1].rev(), repo[node2].rev()],
+            fm,
+            fntemplate=repo.vfs.reljoin(tmproot, template),
+            match=matcher,
+        )
+    label1 = cmdutil.makefilename(repo[node1], template)
+    label2 = cmdutil.makefilename(repo[node2], template)
+    file1 = repo.vfs.reljoin(tmproot, label1)
+    file2 = repo.vfs.reljoin(tmproot, label2)
+    cmdline = formatcmdline(
+        cmdline,
+        repo.root,
+        # no 3way while comparing patches
+        do3way=False,
+        parent1=file1,
+        plabel1=label1,
+        # while comparing patches, there is no second parent
+        parent2=None,
+        plabel2=None,
+        child=file2,
+        clabel=label2,
+    )
+    ui.debug(b'running %r in %s\n' % (pycompat.bytestr(cmdline), tmproot))
+    ui.system(cmdline, cwd=tmproot, blockedtag=b'extdiff')
+    return 1
+
+
+def diffrevs(
+    ui,
+    repo,
+    node1a,
+    node1b,
+    node2,
+    matcher,
+    tmproot,
+    cmdline,
+    do3way,
+    guitool,
+    opts,
+):
+
+    subrepos = opts.get(b'subrepos')
+
+    # calculate list of files changed between both revs
+    st = repo.status(node1a, node2, matcher, listsubrepos=subrepos)
+    mod_a, add_a, rem_a = set(st.modified), set(st.added), set(st.removed)
+    if do3way:
+        stb = repo.status(node1b, node2, matcher, listsubrepos=subrepos)
+        mod_b, add_b, rem_b = (
+            set(stb.modified),
+            set(stb.added),
+            set(stb.removed),
+        )
+    else:
+        mod_b, add_b, rem_b = set(), set(), set()
+    modadd = mod_a | add_a | mod_b | add_b
+    common = modadd | rem_a | rem_b
+    if not common:
+        return 0
+
+    # Always make a copy of node1a (and node1b, if applicable)
+    # dir1a should contain files which are:
+    #   * modified or removed from node1a to node2
+    #   * modified or added from node1b to node2
+    #     (except file added from node1a to node2 as they were not present in
+    #     node1a)
+    dir1a_files = mod_a | rem_a | ((mod_b | add_b) - add_a)
+    dir1a = snapshot(ui, repo, dir1a_files, node1a, tmproot, subrepos)[0]
+    rev1a = b'@%d' % repo[node1a].rev()
+    if do3way:
+        # file calculation criteria same as dir1a
+        dir1b_files = mod_b | rem_b | ((mod_a | add_a) - add_b)
+        dir1b = snapshot(ui, repo, dir1b_files, node1b, tmproot, subrepos)[0]
+        rev1b = b'@%d' % repo[node1b].rev()
+    else:
+        dir1b = None
+        rev1b = b''
+
+    fnsandstat = []
+
+    # If node2 in not the wc or there is >1 change, copy it
+    dir2root = b''
+    rev2 = b''
+    if node2:
+        dir2 = snapshot(ui, repo, modadd, node2, tmproot, subrepos)[0]
+        rev2 = b'@%d' % repo[node2].rev()
+    elif len(common) > 1:
+        # we only actually need to get the files to copy back to
+        # the working dir in this case (because the other cases
+        # are: diffing 2 revisions or single file -- in which case
+        # the file is already directly passed to the diff tool).
+        dir2, fnsandstat = snapshot(ui, repo, modadd, None, tmproot, subrepos)
+    else:
+        # This lets the diff tool open the changed file directly
+        dir2 = b''
+        dir2root = repo.root
+
+    label1a = rev1a
+    label1b = rev1b
+    label2 = rev2
+
+    # If only one change, diff the files instead of the directories
+    # Handle bogus modifies correctly by checking if the files exist
+    if len(common) == 1:
+        common_file = util.localpath(common.pop())
+        dir1a = os.path.join(tmproot, dir1a, common_file)
+        label1a = common_file + rev1a
+        if not os.path.isfile(dir1a):
+            dir1a = pycompat.osdevnull
+        if do3way:
+            dir1b = os.path.join(tmproot, dir1b, common_file)
+            label1b = common_file + rev1b
+            if not os.path.isfile(dir1b):
+                dir1b = pycompat.osdevnull
+        dir2 = os.path.join(dir2root, dir2, common_file)
+        label2 = common_file + rev2
+
+    if not opts.get(b'per_file'):
+        # Run the external tool on the 2 temp directories or the patches
+        cmdline = formatcmdline(
+            cmdline,
+            repo.root,
+            do3way=do3way,
+            parent1=dir1a,
+            plabel1=label1a,
+            parent2=dir1b,
+            plabel2=label1b,
+            child=dir2,
+            clabel=label2,
+        )
+        ui.debug(b'running %r in %s\n' % (pycompat.bytestr(cmdline), tmproot))
+        ui.system(cmdline, cwd=tmproot, blockedtag=b'extdiff')
+    else:
+        # Run the external tool once for each pair of files
+        _runperfilediff(
+            cmdline,
+            repo.root,
+            ui,
+            guitool=guitool,
+            do3way=do3way,
+            confirm=opts.get(b'confirm'),
+            commonfiles=common,
+            tmproot=tmproot,
+            dir1a=dir1a,
+            dir1b=dir1b,
+            dir2root=dir2root,
+            dir2=dir2,
+            rev1a=rev1a,
+            rev1b=rev1b,
+            rev2=rev2,
+        )
+
+    for copy_fn, working_fn, st in fnsandstat:
+        cpstat = os.lstat(copy_fn)
+        # Some tools copy the file and attributes, so mtime may not detect
+        # all changes.  A size check will detect more cases, but not all.
+        # The only certain way to detect every case is to diff all files,
+        # which could be expensive.
+        # copyfile() carries over the permission, so the mode check could
+        # be in an 'elif' branch, but for the case where the file has
+        # changed without affecting mtime or size.
+        if (
+            cpstat[stat.ST_MTIME] != st[stat.ST_MTIME]
+            or cpstat.st_size != st.st_size
+            or (cpstat.st_mode & 0o100) != (st.st_mode & 0o100)
+        ):
+            ui.debug(
+                b'file changed while diffing. '
+                b'Overwriting: %s (src: %s)\n' % (working_fn, copy_fn)
+            )
+            util.copyfile(copy_fn, working_fn)
+
+    return 1
+
+
 def dodiff(ui, repo, cmdline, pats, opts, guitool=False):
     '''Do the actual diff:
 
@@ -360,14 +540,12 b' def dodiff(ui, repo, cmdline, pats, opts'
     - just invoke the diff for a single file in the working dir
     '''
 
+    cmdutil.check_at_most_one_arg(opts, b'rev', b'change')
     revs = opts.get(b'rev')
     change = opts.get(b'change')
     do3way = b'$parent2' in cmdline
 
-    if revs and change:
-        msg = _(b'cannot specify --rev and --change at the same time')
-        raise error.Abort(msg)
-    elif change:
+    if change:
         ctx2 = scmutil.revsingle(repo, change, None)
         ctx1a, ctx1b = ctx2.p1(), ctx2.p2()
     else:
@@ -377,9 +555,6 b' def dodiff(ui, repo, cmdline, pats, opts'
         else:
             ctx1b = repo[nullid]
 
-    perfile = opts.get(b'per_file')
-    confirm = opts.get(b'confirm')
-
     node1a = ctx1a.node()
     node1b = ctx1b.node()
     node2 = ctx2.node()
@@ -389,169 +564,35 b' def dodiff(ui, repo, cmdline, pats, opts'
         if node1b == nullid:
             do3way = False
 
-    subrepos = opts.get(b'subrepos')
-
     matcher = scmutil.match(repo[node2], pats, opts)
 
     if opts.get(b'patch'):
-        if subrepos:
+        if opts.get(b'subrepos'):
             raise error.Abort(_(b'--patch cannot be used with --subrepos'))
-        if perfile:
+        if opts.get(b'per_file'):
             raise error.Abort(_(b'--patch cannot be used with --per-file'))
         if node2 is None:
             raise error.Abort(_(b'--patch requires two revisions'))
-    else:
-        st = repo.status(node1a, node2, matcher, listsubrepos=subrepos)
-        mod_a, add_a, rem_a = set(st.modified), set(st.added), set(st.removed)
-        if do3way:
-            stb = repo.status(node1b, node2, matcher, listsubrepos=subrepos)
-            mod_b, add_b, rem_b = (
-                set(stb.modified),
-                set(stb.added),
-                set(stb.removed),
-            )
-        else:
-            mod_b, add_b, rem_b = set(), set(), set()
-        modadd = mod_a | add_a | mod_b | add_b
-        common = modadd | rem_a | rem_b
-        if not common:
-            return 0
 
     tmproot = pycompat.mkdtemp(prefix=b'extdiff.')
     try:
-        if not opts.get(b'patch'):
-            # Always make a copy of node1a (and node1b, if applicable)
-            dir1a_files = mod_a | rem_a | ((mod_b | add_b) - add_a)
-            dir1a = snapshot(ui, repo, dir1a_files, node1a, tmproot, subrepos)[
-                0
-            ]
-            rev1a = b'@%d' % repo[node1a].rev()
-            if do3way:
-                dir1b_files = mod_b | rem_b | ((mod_a | add_a) - add_b)
-                dir1b = snapshot(
-                    ui, repo, dir1b_files, node1b, tmproot, subrepos
-                )[0]
-                rev1b = b'@%d' % repo[node1b].rev()
-            else:
-                dir1b = None
-                rev1b = b''
-
-            fnsandstat = []
-
-            # If node2 in not the wc or there is >1 change, copy it
-            dir2root = b''
-            rev2 = b''
-            if node2:
-                dir2 = snapshot(ui, repo, modadd, node2, tmproot, subrepos)[0]
-                rev2 = b'@%d' % repo[node2].rev()
-            elif len(common) > 1:
-                # we only actually need to get the files to copy back to
-                # the working dir in this case (because the other cases
-                # are: diffing 2 revisions or single file -- in which case
-                # the file is already directly passed to the diff tool).
-                dir2, fnsandstat = snapshot(
-                    ui, repo, modadd, None, tmproot, subrepos
-                )
-            else:
-                # This lets the diff tool open the changed file directly
-                dir2 = b''
-                dir2root = repo.root
-
-            label1a = rev1a
-            label1b = rev1b
-            label2 = rev2
+        if opts.get(b'patch'):
+            return diffpatch(ui, repo, node1a, node2, tmproot, matcher, cmdline)
 
-            # If only one change, diff the files instead of the directories
-            # Handle bogus modifies correctly by checking if the files exist
-            if len(common) == 1:
-                common_file = util.localpath(common.pop())
-                dir1a = os.path.join(tmproot, dir1a, common_file)
-                label1a = common_file + rev1a
-                if not os.path.isfile(dir1a):
-                    dir1a = pycompat.osdevnull
-                if do3way:
-                    dir1b = os.path.join(tmproot, dir1b, common_file)
-                    label1b = common_file + rev1b
-                    if not os.path.isfile(dir1b):
-                        dir1b = pycompat.osdevnull
-                dir2 = os.path.join(dir2root, dir2, common_file)
-                label2 = common_file + rev2
-        else:
-            template = b'hg-%h.patch'
-            with formatter.nullformatter(ui, b'extdiff', {}) as fm:
-                cmdutil.export(
-                    repo,
-                    [repo[node1a].rev(), repo[node2].rev()],
-                    fm,
-                    fntemplate=repo.vfs.reljoin(tmproot, template),
-                    match=matcher,
-                )
-            label1a = cmdutil.makefilename(repo[node1a], template)
-            label2 = cmdutil.makefilename(repo[node2], template)
-            dir1a = repo.vfs.reljoin(tmproot, label1a)
-            dir2 = repo.vfs.reljoin(tmproot, label2)
-            dir1b = None
-            label1b = None
-            fnsandstat = []
+        return diffrevs(
+            ui,
+            repo,
+            node1a,
+            node1b,
+            node2,
+            matcher,
+            tmproot,
+            cmdline,
+            do3way,
+            guitool,
+            opts,
+        )
 
-        if not perfile:
-            # Run the external tool on the 2 temp directories or the patches
-            cmdline = formatcmdline(
-                cmdline,
-                repo.root,
-                do3way=do3way,
-                parent1=dir1a,
-                plabel1=label1a,
-                parent2=dir1b,
-                plabel2=label1b,
-                child=dir2,
-                clabel=label2,
-            )
-            ui.debug(
-                b'running %r in %s\n' % (pycompat.bytestr(cmdline), tmproot)
-            )
-            ui.system(cmdline, cwd=tmproot, blockedtag=b'extdiff')
-        else:
-            # Run the external tool once for each pair of files
-            _runperfilediff(
-                cmdline,
-                repo.root,
-                ui,
-                guitool=guitool,
-                do3way=do3way,
-                confirm=confirm,
-                commonfiles=common,
-                tmproot=tmproot,
-                dir1a=dir1a,
-                dir1b=dir1b,
-                dir2root=dir2root,
-                dir2=dir2,
-                rev1a=rev1a,
-                rev1b=rev1b,
-                rev2=rev2,
-            )
-
-        for copy_fn, working_fn, st in fnsandstat:
-            cpstat = os.lstat(copy_fn)
-            # Some tools copy the file and attributes, so mtime may not detect
-            # all changes.  A size check will detect more cases, but not all.
-            # The only certain way to detect every case is to diff all files,
-            # which could be expensive.
-            # copyfile() carries over the permission, so the mode check could
-            # be in an 'elif' branch, but for the case where the file has
-            # changed without affecting mtime or size.
-            if (
-                cpstat[stat.ST_MTIME] != st[stat.ST_MTIME]
-                or cpstat.st_size != st.st_size
-                or (cpstat.st_mode & 0o100) != (st.st_mode & 0o100)
-            ):
-                ui.debug(
-                    b'file changed while diffing. '
-                    b'Overwriting: %s (src: %s)\n' % (working_fn, copy_fn)
-                )
-                util.copyfile(copy_fn, working_fn)
-
-        return 1
     finally:
         ui.note(_(b'cleaning up temp directory\n'))
         shutil.rmtree(tmproot)

hgext/fix.py

@@ -144,6 +144,7 b' from mercurial import ('
     match as matchmod,
     mdiff,
     merge,
+    mergestate as mergestatemod,
     pycompat,
     registrar,
     rewriteutil,
@@ -267,8 +268,14 b' def fix(ui, repo, *pats, **opts):'
         workqueue, numitems = getworkqueue(
             ui, repo, pats, opts, revstofix, basectxs
         )
+        basepaths = getbasepaths(repo, opts, workqueue, basectxs)
         fixers = getfixers(ui)
 
+        # Rather than letting each worker independently fetch the files
+        # (which also would add complications for shared/keepalive
+        # connections), prefetch them all first.
+        _prefetchfiles(repo, workqueue, basepaths)
+
         # There are no data dependencies between the workers fixing each file
         # revision, so we can use all available parallelism.
         def getfixes(items):
@@ -276,7 +283,7 b' def fix(ui, repo, *pats, **opts):'
                 ctx = repo[rev]
                 olddata = ctx[path].data()
                 metadata, newdata = fixfile(
-                    ui, repo, opts, fixers, ctx, path, basectxs[rev]
+                    ui, repo, opts, fixers, ctx, path, basepaths, basectxs[rev]
                 )
                 # Don't waste memory/time passing unchanged content back, but
                 # produce one result per item either way.
@@ -426,7 +433,9 b' def getrevstofix(ui, repo, opts):'
     if not (len(revs) == 1 and wdirrev in revs):
         cmdutil.checkunfinished(repo)
         rewriteutil.precheck(repo, revs, b'fix')
-    if wdirrev in revs and list(merge.mergestate.read(repo).unresolved()):
+    if wdirrev in revs and list(
+        mergestatemod.mergestate.read(repo).unresolved()
+    ):
         raise error.Abort(b'unresolved conflicts', hint=b"use 'hg resolve'")
     if not revs:
         raise error.Abort(
@@ -470,7 +479,7 b' def pathstofix(ui, repo, pats, opts, mat'
     return files
 
 
-def lineranges(opts, path, basectxs, fixctx, content2):
+def lineranges(opts, path, basepaths, basectxs, fixctx, content2):
     """Returns the set of line ranges that should be fixed in a file
 
     Of the form [(10, 20), (30, 40)].
@@ -489,7 +498,8 b' def lineranges(opts, path, basectxs, fix'
 
     rangeslist = []
     for basectx in basectxs:
-        basepath = copies.pathcopies(basectx, fixctx).get(path, path)
+        basepath = basepaths.get((basectx.rev(), fixctx.rev(), path), path)
+
         if basepath in basectx:
             content1 = basectx[basepath].data()
         else:
@@ -498,6 +508,21 b' def lineranges(opts, path, basectxs, fix'
     return unionranges(rangeslist)
 
 
+def getbasepaths(repo, opts, workqueue, basectxs):
+    if opts.get(b'whole'):
+        # Base paths will never be fetched for line range determination.
+        return {}
+
+    basepaths = {}
+    for rev, path in workqueue:
+        fixctx = repo[rev]
+        for basectx in basectxs[rev]:
+            basepath = copies.pathcopies(basectx, fixctx).get(path, path)
+            if basepath in basectx:
+                basepaths[(basectx.rev(), fixctx.rev(), path)] = basepath
+    return basepaths
+
+
 def unionranges(rangeslist):
     """Return the union of some closed intervals
 
@@ -610,7 +635,30 b' def getbasectxs(repo, opts, revstofix):'
     return basectxs
 
 
-def fixfile(ui, repo, opts, fixers, fixctx, path, basectxs):
+def _prefetchfiles(repo, workqueue, basepaths):
+    toprefetch = set()
+
+    # Prefetch the files that will be fixed.
+    for rev, path in workqueue:
+        if rev == wdirrev:
+            continue
+        toprefetch.add((rev, path))
+
+    # Prefetch the base contents for lineranges().
+    for (baserev, fixrev, path), basepath in basepaths.items():
+        toprefetch.add((baserev, basepath))
+
+    if toprefetch:
+        scmutil.prefetchfiles(
+            repo,
+            [
+                (rev, scmutil.matchfiles(repo, [path]))
+                for rev, path in toprefetch
+            ],
+        )
+
+
+def fixfile(ui, repo, opts, fixers, fixctx, path, basepaths, basectxs):
     """Run any configured fixers that should affect the file in this context
 
     Returns the file content that results from applying the fixers in some order
@@ -626,7 +674,9 b' def fixfile(ui, repo, opts, fixers, fixc'
     newdata = fixctx[path].data()
     for fixername, fixer in pycompat.iteritems(fixers):
         if fixer.affects(opts, fixctx, path):
-            ranges = lineranges(opts, path, basectxs, fixctx, newdata)
+            ranges = lineranges(
+                opts, path, basepaths, basectxs, fixctx, newdata
+            )
             command = fixer.command(ui, path, ranges)
             if command is None:
                 continue

hgext/git/__init__.py

@@ -16,6 +16,7 b' from mercurial import ('
     extensions,
     localrepo,
     pycompat,
+    registrar,
     scmutil,
     store,
     util,
@@ -28,6 +29,13 b' from . import ('
     index,
 )
 
+configtable = {}
+configitem = registrar.configitem(configtable)
+# git.log-index-cache-miss: internal knob for testing
+configitem(
+    b"git", b"log-index-cache-miss", default=False,
+)
+
 # TODO: extract an interface for this in core
 class gitstore(object):  # store.basicstore):
     def __init__(self, path, vfstype):
@@ -41,13 +49,14 b' class gitstore(object): # store.basicst'
             os.path.normpath(os.path.join(path, b'..', b'.git'))
         )
         self._progress_factory = lambda *args, **kwargs: None
+        self._logfn = lambda x: None
 
     @util.propertycache
     def _db(self):
         # We lazy-create the database because we want to thread a
         # progress callback down to the indexing process if it's
         # required, and we don't have a ui handle in makestore().
-        return index.get_index(self.git, self._progress_factory)
+        return index.get_index(self.git, self._logfn, self._progress_factory)
 
     def join(self, f):
         """Fake store.join method for git repositories.
@@ -276,6 +285,8 b' def reposetup(ui, repo):'
     if repo.local() and isinstance(repo.store, gitstore):
         orig = repo.__class__
         repo.store._progress_factory = repo.ui.makeprogress
+        if ui.configbool(b'git', b'log-index-cache-miss'):
+            repo.store._logfn = repo.ui.warn
 
         class gitlocalrepo(orig):
             def _makedirstate(self):

hgext/git/dirstate.py

@@ -288,6 +288,10 b' class gitdirstate(object):'
         # TODO: track copies?
         return None
 
+    def prefetch_parents(self):
+        # TODO
+        pass
+
     @contextlib.contextmanager
     def parentchange(self):
         # TODO: track this maybe?

hgext/git/gitlog.py

@@ -247,6 +247,60 b' class changelog(baselog):'
     def descendants(self, revs):
         return dagop.descendantrevs(revs, self.revs, self.parentrevs)
 
+    def incrementalmissingrevs(self, common=None):
+        """Return an object that can be used to incrementally compute the
+        revision numbers of the ancestors of arbitrary sets that are not
+        ancestors of common. This is an ancestor.incrementalmissingancestors
+        object.
+
+        'common' is a list of revision numbers. If common is not supplied, uses
+        nullrev.
+        """
+        if common is None:
+            common = [nodemod.nullrev]
+
+        return ancestor.incrementalmissingancestors(self.parentrevs, common)
+
+    def findmissing(self, common=None, heads=None):
+        """Return the ancestors of heads that are not ancestors of common.
+
+        More specifically, return a list of nodes N such that every N
+        satisfies the following constraints:
+
+          1. N is an ancestor of some node in 'heads'
+          2. N is not an ancestor of any node in 'common'
+
+        The list is sorted by revision number, meaning it is
+        topologically sorted.
+
+        'heads' and 'common' are both lists of node IDs.  If heads is
+        not supplied, uses all of the revlog's heads.  If common is not
+        supplied, uses nullid."""
+        if common is None:
+            common = [nodemod.nullid]
+        if heads is None:
+            heads = self.heads()
+
+        common = [self.rev(n) for n in common]
+        heads = [self.rev(n) for n in heads]
+
+        inc = self.incrementalmissingrevs(common=common)
+        return [self.node(r) for r in inc.missingancestors(heads)]
+
+    def children(self, node):
+        """find the children of a given node"""
+        c = []
+        p = self.rev(node)
+        for r in self.revs(start=p + 1):
+            prevs = [pr for pr in self.parentrevs(r) if pr != nodemod.nullrev]
+            if prevs:
+                for pr in prevs:
+                    if pr == p:
+                        c.append(self.node(r))
+            elif p == nodemod.nullrev:
+                c.append(self.node(r))
+        return c
+
     def reachableroots(self, minroot, heads, roots, includepath=False):
         return dagop._reachablerootspure(
             self.parentrevs, minroot, roots, heads, includepath
@@ -270,7 +324,10 b' class changelog(baselog):'
     def parentrevs(self, rev):
         n = self.node(rev)
         hn = gitutil.togitnode(n)
-        c = self.gitrepo[hn]
+        if hn != gitutil.nullgit:
+            c = self.gitrepo[hn]
+        else:
+            return nodemod.nullrev, nodemod.nullrev
         p1 = p2 = nodemod.nullrev
         if c.parents:
             p1 = self.rev(c.parents[0].id.raw)
@@ -342,7 +399,7 b' class changelog(baselog):'
             'refs/hg/internal/latest-commit', oid, force=True
         )
         # Reindex now to pick up changes. We omit the progress
-        # callback because this will be very quick.
+        # and log callbacks because this will be very quick.
         index._index_repo(self.gitrepo, self._db)
         return oid.raw
 

hgext/git/index.py

@@ -216,7 +216,12 b' def fill_in_filelog(gitrepo, db, startco'
     db.commit()
 
 
-def _index_repo(gitrepo, db, progress_factory=lambda *args, **kwargs: None):
+def _index_repo(
+    gitrepo,
+    db,
+    logfn=lambda x: None,
+    progress_factory=lambda *args, **kwargs: None,
+):
     # Identify all references so we can tell the walker to visit all of them.
     all_refs = gitrepo.listall_references()
     possible_heads = set()
@@ -245,11 +250,15 b' def _index_repo(gitrepo, db, progress_fa'
     # TODO: we should figure out how to incrementally index history
     # (preferably by detecting rewinds!) so that we don't have to do a
     # full changelog walk every time a new commit is created.
-    cache_heads = {x[0] for x in db.execute('SELECT node FROM possible_heads')}
+    cache_heads = {
+        pycompat.sysstr(x[0])
+        for x in db.execute('SELECT node FROM possible_heads')
+    }
     walker = None
     cur_cache_heads = {h.hex for h in possible_heads}
     if cur_cache_heads == cache_heads:
         return
+    logfn(b'heads mismatch, rebuilding dagcache\n')
     for start in possible_heads:
         if walker is None:
             walker = gitrepo.walk(start, _OUR_ORDER)
@@ -336,7 +345,9 b' def _index_repo(gitrepo, db, progress_fa'
         prog.complete()
 
 
-def get_index(gitrepo, progress_factory=lambda *args, **kwargs: None):
+def get_index(
+    gitrepo, logfn=lambda x: None, progress_factory=lambda *args, **kwargs: None
+):
     cachepath = os.path.join(
         pycompat.fsencode(gitrepo.path), b'..', b'.hg', b'cache'
     )
@@ -346,5 +357,5 b' def get_index(gitrepo, progress_factory='
     db = _createdb(dbpath)
     # TODO check against gitrepo heads before doing a full index
     # TODO thread a ui.progress call into this layer
-    _index_repo(gitrepo, db, progress_factory)
+    _index_repo(gitrepo, db, logfn, progress_factory)
     return db

hgext/git/manifest.py

@@ -56,8 +56,9 b' class gittreemanifest(object):'
             return val
         t = self._tree
         comps = upath.split('/')
+        te = self._tree
         for comp in comps[:-1]:
-            te = self._tree[comp]
+            te = te[comp]
             t = self._git_repo[te.id]
         ent = t[comps[-1]]
         if ent.filemode == pygit2.GIT_FILEMODE_BLOB:
@@ -125,9 +126,79 b' class gittreemanifest(object):'
     def hasdir(self, dir):
         return dir in self._dirs
 
-    def diff(self, other, match=None, clean=False):
-        # TODO
-        assert False
+    def diff(self, other, match=lambda x: True, clean=False):
+        '''Finds changes between the current manifest and m2.
+
+        The result is returned as a dict with filename as key and
+        values of the form ((n1,fl1),(n2,fl2)), where n1/n2 is the
+        nodeid in the current/other manifest and fl1/fl2 is the flag
+        in the current/other manifest. Where the file does not exist,
+        the nodeid will be None and the flags will be the empty
+        string.
+        '''
+        result = {}
+
+        def _iterativediff(t1, t2, subdir):
+            """compares two trees and appends new tree nodes to examine to
+            the stack"""
+            if t1 is None:
+                t1 = {}
+            if t2 is None:
+                t2 = {}
+
+            for e1 in t1:
+                realname = subdir + pycompat.fsencode(e1.name)
+
+                if e1.type == pygit2.GIT_OBJ_TREE:
+                    try:
+                        e2 = t2[e1.name]
+                        if e2.type != pygit2.GIT_OBJ_TREE:
+                            e2 = None
+                    except KeyError:
+                        e2 = None
+
+                    stack.append((realname + b'/', e1, e2))
+                else:
+                    n1, fl1 = self.find(realname)
+
+                    try:
+                        e2 = t2[e1.name]
+                        n2, fl2 = other.find(realname)
+                    except KeyError:
+                        e2 = None
+                        n2, fl2 = (None, b'')
+
+                    if e2 is not None and e2.type == pygit2.GIT_OBJ_TREE:
+                        stack.append((realname + b'/', None, e2))
+
+                    if not match(realname):
+                        continue
+
+                    if n1 != n2 or fl1 != fl2:
+                        result[realname] = ((n1, fl1), (n2, fl2))
+                    elif clean:
+                        result[realname] = None
+
+            for e2 in t2:
+                if e2.name in t1:
+                    continue
+
+                realname = subdir + pycompat.fsencode(e2.name)
+
+                if e2.type == pygit2.GIT_OBJ_TREE:
+                    stack.append((realname + b'/', None, e2))
+                elif match(realname):
+                    n2, fl2 = other.find(realname)
+                    result[realname] = ((None, b''), (n2, fl2))
+
+        stack = []
+        _iterativediff(self._tree, other._tree, b'')
+        while stack:
+            subdir, t1, t2 = stack.pop()
+            # stack is populated in the function call
+            _iterativediff(t1, t2, subdir)
+
+        return result
 
     def setflag(self, path, flag):
         node, unused_flag = self._resolve_entry(path)
@@ -168,14 +239,13 b' class gittreemanifest(object):'
         for te in tree:
             # TODO: can we prune dir walks with the matcher?
             realname = subdir + pycompat.fsencode(te.name)
-            if te.type == r'tree':
+            if te.type == pygit2.GIT_OBJ_TREE:
                 for inner in self._walkonetree(
                     self._git_repo[te.id], match, realname + b'/'
                 ):
                     yield inner
-            if not match(realname):
-                continue
-            yield pycompat.fsencode(realname)
+            elif match(realname):
+                yield pycompat.fsencode(realname)
 
     def walk(self, match):
         # TODO: this is a very lazy way to merge in the pending
@@ -205,7 +275,7 b' class gittreemanifestctx(object):'
         return memgittreemanifestctx(self._repo, self._tree)
 
     def find(self, path):
-        self.read()[path]
+        return self.read()[path]
 
 
 @interfaceutil.implementer(repository.imanifestrevisionwritable)

hgext/githelp.py

@@ -628,8 +628,17 b' def log(ui, repo, *args, **kwargs):'
         (b'', b'stat', None, b''),
         (b'', b'graph', None, b''),
         (b'p', b'patch', None, b''),
+        (b'G', b'grep-diff', b'', b''),
+        (b'S', b'pickaxe-regex', b'', b''),
     ]
     args, opts = parseoptions(ui, cmdoptions, args)
+    grep_pat = opts.get(b'grep_diff') or opts.get(b'pickaxe_regex')
+    if grep_pat:
+        cmd = Command(b'grep')
+        cmd[b'--diff'] = grep_pat
+        ui.status(b'%s\n' % bytes(cmd))
+        return
+
     ui.status(
         _(
             b'note: -v prints the entire commit message like Git does. To '

hgext/histedit.py

@@ -223,6 +223,7 b' from mercurial import ('
     hg,
     logcmdutil,
     merge as mergemod,
+    mergestate as mergestatemod,
     mergeutil,
     node,
     obsolete,
@@ -2285,7 +2286,7 b' def _getsummary(ctx):'
 def bootstrapcontinue(ui, state, opts):
     repo = state.repo
 
-    ms = mergemod.mergestate.read(repo)
+    ms = mergestatemod.mergestate.read(repo)
     mergeutil.checkunresolved(ms)
 
     if state.actions:

hgext/hooklib/changeset_obsoleted.py

@@ -122,10 +122,18 b' def _report_commit(ui, repo, ctx):'
         )
 
 
+def has_successor(repo, rev):
+    return any(
+        r for r in obsutil.allsuccessors(repo.obsstore, [rev]) if r != rev
+    )
+
+
 def hook(ui, repo, hooktype, node=None, **kwargs):
-    if hooktype != b"pretxnclose":
+    if hooktype != b"txnclose":
         raise error.Abort(
             _(b'Unsupported hook type %r') % pycompat.bytestr(hooktype)
         )
-    for rev in obsutil.getobsoleted(repo, repo.currenttransaction()):
-        _report_commit(ui, repo, repo.unfiltered()[rev])
+    for rev in obsutil.getobsoleted(repo, changes=kwargs['changes']):
+        ctx = repo.unfiltered()[rev]
+        if not has_successor(repo, ctx.node()):
+            _report_commit(ui, repo, ctx)

hgext/infinitepush/__init__.py

@@ -466,7 +466,7 b' def _rebundle(bundlerepo, bundleroots, u'
 
     version = b'02'
     outgoing = discovery.outgoing(
-        bundlerepo, commonheads=bundleroots, missingheads=[unknownhead]
+        bundlerepo, commonheads=bundleroots, ancestorsof=[unknownhead]
     )
     cgstream = changegroup.makestream(bundlerepo, outgoing, version, b'pull')
     cgstream = util.chunkbuffer(cgstream).read()

hgext/largefiles/lfcommands.py

@@ -163,7 +163,7 b' def lfconvert(ui, src, dest, *pats, **op'
             # to the destination repository's requirements.
             if lfiles:
                 rdst.requirements.add(b'largefiles')
-                rdst._writerequirements()
+                scmutil.writereporequirements(rdst)
         else:
 
             class lfsource(filemap.filemap_source):

hgext/largefiles/overrides.py

@@ -31,6 +31,7 b' from mercurial import ('
     logcmdutil,
     match as matchmod,
     merge,
+    mergestate as mergestatemod,
     pathutil,
     pycompat,
     scmutil,
@@ -622,7 +623,7 b' def overridecalculateupdates('
     return actions, diverge, renamedelete
 
 
[e-mail](merge, b'recordupdates')
[e-mail](mergestatemod, b'recordupdates')
 def mergerecordupdates(orig, repo, actions, branchmerge, getfiledata):
     if b'lfmr' in actions:
         lfdirstate = lfutil.openlfdirstate(repo.ui, repo)

hgext/largefiles/reposetup.py

@@ -448,7 +448,7 b' def reposetup(ui, repo):'
             lfutil.shortname + b'/' in f[0] for f in repo.store.datafiles()
         ):
             repo.requirements.add(b'largefiles')
-            repo._writerequirements()
+            scmutil.writereporequirements(repo)
 
     ui.setconfig(
         b'hooks', b'changegroup.lfiles', checkrequireslfiles, b'largefiles'

hgext/lfs/__init__.py

@@ -255,7 +255,7 b' def _reposetup(ui, repo):'
                 ):
                     repo.requirements.add(b'lfs')
                     repo.features.add(repository.REPO_FEATURE_LFS)
-                    repo._writerequirements()
+                    scmutil.writereporequirements(repo)
                     repo.prepushoutgoinghooks.add(b'lfs', wrapper.prepush)
                     break
 

hgext/lfs/wrapper.py

@@ -312,7 +312,7 b' def convertsink(orig, sink):'
                     # membership before assuming it is in the context.
                     if any(f in ctx and ctx[f].islfs() for f, n in files):
                         self.repo.requirements.add(b'lfs')
-                        self.repo._writerequirements()
+                        scmutil.writereporequirements(self.repo)
 
                 return node
 
@@ -337,7 +337,7 b' def vfsinit(orig, self, othervfs):'
             setattr(self, name, getattr(othervfs, name))
 
 
-def _prefetchfiles(repo, revs, match):
+def _prefetchfiles(repo, revmatches):
     """Ensure that required LFS blobs are present, fetching them as a group if
     needed."""
     if not util.safehasattr(repo.svfs, b'lfslocalblobstore'):
@@ -347,7 +347,7 b' def _prefetchfiles(repo, revs, match):'
     oids = set()
     localstore = repo.svfs.lfslocalblobstore
 
-    for rev in revs:
+    for rev, match in revmatches:
         ctx = repo[rev]
         for f in ctx.walk(match):
             p = pointerfromctx(ctx, f)

hgext/mq.py

@@ -836,7 +836,15 b' class queue(object):'
         stat = opts.get(b'stat')
         m = scmutil.match(repo[node1], files, opts)
         logcmdutil.diffordiffstat(
-            self.ui, repo, diffopts, node1, node2, m, changes, stat, fp
+            self.ui,
+            repo,
+            diffopts,
+            repo[node1],
+            repo[node2],
+            m,
+            changes,
+            stat,
+            fp,
         )
 
     def mergeone(self, repo, mergeq, head, patch, rev, diffopts):

hgext/narrow/narrowbundle2.py

@@ -20,6 +20,7 b' from mercurial import ('
     localrepo,
     narrowspec,
     repair,
+    scmutil,
     util,
     wireprototypes,
 )
@@ -179,7 +180,7 b' def _handlechangespec_2(op, inpart):'
 
     if not repository.NARROW_REQUIREMENT in op.repo.requirements:
         op.repo.requirements.add(repository.NARROW_REQUIREMENT)
-        op.repo._writerequirements()
+        scmutil.writereporequirements(op.repo)
     op.repo.setnarrowpats(includepats, excludepats)
     narrowspec.copytoworkingcopy(op.repo)
 
@@ -195,7 +196,7 b' def _handlenarrowspecs(op, inpart):'
 
     if repository.NARROW_REQUIREMENT not in op.repo.requirements:
         op.repo.requirements.add(repository.NARROW_REQUIREMENT)
-        op.repo._writerequirements()
+        scmutil.writereporequirements(op.repo)
     op.repo.setnarrowpats(includepats, excludepats)
     narrowspec.copytoworkingcopy(op.repo)
 

hgext/phabricator.py

@@ -238,8 +238,8 b' def vcrcommand(name, flags, spec, helpca'
 
     def decorate(fn):
         def inner(*args, **kwargs):
-            cassette = pycompat.fsdecode(kwargs.pop('test_vcr', None))
-            if cassette:
+            if kwargs.get('test_vcr'):
+                cassette = pycompat.fsdecode(kwargs.pop('test_vcr'))
                 import hgdemandimport
 
                 with hgdemandimport.deactivated():
@@ -1311,8 +1311,8 b' def phabsend(ui, repo, *revs, **opts):'
     # --fold option implies this, and the auto restacking of orphans requires
     # it.  Otherwise A+C in A->B->C will cause B to be orphaned, and C' to
     # get A' as a parent.
-    def _fail_nonlinear_revs(revs, skiprev, revtype):
-        badnodes = [repo[r].node() for r in revs if r != skiprev]
+    def _fail_nonlinear_revs(revs, revtype):
+        badnodes = [repo[r].node() for r in revs]
         raise error.Abort(
             _(b"cannot phabsend multiple %s revisions: %s")
             % (revtype, scmutil.nodesummaries(repo, badnodes)),
@@ -1321,11 +1321,11 b' def phabsend(ui, repo, *revs, **opts):'
 
     heads = repo.revs(b'heads(%ld)', revs)
     if len(heads) > 1:
-        _fail_nonlinear_revs(heads, heads.max(), b"head")
+        _fail_nonlinear_revs(heads, b"head")
 
     roots = repo.revs(b'roots(%ld)', revs)
     if len(roots) > 1:
-        _fail_nonlinear_revs(roots, roots.min(), b"root")
+        _fail_nonlinear_revs(roots, b"root")
 
     fold = opts.get(b'fold')
     if fold:
@@ -1650,7 +1650,7 b' def _confirmbeforesend(repo, revs, oldma'
         )
 
     if ui.promptchoice(
-        _(b'Send the above changes to %s (yn)?$$ &Yes $$ &No') % url
+        _(b'Send the above changes to %s (Y/n)?$$ &Yes $$ &No') % url
     ):
         return False
 
@@ -2162,8 +2162,14 b' def phabimport(ui, repo, *specs, **opts)'
     [
         (b'', b'accept', False, _(b'accept revisions')),
         (b'', b'reject', False, _(b'reject revisions')),
+        (b'', b'request-review', False, _(b'request review on revisions')),
         (b'', b'abandon', False, _(b'abandon revisions')),
         (b'', b'reclaim', False, _(b'reclaim revisions')),
+        (b'', b'close', False, _(b'close revisions')),
+        (b'', b'reopen', False, _(b'reopen revisions')),
+        (b'', b'plan-changes', False, _(b'plan changes for revisions')),
+        (b'', b'resign', False, _(b'resign as a reviewer from revisions')),
+        (b'', b'commandeer', False, _(b'commandeer revisions')),
         (b'm', b'comment', b'', _(b'comment on the last revision')),
     ],
     _(b'DREVSPEC... [OPTIONS]'),
@@ -2176,7 +2182,19 b' def phabupdate(ui, repo, *specs, **opts)'
     DREVSPEC selects revisions. See :hg:`help phabread` for its usage.
     """
     opts = pycompat.byteskwargs(opts)
-    flags = [n for n in b'accept reject abandon reclaim'.split() if opts.get(n)]
+    transactions = [
+        b'abandon',
+        b'accept',
+        b'close',
+        b'commandeer',
+        b'plan-changes',
+        b'reclaim',
+        b'reject',
+        b'reopen',
+        b'request-review',
+        b'resign',
+    ]
+    flags = [n for n in transactions if opts.get(n.replace(b'-', b'_'))]
     if len(flags) > 1:
         raise error.Abort(_(b'%s cannot be used together') % b', '.join(flags))
 

hgext/purge.py

@@ -64,7 +64,7 b" testedwith = b'ships-with-hg-core'"
     ]
     + cmdutil.walkopts,
     _(b'hg purge [OPTION]... [DIR]...'),
-    helpcategory=command.CATEGORY_MAINTENANCE,
+    helpcategory=command.CATEGORY_WORKING_DIRECTORY,
 )
 def purge(ui, repo, *dirs, **opts):
     '''removes files not tracked by Mercurial

hgext/rebase.py

@@ -36,6 +36,7 b' from mercurial import ('
     extensions,
     hg,
     merge as mergemod,
+    mergestate as mergestatemod,
     mergeutil,
     node as nodemod,
     obsolete,
@@ -205,6 +206,9 b' class rebaseruntime(object):'
         self.backupf = ui.configbool(b'rewrite', b'backup-bundle')
         self.keepf = opts.get(b'keep', False)
         self.keepbranchesf = opts.get(b'keepbranches', False)
+        self.skipemptysuccessorf = rewriteutil.skip_empty_successor(
+            repo.ui, b'rebase'
+        )
         self.obsoletenotrebased = {}
         self.obsoletewithoutsuccessorindestination = set()
         self.inmemory = inmemory
@@ -528,11 +532,11 b' class rebaseruntime(object):'
         extra = {b'rebase_source': ctx.hex()}
         for c in self.extrafns:
             c(ctx, extra)
-        keepbranch = self.keepbranchesf and repo[p1].branch() != ctx.branch()
         destphase = max(ctx.phase(), phases.draft)
-        overrides = {(b'phases', b'new-commit'): destphase}
-        if keepbranch:
-            overrides[(b'ui', b'allowemptycommit')] = True
+        overrides = {
+            (b'phases', b'new-commit'): destphase,
+            (b'ui', b'allowemptycommit'): not self.skipemptysuccessorf,
+        }
         with repo.ui.configoverride(overrides, b'rebase'):
             if self.inmemory:
                 newnode = commitmemorynode(
@@ -544,7 +548,7 b' class rebaseruntime(object):'
                     user=ctx.user(),
                     date=date,
                 )
-                mergemod.mergestate.clean(repo)
+                mergestatemod.mergestate.clean(repo)
             else:
                 newnode = commitnode(
                     repo,
@@ -626,12 +630,7 b' class rebaseruntime(object):'
                         if self.inmemory:
                             raise error.InMemoryMergeConflictsError()
                         else:
-                            raise error.InterventionRequired(
-                                _(
-                                    b'unresolved conflicts (see hg '
-                                    b'resolve, then hg rebase --continue)'
-                                )
-                            )
+                            raise error.ConflictResolutionRequired(b'rebase')
             if not self.collapsef:
                 merging = p2 != nullrev
                 editform = cmdutil.mergeeditform(merging, b'rebase')
@@ -652,6 +651,14 b' class rebaseruntime(object):'
             if newnode is not None:
                 self.state[rev] = repo[newnode].rev()
                 ui.debug(b'rebased as %s\n' % short(newnode))
+                if repo[newnode].isempty():
+                    ui.warn(
+                        _(
+                            b'note: created empty successor for %s, its '
+                            b'destination already has all its changes\n'
+                        )
+                        % desc
+                    )
             else:
                 if not self.collapsef:
                     ui.warn(
@@ -1084,7 +1091,7 b' def rebase(ui, repo, **opts):'
             )
             # TODO: Make in-memory merge not use the on-disk merge state, so
             # we don't have to clean it here
-            mergemod.mergestate.clean(repo)
+            mergestatemod.mergestate.clean(repo)
             clearstatus(repo)
             clearcollapsemsg(repo)
             return _dorebase(ui, repo, action, opts, inmemory=False)
@@ -1191,7 +1198,7 b' def _origrebase('
             if action == b'abort' and opts.get(b'tool', False):
                 ui.warn(_(b'tool option will be ignored\n'))
             if action == b'continue':
-                ms = mergemod.mergestate.read(repo)
+                ms = mergestatemod.mergestate.read(repo)
                 mergeutil.checkunresolved(ms)
 
             retcode = rbsrt._prepareabortorcontinue(
@@ -1429,10 +1436,6 b' def externalparent(repo, state, destance'
 def commitmemorynode(repo, wctx, editor, extra, user, date, commitmsg):
     '''Commit the memory changes with parents p1 and p2.
     Return node of committed revision.'''
-    # Replicates the empty check in ``repo.commit``.
-    if wctx.isempty() and not repo.ui.configbool(b'ui', b'allowemptycommit'):
-        return None
-
     # By convention, ``extra['branch']`` (set by extrafn) clobbers
     # ``branch`` (used when passing ``--keepbranches``).
     branch = None
@@ -1447,6 +1450,8 b' def commitmemorynode(repo, wctx, editor,'
         branch=branch,
         editor=editor,
     )
+    if memctx.isempty() and not repo.ui.configbool(b'ui', b'allowemptycommit'):
+        return None
     commitres = repo.commitctx(memctx)
     wctx.clean()  # Might be reused
     return commitres
@@ -2201,7 +2206,7 b' def abortrebase(ui, repo):'
 def continuerebase(ui, repo):
     with repo.wlock(), repo.lock():
         rbsrt = rebaseruntime(repo, ui)
-        ms = mergemod.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
         mergeutil.checkunresolved(ms)
         retcode = rbsrt._prepareabortorcontinue(isabort=False)
         if retcode is not None:

hgext/releasenotes.py

@@ -30,7 +30,10 b' from mercurial import ('
     scmutil,
     util,
 )
-from mercurial.utils import stringutil
+from mercurial.utils import (
+    procutil,
+    stringutil,
+)
 
 cmdtable = {}
 command = registrar.command(cmdtable)
@@ -689,7 +692,7 b' def releasenotes(ui, repo, file_=None, *'
 def debugparsereleasenotes(ui, path, repo=None):
     """parse release notes and print resulting data structure"""
     if path == b'-':
-        text = pycompat.stdin.read()
+        text = procutil.stdin.read()
     else:
         with open(path, b'rb') as fh:
             text = fh.read()

hgext/remotefilelog/__init__.py

@@ -148,7 +148,7 b' from mercurial import ('
     extensions,
     hg,
     localrepo,
-    match,
+    match as matchmod,
     merge,
     node as nodemod,
     patch,
@@ -361,7 +361,7 b' def cloneshallow(orig, ui, repo, *args, '
                         self.unfiltered().__class__,
                     )
                 self.requirements.add(constants.SHALLOWREPO_REQUIREMENT)
-                self._writerequirements()
+                scmutil.writereporequirements(self)
 
                 # Since setupclient hadn't been called, exchange.pull was not
                 # wrapped. So we need to manually invoke our version of it.
@@ -824,12 +824,12 b' def filelogrevset(orig, repo, subset, x)'
 
     # i18n: "filelog" is a keyword
     pat = revset.getstring(x, _(b"filelog requires a pattern"))
-    m = match.match(
+    m = matchmod.match(
         repo.root, repo.getcwd(), [pat], default=b'relpath', ctx=repo[None]
     )
     s = set()
 
-    if not match.patkind(pat):
+    if not matchmod.patkind(pat):
         # slow
         for r in subset:
             ctx = repo[r]
@@ -1118,10 +1118,10 b' def exchangepull(orig, repo, remote, *ar'
     return orig(repo, remote, *args, **kwargs)
 
 
-def _fileprefetchhook(repo, revs, match):
+def _fileprefetchhook(repo, revmatches):
     if isenabled(repo):
         allfiles = []
-        for rev in revs:
+        for rev, match in revmatches:
             if rev == nodemod.wdirrev or rev is None:
                 continue
             ctx = repo[rev]

hgext/strip.py

@@ -13,7 +13,7 b' from mercurial import ('
     error,
     hg,
     lock as lockmod,
-    merge,
+    mergestate as mergestatemod,
     node as nodemod,
     pycompat,
     registrar,
@@ -269,7 +269,7 b' def stripcmd(ui, repo, *revs, **opts):'
             repo.dirstate.write(repo.currenttransaction())
 
             # clear resolve state
-            merge.mergestate.clean(repo, repo[b'.'].node())
+            mergestatemod.mergestate.clean(repo, repo[b'.'].node())
 
             update = False
 

mercurial/archival.py

@@ -369,7 +369,7 b' def archive('
     if total:
         files.sort()
         scmutil.prefetchfiles(
-            repo, [ctx.rev()], scmutil.matchfiles(repo, files)
+            repo, [(ctx.rev(), scmutil.matchfiles(repo, files))]
         )
         progress = repo.ui.makeprogress(
             _(b'archiving'), unit=_(b'files'), total=total

mercurial/bundle2.py

@@ -166,6 +166,7 b' from . import ('
     phases,
     pushkey,
     pycompat,
+    scmutil,
     streamclone,
     tags,
     url,
@@ -1710,7 +1711,7 b' def _addpartsfromopts(ui, repo, bundler,'
                 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
             )
         if opts.get(b'phases') and repo.revs(
-            b'%ln and secret()', outgoing.missingheads
+            b'%ln and secret()', outgoing.ancestorsof
         ):
             part.addparam(
                 b'targetphase', b'%d' % phases.secret, mandatory=False
@@ -1752,7 +1753,7 b' def addparttagsfnodescache(repo, bundler'
     # consume little memory (1M heads is 40MB) b) we don't want to send the
     # part if we don't have entries and knowing if we have entries requires
     # cache lookups.
-    for node in outgoing.missingheads:
+    for node in outgoing.ancestorsof:
         # Don't compute missing, as this may slow down serving.
         fnode = cache.getfnode(node, computemissing=False)
         if fnode is not None:
@@ -1977,7 +1978,7 b' def handlechangegroup(op, inpart):'
         op.repo.svfs.options = localrepo.resolvestorevfsoptions(
             op.repo.ui, op.repo.requirements, op.repo.features
         )
-        op.repo._writerequirements()
+        scmutil.writereporequirements(op.repo)
 
     bundlesidedata = bool(b'exp-sidedata' in inpart.params)
     reposidedata = bool(b'exp-sidedata-flag' in op.repo.requirements)
@@ -2207,7 +2208,7 b' def handlecheckphases(op, inpart):'
         b'remote repository changed while pushing - please try again '
         b'(%s is %s expected %s)'
     )
-    for expectedphase, nodes in enumerate(phasetonodes):
+    for expectedphase, nodes in pycompat.iteritems(phasetonodes):
         for n in nodes:
             actualphase = phasecache.phase(unfi, cl.rev(n))
             if actualphase != expectedphase:

mercurial/cext/manifest.c

@@ -49,23 +49,35 b' static Py_ssize_t pathlen(line *l)'
 }
 
 /* get the node value of a single line */
-static PyObject *nodeof(line *l)
+static PyObject *nodeof(line *l, char *flag)
 {
 	char *s = l->start;
 	Py_ssize_t llen = pathlen(l);
 	Py_ssize_t hlen = l->len - llen - 2;
-	Py_ssize_t hlen_raw = 20;
+	Py_ssize_t hlen_raw;
 	PyObject *hash;
 	if (llen + 1 + 40 + 1 > l->len) { /* path '\0' hash '\n' */
 		PyErr_SetString(PyExc_ValueError, "manifest line too short");
 		return NULL;
 	}
+	/* Detect flags after the hash first. */
+	switch (s[llen + hlen]) {
+	case 'l':
+	case 't':
+	case 'x':
+		*flag = s[llen + hlen];
+		--hlen;
+		break;
+	default:
+		*flag = '\0';
+		break;
+	}
+
 	switch (hlen) {
 	case 40: /* sha1 */
-	case 41: /* sha1 with cruft for a merge */
+		hlen_raw = 20;
 		break;
 	case 64: /* new hash */
-	case 65: /* new hash with cruft for a merge */
 		hlen_raw = 32;
 		break;
 	default:
@@ -89,24 +101,14 b' static PyObject *nodeof(line *l)'
 /* get the node hash and flags of a line as a tuple */
 static PyObject *hashflags(line *l)
 {
-	char *s = l->start;
-	Py_ssize_t plen = pathlen(l);
-	PyObject *hash = nodeof(l);
-	ssize_t hlen;
-	Py_ssize_t hplen, flen;
+	char flag;
+	PyObject *hash = nodeof(l, &flag);
 	PyObject *flags;
 	PyObject *tup;
 
 	if (!hash)
 		return NULL;
-	/* hash is either 20 or 21 bytes for an old hash, so we use a
-	   ternary here to get the "real" hexlified sha length. */
-	hlen = PyBytes_GET_SIZE(hash) < 22 ? 40 : 64;
-	/* 1 for null byte, 1 for newline */
-	hplen = plen + hlen + 2;
-	flen = l->len - hplen;
-
-	flags = PyBytes_FromStringAndSize(s + hplen - 1, flen);
+	flags = PyBytes_FromStringAndSize(&flag, flag ? 1 : 0);
 	if (!flags) {
 		Py_DECREF(hash);
 		return NULL;
@@ -291,7 +293,7 b' static PyObject *lmiter_iterentriesnext('
 {
 	Py_ssize_t pl;
 	line *l;
-	Py_ssize_t consumed;
+	char flag;
 	PyObject *ret = NULL, *path = NULL, *hash = NULL, *flags = NULL;
 	l = lmiter_nextline((lmIter *)o);
 	if (!l) {
@@ -299,13 +301,11 b' static PyObject *lmiter_iterentriesnext('
 	}
 	pl = pathlen(l);
 	path = PyBytes_FromStringAndSize(l->start, pl);
-	hash = nodeof(l);
+	hash = nodeof(l, &flag);
 	if (!path || !hash) {
 		goto done;
 	}
-	consumed = pl + 41;
-	flags = PyBytes_FromStringAndSize(l->start + consumed,
-					   l->len - consumed - 1);
+	flags = PyBytes_FromStringAndSize(&flag, flag ? 1 : 0);
 	if (!flags) {
 		goto done;
 	}
@@ -568,19 +568,13 b' static int lazymanifest_setitem('
 	pyhash = PyTuple_GetItem(value, 0);
 	if (!PyBytes_Check(pyhash)) {
 		PyErr_Format(PyExc_TypeError,
-			     "node must be a 20-byte string");
+			     "node must be a 20 or 32 bytes string");
 		return -1;
 	}
 	hlen = PyBytes_Size(pyhash);
-	/* Some parts of the codebase try and set 21 or 22
-	 * byte "hash" values in order to perturb things for
-	 * status. We have to preserve at least the 21st
-	 * byte. Sigh. If there's a 22nd byte, we drop it on
-	 * the floor, which works fine.
-	 */
-	if (hlen != 20 && hlen != 21 && hlen != 22) {
+	if (hlen != 20 && hlen != 32) {
 		PyErr_Format(PyExc_TypeError,
-			     "node must be a 20-byte string");
+			     "node must be a 20 or 32 bytes string");
 		return -1;
 	}
 	hash = PyBytes_AsString(pyhash);
@@ -588,28 +582,39 b' static int lazymanifest_setitem('
 	pyflags = PyTuple_GetItem(value, 1);
 	if (!PyBytes_Check(pyflags) || PyBytes_Size(pyflags) > 1) {
 		PyErr_Format(PyExc_TypeError,
-			     "flags must a 0 or 1 byte string");
+			     "flags must a 0 or 1 bytes string");
 		return -1;
 	}
 	if (PyBytes_AsStringAndSize(pyflags, &flags, &flen) == -1) {
 		return -1;
 	}
+	if (flen == 1) {
+		switch (*flags) {
+		case 'l':
+		case 't':
+		case 'x':
+			break;
+		default:
+			PyErr_Format(PyExc_TypeError, "invalid manifest flag");
+			return -1;
+		}
+	}
 	/* one null byte and one newline */
-	dlen = plen + 41 + flen + 1;
+	dlen = plen + hlen * 2 + 1 + flen + 1;
 	dest = malloc(dlen);
 	if (!dest) {
 		PyErr_NoMemory();
 		return -1;
 	}
 	memcpy(dest, path, plen + 1);
-	for (i = 0; i < 20; i++) {
+	for (i = 0; i < hlen; i++) {
 		/* Cast to unsigned, so it will not get sign-extended when promoted
 		 * to int (as is done when passing to a variadic function)
 		 */
 		sprintf(dest + plen + 1 + (i * 2), "%02x", (unsigned char)hash[i]);
 	}
-	memcpy(dest + plen + 41, flags, flen);
-	dest[plen + 41 + flen] = '\n';
+	memcpy(dest + plen + 2 * hlen + 1, flags, flen);
+	dest[plen + 2 * hlen + 1 + flen] = '\n';
 	new.start = dest;
 	new.len = dlen;
 	new.hash_suffix = '\0';

mercurial/cext/osutil.c

@@ -336,7 +336,7 b' static PyObject *makestat(const struct s'
 static PyObject *_listdir_stat(char *path, int pathlen, int keepstat,
 			       char *skip)
 {
-	PyObject *list, *elem, *stat = NULL, *ret = NULL;
+	PyObject *list, *elem, *ret = NULL;
 	char fullpath[PATH_MAX + 10];
 	int kind, err;
 	struct stat st;
@@ -409,7 +409,7 b' static PyObject *_listdir_stat(char *pat'
 		}
 
 		if (keepstat) {
-			stat = makestat(&st);
+			PyObject *stat = makestat(&st);
 			if (!stat)
 				goto error;
 			elem = Py_BuildValue(PY23("siN", "yiN"), ent->d_name,
@@ -419,7 +419,6 b' static PyObject *_listdir_stat(char *pat'
 					     kind);
 		if (!elem)
 			goto error;
-		stat = NULL;
 
 		PyList_Append(list, elem);
 		Py_DECREF(elem);
@@ -430,7 +429,6 b' static PyObject *_listdir_stat(char *pat'
 
 error:
 	Py_DECREF(list);
-	Py_XDECREF(stat);
 error_list:
 	closedir(dir);
 	/* closedir also closes its dirfd */
@@ -480,7 +478,7 b' int attrkind(attrbuf_entry *entry)'
 static PyObject *_listdir_batch(char *path, int pathlen, int keepstat,
 				char *skip, bool *fallback)
 {
-	PyObject *list, *elem, *stat = NULL, *ret = NULL;
+	PyObject *list, *elem, *ret = NULL;
 	int kind, err;
 	unsigned long index;
 	unsigned int count, old_state, new_state;
@@ -586,6 +584,7 b' static PyObject *_listdir_batch(char *pa'
 			}
 
 			if (keepstat) {
+				PyObject *stat = NULL;
 				/* from the getattrlist(2) man page: "Only the
 				   permission bits ... are valid". */
 				st.st_mode = (entry->access_mask & ~S_IFMT) | kind;
@@ -601,7 +600,6 b' static PyObject *_listdir_batch(char *pa'
 						     filename, kind);
 			if (!elem)
 				goto error;
-			stat = NULL;
 
 			PyList_Append(list, elem);
 			Py_DECREF(elem);
@@ -615,7 +613,6 b' static PyObject *_listdir_batch(char *pa'
 
 error:
 	Py_DECREF(list);
-	Py_XDECREF(stat);
 error_dir:
 	close(dfd);
 error_value:

mercurial/cext/parsers.c

@@ -667,7 +667,7 b' void dirs_module_init(PyObject *mod);'
 void manifest_module_init(PyObject *mod);
 void revlog_module_init(PyObject *mod);
 
-static const int version = 16;
+static const int version = 17;
 
 static void module_init(PyObject *mod)
 {

mercurial/cext/revlog.c

@@ -109,6 +109,9 b' static const Py_ssize_t nullrev = -1;'
 
 static Py_ssize_t inline_scan(indexObject *self, const char **offsets);
 
+static int index_find_node(indexObject *self, const char *node,
+                           Py_ssize_t nodelen);
+
 #if LONG_MAX == 0x7fffffffL
 static const char *const tuple_format = PY23("Kiiiiiis#", "Kiiiiiiy#");
 #else
@@ -577,34 +580,6 b' static int check_filter(PyObject *filter'
 	}
 }
 
-static Py_ssize_t add_roots_get_min(indexObject *self, PyObject *list,
-                                    Py_ssize_t marker, char *phases)
-{
-	PyObject *iter = NULL;
-	PyObject *iter_item = NULL;
-	Py_ssize_t min_idx = index_length(self) + 2;
-	long iter_item_long;
-
-	if (PyList_GET_SIZE(list) != 0) {
-		iter = PyObject_GetIter(list);
-		if (iter == NULL)
-			return -2;
-		while ((iter_item = PyIter_Next(iter))) {
-			if (!pylong_to_long(iter_item, &iter_item_long)) {
-				Py_DECREF(iter_item);
-				return -2;
-			}
-			Py_DECREF(iter_item);
-			if (iter_item_long < min_idx)
-				min_idx = iter_item_long;
-			phases[iter_item_long] = (char)marker;
-		}
-		Py_DECREF(iter);
-	}
-
-	return min_idx;
-}
-
 static inline void set_phase_from_parents(char *phases, int parent_1,
                                           int parent_2, Py_ssize_t i)
 {
@@ -773,99 +748,164 b' bail:'
 	return NULL;
 }
 
+static int add_roots_get_min(indexObject *self, PyObject *roots, char *phases,
+                             char phase)
+{
+	Py_ssize_t len = index_length(self);
+	PyObject *item;
+	PyObject *iterator;
+	int rev, minrev = -1;
+	char *node;
+
+	if (!PySet_Check(roots)) {
+		PyErr_SetString(PyExc_TypeError,
+		                "roots must be a set of nodes");
+		return -2;
+	}
+	iterator = PyObject_GetIter(roots);
+	if (iterator == NULL)
+		return -2;
+	while ((item = PyIter_Next(iterator))) {
+		if (node_check(item, &node) == -1)
+			goto failed;
+		rev = index_find_node(self, node, 20);
+		/* null is implicitly public, so negative is invalid */
+		if (rev < 0 || rev >= len)
+			goto failed;
+		phases[rev] = phase;
+		if (minrev == -1 || minrev > rev)
+			minrev = rev;
+		Py_DECREF(item);
+	}
+	Py_DECREF(iterator);
+	return minrev;
+failed:
+	Py_DECREF(iterator);
+	Py_DECREF(item);
+	return -2;
+}
+
 static PyObject *compute_phases_map_sets(indexObject *self, PyObject *args)
 {
+	/* 0: public (untracked), 1: draft, 2: secret, 32: archive,
+	   96: internal */
+	static const char trackedphases[] = {1, 2, 32, 96};
 	PyObject *roots = Py_None;
-	PyObject *ret = NULL;
-	PyObject *phasessize = NULL;
-	PyObject *phaseroots = NULL;
-	PyObject *phaseset = NULL;
-	PyObject *phasessetlist = NULL;
-	PyObject *rev = NULL;
+	PyObject *phasesetsdict = NULL;
+	PyObject *phasesets[4] = {NULL, NULL, NULL, NULL};
 	Py_ssize_t len = index_length(self);
-	Py_ssize_t numphase = 0;
-	Py_ssize_t minrevallphases = 0;
-	Py_ssize_t minrevphase = 0;
-	Py_ssize_t i = 0;
 	char *phases = NULL;
-	long phase;
+	int minphaserev = -1, rev, i;
+	const int numphases = (int)(sizeof(phasesets) / sizeof(phasesets[0]));
 
 	if (!PyArg_ParseTuple(args, "O", &roots))
-		goto done;
-	if (roots == NULL || !PyList_Check(roots)) {
-		PyErr_SetString(PyExc_TypeError, "roots must be a list");
-		goto done;
+		return NULL;
+	if (roots == NULL || !PyDict_Check(roots)) {
+		PyErr_SetString(PyExc_TypeError, "roots must be a dictionary");
+		return NULL;
 	}
 
-	phases = calloc(
-	    len, 1); /* phase per rev: {0: public, 1: draft, 2: secret} */
+	phases = calloc(len, 1);
 	if (phases == NULL) {
 		PyErr_NoMemory();
-		goto done;
+		return NULL;
 	}
-	/* Put the phase information of all the roots in phases */
-	numphase = PyList_GET_SIZE(roots) + 1;
-	minrevallphases = len + 1;
-	phasessetlist = PyList_New(numphase);
-	if (phasessetlist == NULL)
-		goto done;
+
+	for (i = 0; i < numphases; ++i) {
+		PyObject *pyphase = PyInt_FromLong(trackedphases[i]);
+		PyObject *phaseroots = NULL;
+		if (pyphase == NULL)
+			goto release;
+		phaseroots = PyDict_GetItem(roots, pyphase);
+		Py_DECREF(pyphase);
+		if (phaseroots == NULL)
+			continue;
+		rev = add_roots_get_min(self, phaseroots, phases,
+		                        trackedphases[i]);
+		if (rev == -2)
+			goto release;
+		if (rev != -1 && (minphaserev == -1 || rev < minphaserev))
+			minphaserev = rev;
+	}
+
+	for (i = 0; i < numphases; ++i) {
+		phasesets[i] = PySet_New(NULL);
+		if (phasesets[i] == NULL)
+			goto release;
+	}
 
-	PyList_SET_ITEM(phasessetlist, 0, Py_None);
-	Py_INCREF(Py_None);
-
-	for (i = 0; i < numphase - 1; i++) {
-		phaseroots = PyList_GET_ITEM(roots, i);
-		phaseset = PySet_New(NULL);
-		if (phaseset == NULL)
+	if (minphaserev == -1)
+		minphaserev = len;
+	for (rev = minphaserev; rev < len; ++rev) {
+		PyObject *pyphase = NULL;
+		PyObject *pyrev = NULL;
+		int parents[2];
+		/*
+		 * The parent lookup could be skipped for phaseroots, but
+		 * phase --force would historically not recompute them
+		 * correctly, leaving descendents with a lower phase around.
+		 * As such, unconditionally recompute the phase.
+		 */
+		if (index_get_parents(self, rev, parents, (int)len - 1) < 0)
 			goto release;
-		PyList_SET_ITEM(phasessetlist, i + 1, phaseset);
-		if (!PyList_Check(phaseroots)) {
-			PyErr_SetString(PyExc_TypeError,
-			                "roots item must be a list");
+		set_phase_from_parents(phases, parents[0], parents[1], rev);
+		switch (phases[rev]) {
+		case 0:
+			continue;
+		case 1:
+			pyphase = phasesets[0];
+			break;
+		case 2:
+			pyphase = phasesets[1];
+			break;
+		case 32:
+			pyphase = phasesets[2];
+			break;
+		case 96:
+			pyphase = phasesets[3];
+			break;
+		default:
+			/* this should never happen since the phase number is
+			 * specified by this function. */
+			PyErr_SetString(PyExc_SystemError,
+			                "bad phase number in internal list");
 			goto release;
 		}
-		minrevphase =
-		    add_roots_get_min(self, phaseroots, i + 1, phases);
-		if (minrevphase == -2) /* Error from add_roots_get_min */
+		pyrev = PyInt_FromLong(rev);
+		if (pyrev == NULL)
 			goto release;
-		minrevallphases = MIN(minrevallphases, minrevphase);
-	}
-	/* Propagate the phase information from the roots to the revs */
-	if (minrevallphases != -1) {
-		int parents[2];
-		for (i = minrevallphases; i < len; i++) {
-			if (index_get_parents(self, i, parents, (int)len - 1) <
-			    0)
-				goto release;
-			set_phase_from_parents(phases, parents[0], parents[1],
-			                       i);
+		if (PySet_Add(pyphase, pyrev) == -1) {
+			Py_DECREF(pyrev);
+			goto release;
 		}
+		Py_DECREF(pyrev);
 	}
-	/* Transform phase list to a python list */
-	phasessize = PyInt_FromSsize_t(len);
-	if (phasessize == NULL)
+
+	phasesetsdict = _dict_new_presized(numphases);
+	if (phasesetsdict == NULL)
 		goto release;
-	for (i = 0; i < len; i++) {
-		phase = phases[i];
-		/* We only store the sets of phase for non public phase, the
-		 * public phase is computed as a difference */
-		if (phase != 0) {
-			phaseset = PyList_GET_ITEM(phasessetlist, phase);
-			rev = PyInt_FromSsize_t(i);
-			if (rev == NULL)
-				goto release;
-			PySet_Add(phaseset, rev);
-			Py_XDECREF(rev);
+	for (i = 0; i < numphases; ++i) {
+		PyObject *pyphase = PyInt_FromLong(trackedphases[i]);
+		if (pyphase == NULL)
+			goto release;
+		if (PyDict_SetItem(phasesetsdict, pyphase, phasesets[i]) ==
+		    -1) {
+			Py_DECREF(pyphase);
+			goto release;
 		}
+		Py_DECREF(phasesets[i]);
+		phasesets[i] = NULL;
 	}
-	ret = PyTuple_Pack(2, phasessize, phasessetlist);
+
+	return Py_BuildValue("nN", len, phasesetsdict);
 
 release:
-	Py_XDECREF(phasessize);
-	Py_XDECREF(phasessetlist);
-done:
+	for (i = 0; i < numphases; ++i)
+		Py_XDECREF(phasesets[i]);
+	Py_XDECREF(phasesetsdict);
+
 	free(phases);
-	return ret;
+	return NULL;
 }
 
 static PyObject *index_headrevs(indexObject *self, PyObject *args)
@@ -2847,7 +2887,7 b' PyTypeObject HgRevlogIndex_Type = {'
  */
 PyObject *parse_index2(PyObject *self, PyObject *args)
 {
-	PyObject *tuple = NULL, *cache = NULL;
+	PyObject *cache = NULL;
 	indexObject *idx;
 	int ret;
 
@@ -2868,15 +2908,11 b' PyObject *parse_index2(PyObject *self, P'
 		Py_INCREF(cache);
 	}
 
-	tuple = Py_BuildValue("NN", idx, cache);
-	if (!tuple)
-		goto bail;
-	return tuple;
+	return Py_BuildValue("NN", idx, cache);
 
 bail:
 	Py_XDECREF(idx);
 	Py_XDECREF(cache);
-	Py_XDECREF(tuple);
 	return NULL;
 }
 

mercurial/changegroup.py

@@ -1629,7 +1629,7 b' def makestream('
     repo = repo.unfiltered()
     commonrevs = outgoing.common
     csets = outgoing.missing
-    heads = outgoing.missingheads
+    heads = outgoing.ancestorsof
     # We go through the fast path if we get told to, or if all (unfiltered
     # heads have been requested (since we then know there all linkrevs will
     # be pulled by the client).

mercurial/changelog.py

@@ -16,9 +16,9 b' from .node import ('
 from .thirdparty import attr
 
 from . import (
-    copies,
     encoding,
     error,
+    metadata,
     pycompat,
     revlog,
 )
@@ -318,7 +318,7 b' class changelogrevision(object):'
             rawindices = self.extra.get(b'filesadded')
         if rawindices is None:
             return None
-        return copies.decodefileindices(self.files, rawindices)
+        return metadata.decodefileindices(self.files, rawindices)
 
     @property
     def filesremoved(self):
@@ -330,7 +330,7 b' class changelogrevision(object):'
             rawindices = self.extra.get(b'filesremoved')
         if rawindices is None:
             return None
-        return copies.decodefileindices(self.files, rawindices)
+        return metadata.decodefileindices(self.files, rawindices)
 
     @property
     def p1copies(self):
@@ -342,7 +342,7 b' class changelogrevision(object):'
             rawcopies = self.extra.get(b'p1copies')
         if rawcopies is None:
             return None
-        return copies.decodecopies(self.files, rawcopies)
+        return metadata.decodecopies(self.files, rawcopies)
 
     @property
     def p2copies(self):
@@ -354,7 +354,7 b' class changelogrevision(object):'
             rawcopies = self.extra.get(b'p2copies')
         if rawcopies is None:
             return None
-        return copies.decodecopies(self.files, rawcopies)
+        return metadata.decodecopies(self.files, rawcopies)
 
     @property
     def description(self):
@@ -385,9 +385,7 b' class changelog(revlog.revlog):'
             datafile=datafile,
             checkambig=True,
             mmaplargeindex=True,
-            persistentnodemap=opener.options.get(
-                b'exp-persistent-nodemap', False
-            ),
+            persistentnodemap=opener.options.get(b'persistent-nodemap', False),
         )
 
         if self._initempty and (self.version & 0xFFFF == revlog.REVLOGV1):
@@ -572,13 +570,13 b' class changelog(revlog.revlog):'
             ):
                 extra.pop(name, None)
         if p1copies is not None:
-            p1copies = copies.encodecopies(sortedfiles, p1copies)
+            p1copies = metadata.encodecopies(sortedfiles, p1copies)
         if p2copies is not None:
-            p2copies = copies.encodecopies(sortedfiles, p2copies)
+            p2copies = metadata.encodecopies(sortedfiles, p2copies)
         if filesadded is not None:
-            filesadded = copies.encodefileindices(sortedfiles, filesadded)
+            filesadded = metadata.encodefileindices(sortedfiles, filesadded)
         if filesremoved is not None:
-            filesremoved = copies.encodefileindices(sortedfiles, filesremoved)
+            filesremoved = metadata.encodefileindices(sortedfiles, filesremoved)
         if self._copiesstorage == b'extra':
             extrasentries = p1copies, p2copies, filesadded, filesremoved
             if extra is None and any(x is not None for x in extrasentries):

mercurial/chgserver.py

@@ -320,7 +320,7 b' class channeledsystem(object):'
         self.channel = channel
 
     def __call__(self, cmd, environ, cwd=None, type=b'system', cmdtable=None):
-        args = [type, procutil.quotecommand(cmd), os.path.abspath(cwd or b'.')]
+        args = [type, cmd, os.path.abspath(cwd or b'.')]
         args.extend(b'%s=%s' % (k, v) for k, v in pycompat.iteritems(environ))
         data = b'\0'.join(args)
         self.out.write(struct.pack(b'>cI', self.channel, len(data)))
@@ -442,7 +442,20 b' class chgcmdserver(commandserver.server)'
             if newfp is not fp:
                 newfp.close()
             # restore original fd: fp is open again
-            os.dup2(fd, fp.fileno())
+            try:
+                os.dup2(fd, fp.fileno())
+            except OSError as err:
+                # According to issue6330, running chg on heavy loaded systems
+                # can lead to EBUSY. [man dup2] indicates that, on Linux,
+                # EBUSY comes from a race condition between open() and dup2().
+                # However it's not clear why open() race occurred for
+                # newfd=stdin/out/err.
+                self.ui.log(
+                    b'chgserver',
+                    b'got %s while duplicating %s\n',
+                    stringutil.forcebytestr(err),
+                    fn,
+                )
             os.close(fd)
             setattr(self, cn, ch)
             setattr(ui, fn, fp)

mercurial/cmdutil.py

@@ -38,6 +38,7 b' from . import ('
     logcmdutil,
     match as matchmod,
     merge as mergemod,
+    mergestate as mergestatemod,
     mergeutil,
     obsolete,
     patch,
@@ -890,7 +891,7 b' To mark files as resolved: hg resolve -'
 def readmorestatus(repo):
     """Returns a morestatus object if the repo has unfinished state."""
     statetuple = statemod.getrepostate(repo)
-    mergestate = mergemod.mergestate.read(repo)
+    mergestate = mergestatemod.mergestate.read(repo)
     activemerge = mergestate.active()
     if not statetuple and not activemerge:
         return None
@@ -2137,7 +2138,9 b' def _prefetchchangedfiles(repo, revs, ma'
         for file in repo[rev].files():
             if not match or match(file):
                 allfiles.add(file)
-    scmutil.prefetchfiles(repo, revs, scmutil.matchfiles(repo, allfiles))
+    match = scmutil.matchfiles(repo, allfiles)
+    revmatches = [(rev, match) for rev in revs]
+    scmutil.prefetchfiles(repo, revmatches)
 
 
 def export(
@@ -2751,15 +2754,28 b' def files(ui, ctx, m, uipathfn, fm, fmt,'
     ret = 1
 
     needsfctx = ui.verbose or {b'size', b'flags'} & fm.datahint()
-    for f in ctx.matches(m):
-        fm.startitem()
-        fm.context(ctx=ctx)
-        if needsfctx:
-            fc = ctx[f]
-            fm.write(b'size flags', b'% 10d % 1s ', fc.size(), fc.flags())
-        fm.data(path=f)
-        fm.plain(fmt % uipathfn(f))
-        ret = 0
+    if fm.isplain() and not needsfctx:
+        # Fast path. The speed-up comes from skipping the formatter, and batching
+        # calls to ui.write.
+        buf = []
+        for f in ctx.matches(m):
+            buf.append(fmt % uipathfn(f))
+            if len(buf) > 100:
+                ui.write(b''.join(buf))
+                del buf[:]
+            ret = 0
+        if buf:
+            ui.write(b''.join(buf))
+    else:
+        for f in ctx.matches(m):
+            fm.startitem()
+            fm.context(ctx=ctx)
+            if needsfctx:
+                fc = ctx[f]
+                fm.write(b'size flags', b'% 10d % 1s ', fc.size(), fc.flags())
+            fm.data(path=f)
+            fm.plain(fmt % uipathfn(f))
+            ret = 0
 
     for subpath in sorted(ctx.substate):
         submatch = matchmod.subdirmatcher(subpath, m)
@@ -2983,14 +2999,14 b' def cat(ui, repo, ctx, matcher, basefm, '
         try:
             if mfnode and mfl[mfnode].find(file)[0]:
                 if _catfmtneedsdata(basefm):
-                    scmutil.prefetchfiles(repo, [ctx.rev()], matcher)
+                    scmutil.prefetchfiles(repo, [(ctx.rev(), matcher)])
                 write(file)
                 return 0
         except KeyError:
             pass
 
     if _catfmtneedsdata(basefm):
-        scmutil.prefetchfiles(repo, [ctx.rev()], matcher)
+        scmutil.prefetchfiles(repo, [(ctx.rev(), matcher)])
 
     for abs in ctx.walk(matcher):
         write(abs)
@@ -3127,7 +3143,7 b' def amend(ui, repo, old, extra, pats, op'
             if subs:
                 subrepoutil.writestate(repo, newsubstate)
 
-        ms = mergemod.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
         mergeutil.checkunresolved(ms)
 
         filestoamend = {f for f in wctx.files() if matcher(f)}
@@ -3423,9 +3439,9 b' def commitstatus(repo, node, branch, bhe'
         not opts.get(b'amend')
         and bheads
         and node not in bheads
-        and not [
-            x for x in parents if x.node() in bheads and x.branch() == branch
-        ]
+        and not any(
+            p.node() in bheads and p.branch() == branch for p in parents
+        )
     ):
         repo.ui.status(_(b'created new head\n'))
         # The message is not printed for initial roots. For the other
@@ -3755,11 +3771,11 b' def revert(ui, repo, ctx, parents, *pats'
             needdata = (b'revert', b'add', b'undelete')
             oplist = [actions[name][0] for name in needdata]
             prefetch = scmutil.prefetchfiles
-            matchfiles = scmutil.matchfiles
+            matchfiles = scmutil.matchfiles(
+                repo, [f for sublist in oplist for f in sublist]
+            )
             prefetch(
-                repo,
-                [ctx.rev()],
-                matchfiles(repo, [f for sublist in oplist for f in sublist]),
+                repo, [(ctx.rev(), matchfiles)],
             )
             match = scmutil.match(repo[None], pats)
             _performrevert(

mercurial/commands.py

@@ -46,6 +46,7 b' from . import ('
     hg,
     logcmdutil,
     merge as mergemod,
+    mergestate as mergestatemod,
     narrowspec,
     obsolete,
     obsutil,
@@ -2183,7 +2184,8 b' def config(ui, repo, *values, **opts):'
     """
 
     opts = pycompat.byteskwargs(opts)
-    if opts.get(b'edit') or opts.get(b'local') or opts.get(b'global'):
+    editopts = (b'edit', b'local', b'global')
+    if any(opts.get(o) for o in editopts):
         if opts.get(b'local') and opts.get(b'global'):
             raise error.Abort(_(b"can't use --local and --global together"))
 
@@ -2350,7 +2352,7 b' def copy(ui, repo, *pats, **opts):'
     Returns 0 on success, 1 if errors are encountered.
     """
     opts = pycompat.byteskwargs(opts)
-    with repo.wlock(False):
+    with repo.wlock():
         return cmdutil.copy(ui, repo, pats, opts)
 
 
@@ -2475,26 +2477,27 b' def diff(ui, repo, *pats, **opts):'
     Returns 0 on success.
     """
 
+    cmdutil.check_at_most_one_arg(opts, 'rev', 'change')
     opts = pycompat.byteskwargs(opts)
     revs = opts.get(b'rev')
     change = opts.get(b'change')
     stat = opts.get(b'stat')
     reverse = opts.get(b'reverse')
 
-    if revs and change:
-        msg = _(b'cannot specify --rev and --change at the same time')
-        raise error.Abort(msg)
-    elif change:
+    if change:
         repo = scmutil.unhidehashlikerevs(repo, [change], b'nowarn')
         ctx2 = scmutil.revsingle(repo, change, None)
         ctx1 = ctx2.p1()
     else:
         repo = scmutil.unhidehashlikerevs(repo, revs, b'nowarn')
         ctx1, ctx2 = scmutil.revpair(repo, revs)
-    node1, node2 = ctx1.node(), ctx2.node()
 
     if reverse:
-        node1, node2 = node2, node1
+        ctxleft = ctx2
+        ctxright = ctx1
+    else:
+        ctxleft = ctx1
+        ctxright = ctx2
 
     diffopts = patch.diffallopts(ui, opts)
     m = scmutil.match(ctx2, pats, opts)
@@ -2504,8 +2507,8 b' def diff(ui, repo, *pats, **opts):'
         ui,
         repo,
         diffopts,
-        node1,
-        node2,
+        ctxleft,
+        ctxright,
         m,
         stat=stat,
         listsubrepos=opts.get(b'subrepos'),
@@ -2980,68 +2983,47 b' def _dograft(ui, repo, *revs, **opts):'
         editform=b'graft', **pycompat.strkwargs(opts)
     )
 
+    cmdutil.check_at_most_one_arg(opts, b'abort', b'stop', b'continue')
+
     cont = False
     if opts.get(b'no_commit'):
-        if opts.get(b'edit'):
-            raise error.Abort(
-                _(b"cannot specify --no-commit and --edit together")
-            )
-        if opts.get(b'currentuser'):
-            raise error.Abort(
-                _(b"cannot specify --no-commit and --currentuser together")
-            )
-        if opts.get(b'currentdate'):
-            raise error.Abort(
-                _(b"cannot specify --no-commit and --currentdate together")
-            )
-        if opts.get(b'log'):
-            raise error.Abort(
-                _(b"cannot specify --no-commit and --log together")
-            )
+        cmdutil.check_incompatible_arguments(
+            opts,
+            b'no_commit',
+            [b'edit', b'currentuser', b'currentdate', b'log'],
+        )
 
     graftstate = statemod.cmdstate(repo, b'graftstate')
 
     if opts.get(b'stop'):
-        if opts.get(b'continue'):
-            raise error.Abort(
-                _(b"cannot use '--continue' and '--stop' together")
-            )
-        if opts.get(b'abort'):
-            raise error.Abort(_(b"cannot use '--abort' and '--stop' together"))
-
-        if any(
-            (
-                opts.get(b'edit'),
-                opts.get(b'log'),
-                opts.get(b'user'),
-                opts.get(b'date'),
-                opts.get(b'currentdate'),
-                opts.get(b'currentuser'),
-                opts.get(b'rev'),
-            )
-        ):
-            raise error.Abort(_(b"cannot specify any other flag with '--stop'"))
+        cmdutil.check_incompatible_arguments(
+            opts,
+            b'stop',
+            [
+                b'edit',
+                b'log',
+                b'user',
+                b'date',
+                b'currentdate',
+                b'currentuser',
+                b'rev',
+            ],
+        )
         return _stopgraft(ui, repo, graftstate)
     elif opts.get(b'abort'):
-        if opts.get(b'continue'):
-            raise error.Abort(
-                _(b"cannot use '--continue' and '--abort' together")
-            )
-        if any(
-            (
-                opts.get(b'edit'),
-                opts.get(b'log'),
-                opts.get(b'user'),
-                opts.get(b'date'),
-                opts.get(b'currentdate'),
-                opts.get(b'currentuser'),
-                opts.get(b'rev'),
-            )
-        ):
-            raise error.Abort(
-                _(b"cannot specify any other flag with '--abort'")
-            )
-
+        cmdutil.check_incompatible_arguments(
+            opts,
+            b'abort',
+            [
+                b'edit',
+                b'log',
+                b'user',
+                b'date',
+                b'currentdate',
+                b'currentuser',
+                b'rev',
+            ],
+        )
         return cmdutil.abortgraft(ui, repo, graftstate)
     elif opts.get(b'continue'):
         cont = True
@@ -3431,8 +3413,11 b' def grep(ui, repo, pattern, *pats, **opt'
                 m = regexp.search(self.line, p)
                 if not m:
                     break
-                yield m.span()
-                p = m.end()
+                if m.end() == p:
+                    p += 1
+                else:
+                    yield m.span()
+                    p = m.end()
 
     matches = {}
     copies = {}
@@ -3578,56 +3563,68 b' def grep(ui, repo, pattern, *pats, **opt'
 
     getrenamed = scmutil.getrenamedfn(repo)
 
-    def get_file_content(filename, filelog, filenode, context, revision):
-        try:
-            content = filelog.read(filenode)
-        except error.WdirUnsupported:
-            content = context[filename].data()
-        except error.CensoredNodeError:
-            content = None
-            ui.warn(
-                _(b'cannot search in censored file: %(filename)s:%(revnum)s\n')
-                % {b'filename': filename, b'revnum': pycompat.bytestr(revision)}
-            )
-        return content
+    def readfile(ctx, fn):
+        rev = ctx.rev()
+        if rev is None:
+            fctx = ctx[fn]
+            try:
+                return fctx.data()
+            except IOError as e:
+                if e.errno != errno.ENOENT:
+                    raise
+        else:
+            flog = getfile(fn)
+            fnode = ctx.filenode(fn)
+            try:
+                return flog.read(fnode)
+            except error.CensoredNodeError:
+                ui.warn(
+                    _(
+                        b'cannot search in censored file: %(filename)s:%(revnum)s\n'
+                    )
+                    % {b'filename': fn, b'revnum': pycompat.bytestr(rev),}
+                )
 
     def prep(ctx, fns):
         rev = ctx.rev()
         pctx = ctx.p1()
-        parent = pctx.rev()
         matches.setdefault(rev, {})
-        matches.setdefault(parent, {})
+        if diff:
+            parent = pctx.rev()
+            matches.setdefault(parent, {})
         files = revfiles.setdefault(rev, [])
-        for fn in fns:
-            flog = getfile(fn)
-            try:
-                fnode = ctx.filenode(fn)
-            except error.LookupError:
-                continue
-
-            copy = None
-            if follow:
-                copy = getrenamed(fn, rev)
-                if copy:
-                    copies.setdefault(rev, {})[fn] = copy
-                    if fn in skip:
-                        skip.add(copy)
-            if fn in skip:
-                continue
-            files.append(fn)
-
-            if fn not in matches[rev]:
-                content = get_file_content(fn, flog, fnode, ctx, rev)
-                grepbody(fn, rev, content)
-
-            pfn = copy or fn
-            if pfn not in matches[parent]:
-                try:
-                    pfnode = pctx.filenode(pfn)
-                    pcontent = get_file_content(pfn, flog, pfnode, pctx, parent)
-                    grepbody(pfn, parent, pcontent)
-                except error.LookupError:
-                    pass
+        if rev is None:
+            # in `hg grep pattern`, 2/3 of the time is spent is spent in
+            # pathauditor checks without this in mozilla-central
+            contextmanager = repo.wvfs.audit.cached
+        else:
+            contextmanager = util.nullcontextmanager
+        with contextmanager():
+            for fn in fns:
+                # fn might not exist in the revision (could be a file removed by
+                # the revision). We could check `fn not in ctx` even when rev is
+                # None, but it's less racy to protect againt that in readfile.
+                if rev is not None and fn not in ctx:
+                    continue
+
+                copy = None
+                if follow:
+                    copy = getrenamed(fn, rev)
+                    if copy:
+                        copies.setdefault(rev, {})[fn] = copy
+                        if fn in skip:
+                            skip.add(copy)
+                if fn in skip:
+                    continue
+                files.append(fn)
+
+                if fn not in matches[rev]:
+                    grepbody(fn, rev, readfile(ctx, fn))
+
+                if diff:
+                    pfn = copy or fn
+                    if pfn not in matches[parent] and pfn in pctx:
+                        grepbody(pfn, parent, readfile(pctx, pfn))
 
     ui.pager(b'grep')
     fm = ui.formatter(b'grep', opts)
@@ -5812,7 +5809,7 b' def rename(ui, repo, *pats, **opts):'
     Returns 0 on success, 1 if errors are encountered.
     """
     opts = pycompat.byteskwargs(opts)
-    with repo.wlock(False):
+    with repo.wlock():
         return cmdutil.copy(ui, repo, pats, opts, rename=True)
 
 
@@ -5934,7 +5931,7 b' def resolve(ui, repo, *pats, **opts):'
     if show:
         ui.pager(b'resolve')
         fm = ui.formatter(b'resolve', opts)
-        ms = mergemod.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
         wctx = repo[None]
         m = scmutil.match(wctx, pats, opts)
 
@@ -5942,14 +5939,20 b' def resolve(ui, repo, *pats, **opts):'
         # as 'P'.  Resolved path conflicts show as 'R', the same as normal
         # resolved conflicts.
         mergestateinfo = {
-            mergemod.MERGE_RECORD_UNRESOLVED: (b'resolve.unresolved', b'U'),
-            mergemod.MERGE_RECORD_RESOLVED: (b'resolve.resolved', b'R'),
-            mergemod.MERGE_RECORD_UNRESOLVED_PATH: (
+            mergestatemod.MERGE_RECORD_UNRESOLVED: (
+                b'resolve.unresolved',
+                b'U',
+            ),
+            mergestatemod.MERGE_RECORD_RESOLVED: (b'resolve.resolved', b'R'),
+            mergestatemod.MERGE_RECORD_UNRESOLVED_PATH: (
                 b'resolve.unresolved',
                 b'P',
             ),
-            mergemod.MERGE_RECORD_RESOLVED_PATH: (b'resolve.resolved', b'R'),
-            mergemod.MERGE_RECORD_DRIVER_RESOLVED: (
+            mergestatemod.MERGE_RECORD_RESOLVED_PATH: (
+                b'resolve.resolved',
+                b'R',
+            ),
+            mergestatemod.MERGE_RECORD_DRIVER_RESOLVED: (
                 b'resolve.driverresolved',
                 b'D',
             ),
@@ -5959,7 +5962,7 b' def resolve(ui, repo, *pats, **opts):'
             if not m(f):
                 continue
 
-            if ms[f] == mergemod.MERGE_RECORD_MERGED_OTHER:
+            if ms[f] == mergestatemod.MERGE_RECORD_MERGED_OTHER:
                 continue
             label, key = mergestateinfo[ms[f]]
             fm.startitem()
@@ -5971,7 +5974,7 b' def resolve(ui, repo, *pats, **opts):'
         return 0
 
     with repo.wlock():
-        ms = mergemod.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
 
         if not (ms.active() or repo.dirstate.p2() != nullid):
             raise error.Abort(
@@ -5982,7 +5985,7 b' def resolve(ui, repo, *pats, **opts):'
 
         if (
             ms.mergedriver
-            and ms.mdstate() == mergemod.MERGE_DRIVER_STATE_UNMARKED
+            and ms.mdstate() == mergestatemod.MERGE_DRIVER_STATE_UNMARKED
         ):
             proceed = mergemod.driverpreprocess(repo, ms, wctx)
             ms.commit()
@@ -6008,12 +6011,12 b' def resolve(ui, repo, *pats, **opts):'
 
             didwork = True
 
-            if ms[f] == mergemod.MERGE_RECORD_MERGED_OTHER:
+            if ms[f] == mergestatemod.MERGE_RECORD_MERGED_OTHER:
                 continue
 
             # don't let driver-resolved files be marked, and run the conclude
             # step if asked to resolve
-            if ms[f] == mergemod.MERGE_RECORD_DRIVER_RESOLVED:
+            if ms[f] == mergestatemod.MERGE_RECORD_DRIVER_RESOLVED:
                 exact = m.exact(f)
                 if mark:
                     if exact:
@@ -6033,14 +6036,14 b' def resolve(ui, repo, *pats, **opts):'
 
             # path conflicts must be resolved manually
             if ms[f] in (
-                mergemod.MERGE_RECORD_UNRESOLVED_PATH,
-                mergemod.MERGE_RECORD_RESOLVED_PATH,
+                mergestatemod.MERGE_RECORD_UNRESOLVED_PATH,
+                mergestatemod.MERGE_RECORD_RESOLVED_PATH,
             ):
                 if mark:
-                    ms.mark(f, mergemod.MERGE_RECORD_RESOLVED_PATH)
+                    ms.mark(f, mergestatemod.MERGE_RECORD_RESOLVED_PATH)
                 elif unmark:
-                    ms.mark(f, mergemod.MERGE_RECORD_UNRESOLVED_PATH)
-                elif ms[f] == mergemod.MERGE_RECORD_UNRESOLVED_PATH:
+                    ms.mark(f, mergestatemod.MERGE_RECORD_UNRESOLVED_PATH)
+                elif ms[f] == mergestatemod.MERGE_RECORD_UNRESOLVED_PATH:
                     ui.warn(
                         _(b'%s: path conflict must be resolved manually\n')
                         % uipathfn(f)
@@ -6052,12 +6055,12 b' def resolve(ui, repo, *pats, **opts):'
                     fdata = repo.wvfs.tryread(f)
                     if (
                         filemerge.hasconflictmarkers(fdata)
-                        and ms[f] != mergemod.MERGE_RECORD_RESOLVED
+                        and ms[f] != mergestatemod.MERGE_RECORD_RESOLVED
                     ):
                         hasconflictmarkers.append(f)
-                ms.mark(f, mergemod.MERGE_RECORD_RESOLVED)
+                ms.mark(f, mergestatemod.MERGE_RECORD_RESOLVED)
             elif unmark:
-                ms.mark(f, mergemod.MERGE_RECORD_UNRESOLVED)
+                ms.mark(f, mergestatemod.MERGE_RECORD_UNRESOLVED)
             else:
                 # backup pre-resolve (merge uses .orig for its own purposes)
                 a = repo.wjoin(f)
@@ -6126,7 +6129,8 b' def resolve(ui, repo, *pats, **opts):'
                     raise
 
         ms.commit()
-        ms.recordactions()
+        branchmerge = repo.dirstate.p2() != nullid
+        mergestatemod.recordupdates(repo, ms.actions(), branchmerge, None)
 
         if not didwork and pats:
             hint = None
@@ -6660,7 +6664,7 b' def shelve(ui, repo, *pats, **opts):'
         (b'm', b'modified', None, _(b'show only modified files')),
         (b'a', b'added', None, _(b'show only added files')),
         (b'r', b'removed', None, _(b'show only removed files')),
-        (b'd', b'deleted', None, _(b'show only deleted (but tracked) files')),
+        (b'd', b'deleted', None, _(b'show only missing files')),
         (b'c', b'clean', None, _(b'show only files without changes')),
         (b'u', b'unknown', None, _(b'show only unknown (not tracked) files')),
         (b'i', b'ignored', None, _(b'show only ignored files')),
@@ -6791,6 +6795,7 b' def status(ui, repo, *pats, **opts):'
 
     """
 
+    cmdutil.check_at_most_one_arg(opts, 'rev', 'change')
     opts = pycompat.byteskwargs(opts)
     revs = opts.get(b'rev')
     change = opts.get(b'change')
@@ -6801,10 +6806,7 b' def status(ui, repo, *pats, **opts):'
         else:
             terse = ui.config(b'commands', b'status.terse')
 
-    if revs and change:
-        msg = _(b'cannot specify --rev and --change at the same time')
-        raise error.Abort(msg)
-    elif revs and terse:
+    if revs and terse:
         msg = _(b'cannot use --terse with --rev')
         raise error.Abort(msg)
     elif change:
@@ -6940,7 +6942,7 b' def summary(ui, repo, **opts):'
     marks = []
 
     try:
-        ms = mergemod.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
     except error.UnsupportedMergeRecords as e:
         s = b' '.join(e.recordtypes)
         ui.warn(
@@ -7809,7 +7811,7 b' def version_(ui, **opts):'
     names = []
     vers = []
     isinternals = []
-    for name, module in extensions.extensions():
+    for name, module in sorted(extensions.extensions()):
         names.append(name)
         vers.append(extensions.moduleversion(module) or None)
         isinternals.append(extensions.ismoduleinternal(module))

mercurial/commandserver.py

@@ -191,7 +191,6 b' class channeledinput(object):'
 
 
 def _selectmessageencoder(ui):
-    # experimental config: cmdserver.message-encodings
     encnames = ui.configlist(b'cmdserver', b'message-encodings')
     for n in encnames:
         f = _messageencoders.get(n)
@@ -234,9 +233,6 b' class server(object):'
             self.ui = self.ui.copy()
             setuplogging(self.ui, repo=None, fp=self.cdebug)
 
-        # TODO: add this to help/config.txt when stabilized
-        # ``channel``
-        #   Use separate channel for structured output. (Command-server only)
         self.cmsg = None
         if ui.config(b'ui', b'message-output') == b'channel':
             encname, encfn = _selectmessageencoder(ui)
@@ -244,8 +240,23 b' class server(object):'
 
         self.client = fin
 
+        # If shutdown-on-interrupt is off, the default SIGINT handler is
+        # removed so that client-server communication wouldn't be interrupted.
+        # For example, 'runcommand' handler will issue three short read()s.
+        # If one of the first two read()s were interrupted, the communication
+        # channel would be left at dirty state and the subsequent request
+        # wouldn't be parsed. So catching KeyboardInterrupt isn't enough.
+        self._shutdown_on_interrupt = ui.configbool(
+            b'cmdserver', b'shutdown-on-interrupt'
+        )
+        self._old_inthandler = None
+        if not self._shutdown_on_interrupt:
+            self._old_inthandler = signal.signal(signal.SIGINT, signal.SIG_IGN)
+
     def cleanup(self):
         """release and restore resources taken during server session"""
+        if not self._shutdown_on_interrupt:
+            signal.signal(signal.SIGINT, self._old_inthandler)
 
     def _read(self, size):
         if not size:
@@ -278,6 +289,32 b' class server(object):'
         else:
             return []
 
+    def _dispatchcommand(self, req):
+        from . import dispatch  # avoid cycle
+
+        if self._shutdown_on_interrupt:
+            # no need to restore SIGINT handler as it is unmodified.
+            return dispatch.dispatch(req)
+
+        try:
+            signal.signal(signal.SIGINT, self._old_inthandler)
+            return dispatch.dispatch(req)
+        except error.SignalInterrupt:
+            # propagate SIGBREAK, SIGHUP, or SIGTERM.
+            raise
+        except KeyboardInterrupt:
+            # SIGINT may be received out of the try-except block of dispatch(),
+            # so catch it as last ditch. Another KeyboardInterrupt may be
+            # raised while handling exceptions here, but there's no way to
+            # avoid that except for doing everything in C.
+            pass
+        finally:
+            signal.signal(signal.SIGINT, signal.SIG_IGN)
+        # On KeyboardInterrupt, print error message and exit *after* SIGINT
+        # handler removed.
+        req.ui.error(_(b'interrupted!\n'))
+        return -1
+
     def runcommand(self):
         """ reads a list of \0 terminated arguments, executes
         and writes the return code to the result channel """
@@ -318,7 +355,10 b' class server(object):'
         )
 
         try:
-            ret = dispatch.dispatch(req) & 255
+            ret = self._dispatchcommand(req) & 255
+            # If shutdown-on-interrupt is off, it's important to write the
+            # result code *after* SIGINT handler removed. If the result code
+            # were lost, the client wouldn't be able to continue processing.
             self.cresult.write(struct.pack(b'>i', int(ret)))
         finally:
             # restore old cwd

mercurial/configitems.py

@@ -204,7 +204,7 b' coreconfigitem('
     b'cmdserver', b'max-repo-cache', default=0, experimental=True,
 )
 coreconfigitem(
-    b'cmdserver', b'message-encodings', default=list, experimental=True,
+    b'cmdserver', b'message-encodings', default=list,
 )
 coreconfigitem(
     b'cmdserver',
@@ -212,6 +212,9 b' coreconfigitem('
     default=lambda: [b'chgserver', b'cmdserver', b'repocache'],
 )
 coreconfigitem(
+    b'cmdserver', b'shutdown-on-interrupt', default=True,
+)
+coreconfigitem(
     b'color', b'.*', default=None, generic=True,
 )
 coreconfigitem(
@@ -405,18 +408,6 b' coreconfigitem('
 coreconfigitem(
     b'devel', b'legacy.exchange', default=list,
 )
-# TODO before getting `persistent-nodemap` out of experimental
-#
-# * decide for a "status" of the persistent nodemap and associated location
-#   - part of the store next the revlog itself (new requirements)
-#   - part of the cache directory
-#   - part of an `index` directory
-#     (https://www.mercurial-scm.org/wiki/ComputedIndexPlan)
-# * do we want to use this for more than just changelog? if so we need:
-#   - simpler "pending" logic for them
-#   - double check the memory story (we dont want to keep all revlog in memory)
-#   - think about the naming scheme if we are in "cache"
-# * increment the version format to "1" and freeze it.
 coreconfigitem(
     b'devel', b'persistent-nodemap', default=False,
 )
@@ -675,12 +666,6 b' coreconfigitem('
     b'experimental', b'rust.index', default=False,
 )
 coreconfigitem(
-    b'experimental', b'exp-persistent-nodemap', default=False,
-)
-coreconfigitem(
-    b'experimental', b'exp-persistent-nodemap.mmap', default=True,
-)
-coreconfigitem(
     b'experimental', b'server.filesdata.recommended-batch-size', default=50000,
 )
 coreconfigitem(
@@ -783,6 +768,12 b' coreconfigitem('
 coreconfigitem(
     b'format', b'usestore', default=True,
 )
+# Right now, the only efficient implement of the nodemap logic is in Rust, so
+# the persistent nodemap feature needs to stay experimental as long as the Rust
+# extensions are an experimental feature.
+coreconfigitem(
+    b'format', b'use-persistent-nodemap', default=False, experimental=True
+)
 coreconfigitem(
     b'format',
     b'exp-use-copies-side-data-changeset',
@@ -820,9 +811,6 b' coreconfigitem('
     b'hostsecurity', b'ciphers', default=None,
 )
 coreconfigitem(
-    b'hostsecurity', b'disabletls10warning', default=False,
-)
-coreconfigitem(
     b'hostsecurity', b'minimumprotocol', default=dynamicdefault,
 )
 coreconfigitem(
@@ -1080,6 +1068,9 b' coreconfigitem('
     b'rewrite', b'update-timestamp', default=False,
 )
 coreconfigitem(
+    b'rewrite', b'empty-successor', default=b'skip', experimental=True,
+)
+coreconfigitem(
     b'storage', b'new-repo-backend', default=b'revlogv1', experimental=True,
 )
 coreconfigitem(
@@ -1088,6 +1079,14 b' coreconfigitem('
     default=True,
     alias=[(b'format', b'aggressivemergedeltas')],
 )
+# experimental as long as rust is experimental (or a C version is implemented)
+coreconfigitem(
+    b'storage', b'revlog.nodemap.mmap', default=True, experimental=True
+)
+# experimental as long as format.use-persistent-nodemap is.
+coreconfigitem(
+    b'storage', b'revlog.nodemap.mode', default=b'compat', experimental=True
+)
 coreconfigitem(
     b'storage', b'revlog.reuse-external-delta', default=True,
 )
@@ -1235,6 +1234,10 b' coreconfigitem('
     b'ui', b'askusername', default=False,
 )
 coreconfigitem(
+    b'ui', b'available-memory', default=None,
+)
+
+coreconfigitem(
     b'ui', b'clonebundlefallback', default=False,
 )
 coreconfigitem(
@@ -1391,6 +1394,9 b' coreconfigitem('
     b'ui', b'timeout.warn', default=0,
 )
 coreconfigitem(
+    b'ui', b'timestamp-output', default=False,
+)
+coreconfigitem(
     b'ui', b'traceback', default=False,
 )
 coreconfigitem(

mercurial/context.py

@@ -28,12 +28,13 b' from .pycompat import ('
     open,
 )
 from . import (
-    copies,
     dagop,
     encoding,
     error,
     fileset,
     match as matchmod,
+    mergestate as mergestatemod,
+    metadata,
     obsolete as obsmod,
     patch,
     pathutil,
@@ -299,7 +300,7 b' class basectx(object):'
 
     @propertycache
     def _copies(self):
-        return copies.computechangesetcopies(self)
+        return metadata.computechangesetcopies(self)
 
     def p1copies(self):
         return self._copies[0]
@@ -474,6 +475,20 b' class basectx(object):'
 
         return r
 
+    def mergestate(self, clean=False):
+        """Get a mergestate object for this context."""
+        raise NotImplementedError(
+            '%s does not implement mergestate()' % self.__class__
+        )
+
+    def isempty(self):
+        return not (
+            len(self.parents()) > 1
+            or self.branch() != self.p1().branch()
+            or self.closesbranch()
+            or self.files()
+        )
+
 
 class changectx(basectx):
     """A changecontext object makes access to data related to a particular
@@ -582,7 +597,7 b' class changectx(basectx):'
                 filesadded = None
         if filesadded is None:
             if compute_on_none:
-                filesadded = copies.computechangesetfilesadded(self)
+                filesadded = metadata.computechangesetfilesadded(self)
             else:
                 filesadded = []
         return filesadded
@@ -601,7 +616,7 b' class changectx(basectx):'
                 filesremoved = None
         if filesremoved is None:
             if compute_on_none:
-                filesremoved = copies.computechangesetfilesremoved(self)
+                filesremoved = metadata.computechangesetfilesremoved(self)
             else:
                 filesremoved = []
         return filesremoved
@@ -2009,6 +2024,11 b' class workingctx(committablectx):'
 
         sparse.aftercommit(self._repo, node)
 
+    def mergestate(self, clean=False):
+        if clean:
+            return mergestatemod.mergestate.clean(self._repo)
+        return mergestatemod.mergestate.read(self._repo)
+
 
 class committablefilectx(basefilectx):
     """A committablefilectx provides common functionality for a file context
@@ -2310,7 +2330,7 b' class overlayworkingctx(committablectx):'
                 return self._cache[path][b'flags']
             else:
                 raise error.ProgrammingError(
-                    b"No such file or directory: %s" % self._path
+                    b"No such file or directory: %s" % path
                 )
         else:
             return self._wrappedctx[path].flags()
@@ -2427,7 +2447,7 b' class overlayworkingctx(committablectx):'
                 return len(self._cache[path][b'data'])
             else:
                 raise error.ProgrammingError(
-                    b"No such file or directory: %s" % self._path
+                    b"No such file or directory: %s" % path
                 )
         return self._wrappedctx[path].size()
 
@@ -2507,48 +2527,9 b' class overlayworkingctx(committablectx):'
     def isdirty(self, path):
         return path in self._cache
 
-    def isempty(self):
-        # We need to discard any keys that are actually clean before the empty
-        # commit check.
-        self._compact()
-        return len(self._cache) == 0
-
     def clean(self):
         self._cache = {}
 
-    def _compact(self):
-        """Removes keys from the cache that are actually clean, by comparing
-        them with the underlying context.
-
-        This can occur during the merge process, e.g. by passing --tool :local
-        to resolve a conflict.
-        """
-        keys = []
-        # This won't be perfect, but can help performance significantly when
-        # using things like remotefilelog.
-        scmutil.prefetchfiles(
-            self.repo(),
-            [self.p1().rev()],
-            scmutil.matchfiles(self.repo(), self._cache.keys()),
-        )
-
-        for path in self._cache.keys():
-            cache = self._cache[path]
-            try:
-                underlying = self._wrappedctx[path]
-                if (
-                    underlying.data() == cache[b'data']
-                    and underlying.flags() == cache[b'flags']
-                ):
-                    keys.append(path)
-            except error.ManifestLookupError:
-                # Path not in the underlying manifest (created).
-                continue
-
-        for path in keys:
-            del self._cache[path]
-        return keys
-
     def _markdirty(
         self, path, exists, data=None, date=None, flags=b'', copied=None
     ):
@@ -2867,6 +2848,11 b' class memctx(committablectx):'
 
         return scmutil.status(modified, added, removed, [], [], [], [])
 
+    def parents(self):
+        if self._parents[1].node() == nullid:
+            return [self._parents[0]]
+        return self._parents
+
 
 class memfilectx(committablefilectx):
     """memfilectx represents an in-memory file to commit.

mercurial/copies.py

@@ -8,7 +8,6 b''
 from __future__ import absolute_import
 
 import collections
-import multiprocessing
 import os
 
 from .i18n import _
@@ -17,7 +16,6 b' from .i18n import _'
 from .revlogutils.flagutil import REVIDX_SIDEDATA
 
 from . import (
-    error,
     match as matchmod,
     node,
     pathutil,
@@ -25,7 +23,6 b' from . import ('
     util,
 )
 
-from .revlogutils import sidedata as sidedatamod
 
 from .utils import stringutil
 
@@ -183,10 +180,27 b' def _revinfogetter(repo):'
     * p1copies: mapping of copies from p1
     * p2copies: mapping of copies from p2
     * removed: a list of removed files
+    * ismerged: a callback to know if file was merged in that revision
     """
     cl = repo.changelog
     parents = cl.parentrevs
 
+    def get_ismerged(rev):
+        ctx = repo[rev]
+
+        def ismerged(path):
+            if path not in ctx.files():
+                return False
+            fctx = ctx[path]
+            parents = fctx._filelog.parents(fctx._filenode)
+            nb_parents = 0
+            for n in parents:
+                if n != node.nullid:
+                    nb_parents += 1
+            return nb_parents >= 2
+
+        return ismerged
+
     if repo.filecopiesmode == b'changeset-sidedata':
         changelogrevision = cl.changelogrevision
         flags = cl.flags
@@ -218,6 +232,7 b' def _revinfogetter(repo):'
 
         def revinfo(rev):
             p1, p2 = parents(rev)
+            value = None
             if flags(rev) & REVIDX_SIDEDATA:
                 e = merge_caches.pop(rev, None)
                 if e is not None:
@@ -228,12 +243,22 b' def _revinfogetter(repo):'
                 removed = c.filesremoved
                 if p1 != node.nullrev and p2 != node.nullrev:
                     # XXX some case we over cache, IGNORE
-                    merge_caches[rev] = (p1, p2, p1copies, p2copies, removed)
+                    value = merge_caches[rev] = (
+                        p1,
+                        p2,
+                        p1copies,
+                        p2copies,
+                        removed,
+                        get_ismerged(rev),
+                    )
             else:
                 p1copies = {}
                 p2copies = {}
                 removed = []
-            return p1, p2, p1copies, p2copies, removed
+
+            if value is None:
+                value = (p1, p2, p1copies, p2copies, removed, get_ismerged(rev))
+            return value
 
     else:
 
@@ -242,7 +267,7 b' def _revinfogetter(repo):'
             ctx = repo[rev]
             p1copies, p2copies = ctx._copies
             removed = ctx.filesremoved()
-            return p1, p2, p1copies, p2copies, removed
+            return p1, p2, p1copies, p2copies, removed, get_ismerged(rev)
 
     return revinfo
 
@@ -256,6 +281,7 b' def _changesetforwardcopies(a, b, match)'
     revinfo = _revinfogetter(repo)
 
     cl = repo.changelog
+    isancestor = cl.isancestorrev  # XXX we should had chaching to this.
     missingrevs = cl.findmissingrevs(common=[a.rev()], heads=[b.rev()])
     mrset = set(missingrevs)
     roots = set()
@@ -283,10 +309,14 b' def _changesetforwardcopies(a, b, match)'
     iterrevs.update(roots)
     iterrevs.remove(b.rev())
     revs = sorted(iterrevs)
-    return _combinechangesetcopies(revs, children, b.rev(), revinfo, match)
+    return _combinechangesetcopies(
+        revs, children, b.rev(), revinfo, match, isancestor
+    )
 
 
-def _combinechangesetcopies(revs, children, targetrev, revinfo, match):
+def _combinechangesetcopies(
+    revs, children, targetrev, revinfo, match, isancestor
+):
     """combine the copies information for each item of iterrevs
 
     revs: sorted iterable of revision to visit
@@ -305,7 +335,7 b' def _combinechangesetcopies(revs, childr'
             # this is a root
             copies = {}
         for i, c in enumerate(children[r]):
-            p1, p2, p1copies, p2copies, removed = revinfo(c)
+            p1, p2, p1copies, p2copies, removed, ismerged = revinfo(c)
             if r == p1:
                 parent = 1
                 childcopies = p1copies
@@ -319,9 +349,12 b' def _combinechangesetcopies(revs, childr'
                 }
             newcopies = copies
             if childcopies:
-                newcopies = _chain(newcopies, childcopies)
-                # _chain makes a copies, we can avoid doing so in some
-                # simple/linear cases.
+                newcopies = copies.copy()
+                for dest, source in pycompat.iteritems(childcopies):
+                    prev = copies.get(source)
+                    if prev is not None and prev[1] is not None:
+                        source = prev[1]
+                    newcopies[dest] = (c, source)
                 assert newcopies is not copies
             for f in removed:
                 if f in newcopies:
@@ -330,7 +363,7 b' def _combinechangesetcopies(revs, childr'
                         # branches.  when there are no other branches, this
                         # could be avoided.
                         newcopies = copies.copy()
-                    del newcopies[f]
+                    newcopies[f] = (c, None)
             othercopies = all_copies.get(c)
             if othercopies is None:
                 all_copies[c] = newcopies
@@ -338,21 +371,55 b' def _combinechangesetcopies(revs, childr'
                 # we are the second parent to work on c, we need to merge our
                 # work with the other.
                 #
-                # Unlike when copies are stored in the filelog, we consider
-                # it a copy even if the destination already existed on the
-                # other branch. It's simply too expensive to check if the
-                # file existed in the manifest.
-                #
                 # In case of conflict, parent 1 take precedence over parent 2.
                 # This is an arbitrary choice made anew when implementing
                 # changeset based copies. It was made without regards with
                 # potential filelog related behavior.
                 if parent == 1:
-                    othercopies.update(newcopies)
+                    _merge_copies_dict(
+                        othercopies, newcopies, isancestor, ismerged
+                    )
                 else:
-                    newcopies.update(othercopies)
+                    _merge_copies_dict(
+                        newcopies, othercopies, isancestor, ismerged
+                    )
                     all_copies[c] = newcopies
-    return all_copies[targetrev]
+
+    final_copies = {}
+    for dest, (tt, source) in all_copies[targetrev].items():
+        if source is not None:
+            final_copies[dest] = source
+    return final_copies
+
+
+def _merge_copies_dict(minor, major, isancestor, ismerged):
+    """merge two copies-mapping together, minor and major
+
+    In case of conflict, value from "major" will be picked.
+
+    - `isancestors(low_rev, high_rev)`: callable return True if `low_rev` is an
+                                        ancestors of `high_rev`,
+
+    - `ismerged(path)`: callable return True if `path` have been merged in the
+                        current revision,
+    """
+    for dest, value in major.items():
+        other = minor.get(dest)
+        if other is None:
+            minor[dest] = value
+        else:
+            new_tt = value[0]
+            other_tt = other[0]
+            if value[1] == other[1]:
+                continue
+            # content from "major" wins, unless it is older
+            # than the branch point or there is a merge
+            if (
+                new_tt == other_tt
+                or not isancestor(new_tt, other_tt)
+                or ismerged(dest)
+            ):
+                minor[dest] = value
 
 
 def _forwardcopies(a, b, base=None, match=None):
@@ -569,6 +636,12 b' class branch_copies(object):'
         self.dirmove = {} if dirmove is None else dirmove
         self.movewithdir = {} if movewithdir is None else movewithdir
 
+    def __repr__(self):
+        return (
+            '<branch_copies\n  copy=%r\n  renamedelete=%r\n  dirmove=%r\n  movewithdir=%r\n>'
+            % (self.copy, self.renamedelete, self.dirmove, self.movewithdir,)
+        )
+
 
 def _fullcopytracing(repo, c1, c2, base):
     """ The full copytracing algorithm which finds all the new files that were
@@ -922,250 +995,3 b' def graftcopies(wctx, ctx, base):'
     _filter(wctx.p1(), wctx, new_copies)
     for dst, src in pycompat.iteritems(new_copies):
         wctx[dst].markcopied(src)
-
-
-def computechangesetfilesadded(ctx):
-    """return the list of files added in a changeset
-    """
-    added = []
-    for f in ctx.files():
-        if not any(f in p for p in ctx.parents()):
-            added.append(f)
-    return added
-
-
-def computechangesetfilesremoved(ctx):
-    """return the list of files removed in a changeset
-    """
-    removed = []
-    for f in ctx.files():
-        if f not in ctx:
-            removed.append(f)
-    return removed
-
-
-def computechangesetcopies(ctx):
-    """return the copies data for a changeset
-
-    The copies data are returned as a pair of dictionnary (p1copies, p2copies).
-
-    Each dictionnary are in the form: `{newname: oldname}`
-    """
-    p1copies = {}
-    p2copies = {}
-    p1 = ctx.p1()
-    p2 = ctx.p2()
-    narrowmatch = ctx._repo.narrowmatch()
-    for dst in ctx.files():
-        if not narrowmatch(dst) or dst not in ctx:
-            continue
-        copied = ctx[dst].renamed()
-        if not copied:
-            continue
-        src, srcnode = copied
-        if src in p1 and p1[src].filenode() == srcnode:
-            p1copies[dst] = src
-        elif src in p2 and p2[src].filenode() == srcnode:
-            p2copies[dst] = src
-    return p1copies, p2copies
-
-
-def encodecopies(files, copies):
-    items = []
-    for i, dst in enumerate(files):
-        if dst in copies:
-            items.append(b'%d\0%s' % (i, copies[dst]))
-    if len(items) != len(copies):
-        raise error.ProgrammingError(
-            b'some copy targets missing from file list'
-        )
-    return b"\n".join(items)
-
-
-def decodecopies(files, data):
-    try:
-        copies = {}
-        if not data:
-            return copies
-        for l in data.split(b'\n'):
-            strindex, src = l.split(b'\0')
-            i = int(strindex)
-            dst = files[i]
-            copies[dst] = src
-        return copies
-    except (ValueError, IndexError):
-        # Perhaps someone had chosen the same key name (e.g. "p1copies") and
-        # used different syntax for the value.
-        return None
-
-
-def encodefileindices(files, subset):
-    subset = set(subset)
-    indices = []
-    for i, f in enumerate(files):
-        if f in subset:
-            indices.append(b'%d' % i)
-    return b'\n'.join(indices)
-
-
-def decodefileindices(files, data):
-    try:
-        subset = []
-        if not data:
-            return subset
-        for strindex in data.split(b'\n'):
-            i = int(strindex)
-            if i < 0 or i >= len(files):
-                return None
-            subset.append(files[i])
-        return subset
-    except (ValueError, IndexError):
-        # Perhaps someone had chosen the same key name (e.g. "added") and
-        # used different syntax for the value.
-        return None
-
-
-def _getsidedata(srcrepo, rev):
-    ctx = srcrepo[rev]
-    filescopies = computechangesetcopies(ctx)
-    filesadded = computechangesetfilesadded(ctx)
-    filesremoved = computechangesetfilesremoved(ctx)
-    sidedata = {}
-    if any([filescopies, filesadded, filesremoved]):
-        sortedfiles = sorted(ctx.files())
-        p1copies, p2copies = filescopies
-        p1copies = encodecopies(sortedfiles, p1copies)
-        p2copies = encodecopies(sortedfiles, p2copies)
-        filesadded = encodefileindices(sortedfiles, filesadded)
-        filesremoved = encodefileindices(sortedfiles, filesremoved)
-        if p1copies:
-            sidedata[sidedatamod.SD_P1COPIES] = p1copies
-        if p2copies:
-            sidedata[sidedatamod.SD_P2COPIES] = p2copies
-        if filesadded:
-            sidedata[sidedatamod.SD_FILESADDED] = filesadded
-        if filesremoved:
-            sidedata[sidedatamod.SD_FILESREMOVED] = filesremoved
-    return sidedata
-
-
-def getsidedataadder(srcrepo, destrepo):
-    use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
-    if pycompat.iswindows or not use_w:
-        return _get_simple_sidedata_adder(srcrepo, destrepo)
-    else:
-        return _get_worker_sidedata_adder(srcrepo, destrepo)
-
-
-def _sidedata_worker(srcrepo, revs_queue, sidedata_queue, tokens):
-    """The function used by worker precomputing sidedata
-
-    It read an input queue containing revision numbers
-    It write in an output queue containing (rev, <sidedata-map>)
-
-    The `None` input value is used as a stop signal.
-
-    The `tokens` semaphore is user to avoid having too many unprocessed
-    entries. The workers needs to acquire one token before fetching a task.
-    They will be released by the consumer of the produced data.
-    """
-    tokens.acquire()
-    rev = revs_queue.get()
-    while rev is not None:
-        data = _getsidedata(srcrepo, rev)
-        sidedata_queue.put((rev, data))
-        tokens.acquire()
-        rev = revs_queue.get()
-    # processing of `None` is completed, release the token.
-    tokens.release()
-
-
-BUFF_PER_WORKER = 50
-
-
-def _get_worker_sidedata_adder(srcrepo, destrepo):
-    """The parallel version of the sidedata computation
-
-    This code spawn a pool of worker that precompute a buffer of sidedata
-    before we actually need them"""
-    # avoid circular import copies -> scmutil -> worker -> copies
-    from . import worker
-
-    nbworkers = worker._numworkers(srcrepo.ui)
-
-    tokens = multiprocessing.BoundedSemaphore(nbworkers * BUFF_PER_WORKER)
-    revsq = multiprocessing.Queue()
-    sidedataq = multiprocessing.Queue()
-
-    assert srcrepo.filtername is None
-    # queue all tasks beforehand, revision numbers are small and it make
-    # synchronisation simpler
-    #
-    # Since the computation for each node can be quite expensive, the overhead
-    # of using a single queue is not revelant. In practice, most computation
-    # are fast but some are very expensive and dominate all the other smaller
-    # cost.
-    for r in srcrepo.changelog.revs():
-        revsq.put(r)
-    # queue the "no more tasks" markers
-    for i in range(nbworkers):
-        revsq.put(None)
-
-    allworkers = []
-    for i in range(nbworkers):
-        args = (srcrepo, revsq, sidedataq, tokens)
-        w = multiprocessing.Process(target=_sidedata_worker, args=args)
-        allworkers.append(w)
-        w.start()
-
-    # dictionnary to store results for revision higher than we one we are
-    # looking for. For example, if we need the sidedatamap for 42, and 43 is
-    # received, when shelve 43 for later use.
-    staging = {}
-
-    def sidedata_companion(revlog, rev):
-        sidedata = {}
-        if util.safehasattr(revlog, b'filteredrevs'):  # this is a changelog
-            # Is the data previously shelved ?
-            sidedata = staging.pop(rev, None)
-            if sidedata is None:
-                # look at the queued result until we find the one we are lookig
-                # for (shelve the other ones)
-                r, sidedata = sidedataq.get()
-                while r != rev:
-                    staging[r] = sidedata
-                    r, sidedata = sidedataq.get()
-            tokens.release()
-        return False, (), sidedata
-
-    return sidedata_companion
-
-
-def _get_simple_sidedata_adder(srcrepo, destrepo):
-    """The simple version of the sidedata computation
-
-    It just compute it in the same thread on request"""
-
-    def sidedatacompanion(revlog, rev):
-        sidedata = {}
-        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
-            sidedata = _getsidedata(srcrepo, rev)
-        return False, (), sidedata
-
-    return sidedatacompanion
-
-
-def getsidedataremover(srcrepo, destrepo):
-    def sidedatacompanion(revlog, rev):
-        f = ()
-        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
-            if revlog.flags(rev) & REVIDX_SIDEDATA:
-                f = (
-                    sidedatamod.SD_P1COPIES,
-                    sidedatamod.SD_P2COPIES,
-                    sidedatamod.SD_FILESADDED,
-                    sidedatamod.SD_FILESREMOVED,
-                )
-        return False, f, {}
-
-    return sidedatacompanion

mercurial/crecord.py

@@ -20,6 +20,7 b' from .pycompat import ('
     open,
 )
 from . import (
+    diffhelper,
     encoding,
     error,
     patch as patchmod,
@@ -63,15 +64,7 b' try:'
 
     curses.error
 except (ImportError, AttributeError):
-    # I have no idea if wcurses works with crecord...
-    try:
-        import wcurses as curses
-
-        curses.error
-    except (ImportError, AttributeError):
-        # wcurses is not shipped on Windows by default, or python is not
-        # compiled with curses
-        curses = False
+    curses = False
 
 
 class fallbackerror(error.Abort):
@@ -424,7 +417,7 b' class uihunk(patchnode):'
         contextlen = (
             len(self.before) + len(self.after) + removedconvertedtocontext
         )
-        if self.after and self.after[-1] == b'\\ No newline at end of file\n':
+        if self.after and self.after[-1] == diffhelper.MISSING_NEWLINE_MARKER:
             contextlen -= 1
         fromlen = contextlen + self.removed
         tolen = contextlen + self.added
@@ -508,8 +501,12 b' class uihunk(patchnode):'
         """
         dels = []
         adds = []
+        noeol = False
         for line in self.changedlines:
             text = line.linetext
+            if line.linetext == diffhelper.MISSING_NEWLINE_MARKER:
+                noeol = True
+                break
             if line.applied:
                 if text.startswith(b'+'):
                     dels.append(text[1:])
@@ -519,6 +516,9 b' class uihunk(patchnode):'
                 dels.append(text[1:])
                 adds.append(text[1:])
         hunk = [b'-%s' % l for l in dels] + [b'+%s' % l for l in adds]
+        if noeol and hunk:
+            # Remove the newline from the end of the hunk.
+            hunk[-1] = hunk[-1][:-1]
         h = self._hunk
         return patchmod.recordhunk(
             h.header, h.toline, h.fromline, h.proc, h.before, hunk, h.after

mercurial/debugcommands.py

@@ -58,7 +58,7 b' from . import ('
     localrepo,
     lock as lockmod,
     logcmdutil,
-    merge as mergemod,
+    mergestate as mergestatemod,
     obsolete,
     obsutil,
     pathutil,
@@ -127,6 +127,23 b' def debugancestor(ui, repo, *args):'
     ui.write(b'%d:%s\n' % (r.rev(a), hex(a)))
 
 
+@command(b'debugantivirusrunning', [])
+def debugantivirusrunning(ui, repo):
+    """attempt to trigger an antivirus scanner to see if one is active"""
+    with repo.cachevfs.open('eicar-test-file.com', b'wb') as f:
+        f.write(
+            util.b85decode(
+                # This is a base85-armored version of the EICAR test file. See
+                # https://en.wikipedia.org/wiki/EICAR_test_file for details.
+                b'ST#=}P$fV?P+K%yP+C|uG$>GBDK|qyDK~v2MM*<JQY}+dK~6+LQba95P'
+                b'E<)&Nm5l)EmTEQR4qnHOhq9iNGnJx'
+            )
+        )
+    # Give an AV engine time to scan the file.
+    time.sleep(2)
+    util.unlink(repo.cachevfs.join('eicar-test-file.com'))
+
+
 @command(b'debugapplystreamclonebundle', [], b'FILE')
 def debugapplystreamclonebundle(ui, repo, fname):
     """apply a stream clone bundle file"""
@@ -1465,8 +1482,8 b' def debuginstall(ui, **opts):'
     fm = ui.formatter(b'debuginstall', opts)
     fm.startitem()
 
-    # encoding
-    fm.write(b'encoding', _(b"checking encoding (%s)...\n"), encoding.encoding)
+    # encoding might be unknown or wrong. don't translate these messages.
+    fm.write(b'encoding', b"checking encoding (%s)...\n", encoding.encoding)
     err = None
     try:
         codecs.lookup(pycompat.sysstr(encoding.encoding))
@@ -1476,7 +1493,7 b' def debuginstall(ui, **opts):'
     fm.condwrite(
         err,
         b'encodingerror',
-        _(b" %s\n (check that your locale is properly set)\n"),
+        b" %s\n (check that your locale is properly set)\n",
         err,
     )
 
@@ -1650,13 +1667,6 b' def debuginstall(ui, **opts):'
     fm.plain(_(b'checking "re2" regexp engine (%s)\n') % re2)
     fm.data(re2=bool(util._re2))
 
-    rust_debug_mod = policy.importrust("debug")
-    if rust_debug_mod is not None:
-        re2_rust = b'installed' if rust_debug_mod.re2_installed else b'missing'
-
-        msg = b'checking "re2" regexp engine Rust bindings (%s)\n'
-        fm.plain(_(msg % re2_rust))
-
     # templates
     p = templater.templatepaths()
     fm.write(b'templatedirs', b'checking templates (%s)...\n', b' '.join(p))
@@ -1974,7 +1984,7 b' def debugmergestate(ui, repo, *args, **o'
     was chosen."""
 
     if ui.verbose:
-        ms = mergemod.mergestate(repo)
+        ms = mergestatemod.mergestate(repo)
 
         # sort so that reasonable information is on top
         v1records = ms._readrecordsv1()
@@ -2008,7 +2018,7 b' def debugmergestate(ui, repo, *args, **o'
             b'"}'
         )
 
-    ms = mergemod.mergestate.read(repo)
+    ms = mergestatemod.mergestate.read(repo)
 
     fm = ui.formatter(b'debugmergestate', opts)
     fm.startitem()
@@ -2034,8 +2044,8 b' def debugmergestate(ui, repo, *args, **o'
             state = ms._state[f]
             fm_files.data(state=state[0])
             if state[0] in (
-                mergemod.MERGE_RECORD_UNRESOLVED,
-                mergemod.MERGE_RECORD_RESOLVED,
+                mergestatemod.MERGE_RECORD_UNRESOLVED,
+                mergestatemod.MERGE_RECORD_RESOLVED,
             ):
                 fm_files.data(local_key=state[1])
                 fm_files.data(local_path=state[2])
@@ -2045,8 +2055,8 b' def debugmergestate(ui, repo, *args, **o'
                 fm_files.data(other_node=state[6])
                 fm_files.data(local_flags=state[7])
             elif state[0] in (
-                mergemod.MERGE_RECORD_UNRESOLVED_PATH,
-                mergemod.MERGE_RECORD_RESOLVED_PATH,
+                mergestatemod.MERGE_RECORD_UNRESOLVED_PATH,
+                mergestatemod.MERGE_RECORD_RESOLVED_PATH,
             ):
                 fm_files.data(renamed_path=state[1])
                 fm_files.data(rename_side=state[2])
@@ -2658,6 +2668,13 b' def debugrename(ui, repo, *pats, **opts)'
             ui.write(_(b"%s not renamed\n") % rel)
 
 
+@command(b'debugrequires|debugrequirements', [], b'')
+def debugrequirements(ui, repo):
+    """ print the current repo requirements """
+    for r in sorted(repo.requirements):
+        ui.write(b"%s\n" % r)
+
+
 @command(
     b'debugrevlog',
     cmdutil.debugrevlogopts + [(b'd', b'dump', False, _(b'dump index data'))],

mercurial/diffhelper.py

@@ -14,6 +14,8 b' from . import ('
     pycompat,
 )
 
+MISSING_NEWLINE_MARKER = b'\\ No newline at end of file\n'
+
 
 def addlines(fp, hunk, lena, lenb, a, b):
     """Read lines from fp into the hunk
@@ -32,7 +34,7 b' def addlines(fp, hunk, lena, lenb, a, b)'
             s = fp.readline()
             if not s:
                 raise error.ParseError(_(b'incomplete hunk'))
-            if s == b"\\ No newline at end of file\n":
+            if s == MISSING_NEWLINE_MARKER:
                 fixnewline(hunk, a, b)
                 continue
             if s == b'\n' or s == b'\r\n':

mercurial/dirstate.py

@@ -187,7 +187,7 b' class dirstate(object):'
 
     @propertycache
     def _checkexec(self):
-        return util.checkexec(self._root)
+        return bool(util.checkexec(self._root))
 
     @propertycache
     def _checkcase(self):
@@ -1114,6 +1114,7 b' class dirstate(object):'
             unknown,
             warnings,
             bad,
+            traversed,
         ) = rustmod.status(
             self._map._rustmap,
             matcher,
@@ -1124,7 +1125,13 b' class dirstate(object):'
             bool(list_clean),
             bool(list_ignored),
             bool(list_unknown),
+            bool(matcher.traversedir),
         )
+
+        if matcher.traversedir:
+            for dir in traversed:
+                matcher.traversedir(dir)
+
         if self._ui.warn:
             for item in warnings:
                 if isinstance(item, tuple):
@@ -1200,10 +1207,8 b' class dirstate(object):'
             use_rust = False
         elif sparse.enabled:
             use_rust = False
-        elif match.traversedir is not None:
-            use_rust = False
         elif not isinstance(match, allowed_matchers):
-            # Matchers have yet to be implemented
+            # Some matchers have yet to be implemented
             use_rust = False
 
         if use_rust:

mercurial/discovery.py

@@ -41,8 +41,8 b' def findcommonincoming(repo, remote, hea'
       any longer.
     "heads" is either the supplied heads, or else the remote's heads.
     "ancestorsof" if not None, restrict the discovery to a subset defined by
-      these nodes. Changeset outside of this set won't be considered (and
-      won't appears in "common")
+      these nodes. Changeset outside of this set won't be considered (but may
+      still appear in "common").
 
     If you pass heads and they are all known locally, the response lists just
     these heads in "common" and in "heads".
@@ -75,28 +75,35 b' def findcommonincoming(repo, remote, hea'
 
 
 class outgoing(object):
-    '''Represents the set of nodes present in a local repo but not in a
-    (possibly) remote one.
+    '''Represents the result of a findcommonoutgoing() call.
 
     Members:
 
-      missing is a list of all nodes present in local but not in remote.
-      common is a list of all nodes shared between the two repos.
-      excluded is the list of missing changeset that shouldn't be sent remotely.
-      missingheads is the list of heads of missing.
+      ancestorsof is a list of the nodes whose ancestors are included in the
+      outgoing operation.
+
+      missing is a list of those ancestors of ancestorsof that are present in
+      local but not in remote.
+
+      common is a set containing revs common between the local and the remote
+      repository (at least all of those that are ancestors of ancestorsof).
+
       commonheads is the list of heads of common.
 
-    The sets are computed on demand from the heads, unless provided upfront
+      excluded is the list of missing changeset that shouldn't be sent
+      remotely.
+
+    Some members are computed on demand from the heads, unless provided upfront
     by discovery.'''
 
     def __init__(
-        self, repo, commonheads=None, missingheads=None, missingroots=None
+        self, repo, commonheads=None, ancestorsof=None, missingroots=None
     ):
         # at least one of them must not be set
         assert None in (commonheads, missingroots)
         cl = repo.changelog
-        if missingheads is None:
-            missingheads = cl.heads()
+        if ancestorsof is None:
+            ancestorsof = cl.heads()
         if missingroots:
             discbases = []
             for n in missingroots:
@@ -104,14 +111,14 b' class outgoing(object):'
             # TODO remove call to nodesbetween.
             # TODO populate attributes on outgoing instance instead of setting
             # discbases.
-            csets, roots, heads = cl.nodesbetween(missingroots, missingheads)
+            csets, roots, heads = cl.nodesbetween(missingroots, ancestorsof)
             included = set(csets)
-            missingheads = heads
+            ancestorsof = heads
             commonheads = [n for n in discbases if n not in included]
         elif not commonheads:
             commonheads = [nullid]
         self.commonheads = commonheads
-        self.missingheads = missingheads
+        self.ancestorsof = ancestorsof
         self._revlog = cl
         self._common = None
         self._missing = None
@@ -119,7 +126,7 b' class outgoing(object):'
 
     def _computecommonmissing(self):
         sets = self._revlog.findcommonmissing(
-            self.commonheads, self.missingheads
+            self.commonheads, self.ancestorsof
         )
         self._common, self._missing = sets
 
@@ -135,6 +142,17 b' class outgoing(object):'
             self._computecommonmissing()
         return self._missing
 
+    @property
+    def missingheads(self):
+        util.nouideprecwarn(
+            b'outgoing.missingheads never contained what the name suggests and '
+            b'was renamed to outgoing.ancestorsof. check your code for '
+            b'correctness.',
+            b'5.5',
+            stacklevel=2,
+        )
+        return self.ancestorsof
+
 
 def findcommonoutgoing(
     repo, other, onlyheads=None, force=False, commoninc=None, portable=False
@@ -149,7 +167,7 b' def findcommonoutgoing('
     If commoninc is given, it must be the result of a prior call to
     findcommonincoming(repo, other, force) to avoid recomputing it here.
 
-    If portable is given, compute more conservative common and missingheads,
+    If portable is given, compute more conservative common and ancestorsof,
     to make bundles created from the instance more portable.'''
     # declare an empty outgoing object to be filled later
     og = outgoing(repo, None, None)
@@ -164,10 +182,10 b' def findcommonoutgoing('
     # compute outgoing
     mayexclude = repo._phasecache.phaseroots[phases.secret] or repo.obsstore
     if not mayexclude:
-        og.missingheads = onlyheads or repo.heads()
+        og.ancestorsof = onlyheads or repo.heads()
     elif onlyheads is None:
         # use visible heads as it should be cached
-        og.missingheads = repo.filtered(b"served").heads()
+        og.ancestorsof = repo.filtered(b"served").heads()
         og.excluded = [ctx.node() for ctx in repo.set(b'secret() or extinct()')]
     else:
         # compute common, missing and exclude secret stuff
@@ -182,12 +200,12 b' def findcommonoutgoing('
             else:
                 missing.append(node)
         if len(missing) == len(allmissing):
-            missingheads = onlyheads
+            ancestorsof = onlyheads
         else:  # update missing heads
-            missingheads = phases.newheads(repo, onlyheads, excluded)
-        og.missingheads = missingheads
+            ancestorsof = phases.newheads(repo, onlyheads, excluded)
+        og.ancestorsof = ancestorsof
     if portable:
-        # recompute common and missingheads as if -r<rev> had been given for
+        # recompute common and ancestorsof as if -r<rev> had been given for
         # each head of missing, and --base <rev> for each head of the proper
         # ancestors of missing
         og._computecommonmissing()
@@ -195,7 +213,7 b' def findcommonoutgoing('
         missingrevs = {cl.rev(n) for n in og._missing}
         og._common = set(cl.ancestors(missingrevs)) - missingrevs
         commonheads = set(og.commonheads)
-        og.missingheads = [h for h in og.missingheads if h not in commonheads]
+        og.ancestorsof = [h for h in og.ancestorsof if h not in commonheads]
 
     return og
 
@@ -268,7 +286,7 b' def _headssummary(pushop):'
     # If there are no obsstore, no post processing are needed.
     if repo.obsstore:
         torev = repo.changelog.rev
-        futureheads = {torev(h) for h in outgoing.missingheads}
+        futureheads = {torev(h) for h in outgoing.ancestorsof}
         futureheads |= {torev(h) for h in outgoing.commonheads}
         allfuturecommon = repo.changelog.ancestors(futureheads, inclusive=True)
         for branch, heads in sorted(pycompat.iteritems(headssum)):

mercurial/dispatch.py

@@ -104,41 +104,46 b' class request(object):'
 
 def run():
     """run the command in sys.argv"""
-    initstdio()
-    with tracing.log('parse args into request'):
-        req = request(pycompat.sysargv[1:])
-    err = None
     try:
-        status = dispatch(req)
-    except error.StdioError as e:
-        err = e
-        status = -1
-
-    # In all cases we try to flush stdio streams.
-    if util.safehasattr(req.ui, b'fout'):
-        assert req.ui is not None  # help pytype
-        assert req.ui.fout is not None  # help pytype
+        initstdio()
+        with tracing.log('parse args into request'):
+            req = request(pycompat.sysargv[1:])
+        err = None
         try:
-            req.ui.fout.flush()
-        except IOError as e:
+            status = dispatch(req)
+        except error.StdioError as e:
             err = e
             status = -1
 
-    if util.safehasattr(req.ui, b'ferr'):
-        assert req.ui is not None  # help pytype
-        assert req.ui.ferr is not None  # help pytype
-        try:
-            if err is not None and err.errno != errno.EPIPE:
-                req.ui.ferr.write(
-                    b'abort: %s\n' % encoding.strtolocal(err.strerror)
-                )
-            req.ui.ferr.flush()
-        # There's not much we can do about an I/O error here. So (possibly)
-        # change the status code and move on.
-        except IOError:
-            status = -1
+        # In all cases we try to flush stdio streams.
+        if util.safehasattr(req.ui, b'fout'):
+            assert req.ui is not None  # help pytype
+            assert req.ui.fout is not None  # help pytype
+            try:
+                req.ui.fout.flush()
+            except IOError as e:
+                err = e
+                status = -1
 
-    _silencestdio()
+        if util.safehasattr(req.ui, b'ferr'):
+            assert req.ui is not None  # help pytype
+            assert req.ui.ferr is not None  # help pytype
+            try:
+                if err is not None and err.errno != errno.EPIPE:
+                    req.ui.ferr.write(
+                        b'abort: %s\n' % encoding.strtolocal(err.strerror)
+                    )
+                req.ui.ferr.flush()
+            # There's not much we can do about an I/O error here. So (possibly)
+            # change the status code and move on.
+            except IOError:
+                status = -1
+
+        _silencestdio()
+    except KeyboardInterrupt:
+        # Catch early/late KeyboardInterrupt as last ditch. Here nothing will
+        # be printed to console to avoid another IOError/KeyboardInterrupt.
+        status = -1
     sys.exit(status & 255)
 
 

mercurial/error.py

@@ -106,6 +106,22 b' class InterventionRequired(Hint, Excepti'
     __bytes__ = _tobytes
 
 
+class ConflictResolutionRequired(InterventionRequired):
+    """Exception raised when a continuable command required merge conflict resolution."""
+
+    def __init__(self, opname):
+        from .i18n import _
+
+        self.opname = opname
+        InterventionRequired.__init__(
+            self,
+            _(
+                b"unresolved conflicts (see 'hg resolve', then 'hg %s --continue')"
+            )
+            % opname,
+        )
+
+
 class Abort(Hint, Exception):
     """Raised if a command needs to print an error and exit."""
 

mercurial/exchange.py

@@ -503,7 +503,7 b' class pushoperation(object):'
     @util.propertycache
     def futureheads(self):
         """future remote heads if the changeset push succeeds"""
-        return self.outgoing.missingheads
+        return self.outgoing.ancestorsof
 
     @util.propertycache
     def fallbackheads(self):
@@ -512,20 +512,20 b' class pushoperation(object):'
             # not target to push, all common are relevant
             return self.outgoing.commonheads
         unfi = self.repo.unfiltered()
-        # I want cheads = heads(::missingheads and ::commonheads)
-        # (missingheads is revs with secret changeset filtered out)
+        # I want cheads = heads(::ancestorsof and ::commonheads)
+        # (ancestorsof is revs with secret changeset filtered out)
         #
         # This can be expressed as:
-        #     cheads = ( (missingheads and ::commonheads)
-        #              + (commonheads and ::missingheads))"
+        #     cheads = ( (ancestorsof and ::commonheads)
+        #              + (commonheads and ::ancestorsof))"
         #              )
         #
         # while trying to push we already computed the following:
         #     common = (::commonheads)
-        #     missing = ((commonheads::missingheads) - commonheads)
+        #     missing = ((commonheads::ancestorsof) - commonheads)
         #
         # We can pick:
-        # * missingheads part of common (::commonheads)
+        # * ancestorsof part of common (::commonheads)
         common = self.outgoing.common
         rev = self.repo.changelog.index.rev
         cheads = [node for node in self.revs if rev(node) in common]
@@ -905,27 +905,32 b' def _pushcheckoutgoing(pushop):'
         # if repo.obsstore == False --> no obsolete
         # then, save the iteration
         if unfi.obsstore:
-            # this message are here for 80 char limit reason
-            mso = _(b"push includes obsolete changeset: %s!")
-            mspd = _(b"push includes phase-divergent changeset: %s!")
-            mscd = _(b"push includes content-divergent changeset: %s!")
-            mst = {
-                b"orphan": _(b"push includes orphan changeset: %s!"),
-                b"phase-divergent": mspd,
-                b"content-divergent": mscd,
-            }
-            # If we are to push if there is at least one
-            # obsolete or unstable changeset in missing, at
-            # least one of the missinghead will be obsolete or
-            # unstable. So checking heads only is ok
-            for node in outgoing.missingheads:
+            obsoletes = []
+            unstables = []
+            for node in outgoing.missing:
                 ctx = unfi[node]
                 if ctx.obsolete():
-                    raise error.Abort(mso % ctx)
+                    obsoletes.append(ctx)
                 elif ctx.isunstable():
-                    # TODO print more than one instability in the abort
-                    # message
-                    raise error.Abort(mst[ctx.instabilities()[0]] % ctx)
+                    unstables.append(ctx)
+            if obsoletes or unstables:
+                msg = b""
+                if obsoletes:
+                    msg += _(b"push includes obsolete changesets:\n")
+                    msg += b"\n".join(b'  %s' % ctx for ctx in obsoletes)
+                if unstables:
+                    if msg:
+                        msg += b"\n"
+                    msg += _(b"push includes unstable changesets:\n")
+                    msg += b"\n".join(
+                        b'  %s (%s)'
+                        % (
+                            ctx,
+                            b", ".join(_(ins) for ins in ctx.instabilities()),
+                        )
+                        for ctx in unstables
+                    )
+                raise error.Abort(msg)
 
         discovery.checkheads(pushop)
     return True
@@ -969,7 +974,7 b' def _pushb2ctxcheckheads(pushop, bundler'
     """
     # * 'force' do not check for push race,
     # * if we don't push anything, there are nothing to check.
-    if not pushop.force and pushop.outgoing.missingheads:
+    if not pushop.force and pushop.outgoing.ancestorsof:
         allowunrelated = b'related' in bundler.capabilities.get(
             b'checkheads', ()
         )
@@ -1024,12 +1029,12 b' def _pushb2checkphases(pushop, bundler):'
     hasphaseheads = b'heads' in b2caps.get(b'phases', ())
     if pushop.remotephases is not None and hasphaseheads:
         # check that the remote phase has not changed
-        checks = [[] for p in phases.allphases]
+        checks = {p: [] for p in phases.allphases}
         checks[phases.public].extend(pushop.remotephases.publicheads)
         checks[phases.draft].extend(pushop.remotephases.draftroots)
-        if any(checks):
-            for nodes in checks:
-                nodes.sort()
+        if any(pycompat.itervalues(checks)):
+            for phase in checks:
+                checks[phase].sort()
             checkdata = phases.binaryencode(checks)
             bundler.newpart(b'check:phases', data=checkdata)
 
@@ -1104,7 +1109,7 b' def _pushb2phaseheads(pushop, bundler):'
     """push phase information through a bundle2 - binary part"""
     pushop.stepsdone.add(b'phases')
     if pushop.outdatedphases:
-        updates = [[] for p in phases.allphases]
+        updates = {p: [] for p in phases.allphases}
         updates[0].extend(h.node() for h in pushop.outdatedphases)
         phasedata = phases.binaryencode(updates)
         bundler.newpart(b'phase-heads', data=phasedata)
@@ -2658,9 +2663,9 b' def _getbundlephasespart('
                     headsbyphase[phases.public].add(node(r))
 
         # transform data in a format used by the encoding function
-        phasemapping = []
-        for phase in phases.allphases:
-            phasemapping.append(sorted(headsbyphase[phase]))
+        phasemapping = {
+            phase: sorted(headsbyphase[phase]) for phase in phases.allphases
+        }
 
         # generate the actual part
         phasedata = phases.binaryencode(phasemapping)
@@ -3025,6 +3030,23 b' def filterclonebundleentries(repo, entri'
             )
             continue
 
+        if b'REQUIREDRAM' in entry:
+            try:
+                requiredram = util.sizetoint(entry[b'REQUIREDRAM'])
+            except error.ParseError:
+                repo.ui.debug(
+                    b'filtering %s due to a bad REQUIREDRAM attribute\n'
+                    % entry[b'URL']
+                )
+                continue
+            actualram = repo.ui.estimatememory()
+            if actualram is not None and actualram * 0.66 < requiredram:
+                repo.ui.debug(
+                    b'filtering %s as it needs more than 2/3 of system memory\n'
+                    % entry[b'URL']
+                )
+                continue
+
         newentries.append(entry)
 
     return newentries

mercurial/exchangev2.py

@@ -82,15 +82,12 b' def pull(pullop):'
         phases.registernew(repo, tr, phases.draft, csetres[b'added'])
 
     # And adjust the phase of all changesets accordingly.
-    for phase in phases.phasenames:
+    for phasenumber, phase in phases.phasenames.items():
         if phase == b'secret' or not csetres[b'nodesbyphase'][phase]:
             continue
 
         phases.advanceboundary(
-            repo,
-            tr,
-            phases.phasenames.index(phase),
-            csetres[b'nodesbyphase'][phase],
+            repo, tr, phasenumber, csetres[b'nodesbyphase'][phase],
         )
 
     # Write bookmark updates.
@@ -361,7 +358,7 b' def _processchangesetdata(repo, tr, objs'
         # so we can set the linkrev accordingly when manifests are added.
         manifestnodes[cl.rev(node)] = revision.manifest
 
-    nodesbyphase = {phase: set() for phase in phases.phasenames}
+    nodesbyphase = {phase: set() for phase in phases.phasenames.values()}
     remotebookmarks = {}
 
     # addgroup() expects a 7-tuple describing revisions. This normalizes

mercurial/extensions.py

@@ -706,12 +706,17 b' def _disabledpaths():'
     '''find paths of disabled extensions. returns a dict of {name: path}'''
     import hgext
 
-    extpath = os.path.dirname(
-        os.path.abspath(pycompat.fsencode(hgext.__file__))
-    )
-    try:  # might not be a filesystem path
-        files = os.listdir(extpath)
-    except OSError:
+    # The hgext might not have a __file__ attribute (e.g. in PyOxidizer) and
+    # it might not be on a filesystem even if it does.
+    if util.safehasattr(hgext, '__file__'):
+        extpath = os.path.dirname(
+            os.path.abspath(pycompat.fsencode(hgext.__file__))
+        )
+        try:
+            files = os.listdir(extpath)
+        except OSError:
+            return {}
+    else:
         return {}
 
     exts = {}

mercurial/filemerge.py

@@ -98,6 +98,9 b' class absentfilectx(object):'
         self._ctx = ctx
         self._f = f
 
+    def __bytes__(self):
+        return b'absent file %s@%s' % (self._f, self._ctx)
+
     def path(self):
         return self._f
 

mercurial/fileset.py

@@ -16,7 +16,7 b' from . import ('
     error,
     filesetlang,
     match as matchmod,
-    merge,
+    mergestate as mergestatemod,
     pycompat,
     registrar,
     scmutil,
@@ -245,7 +245,7 b' def resolved(mctx, x):'
     getargs(x, 0, 0, _(b"resolved takes no arguments"))
     if mctx.ctx.rev() is not None:
         return mctx.never()
-    ms = merge.mergestate.read(mctx.ctx.repo())
+    ms = mergestatemod.mergestate.read(mctx.ctx.repo())
     return mctx.predicate(
         lambda f: f in ms and ms[f] == b'r', predrepr=b'resolved'
     )
@@ -259,7 +259,7 b' def unresolved(mctx, x):'
     getargs(x, 0, 0, _(b"unresolved takes no arguments"))
     if mctx.ctx.rev() is not None:
         return mctx.never()
-    ms = merge.mergestate.read(mctx.ctx.repo())
+    ms = mergestatemod.mergestate.read(mctx.ctx.repo())
     return mctx.predicate(
         lambda f: f in ms and ms[f] == b'u', predrepr=b'unresolved'
     )

mercurial/help.py

@@ -345,6 +345,11 b' def loaddoc(topic, subdir=None):'
 
 internalstable = sorted(
     [
+        (
+            [b'bid-merge'],
+            _(b'Bid Merge Algorithm'),
+            loaddoc(b'bid-merge', subdir=b'internals'),
+        ),
         ([b'bundle2'], _(b'Bundle2'), loaddoc(b'bundle2', subdir=b'internals')),
         ([b'bundles'], _(b'Bundles'), loaddoc(b'bundles', subdir=b'internals')),
         ([b'cbor'], _(b'CBOR'), loaddoc(b'cbor', subdir=b'internals')),

mercurial/helptext/config.txt

@@ -408,6 +408,24 b' Supported arguments:'
 If no suitable authentication entry is found, the user is prompted
 for credentials as usual if required by the remote.
 
+``cmdserver``
+-------------
+
+Controls command server settings. (ADVANCED)
+
+``message-encodings``
+    List of encodings for the ``m`` (message) channel. The first encoding
+    supported by the server will be selected and advertised in the hello
+    message. This is useful only when ``ui.message-output`` is set to
+    ``channel``. Supported encodings are ``cbor``.
+
+``shutdown-on-interrupt``
+    If set to false, the server's main loop will continue running after
+    SIGINT received. ``runcommand`` requests can still be interrupted by
+    SIGINT. Close the write end of the pipe to shut down the server
+    process gracefully.
+    (default: True)
+
 ``color``
 ---------
 
@@ -1872,6 +1890,15 b' Alias definitions for revsets. See :hg:`'
     applicable for `hg amend`, `hg commit --amend` and `hg uncommit` in the
     current version.
 
+``empty-successor``
+
+    Control what happens with empty successors that are the result of rewrite
+    operations. If set to ``skip``, the successor is not created. If set to
+    ``keep``, the empty successor is created and kept.
+
+    Currently, only the rebase and absorb commands consider this configuration.
+    (EXPERIMENTAL)
+
 ``storage``
 -----------
 
@@ -2371,6 +2398,8 b' User interface controls.'
 ``message-output``
     Where to write status and error messages. (default: ``stdio``)
 
+    ``channel``
+      Use separate channel for structured output. (Command-server only)
     ``stderr``
       Everything to stderr.
     ``stdio``

mercurial/helptext/flags.txt

@@ -10,7 +10,9 b' after the command.'
 
 Every flag has at least a long name, such as --repository. Some flags may also
 have a short one-letter name, such as the equivalent -R. Using the short or long
-name is equivalent and has the same effect.
+name is equivalent and has the same effect. The long name may be abbreviated to
+any unambiguous prefix. For example, :hg:`commit --amend` can be abbreviated
+to :hg:`commit --am`.
 
 Flags that have a short name can also be bundled together - for instance, to
 specify both --edit (short -e) and --interactive (short -i), one could use::

mercurial/helptext/internals/requirements.txt

@@ -142,3 +142,16 b' Support for this requirement was added i'
 August 2019). The requirement will only be present on repositories
 that have opted in to this format (by having
 ``format.bookmarks-in-store=true`` set when they were created).
+
+persistent-nodemap
+==================
+
+The `nodemap` index (mapping nodeid to local revision number) is persisted on
+disk. This provides speed benefit (if the associated native code is used). The
+persistent nodemap is only used for two revlogs: the changelog and the
+manifestlog.
+
+Support for this requirement was added in Mercurial 5.5 (released August 2020).
+Note that as of 5.5, only installations compiled with the Rust extension will
+benefit from a speedup. The other installations will do the necessary work to
+keep the index up to date, but will suffer a slowdown.

mercurial/helptext/internals/revlogs.txt

@@ -161,7 +161,7 b' Version 2 Format'
 
 (In development. Format not finalized or stable.)
 
-Version 2 is identical to version 2 with the following differences.
+Version 2 is identical to version 1 with the following differences.
 
 There is no dedicated *generaldelta* revlog format flag. Instead,
 the feature is implied enabled by default.

mercurial/hg.py

@@ -33,6 +33,7 b' from . import ('
     logcmdutil,
     logexchange,
     merge as mergemod,
+    mergestate as mergestatemod,
     narrowspec,
     node,
     phases,
@@ -355,7 +356,7 b' def unshare(ui, repo):'
 
             repo.requirements.discard(b'shared')
             repo.requirements.discard(b'relshared')
-            repo._writerequirements()
+            scmutil.writereporequirements(repo)
 
     # Removing share changes some fundamental properties of the repo instance.
     # So we instantiate a new repo object and operate on it rather than
@@ -1164,7 +1165,7 b' def merge('
 
 
 def abortmerge(ui, repo):
-    ms = mergemod.mergestate.read(repo)
+    ms = mergestatemod.mergestate.read(repo)
     if ms.active():
         # there were conflicts
         node = ms.localctx.hex()

mercurial/hgweb/server.py

@@ -313,7 +313,7 b' class _httprequesthandlerssl(_httpreques'
         try:
             from .. import sslutil
 
-            sslutil.modernssl
+            sslutil.wrapserversocket
         except ImportError:
             raise error.Abort(_(b"SSL support is unavailable"))
 

mercurial/hook.py

@@ -158,6 +158,10 b' def _exthook(ui, repo, htype, name, cmd,'
     env[b'HG_HOOKNAME'] = name
 
     for k, v in pycompat.iteritems(args):
+        # transaction changes can accumulate MBs of data, so skip it
+        # for external hooks
+        if k == b'changes':
+            continue
         if callable(v):
             v = v()
         if isinstance(v, (dict, list)):

mercurial/interfaces/repository.py

@@ -1395,6 +1395,9 b' class imanifestlog(interfaceutil.Interfa'
         Raises ``error.LookupError`` if the node is not known.
         """
 
+    def update_caches(transaction):
+        """update whatever cache are relevant for the used storage."""
+
 
 class ilocalrepositoryfilestorage(interfaceutil.Interface):
     """Local repository sub-interface providing access to tracked file storage.

mercurial/localrepo.py

@@ -44,8 +44,9 b' from . import ('
     hook,
     lock as lockmod,
     match as matchmod,
-    merge as mergemod,
+    mergestate as mergestatemod,
     mergeutil,
+    metadata,
     namespaces,
     narrowspec,
     obsolete,
@@ -411,13 +412,13 b' class locallegacypeer(localpeer):'
 
     def changegroup(self, nodes, source):
         outgoing = discovery.outgoing(
-            self._repo, missingroots=nodes, missingheads=self._repo.heads()
+            self._repo, missingroots=nodes, ancestorsof=self._repo.heads()
         )
         return changegroup.makechangegroup(self._repo, outgoing, b'01', source)
 
     def changegroupsubset(self, bases, heads, source):
         outgoing = discovery.outgoing(
-            self._repo, missingroots=bases, missingheads=heads
+            self._repo, missingroots=bases, ancestorsof=heads
         )
         return changegroup.makechangegroup(self._repo, outgoing, b'01', source)
 
@@ -445,6 +446,9 b" SIDEDATA_REQUIREMENT = b'exp-sidedata-fl"
 # copies related information in changeset's sidedata.
 COPIESSDC_REQUIREMENT = b'exp-copies-sidedata-changeset'
 
+# The repository use persistent nodemap for the changelog and the manifest.
+NODEMAP_REQUIREMENT = b'persistent-nodemap'
+
 # Functions receiving (ui, features) that extensions can register to impact
 # the ability to load repositories with custom requirements. Only
 # functions defined in loaded extensions are called.
@@ -505,6 +509,11 b' def makelocalrepository(baseui, path, in'
         except OSError as e:
             if e.errno != errno.ENOENT:
                 raise
+        except ValueError as e:
+            # Can be raised on Python 3.8 when path is invalid.
+            raise error.Abort(
+                _(b'invalid path %s: %s') % (path, pycompat.bytestr(e))
+            )
 
         raise error.RepoError(_(b'repository %s not found') % path)
 
@@ -933,10 +942,12 b' def resolverevlogstorevfsoptions(ui, req'
 
     if ui.configbool(b'experimental', b'rust.index'):
         options[b'rust.index'] = True
-    if ui.configbool(b'experimental', b'exp-persistent-nodemap'):
-        options[b'exp-persistent-nodemap'] = True
-    if ui.configbool(b'experimental', b'exp-persistent-nodemap.mmap'):
-        options[b'exp-persistent-nodemap.mmap'] = True
+    if NODEMAP_REQUIREMENT in requirements:
+        options[b'persistent-nodemap'] = True
+    if ui.configbool(b'storage', b'revlog.nodemap.mmap'):
+        options[b'persistent-nodemap.mmap'] = True
+    epnm = ui.config(b'storage', b'revlog.nodemap.mode')
+    options[b'persistent-nodemap.mode'] = epnm
     if ui.configbool(b'devel', b'persistent-nodemap'):
         options[b'devel-force-nodemap'] = True
 
@@ -1021,6 +1032,7 b' class localrepository(object):'
         REVLOGV2_REQUIREMENT,
         SIDEDATA_REQUIREMENT,
         SPARSEREVLOG_REQUIREMENT,
+        NODEMAP_REQUIREMENT,
         bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT,
     }
     _basesupported = supportedformats | {
@@ -1223,8 +1235,9 b' class localrepository(object):'
             if path.startswith(b'cache/'):
                 msg = b'accessing cache with vfs instead of cachevfs: "%s"'
                 repo.ui.develwarn(msg % path, stacklevel=3, config=b"cache-vfs")
-            if path.startswith(b'journal.') or path.startswith(b'undo.'):
-                # journal is covered by 'lock'
+            # path prefixes covered by 'lock'
+            vfs_path_prefixes = (b'journal.', b'undo.', b'strip-backup/')
+            if any(path.startswith(prefix) for prefix in vfs_path_prefixes):
                 if repo._currentlock(repo._lockref) is None:
                     repo.ui.develwarn(
                         b'write with no lock: "%s"' % path,
@@ -1285,9 +1298,6 b' class localrepository(object):'
             caps.add(b'bundle2=' + urlreq.quote(capsblob))
         return caps
 
-    def _writerequirements(self):
-        scmutil.writerequires(self.vfs, self.requirements)
-
     # Don't cache auditor/nofsauditor, or you'll end up with reference cycle:
     # self -> auditor -> self._checknested -> self
 
@@ -2239,6 +2249,7 b' class localrepository(object):'
 
         tr.hookargs[b'txnid'] = txnid
         tr.hookargs[b'txnname'] = desc
+        tr.hookargs[b'changes'] = tr.changes
         # note: writing the fncache only during finalize mean that the file is
         # outdated when running hooks. As fncache is used for streaming clone,
         # this is not expected to break anything that happen during the hooks.
@@ -2461,7 +2472,7 b' class localrepository(object):'
                 ui.status(
                     _(b'working directory now based on revision %d\n') % parents
                 )
-            mergemod.mergestate.clean(self, self[b'.'].node())
+            mergestatemod.mergestate.clean(self, self[b'.'].node())
 
         # TODO: if we know which new heads may result from this rollback, pass
         # them to destroy(), which will prevent the branchhead cache from being
@@ -2511,6 +2522,7 b' class localrepository(object):'
             unfi = self.unfiltered()
 
             self.changelog.update_caches(transaction=tr)
+            self.manifestlog.update_caches(transaction=tr)
 
             rbc = unfi.revbranchcache()
             for r in unfi.changelog:
@@ -2771,6 +2783,22 b' class localrepository(object):'
     ):
         """
         commit an individual file as part of a larger transaction
+
+        input:
+
+            fctx:       a file context with the content we are trying to commit
+            manifest1:  manifest of changeset first parent
+            manifest2:  manifest of changeset second parent
+            linkrev:    revision number of the changeset being created
+            tr:         current transation
+            changelist: list of file being changed (modified inplace)
+            individual: boolean, set to False to skip storing the copy data
+                        (only used by the Google specific feature of using
+                        changeset extra as copy source of truth).
+
+        output:
+
+            The resulting filenode
         """
 
         fname = fctx.path()
@@ -2859,16 +2887,16 b' class localrepository(object):'
                 fparent2 = nullid
             elif not fparentancestors:
                 # TODO: this whole if-else might be simplified much more
-                ms = mergemod.mergestate.read(self)
+                ms = mergestatemod.mergestate.read(self)
                 if (
                     fname in ms
-                    and ms[fname] == mergemod.MERGE_RECORD_MERGED_OTHER
+                    and ms[fname] == mergestatemod.MERGE_RECORD_MERGED_OTHER
                 ):
                     fparent1, fparent2 = fparent2, nullid
 
         # is the file changed?
         text = fctx.data()
-        if fparent2 != nullid or flog.cmp(fparent1, text) or meta:
+        if fparent2 != nullid or meta or flog.cmp(fparent1, text):
             changelist.append(fname)
             return flog.add(text, meta, tr, linkrev, fparent1, fparent2)
         # are just the flags changed during merge?
@@ -2960,18 +2988,13 b' class localrepository(object):'
                 self, status, text, user, date, extra
             )
 
-            ms = mergemod.mergestate.read(self)
+            ms = mergestatemod.mergestate.read(self)
             mergeutil.checkunresolved(ms)
 
             # internal config: ui.allowemptycommit
-            allowemptycommit = (
-                wctx.branch() != wctx.p1().branch()
-                or extra.get(b'close')
-                or merge
-                or cctx.files()
-                or self.ui.configbool(b'ui', b'allowemptycommit')
-            )
-            if not allowemptycommit:
+            if cctx.isempty() and not self.ui.configbool(
+                b'ui', b'allowemptycommit'
+            ):
                 self.ui.debug(b'nothing to commit, clearing merge state\n')
                 ms.reset()
                 return None
@@ -3018,6 +3041,12 b' class localrepository(object):'
                     self.ui.write(
                         _(b'note: commit message saved in %s\n') % msgfn
                     )
+                    self.ui.write(
+                        _(
+                            b"note: use 'hg commit --logfile "
+                            b".hg/last-message.txt --edit' to reuse it\n"
+                        )
+                    )
                 raise
 
         def commithook(unused_success):
@@ -3131,51 +3160,8 b' class localrepository(object):'
                 for f in drop:
                     del m[f]
                 if p2.rev() != nullrev:
-
-                    @util.cachefunc
-                    def mas():
-                        p1n = p1.node()
-                        p2n = p2.node()
-                        cahs = self.changelog.commonancestorsheads(p1n, p2n)
-                        if not cahs:
-                            cahs = [nullrev]
-                        return [self[r].manifest() for r in cahs]
-
-                    def deletionfromparent(f):
-                        # When a file is removed relative to p1 in a merge, this
-                        # function determines whether the absence is due to a
-                        # deletion from a parent, or whether the merge commit
-                        # itself deletes the file. We decide this by doing a
-                        # simplified three way merge of the manifest entry for
-                        # the file. There are two ways we decide the merge
-                        # itself didn't delete a file:
-                        # - neither parent (nor the merge) contain the file
-                        # - exactly one parent contains the file, and that
-                        #   parent has the same filelog entry as the merge
-                        #   ancestor (or all of them if there two). In other
-                        #   words, that parent left the file unchanged while the
-                        #   other one deleted it.
-                        # One way to think about this is that deleting a file is
-                        # similar to emptying it, so the list of changed files
-                        # should be similar either way. The computation
-                        # described above is not done directly in _filecommit
-                        # when creating the list of changed files, however
-                        # it does something very similar by comparing filelog
-                        # nodes.
-                        if f in m1:
-                            return f not in m2 and all(
-                                f in ma and ma.find(f) == m1.find(f)
-                                for ma in mas()
-                            )
-                        elif f in m2:
-                            return all(
-                                f in ma and ma.find(f) == m2.find(f)
-                                for ma in mas()
-                            )
-                        else:
-                            return True
-
-                    removed = [f for f in removed if not deletionfromparent(f)]
+                    rf = metadata.get_removal_filter(ctx, (p1, p2, m1, m2))
+                    removed = [f for f in removed if not rf(f)]
 
                 files = changed + removed
                 md = None
@@ -3653,6 +3639,9 b' def newreporequirements(ui, createopts):'
     if ui.configbool(b'format', b'bookmarks-in-store'):
         requirements.add(bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT)
 
+    if ui.configbool(b'format', b'use-persistent-nodemap'):
+        requirements.add(NODEMAP_REQUIREMENT)
+
     return requirements
 
 

mercurial/logcmdutil.py

@@ -72,8 +72,8 b' def diffordiffstat('
     ui,
     repo,
     diffopts,
-    node1,
-    node2,
+    ctx1,
+    ctx2,
     match,
     changes=None,
     stat=False,
@@ -85,8 +85,6 b' def diffordiffstat('
     hunksfilterfn=None,
 ):
     '''show diff or diffstat.'''
-    ctx1 = repo[node1]
-    ctx2 = repo[node2]
     if root:
         relroot = pathutil.canonpath(repo.root, repo.getcwd(), root)
     else:
@@ -173,6 +171,7 b' def diffordiffstat('
             for chunk, label in chunks:
                 ui.write(chunk, label=label)
 
+    node2 = ctx2.node()
     for subpath, sub in scmutil.itersubrepos(ctx1, ctx2):
         tempnode2 = node2
         try:
@@ -208,15 +207,12 b' class changesetdiffer(object):'
         return None
 
     def showdiff(self, ui, ctx, diffopts, graphwidth=0, stat=False):
-        repo = ctx.repo()
-        node = ctx.node()
-        prev = ctx.p1().node()
         diffordiffstat(
             ui,
-            repo,
+            ctx.repo(),
             diffopts,
-            prev,
-            node,
+            ctx.p1(),
+            ctx,
             match=self._makefilematcher(ctx),
             stat=stat,
             graphwidth=graphwidth,

mercurial/manifest.py

@@ -58,14 +58,16 b' def _parse(data):'
         prev = l
         f, n = l.split(b'\0')
         nl = len(n)
-        if 64 < nl:
-            # modern hash, full width
-            yield f, bin(n[:64]), n[64:]
-        elif 40 < nl < 45:
-            # legacy hash, always sha1
-            yield f, bin(n[:40]), n[40:]
+        flags = n[-1:]
+        if flags in _manifestflags:
+            n = n[:-1]
+            nl -= 1
         else:
-            yield f, bin(n), b''
+            flags = b''
+        if nl not in (40, 64):
+            raise ValueError(b'Invalid manifest line')
+
+        yield f, bin(n), flags
 
 
 def _text(it):
@@ -121,8 +123,20 b' class lazymanifestiterentries(object):'
             self.pos += 1
             return data
         zeropos = data.find(b'\x00', pos)
-        hashval = unhexlify(data, self.lm.extrainfo[self.pos], zeropos + 1, 40)
-        flags = self.lm._getflags(data, self.pos, zeropos)
+        nlpos = data.find(b'\n', pos)
+        if zeropos == -1 or nlpos == -1 or nlpos < zeropos:
+            raise error.StorageError(b'Invalid manifest line')
+        flags = data[nlpos - 1 : nlpos]
+        if flags in _manifestflags:
+            hlen = nlpos - zeropos - 2
+        else:
+            hlen = nlpos - zeropos - 1
+            flags = b''
+        if hlen not in (40, 64):
+            raise error.StorageError(b'Invalid manifest line')
+        hashval = unhexlify(
+            data, self.lm.extrainfo[self.pos], zeropos + 1, hlen
+        )
         self.pos += 1
         return (data[pos:zeropos], hashval, flags)
 
@@ -140,6 +154,9 b' def _cmp(a, b):'
     return (a > b) - (a < b)
 
 
+_manifestflags = {b'', b'l', b't', b'x'}
+
+
 class _lazymanifest(object):
     """A pure python manifest backed by a byte string.  It is supplimented with
     internal lists as it is modified, until it is compacted back to a pure byte
@@ -251,15 +268,6 b' class _lazymanifest(object):'
     def __contains__(self, key):
         return self.bsearch(key) != -1
 
-    def _getflags(self, data, needle, pos):
-        start = pos + 41
-        end = data.find(b"\n", start)
-        if end == -1:
-            end = len(data) - 1
-        if start == end:
-            return b''
-        return self.data[start:end]
-
     def __getitem__(self, key):
         if not isinstance(key, bytes):
             raise TypeError(b"getitem: manifest keys must be a bytes.")
@@ -273,13 +281,17 b' class _lazymanifest(object):'
         nlpos = data.find(b'\n', zeropos)
         assert 0 <= needle <= len(self.positions)
         assert len(self.extrainfo) == len(self.positions)
+        if zeropos == -1 or nlpos == -1 or nlpos < zeropos:
+            raise error.StorageError(b'Invalid manifest line')
         hlen = nlpos - zeropos - 1
-        # Hashes sometimes have an extra byte tucked on the end, so
-        # detect that.
-        if hlen % 2:
+        flags = data[nlpos - 1 : nlpos]
+        if flags in _manifestflags:
             hlen -= 1
+        else:
+            flags = b''
+        if hlen not in (40, 64):
+            raise error.StorageError(b'Invalid manifest line')
         hashval = unhexlify(data, self.extrainfo[needle], zeropos + 1, hlen)
-        flags = self._getflags(data, needle, zeropos)
         return (hashval, flags)
 
     def __delitem__(self, key):
@@ -408,9 +420,7 b' class _lazymanifest(object):'
 
     def _pack(self, d):
         n = d[1]
-        if len(n) == 21 or len(n) == 33:
-            n = n[:-1]
-        assert len(n) == 20 or len(n) == 32
+        assert len(n) in (20, 32)
         return d[0] + b'\x00' + hex(n) + d[2] + b'\n'
 
     def text(self):
@@ -609,6 +619,8 b' class manifestdict(object):'
         return self._lm.diff(m2._lm, clean)
 
     def setflag(self, key, flag):
+        if flag not in _manifestflags:
+            raise TypeError(b"Invalid manifest flag set.")
         self._lm[key] = self[key], flag
 
     def get(self, key, default=None):
@@ -1049,11 +1061,10 b' class treemanifest(object):'
             self._dirs[dir].__setitem__(subpath, n)
         else:
             # manifest nodes are either 20 bytes or 32 bytes,
-            # depending on the hash in use. An extra byte is
-            # occasionally used by hg, but won't ever be
-            # persisted. Trim to 21 or 33 bytes as appropriate.
-            trim = 21 if len(n) < 25 else 33
-            self._files[f] = n[:trim]  # to match manifestdict's behavior
+            # depending on the hash in use. Assert this as historically
+            # sometimes extra bytes were added.
+            assert len(n) in (20, 32)
+            self._files[f] = n
         self._dirty = True
 
     def _load(self):
@@ -1066,6 +1077,8 b' class treemanifest(object):'
 
     def setflag(self, f, flags):
         """Set the flags (symlink, executable) for path f."""
+        if flags not in _manifestflags:
+            raise TypeError(b"Invalid manifest flag set.")
         self._load()
         dir, subpath = _splittopdir(f)
         if dir:
@@ -1599,6 +1612,7 b' class manifestrevlog(object):'
             checkambig=not bool(tree),
             mmaplargeindex=True,
             upperboundcomp=MAXCOMPRESSION,
+            persistentnodemap=opener.options.get(b'persistent-nodemap', False),
         )
 
         self.index = self._revlog.index
@@ -1664,6 +1678,22 b' class manifestrevlog(object):'
         readtree=None,
         match=None,
     ):
+        """add some manifest entry in to the manifest log
+
+        input:
+
+          m:           the manifest dict we want to store
+          transaction: the open transaction
+          p1:          manifest-node of p1
+          p2:          manifest-node of p2
+          added:       file added/changed compared to parent
+          removed:     file removed compared to parent
+
+        tree manifest input:
+
+          readtree:    a function to read a subtree
+          match:       a filematcher for the subpart of the tree manifest
+        """
         try:
             if p1 not in self.fulltextcache:
                 raise FastdeltaUnavailable()
@@ -1959,6 +1989,9 b' class manifestlog(object):'
     def rev(self, node):
         return self._rootstore.rev(node)
 
+    def update_caches(self, transaction):
+        return self._rootstore._revlog.update_caches(transaction=transaction)
+
 
 @interfaceutil.implementer(repository.imanifestrevisionwritable)
 class memmanifestctx(object):

mercurial/mdiff.py

@@ -17,6 +17,7 b' from .pycompat import ('
     setattr,
 )
 from . import (
+    diffhelper,
     encoding,
     error,
     policy,
@@ -25,8 +26,6 b' from . import ('
 )
 from .utils import dateutil
 
-_missing_newline_marker = b"\\ No newline at end of file\n"
-
 bdiff = policy.importmod('bdiff')
 mpatch = policy.importmod('mpatch')
 
@@ -309,7 +308,7 b' def unidiff(a, ad, b, bd, fn1, fn2, bina'
         hunklines = [b"@@ -0,0 +1,%d @@\n" % size] + [b"+" + e for e in b]
         if without_newline:
             hunklines[-1] += b'\n'
-            hunklines.append(_missing_newline_marker)
+            hunklines.append(diffhelper.MISSING_NEWLINE_MARKER)
         hunks = ((hunkrange, hunklines),)
     elif not b:
         without_newline = not a.endswith(b'\n')
@@ -325,7 +324,7 b' def unidiff(a, ad, b, bd, fn1, fn2, bina'
         hunklines = [b"@@ -1,%d +0,0 @@\n" % size] + [b"-" + e for e in a]
         if without_newline:
             hunklines[-1] += b'\n'
-            hunklines.append(_missing_newline_marker)
+            hunklines.append(diffhelper.MISSING_NEWLINE_MARKER)
         hunks = ((hunkrange, hunklines),)
     else:
         hunks = _unidiff(a, b, opts=opts)
@@ -418,13 +417,13 b' def _unidiff(t1, t2, opts=defaultopts):'
                     if hunklines[i].startswith(b' '):
                         skip = True
                     hunklines[i] += b'\n'
-                    hunklines.insert(i + 1, _missing_newline_marker)
+                    hunklines.insert(i + 1, diffhelper.MISSING_NEWLINE_MARKER)
                     break
         if not skip and not t2.endswith(b'\n') and bstart + blen == len(l2) + 1:
             for i in pycompat.xrange(len(hunklines) - 1, -1, -1):
                 if hunklines[i].startswith(b'+'):
                     hunklines[i] += b'\n'
-                    hunklines.insert(i + 1, _missing_newline_marker)
+                    hunklines.insert(i + 1, diffhelper.MISSING_NEWLINE_MARKER)
                     break
         yield hunkrange, hunklines
 

mercurial/merge.py

@@ -8,21 +8,16 b''
 from __future__ import absolute_import
 
 import errno
-import shutil
 import stat
 import struct
 
 from .i18n import _
 from .node import (
     addednodeid,
-    bin,
-    hex,
     modifiednodeid,
-    nullhex,
     nullid,
     nullrev,
 )
-from .pycompat import delattr
 from .thirdparty import attr
 from . import (
     copies,
@@ -30,6 +25,7 b' from . import ('
     error,
     filemerge,
     match as matchmod,
+    mergestate as mergestatemod,
     obsutil,
     pathutil,
     pycompat,
@@ -38,741 +34,11 b' from . import ('
     util,
     worker,
 )
-from .utils import hashutil
 
 _pack = struct.pack
 _unpack = struct.unpack
 
 
-def _droponode(data):
-    # used for compatibility for v1
-    bits = data.split(b'\0')
-    bits = bits[:-2] + bits[-1:]
-    return b'\0'.join(bits)
-
-
-# Merge state record types. See ``mergestate`` docs for more.
-RECORD_LOCAL = b'L'
-RECORD_OTHER = b'O'
-RECORD_MERGED = b'F'
-RECORD_CHANGEDELETE_CONFLICT = b'C'
-RECORD_MERGE_DRIVER_MERGE = b'D'
-RECORD_PATH_CONFLICT = b'P'
-RECORD_MERGE_DRIVER_STATE = b'm'
-RECORD_FILE_VALUES = b'f'
-RECORD_LABELS = b'l'
-RECORD_OVERRIDE = b't'
-RECORD_UNSUPPORTED_MANDATORY = b'X'
-RECORD_UNSUPPORTED_ADVISORY = b'x'
-RECORD_RESOLVED_OTHER = b'R'
-
-MERGE_DRIVER_STATE_UNMARKED = b'u'
-MERGE_DRIVER_STATE_MARKED = b'm'
-MERGE_DRIVER_STATE_SUCCESS = b's'
-
-MERGE_RECORD_UNRESOLVED = b'u'
-MERGE_RECORD_RESOLVED = b'r'
-MERGE_RECORD_UNRESOLVED_PATH = b'pu'
-MERGE_RECORD_RESOLVED_PATH = b'pr'
-MERGE_RECORD_DRIVER_RESOLVED = b'd'
-# represents that the file was automatically merged in favor
-# of other version. This info is used on commit.
-MERGE_RECORD_MERGED_OTHER = b'o'
-
-ACTION_FORGET = b'f'
-ACTION_REMOVE = b'r'
-ACTION_ADD = b'a'
-ACTION_GET = b'g'
-ACTION_PATH_CONFLICT = b'p'
-ACTION_PATH_CONFLICT_RESOLVE = b'pr'
-ACTION_ADD_MODIFIED = b'am'
-ACTION_CREATED = b'c'
-ACTION_DELETED_CHANGED = b'dc'
-ACTION_CHANGED_DELETED = b'cd'
-ACTION_MERGE = b'm'
-ACTION_LOCAL_DIR_RENAME_GET = b'dg'
-ACTION_DIR_RENAME_MOVE_LOCAL = b'dm'
-ACTION_KEEP = b'k'
-ACTION_EXEC = b'e'
-ACTION_CREATED_MERGE = b'cm'
-# GET the other/remote side and store this info in mergestate
-ACTION_GET_OTHER_AND_STORE = b'gs'
-
-
-class mergestate(object):
-    '''track 3-way merge state of individual files
-
-    The merge state is stored on disk when needed. Two files are used: one with
-    an old format (version 1), and one with a new format (version 2). Version 2
-    stores a superset of the data in version 1, including new kinds of records
-    in the future. For more about the new format, see the documentation for
-    `_readrecordsv2`.
-
-    Each record can contain arbitrary content, and has an associated type. This
-    `type` should be a letter. If `type` is uppercase, the record is mandatory:
-    versions of Mercurial that don't support it should abort. If `type` is
-    lowercase, the record can be safely ignored.
-
-    Currently known records:
-
-    L: the node of the "local" part of the merge (hexified version)
-    O: the node of the "other" part of the merge (hexified version)
-    F: a file to be merged entry
-    C: a change/delete or delete/change conflict
-    D: a file that the external merge driver will merge internally
-       (experimental)
-    P: a path conflict (file vs directory)
-    m: the external merge driver defined for this merge plus its run state
-       (experimental)
-    f: a (filename, dictionary) tuple of optional values for a given file
-    X: unsupported mandatory record type (used in tests)
-    x: unsupported advisory record type (used in tests)
-    l: the labels for the parts of the merge.
-
-    Merge driver run states (experimental):
-    u: driver-resolved files unmarked -- needs to be run next time we're about
-       to resolve or commit
-    m: driver-resolved files marked -- only needs to be run before commit
-    s: success/skipped -- does not need to be run any more
-
-    Merge record states (stored in self._state, indexed by filename):
-    u: unresolved conflict
-    r: resolved conflict
-    pu: unresolved path conflict (file conflicts with directory)
-    pr: resolved path conflict
-    d: driver-resolved conflict
-
-    The resolve command transitions between 'u' and 'r' for conflicts and
-    'pu' and 'pr' for path conflicts.
-    '''
-
-    statepathv1 = b'merge/state'
-    statepathv2 = b'merge/state2'
-
-    @staticmethod
-    def clean(repo, node=None, other=None, labels=None):
-        """Initialize a brand new merge state, removing any existing state on
-        disk."""
-        ms = mergestate(repo)
-        ms.reset(node, other, labels)
-        return ms
-
-    @staticmethod
-    def read(repo):
-        """Initialize the merge state, reading it from disk."""
-        ms = mergestate(repo)
-        ms._read()
-        return ms
-
-    def __init__(self, repo):
-        """Initialize the merge state.
-
-        Do not use this directly! Instead call read() or clean()."""
-        self._repo = repo
-        self._dirty = False
-        self._labels = None
-
-    def reset(self, node=None, other=None, labels=None):
-        self._state = {}
-        self._stateextras = {}
-        self._local = None
-        self._other = None
-        self._labels = labels
-        for var in ('localctx', 'otherctx'):
-            if var in vars(self):
-                delattr(self, var)
-        if node:
-            self._local = node
-            self._other = other
-        self._readmergedriver = None
-        if self.mergedriver:
-            self._mdstate = MERGE_DRIVER_STATE_SUCCESS
-        else:
-            self._mdstate = MERGE_DRIVER_STATE_UNMARKED
-        shutil.rmtree(self._repo.vfs.join(b'merge'), True)
-        self._results = {}
-        self._dirty = False
-
-    def _read(self):
-        """Analyse each record content to restore a serialized state from disk
-
-        This function process "record" entry produced by the de-serialization
-        of on disk file.
-        """
-        self._state = {}
-        self._stateextras = {}
-        self._local = None
-        self._other = None
-        for var in ('localctx', 'otherctx'):
-            if var in vars(self):
-                delattr(self, var)
-        self._readmergedriver = None
-        self._mdstate = MERGE_DRIVER_STATE_SUCCESS
-        unsupported = set()
-        records = self._readrecords()
-        for rtype, record in records:
-            if rtype == RECORD_LOCAL:
-                self._local = bin(record)
-            elif rtype == RECORD_OTHER:
-                self._other = bin(record)
-            elif rtype == RECORD_MERGE_DRIVER_STATE:
-                bits = record.split(b'\0', 1)
-                mdstate = bits[1]
-                if len(mdstate) != 1 or mdstate not in (
-                    MERGE_DRIVER_STATE_UNMARKED,
-                    MERGE_DRIVER_STATE_MARKED,
-                    MERGE_DRIVER_STATE_SUCCESS,
-                ):
-                    # the merge driver should be idempotent, so just rerun it
-                    mdstate = MERGE_DRIVER_STATE_UNMARKED
-
-                self._readmergedriver = bits[0]
-                self._mdstate = mdstate
-            elif rtype in (
-                RECORD_MERGED,
-                RECORD_CHANGEDELETE_CONFLICT,
-                RECORD_PATH_CONFLICT,
-                RECORD_MERGE_DRIVER_MERGE,
-                RECORD_RESOLVED_OTHER,
-            ):
-                bits = record.split(b'\0')
-                self._state[bits[0]] = bits[1:]
-            elif rtype == RECORD_FILE_VALUES:
-                filename, rawextras = record.split(b'\0', 1)
-                extraparts = rawextras.split(b'\0')
-                extras = {}
-                i = 0
-                while i < len(extraparts):
-                    extras[extraparts[i]] = extraparts[i + 1]
-                    i += 2
-
-                self._stateextras[filename] = extras
-            elif rtype == RECORD_LABELS:
-                labels = record.split(b'\0', 2)
-                self._labels = [l for l in labels if len(l) > 0]
-            elif not rtype.islower():
-                unsupported.add(rtype)
-        self._results = {}
-        self._dirty = False
-
-        if unsupported:
-            raise error.UnsupportedMergeRecords(unsupported)
-
-    def _readrecords(self):
-        """Read merge state from disk and return a list of record (TYPE, data)
-
-        We read data from both v1 and v2 files and decide which one to use.
-
-        V1 has been used by version prior to 2.9.1 and contains less data than
-        v2. We read both versions and check if no data in v2 contradicts
-        v1. If there is not contradiction we can safely assume that both v1
-        and v2 were written at the same time and use the extract data in v2. If
-        there is contradiction we ignore v2 content as we assume an old version
-        of Mercurial has overwritten the mergestate file and left an old v2
-        file around.
-
-        returns list of record [(TYPE, data), ...]"""
-        v1records = self._readrecordsv1()
-        v2records = self._readrecordsv2()
-        if self._v1v2match(v1records, v2records):
-            return v2records
-        else:
-            # v1 file is newer than v2 file, use it
-            # we have to infer the "other" changeset of the merge
-            # we cannot do better than that with v1 of the format
-            mctx = self._repo[None].parents()[-1]
-            v1records.append((RECORD_OTHER, mctx.hex()))
-            # add place holder "other" file node information
-            # nobody is using it yet so we do no need to fetch the data
-            # if mctx was wrong `mctx[bits[-2]]` may fails.
-            for idx, r in enumerate(v1records):
-                if r[0] == RECORD_MERGED:
-                    bits = r[1].split(b'\0')
-                    bits.insert(-2, b'')
-                    v1records[idx] = (r[0], b'\0'.join(bits))
-            return v1records
-
-    def _v1v2match(self, v1records, v2records):
-        oldv2 = set()  # old format version of v2 record
-        for rec in v2records:
-            if rec[0] == RECORD_LOCAL:
-                oldv2.add(rec)
-            elif rec[0] == RECORD_MERGED:
-                # drop the onode data (not contained in v1)
-                oldv2.add((RECORD_MERGED, _droponode(rec[1])))
-        for rec in v1records:
-            if rec not in oldv2:
-                return False
-        else:
-            return True
-
-    def _readrecordsv1(self):
-        """read on disk merge state for version 1 file
-
-        returns list of record [(TYPE, data), ...]
-
-        Note: the "F" data from this file are one entry short
-              (no "other file node" entry)
-        """
-        records = []
-        try:
-            f = self._repo.vfs(self.statepathv1)
-            for i, l in enumerate(f):
-                if i == 0:
-                    records.append((RECORD_LOCAL, l[:-1]))
-                else:
-                    records.append((RECORD_MERGED, l[:-1]))
-            f.close()
-        except IOError as err:
-            if err.errno != errno.ENOENT:
-                raise
-        return records
-
-    def _readrecordsv2(self):
-        """read on disk merge state for version 2 file
-
-        This format is a list of arbitrary records of the form:
-
-          [type][length][content]
-
-        `type` is a single character, `length` is a 4 byte integer, and
-        `content` is an arbitrary byte sequence of length `length`.
-
-        Mercurial versions prior to 3.7 have a bug where if there are
-        unsupported mandatory merge records, attempting to clear out the merge
-        state with hg update --clean or similar aborts. The 't' record type
-        works around that by writing out what those versions treat as an
-        advisory record, but later versions interpret as special: the first
-        character is the 'real' record type and everything onwards is the data.
-
-        Returns list of records [(TYPE, data), ...]."""
-        records = []
-        try:
-            f = self._repo.vfs(self.statepathv2)
-            data = f.read()
-            off = 0
-            end = len(data)
-            while off < end:
-                rtype = data[off : off + 1]
-                off += 1
-                length = _unpack(b'>I', data[off : (off + 4)])[0]
-                off += 4
-                record = data[off : (off + length)]
-                off += length
-                if rtype == RECORD_OVERRIDE:
-                    rtype, record = record[0:1], record[1:]
-                records.append((rtype, record))
-            f.close()
-        except IOError as err:
-            if err.errno != errno.ENOENT:
-                raise
-        return records
-
-    @util.propertycache
-    def mergedriver(self):
-        # protect against the following:
-        # - A configures a malicious merge driver in their hgrc, then
-        #   pauses the merge
-        # - A edits their hgrc to remove references to the merge driver
-        # - A gives a copy of their entire repo, including .hg, to B
-        # - B inspects .hgrc and finds it to be clean
-        # - B then continues the merge and the malicious merge driver
-        #  gets invoked
-        configmergedriver = self._repo.ui.config(
-            b'experimental', b'mergedriver'
-        )
-        if (
-            self._readmergedriver is not None
-            and self._readmergedriver != configmergedriver
-        ):
-            raise error.ConfigError(
-                _(b"merge driver changed since merge started"),
-                hint=_(b"revert merge driver change or abort merge"),
-            )
-
-        return configmergedriver
-
-    @util.propertycache
-    def local(self):
-        if self._local is None:
-            msg = b"local accessed but self._local isn't set"
-            raise error.ProgrammingError(msg)
-        return self._local
-
-    @util.propertycache
-    def localctx(self):
-        return self._repo[self.local]
-
-    @util.propertycache
-    def other(self):
-        if self._other is None:
-            msg = b"other accessed but self._other isn't set"
-            raise error.ProgrammingError(msg)
-        return self._other
-
-    @util.propertycache
-    def otherctx(self):
-        return self._repo[self.other]
-
-    def active(self):
-        """Whether mergestate is active.
-
-        Returns True if there appears to be mergestate. This is a rough proxy
-        for "is a merge in progress."
-        """
-        return bool(self._local) or bool(self._state)
-
-    def commit(self):
-        """Write current state on disk (if necessary)"""
-        if self._dirty:
-            records = self._makerecords()
-            self._writerecords(records)
-            self._dirty = False
-
-    def _makerecords(self):
-        records = []
-        records.append((RECORD_LOCAL, hex(self._local)))
-        records.append((RECORD_OTHER, hex(self._other)))
-        if self.mergedriver:
-            records.append(
-                (
-                    RECORD_MERGE_DRIVER_STATE,
-                    b'\0'.join([self.mergedriver, self._mdstate]),
-                )
-            )
-        # Write out state items. In all cases, the value of the state map entry
-        # is written as the contents of the record. The record type depends on
-        # the type of state that is stored, and capital-letter records are used
-        # to prevent older versions of Mercurial that do not support the feature
-        # from loading them.
-        for filename, v in pycompat.iteritems(self._state):
-            if v[0] == MERGE_RECORD_DRIVER_RESOLVED:
-                # Driver-resolved merge. These are stored in 'D' records.
-                records.append(
-                    (RECORD_MERGE_DRIVER_MERGE, b'\0'.join([filename] + v))
-                )
-            elif v[0] in (
-                MERGE_RECORD_UNRESOLVED_PATH,
-                MERGE_RECORD_RESOLVED_PATH,
-            ):
-                # Path conflicts. These are stored in 'P' records.  The current
-                # resolution state ('pu' or 'pr') is stored within the record.
-                records.append(
-                    (RECORD_PATH_CONFLICT, b'\0'.join([filename] + v))
-                )
-            elif v[0] == MERGE_RECORD_MERGED_OTHER:
-                records.append(
-                    (RECORD_RESOLVED_OTHER, b'\0'.join([filename] + v))
-                )
-            elif v[1] == nullhex or v[6] == nullhex:
-                # Change/Delete or Delete/Change conflicts. These are stored in
-                # 'C' records. v[1] is the local file, and is nullhex when the
-                # file is deleted locally ('dc'). v[6] is the remote file, and
-                # is nullhex when the file is deleted remotely ('cd').
-                records.append(
-                    (RECORD_CHANGEDELETE_CONFLICT, b'\0'.join([filename] + v))
-                )
-            else:
-                # Normal files.  These are stored in 'F' records.
-                records.append((RECORD_MERGED, b'\0'.join([filename] + v)))
-        for filename, extras in sorted(pycompat.iteritems(self._stateextras)):
-            rawextras = b'\0'.join(
-                b'%s\0%s' % (k, v) for k, v in pycompat.iteritems(extras)
-            )
-            records.append(
-                (RECORD_FILE_VALUES, b'%s\0%s' % (filename, rawextras))
-            )
-        if self._labels is not None:
-            labels = b'\0'.join(self._labels)
-            records.append((RECORD_LABELS, labels))
-        return records
-
-    def _writerecords(self, records):
-        """Write current state on disk (both v1 and v2)"""
-        self._writerecordsv1(records)
-        self._writerecordsv2(records)
-
-    def _writerecordsv1(self, records):
-        """Write current state on disk in a version 1 file"""
-        f = self._repo.vfs(self.statepathv1, b'wb')
-        irecords = iter(records)
-        lrecords = next(irecords)
-        assert lrecords[0] == RECORD_LOCAL
-        f.write(hex(self._local) + b'\n')
-        for rtype, data in irecords:
-            if rtype == RECORD_MERGED:
-                f.write(b'%s\n' % _droponode(data))
-        f.close()
-
-    def _writerecordsv2(self, records):
-        """Write current state on disk in a version 2 file
-
-        See the docstring for _readrecordsv2 for why we use 't'."""
-        # these are the records that all version 2 clients can read
-        allowlist = (RECORD_LOCAL, RECORD_OTHER, RECORD_MERGED)
-        f = self._repo.vfs(self.statepathv2, b'wb')
-        for key, data in records:
-            assert len(key) == 1
-            if key not in allowlist:
-                key, data = RECORD_OVERRIDE, b'%s%s' % (key, data)
-            format = b'>sI%is' % len(data)
-            f.write(_pack(format, key, len(data), data))
-        f.close()
-
-    @staticmethod
-    def getlocalkey(path):
-        """hash the path of a local file context for storage in the .hg/merge
-        directory."""
-
-        return hex(hashutil.sha1(path).digest())
-
-    def add(self, fcl, fco, fca, fd):
-        """add a new (potentially?) conflicting file the merge state
-        fcl: file context for local,
-        fco: file context for remote,
-        fca: file context for ancestors,
-        fd:  file path of the resulting merge.
-
-        note: also write the local version to the `.hg/merge` directory.
-        """
-        if fcl.isabsent():
-            localkey = nullhex
-        else:
-            localkey = mergestate.getlocalkey(fcl.path())
-            self._repo.vfs.write(b'merge/' + localkey, fcl.data())
-        self._state[fd] = [
-            MERGE_RECORD_UNRESOLVED,
-            localkey,
-            fcl.path(),
-            fca.path(),
-            hex(fca.filenode()),
-            fco.path(),
-            hex(fco.filenode()),
-            fcl.flags(),
-        ]
-        self._stateextras[fd] = {b'ancestorlinknode': hex(fca.node())}
-        self._dirty = True
-
-    def addpath(self, path, frename, forigin):
-        """add a new conflicting path to the merge state
-        path:    the path that conflicts
-        frename: the filename the conflicting file was renamed to
-        forigin: origin of the file ('l' or 'r' for local/remote)
-        """
-        self._state[path] = [MERGE_RECORD_UNRESOLVED_PATH, frename, forigin]
-        self._dirty = True
-
-    def addmergedother(self, path):
-        self._state[path] = [MERGE_RECORD_MERGED_OTHER, nullhex, nullhex]
-        self._dirty = True
-
-    def __contains__(self, dfile):
-        return dfile in self._state
-
-    def __getitem__(self, dfile):
-        return self._state[dfile][0]
-
-    def __iter__(self):
-        return iter(sorted(self._state))
-
-    def files(self):
-        return self._state.keys()
-
-    def mark(self, dfile, state):
-        self._state[dfile][0] = state
-        self._dirty = True
-
-    def mdstate(self):
-        return self._mdstate
-
-    def unresolved(self):
-        """Obtain the paths of unresolved files."""
-
-        for f, entry in pycompat.iteritems(self._state):
-            if entry[0] in (
-                MERGE_RECORD_UNRESOLVED,
-                MERGE_RECORD_UNRESOLVED_PATH,
-            ):
-                yield f
-
-    def driverresolved(self):
-        """Obtain the paths of driver-resolved files."""
-
-        for f, entry in self._state.items():
-            if entry[0] == MERGE_RECORD_DRIVER_RESOLVED:
-                yield f
-
-    def extras(self, filename):
-        return self._stateextras.setdefault(filename, {})
-
-    def _resolve(self, preresolve, dfile, wctx):
-        """rerun merge process for file path `dfile`"""
-        if self[dfile] in (MERGE_RECORD_RESOLVED, MERGE_RECORD_DRIVER_RESOLVED):
-            return True, 0
-        if self._state[dfile][0] == MERGE_RECORD_MERGED_OTHER:
-            return True, 0
-        stateentry = self._state[dfile]
-        state, localkey, lfile, afile, anode, ofile, onode, flags = stateentry
-        octx = self._repo[self._other]
-        extras = self.extras(dfile)
-        anccommitnode = extras.get(b'ancestorlinknode')
-        if anccommitnode:
-            actx = self._repo[anccommitnode]
-        else:
-            actx = None
-        fcd = self._filectxorabsent(localkey, wctx, dfile)
-        fco = self._filectxorabsent(onode, octx, ofile)
-        # TODO: move this to filectxorabsent
-        fca = self._repo.filectx(afile, fileid=anode, changectx=actx)
-        # "premerge" x flags
-        flo = fco.flags()
-        fla = fca.flags()
-        if b'x' in flags + flo + fla and b'l' not in flags + flo + fla:
-            if fca.node() == nullid and flags != flo:
-                if preresolve:
-                    self._repo.ui.warn(
-                        _(
-                            b'warning: cannot merge flags for %s '
-                            b'without common ancestor - keeping local flags\n'
-                        )
-                        % afile
-                    )
-            elif flags == fla:
-                flags = flo
-        if preresolve:
-            # restore local
-            if localkey != nullhex:
-                f = self._repo.vfs(b'merge/' + localkey)
-                wctx[dfile].write(f.read(), flags)
-                f.close()
-            else:
-                wctx[dfile].remove(ignoremissing=True)
-            complete, r, deleted = filemerge.premerge(
-                self._repo,
-                wctx,
-                self._local,
-                lfile,
-                fcd,
-                fco,
-                fca,
-                labels=self._labels,
-            )
-        else:
-            complete, r, deleted = filemerge.filemerge(
-                self._repo,
-                wctx,
-                self._local,
-                lfile,
-                fcd,
-                fco,
-                fca,
-                labels=self._labels,
-            )
-        if r is None:
-            # no real conflict
-            del self._state[dfile]
-            self._stateextras.pop(dfile, None)
-            self._dirty = True
-        elif not r:
-            self.mark(dfile, MERGE_RECORD_RESOLVED)
-
-        if complete:
-            action = None
-            if deleted:
-                if fcd.isabsent():
-                    # dc: local picked. Need to drop if present, which may
-                    # happen on re-resolves.
-                    action = ACTION_FORGET
-                else:
-                    # cd: remote picked (or otherwise deleted)
-                    action = ACTION_REMOVE
-            else:
-                if fcd.isabsent():  # dc: remote picked
-                    action = ACTION_GET
-                elif fco.isabsent():  # cd: local picked
-                    if dfile in self.localctx:
-                        action = ACTION_ADD_MODIFIED
-                    else:
-                        action = ACTION_ADD
-                # else: regular merges (no action necessary)
-            self._results[dfile] = r, action
-
-        return complete, r
-
-    def _filectxorabsent(self, hexnode, ctx, f):
-        if hexnode == nullhex:
-            return filemerge.absentfilectx(ctx, f)
-        else:
-            return ctx[f]
-
-    def preresolve(self, dfile, wctx):
-        """run premerge process for dfile
-
-        Returns whether the merge is complete, and the exit code."""
-        return self._resolve(True, dfile, wctx)
-
-    def resolve(self, dfile, wctx):
-        """run merge process (assuming premerge was run) for dfile
-
-        Returns the exit code of the merge."""
-        return self._resolve(False, dfile, wctx)[1]
-
-    def counts(self):
-        """return counts for updated, merged and removed files in this
-        session"""
-        updated, merged, removed = 0, 0, 0
-        for r, action in pycompat.itervalues(self._results):
-            if r is None:
-                updated += 1
-            elif r == 0:
-                if action == ACTION_REMOVE:
-                    removed += 1
-                else:
-                    merged += 1
-        return updated, merged, removed
-
-    def unresolvedcount(self):
-        """get unresolved count for this merge (persistent)"""
-        return len(list(self.unresolved()))
-
-    def actions(self):
-        """return lists of actions to perform on the dirstate"""
-        actions = {
-            ACTION_REMOVE: [],
-            ACTION_FORGET: [],
-            ACTION_ADD: [],
-            ACTION_ADD_MODIFIED: [],
-            ACTION_GET: [],
-        }
-        for f, (r, action) in pycompat.iteritems(self._results):
-            if action is not None:
-                actions[action].append((f, None, b"merge result"))
-        return actions
-
-    def recordactions(self):
-        """record remove/add/get actions in the dirstate"""
-        branchmerge = self._repo.dirstate.p2() != nullid
-        recordupdates(self._repo, self.actions(), branchmerge, None)
-
-    def queueremove(self, f):
-        """queues a file to be removed from the dirstate
-
-        Meant for use by custom merge drivers."""
-        self._results[f] = 0, ACTION_REMOVE
-
-    def queueadd(self, f):
-        """queues a file to be added to the dirstate
-
-        Meant for use by custom merge drivers."""
-        self._results[f] = 0, ACTION_ADD
-
-    def queueget(self, f):
-        """queues a file to be marked modified in the dirstate
-
-        Meant for use by custom merge drivers."""
-        self._results[f] = 0, ACTION_GET
-
-
 def _getcheckunknownconfig(repo, section, name):
     config = repo.ui.config(section, name)
     valid = [b'abort', b'ignore', b'warn']
@@ -885,14 +151,17 b' def _checkunknownfiles(repo, wctx, mctx,'
 
         checkunknowndirs = _unknowndirschecker()
         for f, (m, args, msg) in pycompat.iteritems(actions):
-            if m in (ACTION_CREATED, ACTION_DELETED_CHANGED):
+            if m in (
+                mergestatemod.ACTION_CREATED,
+                mergestatemod.ACTION_DELETED_CHANGED,
+            ):
                 if _checkunknownfile(repo, wctx, mctx, f):
                     fileconflicts.add(f)
                 elif pathconfig and f not in wctx:
                     path = checkunknowndirs(repo, wctx, f)
                     if path is not None:
                         pathconflicts.add(path)
-            elif m == ACTION_LOCAL_DIR_RENAME_GET:
+            elif m == mergestatemod.ACTION_LOCAL_DIR_RENAME_GET:
                 if _checkunknownfile(repo, wctx, mctx, f, args[0]):
                     fileconflicts.add(f)
 
@@ -903,7 +172,7 b' def _checkunknownfiles(repo, wctx, mctx,'
         collectconflicts(unknownconflicts, unknownconfig)
     else:
         for f, (m, args, msg) in pycompat.iteritems(actions):
-            if m == ACTION_CREATED_MERGE:
+            if m == mergestatemod.ACTION_CREATED_MERGE:
                 fl2, anc = args
                 different = _checkunknownfile(repo, wctx, mctx, f)
                 if repo.dirstate._ignore(f):
@@ -924,10 +193,14 b' def _checkunknownfiles(repo, wctx, mctx,'
                 #     don't like an abort happening in the middle of
                 #     merge.update.
                 if not different:
-                    actions[f] = (ACTION_GET, (fl2, False), b'remote created')
+                    actions[f] = (
+                        mergestatemod.ACTION_GET,
+                        (fl2, False),
+                        b'remote created',
+                    )
                 elif mergeforce or config == b'abort':
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f, f, None, False, anc),
                         b'remote differs from untracked local',
                     )
@@ -936,7 +209,11 b' def _checkunknownfiles(repo, wctx, mctx,'
                 else:
                     if config == b'warn':
                         warnconflicts.add(f)
-                    actions[f] = (ACTION_GET, (fl2, True), b'remote created')
+                    actions[f] = (
+                        mergestatemod.ACTION_GET,
+                        (fl2, True),
+                        b'remote created',
+                    )
 
     for f in sorted(abortconflicts):
         warn = repo.ui.warn
@@ -962,14 +239,14 b' def _checkunknownfiles(repo, wctx, mctx,'
             repo.ui.warn(_(b"%s: replacing untracked files in directory\n") % f)
 
     for f, (m, args, msg) in pycompat.iteritems(actions):
-        if m == ACTION_CREATED:
+        if m == mergestatemod.ACTION_CREATED:
             backup = (
                 f in fileconflicts
                 or f in pathconflicts
                 or any(p in pathconflicts for p in pathutil.finddirs(f))
             )
             (flags,) = args
-            actions[f] = (ACTION_GET, (flags, backup), msg)
+            actions[f] = (mergestatemod.ACTION_GET, (flags, backup), msg)
 
 
 def _forgetremoved(wctx, mctx, branchmerge):
@@ -988,9 +265,9 b' def _forgetremoved(wctx, mctx, branchmer'
     """
 
     actions = {}
-    m = ACTION_FORGET
+    m = mergestatemod.ACTION_FORGET
     if branchmerge:
-        m = ACTION_REMOVE
+        m = mergestatemod.ACTION_REMOVE
     for f in wctx.deleted():
         if f not in mctx:
             actions[f] = m, None, b"forget deleted"
@@ -998,7 +275,11 b' def _forgetremoved(wctx, mctx, branchmer'
     if not branchmerge:
         for f in wctx.removed():
             if f not in mctx:
-                actions[f] = ACTION_FORGET, None, b"forget removed"
+                actions[f] = (
+                    mergestatemod.ACTION_FORGET,
+                    None,
+                    b"forget removed",
+                )
 
     return actions
 
@@ -1026,24 +307,24 b' def _checkcollision(repo, wmf, actions):'
     if actions:
         # KEEP and EXEC are no-op
         for m in (
-            ACTION_ADD,
-            ACTION_ADD_MODIFIED,
-            ACTION_FORGET,
-            ACTION_GET,
-            ACTION_CHANGED_DELETED,
-            ACTION_DELETED_CHANGED,
+            mergestatemod.ACTION_ADD,
+            mergestatemod.ACTION_ADD_MODIFIED,
+            mergestatemod.ACTION_FORGET,
+            mergestatemod.ACTION_GET,
+            mergestatemod.ACTION_CHANGED_DELETED,
+            mergestatemod.ACTION_DELETED_CHANGED,
         ):
             for f, args, msg in actions[m]:
                 pmmf.add(f)
-        for f, args, msg in actions[ACTION_REMOVE]:
+        for f, args, msg in actions[mergestatemod.ACTION_REMOVE]:
             pmmf.discard(f)
-        for f, args, msg in actions[ACTION_DIR_RENAME_MOVE_LOCAL]:
+        for f, args, msg in actions[mergestatemod.ACTION_DIR_RENAME_MOVE_LOCAL]:
             f2, flags = args
             pmmf.discard(f2)
             pmmf.add(f)
-        for f, args, msg in actions[ACTION_LOCAL_DIR_RENAME_GET]:
+        for f, args, msg in actions[mergestatemod.ACTION_LOCAL_DIR_RENAME_GET]:
             pmmf.add(f)
-        for f, args, msg in actions[ACTION_MERGE]:
+        for f, args, msg in actions[mergestatemod.ACTION_MERGE]:
             f1, f2, fa, move, anc = args
             if move:
                 pmmf.discard(f1)
@@ -1128,10 +409,10 b' def checkpathconflicts(repo, wctx, mctx,'
 
     for f, (m, args, msg) in actions.items():
         if m in (
-            ACTION_CREATED,
-            ACTION_DELETED_CHANGED,
-            ACTION_MERGE,
-            ACTION_CREATED_MERGE,
+            mergestatemod.ACTION_CREATED,
+            mergestatemod.ACTION_DELETED_CHANGED,
+            mergestatemod.ACTION_MERGE,
+            mergestatemod.ACTION_CREATED_MERGE,
         ):
             # This action may create a new local file.
             createdfiledirs.update(pathutil.finddirs(f))
@@ -1141,13 +422,13 b' def checkpathconflicts(repo, wctx, mctx,'
                 # will be checked once we know what all the deleted files are.
                 remoteconflicts.add(f)
         # Track the names of all deleted files.
-        if m == ACTION_REMOVE:
+        if m == mergestatemod.ACTION_REMOVE:
             deletedfiles.add(f)
-        if m == ACTION_MERGE:
+        if m == mergestatemod.ACTION_MERGE:
             f1, f2, fa, move, anc = args
             if move:
                 deletedfiles.add(f1)
-        if m == ACTION_DIR_RENAME_MOVE_LOCAL:
+        if m == mergestatemod.ACTION_DIR_RENAME_MOVE_LOCAL:
             f2, flags = args
             deletedfiles.add(f2)
 
@@ -1164,10 +445,10 b' def checkpathconflicts(repo, wctx, mctx,'
                 # We will need to rename the local file.
                 localconflicts.add(p)
         if p in actions and actions[p][0] in (
-            ACTION_CREATED,
-            ACTION_DELETED_CHANGED,
-            ACTION_MERGE,
-            ACTION_CREATED_MERGE,
+            mergestatemod.ACTION_CREATED,
+            mergestatemod.ACTION_DELETED_CHANGED,
+            mergestatemod.ACTION_MERGE,
+            mergestatemod.ACTION_CREATED_MERGE,
         ):
             # The file is in a directory which aliases a remote file.
             # This is an internal inconsistency within the remote
@@ -1179,12 +460,17 b' def checkpathconflicts(repo, wctx, mctx,'
         if p not in deletedfiles:
             ctxname = bytes(wctx).rstrip(b'+')
             pnew = util.safename(p, ctxname, wctx, set(actions.keys()))
+            porig = wctx[p].copysource() or p
             actions[pnew] = (
-                ACTION_PATH_CONFLICT_RESOLVE,
-                (p,),
+                mergestatemod.ACTION_PATH_CONFLICT_RESOLVE,
+                (p, porig),
                 b'local path conflict',
             )
-            actions[p] = (ACTION_PATH_CONFLICT, (pnew, b'l'), b'path conflict')
+            actions[p] = (
+                mergestatemod.ACTION_PATH_CONFLICT,
+                (pnew, b'l'),
+                b'path conflict',
+            )
 
     if remoteconflicts:
         # Check if all files in the conflicting directories have been removed.
@@ -1193,20 +479,23 b' def checkpathconflicts(repo, wctx, mctx,'
             if f not in deletedfiles:
                 m, args, msg = actions[p]
                 pnew = util.safename(p, ctxname, wctx, set(actions.keys()))
-                if m in (ACTION_DELETED_CHANGED, ACTION_MERGE):
+                if m in (
+                    mergestatemod.ACTION_DELETED_CHANGED,
+                    mergestatemod.ACTION_MERGE,
+                ):
                     # Action was merge, just update target.
                     actions[pnew] = (m, args, msg)
                 else:
                     # Action was create, change to renamed get action.
                     fl = args[0]
                     actions[pnew] = (
-                        ACTION_LOCAL_DIR_RENAME_GET,
+                        mergestatemod.ACTION_LOCAL_DIR_RENAME_GET,
                         (p, fl),
                         b'remote path conflict',
                     )
                 actions[p] = (
-                    ACTION_PATH_CONFLICT,
-                    (pnew, ACTION_REMOVE),
+                    mergestatemod.ACTION_PATH_CONFLICT,
+                    (pnew, mergestatemod.ACTION_REMOVE),
                     b'path conflict',
                 )
                 remoteconflicts.remove(p)
@@ -1269,6 +558,13 b' def manifestmerge('
     branchmerge and force are as passed in to update
     matcher = matcher to filter file lists
     acceptremote = accept the incoming changes without prompting
+
+    Returns:
+
+    actions: dict of filename as keys and action related info as values
+    diverge: mapping of source name -> list of dest name for divergent renames
+    renamedelete: mapping of source name -> list of destinations for files
+                  deleted on one side and renamed on other.
     """
     if matcher is not None and matcher.always():
         matcher = None
@@ -1340,13 +636,13 b' def manifestmerge('
                 ) or branch_copies2.copy.get(f, None)
                 if fa is not None:
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f, f, fa, False, pa.node()),
                         b'both renamed from %s' % fa,
                     )
                 else:
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f, f, None, False, pa.node()),
                         b'both created',
                     )
@@ -1355,35 +651,43 b' def manifestmerge('
                 fla = ma.flags(f)
                 nol = b'l' not in fl1 + fl2 + fla
                 if n2 == a and fl2 == fla:
-                    actions[f] = (ACTION_KEEP, (), b'remote unchanged')
+                    actions[f] = (
+                        mergestatemod.ACTION_KEEP,
+                        (),
+                        b'remote unchanged',
+                    )
                 elif n1 == a and fl1 == fla:  # local unchanged - use remote
                     if n1 == n2:  # optimization: keep local content
                         actions[f] = (
-                            ACTION_EXEC,
+                            mergestatemod.ACTION_EXEC,
                             (fl2,),
                             b'update permissions',
                         )
                     else:
                         actions[f] = (
-                            ACTION_GET_OTHER_AND_STORE
+                            mergestatemod.ACTION_GET_OTHER_AND_STORE
                             if branchmerge
-                            else ACTION_GET,
+                            else mergestatemod.ACTION_GET,
                             (fl2, False),
                             b'remote is newer',
                         )
                 elif nol and n2 == a:  # remote only changed 'x'
-                    actions[f] = (ACTION_EXEC, (fl2,), b'update permissions')
+                    actions[f] = (
+                        mergestatemod.ACTION_EXEC,
+                        (fl2,),
+                        b'update permissions',
+                    )
                 elif nol and n1 == a:  # local only changed 'x'
                     actions[f] = (
-                        ACTION_GET_OTHER_AND_STORE
+                        mergestatemod.ACTION_GET_OTHER_AND_STORE
                         if branchmerge
-                        else ACTION_GET,
+                        else mergestatemod.ACTION_GET,
                         (fl1, False),
                         b'remote is newer',
                     )
                 else:  # both changed something
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f, f, f, False, pa.node()),
                         b'versions differ',
                     )
@@ -1396,40 +700,51 b' def manifestmerge('
                 f2 = branch_copies1.movewithdir[f]
                 if f2 in m2:
                     actions[f2] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f, f2, None, True, pa.node()),
                         b'remote directory rename, both created',
                     )
                 else:
                     actions[f2] = (
-                        ACTION_DIR_RENAME_MOVE_LOCAL,
+                        mergestatemod.ACTION_DIR_RENAME_MOVE_LOCAL,
                         (f, fl1),
                         b'remote directory rename - move from %s' % f,
                     )
             elif f in branch_copies1.copy:
                 f2 = branch_copies1.copy[f]
                 actions[f] = (
-                    ACTION_MERGE,
+                    mergestatemod.ACTION_MERGE,
                     (f, f2, f2, False, pa.node()),
                     b'local copied/moved from %s' % f2,
                 )
             elif f in ma:  # clean, a different, no remote
                 if n1 != ma[f]:
                     if acceptremote:
-                        actions[f] = (ACTION_REMOVE, None, b'remote delete')
+                        actions[f] = (
+                            mergestatemod.ACTION_REMOVE,
+                            None,
+                            b'remote delete',
+                        )
                     else:
                         actions[f] = (
-                            ACTION_CHANGED_DELETED,
+                            mergestatemod.ACTION_CHANGED_DELETED,
                             (f, None, f, False, pa.node()),
                             b'prompt changed/deleted',
                         )
                 elif n1 == addednodeid:
-                    # This extra 'a' is added by working copy manifest to mark
-                    # the file as locally added. We should forget it instead of
+                    # This file was locally added. We should forget it instead of
                     # deleting it.
-                    actions[f] = (ACTION_FORGET, None, b'remote deleted')
+                    actions[f] = (
+                        mergestatemod.ACTION_FORGET,
+                        None,
+                        b'remote deleted',
+                    )
                 else:
-                    actions[f] = (ACTION_REMOVE, None, b'other deleted')
+                    actions[f] = (
+                        mergestatemod.ACTION_REMOVE,
+                        None,
+                        b'other deleted',
+                    )
         elif n2:  # file exists only on remote side
             if f in copied1:
                 pass  # we'll deal with it on m1 side
@@ -1437,13 +752,13 b' def manifestmerge('
                 f2 = branch_copies2.movewithdir[f]
                 if f2 in m1:
                     actions[f2] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f2, f, None, False, pa.node()),
                         b'local directory rename, both created',
                     )
                 else:
                     actions[f2] = (
-                        ACTION_LOCAL_DIR_RENAME_GET,
+                        mergestatemod.ACTION_LOCAL_DIR_RENAME_GET,
                         (f, fl2),
                         b'local directory rename - get from %s' % f,
                     )
@@ -1451,13 +766,13 b' def manifestmerge('
                 f2 = branch_copies2.copy[f]
                 if f2 in m2:
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f2, f, f2, False, pa.node()),
                         b'remote copied from %s' % f2,
                     )
                 else:
                     actions[f] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (f2, f, f2, True, pa.node()),
                         b'remote moved from %s' % f2,
                     )
@@ -1474,12 +789,20 b' def manifestmerge('
                 # Checking whether the files are different is expensive, so we
                 # don't do that when we can avoid it.
                 if not force:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote created')
+                    actions[f] = (
+                        mergestatemod.ACTION_CREATED,
+                        (fl2,),
+                        b'remote created',
+                    )
                 elif not branchmerge:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote created')
+                    actions[f] = (
+                        mergestatemod.ACTION_CREATED,
+                        (fl2,),
+                        b'remote created',
+                    )
                 else:
                     actions[f] = (
-                        ACTION_CREATED_MERGE,
+                        mergestatemod.ACTION_CREATED_MERGE,
                         (fl2, pa.node()),
                         b'remote created, get or merge',
                     )
@@ -1492,16 +815,20 b' def manifestmerge('
                         break
                 if df is not None and df in m1:
                     actions[df] = (
-                        ACTION_MERGE,
+                        mergestatemod.ACTION_MERGE,
                         (df, f, f, False, pa.node()),
                         b'local directory rename - respect move '
                         b'from %s' % f,
                     )
                 elif acceptremote:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote recreating')
+                    actions[f] = (
+                        mergestatemod.ACTION_CREATED,
+                        (fl2,),
+                        b'remote recreating',
+                    )
                 else:
                     actions[f] = (
-                        ACTION_DELETED_CHANGED,
+                        mergestatemod.ACTION_DELETED_CHANGED,
                         (None, f, f, False, pa.node()),
                         b'prompt deleted/changed',
                     )
@@ -1528,14 +855,14 b' def _resolvetrivial(repo, wctx, mctx, an'
     # actions as we resolve trivial conflicts.
     for f, (m, args, msg) in list(actions.items()):
         if (
-            m == ACTION_CHANGED_DELETED
+            m == mergestatemod.ACTION_CHANGED_DELETED
             and f in ancestor
             and not wctx[f].cmp(ancestor[f])
         ):
             # local did change but ended up with same content
-            actions[f] = ACTION_REMOVE, None, b'prompt same'
+            actions[f] = mergestatemod.ACTION_REMOVE, None, b'prompt same'
         elif (
-            m == ACTION_DELETED_CHANGED
+            m == mergestatemod.ACTION_DELETED_CHANGED
             and f in ancestor
             and not mctx[f].cmp(ancestor[f])
         ):
@@ -1555,7 +882,17 b' def calculateupdates('
     matcher=None,
     mergeforce=False,
 ):
-    """Calculate the actions needed to merge mctx into wctx using ancestors"""
+    """
+    Calculate the actions needed to merge mctx into wctx using ancestors
+
+    Uses manifestmerge() to merge manifest and get list of actions required to
+    perform for merging two manifests. If there are multiple ancestors, uses bid
+    merge if enabled.
+
+    Also filters out actions which are unrequired if repository is sparse.
+
+    Returns same 3 element tuple as manifestmerge().
+    """
     # Avoid cycle.
     from . import sparse
 
@@ -1613,8 +950,8 b' def calculateupdates('
 
             for f, a in sorted(pycompat.iteritems(actions)):
                 m, args, msg = a
-                if m == ACTION_GET_OTHER_AND_STORE:
-                    m = ACTION_GET
+                if m == mergestatemod.ACTION_GET_OTHER_AND_STORE:
+                    m = mergestatemod.ACTION_GET
                 repo.ui.debug(b' %s: %s -> %s\n' % (f, msg, m))
                 if f in fbids:
                     d = fbids[f]
@@ -1638,14 +975,14 b' def calculateupdates('
                     actions[f] = l[0]
                     continue
             # If keep is an option, just do it.
-            if ACTION_KEEP in bids:
+            if mergestatemod.ACTION_KEEP in bids:
                 repo.ui.note(_(b" %s: picking 'keep' action\n") % f)
-                actions[f] = bids[ACTION_KEEP][0]
+                actions[f] = bids[mergestatemod.ACTION_KEEP][0]
                 continue
             # If there are gets and they all agree [how could they not?], do it.
-            if ACTION_GET in bids:
-                ga0 = bids[ACTION_GET][0]
-                if all(a == ga0 for a in bids[ACTION_GET][1:]):
+            if mergestatemod.ACTION_GET in bids:
+                ga0 = bids[mergestatemod.ACTION_GET][0]
+                if all(a == ga0 for a in bids[mergestatemod.ACTION_GET][1:]):
                     repo.ui.note(_(b" %s: picking 'get' action\n") % f)
                     actions[f] = ga0
                     continue
@@ -1790,18 +1127,24 b' def _prefetchfiles(repo, ctx, actions):'
     oplist = [
         actions[a]
         for a in (
-            ACTION_GET,
-            ACTION_DELETED_CHANGED,
-            ACTION_LOCAL_DIR_RENAME_GET,
-            ACTION_MERGE,
+            mergestatemod.ACTION_GET,
+            mergestatemod.ACTION_DELETED_CHANGED,
+            mergestatemod.ACTION_LOCAL_DIR_RENAME_GET,
+            mergestatemod.ACTION_MERGE,
         )
     ]
     prefetch = scmutil.prefetchfiles
     matchfiles = scmutil.matchfiles
     prefetch(
         repo,
-        [ctx.rev()],
-        matchfiles(repo, [f for sublist in oplist for f, args, msg in sublist]),
+        [
+            (
+                ctx.rev(),
+                matchfiles(
+                    repo, [f for sublist in oplist for f, args, msg in sublist]
+                ),
+            )
+        ],
     )
 
 
@@ -1826,21 +1169,21 b' def emptyactions():'
     return {
         m: []
         for m in (
-            ACTION_ADD,
-            ACTION_ADD_MODIFIED,
-            ACTION_FORGET,
-            ACTION_GET,
-            ACTION_CHANGED_DELETED,
-            ACTION_DELETED_CHANGED,
-            ACTION_REMOVE,
-            ACTION_DIR_RENAME_MOVE_LOCAL,
-            ACTION_LOCAL_DIR_RENAME_GET,
-            ACTION_MERGE,
-            ACTION_EXEC,
-            ACTION_KEEP,
-            ACTION_PATH_CONFLICT,
-            ACTION_PATH_CONFLICT_RESOLVE,
-            ACTION_GET_OTHER_AND_STORE,
+            mergestatemod.ACTION_ADD,
+            mergestatemod.ACTION_ADD_MODIFIED,
+            mergestatemod.ACTION_FORGET,
+            mergestatemod.ACTION_GET,
+            mergestatemod.ACTION_CHANGED_DELETED,
+            mergestatemod.ACTION_DELETED_CHANGED,
+            mergestatemod.ACTION_REMOVE,
+            mergestatemod.ACTION_DIR_RENAME_MOVE_LOCAL,
+            mergestatemod.ACTION_LOCAL_DIR_RENAME_GET,
+            mergestatemod.ACTION_MERGE,
+            mergestatemod.ACTION_EXEC,
+            mergestatemod.ACTION_KEEP,
+            mergestatemod.ACTION_PATH_CONFLICT,
+            mergestatemod.ACTION_PATH_CONFLICT_RESOLVE,
+            mergestatemod.ACTION_GET_OTHER_AND_STORE,
         )
     }
 
@@ -1862,10 +1205,12 b' def applyupdates('
     _prefetchfiles(repo, mctx, actions)
 
     updated, merged, removed = 0, 0, 0
-    ms = mergestate.clean(repo, wctx.p1().node(), mctx.node(), labels)
+    ms = mergestatemod.mergestate.clean(
+        repo, wctx.p1().node(), mctx.node(), labels
+    )
 
     # add ACTION_GET_OTHER_AND_STORE to mergestate
-    for e in actions[ACTION_GET_OTHER_AND_STORE]:
+    for e in actions[mergestatemod.ACTION_GET_OTHER_AND_STORE]:
         ms.addmergedother(e[0])
 
     moves = []
@@ -1873,9 +1218,9 b' def applyupdates('
         l.sort()
 
     # 'cd' and 'dc' actions are treated like other merge conflicts
-    mergeactions = sorted(actions[ACTION_CHANGED_DELETED])
-    mergeactions.extend(sorted(actions[ACTION_DELETED_CHANGED]))
-    mergeactions.extend(actions[ACTION_MERGE])
+    mergeactions = sorted(actions[mergestatemod.ACTION_CHANGED_DELETED])
+    mergeactions.extend(sorted(actions[mergestatemod.ACTION_DELETED_CHANGED]))
+    mergeactions.extend(actions[mergestatemod.ACTION_MERGE])
     for f, args, msg in mergeactions:
         f1, f2, fa, move, anc = args
         if f == b'.hgsubstate':  # merged internally
@@ -1906,16 +1251,22 b' def applyupdates('
             wctx[f].audit()
             wctx[f].remove()
 
-    numupdates = sum(len(l) for m, l in actions.items() if m != ACTION_KEEP)
+    numupdates = sum(
+        len(l) for m, l in actions.items() if m != mergestatemod.ACTION_KEEP
+    )
     progress = repo.ui.makeprogress(
         _(b'updating'), unit=_(b'files'), total=numupdates
     )
 
-    if [a for a in actions[ACTION_REMOVE] if a[0] == b'.hgsubstate']:
+    if [
+        a
+        for a in actions[mergestatemod.ACTION_REMOVE]
+        if a[0] == b'.hgsubstate'
+    ]:
         subrepoutil.submerge(repo, wctx, mctx, wctx, overwrite, labels)
 
     # record path conflicts
-    for f, args, msg in actions[ACTION_PATH_CONFLICT]:
+    for f, args, msg in actions[mergestatemod.ACTION_PATH_CONFLICT]:
         f1, fo = args
         s = repo.ui.status
         s(
@@ -1930,7 +1281,7 b' def applyupdates('
         else:
             s(_(b"the remote file has been renamed to %s\n") % f1)
         s(_(b"resolve manually then use 'hg resolve --mark %s'\n") % f)
-        ms.addpath(f, f1, fo)
+        ms.addpathconflict(f, f1, fo)
         progress.increment(item=f)
 
     # When merging in-memory, we can't support worker processes, so set the
@@ -1939,16 +1290,20 b' def applyupdates('
 
     # remove in parallel (must come before resolving path conflicts and getting)
     prog = worker.worker(
-        repo.ui, cost, batchremove, (repo, wctx), actions[ACTION_REMOVE]
+        repo.ui,
+        cost,
+        batchremove,
+        (repo, wctx),
+        actions[mergestatemod.ACTION_REMOVE],
     )
     for i, item in prog:
         progress.increment(step=i, item=item)
-    removed = len(actions[ACTION_REMOVE])
+    removed = len(actions[mergestatemod.ACTION_REMOVE])
 
     # resolve path conflicts (must come before getting)
-    for f, args, msg in actions[ACTION_PATH_CONFLICT_RESOLVE]:
+    for f, args, msg in actions[mergestatemod.ACTION_PATH_CONFLICT_RESOLVE]:
         repo.ui.debug(b" %s: %s -> pr\n" % (f, msg))
-        (f0,) = args
+        (f0, origf0) = args
         if wctx[f0].lexists():
             repo.ui.note(_(b"moving %s to %s\n") % (f0, f))
             wctx[f].audit()
@@ -1965,7 +1320,7 b' def applyupdates('
         cost,
         batchget,
         (repo, mctx, wctx, wantfiledata),
-        actions[ACTION_GET],
+        actions[mergestatemod.ACTION_GET],
         threadsafe=threadsafe,
         hasretval=True,
     )
@@ -1976,33 +1331,33 b' def applyupdates('
         else:
             i, item = res
             progress.increment(step=i, item=item)
-    updated = len(actions[ACTION_GET])
+    updated = len(actions[mergestatemod.ACTION_GET])
 
-    if [a for a in actions[ACTION_GET] if a[0] == b'.hgsubstate']:
+    if [a for a in actions[mergestatemod.ACTION_GET] if a[0] == b'.hgsubstate']:
         subrepoutil.submerge(repo, wctx, mctx, wctx, overwrite, labels)
 
     # forget (manifest only, just log it) (must come first)
-    for f, args, msg in actions[ACTION_FORGET]:
+    for f, args, msg in actions[mergestatemod.ACTION_FORGET]:
         repo.ui.debug(b" %s: %s -> f\n" % (f, msg))
         progress.increment(item=f)
 
     # re-add (manifest only, just log it)
-    for f, args, msg in actions[ACTION_ADD]:
+    for f, args, msg in actions[mergestatemod.ACTION_ADD]:
         repo.ui.debug(b" %s: %s -> a\n" % (f, msg))
         progress.increment(item=f)
 
     # re-add/mark as modified (manifest only, just log it)
-    for f, args, msg in actions[ACTION_ADD_MODIFIED]:
+    for f, args, msg in actions[mergestatemod.ACTION_ADD_MODIFIED]:
         repo.ui.debug(b" %s: %s -> am\n" % (f, msg))
         progress.increment(item=f)
 
     # keep (noop, just log it)
-    for f, args, msg in actions[ACTION_KEEP]:
+    for f, args, msg in actions[mergestatemod.ACTION_KEEP]:
         repo.ui.debug(b" %s: %s -> k\n" % (f, msg))
         # no progress
 
     # directory rename, move local
-    for f, args, msg in actions[ACTION_DIR_RENAME_MOVE_LOCAL]:
+    for f, args, msg in actions[mergestatemod.ACTION_DIR_RENAME_MOVE_LOCAL]:
         repo.ui.debug(b" %s: %s -> dm\n" % (f, msg))
         progress.increment(item=f)
         f0, flags = args
@@ -2013,7 +1368,7 b' def applyupdates('
         updated += 1
 
     # local directory rename, get
-    for f, args, msg in actions[ACTION_LOCAL_DIR_RENAME_GET]:
+    for f, args, msg in actions[mergestatemod.ACTION_LOCAL_DIR_RENAME_GET]:
         repo.ui.debug(b" %s: %s -> dg\n" % (f, msg))
         progress.increment(item=f)
         f0, flags = args
@@ -2022,7 +1377,7 b' def applyupdates('
         updated += 1
 
     # exec
-    for f, args, msg in actions[ACTION_EXEC]:
+    for f, args, msg in actions[mergestatemod.ACTION_EXEC]:
         repo.ui.debug(b" %s: %s -> e\n" % (f, msg))
         progress.increment(item=f)
         (flags,) = args
@@ -2087,7 +1442,7 b' def applyupdates('
     if (
         usemergedriver
         and not unresolved
-        and ms.mdstate() != MERGE_DRIVER_STATE_SUCCESS
+        and ms.mdstate() != mergestatemod.MERGE_DRIVER_STATE_SUCCESS
     ):
         if not driverconclude(repo, ms, wctx, labels=labels):
             # XXX setting unresolved to at least 1 is a hack to make sure we
@@ -2103,10 +1458,10 b' def applyupdates('
 
     extraactions = ms.actions()
     if extraactions:
-        mfiles = {a[0] for a in actions[ACTION_MERGE]}
+        mfiles = {a[0] for a in actions[mergestatemod.ACTION_MERGE]}
         for k, acts in pycompat.iteritems(extraactions):
             actions[k].extend(acts)
-            if k == ACTION_GET and wantfiledata:
+            if k == mergestatemod.ACTION_GET and wantfiledata:
                 # no filedata until mergestate is updated to provide it
                 for a in acts:
                     getfiledata[a[0]] = None
@@ -2128,110 +1483,58 b' def applyupdates('
             # those lists aren't consulted again.
             mfiles.difference_update(a[0] for a in acts)
 
-        actions[ACTION_MERGE] = [
-            a for a in actions[ACTION_MERGE] if a[0] in mfiles
+        actions[mergestatemod.ACTION_MERGE] = [
+            a for a in actions[mergestatemod.ACTION_MERGE] if a[0] in mfiles
         ]
 
     progress.complete()
-    assert len(getfiledata) == (len(actions[ACTION_GET]) if wantfiledata else 0)
+    assert len(getfiledata) == (
+        len(actions[mergestatemod.ACTION_GET]) if wantfiledata else 0
+    )
     return updateresult(updated, merged, removed, unresolved), getfiledata
 
 
-def recordupdates(repo, actions, branchmerge, getfiledata):
-    """record merge actions to the dirstate"""
-    # remove (must come first)
-    for f, args, msg in actions.get(ACTION_REMOVE, []):
-        if branchmerge:
-            repo.dirstate.remove(f)
-        else:
-            repo.dirstate.drop(f)
-
-    # forget (must come first)
-    for f, args, msg in actions.get(ACTION_FORGET, []):
-        repo.dirstate.drop(f)
-
-    # resolve path conflicts
-    for f, args, msg in actions.get(ACTION_PATH_CONFLICT_RESOLVE, []):
-        (f0,) = args
-        origf0 = repo.dirstate.copied(f0) or f0
-        repo.dirstate.add(f)
-        repo.dirstate.copy(origf0, f)
-        if f0 == origf0:
-            repo.dirstate.remove(f0)
-        else:
-            repo.dirstate.drop(f0)
-
-    # re-add
-    for f, args, msg in actions.get(ACTION_ADD, []):
-        repo.dirstate.add(f)
-
-    # re-add/mark as modified
-    for f, args, msg in actions.get(ACTION_ADD_MODIFIED, []):
-        if branchmerge:
-            repo.dirstate.normallookup(f)
-        else:
-            repo.dirstate.add(f)
-
-    # exec change
-    for f, args, msg in actions.get(ACTION_EXEC, []):
-        repo.dirstate.normallookup(f)
-
-    # keep
-    for f, args, msg in actions.get(ACTION_KEEP, []):
-        pass
+def _advertisefsmonitor(repo, num_gets, p1node):
+    # Advertise fsmonitor when its presence could be useful.
+    #
+    # We only advertise when performing an update from an empty working
+    # directory. This typically only occurs during initial clone.
+    #
+    # We give users a mechanism to disable the warning in case it is
+    # annoying.
+    #
+    # We only allow on Linux and MacOS because that's where fsmonitor is
+    # considered stable.
+    fsmonitorwarning = repo.ui.configbool(b'fsmonitor', b'warn_when_unused')
+    fsmonitorthreshold = repo.ui.configint(
+        b'fsmonitor', b'warn_update_file_count'
+    )
+    try:
+        # avoid cycle: extensions -> cmdutil -> merge
+        from . import extensions
 
-    # get
-    for f, args, msg in actions.get(ACTION_GET, []):
-        if branchmerge:
-            repo.dirstate.otherparent(f)
-        else:
-            parentfiledata = getfiledata[f] if getfiledata else None
-            repo.dirstate.normal(f, parentfiledata=parentfiledata)
+        extensions.find(b'fsmonitor')
+        fsmonitorenabled = repo.ui.config(b'fsmonitor', b'mode') != b'off'
+        # We intentionally don't look at whether fsmonitor has disabled
+        # itself because a) fsmonitor may have already printed a warning
+        # b) we only care about the config state here.
+    except KeyError:
+        fsmonitorenabled = False
 
-    # merge
-    for f, args, msg in actions.get(ACTION_MERGE, []):
-        f1, f2, fa, move, anc = args
-        if branchmerge:
-            # We've done a branch merge, mark this file as merged
-            # so that we properly record the merger later
-            repo.dirstate.merge(f)
-            if f1 != f2:  # copy/rename
-                if move:
-                    repo.dirstate.remove(f1)
-                if f1 != f:
-                    repo.dirstate.copy(f1, f)
-                else:
-                    repo.dirstate.copy(f2, f)
-        else:
-            # We've update-merged a locally modified file, so
-            # we set the dirstate to emulate a normal checkout
-            # of that file some time in the past. Thus our
-            # merge will appear as a normal local file
-            # modification.
-            if f2 == f:  # file not locally copied/moved
-                repo.dirstate.normallookup(f)
-            if move:
-                repo.dirstate.drop(f1)
-
-    # directory rename, move local
-    for f, args, msg in actions.get(ACTION_DIR_RENAME_MOVE_LOCAL, []):
-        f0, flag = args
-        if branchmerge:
-            repo.dirstate.add(f)
-            repo.dirstate.remove(f0)
-            repo.dirstate.copy(f0, f)
-        else:
-            repo.dirstate.normal(f)
-            repo.dirstate.drop(f0)
-
-    # directory rename, get
-    for f, args, msg in actions.get(ACTION_LOCAL_DIR_RENAME_GET, []):
-        f0, flag = args
-        if branchmerge:
-            repo.dirstate.add(f)
-            repo.dirstate.copy(f0, f)
-        else:
-            repo.dirstate.normal(f)
+    if (
+        fsmonitorwarning
+        and not fsmonitorenabled
+        and p1node == nullid
+        and num_gets >= fsmonitorthreshold
+        and pycompat.sysplatform.startswith((b'linux', b'darwin'))
+    ):
+        repo.ui.warn(
+            _(
+                b'(warning: large working directory being used without '
+                b'fsmonitor enabled; enable fsmonitor to improve performance; '
+                b'see "hg help -e fsmonitor")\n'
+            )
+        )
 
 
 UPDATECHECK_ABORT = b'abort'  # handled at higher layers
@@ -2334,7 +1637,11 b' def update('
                     ),
                 )
             )
-    with repo.wlock():
+    if wc is not None and wc.isinmemory():
+        maybe_wlock = util.nullcontextmanager()
+    else:
+        maybe_wlock = repo.wlock()
+    with maybe_wlock:
         if wc is None:
             wc = repo[None]
         pl = wc.parents()
@@ -2356,7 +1663,7 b' def update('
         if not overwrite:
             if len(pl) > 1:
                 raise error.Abort(_(b"outstanding uncommitted merge"))
-            ms = mergestate.read(repo)
+            ms = mergestatemod.mergestate.read(repo)
             if list(ms.unresolved()):
                 raise error.Abort(
                     _(b"outstanding merge conflicts"),
@@ -2443,12 +1750,12 b' def update('
         if updatecheck == UPDATECHECK_NO_CONFLICT:
             for f, (m, args, msg) in pycompat.iteritems(actionbyfile):
                 if m not in (
-                    ACTION_GET,
-                    ACTION_KEEP,
-                    ACTION_EXEC,
-                    ACTION_REMOVE,
-                    ACTION_PATH_CONFLICT_RESOLVE,
-                    ACTION_GET_OTHER_AND_STORE,
+                    mergestatemod.ACTION_GET,
+                    mergestatemod.ACTION_KEEP,
+                    mergestatemod.ACTION_EXEC,
+                    mergestatemod.ACTION_REMOVE,
+                    mergestatemod.ACTION_PATH_CONFLICT_RESOLVE,
+                    mergestatemod.ACTION_GET_OTHER_AND_STORE,
                 ):
                     msg = _(b"conflicting changes")
                     hint = _(b"commit or update --clean to discard changes")
@@ -2462,7 +1769,7 b' def update('
             m, args, msg = actionbyfile[f]
             prompts = filemerge.partextras(labels)
             prompts[b'f'] = f
-            if m == ACTION_CHANGED_DELETED:
+            if m == mergestatemod.ACTION_CHANGED_DELETED:
                 if repo.ui.promptchoice(
                     _(
                         b"local%(l)s changed %(f)s which other%(o)s deleted\n"
@@ -2472,16 +1779,24 b' def update('
                     % prompts,
                     0,
                 ):
-                    actionbyfile[f] = (ACTION_REMOVE, None, b'prompt delete')
+                    actionbyfile[f] = (
+                        mergestatemod.ACTION_REMOVE,
+                        None,
+                        b'prompt delete',
+                    )
                 elif f in p1:
                     actionbyfile[f] = (
-                        ACTION_ADD_MODIFIED,
+                        mergestatemod.ACTION_ADD_MODIFIED,
                         None,
                         b'prompt keep',
                     )
                 else:
-                    actionbyfile[f] = (ACTION_ADD, None, b'prompt keep')
-            elif m == ACTION_DELETED_CHANGED:
+                    actionbyfile[f] = (
+                        mergestatemod.ACTION_ADD,
+                        None,
+                        b'prompt keep',
+                    )
+            elif m == mergestatemod.ACTION_DELETED_CHANGED:
                 f1, f2, fa, move, anc = args
                 flags = p2[f2].flags()
                 if (
@@ -2497,7 +1812,7 b' def update('
                     == 0
                 ):
                     actionbyfile[f] = (
-                        ACTION_GET,
+                        mergestatemod.ACTION_GET,
                         (flags, False),
                         b'prompt recreating',
                     )
@@ -2511,9 +1826,9 b' def update('
                 actions[m] = []
             actions[m].append((f, args, msg))
 
-        # ACTION_GET_OTHER_AND_STORE is a ACTION_GET + store in mergestate
-        for e in actions[ACTION_GET_OTHER_AND_STORE]:
-            actions[ACTION_GET].append(e)
+        # ACTION_GET_OTHER_AND_STORE is a mergestatemod.ACTION_GET + store in mergestate
+        for e in actions[mergestatemod.ACTION_GET_OTHER_AND_STORE]:
+            actions[mergestatemod.ACTION_GET].append(e)
 
         if not util.fscasesensitive(repo.path):
             # check collision between files only in p2 for clean update
@@ -2560,46 +1875,9 b' def update('
             # note that we're in the middle of an update
             repo.vfs.write(b'updatestate', p2.hex())
 
-        # Advertise fsmonitor when its presence could be useful.
-        #
-        # We only advertise when performing an update from an empty working
-        # directory. This typically only occurs during initial clone.
-        #
-        # We give users a mechanism to disable the warning in case it is
-        # annoying.
-        #
-        # We only allow on Linux and MacOS because that's where fsmonitor is
-        # considered stable.
-        fsmonitorwarning = repo.ui.configbool(b'fsmonitor', b'warn_when_unused')
-        fsmonitorthreshold = repo.ui.configint(
-            b'fsmonitor', b'warn_update_file_count'
+        _advertisefsmonitor(
+            repo, len(actions[mergestatemod.ACTION_GET]), p1.node()
         )
-        try:
-            # avoid cycle: extensions -> cmdutil -> merge
-            from . import extensions
-
-            extensions.find(b'fsmonitor')
-            fsmonitorenabled = repo.ui.config(b'fsmonitor', b'mode') != b'off'
-            # We intentionally don't look at whether fsmonitor has disabled
-            # itself because a) fsmonitor may have already printed a warning
-            # b) we only care about the config state here.
-        except KeyError:
-            fsmonitorenabled = False
-
-        if (
-            fsmonitorwarning
-            and not fsmonitorenabled
-            and p1.node() == nullid
-            and len(actions[ACTION_GET]) >= fsmonitorthreshold
-            and pycompat.sysplatform.startswith((b'linux', b'darwin'))
-        ):
-            repo.ui.warn(
-                _(
-                    b'(warning: large working directory being used without '
-                    b'fsmonitor enabled; enable fsmonitor to improve performance; '
-                    b'see "hg help -e fsmonitor")\n'
-                )
-            )
 
         wantfiledata = updatedirstate and not branchmerge
         stats, getfiledata = applyupdates(
@@ -2609,7 +1887,9 b' def update('
         if updatedirstate:
             with repo.dirstate.parentchange():
                 repo.setparents(fp1, fp2)
-                recordupdates(repo, actions, branchmerge, getfiledata)
+                mergestatemod.recordupdates(
+                    repo, actions, branchmerge, getfiledata
+                )
                 # update completed, clear state
                 util.unlink(repo.vfs.join(b'updatestate'))
 

mercurial/mergestate.py

@@ -1,42 +1,22 b''
-# merge.py - directory-level update/merge handling for Mercurial
-#
-# Copyright 2006, 2007 Matt Mackall <[e-mail]>
-#
-# This software may be used and distributed according to the terms of the
-# GNU General Public License version 2 or any later version.
-
 from __future__ import absolute_import
 
 import errno
 import shutil
-import stat
 import struct
 
 from .i18n import _
 from .node import (
-    addednodeid,
     bin,
     hex,
-    modifiednodeid,
     nullhex,
     nullid,
-    nullrev,
 )
 from .pycompat import delattr
-from .thirdparty import attr
 from . import (
-    copies,
-    encoding,
     error,
     filemerge,
-    match as matchmod,
-    obsutil,
-    pathutil,
     pycompat,
-    scmutil,
-    subrepoutil,
     util,
-    worker,
 )
 from .utils import hashutil
 
@@ -51,25 +31,48 b' def _droponode(data):'
     return b'\0'.join(bits)
 
 
+def _filectxorabsent(hexnode, ctx, f):
+    if hexnode == nullhex:
+        return filemerge.absentfilectx(ctx, f)
+    else:
+        return ctx[f]
+
+
 # Merge state record types. See ``mergestate`` docs for more.
+
+####
+# merge records which records metadata about a current merge
+# exists only once in a mergestate
+#####
 RECORD_LOCAL = b'L'
 RECORD_OTHER = b'O'
+# record merge labels
+RECORD_LABELS = b'l'
+# store info about merge driver used and it's state
+RECORD_MERGE_DRIVER_STATE = b'm'
+
+#####
+# record extra information about files, with one entry containing info about one
+# file. Hence, multiple of them can exists
+#####
+RECORD_FILE_VALUES = b'f'
+
+#####
+# merge records which represents state of individual merges of files/folders
+# These are top level records for each entry containing merge related info.
+# Each record of these has info about one file. Hence multiple of them can
+# exists
+#####
 RECORD_MERGED = b'F'
 RECORD_CHANGEDELETE_CONFLICT = b'C'
 RECORD_MERGE_DRIVER_MERGE = b'D'
+# the path was dir on one side of merge and file on another
 RECORD_PATH_CONFLICT = b'P'
-RECORD_MERGE_DRIVER_STATE = b'm'
-RECORD_FILE_VALUES = b'f'
-RECORD_LABELS = b'l'
-RECORD_OVERRIDE = b't'
-RECORD_UNSUPPORTED_MANDATORY = b'X'
-RECORD_UNSUPPORTED_ADVISORY = b'x'
-RECORD_RESOLVED_OTHER = b'R'
 
-MERGE_DRIVER_STATE_UNMARKED = b'u'
-MERGE_DRIVER_STATE_MARKED = b'm'
-MERGE_DRIVER_STATE_SUCCESS = b's'
-
+#####
+# possible state which a merge entry can have. These are stored inside top-level
+# merge records mentioned just above.
+#####
 MERGE_RECORD_UNRESOLVED = b'u'
 MERGE_RECORD_RESOLVED = b'r'
 MERGE_RECORD_UNRESOLVED_PATH = b'pu'
@@ -79,6 +82,21 b" MERGE_RECORD_DRIVER_RESOLVED = b'd'"
 # of other version. This info is used on commit.
 MERGE_RECORD_MERGED_OTHER = b'o'
 
+#####
+# top level record which stores other unknown records. Multiple of these can
+# exists
+#####
+RECORD_OVERRIDE = b't'
+
+#####
+# possible states which a merge driver can have. These are stored inside a
+# RECORD_MERGE_DRIVER_STATE entry
+#####
+MERGE_DRIVER_STATE_UNMARKED = b'u'
+MERGE_DRIVER_STATE_MARKED = b'm'
+MERGE_DRIVER_STATE_SUCCESS = b's'
+
+
 ACTION_FORGET = b'f'
 ACTION_REMOVE = b'r'
 ACTION_ADD = b'a'
@@ -125,8 +143,6 b' class mergestate(object):'
     m: the external merge driver defined for this merge plus its run state
        (experimental)
     f: a (filename, dictionary) tuple of optional values for a given file
-    X: unsupported mandatory record type (used in tests)
-    x: unsupported advisory record type (used in tests)
     l: the labels for the parts of the merge.
 
     Merge driver run states (experimental):
@@ -233,7 +249,6 b' class mergestate(object):'
                 RECORD_CHANGEDELETE_CONFLICT,
                 RECORD_PATH_CONFLICT,
                 RECORD_MERGE_DRIVER_MERGE,
-                RECORD_RESOLVED_OTHER,
             ):
                 bits = record.split(b'\0')
                 self._state[bits[0]] = bits[1:]
@@ -252,6 +267,11 b' class mergestate(object):'
                 self._labels = [l for l in labels if len(l) > 0]
             elif not rtype.islower():
                 unsupported.add(rtype)
+        # contains a mapping of form:
+        # {filename : (merge_return_value, action_to_be_performed}
+        # these are results of re-running merge process
+        # this dict is used to perform actions on dirstate caused by re-running
+        # the merge
         self._results = {}
         self._dirty = False
 
@@ -461,9 +481,7 b' class mergestate(object):'
                     (RECORD_PATH_CONFLICT, b'\0'.join([filename] + v))
                 )
             elif v[0] == MERGE_RECORD_MERGED_OTHER:
-                records.append(
-                    (RECORD_RESOLVED_OTHER, b'\0'.join([filename] + v))
-                )
+                records.append((RECORD_MERGED, b'\0'.join([filename] + v)))
             elif v[1] == nullhex or v[6] == nullhex:
                 # Change/Delete or Delete/Change conflicts. These are stored in
                 # 'C' records. v[1] is the local file, and is nullhex when the
@@ -553,7 +571,7 b' class mergestate(object):'
         self._stateextras[fd] = {b'ancestorlinknode': hex(fca.node())}
         self._dirty = True
 
-    def addpath(self, path, frename, forigin):
+    def addpathconflict(self, path, frename, forigin):
         """add a new conflicting path to the merge state
         path:    the path that conflicts
         frename: the filename the conflicting file was renamed to
@@ -606,7 +624,10 b' class mergestate(object):'
         return self._stateextras.setdefault(filename, {})
 
     def _resolve(self, preresolve, dfile, wctx):
-        """rerun merge process for file path `dfile`"""
+        """rerun merge process for file path `dfile`.
+        Returns whether the merge was completed and the return value of merge
+        obtained from filemerge._filemerge().
+        """
         if self[dfile] in (MERGE_RECORD_RESOLVED, MERGE_RECORD_DRIVER_RESOLVED):
             return True, 0
         if self._state[dfile][0] == MERGE_RECORD_MERGED_OTHER:
@@ -620,8 +641,8 b' class mergestate(object):'
             actx = self._repo[anccommitnode]
         else:
             actx = None
-        fcd = self._filectxorabsent(localkey, wctx, dfile)
-        fco = self._filectxorabsent(onode, octx, ofile)
+        fcd = _filectxorabsent(localkey, wctx, dfile)
+        fco = _filectxorabsent(onode, octx, ofile)
         # TODO: move this to filectxorabsent
         fca = self._repo.filectx(afile, fileid=anode, changectx=actx)
         # "premerge" x flags
@@ -647,7 +668,7 b' class mergestate(object):'
                 f.close()
             else:
                 wctx[dfile].remove(ignoremissing=True)
-            complete, r, deleted = filemerge.premerge(
+            complete, merge_ret, deleted = filemerge.premerge(
                 self._repo,
                 wctx,
                 self._local,
@@ -658,7 +679,7 b' class mergestate(object):'
                 labels=self._labels,
             )
         else:
-            complete, r, deleted = filemerge.filemerge(
+            complete, merge_ret, deleted = filemerge.filemerge(
                 self._repo,
                 wctx,
                 self._local,
@@ -668,12 +689,12 b' class mergestate(object):'
                 fca,
                 labels=self._labels,
             )
-        if r is None:
-            # no real conflict
+        if merge_ret is None:
+            # If return value of merge is None, then there are no real conflict
             del self._state[dfile]
             self._stateextras.pop(dfile, None)
             self._dirty = True
-        elif not r:
+        elif not merge_ret:
             self.mark(dfile, MERGE_RECORD_RESOLVED)
 
         if complete:
@@ -695,15 +716,9 b' class mergestate(object):'
                     else:
                         action = ACTION_ADD
                 # else: regular merges (no action necessary)
-            self._results[dfile] = r, action
-
-        return complete, r
+            self._results[dfile] = merge_ret, action
 
-    def _filectxorabsent(self, hexnode, ctx, f):
-        if hexnode == nullhex:
-            return filemerge.absentfilectx(ctx, f)
-        else:
-            return ctx[f]
+        return complete, merge_ret
 
     def preresolve(self, dfile, wctx):
         """run premerge process for dfile
@@ -749,11 +764,6 b' class mergestate(object):'
                 actions[action].append((f, None, b"merge result"))
         return actions
 
-    def recordactions(self):
-        """record remove/add/get actions in the dirstate"""
-        branchmerge = self._repo.dirstate.p2() != nullid
-        recordupdates(self._repo, self.actions(), branchmerge, None)
-
     def queueremove(self, f):
         """queues a file to be removed from the dirstate
 
@@ -773,1370 +783,6 b' class mergestate(object):'
         self._results[f] = 0, ACTION_GET
 
 
-def _getcheckunknownconfig(repo, section, name):
-    config = repo.ui.config(section, name)
-    valid = [b'abort', b'ignore', b'warn']
-    if config not in valid:
-        validstr = b', '.join([b"'" + v + b"'" for v in valid])
-        raise error.ConfigError(
-            _(b"%s.%s not valid ('%s' is none of %s)")
-            % (section, name, config, validstr)
-        )
-    return config
-
-
-def _checkunknownfile(repo, wctx, mctx, f, f2=None):
-    if wctx.isinmemory():
-        # Nothing to do in IMM because nothing in the "working copy" can be an
-        # unknown file.
-        #
-        # Note that we should bail out here, not in ``_checkunknownfiles()``,
-        # because that function does other useful work.
-        return False
-
-    if f2 is None:
-        f2 = f
-    return (
-        repo.wvfs.audit.check(f)
-        and repo.wvfs.isfileorlink(f)
-        and repo.dirstate.normalize(f) not in repo.dirstate
-        and mctx[f2].cmp(wctx[f])
-    )
-
-
-class _unknowndirschecker(object):
-    """
-    Look for any unknown files or directories that may have a path conflict
-    with a file.  If any path prefix of the file exists as a file or link,
-    then it conflicts.  If the file itself is a directory that contains any
-    file that is not tracked, then it conflicts.
-
-    Returns the shortest path at which a conflict occurs, or None if there is
-    no conflict.
-    """
-
-    def __init__(self):
-        # A set of paths known to be good.  This prevents repeated checking of
-        # dirs.  It will be updated with any new dirs that are checked and found
-        # to be safe.
-        self._unknowndircache = set()
-
-        # A set of paths that are known to be absent.  This prevents repeated
-        # checking of subdirectories that are known not to exist. It will be
-        # updated with any new dirs that are checked and found to be absent.
-        self._missingdircache = set()
-
-    def __call__(self, repo, wctx, f):
-        if wctx.isinmemory():
-            # Nothing to do in IMM for the same reason as ``_checkunknownfile``.
-            return False
-
-        # Check for path prefixes that exist as unknown files.
-        for p in reversed(list(pathutil.finddirs(f))):
-            if p in self._missingdircache:
-                return
-            if p in self._unknowndircache:
-                continue
-            if repo.wvfs.audit.check(p):
-                if (
-                    repo.wvfs.isfileorlink(p)
-                    and repo.dirstate.normalize(p) not in repo.dirstate
-                ):
-                    return p
-                if not repo.wvfs.lexists(p):
-                    self._missingdircache.add(p)
-                    return
-                self._unknowndircache.add(p)
-
-        # Check if the file conflicts with a directory containing unknown files.
-        if repo.wvfs.audit.check(f) and repo.wvfs.isdir(f):
-            # Does the directory contain any files that are not in the dirstate?
-            for p, dirs, files in repo.wvfs.walk(f):
-                for fn in files:
-                    relf = util.pconvert(repo.wvfs.reljoin(p, fn))
-                    relf = repo.dirstate.normalize(relf, isknown=True)
-                    if relf not in repo.dirstate:
-                        return f
-        return None
-
-
-def _checkunknownfiles(repo, wctx, mctx, force, actions, mergeforce):
-    """
-    Considers any actions that care about the presence of conflicting unknown
-    files. For some actions, the result is to abort; for others, it is to
-    choose a different action.
-    """
-    fileconflicts = set()
-    pathconflicts = set()
-    warnconflicts = set()
-    abortconflicts = set()
-    unknownconfig = _getcheckunknownconfig(repo, b'merge', b'checkunknown')
-    ignoredconfig = _getcheckunknownconfig(repo, b'merge', b'checkignored')
-    pathconfig = repo.ui.configbool(
-        b'experimental', b'merge.checkpathconflicts'
-    )
-    if not force:
-
-        def collectconflicts(conflicts, config):
-            if config == b'abort':
-                abortconflicts.update(conflicts)
-            elif config == b'warn':
-                warnconflicts.update(conflicts)
-
-        checkunknowndirs = _unknowndirschecker()
-        for f, (m, args, msg) in pycompat.iteritems(actions):
-            if m in (ACTION_CREATED, ACTION_DELETED_CHANGED):
-                if _checkunknownfile(repo, wctx, mctx, f):
-                    fileconflicts.add(f)
-                elif pathconfig and f not in wctx:
-                    path = checkunknowndirs(repo, wctx, f)
-                    if path is not None:
-                        pathconflicts.add(path)
-            elif m == ACTION_LOCAL_DIR_RENAME_GET:
-                if _checkunknownfile(repo, wctx, mctx, f, args[0]):
-                    fileconflicts.add(f)
-
-        allconflicts = fileconflicts | pathconflicts
-        ignoredconflicts = {c for c in allconflicts if repo.dirstate._ignore(c)}
-        unknownconflicts = allconflicts - ignoredconflicts
-        collectconflicts(ignoredconflicts, ignoredconfig)
-        collectconflicts(unknownconflicts, unknownconfig)
-    else:
-        for f, (m, args, msg) in pycompat.iteritems(actions):
-            if m == ACTION_CREATED_MERGE:
-                fl2, anc = args
-                different = _checkunknownfile(repo, wctx, mctx, f)
-                if repo.dirstate._ignore(f):
-                    config = ignoredconfig
-                else:
-                    config = unknownconfig
-
-                # The behavior when force is True is described by this table:
-                #  config  different  mergeforce  |    action    backup
-                #    *         n          *       |      get        n
-                #    *         y          y       |     merge       -
-                #   abort      y          n       |     merge       -   (1)
-                #   warn       y          n       |  warn + get     y
-                #  ignore      y          n       |      get        y
-                #
-                # (1) this is probably the wrong behavior here -- we should
-                #     probably abort, but some actions like rebases currently
-                #     don't like an abort happening in the middle of
-                #     merge.update.
-                if not different:
-                    actions[f] = (ACTION_GET, (fl2, False), b'remote created')
-                elif mergeforce or config == b'abort':
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f, f, None, False, anc),
-                        b'remote differs from untracked local',
-                    )
-                elif config == b'abort':
-                    abortconflicts.add(f)
-                else:
-                    if config == b'warn':
-                        warnconflicts.add(f)
-                    actions[f] = (ACTION_GET, (fl2, True), b'remote created')
-
-    for f in sorted(abortconflicts):
-        warn = repo.ui.warn
-        if f in pathconflicts:
-            if repo.wvfs.isfileorlink(f):
-                warn(_(b"%s: untracked file conflicts with directory\n") % f)
-            else:
-                warn(_(b"%s: untracked directory conflicts with file\n") % f)
-        else:
-            warn(_(b"%s: untracked file differs\n") % f)
-    if abortconflicts:
-        raise error.Abort(
-            _(
-                b"untracked files in working directory "
-                b"differ from files in requested revision"
-            )
-        )
-
-    for f in sorted(warnconflicts):
-        if repo.wvfs.isfileorlink(f):
-            repo.ui.warn(_(b"%s: replacing untracked file\n") % f)
-        else:
-            repo.ui.warn(_(b"%s: replacing untracked files in directory\n") % f)
-
-    for f, (m, args, msg) in pycompat.iteritems(actions):
-        if m == ACTION_CREATED:
-            backup = (
-                f in fileconflicts
-                or f in pathconflicts
-                or any(p in pathconflicts for p in pathutil.finddirs(f))
-            )
-            (flags,) = args
-            actions[f] = (ACTION_GET, (flags, backup), msg)
-
-
-def _forgetremoved(wctx, mctx, branchmerge):
-    """
-    Forget removed files
-
-    If we're jumping between revisions (as opposed to merging), and if
-    neither the working directory nor the target rev has the file,
-    then we need to remove it from the dirstate, to prevent the
-    dirstate from listing the file when it is no longer in the
-    manifest.
-
-    If we're merging, and the other revision has removed a file
-    that is not present in the working directory, we need to mark it
-    as removed.
-    """
-
-    actions = {}
-    m = ACTION_FORGET
-    if branchmerge:
-        m = ACTION_REMOVE
-    for f in wctx.deleted():
-        if f not in mctx:
-            actions[f] = m, None, b"forget deleted"
-
-    if not branchmerge:
-        for f in wctx.removed():
-            if f not in mctx:
-                actions[f] = ACTION_FORGET, None, b"forget removed"
-
-    return actions
-
-
-def _checkcollision(repo, wmf, actions):
-    """
-    Check for case-folding collisions.
-    """
-    # If the repo is narrowed, filter out files outside the narrowspec.
-    narrowmatch = repo.narrowmatch()
-    if not narrowmatch.always():
-        pmmf = set(wmf.walk(narrowmatch))
-        if actions:
-            narrowactions = {}
-            for m, actionsfortype in pycompat.iteritems(actions):
-                narrowactions[m] = []
-                for (f, args, msg) in actionsfortype:
-                    if narrowmatch(f):
-                        narrowactions[m].append((f, args, msg))
-            actions = narrowactions
-    else:
-        # build provisional merged manifest up
-        pmmf = set(wmf)
-
-    if actions:
-        # KEEP and EXEC are no-op
-        for m in (
-            ACTION_ADD,
-            ACTION_ADD_MODIFIED,
-            ACTION_FORGET,
-            ACTION_GET,
-            ACTION_CHANGED_DELETED,
-            ACTION_DELETED_CHANGED,
-        ):
-            for f, args, msg in actions[m]:
-                pmmf.add(f)
-        for f, args, msg in actions[ACTION_REMOVE]:
-            pmmf.discard(f)
-        for f, args, msg in actions[ACTION_DIR_RENAME_MOVE_LOCAL]:
-            f2, flags = args
-            pmmf.discard(f2)
-            pmmf.add(f)
-        for f, args, msg in actions[ACTION_LOCAL_DIR_RENAME_GET]:
-            pmmf.add(f)
-        for f, args, msg in actions[ACTION_MERGE]:
-            f1, f2, fa, move, anc = args
-            if move:
-                pmmf.discard(f1)
-            pmmf.add(f)
-
-    # check case-folding collision in provisional merged manifest
-    foldmap = {}
-    for f in pmmf:
-        fold = util.normcase(f)
-        if fold in foldmap:
-            raise error.Abort(
-                _(b"case-folding collision between %s and %s")
-                % (f, foldmap[fold])
-            )
-        foldmap[fold] = f
-
-    # check case-folding of directories
-    foldprefix = unfoldprefix = lastfull = b''
-    for fold, f in sorted(foldmap.items()):
-        if fold.startswith(foldprefix) and not f.startswith(unfoldprefix):
-            # the folded prefix matches but actual casing is different
-            raise error.Abort(
-                _(b"case-folding collision between %s and directory of %s")
-                % (lastfull, f)
-            )
-        foldprefix = fold + b'/'
-        unfoldprefix = f + b'/'
-        lastfull = f
-
-
-def driverpreprocess(repo, ms, wctx, labels=None):
-    """run the preprocess step of the merge driver, if any
-
-    This is currently not implemented -- it's an extension point."""
-    return True
-
-
-def driverconclude(repo, ms, wctx, labels=None):
-    """run the conclude step of the merge driver, if any
-
-    This is currently not implemented -- it's an extension point."""
-    return True
-
-
-def _filesindirs(repo, manifest, dirs):
-    """
-    Generator that yields pairs of all the files in the manifest that are found
-    inside the directories listed in dirs, and which directory they are found
-    in.
-    """
-    for f in manifest:
-        for p in pathutil.finddirs(f):
-            if p in dirs:
-                yield f, p
-                break
-
-
-def checkpathconflicts(repo, wctx, mctx, actions):
-    """
-    Check if any actions introduce path conflicts in the repository, updating
-    actions to record or handle the path conflict accordingly.
-    """
-    mf = wctx.manifest()
-
-    # The set of local files that conflict with a remote directory.
-    localconflicts = set()
-
-    # The set of directories that conflict with a remote file, and so may cause
-    # conflicts if they still contain any files after the merge.
-    remoteconflicts = set()
-
-    # The set of directories that appear as both a file and a directory in the
-    # remote manifest.  These indicate an invalid remote manifest, which
-    # can't be updated to cleanly.
-    invalidconflicts = set()
-
-    # The set of directories that contain files that are being created.
-    createdfiledirs = set()
-
-    # The set of files deleted by all the actions.
-    deletedfiles = set()
-
-    for f, (m, args, msg) in actions.items():
-        if m in (
-            ACTION_CREATED,
-            ACTION_DELETED_CHANGED,
-            ACTION_MERGE,
-            ACTION_CREATED_MERGE,
-        ):
-            # This action may create a new local file.
-            createdfiledirs.update(pathutil.finddirs(f))
-            if mf.hasdir(f):
-                # The file aliases a local directory.  This might be ok if all
-                # the files in the local directory are being deleted.  This
-                # will be checked once we know what all the deleted files are.
-                remoteconflicts.add(f)
-        # Track the names of all deleted files.
-        if m == ACTION_REMOVE:
-            deletedfiles.add(f)
-        if m == ACTION_MERGE:
-            f1, f2, fa, move, anc = args
-            if move:
-                deletedfiles.add(f1)
-        if m == ACTION_DIR_RENAME_MOVE_LOCAL:
-            f2, flags = args
-            deletedfiles.add(f2)
-
-    # Check all directories that contain created files for path conflicts.
-    for p in createdfiledirs:
-        if p in mf:
-            if p in mctx:
-                # A file is in a directory which aliases both a local
-                # and a remote file.  This is an internal inconsistency
-                # within the remote manifest.
-                invalidconflicts.add(p)
-            else:
-                # A file is in a directory which aliases a local file.
-                # We will need to rename the local file.
-                localconflicts.add(p)
-        if p in actions and actions[p][0] in (
-            ACTION_CREATED,
-            ACTION_DELETED_CHANGED,
-            ACTION_MERGE,
-            ACTION_CREATED_MERGE,
-        ):
-            # The file is in a directory which aliases a remote file.
-            # This is an internal inconsistency within the remote
-            # manifest.
-            invalidconflicts.add(p)
-
-    # Rename all local conflicting files that have not been deleted.
-    for p in localconflicts:
-        if p not in deletedfiles:
-            ctxname = bytes(wctx).rstrip(b'+')
-            pnew = util.safename(p, ctxname, wctx, set(actions.keys()))
-            actions[pnew] = (
-                ACTION_PATH_CONFLICT_RESOLVE,
-                (p,),
-                b'local path conflict',
-            )
-            actions[p] = (ACTION_PATH_CONFLICT, (pnew, b'l'), b'path conflict')
-
-    if remoteconflicts:
-        # Check if all files in the conflicting directories have been removed.
-        ctxname = bytes(mctx).rstrip(b'+')
-        for f, p in _filesindirs(repo, mf, remoteconflicts):
-            if f not in deletedfiles:
-                m, args, msg = actions[p]
-                pnew = util.safename(p, ctxname, wctx, set(actions.keys()))
-                if m in (ACTION_DELETED_CHANGED, ACTION_MERGE):
-                    # Action was merge, just update target.
-                    actions[pnew] = (m, args, msg)
-                else:
-                    # Action was create, change to renamed get action.
-                    fl = args[0]
-                    actions[pnew] = (
-                        ACTION_LOCAL_DIR_RENAME_GET,
-                        (p, fl),
-                        b'remote path conflict',
-                    )
-                actions[p] = (
-                    ACTION_PATH_CONFLICT,
-                    (pnew, ACTION_REMOVE),
-                    b'path conflict',
-                )
-                remoteconflicts.remove(p)
-                break
-
-    if invalidconflicts:
-        for p in invalidconflicts:
-            repo.ui.warn(_(b"%s: is both a file and a directory\n") % p)
-        raise error.Abort(_(b"destination manifest contains path conflicts"))
-
-
-def _filternarrowactions(narrowmatch, branchmerge, actions):
-    """
-    Filters out actions that can ignored because the repo is narrowed.
-
-    Raise an exception if the merge cannot be completed because the repo is
-    narrowed.
-    """
-    nooptypes = {b'k'}  # TODO: handle with nonconflicttypes
-    nonconflicttypes = set(b'a am c cm f g gs r e'.split())
-    # We mutate the items in the dict during iteration, so iterate
-    # over a copy.
-    for f, action in list(actions.items()):
-        if narrowmatch(f):
-            pass
-        elif not branchmerge:
-            del actions[f]  # just updating, ignore changes outside clone
-        elif action[0] in nooptypes:
-            del actions[f]  # merge does not affect file
-        elif action[0] in nonconflicttypes:
-            raise error.Abort(
-                _(
-                    b'merge affects file \'%s\' outside narrow, '
-                    b'which is not yet supported'
-                )
-                % f,
-                hint=_(b'merging in the other direction may work'),
-            )
-        else:
-            raise error.Abort(
-                _(b'conflict in file \'%s\' is outside narrow clone') % f
-            )
-
-
-def manifestmerge(
-    repo,
-    wctx,
-    p2,
-    pa,
-    branchmerge,
-    force,
-    matcher,
-    acceptremote,
-    followcopies,
-    forcefulldiff=False,
-):
-    """
-    Merge wctx and p2 with ancestor pa and generate merge action list
-
-    branchmerge and force are as passed in to update
-    matcher = matcher to filter file lists
-    acceptremote = accept the incoming changes without prompting
-    """
-    if matcher is not None and matcher.always():
-        matcher = None
-
-    # manifests fetched in order are going to be faster, so prime the caches
-    [
-        x.manifest()
-        for x in sorted(wctx.parents() + [p2, pa], key=scmutil.intrev)
-    ]
-
-    branch_copies1 = copies.branch_copies()
-    branch_copies2 = copies.branch_copies()
-    diverge = {}
-    if followcopies:
-        branch_copies1, branch_copies2, diverge = copies.mergecopies(
-            repo, wctx, p2, pa
-        )
-
-    boolbm = pycompat.bytestr(bool(branchmerge))
-    boolf = pycompat.bytestr(bool(force))
-    boolm = pycompat.bytestr(bool(matcher))
-    repo.ui.note(_(b"resolving manifests\n"))
-    repo.ui.debug(
-        b" branchmerge: %s, force: %s, partial: %s\n" % (boolbm, boolf, boolm)
-    )
-    repo.ui.debug(b" ancestor: %s, local: %s, remote: %s\n" % (pa, wctx, p2))
-
-    m1, m2, ma = wctx.manifest(), p2.manifest(), pa.manifest()
-    copied1 = set(branch_copies1.copy.values())
-    copied1.update(branch_copies1.movewithdir.values())
-    copied2 = set(branch_copies2.copy.values())
-    copied2.update(branch_copies2.movewithdir.values())
-
-    if b'.hgsubstate' in m1 and wctx.rev() is None:
-        # Check whether sub state is modified, and overwrite the manifest
-        # to flag the change. If wctx is a committed revision, we shouldn't
-        # care for the dirty state of the working directory.
-        if any(wctx.sub(s).dirty() for s in wctx.substate):
-            m1[b'.hgsubstate'] = modifiednodeid
-
-    # Don't use m2-vs-ma optimization if:
-    # - ma is the same as m1 or m2, which we're just going to diff again later
-    # - The caller specifically asks for a full diff, which is useful during bid
-    #   merge.
-    if pa not in ([wctx, p2] + wctx.parents()) and not forcefulldiff:
-        # Identify which files are relevant to the merge, so we can limit the
-        # total m1-vs-m2 diff to just those files. This has significant
-        # performance benefits in large repositories.
-        relevantfiles = set(ma.diff(m2).keys())
-
-        # For copied and moved files, we need to add the source file too.
-        for copykey, copyvalue in pycompat.iteritems(branch_copies1.copy):
-            if copyvalue in relevantfiles:
-                relevantfiles.add(copykey)
-        for movedirkey in branch_copies1.movewithdir:
-            relevantfiles.add(movedirkey)
-        filesmatcher = scmutil.matchfiles(repo, relevantfiles)
-        matcher = matchmod.intersectmatchers(matcher, filesmatcher)
-
-    diff = m1.diff(m2, match=matcher)
-
-    actions = {}
-    for f, ((n1, fl1), (n2, fl2)) in pycompat.iteritems(diff):
-        if n1 and n2:  # file exists on both local and remote side
-            if f not in ma:
-                # TODO: what if they're renamed from different sources?
-                fa = branch_copies1.copy.get(
-                    f, None
-                ) or branch_copies2.copy.get(f, None)
-                if fa is not None:
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f, f, fa, False, pa.node()),
-                        b'both renamed from %s' % fa,
-                    )
-                else:
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f, f, None, False, pa.node()),
-                        b'both created',
-                    )
-            else:
-                a = ma[f]
-                fla = ma.flags(f)
-                nol = b'l' not in fl1 + fl2 + fla
-                if n2 == a and fl2 == fla:
-                    actions[f] = (ACTION_KEEP, (), b'remote unchanged')
-                elif n1 == a and fl1 == fla:  # local unchanged - use remote
-                    if n1 == n2:  # optimization: keep local content
-                        actions[f] = (
-                            ACTION_EXEC,
-                            (fl2,),
-                            b'update permissions',
-                        )
-                    else:
-                        actions[f] = (
-                            ACTION_GET_OTHER_AND_STORE
-                            if branchmerge
-                            else ACTION_GET,
-                            (fl2, False),
-                            b'remote is newer',
-                        )
-                elif nol and n2 == a:  # remote only changed 'x'
-                    actions[f] = (ACTION_EXEC, (fl2,), b'update permissions')
-                elif nol and n1 == a:  # local only changed 'x'
-                    actions[f] = (
-                        ACTION_GET_OTHER_AND_STORE
-                        if branchmerge
-                        else ACTION_GET,
-                        (fl1, False),
-                        b'remote is newer',
-                    )
-                else:  # both changed something
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f, f, f, False, pa.node()),
-                        b'versions differ',
-                    )
-        elif n1:  # file exists only on local side
-            if f in copied2:
-                pass  # we'll deal with it on m2 side
-            elif (
-                f in branch_copies1.movewithdir
-            ):  # directory rename, move local
-                f2 = branch_copies1.movewithdir[f]
-                if f2 in m2:
-                    actions[f2] = (
-                        ACTION_MERGE,
-                        (f, f2, None, True, pa.node()),
-                        b'remote directory rename, both created',
-                    )
-                else:
-                    actions[f2] = (
-                        ACTION_DIR_RENAME_MOVE_LOCAL,
-                        (f, fl1),
-                        b'remote directory rename - move from %s' % f,
-                    )
-            elif f in branch_copies1.copy:
-                f2 = branch_copies1.copy[f]
-                actions[f] = (
-                    ACTION_MERGE,
-                    (f, f2, f2, False, pa.node()),
-                    b'local copied/moved from %s' % f2,
-                )
-            elif f in ma:  # clean, a different, no remote
-                if n1 != ma[f]:
-                    if acceptremote:
-                        actions[f] = (ACTION_REMOVE, None, b'remote delete')
-                    else:
-                        actions[f] = (
-                            ACTION_CHANGED_DELETED,
-                            (f, None, f, False, pa.node()),
-                            b'prompt changed/deleted',
-                        )
-                elif n1 == addednodeid:
-                    # This extra 'a' is added by working copy manifest to mark
-                    # the file as locally added. We should forget it instead of
-                    # deleting it.
-                    actions[f] = (ACTION_FORGET, None, b'remote deleted')
-                else:
-                    actions[f] = (ACTION_REMOVE, None, b'other deleted')
-        elif n2:  # file exists only on remote side
-            if f in copied1:
-                pass  # we'll deal with it on m1 side
-            elif f in branch_copies2.movewithdir:
-                f2 = branch_copies2.movewithdir[f]
-                if f2 in m1:
-                    actions[f2] = (
-                        ACTION_MERGE,
-                        (f2, f, None, False, pa.node()),
-                        b'local directory rename, both created',
-                    )
-                else:
-                    actions[f2] = (
-                        ACTION_LOCAL_DIR_RENAME_GET,
-                        (f, fl2),
-                        b'local directory rename - get from %s' % f,
-                    )
-            elif f in branch_copies2.copy:
-                f2 = branch_copies2.copy[f]
-                if f2 in m2:
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f2, f, f2, False, pa.node()),
-                        b'remote copied from %s' % f2,
-                    )
-                else:
-                    actions[f] = (
-                        ACTION_MERGE,
-                        (f2, f, f2, True, pa.node()),
-                        b'remote moved from %s' % f2,
-                    )
-            elif f not in ma:
-                # local unknown, remote created: the logic is described by the
-                # following table:
-                #
-                # force  branchmerge  different  |  action
-                #   n         *           *      |   create
-                #   y         n           *      |   create
-                #   y         y           n      |   create
-                #   y         y           y      |   merge
-                #
-                # Checking whether the files are different is expensive, so we
-                # don't do that when we can avoid it.
-                if not force:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote created')
-                elif not branchmerge:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote created')
-                else:
-                    actions[f] = (
-                        ACTION_CREATED_MERGE,
-                        (fl2, pa.node()),
-                        b'remote created, get or merge',
-                    )
-            elif n2 != ma[f]:
-                df = None
-                for d in branch_copies1.dirmove:
-                    if f.startswith(d):
-                        # new file added in a directory that was moved
-                        df = branch_copies1.dirmove[d] + f[len(d) :]
-                        break
-                if df is not None and df in m1:
-                    actions[df] = (
-                        ACTION_MERGE,
-                        (df, f, f, False, pa.node()),
-                        b'local directory rename - respect move '
-                        b'from %s' % f,
-                    )
-                elif acceptremote:
-                    actions[f] = (ACTION_CREATED, (fl2,), b'remote recreating')
-                else:
-                    actions[f] = (
-                        ACTION_DELETED_CHANGED,
-                        (None, f, f, False, pa.node()),
-                        b'prompt deleted/changed',
-                    )
-
-    if repo.ui.configbool(b'experimental', b'merge.checkpathconflicts'):
-        # If we are merging, look for path conflicts.
-        checkpathconflicts(repo, wctx, p2, actions)
-
-    narrowmatch = repo.narrowmatch()
-    if not narrowmatch.always():
-        # Updates "actions" in place
-        _filternarrowactions(narrowmatch, branchmerge, actions)
-
-    renamedelete = branch_copies1.renamedelete
-    renamedelete.update(branch_copies2.renamedelete)
-
-    return actions, diverge, renamedelete
-
-
-def _resolvetrivial(repo, wctx, mctx, ancestor, actions):
-    """Resolves false conflicts where the nodeid changed but the content
-       remained the same."""
-    # We force a copy of actions.items() because we're going to mutate
-    # actions as we resolve trivial conflicts.
-    for f, (m, args, msg) in list(actions.items()):
-        if (
-            m == ACTION_CHANGED_DELETED
-            and f in ancestor
-            and not wctx[f].cmp(ancestor[f])
-        ):
-            # local did change but ended up with same content
-            actions[f] = ACTION_REMOVE, None, b'prompt same'
-        elif (
-            m == ACTION_DELETED_CHANGED
-            and f in ancestor
-            and not mctx[f].cmp(ancestor[f])
-        ):
-            # remote did change but ended up with same content
-            del actions[f]  # don't get = keep local deleted
-
-
-def calculateupdates(
-    repo,
-    wctx,
-    mctx,
-    ancestors,
-    branchmerge,
-    force,
-    acceptremote,
-    followcopies,
-    matcher=None,
-    mergeforce=False,
-):
-    """Calculate the actions needed to merge mctx into wctx using ancestors"""
-    # Avoid cycle.
-    from . import sparse
-
-    if len(ancestors) == 1:  # default
-        actions, diverge, renamedelete = manifestmerge(
-            repo,
-            wctx,
-            mctx,
-            ancestors[0],
-            branchmerge,
-            force,
-            matcher,
-            acceptremote,
-            followcopies,
-        )
-        _checkunknownfiles(repo, wctx, mctx, force, actions, mergeforce)
-
-    else:  # only when merge.preferancestor=* - the default
-        repo.ui.note(
-            _(b"note: merging %s and %s using bids from ancestors %s\n")
-            % (
-                wctx,
-                mctx,
-                _(b' and ').join(pycompat.bytestr(anc) for anc in ancestors),
-            )
-        )
-
-        # Call for bids
-        fbids = (
-            {}
-        )  # mapping filename to bids (action method to list af actions)
-        diverge, renamedelete = None, None
-        for ancestor in ancestors:
-            repo.ui.note(_(b'\ncalculating bids for ancestor %s\n') % ancestor)
-            actions, diverge1, renamedelete1 = manifestmerge(
-                repo,
-                wctx,
-                mctx,
-                ancestor,
-                branchmerge,
-                force,
-                matcher,
-                acceptremote,
-                followcopies,
-                forcefulldiff=True,
-            )
-            _checkunknownfiles(repo, wctx, mctx, force, actions, mergeforce)
-
-            # Track the shortest set of warning on the theory that bid
-            # merge will correctly incorporate more information
-            if diverge is None or len(diverge1) < len(diverge):
-                diverge = diverge1
-            if renamedelete is None or len(renamedelete) < len(renamedelete1):
-                renamedelete = renamedelete1
-
-            for f, a in sorted(pycompat.iteritems(actions)):
-                m, args, msg = a
-                if m == ACTION_GET_OTHER_AND_STORE:
-                    m = ACTION_GET
-                repo.ui.debug(b' %s: %s -> %s\n' % (f, msg, m))
-                if f in fbids:
-                    d = fbids[f]
-                    if m in d:
-                        d[m].append(a)
-                    else:
-                        d[m] = [a]
-                else:
-                    fbids[f] = {m: [a]}
-
-        # Pick the best bid for each file
-        repo.ui.note(_(b'\nauction for merging merge bids\n'))
-        actions = {}
-        for f, bids in sorted(fbids.items()):
-            # bids is a mapping from action method to list af actions
-            # Consensus?
-            if len(bids) == 1:  # all bids are the same kind of method
-                m, l = list(bids.items())[0]
-                if all(a == l[0] for a in l[1:]):  # len(bids) is > 1
-                    repo.ui.note(_(b" %s: consensus for %s\n") % (f, m))
-                    actions[f] = l[0]
-                    continue
-            # If keep is an option, just do it.
-            if ACTION_KEEP in bids:
-                repo.ui.note(_(b" %s: picking 'keep' action\n") % f)
-                actions[f] = bids[ACTION_KEEP][0]
-                continue
-            # If there are gets and they all agree [how could they not?], do it.
-            if ACTION_GET in bids:
-                ga0 = bids[ACTION_GET][0]
-                if all(a == ga0 for a in bids[ACTION_GET][1:]):
-                    repo.ui.note(_(b" %s: picking 'get' action\n") % f)
-                    actions[f] = ga0
-                    continue
-            # TODO: Consider other simple actions such as mode changes
-            # Handle inefficient democrazy.
-            repo.ui.note(_(b' %s: multiple bids for merge action:\n') % f)
-            for m, l in sorted(bids.items()):
-                for _f, args, msg in l:
-                    repo.ui.note(b'  %s -> %s\n' % (msg, m))
-            # Pick random action. TODO: Instead, prompt user when resolving
-            m, l = list(bids.items())[0]
-            repo.ui.warn(
-                _(b' %s: ambiguous merge - picked %s action\n') % (f, m)
-            )
-            actions[f] = l[0]
-            continue
-        repo.ui.note(_(b'end of auction\n\n'))
-
-    if wctx.rev() is None:
-        fractions = _forgetremoved(wctx, mctx, branchmerge)
-        actions.update(fractions)
-
-    prunedactions = sparse.filterupdatesactions(
-        repo, wctx, mctx, branchmerge, actions
-    )
-    _resolvetrivial(repo, wctx, mctx, ancestors[0], actions)
-
-    return prunedactions, diverge, renamedelete
-
-
-def _getcwd():
-    try:
-        return encoding.getcwd()
-    except OSError as err:
-        if err.errno == errno.ENOENT:
-            return None
-        raise
-
-
-def batchremove(repo, wctx, actions):
-    """apply removes to the working directory
-
-    yields tuples for progress updates
-    """
-    verbose = repo.ui.verbose
-    cwd = _getcwd()
-    i = 0
-    for f, args, msg in actions:
-        repo.ui.debug(b" %s: %s -> r\n" % (f, msg))
-        if verbose:
-            repo.ui.note(_(b"removing %s\n") % f)
-        wctx[f].audit()
-        try:
-            wctx[f].remove(ignoremissing=True)
-        except OSError as inst:
-            repo.ui.warn(
-                _(b"update failed to remove %s: %s!\n") % (f, inst.strerror)
-            )
-        if i == 100:
-            yield i, f
-            i = 0
-        i += 1
-    if i > 0:
-        yield i, f
-
-    if cwd and not _getcwd():
-        # cwd was removed in the course of removing files; print a helpful
-        # warning.
-        repo.ui.warn(
-            _(
-                b"current directory was removed\n"
-                b"(consider changing to repo root: %s)\n"
-            )
-            % repo.root
-        )
-
-
-def batchget(repo, mctx, wctx, wantfiledata, actions):
-    """apply gets to the working directory
-
-    mctx is the context to get from
-
-    Yields arbitrarily many (False, tuple) for progress updates, followed by
-    exactly one (True, filedata). When wantfiledata is false, filedata is an
-    empty dict. When wantfiledata is true, filedata[f] is a triple (mode, size,
-    mtime) of the file f written for each action.
-    """
-    filedata = {}
-    verbose = repo.ui.verbose
-    fctx = mctx.filectx
-    ui = repo.ui
-    i = 0
-    with repo.wvfs.backgroundclosing(ui, expectedcount=len(actions)):
-        for f, (flags, backup), msg in actions:
-            repo.ui.debug(b" %s: %s -> g\n" % (f, msg))
-            if verbose:
-                repo.ui.note(_(b"getting %s\n") % f)
-
-            if backup:
-                # If a file or directory exists with the same name, back that
-                # up.  Otherwise, look to see if there is a file that conflicts
-                # with a directory this file is in, and if so, back that up.
-                conflicting = f
-                if not repo.wvfs.lexists(f):
-                    for p in pathutil.finddirs(f):
-                        if repo.wvfs.isfileorlink(p):
-                            conflicting = p
-                            break
-                if repo.wvfs.lexists(conflicting):
-                    orig = scmutil.backuppath(ui, repo, conflicting)
-                    util.rename(repo.wjoin(conflicting), orig)
-            wfctx = wctx[f]
-            wfctx.clearunknown()
-            atomictemp = ui.configbool(b"experimental", b"update.atomic-file")
-            size = wfctx.write(
-                fctx(f).data(),
-                flags,
-                backgroundclose=True,
-                atomictemp=atomictemp,
-            )
-            if wantfiledata:
-                s = wfctx.lstat()
-                mode = s.st_mode
-                mtime = s[stat.ST_MTIME]
-                filedata[f] = (mode, size, mtime)  # for dirstate.normal
-            if i == 100:
-                yield False, (i, f)
-                i = 0
-            i += 1
-    if i > 0:
-        yield False, (i, f)
-    yield True, filedata
-
-
-def _prefetchfiles(repo, ctx, actions):
-    """Invoke ``scmutil.prefetchfiles()`` for the files relevant to the dict
-    of merge actions.  ``ctx`` is the context being merged in."""
-
-    # Skipping 'a', 'am', 'f', 'r', 'dm', 'e', 'k', 'p' and 'pr', because they
-    # don't touch the context to be merged in.  'cd' is skipped, because
-    # changed/deleted never resolves to something from the remote side.
-    oplist = [
-        actions[a]
-        for a in (
-            ACTION_GET,
-            ACTION_DELETED_CHANGED,
-            ACTION_LOCAL_DIR_RENAME_GET,
-            ACTION_MERGE,
-        )
-    ]
-    prefetch = scmutil.prefetchfiles
-    matchfiles = scmutil.matchfiles
-    prefetch(
-        repo,
-        [ctx.rev()],
-        matchfiles(repo, [f for sublist in oplist for f, args, msg in sublist]),
-    )
-
-
-@attr.s(frozen=True)
-class updateresult(object):
-    updatedcount = attr.ib()
-    mergedcount = attr.ib()
-    removedcount = attr.ib()
-    unresolvedcount = attr.ib()
-
-    def isempty(self):
-        return not (
-            self.updatedcount
-            or self.mergedcount
-            or self.removedcount
-            or self.unresolvedcount
-        )
-
-
-def emptyactions():
-    """create an actions dict, to be populated and passed to applyupdates()"""
-    return {
-        m: []
-        for m in (
-            ACTION_ADD,
-            ACTION_ADD_MODIFIED,
-            ACTION_FORGET,
-            ACTION_GET,
-            ACTION_CHANGED_DELETED,
-            ACTION_DELETED_CHANGED,
-            ACTION_REMOVE,
-            ACTION_DIR_RENAME_MOVE_LOCAL,
-            ACTION_LOCAL_DIR_RENAME_GET,
-            ACTION_MERGE,
-            ACTION_EXEC,
-            ACTION_KEEP,
-            ACTION_PATH_CONFLICT,
-            ACTION_PATH_CONFLICT_RESOLVE,
-            ACTION_GET_OTHER_AND_STORE,
-        )
-    }
-
-
-def applyupdates(
-    repo, actions, wctx, mctx, overwrite, wantfiledata, labels=None
-):
-    """apply the merge action list to the working directory
-
-    wctx is the working copy context
-    mctx is the context to be merged into the working copy
-
-    Return a tuple of (counts, filedata), where counts is a tuple
-    (updated, merged, removed, unresolved) that describes how many
-    files were affected by the update, and filedata is as described in
-    batchget.
-    """
-
-    _prefetchfiles(repo, mctx, actions)
-
-    updated, merged, removed = 0, 0, 0
-    ms = mergestate.clean(repo, wctx.p1().node(), mctx.node(), labels)
-
-    # add ACTION_GET_OTHER_AND_STORE to mergestate
-    for e in actions[ACTION_GET_OTHER_AND_STORE]:
-        ms.addmergedother(e[0])
-
-    moves = []
-    for m, l in actions.items():
-        l.sort()
-
-    # 'cd' and 'dc' actions are treated like other merge conflicts
-    mergeactions = sorted(actions[ACTION_CHANGED_DELETED])
-    mergeactions.extend(sorted(actions[ACTION_DELETED_CHANGED]))
-    mergeactions.extend(actions[ACTION_MERGE])
-    for f, args, msg in mergeactions:
-        f1, f2, fa, move, anc = args
-        if f == b'.hgsubstate':  # merged internally
-            continue
-        if f1 is None:
-            fcl = filemerge.absentfilectx(wctx, fa)
-        else:
-            repo.ui.debug(b" preserving %s for resolve of %s\n" % (f1, f))
-            fcl = wctx[f1]
-        if f2 is None:
-            fco = filemerge.absentfilectx(mctx, fa)
-        else:
-            fco = mctx[f2]
-        actx = repo[anc]
-        if fa in actx:
-            fca = actx[fa]
-        else:
-            # TODO: move to absentfilectx
-            fca = repo.filectx(f1, fileid=nullrev)
-        ms.add(fcl, fco, fca, f)
-        if f1 != f and move:
-            moves.append(f1)
-
-    # remove renamed files after safely stored
-    for f in moves:
-        if wctx[f].lexists():
-            repo.ui.debug(b"removing %s\n" % f)
-            wctx[f].audit()
-            wctx[f].remove()
-
-    numupdates = sum(len(l) for m, l in actions.items() if m != ACTION_KEEP)
-    progress = repo.ui.makeprogress(
-        _(b'updating'), unit=_(b'files'), total=numupdates
-    )
-
-    if [a for a in actions[ACTION_REMOVE] if a[0] == b'.hgsubstate']:
-        subrepoutil.submerge(repo, wctx, mctx, wctx, overwrite, labels)
-
-    # record path conflicts
-    for f, args, msg in actions[ACTION_PATH_CONFLICT]:
-        f1, fo = args
-        s = repo.ui.status
-        s(
-            _(
-                b"%s: path conflict - a file or link has the same name as a "
-                b"directory\n"
-            )
-            % f
-        )
-        if fo == b'l':
-            s(_(b"the local file has been renamed to %s\n") % f1)
-        else:
-            s(_(b"the remote file has been renamed to %s\n") % f1)
-        s(_(b"resolve manually then use 'hg resolve --mark %s'\n") % f)
-        ms.addpath(f, f1, fo)
-        progress.increment(item=f)
-
-    # When merging in-memory, we can't support worker processes, so set the
-    # per-item cost at 0 in that case.
-    cost = 0 if wctx.isinmemory() else 0.001
-
-    # remove in parallel (must come before resolving path conflicts and getting)
-    prog = worker.worker(
-        repo.ui, cost, batchremove, (repo, wctx), actions[ACTION_REMOVE]
-    )
-    for i, item in prog:
-        progress.increment(step=i, item=item)
-    removed = len(actions[ACTION_REMOVE])
-
-    # resolve path conflicts (must come before getting)
-    for f, args, msg in actions[ACTION_PATH_CONFLICT_RESOLVE]:
-        repo.ui.debug(b" %s: %s -> pr\n" % (f, msg))
-        (f0,) = args
-        if wctx[f0].lexists():
-            repo.ui.note(_(b"moving %s to %s\n") % (f0, f))
-            wctx[f].audit()
-            wctx[f].write(wctx.filectx(f0).data(), wctx.filectx(f0).flags())
-            wctx[f0].remove()
-        progress.increment(item=f)
-
-    # get in parallel.
-    threadsafe = repo.ui.configbool(
-        b'experimental', b'worker.wdir-get-thread-safe'
-    )
-    prog = worker.worker(
-        repo.ui,
-        cost,
-        batchget,
-        (repo, mctx, wctx, wantfiledata),
-        actions[ACTION_GET],
-        threadsafe=threadsafe,
-        hasretval=True,
-    )
-    getfiledata = {}
-    for final, res in prog:
-        if final:
-            getfiledata = res
-        else:
-            i, item = res
-            progress.increment(step=i, item=item)
-    updated = len(actions[ACTION_GET])
-
-    if [a for a in actions[ACTION_GET] if a[0] == b'.hgsubstate']:
-        subrepoutil.submerge(repo, wctx, mctx, wctx, overwrite, labels)
-
-    # forget (manifest only, just log it) (must come first)
-    for f, args, msg in actions[ACTION_FORGET]:
-        repo.ui.debug(b" %s: %s -> f\n" % (f, msg))
-        progress.increment(item=f)
-
-    # re-add (manifest only, just log it)
-    for f, args, msg in actions[ACTION_ADD]:
-        repo.ui.debug(b" %s: %s -> a\n" % (f, msg))
-        progress.increment(item=f)
-
-    # re-add/mark as modified (manifest only, just log it)
-    for f, args, msg in actions[ACTION_ADD_MODIFIED]:
-        repo.ui.debug(b" %s: %s -> am\n" % (f, msg))
-        progress.increment(item=f)
-
-    # keep (noop, just log it)
-    for f, args, msg in actions[ACTION_KEEP]:
-        repo.ui.debug(b" %s: %s -> k\n" % (f, msg))
-        # no progress
-
-    # directory rename, move local
-    for f, args, msg in actions[ACTION_DIR_RENAME_MOVE_LOCAL]:
-        repo.ui.debug(b" %s: %s -> dm\n" % (f, msg))
-        progress.increment(item=f)
-        f0, flags = args
-        repo.ui.note(_(b"moving %s to %s\n") % (f0, f))
-        wctx[f].audit()
-        wctx[f].write(wctx.filectx(f0).data(), flags)
-        wctx[f0].remove()
-        updated += 1
-
-    # local directory rename, get
-    for f, args, msg in actions[ACTION_LOCAL_DIR_RENAME_GET]:
-        repo.ui.debug(b" %s: %s -> dg\n" % (f, msg))
-        progress.increment(item=f)
-        f0, flags = args
-        repo.ui.note(_(b"getting %s to %s\n") % (f0, f))
-        wctx[f].write(mctx.filectx(f0).data(), flags)
-        updated += 1
-
-    # exec
-    for f, args, msg in actions[ACTION_EXEC]:
-        repo.ui.debug(b" %s: %s -> e\n" % (f, msg))
-        progress.increment(item=f)
-        (flags,) = args
-        wctx[f].audit()
-        wctx[f].setflags(b'l' in flags, b'x' in flags)
-        updated += 1
-
-    # the ordering is important here -- ms.mergedriver will raise if the merge
-    # driver has changed, and we want to be able to bypass it when overwrite is
-    # True
-    usemergedriver = not overwrite and mergeactions and ms.mergedriver
-
-    if usemergedriver:
-        if wctx.isinmemory():
-            raise error.InMemoryMergeConflictsError(
-                b"in-memory merge does not support mergedriver"
-            )
-        ms.commit()
-        proceed = driverpreprocess(repo, ms, wctx, labels=labels)
-        # the driver might leave some files unresolved
-        unresolvedf = set(ms.unresolved())
-        if not proceed:
-            # XXX setting unresolved to at least 1 is a hack to make sure we
-            # error out
-            return updateresult(
-                updated, merged, removed, max(len(unresolvedf), 1)
-            )
-        newactions = []
-        for f, args, msg in mergeactions:
-            if f in unresolvedf:
-                newactions.append((f, args, msg))
-        mergeactions = newactions
-
-    try:
-        # premerge
-        tocomplete = []
-        for f, args, msg in mergeactions:
-            repo.ui.debug(b" %s: %s -> m (premerge)\n" % (f, msg))
-            progress.increment(item=f)
-            if f == b'.hgsubstate':  # subrepo states need updating
-                subrepoutil.submerge(
-                    repo, wctx, mctx, wctx.ancestor(mctx), overwrite, labels
-                )
-                continue
-            wctx[f].audit()
-            complete, r = ms.preresolve(f, wctx)
-            if not complete:
-                numupdates += 1
-                tocomplete.append((f, args, msg))
-
-        # merge
-        for f, args, msg in tocomplete:
-            repo.ui.debug(b" %s: %s -> m (merge)\n" % (f, msg))
-            progress.increment(item=f, total=numupdates)
-            ms.resolve(f, wctx)
-
-    finally:
-        ms.commit()
-
-    unresolved = ms.unresolvedcount()
-
-    if (
-        usemergedriver
-        and not unresolved
-        and ms.mdstate() != MERGE_DRIVER_STATE_SUCCESS
-    ):
-        if not driverconclude(repo, ms, wctx, labels=labels):
-            # XXX setting unresolved to at least 1 is a hack to make sure we
-            # error out
-            unresolved = max(unresolved, 1)
-
-        ms.commit()
-
-    msupdated, msmerged, msremoved = ms.counts()
-    updated += msupdated
-    merged += msmerged
-    removed += msremoved
-
-    extraactions = ms.actions()
-    if extraactions:
-        mfiles = {a[0] for a in actions[ACTION_MERGE]}
-        for k, acts in pycompat.iteritems(extraactions):
-            actions[k].extend(acts)
-            if k == ACTION_GET and wantfiledata:
-                # no filedata until mergestate is updated to provide it
-                for a in acts:
-                    getfiledata[a[0]] = None
-            # Remove these files from actions[ACTION_MERGE] as well. This is
-            # important because in recordupdates, files in actions[ACTION_MERGE]
-            # are processed after files in other actions, and the merge driver
-            # might add files to those actions via extraactions above. This can
-            # lead to a file being recorded twice, with poor results. This is
-            # especially problematic for actions[ACTION_REMOVE] (currently only
-            # possible with the merge driver in the initial merge process;
-            # interrupted merges don't go through this flow).
-            #
-            # The real fix here is to have indexes by both file and action so
-            # that when the action for a file is changed it is automatically
-            # reflected in the other action lists. But that involves a more
-            # complex data structure, so this will do for now.
-            #
-            # We don't need to do the same operation for 'dc' and 'cd' because
-            # those lists aren't consulted again.
-            mfiles.difference_update(a[0] for a in acts)
-
-        actions[ACTION_MERGE] = [
-            a for a in actions[ACTION_MERGE] if a[0] in mfiles
-        ]
-
-    progress.complete()
-    assert len(getfiledata) == (len(actions[ACTION_GET]) if wantfiledata else 0)
-    return updateresult(updated, merged, removed, unresolved), getfiledata
-
-
 def recordupdates(repo, actions, branchmerge, getfiledata):
     """record merge actions to the dirstate"""
     # remove (must come first)
@@ -2152,8 +798,7 b' def recordupdates(repo, actions, branchm'
 
     # resolve path conflicts
     for f, args, msg in actions.get(ACTION_PATH_CONFLICT_RESOLVE, []):
-        (f0,) = args
-        origf0 = repo.dirstate.copied(f0) or f0
+        (f0, origf0) = args
         repo.dirstate.add(f)
         repo.dirstate.copy(origf0, f)
         if f0 == origf0:
@@ -2232,594 +877,3 b' def recordupdates(repo, actions, branchm'
             repo.dirstate.copy(f0, f)
         else:
             repo.dirstate.normal(f)
-
-
-UPDATECHECK_ABORT = b'abort'  # handled at higher layers
-UPDATECHECK_NONE = b'none'
-UPDATECHECK_LINEAR = b'linear'
-UPDATECHECK_NO_CONFLICT = b'noconflict'
-
-
-def update(
-    repo,
-    node,
-    branchmerge,
-    force,
-    ancestor=None,
-    mergeancestor=False,
-    labels=None,
-    matcher=None,
-    mergeforce=False,
-    updatedirstate=True,
-    updatecheck=None,
-    wc=None,
-):
-    """
-    Perform a merge between the working directory and the given node
-
-    node = the node to update to
-    branchmerge = whether to merge between branches
-    force = whether to force branch merging or file overwriting
-    matcher = a matcher to filter file lists (dirstate not updated)
-    mergeancestor = whether it is merging with an ancestor. If true,
-      we should accept the incoming changes for any prompts that occur.
-      If false, merging with an ancestor (fast-forward) is only allowed
-      between different named branches. This flag is used by rebase extension
-      as a temporary fix and should be avoided in general.
-    labels = labels to use for base, local and other
-    mergeforce = whether the merge was run with 'merge --force' (deprecated): if
-      this is True, then 'force' should be True as well.
-
-    The table below shows all the behaviors of the update command given the
-    -c/--check and -C/--clean or no options, whether the working directory is
-    dirty, whether a revision is specified, and the relationship of the parent
-    rev to the target rev (linear or not). Match from top first. The -n
-    option doesn't exist on the command line, but represents the
-    experimental.updatecheck=noconflict option.
-
-    This logic is tested by test-update-branches.t.
-
-    -c  -C  -n  -m  dirty  rev  linear  |  result
-     y   y   *   *    *     *     *     |    (1)
-     y   *   y   *    *     *     *     |    (1)
-     y   *   *   y    *     *     *     |    (1)
-     *   y   y   *    *     *     *     |    (1)
-     *   y   *   y    *     *     *     |    (1)
-     *   *   y   y    *     *     *     |    (1)
-     *   *   *   *    *     n     n     |     x
-     *   *   *   *    n     *     *     |    ok
-     n   n   n   n    y     *     y     |   merge
-     n   n   n   n    y     y     n     |    (2)
-     n   n   n   y    y     *     *     |   merge
-     n   n   y   n    y     *     *     |  merge if no conflict
-     n   y   n   n    y     *     *     |  discard
-     y   n   n   n    y     *     *     |    (3)
-
-    x = can't happen
-    * = don't-care
-    1 = incompatible options (checked in commands.py)
-    2 = abort: uncommitted changes (commit or update --clean to discard changes)
-    3 = abort: uncommitted changes (checked in commands.py)
-
-    The merge is performed inside ``wc``, a workingctx-like objects. It defaults
-    to repo[None] if None is passed.
-
-    Return the same tuple as applyupdates().
-    """
-    # Avoid cycle.
-    from . import sparse
-
-    # This function used to find the default destination if node was None, but
-    # that's now in destutil.py.
-    assert node is not None
-    if not branchmerge and not force:
-        # TODO: remove the default once all callers that pass branchmerge=False
-        # and force=False pass a value for updatecheck. We may want to allow
-        # updatecheck='abort' to better suppport some of these callers.
-        if updatecheck is None:
-            updatecheck = UPDATECHECK_LINEAR
-        if updatecheck not in (
-            UPDATECHECK_NONE,
-            UPDATECHECK_LINEAR,
-            UPDATECHECK_NO_CONFLICT,
-        ):
-            raise ValueError(
-                r'Invalid updatecheck %r (can accept %r)'
-                % (
-                    updatecheck,
-                    (
-                        UPDATECHECK_NONE,
-                        UPDATECHECK_LINEAR,
-                        UPDATECHECK_NO_CONFLICT,
-                    ),
-                )
-            )
-    with repo.wlock():
-        if wc is None:
-            wc = repo[None]
-        pl = wc.parents()
-        p1 = pl[0]
-        p2 = repo[node]
-        if ancestor is not None:
-            pas = [repo[ancestor]]
-        else:
-            if repo.ui.configlist(b'merge', b'preferancestor') == [b'*']:
-                cahs = repo.changelog.commonancestorsheads(p1.node(), p2.node())
-                pas = [repo[anc] for anc in (sorted(cahs) or [nullid])]
-            else:
-                pas = [p1.ancestor(p2, warn=branchmerge)]
-
-        fp1, fp2, xp1, xp2 = p1.node(), p2.node(), bytes(p1), bytes(p2)
-
-        overwrite = force and not branchmerge
-        ### check phase
-        if not overwrite:
-            if len(pl) > 1:
-                raise error.Abort(_(b"outstanding uncommitted merge"))
-            ms = mergestate.read(repo)
-            if list(ms.unresolved()):
-                raise error.Abort(
-                    _(b"outstanding merge conflicts"),
-                    hint=_(b"use 'hg resolve' to resolve"),
-                )
-        if branchmerge:
-            if pas == [p2]:
-                raise error.Abort(
-                    _(
-                        b"merging with a working directory ancestor"
-                        b" has no effect"
-                    )
-                )
-            elif pas == [p1]:
-                if not mergeancestor and wc.branch() == p2.branch():
-                    raise error.Abort(
-                        _(b"nothing to merge"),
-                        hint=_(b"use 'hg update' or check 'hg heads'"),
-                    )
-            if not force and (wc.files() or wc.deleted()):
-                raise error.Abort(
-                    _(b"uncommitted changes"),
-                    hint=_(b"use 'hg status' to list changes"),
-                )
-            if not wc.isinmemory():
-                for s in sorted(wc.substate):
-                    wc.sub(s).bailifchanged()
-
-        elif not overwrite:
-            if p1 == p2:  # no-op update
-                # call the hooks and exit early
-                repo.hook(b'preupdate', throw=True, parent1=xp2, parent2=b'')
-                repo.hook(b'update', parent1=xp2, parent2=b'', error=0)
-                return updateresult(0, 0, 0, 0)
-
-            if updatecheck == UPDATECHECK_LINEAR and pas not in (
-                [p1],
-                [p2],
-            ):  # nonlinear
-                dirty = wc.dirty(missing=True)
-                if dirty:
-                    # Branching is a bit strange to ensure we do the minimal
-                    # amount of call to obsutil.foreground.
-                    foreground = obsutil.foreground(repo, [p1.node()])
-                    # note: the <node> variable contains a random identifier
-                    if repo[node].node() in foreground:
-                        pass  # allow updating to successors
-                    else:
-                        msg = _(b"uncommitted changes")
-                        hint = _(b"commit or update --clean to discard changes")
-                        raise error.UpdateAbort(msg, hint=hint)
-                else:
-                    # Allow jumping branches if clean and specific rev given
-                    pass
-
-        if overwrite:
-            pas = [wc]
-        elif not branchmerge:
-            pas = [p1]
-
-        # deprecated config: merge.followcopies
-        followcopies = repo.ui.configbool(b'merge', b'followcopies')
-        if overwrite:
-            followcopies = False
-        elif not pas[0]:
-            followcopies = False
-        if not branchmerge and not wc.dirty(missing=True):
-            followcopies = False
-
-        ### calculate phase
-        actionbyfile, diverge, renamedelete = calculateupdates(
-            repo,
-            wc,
-            p2,
-            pas,
-            branchmerge,
-            force,
-            mergeancestor,
-            followcopies,
-            matcher=matcher,
-            mergeforce=mergeforce,
-        )
-
-        if updatecheck == UPDATECHECK_NO_CONFLICT:
-            for f, (m, args, msg) in pycompat.iteritems(actionbyfile):
-                if m not in (
-                    ACTION_GET,
-                    ACTION_KEEP,
-                    ACTION_EXEC,
-                    ACTION_REMOVE,
-                    ACTION_PATH_CONFLICT_RESOLVE,
-                    ACTION_GET_OTHER_AND_STORE,
-                ):
-                    msg = _(b"conflicting changes")
-                    hint = _(b"commit or update --clean to discard changes")
-                    raise error.Abort(msg, hint=hint)
-
-        # Prompt and create actions. Most of this is in the resolve phase
-        # already, but we can't handle .hgsubstate in filemerge or
-        # subrepoutil.submerge yet so we have to keep prompting for it.
-        if b'.hgsubstate' in actionbyfile:
-            f = b'.hgsubstate'
-            m, args, msg = actionbyfile[f]
-            prompts = filemerge.partextras(labels)
-            prompts[b'f'] = f
-            if m == ACTION_CHANGED_DELETED:
-                if repo.ui.promptchoice(
-                    _(
-                        b"local%(l)s changed %(f)s which other%(o)s deleted\n"
-                        b"use (c)hanged version or (d)elete?"
-                        b"$$ &Changed $$ &Delete"
-                    )
-                    % prompts,
-                    0,
-                ):
-                    actionbyfile[f] = (ACTION_REMOVE, None, b'prompt delete')
-                elif f in p1:
-                    actionbyfile[f] = (
-                        ACTION_ADD_MODIFIED,
-                        None,
-                        b'prompt keep',
-                    )
-                else:
-                    actionbyfile[f] = (ACTION_ADD, None, b'prompt keep')
-            elif m == ACTION_DELETED_CHANGED:
-                f1, f2, fa, move, anc = args
-                flags = p2[f2].flags()
-                if (
-                    repo.ui.promptchoice(
-                        _(
-                            b"other%(o)s changed %(f)s which local%(l)s deleted\n"
-                            b"use (c)hanged version or leave (d)eleted?"
-                            b"$$ &Changed $$ &Deleted"
-                        )
-                        % prompts,
-                        0,
-                    )
-                    == 0
-                ):
-                    actionbyfile[f] = (
-                        ACTION_GET,
-                        (flags, False),
-                        b'prompt recreating',
-                    )
-                else:
-                    del actionbyfile[f]
-
-        # Convert to dictionary-of-lists format
-        actions = emptyactions()
-        for f, (m, args, msg) in pycompat.iteritems(actionbyfile):
-            if m not in actions:
-                actions[m] = []
-            actions[m].append((f, args, msg))
-
-        # ACTION_GET_OTHER_AND_STORE is a ACTION_GET + store in mergestate
-        for e in actions[ACTION_GET_OTHER_AND_STORE]:
-            actions[ACTION_GET].append(e)
-
-        if not util.fscasesensitive(repo.path):
-            # check collision between files only in p2 for clean update
-            if not branchmerge and (
-                force or not wc.dirty(missing=True, branch=False)
-            ):
-                _checkcollision(repo, p2.manifest(), None)
-            else:
-                _checkcollision(repo, wc.manifest(), actions)
-
-        # divergent renames
-        for f, fl in sorted(pycompat.iteritems(diverge)):
-            repo.ui.warn(
-                _(
-                    b"note: possible conflict - %s was renamed "
-                    b"multiple times to:\n"
-                )
-                % f
-            )
-            for nf in sorted(fl):
-                repo.ui.warn(b" %s\n" % nf)
-
-        # rename and delete
-        for f, fl in sorted(pycompat.iteritems(renamedelete)):
-            repo.ui.warn(
-                _(
-                    b"note: possible conflict - %s was deleted "
-                    b"and renamed to:\n"
-                )
-                % f
-            )
-            for nf in sorted(fl):
-                repo.ui.warn(b" %s\n" % nf)
-
-        ### apply phase
-        if not branchmerge:  # just jump to the new rev
-            fp1, fp2, xp1, xp2 = fp2, nullid, xp2, b''
-        # If we're doing a partial update, we need to skip updating
-        # the dirstate.
-        always = matcher is None or matcher.always()
-        updatedirstate = updatedirstate and always and not wc.isinmemory()
-        if updatedirstate:
-            repo.hook(b'preupdate', throw=True, parent1=xp1, parent2=xp2)
-            # note that we're in the middle of an update
-            repo.vfs.write(b'updatestate', p2.hex())
-
-        # Advertise fsmonitor when its presence could be useful.
-        #
-        # We only advertise when performing an update from an empty working
-        # directory. This typically only occurs during initial clone.
-        #
-        # We give users a mechanism to disable the warning in case it is
-        # annoying.
-        #
-        # We only allow on Linux and MacOS because that's where fsmonitor is
-        # considered stable.
-        fsmonitorwarning = repo.ui.configbool(b'fsmonitor', b'warn_when_unused')
-        fsmonitorthreshold = repo.ui.configint(
-            b'fsmonitor', b'warn_update_file_count'
-        )
-        try:
-            # avoid cycle: extensions -> cmdutil -> merge
-            from . import extensions
-
-            extensions.find(b'fsmonitor')
-            fsmonitorenabled = repo.ui.config(b'fsmonitor', b'mode') != b'off'
-            # We intentionally don't look at whether fsmonitor has disabled
-            # itself because a) fsmonitor may have already printed a warning
-            # b) we only care about the config state here.
-        except KeyError:
-            fsmonitorenabled = False
-
-        if (
-            fsmonitorwarning
-            and not fsmonitorenabled
-            and p1.node() == nullid
-            and len(actions[ACTION_GET]) >= fsmonitorthreshold
-            and pycompat.sysplatform.startswith((b'linux', b'darwin'))
-        ):
-            repo.ui.warn(
-                _(
-                    b'(warning: large working directory being used without '
-                    b'fsmonitor enabled; enable fsmonitor to improve performance; '
-                    b'see "hg help -e fsmonitor")\n'
-                )
-            )
-
-        wantfiledata = updatedirstate and not branchmerge
-        stats, getfiledata = applyupdates(
-            repo, actions, wc, p2, overwrite, wantfiledata, labels=labels
-        )
-
-        if updatedirstate:
-            with repo.dirstate.parentchange():
-                repo.setparents(fp1, fp2)
-                recordupdates(repo, actions, branchmerge, getfiledata)
-                # update completed, clear state
-                util.unlink(repo.vfs.join(b'updatestate'))
-
-                if not branchmerge:
-                    repo.dirstate.setbranch(p2.branch())
-
-    # If we're updating to a location, clean up any stale temporary includes
-    # (ex: this happens during hg rebase --abort).
-    if not branchmerge:
-        sparse.prunetemporaryincludes(repo)
-
-    if updatedirstate:
-        repo.hook(
-            b'update', parent1=xp1, parent2=xp2, error=stats.unresolvedcount
-        )
-    return stats
-
-
-def merge(ctx, labels=None, force=False, wc=None):
-    """Merge another topological branch into the working copy.
-
-    force = whether the merge was run with 'merge --force' (deprecated)
-    """
-
-    return update(
-        ctx.repo(),
-        ctx.rev(),
-        labels=labels,
-        branchmerge=True,
-        force=force,
-        mergeforce=force,
-        wc=wc,
-    )
-
-
-def clean_update(ctx, wc=None):
-    """Do a clean update to the given commit.
-
-    This involves updating to the commit and discarding any changes in the
-    working copy.
-    """
-    return update(ctx.repo(), ctx.rev(), branchmerge=False, force=True, wc=wc)
-
-
-def revert_to(ctx, matcher=None, wc=None):
-    """Revert the working copy to the given commit.
-
-    The working copy will keep its current parent(s) but its content will
-    be the same as in the given commit.
-    """
-
-    return update(
-        ctx.repo(),
-        ctx.rev(),
-        branchmerge=False,
-        force=True,
-        updatedirstate=False,
-        matcher=matcher,
-        wc=wc,
-    )
-
-
-def graft(
-    repo,
-    ctx,
-    base=None,
-    labels=None,
-    keepparent=False,
-    keepconflictparent=False,
-    wctx=None,
-):
-    """Do a graft-like merge.
-
-    This is a merge where the merge ancestor is chosen such that one
-    or more changesets are grafted onto the current changeset. In
-    addition to the merge, this fixes up the dirstate to include only
-    a single parent (if keepparent is False) and tries to duplicate any
-    renames/copies appropriately.
-
-    ctx - changeset to rebase
-    base - merge base, or ctx.p1() if not specified
-    labels - merge labels eg ['local', 'graft']
-    keepparent - keep second parent if any
-    keepconflictparent - if unresolved, keep parent used for the merge
-
-    """
-    # If we're grafting a descendant onto an ancestor, be sure to pass
-    # mergeancestor=True to update. This does two things: 1) allows the merge if
-    # the destination is the same as the parent of the ctx (so we can use graft
-    # to copy commits), and 2) informs update that the incoming changes are
-    # newer than the destination so it doesn't prompt about "remote changed foo
-    # which local deleted".
-    # We also pass mergeancestor=True when base is the same revision as p1. 2)
-    # doesn't matter as there can't possibly be conflicts, but 1) is necessary.
-    wctx = wctx or repo[None]
-    pctx = wctx.p1()
-    base = base or ctx.p1()
-    mergeancestor = (
-        repo.changelog.isancestor(pctx.node(), ctx.node())
-        or pctx.rev() == base.rev()
-    )
-
-    stats = update(
-        repo,
-        ctx.node(),
-        True,
-        True,
-        base.node(),
-        mergeancestor=mergeancestor,
-        labels=labels,
-        wc=wctx,
-    )
-
-    if keepconflictparent and stats.unresolvedcount:
-        pother = ctx.node()
-    else:
-        pother = nullid
-        parents = ctx.parents()
-        if keepparent and len(parents) == 2 and base in parents:
-            parents.remove(base)
-            pother = parents[0].node()
-    # Never set both parents equal to each other
-    if pother == pctx.node():
-        pother = nullid
-
-    if wctx.isinmemory():
-        wctx.setparents(pctx.node(), pother)
-        # fix up dirstate for copies and renames
-        copies.graftcopies(wctx, ctx, base)
-    else:
-        with repo.dirstate.parentchange():
-            repo.setparents(pctx.node(), pother)
-            repo.dirstate.write(repo.currenttransaction())
-            # fix up dirstate for copies and renames
-            copies.graftcopies(wctx, ctx, base)
-    return stats
-
-
-def purge(
-    repo,
-    matcher,
-    unknown=True,
-    ignored=False,
-    removeemptydirs=True,
-    removefiles=True,
-    abortonerror=False,
-    noop=False,
-):
-    """Purge the working directory of untracked files.
-
-    ``matcher`` is a matcher configured to scan the working directory -
-    potentially a subset.
-
-    ``unknown`` controls whether unknown files should be purged.
-
-    ``ignored`` controls whether ignored files should be purged.
-
-    ``removeemptydirs`` controls whether empty directories should be removed.
-
-    ``removefiles`` controls whether files are removed.
-
-    ``abortonerror`` causes an exception to be raised if an error occurs
-    deleting a file or directory.
-
-    ``noop`` controls whether to actually remove files. If not defined, actions
-    will be taken.
-
-    Returns an iterable of relative paths in the working directory that were
-    or would be removed.
-    """
-
-    def remove(removefn, path):
-        try:
-            removefn(path)
-        except OSError:
-            m = _(b'%s cannot be removed') % path
-            if abortonerror:
-                raise error.Abort(m)
-            else:
-                repo.ui.warn(_(b'warning: %s\n') % m)
-
-    # There's no API to copy a matcher. So mutate the passed matcher and
-    # restore it when we're done.
-    oldtraversedir = matcher.traversedir
-
-    res = []
-
-    try:
-        if removeemptydirs:
-            directories = []
-            matcher.traversedir = directories.append
-
-        status = repo.status(match=matcher, ignored=ignored, unknown=unknown)
-
-        if removefiles:
-            for f in sorted(status.unknown + status.ignored):
-                if not noop:
-                    repo.ui.note(_(b'removing file %s\n') % f)
-                    remove(repo.wvfs.unlink, f)
-                res.append(f)
-
-        if removeemptydirs:
-            for f in sorted(directories, reverse=True):
-                if matcher(f) and not repo.wvfs.listdir(f):
-                    if not noop:
-                        repo.ui.note(_(b'removing directory %s\n') % f)
-                        remove(repo.wvfs.rmdir, f)
-                    res.append(f)
-
-        return res
-
-    finally:
-        matcher.traversedir = oldtraversedir

mercurial/narrowspec.py

@@ -14,6 +14,7 b' from . import ('
     error,
     match as matchmod,
     merge,
+    mergestate as mergestatemod,
     scmutil,
     sparse,
     util,
@@ -272,7 +273,7 b' def _deletecleanfiles(repo, files):'
 
 def _writeaddedfiles(repo, pctx, files):
     actions = merge.emptyactions()
-    addgaction = actions[merge.ACTION_GET].append
+    addgaction = actions[mergestatemod.ACTION_GET].append
     mf = repo[b'.'].manifest()
     for f in files:
         if not repo.wvfs.exists(f):

mercurial/obsutil.py

@@ -13,6 +13,7 b' from .i18n import _'
 from . import (
     diffutil,
     encoding,
+    error,
     node as nodemod,
     phases,
     pycompat,
@@ -481,14 +482,23 b' def geteffectflag(source, successors):'
     return effects
 
 
-def getobsoleted(repo, tr):
-    """return the set of pre-existing revisions obsoleted by a transaction"""
+def getobsoleted(repo, tr=None, changes=None):
+    """return the set of pre-existing revisions obsoleted by a transaction
+
+    Either the transaction or changes item of the transaction (for hooks)
+    must be provided, but not both.
+    """
+    if (tr is None) == (changes is None):
+        e = b"exactly one of tr and changes must be provided"
+        raise error.ProgrammingError(e)
     torev = repo.unfiltered().changelog.index.get_rev
     phase = repo._phasecache.phase
     succsmarkers = repo.obsstore.successors.get
     public = phases.public
-    addedmarkers = tr.changes[b'obsmarkers']
-    origrepolen = tr.changes[b'origrepolen']
+    if changes is None:
+        changes = tr.changes
+    addedmarkers = changes[b'obsmarkers']
+    origrepolen = changes[b'origrepolen']
     seenrevs = set()
     obsoleted = set()
     for mark in addedmarkers:

mercurial/patch.py

@@ -785,7 +785,7 b' class patchfile(object):'
             for l in x.hunk:
                 lines.append(l)
                 if l[-1:] != b'\n':
-                    lines.append(b"\n\\ No newline at end of file\n")
+                    lines.append(b'\n' + diffhelper.MISSING_NEWLINE_MARKER)
         self.backend.writerej(self.fname, len(self.rej), self.hunks, lines)
 
     def apply(self, h):
@@ -1069,7 +1069,7 b' class recordhunk(object):'
 
     def write(self, fp):
         delta = len(self.before) + len(self.after)
-        if self.after and self.after[-1] == b'\\ No newline at end of file\n':
+        if self.after and self.after[-1] == diffhelper.MISSING_NEWLINE_MARKER:
             delta -= 1
         fromlen = delta + self.removed
         tolen = delta + self.added
@@ -2666,7 +2666,11 b' def diffhunks('
     prefetchmatch = scmutil.matchfiles(
         repo, list(modifiedset | addedset | removedset)
     )
-    scmutil.prefetchfiles(repo, [ctx1.rev(), ctx2.rev()], prefetchmatch)
+    revmatches = [
+        (ctx1.rev(), prefetchmatch),
+        (ctx2.rev(), prefetchmatch),
+    ]
+    scmutil.prefetchfiles(repo, revmatches)
 
     def difffn(opts, losedata):
         return trydiff(
@@ -2918,6 +2922,18 b' def _filepairs(modified, added, removed,'
         yield f1, f2, copyop
 
 
+def _gitindex(text):
+    if not text:
+        text = b""
+    l = len(text)
+    s = hashutil.sha1(b'blob %d\0' % l)
+    s.update(text)
+    return hex(s.digest())
+
+
+_gitmode = {b'l': b'120000', b'x': b'100755', b'': b'100644'}
+
+
 def trydiff(
     repo,
     revs,
@@ -2940,14 +2956,6 b' def trydiff('
     pathfn is applied to every path in the diff output.
     '''
 
-    def gitindex(text):
-        if not text:
-            text = b""
-        l = len(text)
-        s = hashutil.sha1(b'blob %d\0' % l)
-        s.update(text)
-        return hex(s.digest())
-
     if opts.noprefix:
         aprefix = bprefix = b''
     else:
@@ -2964,8 +2972,6 b' def trydiff('
     date1 = dateutil.datestr(ctx1.date())
     date2 = dateutil.datestr(ctx2.date())
 
-    gitmode = {b'l': b'120000', b'x': b'100755', b'': b'100644'}
-
     if not pathfn:
         pathfn = lambda f: f
 
@@ -3019,11 +3025,11 b' def trydiff('
                 b'diff --git %s%s %s%s' % (aprefix, path1, bprefix, path2)
             )
             if not f1:  # added
-                header.append(b'new file mode %s' % gitmode[flag2])
+                header.append(b'new file mode %s' % _gitmode[flag2])
             elif not f2:  # removed
-                header.append(b'deleted file mode %s' % gitmode[flag1])
+                header.append(b'deleted file mode %s' % _gitmode[flag1])
             else:  # modified/copied/renamed
-                mode1, mode2 = gitmode[flag1], gitmode[flag2]
+                mode1, mode2 = _gitmode[flag1], _gitmode[flag2]
                 if mode1 != mode2:
                     header.append(b'old mode %s' % mode1)
                     header.append(b'new mode %s' % mode2)
@@ -3067,39 +3073,66 b' def trydiff('
             if fctx2 is not None:
                 content2 = fctx2.data()
 
-        if binary and opts.git and not opts.nobinary:
-            text = mdiff.b85diff(content1, content2)
-            if text:
-                header.append(
-                    b'index %s..%s' % (gitindex(content1), gitindex(content2))
+        data1 = (ctx1, fctx1, path1, flag1, content1, date1)
+        data2 = (ctx2, fctx2, path2, flag2, content2, date2)
+        yield diffcontent(data1, data2, header, binary, opts)
+
+
+def diffcontent(data1, data2, header, binary, opts):
+    """ diffs two versions of a file.
+
+    data1 and data2 are tuples containg:
+
+        * ctx: changeset for the file
+        * fctx: file context for that file
+        * path1: name of the file
+        * flag: flags of the file
+        * content: full content of the file (can be null in case of binary)
+        * date: date of the changeset
+
+    header: the patch header
+    binary: whether the any of the version of file is binary or not
+    opts:   user passed options
+
+    It exists as a separate function so that extensions like extdiff can wrap
+    it and use the file content directly.
+    """
+
+    ctx1, fctx1, path1, flag1, content1, date1 = data1
+    ctx2, fctx2, path2, flag2, content2, date2 = data2
+    if binary and opts.git and not opts.nobinary:
+        text = mdiff.b85diff(content1, content2)
+        if text:
+            header.append(
+                b'index %s..%s' % (_gitindex(content1), _gitindex(content2))
+            )
+        hunks = ((None, [text]),)
+    else:
+        if opts.git and opts.index > 0:
+            flag = flag1
+            if flag is None:
+                flag = flag2
+            header.append(
+                b'index %s..%s %s'
+                % (
+                    _gitindex(content1)[0 : opts.index],
+                    _gitindex(content2)[0 : opts.index],
+                    _gitmode[flag],
                 )
-            hunks = ((None, [text]),)
-        else:
-            if opts.git and opts.index > 0:
-                flag = flag1
-                if flag is None:
-                    flag = flag2
-                header.append(
-                    b'index %s..%s %s'
-                    % (
-                        gitindex(content1)[0 : opts.index],
-                        gitindex(content2)[0 : opts.index],
-                        gitmode[flag],
-                    )
-                )
-
-            uheaders, hunks = mdiff.unidiff(
-                content1,
-                date1,
-                content2,
-                date2,
-                path1,
-                path2,
-                binary=binary,
-                opts=opts,
             )
-            header.extend(uheaders)
-        yield fctx1, fctx2, header, hunks
+
+        uheaders, hunks = mdiff.unidiff(
+            content1,
+            date1,
+            content2,
+            date2,
+            path1,
+            path2,
+            binary=binary,
+            opts=opts,
+        )
+        header.extend(uheaders)
+    return fctx1, fctx2, header, hunks
 
 
 def diffstatsum(stats):

mercurial/pathutil.py

@@ -1,5 +1,6 b''
 from __future__ import absolute_import
 
+import contextlib
 import errno
 import os
 import posixpath
@@ -148,6 +149,19 b' class pathauditor(object):'
         except (OSError, error.Abort):
             return False
 
+    @contextlib.contextmanager
+    def cached(self):
+        if self._cached:
+            yield
+        else:
+            try:
+                self._cached = True
+                yield
+            finally:
+                self.audited.clear()
+                self.auditeddir.clear()
+                self._cached = False
+
 
 def canonpath(root, cwd, myname, auditor=None):
     '''return the canonical path of myname, given cwd and root

mercurial/phases.py

@@ -128,25 +128,28 b' from . import ('
 
 _fphasesentry = struct.Struct(b'>i20s')
 
-INTERNAL_FLAG = 64  # Phases for mercurial internal usage only
-HIDEABLE_FLAG = 32  # Phases that are hideable
-
 # record phase index
 public, draft, secret = range(3)
-internal = INTERNAL_FLAG | HIDEABLE_FLAG
-archived = HIDEABLE_FLAG
-allphases = list(range(internal + 1))
-trackedphases = allphases[1:]
+archived = 32  # non-continuous for compatibility
+internal = 96  # non-continuous for compatibility
+allphases = (public, draft, secret, archived, internal)
+trackedphases = (draft, secret, archived, internal)
 # record phase names
 cmdphasenames = [b'public', b'draft', b'secret']  # known to `hg phase` command
-phasenames = [None] * len(allphases)
-phasenames[: len(cmdphasenames)] = cmdphasenames
+phasenames = dict(enumerate(cmdphasenames))
 phasenames[archived] = b'archived'
 phasenames[internal] = b'internal'
+# map phase name to phase number
+phasenumber = {name: phase for phase, name in phasenames.items()}
+# like phasenumber, but also include maps for the numeric and binary
+# phase number to the phase number
+phasenumber2 = phasenumber.copy()
+phasenumber2.update({phase: phase for phase in phasenames})
+phasenumber2.update({b'%i' % phase: phase for phase in phasenames})
 # record phase property
-mutablephases = tuple(allphases[1:])
-remotehiddenphases = tuple(allphases[2:])
-localhiddenphases = tuple(p for p in allphases if p & HIDEABLE_FLAG)
+mutablephases = (draft, secret, archived, internal)
+remotehiddenphases = (secret, archived, internal)
+localhiddenphases = (internal, archived)
 
 
 def supportinternal(repo):
@@ -167,7 +170,7 b' def _readroots(repo, phasedefaults=None)'
     """
     repo = repo.unfiltered()
     dirty = False
-    roots = [set() for i in allphases]
+    roots = {i: set() for i in allphases}
     try:
         f, pending = txnutil.trypending(repo.root, repo.svfs, b'phaseroots')
         try:
@@ -189,11 +192,10 b' def _readroots(repo, phasedefaults=None)'
 def binaryencode(phasemapping):
     """encode a 'phase -> nodes' mapping into a binary stream
 
-    Since phases are integer the mapping is actually a python list:
-    [[PUBLIC_HEADS], [DRAFTS_HEADS], [SECRET_HEADS]]
+    The revision lists are encoded as (phase, root) pairs.
     """
     binarydata = []
-    for phase, nodes in enumerate(phasemapping):
+    for phase, nodes in pycompat.iteritems(phasemapping):
         for head in nodes:
             binarydata.append(_fphasesentry.pack(phase, head))
     return b''.join(binarydata)
@@ -202,8 +204,9 b' def binaryencode(phasemapping):'
 def binarydecode(stream):
     """decode a binary stream into a 'phase -> nodes' mapping
 
-    Since phases are integer the mapping is actually a python list."""
-    headsbyphase = [[] for i in allphases]
+    The (phase, root) pairs are turned back into a dictionary with
+    the phase as index and the aggregated roots of that phase as value."""
+    headsbyphase = {i: [] for i in allphases}
     entrysize = _fphasesentry.size
     while True:
         entry = stream.read(entrysize)
@@ -323,6 +326,38 b' class phasecache(object):'
             self.filterunknown(repo)
             self.opener = repo.svfs
 
+    def hasnonpublicphases(self, repo):
+        """detect if there are revisions with non-public phase"""
+        repo = repo.unfiltered()
+        cl = repo.changelog
+        if len(cl) >= self._loadedrevslen:
+            self.invalidate()
+            self.loadphaserevs(repo)
+        return any(
+            revs
+            for phase, revs in pycompat.iteritems(self.phaseroots)
+            if phase != public
+        )
+
+    def nonpublicphaseroots(self, repo):
+        """returns the roots of all non-public phases
+
+        The roots are not minimized, so if the secret revisions are
+        descendants of draft revisions, their roots will still be present.
+        """
+        repo = repo.unfiltered()
+        cl = repo.changelog
+        if len(cl) >= self._loadedrevslen:
+            self.invalidate()
+            self.loadphaserevs(repo)
+        return set().union(
+            *[
+                revs
+                for phase, revs in pycompat.iteritems(self.phaseroots)
+                if phase != public
+            ]
+        )
+
     def getrevset(self, repo, phases, subset=None):
         """return a smartset for the given phases"""
         self.loadphaserevs(repo)  # ensure phase's sets are loaded
@@ -380,7 +415,7 b' class phasecache(object):'
         # Shallow copy meant to ensure isolation in
         # advance/retractboundary(), nothing more.
         ph = self.__class__(None, None, _load=False)
-        ph.phaseroots = self.phaseroots[:]
+        ph.phaseroots = self.phaseroots.copy()
         ph.dirty = self.dirty
         ph.opener = self.opener
         ph._loadedrevslen = self._loadedrevslen
@@ -400,17 +435,12 b' class phasecache(object):'
 
     def _getphaserevsnative(self, repo):
         repo = repo.unfiltered()
-        nativeroots = []
-        for phase in trackedphases:
-            nativeroots.append(
-                pycompat.maplist(repo.changelog.rev, self.phaseroots[phase])
-            )
-        return repo.changelog.computephases(nativeroots)
+        return repo.changelog.computephases(self.phaseroots)
 
     def _computephaserevspure(self, repo):
         repo = repo.unfiltered()
         cl = repo.changelog
-        self._phasesets = [set() for phase in allphases]
+        self._phasesets = {phase: set() for phase in allphases}
         lowerroots = set()
         for phase in reversed(trackedphases):
             roots = pycompat.maplist(cl.rev, self.phaseroots[phase])
@@ -464,7 +494,7 b' class phasecache(object):'
             f.close()
 
     def _write(self, fp):
-        for phase, roots in enumerate(self.phaseroots):
+        for phase, roots in pycompat.iteritems(self.phaseroots):
             for h in sorted(roots):
                 fp.write(b'%i %s\n' % (phase, hex(h)))
         self.dirty = False
@@ -511,7 +541,7 b' class phasecache(object):'
 
         changes = set()  # set of revisions to be changed
         delroots = []  # set of root deleted by this path
-        for phase in pycompat.xrange(targetphase + 1, len(allphases)):
+        for phase in (phase for phase in allphases if phase > targetphase):
             # filter nodes that are not in a compatible phase already
             nodes = [
                 n for n in nodes if self.phase(repo, repo[n].rev()) >= phase
@@ -546,7 +576,11 b' class phasecache(object):'
         return changes
 
     def retractboundary(self, repo, tr, targetphase, nodes):
-        oldroots = self.phaseroots[: targetphase + 1]
+        oldroots = {
+            phase: revs
+            for phase, revs in pycompat.iteritems(self.phaseroots)
+            if phase <= targetphase
+        }
         if tr is None:
             phasetracking = None
         else:
@@ -565,7 +599,7 b' class phasecache(object):'
             # find the phase of the affected revision
             for phase in pycompat.xrange(targetphase, -1, -1):
                 if phase:
-                    roots = oldroots[phase]
+                    roots = oldroots.get(phase, [])
                     revs = set(repo.revs(b'%ln::%ld', roots, affected))
                     affected -= revs
                 else:  # public phase
@@ -583,30 +617,32 b' class phasecache(object):'
             raise error.ProgrammingError(msg)
 
         repo = repo.unfiltered()
-        currentroots = self.phaseroots[targetphase]
+        torev = repo.changelog.rev
+        tonode = repo.changelog.node
+        currentroots = {torev(node) for node in self.phaseroots[targetphase]}
         finalroots = oldroots = set(currentroots)
+        newroots = [torev(node) for node in nodes]
         newroots = [
-            n for n in nodes if self.phase(repo, repo[n].rev()) < targetphase
+            rev for rev in newroots if self.phase(repo, rev) < targetphase
         ]
+
         if newroots:
-
-            if nullid in newroots:
+            if nullrev in newroots:
                 raise error.Abort(_(b'cannot change null revision phase'))
-            currentroots = currentroots.copy()
             currentroots.update(newroots)
 
             # Only compute new roots for revs above the roots that are being
             # retracted.
-            minnewroot = min(repo[n].rev() for n in newroots)
-            aboveroots = [
-                n for n in currentroots if repo[n].rev() >= minnewroot
-            ]
-            updatedroots = repo.set(b'roots(%ln::)', aboveroots)
+            minnewroot = min(newroots)
+            aboveroots = [rev for rev in currentroots if rev >= minnewroot]
+            updatedroots = repo.revs(b'roots(%ld::)', aboveroots)
 
-            finalroots = {n for n in currentroots if repo[n].rev() < minnewroot}
-            finalroots.update(ctx.node() for ctx in updatedroots)
+            finalroots = {rev for rev in currentroots if rev < minnewroot}
+            finalroots.update(updatedroots)
         if finalroots != oldroots:
-            self._updateroots(targetphase, finalroots, tr)
+            self._updateroots(
+                targetphase, {tonode(rev) for rev in finalroots}, tr
+            )
             return True
         return False
 
@@ -617,7 +653,7 b' class phasecache(object):'
         """
         filtered = False
         has_node = repo.changelog.index.has_node  # to filter unknown nodes
-        for phase, nodes in enumerate(self.phaseroots):
+        for phase, nodes in pycompat.iteritems(self.phaseroots):
             missing = sorted(node for node in nodes if not has_node(node))
             if missing:
                 for mnode in missing:
@@ -742,7 +778,7 b' def subsetphaseheads(repo, subset):'
     """
     cl = repo.changelog
 
-    headsbyphase = [[] for i in allphases]
+    headsbyphase = {i: [] for i in allphases}
     # No need to keep track of secret phase; any heads in the subset that
     # are not mentioned are implicitly secret.
     for phase in allphases[:secret]:
@@ -753,12 +789,12 b' def subsetphaseheads(repo, subset):'
 
 def updatephases(repo, trgetter, headsbyphase):
     """Updates the repo with the given phase heads"""
-    # Now advance phase boundaries of all but secret phase
+    # Now advance phase boundaries of all phases
     #
     # run the update (and fetch transaction) only if there are actually things
     # to update. This avoid creating empty transaction during no-op operation.
 
-    for phase in allphases[:-1]:
+    for phase in allphases:
         revset = b'%ln - _phase(%s)'
         heads = [c.node() for c in repo.set(revset, headsbyphase[phase], phase)]
         if heads:
@@ -873,18 +909,16 b' def newcommitphase(ui):'
     """
     v = ui.config(b'phases', b'new-commit')
     try:
-        return phasenames.index(v)
-    except ValueError:
-        try:
-            return int(v)
-        except ValueError:
-            msg = _(b"phases.new-commit: not a valid phase name ('%s')")
-            raise error.ConfigError(msg % v)
+        return phasenumber2[v]
+    except KeyError:
+        raise error.ConfigError(
+            _(b"phases.new-commit: not a valid phase name ('%s')") % v
+        )
 
 
 def hassecret(repo):
     """utility function that check if a repo have any secret changeset."""
-    return bool(repo._phasecache.phaseroots[2])
+    return bool(repo._phasecache.phaseroots[secret])
 
 
 def preparehookargs(node, old, new):

mercurial/policy.py

@@ -80,7 +80,7 b' def _importfrom(pkgname, modname):'
     ('cext', 'bdiff'): 3,
     ('cext', 'mpatch'): 1,
     ('cext', 'osutil'): 4,
-    ('cext', 'parsers'): 16,
+    ('cext', 'parsers'): 17,
 }
 
 # map import request to other package or module

mercurial/posix.py

@@ -538,10 +538,6 b' def shellsplit(s):'
     return pycompat.shlexsplit(s, posix=True)
 
 
-def quotecommand(cmd):
-    return cmd
-
-
 def testpid(pid):
     '''return False if pid dead, True if running or not sure'''
     if pycompat.sysplatform == b'OpenVMS':

mercurial/pycompat.py

@@ -98,7 +98,6 b' if ispy3:'
     import codecs
     import functools
     import io
-    import locale
     import struct
 
     if os.name == r'nt' and sys.version_info >= (3, 6):
@@ -143,29 +142,12 b' if ispy3:'
 
     long = int
 
-    # Warning: sys.stdout.buffer and sys.stderr.buffer do not necessarily have
-    # the same buffering behavior as sys.stdout and sys.stderr. The interpreter
-    # initializes them with block-buffered streams or unbuffered streams (when
-    # the -u option or the PYTHONUNBUFFERED environment variable is set), never
-    # with a line-buffered stream.
-    # TODO: .buffer might not exist if std streams were replaced; we'll need
-    # a silly wrapper to make a bytes stream backed by a unicode one.
-    stdin = sys.stdin.buffer
-    stdout = sys.stdout.buffer
-    stderr = sys.stderr.buffer
-
     if getattr(sys, 'argv', None) is not None:
         # On POSIX, the char** argv array is converted to Python str using
-        # Py_DecodeLocale(). The inverse of this is Py_EncodeLocale(), which isn't
-        # directly callable from Python code. So, we need to emulate it.
-        # Py_DecodeLocale() calls mbstowcs() and falls back to mbrtowc() with
-        # surrogateescape error handling on failure. These functions take the
-        # current system locale into account. So, the inverse operation is to
-        # .encode() using the system locale's encoding and using the
-        # surrogateescape error handler. The only tricky part here is getting
-        # the system encoding correct, since `locale.getlocale()` can return
-        # None. We fall back to the filesystem encoding if lookups via `locale`
-        # fail, as this seems like a reasonable thing to do.
+        # Py_DecodeLocale(). The inverse of this is Py_EncodeLocale(), which
+        # isn't directly callable from Python code. In practice, os.fsencode()
+        # can be used instead (this is recommended by Python's documentation
+        # for sys.argv).
         #
         # On Windows, the wchar_t **argv is passed into the interpreter as-is.
         # Like POSIX, we need to emulate what Py_EncodeLocale() would do. But
@@ -178,19 +160,7 b' if ispy3:'
         if os.name == r'nt':
             sysargv = [a.encode("mbcs", "ignore") for a in sys.argv]
         else:
-
-            def getdefaultlocale_if_known():
-                try:
-                    return locale.getdefaultlocale()
-                except ValueError:
-                    return None, None
-
-            encoding = (
-                locale.getlocale()[1]
-                or getdefaultlocale_if_known()[1]
-                or sys.getfilesystemencoding()
-            )
-            sysargv = [a.encode(encoding, "surrogateescape") for a in sys.argv]
+            sysargv = [fsencode(a) for a in sys.argv]
 
     bytechr = struct.Struct('>B').pack
     byterepr = b'%r'.__mod__
@@ -495,9 +465,6 b' else:'
     osaltsep = os.altsep
     osdevnull = os.devnull
     long = long
-    stdin = sys.stdin
-    stdout = sys.stdout
-    stderr = sys.stderr
     if getattr(sys, 'argv', None) is not None:
         sysargv = sys.argv
     sysplatform = sys.platform

mercurial/repair.py

@@ -66,7 +66,7 b' def backupbundle('
     else:
         bundletype = b"HG10UN"
 
-    outgoing = discovery.outgoing(repo, missingroots=bases, missingheads=heads)
+    outgoing = discovery.outgoing(repo, missingroots=bases, ancestorsof=heads)
     contentopts = {
         b'cg.version': cgversion,
         b'obsolescence': obsolescence,

mercurial/repoview.py

@@ -129,10 +129,8 b' def computeunserved(repo, visibilityexce'
 def computemutable(repo, visibilityexceptions=None):
     assert not repo.changelog.filteredrevs
     # fast check to avoid revset call on huge repo
-    if any(repo._phasecache.phaseroots[1:]):
-        getphase = repo._phasecache.phase
-        maymutable = filterrevs(repo, b'base')
-        return frozenset(r for r in maymutable if getphase(repo, r))
+    if repo._phasecache.hasnonpublicphases(repo):
+        return frozenset(repo._phasecache.getrevset(repo, phases.mutablephases))
     return frozenset()
 
 
@@ -154,9 +152,9 b' def computeimpactable(repo, visibilityex'
     assert not repo.changelog.filteredrevs
     cl = repo.changelog
     firstmutable = len(cl)
-    for roots in repo._phasecache.phaseroots[1:]:
-        if roots:
-            firstmutable = min(firstmutable, min(cl.rev(r) for r in roots))
+    roots = repo._phasecache.nonpublicphaseroots(repo)
+    if roots:
+        firstmutable = min(firstmutable, min(cl.rev(r) for r in roots))
     # protect from nullrev root
     firstmutable = max(0, firstmutable)
     return frozenset(pycompat.xrange(firstmutable, len(cl)))

mercurial/revlog.py

@@ -1523,7 +1523,7 b' class revlog(object):'
 
         def disambiguate(hexnode, minlength):
             """Disambiguate against wdirid."""
-            for length in range(minlength, 41):
+            for length in range(minlength, len(hexnode) + 1):
                 prefix = hexnode[:length]
                 if not maybewdir(prefix):
                     return prefix
@@ -1540,12 +1540,12 b' class revlog(object):'
                 pass
 
         if node == wdirid:
-            for length in range(minlength, 41):
+            for length in range(minlength, len(hexnode) + 1):
                 prefix = hexnode[:length]
                 if isvalid(prefix):
                     return prefix
 
-        for length in range(minlength, 41):
+        for length in range(minlength, len(hexnode) + 1):
             prefix = hexnode[:length]
             if isvalid(prefix):
                 return disambiguate(hexnode, length)

mercurial/revlogutils/nodemap.py

@@ -13,6 +13,8 b' import os'
 import re
 import struct
 
+from ..i18n import _
+
 from .. import (
     error,
     node as nodemod,
@@ -48,7 +50,7 b' def persisted_data(revlog):'
     docket.data_unused = data_unused
 
     filename = _rawdata_filepath(revlog, docket)
-    use_mmap = revlog.opener.options.get(b"exp-persistent-nodemap.mmap")
+    use_mmap = revlog.opener.options.get(b"persistent-nodemap.mmap")
     try:
         with revlog.opener(filename) as fd:
             if use_mmap:
@@ -105,6 +107,9 b' class _NoTransaction(object):'
     def addabort(self, *args, **kwargs):
         pass
 
+    def _report(self, *args):
+        pass
+
 
 def update_persistent_nodemap(revlog):
     """update the persistent nodemap right now
@@ -137,7 +142,14 b' def _persist_nodemap(tr, revlog, pending'
     can_incremental = util.safehasattr(revlog.index, "nodemap_data_incremental")
     ondisk_docket = revlog._nodemap_docket
     feed_data = util.safehasattr(revlog.index, "update_nodemap_data")
-    use_mmap = revlog.opener.options.get(b"exp-persistent-nodemap.mmap")
+    use_mmap = revlog.opener.options.get(b"persistent-nodemap.mmap")
+    mode = revlog.opener.options.get(b"persistent-nodemap.mode")
+    if not can_incremental:
+        msg = _(b"persistent nodemap in strict mode without efficient method")
+        if mode == b'warn':
+            tr._report(b"%s\n" % msg)
+        elif mode == b'strict':
+            raise error.Abort(msg)
 
     data = None
     # first attemp an incremental update of the data
@@ -255,8 +267,7 b' def _persist_nodemap(tr, revlog, pending'
 # data. Its content is currently very light, but it will expand as the on disk
 # nodemap gains the necessary features to be used in production.
 
-# version 0 is experimental, no BC garantee, do no use outside of tests.
-ONDISK_VERSION = 0
+ONDISK_VERSION = 1
 S_VERSION = struct.Struct(">B")
 S_HEADER = struct.Struct(">BQQQQ")
 

mercurial/revset.py

@@ -789,9 +789,9 b' def conflictlocal(repo, subset, x):'
     "merge" here includes merge conflicts from e.g. 'hg rebase' or 'hg graft'.
     """
     getargs(x, 0, 0, _(b"conflictlocal takes no arguments"))
-    from . import merge
-
-    mergestate = merge.mergestate.read(repo)
+    from . import mergestate as mergestatemod
+
+    mergestate = mergestatemod.mergestate.read(repo)
     if mergestate.active() and repo.changelog.hasnode(mergestate.local):
         return subset & {repo.changelog.rev(mergestate.local)}
 
@@ -805,9 +805,9 b' def conflictother(repo, subset, x):'
     "merge" here includes merge conflicts from e.g. 'hg rebase' or 'hg graft'.
     """
     getargs(x, 0, 0, _(b"conflictother takes no arguments"))
-    from . import merge
-
-    mergestate = merge.mergestate.read(repo)
+    from . import mergestate as mergestatemod
+
+    mergestate = mergestatemod.mergestate.read(repo)
     if mergestate.active() and repo.changelog.hasnode(mergestate.other):
         return subset & {repo.changelog.rev(mergestate.other)}
 

mercurial/rewriteutil.py

@@ -53,3 +53,20 b' def disallowednewunstable(repo, revs):'
     if allowunstable:
         return revset.baseset()
     return repo.revs(b"(%ld::) - %ld", revs, revs)
+
+
+def skip_empty_successor(ui, command):
+    empty_successor = ui.config(b'rewrite', b'empty-successor')
+    if empty_successor == b'skip':
+        return True
+    elif empty_successor == b'keep':
+        return False
+    else:
+        raise error.ConfigError(
+            _(
+                b"%s doesn't know how to handle config "
+                b"rewrite.empty-successor=%s (only 'skip' and 'keep' are "
+                b"supported)"
+            )
+            % (command, empty_successor)
+        )

mercurial/scmutil.py

@@ -456,9 +456,7 b' def formatrevnode(ui, rev, node):'
 
 
 def resolvehexnodeidprefix(repo, prefix):
-    if prefix.startswith(b'x') and repo.ui.configbool(
-        b'experimental', b'revisions.prefixhexnode'
-    ):
+    if prefix.startswith(b'x'):
         prefix = prefix[1:]
     try:
         # Uses unfiltered repo because it's faster when prefix is ambiguous/
@@ -805,9 +803,12 b' def getuipathfn(repo, legacyrelativevalu'
 
     if relative:
         cwd = repo.getcwd()
-        pathto = repo.pathto
-        return lambda f: pathto(f, cwd)
-    elif repo.ui.configbool(b'ui', b'slash'):
+        if cwd != b'':
+            # this branch would work even if cwd == b'' (ie cwd = repo
+            # root), but its generality makes the returned function slower
+            pathto = repo.pathto
+            return lambda f: pathto(f, cwd)
+    if repo.ui.configbool(b'ui', b'slash'):
         return lambda f: f
     else:
         return util.localpath
@@ -1469,6 +1470,13 b' def movedirstate(repo, newctx, match=Non'
     repo._quick_access_changeid_invalidate()
 
 
+def writereporequirements(repo, requirements=None):
+    """ writes requirements for the repo to .hg/requires """
+    if requirements:
+        repo.requirements = requirements
+    writerequires(repo.vfs, repo.requirements)
+
+
 def writerequires(opener, requirements):
     with opener(b'requires', b'w', atomictemp=True) as fp:
         for r in sorted(requirements):
@@ -1879,18 +1887,29 b' class simplekeyvaluefile(object):'
 ]
 
 
-def prefetchfiles(repo, revs, match):
+def prefetchfiles(repo, revmatches):
     """Invokes the registered file prefetch functions, allowing extensions to
     ensure the corresponding files are available locally, before the command
-    uses them."""
-    if match:
-        # The command itself will complain about files that don't exist, so
-        # don't duplicate the message.
-        match = matchmod.badmatch(match, lambda fn, msg: None)
-    else:
-        match = matchall(repo)
+    uses them.
+
+    Args:
+      revmatches: a list of (revision, match) tuples to indicate the files to
+      fetch at each revision. If any of the match elements is None, it matches
+      all files.
+    """
 
-    fileprefetchhooks(repo, revs, match)
+    def _matcher(m):
+        if m:
+            assert isinstance(m, matchmod.basematcher)
+            # The command itself will complain about files that don't exist, so
+            # don't duplicate the message.
+            return matchmod.badmatch(m, lambda fn, msg: None)
+        else:
+            return matchall(repo)
+
+    revbadmatches = [(rev, _matcher(match)) for (rev, match) in revmatches]
+
+    fileprefetchhooks(repo, revbadmatches)
 
 
 # a list of (repo, revs, match) prefetch functions

mercurial/shelve.py

@@ -42,6 +42,7 b' from . import ('
     lock as lockmod,
     mdiff,
     merge,
+    mergestate as mergestatemod,
     node as nodemod,
     patch,
     phases,
@@ -161,7 +162,7 b' class shelvedfile(object):'
         repo = self.repo.unfiltered()
 
         outgoing = discovery.outgoing(
-            repo, missingroots=bases, missingheads=[node]
+            repo, missingroots=bases, ancestorsof=[node]
         )
         cg = changegroup.makechangegroup(repo, outgoing, cgversion, b'shelve')
 
@@ -801,7 +802,7 b' def unshelvecontinue(ui, repo, state, op'
     basename = state.name
     with repo.lock():
         checkparents(repo, state)
-        ms = merge.mergestate.read(repo)
+        ms = mergestatemod.mergestate.read(repo)
         if list(ms.unresolved()):
             raise error.Abort(
                 _(b"unresolved conflicts, can't continue"),
@@ -1013,12 +1014,7 b' def _rebaserestoredcommit('
                 activebookmark,
                 interactive,
             )
-            raise error.InterventionRequired(
-                _(
-                    b"unresolved conflicts (see 'hg resolve', then "
-                    b"'hg unshelve --continue')"
-                )
-            )
+            raise error.ConflictResolutionRequired(b'unshelve')
 
         with repo.dirstate.parentchange():
             repo.setparents(tmpwctx.node(), nodemod.nullid)

mercurial/simplemerge.py

@@ -451,12 +451,7 b' def _picklabels(defaults, overrides):'
     return result
 
 
-def _bytes_to_set(b):
-    """turns a multiple bytes (usually flags) into a set of individual byte"""
-    return set(b[x : x + 1] for x in range(len(b)))
-
-
-def is_null(ctx):
+def is_not_null(ctx):
     if not util.safehasattr(ctx, "node"):
         return False
     return ctx.node() != nodemod.nullid
@@ -518,15 +513,13 b' def simplemerge(ui, localctx, basectx, o'
 
     # merge flags if necessary
     flags = localctx.flags()
-    localflags = _bytes_to_set(flags)
-    otherflags = _bytes_to_set(otherctx.flags())
-    if is_null(basectx) and localflags != otherflags:
-        baseflags = _bytes_to_set(basectx.flags())
-        flags = localflags & otherflags
-        for f in localflags.symmetric_difference(otherflags):
-            if f not in baseflags:
-                flags.add(f)
-        flags = b''.join(sorted(flags))
+    localflags = set(pycompat.iterbytestr(flags))
+    otherflags = set(pycompat.iterbytestr(otherctx.flags()))
+    if is_not_null(basectx) and localflags != otherflags:
+        baseflags = set(pycompat.iterbytestr(basectx.flags()))
+        commonflags = localflags & otherflags
+        addedflags = (localflags ^ otherflags) - baseflags
+        flags = b''.join(sorted(commonflags | addedflags))
 
     if not opts.get(b'print'):
         localctx.write(mergedtext, flags)

mercurial/sparse.py

@@ -18,6 +18,7 b' from . import ('
     error,
     match as matchmod,
     merge as mergemod,
+    mergestate as mergestatemod,
     pathutil,
     pycompat,
     scmutil,
@@ -406,7 +407,7 b' def filterupdatesactions(repo, wctx, mct'
         elif file in wctx:
             prunedactions[file] = (b'r', args, msg)
 
-        if branchmerge and type == mergemod.ACTION_MERGE:
+        if branchmerge and type == mergestatemod.ACTION_MERGE:
             f1, f2, fa, move, anc = args
             if not sparsematch(f1):
                 temporaryfiles.append(f1)
@@ -600,10 +601,10 b' def _updateconfigandrefreshwdir('
 
     if b'exp-sparse' in oldrequires and removing:
         repo.requirements.discard(b'exp-sparse')
-        scmutil.writerequires(repo.vfs, repo.requirements)
+        scmutil.writereporequirements(repo)
     elif b'exp-sparse' not in oldrequires:
         repo.requirements.add(b'exp-sparse')
-        scmutil.writerequires(repo.vfs, repo.requirements)
+        scmutil.writereporequirements(repo)
 
     try:
         writeconfig(repo, includes, excludes, profiles)
@@ -612,7 +613,7 b' def _updateconfigandrefreshwdir('
         if repo.requirements != oldrequires:
             repo.requirements.clear()
             repo.requirements |= oldrequires
-            scmutil.writerequires(repo.vfs, repo.requirements)
+            scmutil.writereporequirements(repo)
         writeconfig(repo, oldincludes, oldexcludes, oldprofiles)
         raise
 

mercurial/sshpeer.py

@@ -36,15 +36,16 b' def _serverquote(s):'
     return b"'%s'" % s.replace(b"'", b"'\\''")
 
 
-def _forwardoutput(ui, pipe):
+def _forwardoutput(ui, pipe, warn=False):
     """display all data currently available on pipe as remote output.
 
     This is non blocking."""
     if pipe:
         s = procutil.readpipe(pipe)
         if s:
+            display = ui.warn if warn else ui.status
             for l in s.splitlines():
-                ui.status(_(b"remote: "), l, b'\n')
+                display(_(b"remote: "), l, b'\n')
 
 
 class doublepipe(object):
@@ -178,7 +179,6 b' def _makeconnection(ui, sshcmd, args, re'
     )
 
     ui.debug(b'running %s\n' % cmd)
-    cmd = procutil.quotecommand(cmd)
 
     # no buffer allow the use of 'select'
     # feel free to remove buffering and select usage when we ultimately
@@ -204,8 +204,12 b' def _clientcapabilities():'
 
 def _performhandshake(ui, stdin, stdout, stderr):
     def badresponse():
-        # Flush any output on stderr.
-        _forwardoutput(ui, stderr)
+        # Flush any output on stderr. In general, the stderr contains errors
+        # from the remote (ssh errors, some hg errors), and status indications
+        # (like "adding changes"), with no current way to tell them apart.
+        # Here we failed so early that it's almost certainly only errors, so
+        # use warn=True so -q doesn't hide them.
+        _forwardoutput(ui, stderr, warn=True)
 
         msg = _(b'no suitable response from remote hg')
         hint = ui.config(b'ui', b'ssherrorhint')
@@ -307,7 +311,7 b' def _performhandshake(ui, stdin, stdout,'
     while lines[-1] and max_noise:
         try:
             l = stdout.readline()
-            _forwardoutput(ui, stderr)
+            _forwardoutput(ui, stderr, warn=True)
 
             # Look for reply to protocol upgrade request. It has a token
             # in it, so there should be no false positives.
@@ -374,7 +378,7 b' def _performhandshake(ui, stdin, stdout,'
         badresponse()
 
     # Flush any output on stderr before proceeding.
-    _forwardoutput(ui, stderr)
+    _forwardoutput(ui, stderr, warn=True)
 
     return protoname, caps
 

mercurial/sslutil.py

@@ -33,9 +33,8 b' from .utils import ('
 # support for TLS 1.1, TLS 1.2, SNI, system CA stores, etc. These features are
 # all exposed via the "ssl" module.
 #
-# Depending on the version of Python being used, SSL/TLS support is either
-# modern/secure or legacy/insecure. Many operations in this module have
-# separate code paths depending on support in Python.
+# We require in setup.py the presence of ssl.SSLContext, which indicates modern
+# SSL/TLS support.
 
 configprotocols = {
     b'tls1.0',
@@ -45,76 +44,19 b' configprotocols = {'
 
 hassni = getattr(ssl, 'HAS_SNI', False)
 
-# TLS 1.1 and 1.2 may not be supported if the OpenSSL Python is compiled
-# against doesn't support them.
-supportedprotocols = {b'tls1.0'}
-if util.safehasattr(ssl, b'PROTOCOL_TLSv1_1'):
+# ssl.HAS_TLSv1* are preferred to check support but they were added in Python
+# 3.7. Prior to CPython commit 6e8cda91d92da72800d891b2fc2073ecbc134d98
+# (backported to the 3.7 branch), ssl.PROTOCOL_TLSv1_1 / ssl.PROTOCOL_TLSv1_2
+# were defined only if compiled against a OpenSSL version with TLS 1.1 / 1.2
+# support. At the mentioned commit, they were unconditionally defined.
+supportedprotocols = set()
+if getattr(ssl, 'HAS_TLSv1', util.safehasattr(ssl, 'PROTOCOL_TLSv1')):
+    supportedprotocols.add(b'tls1.0')
+if getattr(ssl, 'HAS_TLSv1_1', util.safehasattr(ssl, 'PROTOCOL_TLSv1_1')):
     supportedprotocols.add(b'tls1.1')
-if util.safehasattr(ssl, b'PROTOCOL_TLSv1_2'):
+if getattr(ssl, 'HAS_TLSv1_2', util.safehasattr(ssl, 'PROTOCOL_TLSv1_2')):
     supportedprotocols.add(b'tls1.2')
 
-try:
-    # ssl.SSLContext was added in 2.7.9 and presence indicates modern
-    # SSL/TLS features are available.
-    SSLContext = ssl.SSLContext
-    modernssl = True
-    _canloaddefaultcerts = util.safehasattr(SSLContext, b'load_default_certs')
-except AttributeError:
-    modernssl = False
-    _canloaddefaultcerts = False
-
-    # We implement SSLContext using the interface from the standard library.
-    class SSLContext(object):
-        def __init__(self, protocol):
-            # From the public interface of SSLContext
-            self.protocol = protocol
-            self.check_hostname = False
-            self.options = 0
-            self.verify_mode = ssl.CERT_NONE
-
-            # Used by our implementation.
-            self._certfile = None
-            self._keyfile = None
-            self._certpassword = None
-            self._cacerts = None
-            self._ciphers = None
-
-        def load_cert_chain(self, certfile, keyfile=None, password=None):
-            self._certfile = certfile
-            self._keyfile = keyfile
-            self._certpassword = password
-
-        def load_default_certs(self, purpose=None):
-            pass
-
-        def load_verify_locations(self, cafile=None, capath=None, cadata=None):
-            if capath:
-                raise error.Abort(_(b'capath not supported'))
-            if cadata:
-                raise error.Abort(_(b'cadata not supported'))
-
-            self._cacerts = cafile
-
-        def set_ciphers(self, ciphers):
-            self._ciphers = ciphers
-
-        def wrap_socket(self, socket, server_hostname=None, server_side=False):
-            # server_hostname is unique to SSLContext.wrap_socket and is used
-            # for SNI in that context. So there's nothing for us to do with it
-            # in this legacy code since we don't support SNI.
-
-            args = {
-                'keyfile': self._keyfile,
-                'certfile': self._certfile,
-                'server_side': server_side,
-                'cert_reqs': self.verify_mode,
-                'ssl_version': self.protocol,
-                'ca_certs': self._cacerts,
-                'ciphers': self._ciphers,
-            }
-
-            return ssl.wrap_socket(socket, **args)
-
 
 def _hostsettings(ui, hostname):
     """Obtain security settings for a hostname.
@@ -135,15 +77,11 b' def _hostsettings(ui, hostname):'
         b'disablecertverification': False,
         # Whether the legacy [hostfingerprints] section has data for this host.
         b'legacyfingerprint': False,
-        # PROTOCOL_* constant to use for SSLContext.__init__.
-        b'protocol': None,
         # String representation of minimum protocol to be used for UI
         # presentation.
-        b'protocolui': None,
+        b'minimumprotocol': None,
         # ssl.CERT_* constant used by SSLContext.verify_mode.
         b'verifymode': None,
-        # Defines extra ssl.OP* bitwise options to set.
-        b'ctxoptions': None,
         # OpenSSL Cipher List to use (instead of default).
         b'ciphers': None,
     }
@@ -158,45 +96,30 b' def _hostsettings(ui, hostname):'
                 % b' '.join(sorted(configprotocols)),
             )
 
-    # We default to TLS 1.1+ where we can because TLS 1.0 has known
-    # vulnerabilities (like BEAST and POODLE). We allow users to downgrade to
-    # TLS 1.0+ via config options in case a legacy server is encountered.
-    if b'tls1.1' in supportedprotocols:
-        defaultprotocol = b'tls1.1'
-    else:
-        # Let people know they are borderline secure.
-        # We don't document this config option because we want people to see
-        # the bold warnings on the web site.
-        # internal config: hostsecurity.disabletls10warning
-        if not ui.configbool(b'hostsecurity', b'disabletls10warning'):
-            ui.warn(
-                _(
-                    b'warning: connecting to %s using legacy security '
-                    b'technology (TLS 1.0); see '
-                    b'https://mercurial-scm.org/wiki/SecureConnections for '
-                    b'more info\n'
-                )
-                % bhostname
-            )
-        defaultprotocol = b'tls1.0'
+    # We default to TLS 1.1+ because TLS 1.0 has known vulnerabilities (like
+    # BEAST and POODLE). We allow users to downgrade to TLS 1.0+ via config
+    # options in case a legacy server is encountered.
+
+    # setup.py checks that TLS 1.1 or TLS 1.2 is present, so the following
+    # assert should not fail.
+    assert supportedprotocols - {b'tls1.0'}
+    defaultminimumprotocol = b'tls1.1'
 
     key = b'minimumprotocol'
-    protocol = ui.config(b'hostsecurity', key, defaultprotocol)
-    validateprotocol(protocol, key)
+    minimumprotocol = ui.config(b'hostsecurity', key, defaultminimumprotocol)
+    validateprotocol(minimumprotocol, key)
 
     key = b'%s:minimumprotocol' % bhostname
-    protocol = ui.config(b'hostsecurity', key, protocol)
-    validateprotocol(protocol, key)
+    minimumprotocol = ui.config(b'hostsecurity', key, minimumprotocol)
+    validateprotocol(minimumprotocol, key)
 
     # If --insecure is used, we allow the use of TLS 1.0 despite config options.
     # We always print a "connection security to %s is disabled..." message when
     # --insecure is used. So no need to print anything more here.
     if ui.insecureconnections:
-        protocol = b'tls1.0'
+        minimumprotocol = b'tls1.0'
 
-    s[b'protocol'], s[b'ctxoptions'], s[b'protocolui'] = protocolsettings(
-        protocol
-    )
+    s[b'minimumprotocol'] = minimumprotocol
 
     ciphers = ui.config(b'hostsecurity', b'ciphers')
     ciphers = ui.config(b'hostsecurity', b'%s:ciphers' % bhostname, ciphers)
@@ -288,7 +211,7 b' def _hostsettings(ui, hostname):'
 
         # Require certificate validation if CA certs are being loaded and
         # verification hasn't been disabled above.
-        if cafile or (_canloaddefaultcerts and s[b'allowloaddefaultcerts']):
+        if cafile or s[b'allowloaddefaultcerts']:
             s[b'verifymode'] = ssl.CERT_REQUIRED
         else:
             # At this point we don't have a fingerprint, aren't being
@@ -298,59 +221,26 b' def _hostsettings(ui, hostname):'
             # user).
             s[b'verifymode'] = ssl.CERT_NONE
 
-    assert s[b'protocol'] is not None
-    assert s[b'ctxoptions'] is not None
     assert s[b'verifymode'] is not None
 
     return s
 
 
-def protocolsettings(protocol):
-    """Resolve the protocol for a config value.
-
-    Returns a 3-tuple of (protocol, options, ui value) where the first
-    2 items are values used by SSLContext and the last is a string value
-    of the ``minimumprotocol`` config option equivalent.
+def commonssloptions(minimumprotocol):
+    """Return SSLContext options common to servers and clients.
     """
-    if protocol not in configprotocols:
-        raise ValueError(b'protocol value not supported: %s' % protocol)
-
-    # Despite its name, PROTOCOL_SSLv23 selects the highest protocol
-    # that both ends support, including TLS protocols. On legacy stacks,
-    # the highest it likely goes is TLS 1.0. On modern stacks, it can
-    # support TLS 1.2.
-    #
-    # The PROTOCOL_TLSv* constants select a specific TLS version
-    # only (as opposed to multiple versions). So the method for
-    # supporting multiple TLS versions is to use PROTOCOL_SSLv23 and
-    # disable protocols via SSLContext.options and OP_NO_* constants.
-    # However, SSLContext.options doesn't work unless we have the
-    # full/real SSLContext available to us.
-    if supportedprotocols == {b'tls1.0'}:
-        if protocol != b'tls1.0':
-            raise error.Abort(
-                _(b'current Python does not support protocol setting %s')
-                % protocol,
-                hint=_(
-                    b'upgrade Python or disable setting since '
-                    b'only TLS 1.0 is supported'
-                ),
-            )
-
-        return ssl.PROTOCOL_TLSv1, 0, b'tls1.0'
-
-    # WARNING: returned options don't work unless the modern ssl module
-    # is available. Be careful when adding options here.
+    if minimumprotocol not in configprotocols:
+        raise ValueError(b'protocol value not supported: %s' % minimumprotocol)
 
     # SSLv2 and SSLv3 are broken. We ban them outright.
     options = ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
 
-    if protocol == b'tls1.0':
+    if minimumprotocol == b'tls1.0':
         # Defaults above are to use TLS 1.0+
         pass
-    elif protocol == b'tls1.1':
+    elif minimumprotocol == b'tls1.1':
         options |= ssl.OP_NO_TLSv1
-    elif protocol == b'tls1.2':
+    elif minimumprotocol == b'tls1.2':
         options |= ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1
     else:
         raise error.Abort(_(b'this should not happen'))
@@ -359,7 +249,7 b' def protocolsettings(protocol):'
     # There is no guarantee this attribute is defined on the module.
     options |= getattr(ssl, 'OP_NO_COMPRESSION', 0)
 
-    return ssl.PROTOCOL_SSLv23, options, protocol
+    return options
 
 
 def wrapsocket(sock, keyfile, certfile, ui, serverhostname=None):
@@ -414,12 +304,12 b' def wrapsocket(sock, keyfile, certfile, '
     # bundle with a specific CA cert removed. If the system/default CA bundle
     # is loaded and contains that removed CA, you've just undone the user's
     # choice.
-    sslcontext = SSLContext(settings[b'protocol'])
-
-    # This is a no-op unless using modern ssl.
-    sslcontext.options |= settings[b'ctxoptions']
-
-    # This still works on our fake SSLContext.
+    #
+    # Despite its name, PROTOCOL_SSLv23 selects the highest protocol that both
+    # ends support, including TLS protocols. commonssloptions() restricts the
+    # set of allowed protocols.
+    sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
+    sslcontext.options |= commonssloptions(settings[b'minimumprotocol'])
     sslcontext.verify_mode = settings[b'verifymode']
 
     if settings[b'ciphers']:
@@ -468,8 +358,6 b' def wrapsocket(sock, keyfile, certfile, '
         # If we're doing certificate verification and no CA certs are loaded,
         # that is almost certainly the reason why verification failed. Provide
         # a hint to the user.
-        # Only modern ssl module exposes SSLContext.get_ca_certs() so we can
-        # only show this warning if modern ssl is available.
         # The exception handler is here to handle bugs around cert attributes:
         # https://bugs.python.org/issue20916#msg213479.  (See issues5313.)
         # When the main 20916 bug occurs, 'sslcontext.get_ca_certs()' is a
@@ -478,7 +366,6 b' def wrapsocket(sock, keyfile, certfile, '
             if (
                 caloaded
                 and settings[b'verifymode'] == ssl.CERT_REQUIRED
-                and modernssl
                 and not sslcontext.get_ca_certs()
             ):
                 ui.warn(
@@ -502,7 +389,7 b' def wrapsocket(sock, keyfile, certfile, '
             # reason, try to emit an actionable warning.
             if e.reason == 'UNSUPPORTED_PROTOCOL':
                 # We attempted TLS 1.0+.
-                if settings[b'protocolui'] == b'tls1.0':
+                if settings[b'minimumprotocol'] == b'tls1.0':
                     # We support more than just TLS 1.0+. If this happens,
                     # the likely scenario is either the client or the server
                     # is really old. (e.g. server doesn't support TLS 1.0+ or
@@ -547,7 +434,7 b' def wrapsocket(sock, keyfile, certfile, '
                             b'to be more secure than the server can support)\n'
                         )
                         % (
-                            settings[b'protocolui'],
+                            settings[b'minimumprotocol'],
                             pycompat.bytesurl(serverhostname),
                         )
                     )
@@ -618,12 +505,18 b' def wrapserversocket('
                 _(b'referenced certificate file (%s) does not exist') % f
             )
 
-    protocol, options, _protocolui = protocolsettings(b'tls1.0')
+    # Despite its name, PROTOCOL_SSLv23 selects the highest protocol that both
+    # ends support, including TLS protocols. commonssloptions() restricts the
+    # set of allowed protocols.
+    protocol = ssl.PROTOCOL_SSLv23
+    options = commonssloptions(b'tls1.0')
 
     # This config option is intended for use in tests only. It is a giant
     # footgun to kill security. Don't define it.
     exactprotocol = ui.config(b'devel', b'serverexactprotocol')
     if exactprotocol == b'tls1.0':
+        if b'tls1.0' not in supportedprotocols:
+            raise error.Abort(_(b'TLS 1.0 not supported by this Python'))
         protocol = ssl.PROTOCOL_TLSv1
     elif exactprotocol == b'tls1.1':
         if b'tls1.1' not in supportedprotocols:
@@ -638,23 +531,20 b' def wrapserversocket('
             _(b'invalid value for serverexactprotocol: %s') % exactprotocol
         )
 
-    if modernssl:
-        # We /could/ use create_default_context() here since it doesn't load
-        # CAs when configured for client auth. However, it is hard-coded to
-        # use ssl.PROTOCOL_SSLv23 which may not be appropriate here.
-        sslcontext = SSLContext(protocol)
-        sslcontext.options |= options
+    # We /could/ use create_default_context() here since it doesn't load
+    # CAs when configured for client auth. However, it is hard-coded to
+    # use ssl.PROTOCOL_SSLv23 which may not be appropriate here.
+    sslcontext = ssl.SSLContext(protocol)
+    sslcontext.options |= options
 
-        # Improve forward secrecy.
-        sslcontext.options |= getattr(ssl, 'OP_SINGLE_DH_USE', 0)
-        sslcontext.options |= getattr(ssl, 'OP_SINGLE_ECDH_USE', 0)
+    # Improve forward secrecy.
+    sslcontext.options |= getattr(ssl, 'OP_SINGLE_DH_USE', 0)
+    sslcontext.options |= getattr(ssl, 'OP_SINGLE_ECDH_USE', 0)
 
-        # Use the list of more secure ciphers if found in the ssl module.
-        if util.safehasattr(ssl, b'_RESTRICTED_SERVER_CIPHERS'):
-            sslcontext.options |= getattr(ssl, 'OP_CIPHER_SERVER_PREFERENCE', 0)
-            sslcontext.set_ciphers(ssl._RESTRICTED_SERVER_CIPHERS)
-    else:
-        sslcontext = SSLContext(ssl.PROTOCOL_TLSv1)
+    # Use the list of more secure ciphers if found in the ssl module.
+    if util.safehasattr(ssl, b'_RESTRICTED_SERVER_CIPHERS'):
+        sslcontext.options |= getattr(ssl, 'OP_CIPHER_SERVER_PREFERENCE', 0)
+        sslcontext.set_ciphers(ssl._RESTRICTED_SERVER_CIPHERS)
 
     if requireclientcert:
         sslcontext.verify_mode = ssl.CERT_REQUIRED
@@ -797,14 +687,6 b' def _plainapplepython():'
     )
 
 
-_systemcacertpaths = [
-    # RHEL, CentOS, and Fedora
-    b'/etc/pki/tls/certs/ca-bundle.trust.crt',
-    # Debian, Ubuntu, Gentoo
-    b'/etc/ssl/certs/ca-certificates.crt',
-]
-
-
 def _defaultcacerts(ui):
     """return path to default CA certificates or None.
 
@@ -827,23 +709,6 b' def _defaultcacerts(ui):'
     except (ImportError, AttributeError):
         pass
 
-    # On Windows, only the modern ssl module is capable of loading the system
-    # CA certificates. If we're not capable of doing that, emit a warning
-    # because we'll get a certificate verification error later and the lack
-    # of loaded CA certificates will be the reason why.
-    # Assertion: this code is only called if certificates are being verified.
-    if pycompat.iswindows:
-        if not _canloaddefaultcerts:
-            ui.warn(
-                _(
-                    b'(unable to load Windows CA certificates; see '
-                    b'https://mercurial-scm.org/wiki/SecureConnections for '
-                    b'how to configure Mercurial to avoid this message)\n'
-                )
-            )
-
-        return None
-
     # Apple's OpenSSL has patches that allow a specially constructed certificate
     # to load the system CA store. If we're running on Apple Python, use this
     # trick.
@@ -854,58 +719,6 b' def _defaultcacerts(ui):'
         if os.path.exists(dummycert):
             return dummycert
 
-    # The Apple OpenSSL trick isn't available to us. If Python isn't able to
-    # load system certs, we're out of luck.
-    if pycompat.isdarwin:
-        # FUTURE Consider looking for Homebrew or MacPorts installed certs
-        # files. Also consider exporting the keychain certs to a file during
-        # Mercurial install.
-        if not _canloaddefaultcerts:
-            ui.warn(
-                _(
-                    b'(unable to load CA certificates; see '
-                    b'https://mercurial-scm.org/wiki/SecureConnections for '
-                    b'how to configure Mercurial to avoid this message)\n'
-                )
-            )
-        return None
-
-    # / is writable on Windows. Out of an abundance of caution make sure
-    # we're not on Windows because paths from _systemcacerts could be installed
-    # by non-admin users.
-    assert not pycompat.iswindows
-
-    # Try to find CA certificates in well-known locations. We print a warning
-    # when using a found file because we don't want too much silent magic
-    # for security settings. The expectation is that proper Mercurial
-    # installs will have the CA certs path defined at install time and the
-    # installer/packager will make an appropriate decision on the user's
-    # behalf. We only get here and perform this setting as a feature of
-    # last resort.
-    if not _canloaddefaultcerts:
-        for path in _systemcacertpaths:
-            if os.path.isfile(path):
-                ui.warn(
-                    _(
-                        b'(using CA certificates from %s; if you see this '
-                        b'message, your Mercurial install is not properly '
-                        b'configured; see '
-                        b'https://mercurial-scm.org/wiki/SecureConnections '
-                        b'for how to configure Mercurial to avoid this '
-                        b'message)\n'
-                    )
-                    % path
-                )
-                return path
-
-        ui.warn(
-            _(
-                b'(unable to load CA certificates; see '
-                b'https://mercurial-scm.org/wiki/SecureConnections for '
-                b'how to configure Mercurial to avoid this message)\n'
-            )
-        )
-
     return None
 
 

mercurial/state.py

@@ -19,6 +19,8 b' the data.'
 
 from __future__ import absolute_import
 
+import contextlib
+
 from .i18n import _
 
 from . import (
@@ -119,6 +121,7 b' class _statecheck(object):'
         reportonly,
         continueflag,
         stopflag,
+        childopnames,
         cmdmsg,
         cmdhint,
         statushint,
@@ -132,6 +135,8 b' class _statecheck(object):'
         self._reportonly = reportonly
         self._continueflag = continueflag
         self._stopflag = stopflag
+        self._childopnames = childopnames
+        self._delegating = False
         self._cmdmsg = cmdmsg
         self._cmdhint = cmdhint
         self._statushint = statushint
@@ -181,12 +186,15 b' class _statecheck(object):'
         """
         if self._opname == b'merge':
             return len(repo[None].parents()) > 1
+        elif self._delegating:
+            return False
         else:
             return repo.vfs.exists(self._fname)
 
 
 # A list of statecheck objects for multistep operations like graft.
 _unfinishedstates = []
+_unfinishedstatesbyname = {}
 
 
 def addunfinished(
@@ -197,6 +205,7 b' def addunfinished('
     reportonly=False,
     continueflag=False,
     stopflag=False,
+    childopnames=None,
     cmdmsg=b"",
     cmdhint=b"",
     statushint=b"",
@@ -218,6 +227,8 b' def addunfinished('
     `--continue` option or not.
     stopflag is a boolean that determines whether or not a command supports
     --stop flag
+    childopnames is a list of other opnames this op uses as sub-steps of its
+    own execution. They must already be added.
     cmdmsg is used to pass a different status message in case standard
     message of the format "abort: cmdname in progress" is not desired.
     cmdhint is used to pass a different hint message in case standard
@@ -230,6 +241,7 b' def addunfinished('
     continuefunc stores the function required to finish an interrupted
     operation.
     """
+    childopnames = childopnames or []
     statecheckobj = _statecheck(
         opname,
         fname,
@@ -238,17 +250,98 b' def addunfinished('
         reportonly,
         continueflag,
         stopflag,
+        childopnames,
         cmdmsg,
         cmdhint,
         statushint,
         abortfunc,
         continuefunc,
     )
+
     if opname == b'merge':
         _unfinishedstates.append(statecheckobj)
     else:
+        # This check enforces that for any op 'foo' which depends on op 'bar',
+        # 'foo' comes before 'bar' in _unfinishedstates. This ensures that
+        # getrepostate() always returns the most specific applicable answer.
+        for childopname in childopnames:
+            if childopname not in _unfinishedstatesbyname:
+                raise error.ProgrammingError(
+                    _(b'op %s depends on unknown op %s') % (opname, childopname)
+                )
+
         _unfinishedstates.insert(0, statecheckobj)
 
+    if opname in _unfinishedstatesbyname:
+        raise error.ProgrammingError(_(b'op %s registered twice') % opname)
+    _unfinishedstatesbyname[opname] = statecheckobj
+
+
+def _getparentandchild(opname, childopname):
+    p = _unfinishedstatesbyname.get(opname, None)
+    if not p:
+        raise error.ProgrammingError(_(b'unknown op %s') % opname)
+    if childopname not in p._childopnames:
+        raise error.ProgrammingError(
+            _(b'op %s does not delegate to %s') % (opname, childopname)
+        )
+    c = _unfinishedstatesbyname[childopname]
+    return p, c
+
+
[e-mail]
+def delegating(repo, opname, childopname):
+    """context wrapper for delegations from opname to childopname.
+
+    requires that childopname was specified when opname was registered.
+
+    Usage:
+      def my_command_foo_that_uses_rebase(...):
+        ...
+        with state.delegating(repo, 'foo', 'rebase'):
+          _run_rebase(...)
+        ...
+    """
+
+    p, c = _getparentandchild(opname, childopname)
+    if p._delegating:
+        raise error.ProgrammingError(
+            _(b'cannot delegate from op %s recursively') % opname
+        )
+    p._delegating = True
+    try:
+        yield
+    except error.ConflictResolutionRequired as e:
+        # Rewrite conflict resolution advice for the parent opname.
+        if e.opname == childopname:
+            raise error.ConflictResolutionRequired(opname)
+        raise e
+    finally:
+        p._delegating = False
+
+
+def ischildunfinished(repo, opname, childopname):
+    """Returns true if both opname and childopname are unfinished."""
+
+    p, c = _getparentandchild(opname, childopname)
+    return (p._delegating or p.isunfinished(repo)) and c.isunfinished(repo)
+
+
+def continuechild(ui, repo, opname, childopname):
+    """Checks that childopname is in progress, and continues it."""
+
+    p, c = _getparentandchild(opname, childopname)
+    if not ischildunfinished(repo, opname, childopname):
+        raise error.ProgrammingError(
+            _(b'child op %s of parent %s is not unfinished')
+            % (childopname, opname)
+        )
+    if not c.continuefunc:
+        raise error.ProgrammingError(
+            _(b'op %s has no continue function') % childopname
+        )
+    return c.continuefunc(ui, repo)
+
 
 addunfinished(
     b'update',

mercurial/streamclone.py

@@ -20,6 +20,7 b' from . import ('
     narrowspec,
     phases,
     pycompat,
+    scmutil,
     store,
     util,
 )
@@ -187,7 +188,7 b' def maybeperformlegacystreamclone(pullop'
         repo.svfs.options = localrepo.resolvestorevfsoptions(
             repo.ui, repo.requirements, repo.features
         )
-        repo._writerequirements()
+        scmutil.writereporequirements(repo)
 
         if rbranchmap:
             repo._branchcaches.replace(repo, rbranchmap)
@@ -730,4 +731,4 b' def applybundlev2(repo, fp, filecount, f'
     repo.svfs.options = localrepo.resolvestorevfsoptions(
         repo.ui, repo.requirements, repo.features
     )
-    repo._writerequirements()
+    scmutil.writereporequirements(repo)

mercurial/subrepo.py

@@ -617,8 +617,8 b' class hgsubrepo(abstractsubrepo):'
                 ui,
                 self._repo,
                 diffopts,
-                node1,
-                node2,
+                self._repo[node1],
+                self._repo[node2],
                 match,
                 prefix=prefix,
                 listsubrepos=True,
@@ -639,7 +639,7 b' class hgsubrepo(abstractsubrepo):'
         rev = self._state[1]
         ctx = self._repo[rev]
         scmutil.prefetchfiles(
-            self._repo, [ctx.rev()], scmutil.matchfiles(self._repo, files)
+            self._repo, [(ctx.rev(), scmutil.matchfiles(self._repo, files))]
         )
         total = abstractsubrepo.archive(self, archiver, prefix, match)
         for subpath in ctx.substate:

mercurial/templatekw.py

@@ -419,9 +419,9 b' def getgraphnodecurrent(repo, ctx, cache'
     else:
         merge_nodes = cache.get(b'merge_nodes')
         if merge_nodes is None:
-            from . import merge
+            from . import mergestate as mergestatemod
 
-            mergestate = merge.mergestate.read(repo)
+            mergestate = mergestatemod.mergestate.read(repo)
             if mergestate.active():
                 merge_nodes = (mergestate.local, mergestate.other)
             else:

mercurial/ui.py

@@ -9,6 +9,7 b' from __future__ import absolute_import'
 
 import collections
 import contextlib
+import datetime
 import errno
 import getpass
 import inspect
@@ -242,6 +243,7 b' class ui(object):'
         self._terminfoparams = {}
         self._styles = {}
         self._uninterruptible = False
+        self.showtimestamp = False
 
         if src:
             self._fout = src._fout
@@ -561,6 +563,7 b' class ui(object):'
             self._reportuntrusted = self.debugflag or self.configbool(
                 b"ui", b"report_untrusted"
             )
+            self.showtimestamp = self.configbool(b'ui', b'timestamp-output')
             self.tracebackflag = self.configbool(b'ui', b'traceback')
             self.logblockedtimes = self.configbool(b'ui', b'logblockedtimes')
 
@@ -1200,7 +1203,7 b' class ui(object):'
                 dest.write(msg)
             # stderr may be buffered under win32 when redirected to files,
             # including stdout.
-            if dest is self._ferr and not getattr(self._ferr, 'closed', False):
+            if dest is self._ferr and not getattr(dest, 'closed', False):
                 dest.flush()
         except IOError as err:
             if dest is self._ferr and err.errno in (
@@ -1217,7 +1220,21 b' class ui(object):'
             ) * 1000
 
     def _writemsg(self, dest, *args, **opts):
+        timestamp = self.showtimestamp and opts.get('type') in {
+            b'debug',
+            b'error',
+            b'note',
+            b'status',
+            b'warning',
+        }
+        if timestamp:
+            args = (
+                b'[%s] '
+                % pycompat.bytestr(datetime.datetime.now().isoformat()),
+            ) + args
         _writemsgwith(self._write, dest, *args, **opts)
+        if timestamp:
+            dest.flush()
 
     def _writemsgnobuf(self, dest, *args, **opts):
         _writemsgwith(self._writenobuf, dest, *args, **opts)
@@ -2102,6 +2119,22 b' class ui(object):'
             if (b'ui', b'quiet') in overrides:
                 self.fixconfig(section=b'ui')
 
+    def estimatememory(self):
+        """Provide an estimate for the available system memory in Bytes.
+
+        This can be overriden via ui.available-memory. It returns None, if
+        no estimate can be computed.
+        """
+        value = self.config(b'ui', b'available-memory')
+        if value is not None:
+            try:
+                return util.sizetoint(value)
+            except error.ParseError:
+                raise error.ConfigError(
+                    _(b"ui.available-memory value is invalid ('%s')") % value
+                )
+        return util._estimatememory()
+
 
 class paths(dict):
     """Represents a collection of paths and their configs.

mercurial/upgrade.py

@@ -13,12 +13,12 b' from .i18n import _'
 from .pycompat import getattr
 from . import (
     changelog,
-    copies,
     error,
     filelog,
     hg,
     localrepo,
     manifest,
+    metadata,
     pycompat,
     revlog,
     scmutil,
@@ -78,6 +78,7 b' def supportremovedrequirements(repo):'
         localrepo.SPARSEREVLOG_REQUIREMENT,
         localrepo.SIDEDATA_REQUIREMENT,
         localrepo.COPIESSDC_REQUIREMENT,
+        localrepo.NODEMAP_REQUIREMENT,
     }
     for name in compression.compengines:
         engine = compression.compengines[name]
@@ -105,6 +106,7 b' def supporteddestrequirements(repo):'
         localrepo.SPARSEREVLOG_REQUIREMENT,
         localrepo.SIDEDATA_REQUIREMENT,
         localrepo.COPIESSDC_REQUIREMENT,
+        localrepo.NODEMAP_REQUIREMENT,
     }
     for name in compression.compengines:
         engine = compression.compengines[name]
@@ -132,6 +134,7 b' def allowednewrequirements(repo):'
         localrepo.SPARSEREVLOG_REQUIREMENT,
         localrepo.SIDEDATA_REQUIREMENT,
         localrepo.COPIESSDC_REQUIREMENT,
+        localrepo.NODEMAP_REQUIREMENT,
     }
     for name in compression.compengines:
         engine = compression.compengines[name]
@@ -374,6 +377,21 b' class sidedata(requirementformatvariant)'
 
 
 @registerformatvariant
+class persistentnodemap(requirementformatvariant):
+    name = b'persistent-nodemap'
+
+    _requirement = localrepo.NODEMAP_REQUIREMENT
+
+    default = False
+
+    description = _(
+        b'persist the node -> rev mapping on disk to speedup lookup'
+    )
+
+    upgrademessage = _(b'Speedup revision lookup by node id.')
+
+
+@registerformatvariant
 class copiessdc(requirementformatvariant):
     name = b'copies-sdc'
 
@@ -716,9 +734,9 b' def getsidedatacompanion(srcrepo, dstrep'
             return False, (), {}
 
     elif localrepo.COPIESSDC_REQUIREMENT in addedreqs:
-        sidedatacompanion = copies.getsidedataadder(srcrepo, dstrepo)
+        sidedatacompanion = metadata.getsidedataadder(srcrepo, dstrepo)
     elif localrepo.COPIESSDC_REQUIREMENT in removedreqs:
-        sidedatacompanion = copies.getsidedataremover(srcrepo, dstrepo)
+        sidedatacompanion = metadata.getsidedataremover(srcrepo, dstrepo)
     return sidedatacompanion
 
 
@@ -807,14 +825,14 b' def _clonerevlogs('
     if not revcount:
         return
 
-    ui.write(
+    ui.status(
         _(
             b'migrating %d total revisions (%d in filelogs, %d in manifests, '
             b'%d in changelog)\n'
         )
         % (revcount, frevcount, mrevcount, crevcount)
     )
-    ui.write(
+    ui.status(
         _(b'migrating %s in store; %s tracked data\n')
         % ((util.bytecount(srcsize), util.bytecount(srcrawsize)))
     )
@@ -837,7 +855,7 b' def _clonerevlogs('
         oldrl = _revlogfrompath(srcrepo, unencoded)
 
         if isinstance(oldrl, changelog.changelog) and b'c' not in seen:
-            ui.write(
+            ui.status(
                 _(
                     b'finished migrating %d manifest revisions across %d '
                     b'manifests; change in size: %s\n'
@@ -845,7 +863,7 b' def _clonerevlogs('
                 % (mrevcount, mcount, util.bytecount(mdstsize - msrcsize))
             )
 
-            ui.write(
+            ui.status(
                 _(
                     b'migrating changelog containing %d revisions '
                     b'(%s in store; %s tracked data)\n'
@@ -861,7 +879,7 b' def _clonerevlogs('
                 _(b'changelog revisions'), total=crevcount
             )
         elif isinstance(oldrl, manifest.manifestrevlog) and b'm' not in seen:
-            ui.write(
+            ui.status(
                 _(
                     b'finished migrating %d filelog revisions across %d '
                     b'filelogs; change in size: %s\n'
@@ -869,7 +887,7 b' def _clonerevlogs('
                 % (frevcount, fcount, util.bytecount(fdstsize - fsrcsize))
             )
 
-            ui.write(
+            ui.status(
                 _(
                     b'migrating %d manifests containing %d revisions '
                     b'(%s in store; %s tracked data)\n'
@@ -888,7 +906,7 b' def _clonerevlogs('
                 _(b'manifest revisions'), total=mrevcount
             )
         elif b'f' not in seen:
-            ui.write(
+            ui.status(
                 _(
                     b'migrating %d filelogs containing %d revisions '
                     b'(%s in store; %s tracked data)\n'
@@ -941,7 +959,7 b' def _clonerevlogs('
 
     progress.complete()
 
-    ui.write(
+    ui.status(
         _(
             b'finished migrating %d changelog revisions; change in size: '
             b'%s\n'
@@ -949,7 +967,7 b' def _clonerevlogs('
         % (crevcount, util.bytecount(cdstsize - csrcsize))
     )
 
-    ui.write(
+    ui.status(
         _(
             b'finished migrating %d total revisions; total change in store '
             b'size: %s\n'
@@ -975,7 +993,7 b' def _filterstorefile(srcrepo, dstrepo, r'
     Function should return ``True`` if the file is to be copied.
     """
     # Skip revlogs.
-    if path.endswith((b'.i', b'.d')):
+    if path.endswith((b'.i', b'.d', b'.n', b'.nd')):
         return False
     # Skip transaction related files.
     if path.startswith(b'undo'):
@@ -1013,7 +1031,7 b' def _upgraderepo('
     assert srcrepo.currentwlock()
     assert dstrepo.currentwlock()
 
-    ui.write(
+    ui.status(
         _(
             b'(it is safe to interrupt this process any time before '
             b'data migration completes)\n'
@@ -1048,14 +1066,14 b' def _upgraderepo('
         if not _filterstorefile(srcrepo, dstrepo, requirements, p, kind, st):
             continue
 
-        srcrepo.ui.write(_(b'copying %s\n') % p)
+        srcrepo.ui.status(_(b'copying %s\n') % p)
         src = srcrepo.store.rawvfs.join(p)
         dst = dstrepo.store.rawvfs.join(p)
         util.copyfile(src, dst, copystat=True)
 
     _finishdatamigration(ui, srcrepo, dstrepo, requirements)
 
-    ui.write(_(b'data fully migrated to temporary repository\n'))
+    ui.status(_(b'data fully migrated to temporary repository\n'))
 
     backuppath = pycompat.mkdtemp(prefix=b'upgradebackup.', dir=srcrepo.path)
     backupvfs = vfsmod.vfs(backuppath)
@@ -1067,28 +1085,28 b' def _upgraderepo('
     # as a mechanism to lock out new clients during the data swap. This is
     # better than allowing a client to continue while the repository is in
     # an inconsistent state.
-    ui.write(
+    ui.status(
         _(
             b'marking source repository as being upgraded; clients will be '
             b'unable to read from repository\n'
         )
     )
-    scmutil.writerequires(
-        srcrepo.vfs, srcrepo.requirements | {b'upgradeinprogress'}
+    scmutil.writereporequirements(
+        srcrepo, srcrepo.requirements | {b'upgradeinprogress'}
     )
 
-    ui.write(_(b'starting in-place swap of repository data\n'))
-    ui.write(_(b'replaced files will be backed up at %s\n') % backuppath)
+    ui.status(_(b'starting in-place swap of repository data\n'))
+    ui.status(_(b'replaced files will be backed up at %s\n') % backuppath)
 
     # Now swap in the new store directory. Doing it as a rename should make
     # the operation nearly instantaneous and atomic (at least in well-behaved
     # environments).
-    ui.write(_(b'replacing store...\n'))
+    ui.status(_(b'replacing store...\n'))
     tstart = util.timer()
     util.rename(srcrepo.spath, backupvfs.join(b'store'))
     util.rename(dstrepo.spath, srcrepo.spath)
     elapsed = util.timer() - tstart
-    ui.write(
+    ui.status(
         _(
             b'store replacement complete; repository was inconsistent for '
             b'%0.1fs\n'
@@ -1098,13 +1116,13 b' def _upgraderepo('
 
     # We first write the requirements file. Any new requirements will lock
     # out legacy clients.
-    ui.write(
+    ui.status(
         _(
             b'finalizing requirements file and making repository readable '
             b'again\n'
         )
     )
-    scmutil.writerequires(srcrepo.vfs, requirements)
+    scmutil.writereporequirements(srcrepo, requirements)
 
     # The lock file from the old store won't be removed because nothing has a
     # reference to its new location. So clean it up manually. Alternatively, we
@@ -1274,9 +1292,20 b' def upgraderepo('
             ui.write((b'\n'))
         ui.write(b'\n')
 
+    def printoptimisations():
+        optimisations = [a for a in actions if a.type == optimisation]
+        optimisations.sort(key=lambda a: a.name)
+        if optimisations:
+            ui.write(_(b'optimisations: '))
+            write_labeled(
+                [a.name for a in optimisations],
+                "upgrade-repo.optimisation.performed",
+            )
+            ui.write(b'\n\n')
+
     def printupgradeactions():
         for a in actions:
-            ui.write(b'%s\n   %s\n\n' % (a.name, a.upgrademessage))
+            ui.status(b'%s\n   %s\n\n' % (a.name, a.upgrademessage))
 
     if not run:
         fromconfig = []
@@ -1291,35 +1320,35 b' def upgraderepo('
         if fromconfig or onlydefault:
 
             if fromconfig:
-                ui.write(
+                ui.status(
                     _(
                         b'repository lacks features recommended by '
                         b'current config options:\n\n'
                     )
                 )
                 for i in fromconfig:
-                    ui.write(b'%s\n   %s\n\n' % (i.name, i.description))
+                    ui.status(b'%s\n   %s\n\n' % (i.name, i.description))
 
             if onlydefault:
-                ui.write(
+                ui.status(
                     _(
                         b'repository lacks features used by the default '
                         b'config options:\n\n'
                     )
                 )
                 for i in onlydefault:
-                    ui.write(b'%s\n   %s\n\n' % (i.name, i.description))
+                    ui.status(b'%s\n   %s\n\n' % (i.name, i.description))
 
-            ui.write(b'\n')
+            ui.status(b'\n')
         else:
-            ui.write(
+            ui.status(
                 _(
                     b'(no feature deficiencies found in existing '
                     b'repository)\n'
                 )
             )
 
-        ui.write(
+        ui.status(
             _(
                 b'performing an upgrade with "--run" will make the following '
                 b'changes:\n\n'
@@ -1327,31 +1356,33 b' def upgraderepo('
         )
 
         printrequirements()
+        printoptimisations()
         printupgradeactions()
 
         unusedoptimize = [i for i in alloptimizations if i not in actions]
 
         if unusedoptimize:
-            ui.write(
+            ui.status(
                 _(
                     b'additional optimizations are available by specifying '
                     b'"--optimize <name>":\n\n'
                 )
             )
             for i in unusedoptimize:
-                ui.write(_(b'%s\n   %s\n\n') % (i.name, i.description))
+                ui.status(_(b'%s\n   %s\n\n') % (i.name, i.description))
         return
 
     # Else we're in the run=true case.
     ui.write(_(b'upgrade will perform the following actions:\n\n'))
     printrequirements()
+    printoptimisations()
     printupgradeactions()
 
     upgradeactions = [a.name for a in actions]
 
-    ui.write(_(b'beginning upgrade...\n'))
+    ui.status(_(b'beginning upgrade...\n'))
     with repo.wlock(), repo.lock():
-        ui.write(_(b'repository locked and read-only\n'))
+        ui.status(_(b'repository locked and read-only\n'))
         # Our strategy for upgrading the repository is to create a new,
         # temporary repository, write data to it, then do a swap of the
         # data. There are less heavyweight ways to do this, but it is easier
@@ -1360,7 +1391,7 b' def upgraderepo('
         tmppath = pycompat.mkdtemp(prefix=b'upgrade.', dir=repo.path)
         backuppath = None
         try:
-            ui.write(
+            ui.status(
                 _(
                     b'creating temporary repository to stage migrated '
                     b'data: %s\n'
@@ -1377,15 +1408,17 b' def upgraderepo('
                     ui, repo, dstrepo, newreqs, upgradeactions, revlogs=revlogs
                 )
             if not (backup or backuppath is None):
-                ui.write(_(b'removing old repository content%s\n') % backuppath)
+                ui.status(
+                    _(b'removing old repository content%s\n') % backuppath
+                )
                 repo.vfs.rmtree(backuppath, forcibly=True)
                 backuppath = None
 
         finally:
-            ui.write(_(b'removing temporary repository %s\n') % tmppath)
+            ui.status(_(b'removing temporary repository %s\n') % tmppath)
             repo.vfs.rmtree(tmppath, forcibly=True)
 
-            if backuppath:
+            if backuppath and not ui.quiet:
                 ui.warn(
                     _(b'copy of old repository backed up at %s\n') % backuppath
                 )

mercurial/util.py

@@ -205,6 +205,8 b' def nouideprecwarn(msg, version, stackle'
             b" update your code.)"
         ) % version
         warnings.warn(pycompat.sysstr(msg), DeprecationWarning, stacklevel + 1)
+        # on python 3 with chg, we will need to explicitly flush the output
+        sys.stderr.flush()
 
 
 DIGESTS = {
@@ -1379,8 +1381,8 b' def acceptintervention(tr=None):'
 
 
 @contextlib.contextmanager
-def nullcontextmanager():
-    yield
+def nullcontextmanager(enter_result=None):
+    yield enter_result
 
 
 class _lrucachenode(object):
@@ -2845,7 +2847,7 b' if pyplatform.python_implementation() =='
     # [1]: fixed by changeset 67dc99a989cd in the cpython hg repo.
     #
     # Here we workaround the EINTR issue for fileobj.__iter__. Other methods
-    # like "read*" are ignored for now, as Python < 2.7.4 is a minority.
+    # like "read*" work fine, as we do not support Python < 2.7.4.
     #
     # Although we can workaround the EINTR issue for fp.__iter__, it is slower:
     # "for x in fp" is 4x faster than "for x in iter(fp.readline, '')" in
@@ -2857,39 +2859,6 b' if pyplatform.python_implementation() =='
     # affects things like pipes, sockets, ttys etc. We treat "normal" (S_ISREG)
     # files approximately as "fast" files and use the fast (unsafe) code path,
     # to minimize the performance impact.
-    if sys.version_info >= (2, 7, 4):
-        # fp.readline deals with EINTR correctly, use it as a workaround.
-        def _safeiterfile(fp):
-            return iter(fp.readline, b'')
-
-    else:
-        # fp.read* are broken too, manually deal with EINTR in a stupid way.
-        # note: this may block longer than necessary because of bufsize.
-        def _safeiterfile(fp, bufsize=4096):
-            fd = fp.fileno()
-            line = b''
-            while True:
-                try:
-                    buf = os.read(fd, bufsize)
-                except OSError as ex:
-                    # os.read only raises EINTR before any data is read
-                    if ex.errno == errno.EINTR:
-                        continue
-                    else:
-                        raise
-                line += buf
-                if b'\n' in buf:
-                    splitted = line.splitlines(True)
-                    line = b''
-                    for l in splitted:
-                        if l[-1] == b'\n':
-                            yield l
-                        else:
-                            line = l
-                if not buf:
-                    break
-            if line:
-                yield line
 
     def iterfile(fp):
         fastpath = True
@@ -2898,7 +2867,8 b' if pyplatform.python_implementation() =='
         if fastpath:
             return fp
         else:
-            return _safeiterfile(fp)
+            # fp.readline deals with EINTR correctly, use it as a workaround.
+            return iter(fp.readline, b'')
 
 
 else:
@@ -3656,3 +3626,44 b' def with_lc_ctype():'
             locale.setlocale(locale.LC_CTYPE, oldloc)
     else:
         yield
+
+
+def _estimatememory():
+    """Provide an estimate for the available system memory in Bytes.
+
+    If no estimate can be provided on the platform, returns None.
+    """
+    if pycompat.sysplatform.startswith(b'win'):
+        # On Windows, use the GlobalMemoryStatusEx kernel function directly.
+        from ctypes import c_long as DWORD, c_ulonglong as DWORDLONG
+        from ctypes.wintypes import Structure, byref, sizeof, windll
+
+        class MEMORYSTATUSEX(Structure):
+            _fields_ = [
+                ('dwLength', DWORD),
+                ('dwMemoryLoad', DWORD),
+                ('ullTotalPhys', DWORDLONG),
+                ('ullAvailPhys', DWORDLONG),
+                ('ullTotalPageFile', DWORDLONG),
+                ('ullAvailPageFile', DWORDLONG),
+                ('ullTotalVirtual', DWORDLONG),
+                ('ullAvailVirtual', DWORDLONG),
+                ('ullExtendedVirtual', DWORDLONG),
+            ]
+
+        x = MEMORYSTATUSEX()
+        x.dwLength = sizeof(x)
+        windll.kernel32.GlobalMemoryStatusEx(byref(x))
+        return x.ullAvailPhys
+
+    # On newer Unix-like systems and Mac OSX, the sysconf interface
+    # can be used. _SC_PAGE_SIZE is part of POSIX; _SC_PHYS_PAGES
+    # seems to be implemented on most systems.
+    try:
+        pagesize = os.sysconf(os.sysconf_names['SC_PAGE_SIZE'])
+        pages = os.sysconf(os.sysconf_names['SC_PHYS_PAGES'])
+        return pagesize * pages
+    except OSError:  # sysconf can fail
+        pass
+    except KeyError:  # unknown parameter
+        pass

mercurial/utils/procutil.py

@@ -37,9 +37,10 b' from ..utils import resourceutil'
 
 osutil = policy.importmod('osutil')
 
-stderr = pycompat.stderr
-stdin = pycompat.stdin
-stdout = pycompat.stdout
+if pycompat.iswindows:
+    from .. import windows as platform
+else:
+    from .. import posix as platform
 
 
 def isatty(fp):
@@ -49,33 +50,108 b' def isatty(fp):'
         return False
 
 
-# Python 2 uses the C library's standard I/O streams. Glibc determines
-# buffering on first write to stdout - if we replace a TTY destined stdout with
-# a pipe destined stdout (e.g. pager), we want line buffering (or unbuffered,
-# on Windows).
-# Python 3 rolls its own standard I/O streams.
-if isatty(stdout):
+class LineBufferedWrapper(object):
+    def __init__(self, orig):
+        self.orig = orig
+
+    def __getattr__(self, attr):
+        return getattr(self.orig, attr)
+
+    def write(self, s):
+        orig = self.orig
+        res = orig.write(s)
+        if s.endswith(b'\n'):
+            orig.flush()
+        return res
+
+
+io.BufferedIOBase.register(LineBufferedWrapper)
+
+
+def make_line_buffered(stream):
+    if pycompat.ispy3 and not isinstance(stream, io.BufferedIOBase):
+        # On Python 3, buffered streams can be expected to subclass
+        # BufferedIOBase. This is definitively the case for the streams
+        # initialized by the interpreter. For unbuffered streams, we don't need
+        # to emulate line buffering.
+        return stream
+    if isinstance(stream, LineBufferedWrapper):
+        return stream
+    return LineBufferedWrapper(stream)
+
+
+class WriteAllWrapper(object):
+    def __init__(self, orig):
+        self.orig = orig
+
+    def __getattr__(self, attr):
+        return getattr(self.orig, attr)
+
+    def write(self, s):
+        write1 = self.orig.write
+        m = memoryview(s)
+        total_to_write = len(s)
+        total_written = 0
+        while total_written < total_to_write:
+            total_written += write1(m[total_written:])
+        return total_written
+
+
+io.IOBase.register(WriteAllWrapper)
+
+
+def _make_write_all(stream):
+    assert pycompat.ispy3
+    if isinstance(stream, WriteAllWrapper):
+        return stream
+    if isinstance(stream, io.BufferedIOBase):
+        # The io.BufferedIOBase.write() contract guarantees that all data is
+        # written.
+        return stream
+    # In general, the write() method of streams is free to write only part of
+    # the data.
+    return WriteAllWrapper(stream)
+
+
+if pycompat.ispy3:
+    # Python 3 implements its own I/O streams.
+    # TODO: .buffer might not exist if std streams were replaced; we'll need
+    # a silly wrapper to make a bytes stream backed by a unicode one.
+    stdin = sys.stdin.buffer
+    stdout = _make_write_all(sys.stdout.buffer)
+    stderr = _make_write_all(sys.stderr.buffer)
     if pycompat.iswindows:
-        # Windows doesn't support line buffering
-        stdout = os.fdopen(stdout.fileno(), 'wb', 0)
-    elif not pycompat.ispy3:
-        # on Python 3, stdout (sys.stdout.buffer) is already line buffered and
-        # buffering=1 is not handled in binary mode
-        stdout = os.fdopen(stdout.fileno(), 'wb', 1)
+        # Work around Windows bugs.
+        stdout = platform.winstdout(stdout)
+        stderr = platform.winstdout(stderr)
+    if isatty(stdout):
+        # The standard library doesn't offer line-buffered binary streams.
+        stdout = make_line_buffered(stdout)
+else:
+    # Python 2 uses the I/O streams provided by the C library.
+    stdin = sys.stdin
+    stdout = sys.stdout
+    stderr = sys.stderr
+    if pycompat.iswindows:
+        # Work around Windows bugs.
+        stdout = platform.winstdout(stdout)
+        stderr = platform.winstdout(stderr)
+    if isatty(stdout):
+        if pycompat.iswindows:
+            # The Windows C runtime library doesn't support line buffering.
+            stdout = make_line_buffered(stdout)
+        else:
+            # glibc determines buffering on first write to stdout - if we
+            # replace a TTY destined stdout with a pipe destined stdout (e.g.
+            # pager), we want line buffering.
+            stdout = os.fdopen(stdout.fileno(), 'wb', 1)
 
-if pycompat.iswindows:
-    from .. import windows as platform
-
-    stdout = platform.winstdout(stdout)
-else:
-    from .. import posix as platform
 
 findexe = platform.findexe
 _gethgcmd = platform.gethgcmd
 getuser = platform.getuser
 getpid = os.getpid
 hidewindow = platform.hidewindow
-quotecommand = platform.quotecommand
 readpipe = platform.readpipe
 setbinary = platform.setbinary
 setsignalhandler = platform.setsignalhandler
@@ -140,7 +216,7 b" def popen(cmd, mode=b'rb', bufsize=-1):"
 
 def _popenreader(cmd, bufsize):
     p = subprocess.Popen(
-        tonativestr(quotecommand(cmd)),
+        tonativestr(cmd),
         shell=True,
         bufsize=bufsize,
         close_fds=closefds,
@@ -151,7 +227,7 b' def _popenreader(cmd, bufsize):'
 
 def _popenwriter(cmd, bufsize):
     p = subprocess.Popen(
-        tonativestr(quotecommand(cmd)),
+        tonativestr(cmd),
         shell=True,
         bufsize=bufsize,
         close_fds=closefds,
@@ -397,7 +473,6 b' def system(cmd, environ=None, cwd=None, '
         stdout.flush()
     except Exception:
         pass
-    cmd = quotecommand(cmd)
     env = shellenviron(environ)
     if out is None or isstdout(out):
         rc = subprocess.call(

mercurial/windows.py

@@ -186,11 +186,26 b" def posixfile(name, mode=b'r', buffering"
 listdir = osutil.listdir
 
 
+# copied from .utils.procutil, remove after Python 2 support was dropped
+def _isatty(fp):
+    try:
+        return fp.isatty()
+    except AttributeError:
+        return False
+
+
 class winstdout(object):
-    '''stdout on windows misbehaves if sent through a pipe'''
+    '''Some files on Windows misbehave.
+
+    When writing to a broken pipe, EINVAL instead of EPIPE may be raised.
+
+    When writing too many bytes to a console at the same, a "Not enough space"
+    error may happen. Python 3 already works around that.
+    '''
 
     def __init__(self, fp):
         self.fp = fp
+        self.throttle = not pycompat.ispy3 and _isatty(fp)
 
     def __getattr__(self, key):
         return getattr(self.fp, key)
@@ -203,12 +218,13 b' class winstdout(object):'
 
     def write(self, s):
         try:
+            if not self.throttle:
+                return self.fp.write(s)
             # This is workaround for "Not enough space" error on
             # writing large size of data to console.
             limit = 16000
             l = len(s)
             start = 0
-            self.softspace = 0
             while start < l:
                 end = start + limit
                 self.fp.write(s[start:end])
@@ -474,14 +490,6 b' def shellsplit(s):'
     return pycompat.maplist(_unquote, pycompat.shlexsplit(s, posix=False))
 
 
-def quotecommand(cmd):
-    """Build a command string suitable for os.popen* calls."""
-    if sys.version_info < (2, 7, 1):
-        # Python versions since 2.7.1 do this extra quoting themselves
-        return b'"' + cmd + b'"'
-    return cmd
-
-
 # if you change this stub into a real check, please try to implement the
 # username and groupname functions above, too.
 def isowner(st):

mercurial/wireprotov1server.py

@@ -339,7 +339,7 b' def capabilities(repo, proto):'
 def changegroup(repo, proto, roots):
     nodes = wireprototypes.decodelist(roots)
     outgoing = discovery.outgoing(
-        repo, missingroots=nodes, missingheads=repo.heads()
+        repo, missingroots=nodes, ancestorsof=repo.heads()
     )
     cg = changegroupmod.makechangegroup(repo, outgoing, b'01', b'serve')
     gen = iter(lambda: cg.read(32768), b'')
@@ -350,7 +350,7 b' def changegroup(repo, proto, roots):'
 def changegroupsubset(repo, proto, bases, heads):
     bases = wireprototypes.decodelist(bases)
     heads = wireprototypes.decodelist(heads)
-    outgoing = discovery.outgoing(repo, missingroots=bases, missingheads=heads)
+    outgoing = discovery.outgoing(repo, missingroots=bases, ancestorsof=heads)
     cg = changegroupmod.makechangegroup(repo, outgoing, b'01', b'serve')
     gen = iter(lambda: cg.read(32768), b'')
     return wireprototypes.streamres(gen=gen)

relnotes/next

@@ -1,11 +1,45 b''
 == New Features ==
 
+ * clonebundles can be annotated with the expected memory requirements
+   using the `REQUIREDRAM` option. This allows clients to skip
+   bundles created with large zstd windows and fallback to larger, but
+   less demanding bundles.
+
+ * The `phabricator` extension now provides more functionality of the
+   arcanist CLI like changing the status of a differential.
+
+ * Phases processing is much faster, especially for repositories with
+   old non-public changesets.
 
 == New Experimental Features ==
 
+ * The core of some hg operations have been (and are being)
+   implemented in rust, for speed. `hg status` on a repository with
+   300k tracked files goes from 1.8s to 0.6s for instance.
+   This has currently been tested only on linux, and does not build on
+   windows. See rust/README.rst in the mercurial repository for
+   instructions to opt into this.
 
 == Backwards Compatibility Changes ==
 
+* Mercurial now requires at least Python 2.7.9 or a Python version that
+  backported modern SSL/TLS features (as defined in PEP 466), and that Python
+  was compiled against a OpenSSL version supporting TLS 1.1 or TLS 1.2
+  (likely this requires the OpenSSL version to be at least 1.0.1).
+
+* The `hg perfwrite` command from contrib/perf.py was made more flexible and
+  changed its default behavior. To get the previous behavior, run `hg perfwrite
+  --nlines=100000 --nitems=1 --item='Testing write performance' --batch-line`.
+
 
 == Internal API Changes ==
 
+ * logcmdutil.diffordiffstat() now takes contexts instead of nodes.
+
+ * The `mergestate` class along with some related methods and constants have
+   moved from `mercurial.merge` to a new `mercurial.mergestate` module.
+
+ * The `phasecache` class now uses sparse dictionaries for the phase data.
+   New accessors are provided to detect if any non-public changeset exists
+   (`hasnonpublicphases`) and get the correponsponding root set
+   (`nonpublicphaseroots`).

rust/Cargo.lock

@@ -42,11 +42,6 b' version = "1.3.4"'
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
-name = "cc"
-version = "1.0.50"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-
-[[package]]
 name = "cfg-if"
 version = "0.1.10"
 source = "registry+https://github.com/rust-lang/crates.io-index"
@@ -63,7 +58,7 b' dependencies = ['
 
 [[package]]
 name = "clap"
-version = "2.33.0"
+version = "2.33.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -208,22 +203,20 b' name = "hg-core"'
 version = "0.1.0"
 dependencies = [
  "byteorder 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)",
- "cc 1.0.50 (registry+https://github.com/rust-lang/crates.io-index)",
- "clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "clap 2.33.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "crossbeam 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "hex 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "libc 0.2.67 (registry+https://github.com/rust-lang/crates.io-index)",
  "log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)",
  "memchr 2.3.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "memmap 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "micro-timer 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "micro-timer 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "pretty_assertions 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_distr 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_pcg 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "rayon 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "regex 1.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
+ "regex 1.3.9 (registry+https://github.com/rust-lang/crates.io-index)",
  "same-file 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
  "tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "twox-hash 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -287,16 +280,16 b' dependencies = ['
 
 [[package]]
 name = "micro-timer"
-version = "0.2.1"
+version = "0.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
- "micro-timer-macros 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "micro-timer-macros 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "scopeguard 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "micro-timer-macros"
-version = "0.2.0"
+version = "0.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "proc-macro2 1.0.9 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -369,7 +362,7 b' version = "0.4.1"'
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "libc 0.2.67 (registry+https://github.com/rust-lang/crates.io-index)",
- "regex 1.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
+ "regex 1.3.9 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -378,7 +371,7 b' version = "0.4.1"'
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "libc 0.2.67 (registry+https://github.com/rust-lang/crates.io-index)",
- "regex 1.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
+ "regex 1.3.9 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -471,18 +464,18 b' source = "registry+https://github.com/ru'
 
 [[package]]
 name = "regex"
-version = "1.3.6"
+version = "1.3.9"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "aho-corasick 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)",
  "memchr 2.3.3 (registry+https://github.com/rust-lang/crates.io-index)",
- "regex-syntax 0.6.17 (registry+https://github.com/rust-lang/crates.io-index)",
+ "regex-syntax 0.6.18 (registry+https://github.com/rust-lang/crates.io-index)",
  "thread_local 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "regex-syntax"
-version = "0.6.17"
+version = "0.6.18"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 
 [[package]]
@@ -494,6 +487,14 b' dependencies = ['
 ]
 
 [[package]]
+name = "rhg"
+version = "0.1.0"
+dependencies = [
+ "clap 2.33.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "hg-core 0.1.0",
+]
+
+[[package]]
 name = "rustc_version"
 version = "0.2.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
@@ -655,10 +656,9 b' source = "registry+https://github.com/ru'
 "checksum autocfg 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "f8aac770f1885fd7e387acedd76065302551364496e46b3dd00860b2f8359b9d"
 "checksum bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
 "checksum byteorder 1.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "08c48aae112d48ed9f069b33538ea9e3e90aa263cfa3d1c24309612b1f7472de"
-"checksum cc 1.0.50 (registry+https://github.com/rust-lang/crates.io-index)" = "95e28fa049fda1c330bcf9d723be7663a899c4679724b34c81e9f5a326aab8cd"
 "checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
 "checksum chrono 0.4.11 (registry+https://github.com/rust-lang/crates.io-index)" = "80094f509cf8b5ae86a4966a39b3ff66cd7e2a3e594accec3743ff3fabeab5b2"
-"checksum clap 2.33.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5067f5bb2d80ef5d68b4c87db81601f0b75bca627bc2ef76b141d7b846a3c6d9"
+"checksum clap 2.33.1 (registry+https://github.com/rust-lang/crates.io-index)" = "bdfa80d47f954d53a35a64987ca1422f495b8d6483c0fe9f7117b36c2a792129"
 "checksum colored 1.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "f4ffc801dacf156c5854b9df4f425a626539c3a6ef7893cc0c5084a23f0b6c59"
 "checksum cpython 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "bfaf3847ab963e40c4f6dd8d6be279bdf74007ae2413786a0dcbb28c52139a95"
 "checksum crossbeam 0.7.3 (registry+https://github.com/rust-lang/crates.io-index)" = "69323bff1fb41c635347b8ead484a5ca6c3f11914d784170b158d8449ab07f8e"
@@ -680,8 +680,8 b' source = "registry+https://github.com/ru'
 "checksum memchr 2.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3728d817d99e5ac407411fa471ff9800a778d88a24685968b36824eaf4bee400"
 "checksum memmap 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6585fd95e7bb50d6cc31e20d4cf9afb4e2ba16c5846fc76793f11218da9c475b"
 "checksum memoffset 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "75189eb85871ea5c2e2c15abbdd541185f63b408415e5051f5cac122d8c774b9"
-"checksum micro-timer 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "987429cd6162a80ed5ff44fc790f5090b1c6d617ac73a2e272965ed91201d79b"
-"checksum micro-timer-macros 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "43cec5c0b38783eb33ef7bccf4b250b7a085703e11f5f2238fa31969e629388a"
+"checksum micro-timer 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "25b31d6cb9112984323d05d7a353f272ae5d7a307074f9ab9b25c00121b8c947"
+"checksum micro-timer-macros 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5694085dd384bb9e824207facc040c248d9df653f55e28c3ad0686958b448504"
 "checksum num-integer 0.1.42 (registry+https://github.com/rust-lang/crates.io-index)" = "3f6ea62e9d81a77cd3ee9a2a5b9b609447857f3d358704331e4ef39eb247fcba"
 "checksum num-traits 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)" = "c62be47e61d1842b9170f0fdeec8eba98e60e90e5446449a0545e5152acd7096"
 "checksum num_cpus 1.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "46203554f085ff89c235cd12f7075f3233af9b11ed7c9e16dfe2560d03313ce6"
@@ -701,8 +701,8 b' source = "registry+https://github.com/ru'
 "checksum rayon 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "db6ce3297f9c85e16621bb8cca38a06779ffc31bb8184e1be4bed2be4678a098"
 "checksum rayon-core 1.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "08a89b46efaf957e52b18062fb2f4660f8b8a4dde1807ca002690868ef2c85a9"
 "checksum redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)" = "2439c63f3f6139d1b57529d16bc3b8bb855230c8efcc5d3a896c8bea7c3b1e84"
-"checksum regex 1.3.6 (registry+https://github.com/rust-lang/crates.io-index)" = "7f6946991529684867e47d86474e3a6d0c0ab9b82d5821e314b1ede31fa3a4b3"
-"checksum regex-syntax 0.6.17 (registry+https://github.com/rust-lang/crates.io-index)" = "7fe5bd57d1d7414c6b5ed48563a2c855d995ff777729dcd91c369ec7fea395ae"
+"checksum regex 1.3.9 (registry+https://github.com/rust-lang/crates.io-index)" = "9c3780fcf44b193bc4d09f36d2a3c87b251da4a046c87795a0d35f4f927ad8e6"
+"checksum regex-syntax 0.6.18 (registry+https://github.com/rust-lang/crates.io-index)" = "26412eb97c6b088a6997e05f69403a802a92d520de2f8e63c2b65f9e0f47c4e8"
 "checksum remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4a83fa3702a688b9359eccba92d153ac33fd2e8462f9e0e3fdf155239ea7792e"
 "checksum rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a"
 "checksum same-file 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"

rust/Cargo.toml

@@ -1,3 +1,3 b''
 [workspace]
-members = ["hg-core", "hg-cpython"]
+members = ["hg-core", "hg-cpython", "rhg"]
 exclude = ["chg", "hgcli"]

rust/README.rst

@@ -8,9 +8,9 b' improves performance in some areas.'
 
 There are currently three independent rust projects:
 - chg. An implementation of chg, in rust instead of C.
-- hgcli. A experiment for starting hg in rust rather than in python,
-  by linking with the python runtime. Probably meant to be replaced by
-  PyOxidizer at some point.
+- hgcli. A project that provide a (mostly) self-contained "hg" binary,
+  for ease of deployment and a bit of speed, using PyOxidizer. See
+  hgcli/README.md.
 - hg-core (and hg-cpython): implementation of some
   functionality of mercurial in rust, e.g. ancestry computations in
   revision graphs, status or pull discovery. The top-level ``Cargo.toml`` file
@@ -27,8 +27,6 b' built without rust previously)::'
   $ ./hg debuginstall | grep -i rust # to validate rust is in use
   checking Rust extensions (installed)
   checking module policy (rust+c-allow)
-  checking "re2" regexp engine Rust bindings (installed)
-
 
 If the environment variable ``HGWITHRUSTEXT=cpython`` is set, the Rust
 extension will be used by default unless ``--no-rust``.
@@ -36,35 +34,20 b' extension will be used by default unless'
 One day we may use this environment variable to switch to new experimental
 binding crates like a hypothetical ``HGWITHRUSTEXT=hpy``.
 
-Using the fastest ``hg status``
--------------------------------
-
-The code for ``hg status`` needs to conform to ``.hgignore`` rules, which are
-all translated into regex. 
-
-In the first version, for compatibility and ease of development reasons, the 
-Re2 regex engine was chosen until we figured out if the ``regex`` crate had
-similar enough behavior.
-
-Now that that work has been done, the default behavior is to use the ``regex``
-crate, that provides a significant performance boost compared to the standard 
-Python + C path in many commands such as ``status``, ``diff`` and ``commit``,
+Profiling
+=========
 
-However, the ``Re2`` path remains slightly faster for our use cases and remains
-a better option for getting the most speed out of your Mercurial. 
+Setting the environment variable ``RUST_LOG=trace`` will make hg print
+a few high level rust-related performance numbers. It can also
+indicate why the rust code cannot be used (say, using lookarounds in
+hgignore).
 
-If you want to use ``Re2``, you need to install ``Re2`` following Google's 
-guidelines: https://github.com/google/re2/wiki/Install.
-Then, use ``HG_RUST_FEATURES=with-re2`` and 
-``HG_RE2_PATH=system|<path to your re2 install>`` when building ``hg`` to 
-signal the use of Re2. Using the local path instead of the "system" RE2 links
-it statically.
-
-For example::
-
-  $ HG_RUST_FEATURES=with-re2 HG_RE2_PATH=system make PURE=--rust
-  $ # OR
-  $ HG_RUST_FEATURES=with-re2 HG_RE2_PATH=/path/to/re2 make PURE=--rust
+``py-spy`` (https://github.com/benfred/py-spy) can be used to
+construct a single profile with rust functions and python functions
+(as opposed to ``hg --profile``, which attributes time spent in rust
+to some unlucky python code running shortly after the rust code, and
+as opposed to tools for native code like ``perf``, which attribute
+time to the python interpreter instead of python functions).
 
 Developing Rust
 ===============
@@ -114,14 +97,3 b' To format the entire Rust workspace::'
   $ cargo +nightly fmt
 
 This requires you to have the nightly toolchain installed.
-
-Additional features
--------------------
-
-As mentioned in the section about ``hg status``, code paths using ``re2`` are
-opt-in.
-
-For example::
-
-  $ cargo check --features with-re2
-