branching: merge stable into default

relnotes/6.3

@@ -0,0 +1,98 b''
+= Mercurial 6.3 =
+
+== New Features ==
+
+ * testlib: add `--raw-sha1` option to `f`
+ * rhg: add `config.rhg` helptext
+ * config: add alias from `hg help rhg` to `hg help rust`
+ * rhg: add a config option to fall back immediately
+ * bundle: introduce a --exact option
+ * perf-bundle: add a new command to benchmark bundle creation time
+ * perf-bundle: accept --rev arguments
+ * perf-bundle: accept --type argument
+ * perf-unbundle: add a perf command to time the unbundle operation
+ * perf: introduce a benchmark for delta-find
+ * contrib: add support for rhel9
+ * phase-shelve: Implement a 'shelve.store' experimental config
+ * debug-delta-find: introduce a quiet mode
+ * sort-revset: introduce a `random` variant
+ * phase: introduce a dedicated requirement for the `archived` phase
+ * rebase: add boolean config item rebase.store-source
+ * rhg: make [rhg status -v] work when it needs no extra output
+ * rhg: support "!" syntax for disabling extensions
+ * rhg: add debugrhgsparse command to help figure out bugs in rhg
+ * rhg: add sparse support
+ * rhg-status: add support for narrow clones
+ * templates: add filter to reverse list
+ * contrib: add pull_logger extension
+ * revset: handle wdir() in `roots()`
+ * revset: handle wdir() in `sort(..., -topo)`
+ * rhg: support tweakdefaults
+ * rhg: parallellize computation of [unsure_is_modified]
+
+== Default Format Change ==
+
+These changes affect newly created repositories (or new clones) done with
+Mercurial 6.3.
+
+== New Experimental Features ==
+
+== Bug Fixes ==
+
+ * shelve: demonstrate that the state is different across platforms (issue6735)
+ * shelve: in test for trailing whitespace, strip commit (issue6735)
+ * shelve: remove strip and rely on prior state (issue6735)
+ * tests: fix http-bad-server expected errors for python 3.10 (issue6643)
+ * status: let `--no-copies` override `ui.statuscopies`
+ * releasenotes: use re.MULTILINE mode when checking admonitions
+ * rhg: fallback to slow path on invalid patterns in hgignore
+ * Fix a bunch of leftover str/bytes issues from Python 3 migration
+ * keepalive: ensure `close_all()` actually closes all cached connections
+ * lfs: fix blob corruption when tranferring with workers on posix
+ * lfs: avoid closing connections when the worker doesn't fork
+ * dirstate-v2: update constant that wasn't kept in sync
+ * dirstate-v2: fix edge case where entries aren't sorted
+ * upgrade: no longer keep all revlogs in memory at any point
+ * rust-status: save new dircache even if just invalidated
+ * dirstate-v2: hash the source of the ignore patterns as well
+ * rhg: fallback when encountering ellipsis revisions
+ * shelve: handle empty parents and nodestoremove in shelvedstate (issue6748)
+ * profile: prevent a crash when line number is unknown
+ * tags-fnode-cache: do not repeatedly open the filelog in a loop
+ * tags-fnode-cache: skip building a changectx in getfnode
+ * rust: create wrapper struct to reduce `regex` contention issues
+
+== Backwards Compatibility Changes ==
+
+ * chg worker processes will now correctly load per-repository configuration
+   when given a both a relative `--repository` path and an alternate working
+   directory via `--cwd`. A side-effect of this change is that these workers
+   will now return an error if hg cannot find the current working directory,
+   even when a different directory is specified via `--cwd`.
+ * phase: rename the requirement for internal-phase from `internal-phase` to `use-internal-phase` (see 74fb1842f8b962cf03d7cd5b841dbcf2ae065587)
+
+== Internal API Changes ==
+
+== Miscellaneous ==
+
+ * sslutil: use proper attribute to select python 3.7+
+ * typing: suppress a few pyi-errors with more recent pytype
+ * ci: bump pytype to 2022.03.29
+ * bundlespec: add documentation about existing option
+ * subrepo: avoid opening console window for non-native subrepos on Windows
+ * setup: unconditionally enable the `long-paths-support` option on Windows
+ * setup: use the full executable manifest from `python.exe`
+ * tests: work around libmagic bug in svn subrepo tests
+ * packagelib: use python3 by default
+ * Improve `hg bisect` performance
+ * perf: properly process formatter option in perf::unbundle
+ * compare-disco: miscellaneous display improvements
+ * fsmonitor: better compatibility with newer Pythons
+ * revlog: finer computation of "issnapshot"
+ * rhg: don't fallback if `strip` or `rebase` are activated
+ * perf: make perf::bundle compatible before 61ba04693d65
+ * perf: make perf::bundle compatible down to 5.2
+ * perf-unbundle: improve compatibility
+ * run-tests: display the time it took to install Mercurial
+ * mergetools: don't let meld open all changed files on startup
+ * dirstate-v2: skip evaluation of hgignore regex on cached directories

.hgsigs

@@ -234,3 +234,5 b' 094a5fa3cf52f936e0de3f1e507c818bee5ece6b'
 f69bffd00abe3a1b94d1032eb2c92e611d16a192 0 iQHNBAABCgA3FiEEH2b4zfZU6QXBHaBhoR4BzQ4F2VYFAmLifPsZHGFscGhhcmVAcmFwaGFlbGdvbWVzLmRldgAKCRChHgHNDgXZVukEC/oCa6AzaJlWh6G45Ap7BCWyB3EDWmcep07W8zRTfHQuuXslNFxRfj8O1DLVP05nDa1Uo2u1nkDxTH+x1fX0q4G8U/yLzCNsiBkCWSeEM8IeolarzzzvFe9Zk+UoRoRlc+vKAjxChtYTEnggQXjLdK+EdbXfEz2kJwdYlGX3lLr0Q2BKnBjSUvFe1Ma/1wxEjZIhDr6t7o8I/49QmPjK7RCYW1WBv77gnml0Oo8cxjDUR9cjqfeKtXKbMJiCsoXCS0hx3vJkBOzcs4ONEIw934is38qPNBBsaUjMrrqm0Mxs6yFricYqGVpmtNijsSRsfS7ZgNfaGaC2Bnu1E7P0A+AzPMPf/BP4uW9ixMbP1hNdr/6N41n19lkdjyQXVWGhB8RM+muf3jc6ZVvgZPMlxvFiz4/rP9nVOdrB96ssFZ9V2Ca/j2tU40AOgjI6sYsAR8pSSgmIdqe+DZQISHTT8D+4uVbtwYD49VklBcxudlbd3dAc5z9rVI3upsyByfRMROc=
 b5c8524827d20fe2e0ca8fb1234a0fe35a1a36c7 0 iQHNBAABCgA3FiEEH2b4zfZU6QXBHaBhoR4BzQ4F2VYFAmMQxRoZHGFscGhhcmVAcmFwaGFlbGdvbWVzLmRldgAKCRChHgHNDgXZVm2gC/9HikIaOE49euIoLj6ctYsJY9PSQK4Acw7BXvdsTVMmW27o87NxH75bGBbmPQ57X1iuKLCQ1RoU3p2Eh1gPbkIsouWO3enBIfsFmkPtWQz28zpCrI9CUXg2ug4PGFPN9XyxNmhJ7vJ4Cst2tRxz9PBKUBO2EXJN1UKIdMvurIeT2sQrDQf1ePc85QkXx79231wZyF98smnV7UYU9ZPFnAzfcuRzdFn7UmH3KKxHTZQ6wAevj/fJXf5NdTlqbeNmq/t75/nGKXSFPWtRGfFs8JHGkkLgBiTJVsHYSqcnKNdVldIFUoJP4c2/SPyoBkqNvoIrr73XRo8tdDF1iY4ddmhHMSmKgSRqLnIEgew3Apa/IwPdolg+lMsOtcjgz4CB9agJ+O0+rdZd2ZUBNMN0nBSUh+lrkMjat8TJAlvut9h/6HAe4Dz8WheoWol8f8t1jLOJvbdvsMYi+Hf9CZjp7PlHT9y/TnDarcw2YIrf6Bv+Fm14ZDelu9VlF2zR1X8cofY=
 dbdee8ac3e3fcdda1fa55b90c0a235125b7f8e6f 0 iQHNBAABCgA3FiEEH2b4zfZU6QXBHaBhoR4BzQ4F2VYFAmM77dQZHGFscGhhcmVAcmFwaGFlbGdvbWVzLmRldgAKCRChHgHNDgXZViOTC/sEPicecV3h3v47VAIUigyKNWpcJ+epbRRaH6gqHTkexvULOPL6nJrdfBHkNry1KRtOcjaxQvtWZM+TRCfqsE++Q3ZYakRpWKontb/8xQSbmENvbnElLh6k0STxN/JVc480us7viDG5pHS9DLsgbkHmdCv5KdmSE0hphRrWX+5X7RTqpAfCgdwTkacB5Geu9QfRnuYjz6lvqbs5ITKtBGUYbg3hKzw2894FHtMqV6qa5rk1ZMmVDbQfKQaMVG41UWNoN7bLESi69EmF4q5jsXdIbuBy0KtNXmB+gdAaHN03B5xtc+IsQZOTHEUNlMgov3yEVTcA6fSG9/Z+CMsdCbyQxqkwakbwWS1L2WcAsrkHyafvbNdR2FU34iYRWOck8IUg2Ffv7UFrHabJDy+nY7vcTLb0f7lV4jLXMWEt1hvXWMYek6Y4jtWahg6fjmAdD3Uf4BMfsTdnQKPvJpWXx303jnST3xvFvuqbbbDlhLfAB9M6kxVntvCVkMlMpe39+gM=
+a3356ab610fc50000cf0ba55c424a4d96da11db7 0 iQHNBAABCgA3FiEEH2b4zfZU6QXBHaBhoR4BzQ4F2VYFAmNWr44ZHGFscGhhcmVAcmFwaGFlbGdvbWVzLmRldgAKCRChHgHNDgXZVjalC/9ddIeZ1qc3ykUZb+vKw+rZ6WS0rnDgrfFYBQFooK106lB+IC2PlghXSrY2hXn/7Dk95bK90S9AO4TFidDPiRYuBYdXR+G+CzmYFtCQzGBgGyrWgpUYsZUeA3VNqZ+Zbwn/vRNiFVNDsrFudjE6xEwaYdepmoXJsv3NdgZME7T0ZcDIujIa7ihiXvGFPVzMyF/VZg4QvdmerC4pvkeKC3KRNjhBkMQbf0GtQ4kpgMFBj5bmgXbq9rftL5yYy+rDiRQ0qzpOMHbdxvSZjPhK/do5M3rt2cjPxtF+7R3AHxQ6plOf0G89BONYebopY92OIyA3Qg9d/zIKDmibhgyxj4G9YU3+38gPEpsNeEw0fkyxhQbCY3QpNX4JGFaxq5GVCUywvVIuqoiOcQeXlTDN70zhAQHUx0rcGe1Lc6I+rT6Y2lNjJIdiCiMAWIl0D+4SVrLqdMYdSMXcBajTxOudb9KZnu03zNMXuLb8FFk1lFzkY7AcWA++d02f15P3sVZsDXE=
+04f1dba53c961dfdb875c8469adc96fa999cfbed 0 iQHNBAABCgA3FiEEH2b4zfZU6QXBHaBhoR4BzQ4F2VYFAmNyC5sZHGFscGhhcmVAcmFwaGFlbGdvbWVzLmRldgAKCRChHgHNDgXZVqF+C/4uLaV/4nizZkWD3PjU1WyFYDg4bWDFOHb+PWuQ/3uoHXu1/EaYRnqmcDyOSJ99aXZBQ78rm9xhjxdmbklZ4ll1EGkqfTiYH+ld+rqE8iaqlc/DVy7pFXaenYwxletzO1OezzwF4XDLi6hcqzY9CXA3NM40vf6W4Rs5bEIi4eSbgJSNB1ll6ZzjvkU5bWTUoxSH+fxIJUuo27El2etdlKFQkS3/oTzWHejpVn6SQ1KyojTHMQBDRK4rqJBISp3gTf4TEezb0q0HTutJYDFdQNIRqx7V1Ao4Ei+YNbenJzcWJOA/2uk4V0AvZ4tnjgAzBYKwvIL1HfoQ0OmILeXjlVzV7Xu0G57lavum0sKkz/KZLKyYhKQHjYQLE7YMSM2y6/UEoFNN577vB47CHUq446PSMb8dGs2rmj66rj4iz5ml0yX+V9O2PpmIKoPAu1Y5/6zB9rCL76MRx182IW2m3rm4lsTfXPBPtea/OFt6ylxqCJRxaA0pht4FiAOvicPKXh4=

.hgtags

@@ -247,3 +247,5 b' 094a5fa3cf52f936e0de3f1e507c818bee5ece6b'
 f69bffd00abe3a1b94d1032eb2c92e611d16a192 6.2.1
 b5c8524827d20fe2e0ca8fb1234a0fe35a1a36c7 6.2.2
 dbdee8ac3e3fcdda1fa55b90c0a235125b7f8e6f 6.2.3
+a3356ab610fc50000cf0ba55c424a4d96da11db7 6.3rc0
+04f1dba53c961dfdb875c8469adc96fa999cfbed 6.3.0

contrib/perf.py

@@ -2676,11 +2676,34 b' def perf_unbundle(ui, repo, fname, **opt'
     """benchmark application of a bundle in a repository.
 
     This does not include the final transaction processing"""
+
     from mercurial import exchange
     from mercurial import bundle2
+    from mercurial import transaction
 
     opts = _byteskwargs(opts)
 
+    ###  some compatibility hotfix
+    #
+    # the data attribute is dropped in 63edc384d3b7 a changeset introducing a
+    # critical regression that break transaction rollback for files that are
+    # de-inlined.
+    method = transaction.transaction._addentry
+    pre_63edc384d3b7 = "data" in getargspec(method).args
+    # the `detailed_exit_code` attribute is introduced in 33c0c25d0b0f
+    # a changeset that is a close descendant of 18415fc918a1, the changeset
+    # that conclude the fix run for the bug introduced in 63edc384d3b7.
+    args = getargspec(error.Abort.__init__).args
+    post_18415fc918a1 = "detailed_exit_code" in args
+
+    old_max_inline = None
+    try:
+        if not (pre_63edc384d3b7 or post_18415fc918a1):
+            # disable inlining
+            old_max_inline = mercurial.revlog._maxinline
+            # large enough to never happen
+            mercurial.revlog._maxinline = 2 ** 50
+
     with repo.lock():
         bundle = [None, None]
         orig_quiet = repo.ui.quiet
@@ -2699,7 +2722,8 b' def perf_unbundle(ui, repo, fname, **opt'
                     f.seek(0)
                     bundle[0] = exchange.readbundle(ui, f, fname)
                     bundle[1] = repo.transaction(b'perf::unbundle')
-                    bundle[1]._report = noop_report  # silence the transaction
+                        # silence the transaction
+                        bundle[1]._report = noop_report
 
                 def apply():
                     gen, tr = bundle
@@ -2719,6 +2743,9 b' def perf_unbundle(ui, repo, fname, **opt'
             gen, tr = bundle
             if tr is not None:
                 tr.abort()
+    finally:
+        if old_max_inline is not None:
+            mercurial.revlog._maxinline = old_max_inline
 
 
 @command(

hgext/convert/bzr.py

@@ -23,9 +23,9 b' from . import common'
 # these do not work with demandimport, blacklist
 demandimport.IGNORES.update(
     [
-        b'breezy.transactions',
-        b'breezy.urlutils',
-        b'ElementPath',
+        'breezy.transactions',
+        'breezy.urlutils',
+        'ElementPath',
     ]
 )
 

hgext/highlight/highlight.py

@@ -11,7 +11,7 b''
 
 from mercurial import demandimport
 
-demandimport.IGNORES.update([b'pkgutil', b'pkg_resources', b'__main__'])
+demandimport.IGNORES.update(['pkgutil', 'pkg_resources', '__main__'])
 
 from mercurial import (
     encoding,

hgext/lfs/blobstore.py

@@ -601,14 +601,30 b' class _gitlfsremote:'
                             continue
                         raise
 
-        # Until https multiplexing gets sorted out
+        # Until https multiplexing gets sorted out.  It's not clear if
+        # ConnectionManager.set_ready() is externally synchronized for thread
+        # safety with Windows workers.
         if self.ui.configbool(b'experimental', b'lfs.worker-enable'):
+            # The POSIX workers are forks of this process, so before spinning
+            # them up, close all pooled connections.  Otherwise, there's no way
+            # to coordinate between them about who is using what, and the
+            # transfers will get corrupted.
+            #
+            # TODO: add a function to keepalive.ConnectionManager to mark all
+            #  ready connections as in use, and roll that back after the fork?
+            #  That would allow the existing pool of connections in this process
+            #  to be preserved.
+            def prefork():
+                for h in self.urlopener.handlers:
+                    getattr(h, "close_all", lambda: None)()
+
             oids = worker.worker(
                 self.ui,
                 0.1,
                 transfer,
                 (),
                 sorted(objects, key=lambda o: o.get(b'oid')),
+                prefork=prefork,
             )
         else:
             oids = transfer(sorted(objects, key=lambda o: o.get(b'oid')))

mercurial/dirstateutils/v2.py

@@ -56,11 +56,11 b' assert TREE_METADATA_SIZE == TREE_METADA'
 assert NODE_SIZE == NODE.size
 
 # match constant in mercurial/pure/parsers.py
-DIRSTATE_V2_DIRECTORY = 1 << 5
+DIRSTATE_V2_DIRECTORY = 1 << 13
 
 
 def parse_dirstate(map, copy_map, data, tree_metadata):
-    """parse a full v2-dirstate from a binary data into dictionnaries:
+    """parse a full v2-dirstate from a binary data into dictionaries:
 
     - map: a {path: entry} mapping that will be filled
     - copy_map: a {path: copy-source} mapping that will be filled
@@ -176,7 +176,7 b' class Node:'
 def pack_dirstate(map, copy_map):
     """
     Pack `map` and `copy_map` into the dirstate v2 binary format and return
-    the bytearray.
+    the tuple of (data, metadata) bytearrays.
 
     The on-disk format expects a tree-like structure where the leaves are
     written first (and sorted per-directory), going up levels until the root
@@ -191,7 +191,7 b' def pack_dirstate(map, copy_map):'
     # Algorithm explanation
 
     This explanation does not talk about the different counters for tracked
-    descendents and storing the copies, but that work is pretty simple once this
+    descendants and storing the copies, but that work is pretty simple once this
     algorithm is in place.
 
     ## Building a subtree
@@ -272,9 +272,9 b' def pack_dirstate(map, copy_map):'
         )
         return data, tree_metadata
 
-    sorted_map = sorted(map.items(), key=lambda x: x[0])
+    sorted_map = sorted(map.items(), key=lambda x: x[0].split(b"/"))
 
-    # Use a stack to not have to only remember the nodes we currently need
+    # Use a stack to have to only remember the nodes we currently need
     # instead of building the entire tree in memory
     stack = []
     current_node = Node(b"", None)

mercurial/help.py

@@ -486,6 +486,7 b' helptable = sorted('
             [
                 b'rust',
                 b'rustext',
+                b'rhg',
             ],
             _(b'Rust in Mercurial'),
             loaddoc(b'rust'),

mercurial/helptext/config.txt

@@ -2156,6 +2156,51 b' Alias definitions for revsets. See :hg:`'
     Currently, only the rebase and absorb commands consider this configuration.
     (EXPERIMENTAL)
 
+``rhg``
+-------
+
+The pure Rust fast-path for Mercurial. See `rust/README.rst` in the Mercurial repository.
+
+``fallback-executable``
+    Path to the executable to run in a sub-process when falling back to
+    another implementation of Mercurial.
+
+``fallback-immediately``
+    Fall back to ``fallback-executable`` as soon as possible, regardless of
+    the `rhg.on-unsupported` configuration. Useful for debugging, for example to
+    bypass `rhg` if the deault `hg` points to `rhg`.
+
+    Note that because this requires loading the configuration, it is possible
+    that `rhg` error out before being able to fall back.
+
+``ignored-extensions``
+    Controls which extensions should be ignored by `rhg`. By default, `rhg`
+    triggers the `rhg.on-unsupported` behavior any unsupported extensions.
+    Users can disable that behavior when they know that a given extension
+    does not need support from `rhg`.
+
+    Expects a list of extension names, or ``*`` to ignore all extensions.
+
+    Note: ``*:<suboption>`` is also a valid extension name for this
+    configuration option.
+    As of this writing, the only valid "global" suboption is ``required``.
+
+``on-unsupported``
+    Controls the behavior of `rhg` when detecting unsupported features.
+
+    Possible values are `abort` (default), `abort-silent` and `fallback`.
+
+    ``abort``
+        Print an error message describing what feature is not supported,
+        and exit with code 252
+
+    ``abort-silent``
+        Silently exit with code 252
+
+    ``fallback``
+        Try running the fallback executable with the same parameters
+        (and trace the fallback reason, use `RUST_LOG=trace` to see).
+
 ``share``
 ---------
 
@@ -2171,9 +2216,9 b' Alias definitions for revsets. See :hg:`'
     ``allow``
     Respects the feature presence in the share source
     ``upgrade-abort``
-    tries to upgrade the share to use share-safe; if it fails, aborts
+        Tries to upgrade the share to use share-safe; if it fails, aborts
     ``upgrade-allow``
-    tries to upgrade the share; if it fails, continue by
+        Tries to upgrade the share; if it fails, continue by
     respecting the share source setting
 
     Check :hg:`help config.format.use-share-safe` for details about the
@@ -2199,9 +2244,9 b' Alias definitions for revsets. See :hg:`'
     ``allow``
     Respects the feature presence in the share source
     ``downgrade-abort``
-    tries to downgrade the share to not use share-safe; if it fails, aborts
+        Tries to downgrade the share to not use share-safe; if it fails, aborts
     ``downgrade-allow``
-    tries to downgrade the share to not use share-safe;
+        Tries to downgrade the share to not use share-safe;
     if it fails, continue by respecting the shared source setting
 
     Check :hg:`help config.format.use-share-safe` for details about the

mercurial/helptext/internals/dirstate-v2.txt

@@ -283,8 +283,16 b' We define:'
   in inclusion order. This definition is recursive, as included files can
   themselves include more files.
 
-This hash is defined as the SHA-1 of the concatenation (in sorted
-order) of the "expanded contents" of each "root" ignore file.
+* "filepath" as the bytes of the ignore file path
+  relative to the root of the repository if inside the repository,
+  or the untouched path as defined in the configuration.
+
+This hash is defined as the SHA-1 of the following line format:
+
+<filepath> <sha1 of the "expanded contents">\n
+
+for each "root" ignore file. (in sorted order)
+
 (Note that computing this does not require actually concatenating
 into a single contiguous byte sequence.
 Instead a SHA-1 hasher object can be created

mercurial/helptext/rust.txt

@@ -89,6 +89,8 b' execution of certain commands while addi'
 The only way of trying it out is by building it from source. Please refer to
 `rust/README.rst` in the Mercurial repository.
 
+See `hg help config.rhg` for configuration options.
+
 Contributing
 ============
 

mercurial/keepalive.py

@@ -166,7 +166,9 b' class ConnectionManager:'
         if host:
             return list(self._hostmap[host])
         else:
-            return dict(self._hostmap)
+            return dict(
+                {h: list(conns) for (h, conns) in self._hostmap.items()}
+            )
 
 
 class KeepAliveHandler:
@@ -700,6 +702,17 b' class HTTPConnection(httplib.HTTPConnect'
         self.sentbytescount = 0
         self.receivedbytescount = 0
 
+    def __repr__(self):
+        base = super(HTTPConnection, self).__repr__()
+        local = "(unconnected)"
+        s = self.sock
+        if s:
+            try:
+                local = "%s:%d" % s.getsockname()
+            except OSError:
+                pass  # Likely not connected
+        return "<%s: %s <--> %s:%d>" % (base, local, self.host, self.port)
+
 
 #########################################################################
 #####   TEST FUNCTIONS

mercurial/localrepo.py

@@ -3738,8 +3738,10 b' def newreporequirements(ui, createopts):'
 
     if ui.configbool(b'format', b'use-dirstate-tracked-hint'):
         version = ui.configint(b'format', b'use-dirstate-tracked-hint.version')
-        msg = _("ignoring unknown tracked key version: %d\n")
-        hint = _("see `hg help config.format.use-dirstate-tracked-hint-version")
+        msg = _(b"ignoring unknown tracked key version: %d\n")
+        hint = _(
+            b"see `hg help config.format.use-dirstate-tracked-hint-version"
+        )
         if version != 1:
             ui.warn(msg % version, hint=hint)
         else:

mercurial/shelve.py

@@ -278,9 +278,9 b' class shelvedstate:'
         try:
             d[b'originalwctx'] = bin(d[b'originalwctx'])
             d[b'pendingctx'] = bin(d[b'pendingctx'])
-            d[b'parents'] = [bin(h) for h in d[b'parents'].split(b' ')]
+            d[b'parents'] = [bin(h) for h in d[b'parents'].split(b' ') if h]
             d[b'nodestoremove'] = [
-                bin(h) for h in d[b'nodestoremove'].split(b' ')
+                bin(h) for h in d[b'nodestoremove'].split(b' ') if h
             ]
         except (ValueError, KeyError) as err:
             raise error.CorruptedState(stringutil.forcebytestr(err))

mercurial/statprof.py

@@ -236,8 +236,8 b' class CodeSite:'
 
     def getsource(self, length):
         if self.source is None:
-            lineno = self.lineno - 1
             try:
+                lineno = self.lineno - 1  # lineno can be None
                 with open(self.path, b'rb') as fp:
                     for i, line in enumerate(fp):
                         if i == lineno:
@@ -773,7 +773,7 b' def display_hotpath(data, fp, limit=0.05'
             codestring = codepattern % (
                 prefix,
                 b'line'.rjust(spacing_len),
-                site.lineno,
+                site.lineno if site.lineno is not None else -1,
                 b''.ljust(max(0, 4 - len(str(site.lineno)))),
                 site.getsource(30),
             )

mercurial/tags.py

@@ -491,11 +491,14 b' def _getfnodes(ui, repo, nodes):'
     cachefnode = {}
     validated_fnodes = set()
     unknown_entries = set()
+
+    flog = None
     for node in nodes:
         fnode = fnodescache.getfnode(node)
-        flog = repo.file(b'.hgtags')
         if fnode != repo.nullid:
             if fnode not in validated_fnodes:
+                if flog is None:
+                    flog = repo.file(b'.hgtags')
                 if flog.hasnode(fnode):
                     validated_fnodes.add(fnode)
                 else:
@@ -758,8 +761,7 b' class hgtagsfnodescache:'
         if node == self._repo.nullid:
             return node
 
-        ctx = self._repo[node]
-        rev = ctx.rev()
+        rev = self._repo.changelog.rev(node)
 
         self.lookupcount += 1
 

mercurial/upgrade_utils/actions.py

@@ -852,7 +852,7 b' class UpgradeOperation(BaseOperation):'
 
         return False
 
-    def _write_labeled(self, l, label):
+    def _write_labeled(self, l, label: bytes):
         """
         Utility function to aid writing of a list under one label
         """
@@ -867,19 +867,19 b' class UpgradeOperation(BaseOperation):'
         self.ui.write(_(b'requirements\n'))
         self.ui.write(_(b'   preserved: '))
         self._write_labeled(
-            self._preserved_requirements, "upgrade-repo.requirement.preserved"
+            self._preserved_requirements, b"upgrade-repo.requirement.preserved"
         )
         self.ui.write((b'\n'))
         if self._removed_requirements:
             self.ui.write(_(b'   removed: '))
             self._write_labeled(
-                self._removed_requirements, "upgrade-repo.requirement.removed"
+                self._removed_requirements, b"upgrade-repo.requirement.removed"
             )
             self.ui.write((b'\n'))
         if self._added_requirements:
             self.ui.write(_(b'   added: '))
             self._write_labeled(
-                self._added_requirements, "upgrade-repo.requirement.added"
+                self._added_requirements, b"upgrade-repo.requirement.added"
             )
             self.ui.write((b'\n'))
         self.ui.write(b'\n')
@@ -893,7 +893,7 b' class UpgradeOperation(BaseOperation):'
             self.ui.write(_(b'optimisations: '))
             self._write_labeled(
                 [a.name for a in optimisations],
-                "upgrade-repo.optimisation.performed",
+                b"upgrade-repo.optimisation.performed",
             )
             self.ui.write(b'\n\n')
 

mercurial/upgrade_utils/engine.py

@@ -233,18 +233,18 b' def _clonerevlogs('
 
         # This is for the separate progress bars.
         if rl_type & store.FILEFLAGS_CHANGELOG:
-            changelogs[unencoded] = (rl_type, rl)
+            changelogs[unencoded] = rl_type
             crevcount += len(rl)
             csrcsize += datasize
             crawsize += rawsize
         elif rl_type & store.FILEFLAGS_MANIFESTLOG:
-            manifests[unencoded] = (rl_type, rl)
+            manifests[unencoded] = rl_type
             mcount += 1
             mrevcount += len(rl)
             msrcsize += datasize
             mrawsize += rawsize
         elif rl_type & store.FILEFLAGS_FILELOG:
-            filelogs[unencoded] = (rl_type, rl)
+            filelogs[unencoded] = rl_type
             fcount += 1
             frevcount += len(rl)
             fsrcsize += datasize
@@ -289,7 +289,9 b' def _clonerevlogs('
         )
     )
     progress = srcrepo.ui.makeprogress(_(b'file revisions'), total=frevcount)
-    for unencoded, (rl_type, oldrl) in sorted(filelogs.items()):
+    for unencoded, rl_type in sorted(filelogs.items()):
+        oldrl = _revlogfrompath(srcrepo, rl_type, unencoded)
+
         newrl = _perform_clone(
             ui,
             dstrepo,
@@ -329,7 +331,8 b' def _clonerevlogs('
     progress = srcrepo.ui.makeprogress(
         _(b'manifest revisions'), total=mrevcount
     )
-    for unencoded, (rl_type, oldrl) in sorted(manifests.items()):
+    for unencoded, rl_type in sorted(manifests.items()):
+        oldrl = _revlogfrompath(srcrepo, rl_type, unencoded)
         newrl = _perform_clone(
             ui,
             dstrepo,
@@ -368,7 +371,8 b' def _clonerevlogs('
     progress = srcrepo.ui.makeprogress(
         _(b'changelog revisions'), total=crevcount
     )
-    for unencoded, (rl_type, oldrl) in sorted(changelogs.items()):
+    for unencoded, rl_type in sorted(changelogs.items()):
+        oldrl = _revlogfrompath(srcrepo, rl_type, unencoded)
         newrl = _perform_clone(
             ui,
             dstrepo,

mercurial/worker.py

@@ -125,7 +125,14 b' def worthwhile(ui, costperop, nops, thre'
 
 
 def worker(
-    ui, costperarg, func, staticargs, args, hasretval=False, threadsafe=True
+    ui,
+    costperarg,
+    func,
+    staticargs,
+    args,
+    hasretval=False,
+    threadsafe=True,
+    prefork=None,
 ):
     """run a function, possibly in parallel in multiple worker
     processes.
@@ -149,6 +156,10 b' def worker('
     threadsafe - whether work items are thread safe and can be executed using
     a thread-based worker. Should be disabled for CPU heavy tasks that don't
     release the GIL.
+
+    prefork - a parameterless Callable that is invoked prior to forking the
+    process.  fork() is only used on non-Windows platforms, but is also not
+    called on POSIX platforms if the work amount doesn't warrant a worker.
     """
     enabled = ui.configbool(b'worker', b'enabled')
     if enabled and _platformworker is _posixworker and not ismainthread():
@@ -157,11 +168,13 b' def worker('
         enabled = False
 
     if enabled and worthwhile(ui, costperarg, len(args), threadsafe=threadsafe):
-        return _platformworker(ui, func, staticargs, args, hasretval)
+        return _platformworker(
+            ui, func, staticargs, args, hasretval, prefork=prefork
+        )
     return func(*staticargs + (args,))
 
 
-def _posixworker(ui, func, staticargs, args, hasretval):
+def _posixworker(ui, func, staticargs, args, hasretval, prefork=None):
     workers = _numworkers(ui)
     oldhandler = signal.getsignal(signal.SIGINT)
     signal.signal(signal.SIGINT, signal.SIG_IGN)
@@ -207,6 +220,10 b' def _posixworker(ui, func, staticargs, a'
     parentpid = os.getpid()
     pipes = []
     retval = {}
+
+    if prefork:
+        prefork()
+
     for pargs in partition(args, min(workers, len(args))):
         # Every worker gets its own pipe to send results on, so we don't have to
         # implement atomic writes larger than PIPE_BUF. Each forked process has
@@ -316,7 +333,7 b' def _posixexitstatus(code):'
         return -(os.WTERMSIG(code))
 
 
-def _windowsworker(ui, func, staticargs, args, hasretval):
+def _windowsworker(ui, func, staticargs, args, hasretval, prefork=None):
     class Worker(threading.Thread):
         def __init__(
             self, taskqueue, resultqueue, func, staticargs, *args, **kwargs

relnotes/next

@@ -13,12 +13,6 b' Mercurial XXX.'
 
 == Backwards Compatibility Changes ==
 
- * chg worker processes will now correctly load per-repository configuration 
-   when given a both a relative `--repository` path and an alternate working
-   directory via `--cwd`. A side-effect of this change is that these workers
-   will now return an error if hg cannot find the current working directory,
-   even when a different directory is specified via `--cwd`.
-
 == Internal API Changes ==
 
 == Miscellaneous ==

rust/Cargo.lock

@@ -479,6 +479,7 b' dependencies = ['
  "same-file",
  "sha-1 0.10.0",
  "tempfile",
+ "thread_local",
  "twox-hash",
  "zstd",
 ]
@@ -1120,6 +1121,15 b' dependencies = ['
 ]
 
 [[package]]
+name = "thread_local"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5516c27b78311c50bf42c071425c560ac799b11c30b31f87e3081965fe5e0180"
+dependencies = [
+ "once_cell",
+]
+
+[[package]]
 name = "time"
 version = "0.1.44"
 source = "registry+https://github.com/rust-lang/crates.io-index"

rust/hg-core/Cargo.toml

@@ -29,6 +29,7 b' sha-1 = "0.10.0"'
 twox-hash = "1.6.2"
 same-file = "1.0.6"
 tempfile = "3.1.0"
+thread_local = "1.1.4"
 crossbeam-channel = "0.5.0"
 micro-timer = "0.4.0"
 log = "0.4.8"

rust/hg-core/src/dirstate_tree/status.rs

@@ -10,6 +10,7 b' use crate::dirstate_tree::on_disk::Dirst'
 use crate::matchers::get_ignore_function;
 use crate::matchers::Matcher;
 use crate::utils::files::get_bytes_from_os_string;
+use crate::utils::files::get_bytes_from_path;
 use crate::utils::files::get_path_from_bytes;
 use crate::utils::hg_path::HgPath;
 use crate::BadMatch;
@@ -67,7 +68,7 b" pub fn status<'dirstate>("
                     let (ignore_fn, warnings) = get_ignore_function(
                         ignore_files,
                         &root_dir,
-                        &mut |_pattern_bytes| {},
+                        &mut |_source, _pattern_bytes| {},
                     )?;
                     (ignore_fn, warnings, None)
                 }
@@ -76,7 +77,24 b" pub fn status<'dirstate>("
                     let (ignore_fn, warnings) = get_ignore_function(
                         ignore_files,
                         &root_dir,
-                        &mut |pattern_bytes| hasher.update(pattern_bytes),
+                        &mut |source, pattern_bytes| {
+                            // If inside the repo, use the relative version to
+                            // make it deterministic inside tests.
+                            // The performance hit should be negligible.
+                            let source = source
+                                .strip_prefix(&root_dir)
+                                .unwrap_or(source);
+                            let source = get_bytes_from_path(source);
+
+                            let mut subhasher = Sha1::new();
+                            subhasher.update(pattern_bytes);
+                            let patterns_hash = subhasher.finalize();
+
+                            hasher.update(source);
+                            hasher.update(b" ");
+                            hasher.update(patterns_hash);
+                            hasher.update(b"\n");
+                        },
                     )?;
                     let new_hash = *hasher.finalize().as_ref();
                     let changed = new_hash != dmap.ignore_patterns_hash;
@@ -122,8 +140,8 b" pub fn status<'dirstate>("
         ignore_fn,
         outcome: Mutex::new(outcome),
         ignore_patterns_have_changed: patterns_changed,
-        new_cachable_directories: Default::default(),
-        outated_cached_directories: Default::default(),
+        new_cacheable_directories: Default::default(),
+        outdated_cached_directories: Default::default(),
         filesystem_time_at_status_start,
     };
     let is_at_repo_root = true;
@@ -147,12 +165,12 b" pub fn status<'dirstate>("
         is_at_repo_root,
     )?;
     let mut outcome = common.outcome.into_inner().unwrap();
-    let new_cachable = common.new_cachable_directories.into_inner().unwrap();
-    let outdated = common.outated_cached_directories.into_inner().unwrap();
+    let new_cacheable = common.new_cacheable_directories.into_inner().unwrap();
+    let outdated = common.outdated_cached_directories.into_inner().unwrap();
 
     outcome.dirty = common.ignore_patterns_have_changed == Some(true)
         || !outdated.is_empty()
-        || (!new_cachable.is_empty()
+        || (!new_cacheable.is_empty()
             && dmap.dirstate_version == DirstateVersion::V2);
 
     // Remove outdated mtimes before adding new mtimes, in case a given
@@ -160,7 +178,7 b" pub fn status<'dirstate>("
     for path in &outdated {
         dmap.clear_cached_mtime(path)?;
     }
-    for (path, mtime) in &new_cachable {
+    for (path, mtime) in &new_cacheable {
         dmap.set_cached_mtime(path, *mtime)?;
     }
 
@@ -175,9 +193,11 b" struct StatusCommon<'a, 'tree, 'on_disk:"
     matcher: &'a (dyn Matcher + Sync),
     ignore_fn: IgnoreFnType<'a>,
     outcome: Mutex<DirstateStatus<'on_disk>>,
-    new_cachable_directories:
+    /// New timestamps of directories to be used for caching their readdirs
+    new_cacheable_directories:
         Mutex<Vec<(Cow<'on_disk, HgPath>, TruncatedTimestamp)>>,
-    outated_cached_directories: Mutex<Vec<Cow<'on_disk, HgPath>>>,
+    /// Used to invalidate the readdir cache of directories
+    outdated_cached_directories: Mutex<Vec<Cow<'on_disk, HgPath>>>,
 
     /// Whether ignore files like `.hgignore` have changed since the previous
     /// time a `status()` call wrote their hash to the dirstate. `None` means
@@ -305,17 +325,18 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
     fn check_for_outdated_directory_cache(
         &self,
         dirstate_node: &NodeRef<'tree, 'on_disk>,
-    ) -> Result<(), DirstateV2ParseError> {
+    ) -> Result<bool, DirstateV2ParseError> {
         if self.ignore_patterns_have_changed == Some(true)
             && dirstate_node.cached_directory_mtime()?.is_some()
         {
-            self.outated_cached_directories.lock().unwrap().push(
+            self.outdated_cached_directories.lock().unwrap().push(
                 dirstate_node
                     .full_path_borrowed(self.dmap.on_disk)?
                     .detach_from_tree(),
-            )
+            );
+            return Ok(true);
         }
-        Ok(())
+        Ok(false)
     }
 
     /// If this returns true, we can get accurate results by only using
@@ -487,6 +508,7 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
         dirstate_node: NodeRef<'tree, 'on_disk>,
         has_ignored_ancestor: &'ancestor HasIgnoredAncestor<'ancestor>,
     ) -> Result<(), DirstateV2ParseError> {
+        let outdated_dircache =
         self.check_for_outdated_directory_cache(&dirstate_node)?;
         let hg_path = &dirstate_node.full_path_borrowed(self.dmap.on_disk)?;
         let file_or_symlink = fs_entry.is_file() || fs_entry.is_symlink();
@@ -522,6 +544,7 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
                 children_all_have_dirstate_node_or_are_ignored,
                 fs_entry,
                 dirstate_node,
+                outdated_dircache,
             )?
         } else {
             if file_or_symlink && self.matcher.matches(&hg_path) {
@@ -561,11 +584,17 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
         Ok(())
     }
 
+    /// Save directory mtime if applicable.
+    ///
+    /// `outdated_directory_cache` is `true` if we've just invalidated the
+    /// cache for this directory in `check_for_outdated_directory_cache`,
+    /// which forces the update.
     fn maybe_save_directory_mtime(
         &self,
         children_all_have_dirstate_node_or_are_ignored: bool,
         directory_entry: &DirEntry,
         dirstate_node: NodeRef<'tree, 'on_disk>,
+        outdated_directory_cache: bool,
     ) -> Result<(), DirstateV2ParseError> {
         if !children_all_have_dirstate_node_or_are_ignored {
             return Ok(());
@@ -635,9 +664,10 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
         // We deem this scenario (unlike the previous one) to be
         // unlikely enough in practice.
 
-        let is_up_to_date =
-            if let Some(cached) = dirstate_node.cached_directory_mtime()? {
-                cached.likely_equal(directory_mtime)
+        let is_up_to_date = if let Some(cached) =
+            dirstate_node.cached_directory_mtime()?
+        {
+            !outdated_directory_cache && cached.likely_equal(directory_mtime)
             } else {
                 false
             };
@@ -645,7 +675,7 b" impl<'a, 'tree, 'on_disk> StatusCommon<'"
             let hg_path = dirstate_node
                 .full_path_borrowed(self.dmap.on_disk)?
                 .detach_from_tree();
-            self.new_cachable_directories
+            self.new_cacheable_directories
                 .lock()
                 .unwrap()
                 .push((hg_path, directory_mtime))

rust/hg-core/src/filepatterns.rs

@@ -412,11 +412,11 b' pub fn parse_pattern_file_contents('
 pub fn read_pattern_file(
     file_path: &Path,
     warn: bool,
-    inspect_pattern_bytes: &mut impl FnMut(&[u8]),
+    inspect_pattern_bytes: &mut impl FnMut(&Path, &[u8]),
 ) -> Result<(Vec<IgnorePattern>, Vec<PatternFileWarning>), PatternError> {
     match std::fs::read(file_path) {
         Ok(contents) => {
-            inspect_pattern_bytes(&contents);
+            inspect_pattern_bytes(file_path, &contents);
             parse_pattern_file_contents(&contents, file_path, None, warn)
         }
         Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok((
@@ -455,7 +455,7 b' pub type PatternResult<T> = Result<T, Pa'
 pub fn get_patterns_from_file(
     pattern_file: &Path,
     root_dir: &Path,
-    inspect_pattern_bytes: &mut impl FnMut(&[u8]),
+    inspect_pattern_bytes: &mut impl FnMut(&Path, &[u8]),
 ) -> PatternResult<(Vec<IgnorePattern>, Vec<PatternFileWarning>)> {
     let (patterns, mut warnings) =
         read_pattern_file(pattern_file, true, inspect_pattern_bytes)?;

rust/hg-core/src/matchers.rs

@@ -573,6 +573,39 b' impl DifferenceMatcher {'
     }
 }
 
+/// Wraps [`regex::bytes::Regex`] to improve performance in multithreaded
+/// contexts.
+///
+/// The `status` algorithm makes heavy use of threads, and calling `is_match`
+/// from many threads at once is prone to contention, probably within the
+/// scratch space needed as the regex DFA is built lazily.
+///
+/// We are in the process of raising the issue upstream, but for now
+/// the workaround used here is to store the `Regex` in a lazily populated
+/// thread-local variable, sharing the initial read-only compilation, but
+/// not the lazy dfa scratch space mentioned above.
+///
+/// This reduces the contention observed with 16+ threads, but does not
+/// completely remove it. Hopefully this can be addressed upstream.
+struct RegexMatcher {
+    /// Compiled at the start of the status algorithm, used as a base for
+    /// cloning in each thread-local `self.local`, thus sharing the expensive
+    /// first compilation.
+    base: regex::bytes::Regex,
+    /// Thread-local variable that holds the `Regex` that is actually queried
+    /// from each thread.
+    local: thread_local::ThreadLocal<regex::bytes::Regex>,
+}
+
+impl RegexMatcher {
+    /// Returns whether the path matches the stored `Regex`.
+    pub fn is_match(&self, path: &HgPath) -> bool {
+        self.local
+            .get_or(|| self.base.clone())
+            .is_match(path.as_bytes())
+    }
+}
+
 /// Returns a function that matches an `HgPath` against the given regex
 /// pattern.
 ///
@@ -580,9 +613,7 b' impl DifferenceMatcher {'
 /// underlying engine (the `regex` crate), for instance anything with
 /// back-references.
 #[timed]
-fn re_matcher(
-    pattern: &[u8],
-) -> PatternResult<impl Fn(&HgPath) -> bool + Sync> {
+fn re_matcher(pattern: &[u8]) -> PatternResult<RegexMatcher> {
     use std::io::Write;
 
     // The `regex` crate adds `.*` to the start and end of expressions if there
@@ -611,7 +642,10 b' fn re_matcher('
         .build()
         .map_err(|e| PatternError::UnsupportedSyntax(e.to_string()))?;
 
-    Ok(move |path: &HgPath| re.is_match(path.as_bytes()))
+    Ok(RegexMatcher {
+        base: re,
+        local: Default::default(),
+    })
 }
 
 /// Returns the regex pattern and a function that matches an `HgPath` against
@@ -638,7 +672,7 b" fn build_regex_match<'a, 'b>("
     let func = if !(regexps.is_empty()) {
         let matcher = re_matcher(&full_regex)?;
         let func = move |filename: &HgPath| {
-            exact_set.contains(filename) || matcher(filename)
+            exact_set.contains(filename) || matcher.is_match(filename)
         };
         Box::new(func) as IgnoreFnType
     } else {
@@ -838,7 +872,7 b" fn build_match<'a, 'b>("
 pub fn get_ignore_matcher<'a>(
     mut all_pattern_files: Vec<PathBuf>,
     root_dir: &Path,
-    inspect_pattern_bytes: &mut impl FnMut(&[u8]),
+    inspect_pattern_bytes: &mut impl FnMut(&Path, &[u8]),
 ) -> PatternResult<(IncludeMatcher<'a>, Vec<PatternFileWarning>)> {
     let mut all_patterns = vec![];
     let mut all_warnings = vec![];
@@ -871,7 +905,7 b" pub fn get_ignore_matcher<'a>("
 pub fn get_ignore_function<'a>(
     all_pattern_files: Vec<PathBuf>,
     root_dir: &Path,
-    inspect_pattern_bytes: &mut impl FnMut(&[u8]),
+    inspect_pattern_bytes: &mut impl FnMut(&Path, &[u8]),
 ) -> PatternResult<(IgnoreFnType<'a>, Vec<PatternFileWarning>)> {
     let res =
         get_ignore_matcher(all_pattern_files, root_dir, inspect_pattern_bytes);

rust/hg-core/src/revlog/revlog.rs

@@ -447,6 +447,11 b" impl<'a> RevlogEntry<'a> {"
         ) {
             Ok(data)
         } else {
+            if (self.flags & REVISION_FLAG_ELLIPSIS) != 0 {
+                return Err(HgError::unsupported(
+                    "ellipsis revisions are not supported by rhg",
+                ));
+            }
             Err(corrupted(format!(
                 "hash check failed for revision {}",
                 self.rev

rust/rhg/README.md

@@ -19,35 +19,9 b' The executable can then be found at `rus'
 
 `rhg` reads Mercurial configuration from the usual sources:
 the user’s `~/.hgrc`, a repository’s `.hg/hgrc`, command line `--config`, etc.
-It has some specific configuration in the `[rhg]` section:
-
-* `on-unsupported` governs the behavior of `rhg` when it encounters something
-  that it does not support but “full” `hg` possibly does.
-  This can be in configuration, on the command line, or in a repository.
-
-  - `abort`, the default value, makes `rhg` print a message to stderr
-    to explain what is not supported, then terminate with a 252 exit code.
-  - `abort-silent` makes it terminate with the same exit code,
-    but without printing anything.
-  - `fallback` makes it silently call a (presumably Python-based) `hg`
-    subprocess with the same command-line parameters.
-    The `rhg.fallback-executable` configuration must be set.
+It has some specific configuration in the `[rhg]` section.
 
-* `fallback-executable`: path to the executable to run in a sub-process
-  when falling back to a Python implementation of Mercurial.
-
-* `allowed-extensions`: a list of extension names that `rhg` can ignore.
-
-  Mercurial extensions can modify the behavior of existing `hg` sub-commands,
-  including those that `rhg` otherwise supports.
-  Because it cannot load Python extensions, finding them
-  enabled in configuration is considered “unsupported” (see above).
-  A few exceptions are made for extensions that `rhg` does know about,
-  with the Rust implementation duplicating their behavior.
-
-  This configuration makes additional exceptions: `rhg` will proceed even if
-  those extensions are enabled.
-
+See `hg help config.rhg` for details.
 
 ## Installation and configuration example
 

rust/rhg/src/commands/debugignorerhg.rs

@@ -25,7 +25,7 b' pub fn run(invocation: &crate::CliInvoca'
     let (ignore_matcher, warnings) = get_ignore_matcher(
         vec![ignore_file],
         &repo.working_directory_path().to_owned(),
-        &mut |_pattern_bytes| (),
+        &mut |_source, _pattern_bytes| (),
     )
     .map_err(|e| StatusError::from(e))?;
 

rust/rhg/src/error.rs

@@ -221,7 +221,12 b' impl From<(RevlogError, &str)> for Comma'
 
 impl From<StatusError> for CommandError {
     fn from(error: StatusError) -> Self {
-        CommandError::abort(format!("{}", error))
+        match error {
+            StatusError::Pattern(_) => {
+                CommandError::unsupported(format!("{}", error))
+            }
+            _ => CommandError::abort(format!("{}", error)),
+        }
     }
 }
 

rust/rhg/src/main.rs

@@ -301,7 +301,7 b' fn rhg_main(argv: Vec<OsString>) -> ! {'
         }
     };
 
-    let exit =
+    let simple_exit =
         |ui: &Ui, config: &Config, result: Result<(), CommandError>| -> ! {
             exit(
                 &argv,
@@ -317,7 +317,7 b' fn rhg_main(argv: Vec<OsString>) -> ! {'
             )
         };
     let early_exit = |config: &Config, error: CommandError| -> ! {
-        exit(&Ui::new_infallible(config), &config, Err(error))
+        simple_exit(&Ui::new_infallible(config), &config, Err(error))
     };
     let repo_result = match Repo::find(&non_repo_config, repo_path.to_owned())
     {
@@ -348,6 +348,24 b' fn rhg_main(argv: Vec<OsString>) -> ! {'
     let config = config_cow.as_ref();
     let ui = Ui::new(&config)
         .unwrap_or_else(|error| early_exit(&config, error.into()));
+
+    if let Ok(true) = config.get_bool(b"rhg", b"fallback-immediately") {
+        exit(
+            &argv,
+            &initial_current_dir,
+            &ui,
+            OnUnsupported::Fallback {
+                executable: config
+                    .get(b"rhg", b"fallback-executable")
+                    .map(ToOwned::to_owned),
+            },
+            Err(CommandError::unsupported(
+                "`rhg.fallback-immediately is true`",
+            )),
+            false,
+        )
+    }
+
     let result = main_with_result(
         argv.iter().map(|s| s.to_owned()).collect(),
         &process_start_time,
@@ -355,7 +373,7 b' fn rhg_main(argv: Vec<OsString>) -> ! {'
         repo_result.as_ref(),
         config,
     );
-    exit(&ui, &config, result)
+    simple_exit(&ui, &config, result)
 }
 
 fn main() -> ! {

tests/f

@@ -63,7 +63,16 b' def visit(opts, filenames, outfile):'
         if isfile:
             if opts.type:
                 facts.append(b'file')
-            if any((opts.hexdump, opts.dump, opts.md5, opts.sha1, opts.sha256)):
+            needs_reading = (
+                opts.hexdump,
+                opts.dump,
+                opts.md5,
+                opts.sha1,
+                opts.raw_sha1,
+                opts.sha256,
+            )
+
+            if any(needs_reading):
                 with open(f, 'rb') as fobj:
                     content = fobj.read()
         elif islink:
@@ -101,6 +110,9 b' def visit(opts, filenames, outfile):'
         if opts.md5 and content is not None:
             h = hashlib.md5(content)
             facts.append(b'md5=%s' % binascii.hexlify(h.digest())[: opts.bytes])
+        if opts.raw_sha1 and content is not None:
+            h = hashlib.sha1(content)
+            facts.append(b'raw-sha1=%s' % h.digest()[: opts.bytes])
         if opts.sha1 and content is not None:
             h = hashlib.sha1(content)
             facts.append(
@@ -186,6 +198,12 b' if __name__ == "__main__":'
     )
     parser.add_option(
         "",
+        "--raw-sha1",
+        action="store_true",
+        help="show raw bytes of the sha1 hash of the content",
+    )
+    parser.add_option(
+        "",
         "--sha256",
         action="store_true",
         help="show sha256 hash of the content",

tests/test-dirstate.t

@@ -245,3 +245,17 b' We should make sure all of it (docket + '
 
   $ hg status
   A foo
+  $ cd ..
+
+Check dirstate ordering
+(e.g. `src/dirstate/` and `src/dirstate.rs` shouldn't cause issues)
+
+  $ hg init repro
+  $ cd repro
+  $ mkdir src
+  $ mkdir src/dirstate
+  $ touch src/dirstate/file1 src/dirstate/file2 src/dirstate.rs
+  $ touch file1 file2
+  $ hg commit -Aqm1
+  $ hg st
+  $ cd ..

tests/test-hgignore.t

@@ -59,18 +59,24 b' Should display baz only:'
   ? syntax
 
   $ echo "*.o" > .hgignore
-#if no-rhg
   $ hg status
   abort: $TESTTMP/ignorerepo/.hgignore: invalid pattern (relre): *.o (glob)
   [255]
-#endif
-#if rhg
+
+  $ echo 're:^(?!a).*\.o$' > .hgignore
   $ hg status
-  Unsupported syntax regex parse error:
-      ^(?:*.o)
-          ^
-  error: repetition operator missing expression
-  [255]
+  A dir/b.o
+  ? .hgignore
+  ? a.c
+  ? a.o
+  ? syntax
+#if rhg
+  $ hg status --config rhg.on-unsupported=abort
+  unsupported feature: Unsupported syntax regex parse error:
+      ^(?:^(?!a).*\.o$)
+           ^^^
+  error: look-around, including look-ahead and look-behind, is not supported
+  [252]
 #endif
 
 Ensure given files are relative to cwd
@@ -415,17 +421,51 b' Windows paths are accepted on input'
 Check the hash of ignore patterns written in the dirstate
 This is an optimization that is only relevant when using the Rust extensions
 
+  $ cat_filename_and_hash () {
+  >     for i in "$@"; do
+  >         printf "$i "
+  >         cat "$i" | "$TESTDIR"/f --raw-sha1 | sed 's/^raw-sha1=//'
+  >     done
+  > }
   $ hg status > /dev/null
-  $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1
-  sha1=6e315b60f15fb5dfa02be00f3e2c8f923051f5ff
+  $ cat_filename_and_hash .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1
+  sha1=c0beb296395d48ced8e14f39009c4ea6e409bfe6
   $ hg debugstate --docket | grep ignore
-  ignore pattern hash: 6e315b60f15fb5dfa02be00f3e2c8f923051f5ff
+  ignore pattern hash: c0beb296395d48ced8e14f39009c4ea6e409bfe6
 
   $ echo rel > .hg/testhgignorerel
   $ hg status > /dev/null
-  $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1
-  sha1=dea19cc7119213f24b6b582a4bae7b0cb063e34e
+  $ cat_filename_and_hash .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1
+  sha1=b8e63d3428ec38abc68baa27631516d5ec46b7fa
   $ hg debugstate --docket | grep ignore
-  ignore pattern hash: dea19cc7119213f24b6b582a4bae7b0cb063e34e
+  ignore pattern hash: b8e63d3428ec38abc68baa27631516d5ec46b7fa
+  $ cd ..
+
+Check that the hash depends on the source of the hgignore patterns
+(otherwise the context is lost and things like subinclude are cached improperly)
+
+  $ hg init ignore-collision
+  $ cd ignore-collision
+  $ echo > .hg/testhgignorerel
+
+  $ mkdir dir1/ dir1/subdir
+  $ touch dir1/subdir/f dir1/subdir/ignored1
+  $ echo 'ignored1' > dir1/.hgignore
+
+  $ mkdir dir2 dir2/subdir
+  $ touch dir2/subdir/f dir2/subdir/ignored2
+  $ echo 'ignored2' > dir2/.hgignore
+  $ echo 'subinclude:dir2/.hgignore' >> .hgignore
+  $ echo 'subinclude:dir1/.hgignore' >> .hgignore
+
+  $ hg commit -Aqm_
+
+  $ > dir1/.hgignore
+  $ echo 'ignored' > dir2/.hgignore
+  $ echo 'ignored1' >> dir2/.hgignore
+  $ hg status
+  M dir1/.hgignore
+  M dir2/.hgignore
+  ? dir1/subdir/ignored1
 
 #endif

tests/test-rhg.t

@@ -168,6 +168,10 b' Fallback to Python'
   $ rhg cat original --exclude="*.rs"
   original content
 
+Check that `fallback-immediately` overrides `$NO_FALLBACK`
+  $ $NO_FALLBACK rhg cat original --exclude="*.rs" --config rhg.fallback-immediately=1
+  original content
+
   $ (unset RHG_FALLBACK_EXECUTABLE; rhg cat original --exclude="*.rs")
   abort: 'rhg.on-unsupported=fallback' without 'rhg.fallback-executable' set.
   [255]

tests/test-status.t

@@ -944,7 +944,7 b' It is still not set when there are unkno'
   $ hg debugdirstate --all --no-dates | grep '^ '
       0         -1 unset               subdir
 
-Now the directory is eligible for caching, so its mtime is save in the dirstate
+Now the directory is eligible for caching, so its mtime is saved in the dirstate
 
   $ rm subdir/unknown
   $ sleep 0.1 # ensure the kernel’s internal clock for mtimes has ticked
@@ -976,4 +976,27 b' Removing a node from the dirstate resets'
   $ hg status
   ? subdir/a
 
+Changing the hgignore rules makes us recompute the status (and rewrite the dirstate).
+
+  $ rm subdir/a
+  $ mkdir another-subdir
+  $ touch another-subdir/something-else
+
+  $ cat > "$TESTDIR"/extra-hgignore <<EOF
+  > something-else
+  > EOF
+
+  $ hg status --config ui.ignore.global="$TESTDIR"/extra-hgignore
+  $ hg debugdirstate --all --no-dates | grep '^ '
+      0         -1 set                 subdir
+
+  $ hg status
+  ? another-subdir/something-else
+
+One invocation of status is enough to populate the cache even if it's invalidated
+in the same run.
+
+  $ hg debugdirstate --all --no-dates | grep '^ '
+      0         -1 set                 subdir
+
 #endif

tests/test-tools.t

@@ -13,6 +13,7 b' Tests of the file helper tool'
                           check if file is newer (or same)
     -r, --recurse         recurse into directories
     -S, --sha1            show sha1 hash of the content
+    --raw-sha1            show raw bytes of the sha1 hash of the content
     --sha256              show sha256 hash of the content
     -M, --md5             show md5 hash of the content
     -D, --dump            dump file content