##// END OF EJS Templates
mmap: populate the mapping by default...
mmap: populate the mapping by default Without pre-population, accessing all data through a mmap can result in many pagefault, reducing performance significantly. If the mmap is prepopulated, the performance can no longer get slower than a full read. (See benchmark number below) In some cases were very few data is read, prepopulating can be overkill and slower than populating on access (through page fault). So that behavior can be controlled when the caller can pre-determine the best behavior. (See benchmark number below) In addition, testing with populating in a secondary thread yield great result combining the best of each approach. This might be implemented in later changesets. In all cases, using mmap has a great effect on memory usage when many processes run in parallel on the same machine. ### Benchmarks # What did I run A couple of month back I ran a large benchmark campaign to assess the impact of various approach for using mmap with the revlog (and other files), it highlighted a few benchmarks that capture the impact of the changes well. So to validate this change I checked the following: - log command displaying various revisions (read the changelog index) - log command displaying the patch of listed revisions (read the changelog index, the manifest index and a few files indexes) - unbundling a few revisions (read and write changelog, manifest and few files indexes, and walk the graph to update some cache) - pushing a few revisions (read and write changelog, manifest and few files indexes, walk the graph to update some cache, performs various accesses locally and remotely during discovery) Benchmarks were run using the default module policy (c+py) and the rust one. No significant difference were found between the two implementation, so we will present result using the default policy (unless otherwise specified). I ran them on a few repositories : - mercurial: a "public changeset only" copy of mercurial from 2018-08-01 using zstd compression and sparse-revlog - pypy: a copy of pypy from 2018-08-01 using zstd compression and sparse-revlog - netbeans: a copy of netbeans from 2018-08-01 using zstd compression and sparse-revlog - mozilla-try: a copy of mozilla-try from 2019-02-18 using zstd compression and sparse-revlog - mozilla-try persistent-nodemap: Same as the above but with a persistent nodemap. Used for the log --patch benchmark only # Results For the smaller repositories (mercurial, pypy), the impact of mmap is almost imperceptible, other cost dominating the operation. The impact of prepopulating is undiscernible in the benchmark we ran. For larger repositories the benchmark support explanation given above: On netbeans, the log can be about 1% faster without repopulation (for a difference < 100ms) but unbundle becomes a bit slower, even when small. ### data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog # benchmark.name = hg.command.unbundle # benchmark.variants.issue6528 = disabled # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle # benchmark.variants.verbosity = quiet with-populate: 0.240157 no-populate: 0.265087 (+10.38%, +0.02) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.459518 no-populate: 1.481290 (+1.49%, +0.02) ## benchmark.name = hg.command.push # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev with-populate: 0.771919 no-populate: 0.792025 (+2.60%, +0.02) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.459518 no-populate: 1.481290 (+1.49%, +0.02) For mozilla-try, the "slow down" from pre-populate for small `hg log` is more visible, but still small in absolute time. (using rust value for the persistent nodemap value to be relevant). ### data-env-vars.name = mozilla-try-2019-02-18-ds2-pnm # benchmark.name = hg.command.log # bin-env-vars.hg.flavor = rust # benchmark.variants.patch = yes # benchmark.variants.limit-rev = 1 with-populate: 0.237813 no-populate: 0.229452 (-3.52%, -0.01) # benchmark.variants.limit-rev = 10 # benchmark.variants.patch = yes with-populate: 1.213578 no-populate: 1.205189 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.variants.limit-rev = 1000 # benchmark.variants.patch = no # benchmark.variants.rev = tip with-populate: 0.198607 no-populate: 0.195038 (-1.80%, -0.00) However pre-populating provide a significant boost on more complex operations like unbundle or push: ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = hg.command.push # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev with-populate: 4.798632 no-populate: 4.953295 (+3.22%, +0.15) # benchmark.variants.revs = any-100-extra-rev with-populate: 4.903618 no-populate: 5.014963 (+2.27%, +0.11) ## benchmark.name = hg.command.unbundle # benchmark.variants.revs = any-1-extra-rev with-populate: 1.423411 no-populate: 1.585365 (+11.38%, +0.16) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.537909 no-populate: 1.688489 (+9.79%, +0.15)

File last commit:

r45482:b77d5b56 stable
r52574:522b4d72 default
Show More
hgignore.txt
103 lines | 3.3 KiB | text/plain | TextLexer
Synopsis
========
The Mercurial system uses a file called ``.hgignore`` in the root
directory of a repository to control its behavior when it searches
for files that it is not currently tracking.
Description
===========
The working directory of a Mercurial repository will often contain
files that should not be tracked by Mercurial. These include backup
files created by editors and build products created by compilers.
These files can be ignored by listing them in a ``.hgignore`` file in
the root of the working directory. The ``.hgignore`` file must be
created manually. It is typically put under version control, so that
the settings will propagate to other repositories with push and pull.
An untracked file is ignored if its path relative to the repository
root directory, or any prefix path of that path, is matched against
any pattern in ``.hgignore``.
For example, say we have an untracked file, ``file.c``, at
``a/b/file.c`` inside our repository. Mercurial will ignore ``file.c``
if any pattern in ``.hgignore`` matches ``a/b/file.c``, ``a/b`` or ``a``.
In addition, a Mercurial configuration file can reference a set of
per-user or global ignore files. See the ``ignore`` configuration
key on the ``[ui]`` section of :hg:`help config` for details of how to
configure these files.
To control Mercurial's handling of files that it manages, many
commands support the ``-I`` and ``-X`` options; see
:hg:`help <command>` and :hg:`help patterns` for details.
Files that are already tracked are not affected by .hgignore, even
if they appear in .hgignore. An untracked file X can be explicitly
added with :hg:`add X`, even if X would be excluded by a pattern
in .hgignore.
Syntax
======
An ignore file is a plain text file consisting of a list of patterns,
with one pattern per line. Empty lines are skipped. The ``#``
character is treated as a comment character, and the ``\`` character
is treated as an escape character.
Mercurial supports several pattern syntaxes. The default syntax used
is Python/Perl-style regular expressions.
To change the syntax used, use a line of the following form::
syntax: NAME
where ``NAME`` is one of the following:
``regexp``
Regular expression, Python/Perl syntax.
``glob``
Shell-style glob.
``rootglob``
A variant of ``glob`` that is rooted (see below).
The chosen syntax stays in effect when parsing all patterns that
follow, until another syntax is selected.
Neither ``glob`` nor regexp patterns are rooted. A glob-syntax
pattern of the form ``*.c`` will match a file ending in ``.c`` in any
directory, and a regexp pattern of the form ``\.c$`` will do the
same. To root a regexp pattern, start it with ``^``. To get the same
effect with glob-syntax, you have to use ``rootglob``.
Subdirectories can have their own .hgignore settings by adding
``subinclude:path/to/subdir/.hgignore`` to the root ``.hgignore``. See
:hg:`help patterns` for details on ``subinclude:`` and ``include:``.
.. note::
Patterns specified in other than ``.hgignore`` are always rooted.
Please see :hg:`help patterns` for details.
Example
=======
Here is an example ignore file. ::
# use glob syntax.
syntax: glob
*.elc
*.pyc
*~
# switch to regexp syntax.
syntax: regexp
^\.pc/
Debugging
=========
Use the ``debugignore`` command to see if and why a file is ignored, or to
see the combined ignore pattern. See :hg:`help debugignore` for details.