##// END OF EJS Templates
mmap: populate the mapping by default...
mmap: populate the mapping by default Without pre-population, accessing all data through a mmap can result in many pagefault, reducing performance significantly. If the mmap is prepopulated, the performance can no longer get slower than a full read. (See benchmark number below) In some cases were very few data is read, prepopulating can be overkill and slower than populating on access (through page fault). So that behavior can be controlled when the caller can pre-determine the best behavior. (See benchmark number below) In addition, testing with populating in a secondary thread yield great result combining the best of each approach. This might be implemented in later changesets. In all cases, using mmap has a great effect on memory usage when many processes run in parallel on the same machine. ### Benchmarks # What did I run A couple of month back I ran a large benchmark campaign to assess the impact of various approach for using mmap with the revlog (and other files), it highlighted a few benchmarks that capture the impact of the changes well. So to validate this change I checked the following: - log command displaying various revisions (read the changelog index) - log command displaying the patch of listed revisions (read the changelog index, the manifest index and a few files indexes) - unbundling a few revisions (read and write changelog, manifest and few files indexes, and walk the graph to update some cache) - pushing a few revisions (read and write changelog, manifest and few files indexes, walk the graph to update some cache, performs various accesses locally and remotely during discovery) Benchmarks were run using the default module policy (c+py) and the rust one. No significant difference were found between the two implementation, so we will present result using the default policy (unless otherwise specified). I ran them on a few repositories : - mercurial: a "public changeset only" copy of mercurial from 2018-08-01 using zstd compression and sparse-revlog - pypy: a copy of pypy from 2018-08-01 using zstd compression and sparse-revlog - netbeans: a copy of netbeans from 2018-08-01 using zstd compression and sparse-revlog - mozilla-try: a copy of mozilla-try from 2019-02-18 using zstd compression and sparse-revlog - mozilla-try persistent-nodemap: Same as the above but with a persistent nodemap. Used for the log --patch benchmark only # Results For the smaller repositories (mercurial, pypy), the impact of mmap is almost imperceptible, other cost dominating the operation. The impact of prepopulating is undiscernible in the benchmark we ran. For larger repositories the benchmark support explanation given above: On netbeans, the log can be about 1% faster without repopulation (for a difference < 100ms) but unbundle becomes a bit slower, even when small. ### data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog # benchmark.name = hg.command.unbundle # benchmark.variants.issue6528 = disabled # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle # benchmark.variants.verbosity = quiet with-populate: 0.240157 no-populate: 0.265087 (+10.38%, +0.02) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.459518 no-populate: 1.481290 (+1.49%, +0.02) ## benchmark.name = hg.command.push # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev with-populate: 0.771919 no-populate: 0.792025 (+2.60%, +0.02) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.459518 no-populate: 1.481290 (+1.49%, +0.02) For mozilla-try, the "slow down" from pre-populate for small `hg log` is more visible, but still small in absolute time. (using rust value for the persistent nodemap value to be relevant). ### data-env-vars.name = mozilla-try-2019-02-18-ds2-pnm # benchmark.name = hg.command.log # bin-env-vars.hg.flavor = rust # benchmark.variants.patch = yes # benchmark.variants.limit-rev = 1 with-populate: 0.237813 no-populate: 0.229452 (-3.52%, -0.01) # benchmark.variants.limit-rev = 10 # benchmark.variants.patch = yes with-populate: 1.213578 no-populate: 1.205189 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.variants.limit-rev = 1000 # benchmark.variants.patch = no # benchmark.variants.rev = tip with-populate: 0.198607 no-populate: 0.195038 (-1.80%, -0.00) However pre-populating provide a significant boost on more complex operations like unbundle or push: ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = hg.command.push # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev with-populate: 4.798632 no-populate: 4.953295 (+3.22%, +0.15) # benchmark.variants.revs = any-100-extra-rev with-populate: 4.903618 no-populate: 5.014963 (+2.27%, +0.11) ## benchmark.name = hg.command.unbundle # benchmark.variants.revs = any-1-extra-rev with-populate: 1.423411 no-populate: 1.585365 (+11.38%, +0.16) # benchmark.variants.revs = any-100-extra-rev with-populate: 1.537909 no-populate: 1.688489 (+9.79%, +0.15)

File last commit:

r51588:1c31b343 default
r52574:522b4d72 default
Show More
patterns.txt
93 lines | 4.1 KiB | text/plain | TextLexer
Matt Harbison
help: create packages for the help text...
r44031 Mercurial accepts several notations for identifying one or more files
at a time.
Jordi Gutiérrez Hermoso
help: properly document the default pattern...
r49352 By default, Mercurial treats filenames verbatim without pattern
matching, relative to the current working directory. Note that your
system shell might perform pattern matching of its own before passing
filenames into Mercurial.
Matt Harbison
help: create packages for the help text...
r44031
Alternate pattern notations must be specified explicitly.
.. note::
Patterns specified in ``.hgignore`` are not rooted.
Please see :hg:`help hgignore` for details.
To use a plain path name without any pattern matching, start it with
``path:``. These path names must completely match starting at the
current repository root, and when the path points to a directory, it is matched
recursively. To match all files in a directory non-recursively (not including
any files in subdirectories), ``rootfilesin:`` can be used, specifying an
Raphaël Gomès
match: add `filepath:` pattern to match an exact filepath relative to the root...
r51588 absolute path (relative to the repository root). To match a single file exactly,
relative to the repository root, you can use ``filepath:``.
Matt Harbison
help: create packages for the help text...
r44031
To use an extended glob, start a name with ``glob:``. Globs are rooted
at the current directory; a glob such as ``*.c`` will only match files
in the current directory ending with ``.c``. ``rootglob:`` can be used
instead of ``glob:`` for a glob that is rooted at the root of the
repository.
The supported glob syntax extensions are ``**`` to match any string
across path separators and ``{a,b}`` to mean "a or b".
To use a Perl/Python regular expression, start a name with ``re:``.
Regexp pattern matching is anchored at the root of the repository.
To read name patterns from a file, use ``listfile:`` or ``listfile0:``.
The latter expects null delimited patterns while the former expects line
feeds. Each string read from the file is itself treated as a file
pattern.
To read a set of patterns from a file, use ``include:`` or ``subinclude:``.
``include:`` will use all the patterns from the given file and treat them as if
they had been passed in manually. ``subinclude:`` will only apply the patterns
against files that are under the subinclude file's directory. See :hg:`help
hgignore` for details on the format of these files.
All patterns, except for ``glob:`` specified in command line (not for
``-I`` or ``-X`` options), can match also against directories: files
under matched directories are treated as matched.
For ``-I`` and ``-X`` options, ``glob:`` will match directories recursively.
Plain examples::
Raphaël Gomès
match: add `filepath:` pattern to match an exact filepath relative to the root...
r51588 path:foo/bar a name bar in a directory named foo in the root
of the repository
path:some/path a file or directory named "some/path"
filepath:some/path/to/a/file exactly a single file named
"some/path/to/a/file", relative to the root
of the repository
rootfilesin:foo/bar the files in a directory called foo/bar, but
not any files in its subdirectories and not
a file bar in directory foo
Matt Harbison
help: create packages for the help text...
r44031
Glob examples::
glob:*.c any name ending in ".c" in the current directory
*.c any name ending in ".c" in the current directory
**.c any name ending in ".c" in any subdirectory of the
current directory including itself.
foo/* any file in directory foo
foo/** any file in directory foo plus all its subdirectories,
recursively
foo/*.c any name ending in ".c" in the directory foo
foo/**.c any name ending in ".c" in any subdirectory of foo
including itself.
rootglob:*.c any name ending in ".c" in the root of the repository
Regexp examples::
re:.*\.c$ any name ending in ".c", anywhere in the repository
File examples::
listfile:list.txt read list from list.txt with one file pattern per line
listfile0:list.txt read list from list.txt with null byte delimiters
See also :hg:`help filesets`.
Include examples::
include:path/to/mypatternfile reads patterns to be applied to all paths
subinclude:path/to/subignorefile reads patterns specifically for paths in the
subdirectory