##// END OF EJS Templates
manifest: persist the manifestfulltext cache...
manifest: persist the manifestfulltext cache Reconstructing the manifest from the revlog takes time, so much so that there already is a LRU cache to avoid having to load a manifest multiple times. This patch persists that LRU cache in the .hg/cache directory, so we can re-use this cache across hg commands. Commit benchmark (run on Macos 10.13 on a 2017-model Macbook Pro with Core i7 2.9GHz and flash drive), testing without and with patch run 5 times, baseline is r2a227782e754: * committing to an existing file, against the mozilla-central repository. Baseline real time average 1.9692, with patch 1.3786. A new debugcommand "hg debugmanifestfulltextcache" lets you inspect the cache, clear it, or add specific manifest nodeids to it. When calling repo.updatecaches(), the manifest(s) for the working copy parents are added to the cache. The hg perfmanifest command has an additional --clear-disk switch to clear this cache when testing manifest loading performance. Using this command to test performance on the firefox repository for revision f947d902ed91, whose manifest has a delta chain length of 60540, we see: $ hg perfmanifest f947d902ed91 --clear-disk ! wall 0.972253 comb 0.970000 user 0.850000 sys 0.120000 (best of 10) $ hg debugmanifestfulltextcache -a `hg log --debug -r f947d902ed91 | grep manifest | cut -d: -f3` Cache contains 1 manifest entries, in order of most to least recent: id: 0294517df4aad07c70701db43bc7ff24c3ce7dbc, size 25.6 MB Total cache data size 25.6 MB, on-disk 0 bytes $ hg perfmanifest f947d902ed91 ! wall 0.036748 comb 0.040000 user 0.020000 sys 0.020000 (best of 100) Worst-case scenario: a manifest text loaded from a single delta; in the firefox repository manifest node 9a1246ff762e is the chain base for the manifest attached to revision f947d902ed91. Loading this from a full cache file is just as fast as without the cache; the extra node ids ensure a big full cache: $ for node in 9a1246ff762e 1a1922c14a3e 54a31d11a36a 0294517df4aa; do > hgd debugmanifestfulltextcache -a $node > /dev/null > done $ hgd perfmanifest -m 9a1246ff762e ! wall 0.077513 comb 0.080000 user 0.030000 sys 0.050000 (best of 100) $ hgd perfmanifest -m 9a1246ff762e --clear-disk ! wall 0.078547 comb 0.080000 user 0.070000 sys 0.010000 (best of 100)

File last commit:

r36469:1fa35ca3 default
r38803:0a57945a default
Show More
bundles.txt
93 lines | 3.2 KiB | text/plain | TextLexer
A bundle is a container for repository data.
Bundles are used as standalone files as well as the interchange format
over the wire protocol used when two Mercurial peers communicate with
each other.
Headers
=======
Bundles produced since Mercurial 0.7 (September 2005) have a 4 byte
header identifying the major bundle type. The header always begins with
``HG`` and the follow 2 bytes indicate the bundle type/version. Some
bundle types have additional data after this 4 byte header.
The following sections describe each bundle header/type.
HG10
----
``HG10`` headers indicate a *changegroup bundle*. This is the original
bundle format, so it is sometimes referred to as *bundle1*. It has been
present since version 0.7 (released September 2005).
This header is followed by 2 bytes indicating the compression algorithm
used for data that follows. All subsequent data following this
compression identifier is compressed according to the algorithm/method
specified.
Supported algorithms include the following.
``BZ``
*bzip2* compression.
Bzip2 compressors emit a leading ``BZ`` header. Mercurial uses this
leading ``BZ`` as part of the bundle header. Therefore consumers
of bzip2 bundles need to *seed* the bzip2 decompressor with ``BZ`` or
seek the input stream back to the beginning of the algorithm component
of the bundle header so that decompressor input is valid. This behavior
is unique among supported compression algorithms.
Supported since version 0.7 (released December 2006).
``GZ``
*zlib* compression.
Supported since version 0.9.2 (released December 2006).
``UN``
*Uncompressed* or no compression. Unmodified changegroup data follows.
Supported since version 0.9.2 (released December 2006).
3rd party extensions may implement their own compression. However, no
authority reserves values for their compression algorithm identifiers.
HG2X
----
``HG2X`` headers (where ``X`` is any value) denote a *bundle2* bundle.
Bundle2 bundles are a container format for various kinds of repository
data and capabilities, beyond changegroup data (which was the only data
supported by ``HG10`` bundles.
``HG20`` is currently the only defined bundle2 version.
The ``HG20`` format is documented at :hg:`help internals.bundle2`.
Initial ``HG20`` support was added in Mercurial 3.0 (released May
2014). However, bundle2 bundles were hidden behind an experimental flag
until version 3.5 (released August 2015), when they were enabled in the
wire protocol. Various commands (including ``hg bundle``) did not
support generating bundle2 files until Mercurial 3.6 (released November
2015).
HGS1
----
*Experimental*
A ``HGS1`` header indicates a *streaming clone bundle*. This is a bundle
that contains raw revlog data from a repository store. (Typically revlog
data is exchanged in the form of changegroups.)
The purpose of *streaming clone bundles* are to *clone* repository data
very efficiently.
The ``HGS1`` header is always followed by 2 bytes indicating a
compression algorithm of the data that follows. Only ``UN``
(uncompressed data) is currently allowed.
``HGS1UN`` support was added as an experimental feature in version 3.6
(released November 2015) as part of the initial offering of the *clone
bundles* feature.