##// END OF EJS Templates
compression: introduce an official `format.revlog-compression` option...
compression: introduce an official `format.revlog-compression` option This option supersedes the `experiment.format.compression` option. The value currently supported are zlib (default) and zstd (if Mercurial was compiled with zstd support). The option gained an explicit reference to `revlog` since this is the target usage here. Different storage methods might require different compression strategies. In our tests, using zstd give a significant CPU usage improvement (both compression and decompressing) while keeping similar repository size. Zstd as other interresting mode (dictionnary, pre-text, etc…) that are probably worth exploring. However, just plain switching from zlib to zstd provide a large benefit.

File last commit:

r42213:4ee906aa default
r42213:4ee906aa default
Show More
test-repo-compengines.t
195 lines | 4.9 KiB | text/troff | Tads3Lexer
/ tests / test-repo-compengines.t
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 A new repository uses zlib storage, which doesn't need a requirement
$ hg init default
$ cd default
$ cat .hg/requires
dotencode
fncache
generaldelta
revlogv1
Boris Feld
test: enable sparse-revlog for test-repo-compengines.t...
r40941 sparserevlog
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 store
Gregory Szorc
simplestore: use a custom store for the simple store repo...
r37433 testonly-simplestore (reposimplestore !)
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818
$ touch foo
$ hg -q commit -A -m 'initial commit with a lot of repeated repeated repeated text to trigger compression'
$ hg debugrevlog -c | grep 0x78
0x78 (x) : 1 (100.00%)
0x78 (x) : 110 (100.00%)
$ cd ..
Unknown compression engine to format.compression aborts
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg --config format.revlog-compression=unknown init unknown
abort: compression engine unknown defined by format.revlog-compression not available
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 (run "hg debuginstall" to list available compression engines)
[255]
A requirement specifying an unknown compression engine results in bail
$ hg init unknownrequirement
$ cd unknownrequirement
$ echo exp-compression-unknown >> .hg/requires
$ hg log
abort: repository requires features unknown to this Mercurial: exp-compression-unknown!
(see https://mercurial-scm.org/wiki/MissingRequirement for more information)
[255]
$ cd ..
#if zstd
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg --config format.revlog-compression=zstd init zstd
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 $ cd zstd
$ cat .hg/requires
dotencode
exp-compression-zstd
fncache
generaldelta
revlogv1
Boris Feld
test: enable sparse-revlog for test-repo-compengines.t...
r40941 sparserevlog
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 store
Gregory Szorc
simplestore: use a custom store for the simple store repo...
r37433 testonly-simplestore (reposimplestore !)
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818
$ touch foo
$ hg -q commit -A -m 'initial commit with a lot of repeated repeated repeated text'
$ hg debugrevlog -c | grep 0x28
0x28 : 1 (100.00%)
0x28 : 98 (100.00%)
$ cd ..
Specifying a new format.compression on an existing repo won't introduce data
with that engine or a requirement
$ cd default
$ touch bar
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg --config format.revlog-compression=zstd -q commit -A -m 'add bar with a lot of repeated repeated repeated text'
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818
$ cat .hg/requires
dotencode
fncache
generaldelta
revlogv1
Boris Feld
test: enable sparse-revlog for test-repo-compengines.t...
r40941 sparserevlog
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818 store
Gregory Szorc
simplestore: use a custom store for the simple store repo...
r37433 testonly-simplestore (reposimplestore !)
Gregory Szorc
localrepo: experimental support for non-zlib revlog compression...
r30818
$ hg debugrevlog -c | grep 0x78
0x78 (x) : 2 (100.00%)
0x78 (x) : 199 (100.00%)
#endif
compression: introduce a `storage.revlog.zlib.level` configuration...
r42210
checking zlib options
=====================
$ hg init zlib-level-default
$ hg init zlib-level-1
$ cat << EOF >> zlib-level-1/.hg/hgrc
> [storage]
> revlog.zlib.level=1
> EOF
$ hg init zlib-level-9
$ cat << EOF >> zlib-level-9/.hg/hgrc
> [storage]
> revlog.zlib.level=9
> EOF
$ commitone() {
> repo=$1
> cp $RUNTESTDIR/bundles/issue4438-r1.hg $repo/a
> hg -R $repo add $repo/a
> hg -R $repo commit -m some-commit
> }
$ for repo in zlib-level-default zlib-level-1 zlib-level-9; do
> commitone $repo
> done
$ $RUNTESTDIR/f -s */.hg/store/data/*
zlib-level-1/.hg/store/data/a.i: size=4146
zlib-level-9/.hg/store/data/a.i: size=4138
zlib-level-default/.hg/store/data/a.i: size=4138
Test error cases
$ hg init zlib-level-invalid
$ cat << EOF >> zlib-level-invalid/.hg/hgrc
> [storage]
> revlog.zlib.level=foobar
> EOF
$ commitone zlib-level-invalid
abort: storage.revlog.zlib.level is not a valid integer ('foobar')
abort: storage.revlog.zlib.level is not a valid integer ('foobar')
[255]
$ hg init zlib-level-out-of-range
$ cat << EOF >> zlib-level-out-of-range/.hg/hgrc
> [storage]
> revlog.zlib.level=42
> EOF
$ commitone zlib-level-out-of-range
abort: invalid value for `storage.revlog.zlib.level` config: 42
abort: invalid value for `storage.revlog.zlib.level` config: 42
[255]
compression: introduce a `storage.revlog.zstd.level` configuration...
r42211 checking zstd options
=====================
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg init zstd-level-default --config format.revlog-compression=zstd
$ hg init zstd-level-1 --config format.revlog-compression=zstd
compression: introduce a `storage.revlog.zstd.level` configuration...
r42211 $ cat << EOF >> zstd-level-1/.hg/hgrc
> [storage]
> revlog.zstd.level=1
> EOF
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg init zstd-level-22 --config format.revlog-compression=zstd
compression: introduce a `storage.revlog.zstd.level` configuration...
r42211 $ cat << EOF >> zstd-level-22/.hg/hgrc
> [storage]
> revlog.zstd.level=22
> EOF
$ commitone() {
> repo=$1
> cp $RUNTESTDIR/bundles/issue4438-r1.hg $repo/a
> hg -R $repo add $repo/a
> hg -R $repo commit -m some-commit
> }
$ for repo in zstd-level-default zstd-level-1 zstd-level-22; do
> commitone $repo
> done
$ $RUNTESTDIR/f -s zstd-*/.hg/store/data/*
zstd-level-1/.hg/store/data/a.i: size=4097
zstd-level-22/.hg/store/data/a.i: size=4091
zstd-level-default/.hg/store/data/a.i: size=4094
Test error cases
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg init zstd-level-invalid --config format.revlog-compression=zstd
compression: introduce a `storage.revlog.zstd.level` configuration...
r42211 $ cat << EOF >> zstd-level-invalid/.hg/hgrc
> [storage]
> revlog.zstd.level=foobar
> EOF
$ commitone zstd-level-invalid
abort: storage.revlog.zstd.level is not a valid integer ('foobar')
abort: storage.revlog.zstd.level is not a valid integer ('foobar')
[255]
compression: introduce an official `format.revlog-compression` option...
r42213 $ hg init zstd-level-out-of-range --config format.revlog-compression=zstd
compression: introduce a `storage.revlog.zstd.level` configuration...
r42211 $ cat << EOF >> zstd-level-out-of-range/.hg/hgrc
> [storage]
> revlog.zstd.level=42
> EOF
$ commitone zstd-level-out-of-range
abort: invalid value for `storage.revlog.zstd.level` config: 42
abort: invalid value for `storage.revlog.zstd.level` config: 42
[255]