##// END OF EJS Templates
share-safe: enable by default (BC)...
share-safe: enable by default (BC) The feature have been around for a year (4 version) and is quite important. Lets make it enabled by default. Differential Revision: https://phab.mercurial-scm.org/D11997

File last commit:

r49515:7ee07e1a default
r49515:7ee07e1a default
Show More
test-persistent-nodemap.t
1262 lines | 41.2 KiB | text/troff | Tads3Lexer
/ tests / test-persistent-nodemap.t
nodemap: have some python code writing a nodemap in persistent binary form...
r44788 ===================================
Test the persistent on-disk nodemap
===================================
persistent-nodemap: enable the feature by default when using Rust...
r47646
test: enable share-safe in test-persistent-nodemap...
r48851 $ cat << EOF >> $HGRCPATH
> [format]
> use-share-safe=yes
> [extensions]
> share=
> EOF
persistent-nodemap: enable the feature by default when using Rust...
r47646 #if no-rust
nodemap: gate the feature behind a new requirement...
r45295 $ cat << EOF >> $HGRCPATH
nodemap: move the main switch to the `format` section...
r45297 > [format]
> use-persistent-nodemap=yes
nodemap: add a (python) index class for persistent nodemap testing...
r44794 > [devel]
> persistent-nodemap=yes
nodemap: write nodemap data on disk...
r44789 > EOF
persistent-nodemap: add a "warn" option to the slow-path config...
r47028
persistent-nodemap: enable the feature by default when using Rust...
r47646 #endif
persistent-nodemap: add a "warn" option to the slow-path config...
r47028 $ hg init test-repo --config storage.revlog.persistent-nodemap.slow-path=allow
nodemap: gate the feature behind a new requirement...
r45295 $ cd test-repo
persistent-nodemap: add a "warn" option to the slow-path config...
r47028
Check handling of the default slow-path value
#if no-pure no-rust
$ hg id
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 abort: accessing `persistent-nodemap` repository without associated fast implementation.
persistent-nodemap: add a "warn" option to the slow-path config...
r47028 (check `hg help config.format.use-persistent-nodemap` for details)
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 [255]
persistent-nodemap: add a "warn" option to the slow-path config...
r47028
Unlock further check (we are here to test the feature)
$ cat << EOF >> $HGRCPATH
> [storage]
> # to avoid spamming the test
> revlog.persistent-nodemap.slow-path=allow
> EOF
#endif
Simon Sapin
persistent-nodemap: add test case reproducing a Rust panic...
r47140 #if rust
Simon Sapin
persistent-nodemap: Fix Rust declarations for Revlog_CAPI signatures...
r47141 Regression test for a previous bug in Rust/C FFI for the `Revlog_CAPI` capsule:
in places where `mercurial/cext/revlog.c` function signatures use `Py_ssize_t`
(64 bits on Linux x86_64), corresponding declarations in `rust/hg-cpython/src/cindex.rs`
incorrectly used `libc::c_int` (32 bits).
As a result, -1 passed from Rust for the null revision became 4294967295 in C.
Simon Sapin
persistent-nodemap: add test case reproducing a Rust panic...
r47140
Simon Sapin
persistent-nodemap: Fix Rust declarations for Revlog_CAPI signatures...
r47141 $ hg log -r 00000000
changeset: -1:000000000000
tag: tip
user:
date: Thu Jan 01 00:00:00 1970 +0000
Simon Sapin
persistent-nodemap: add test case reproducing a Rust panic...
r47140
#endif
persistent-nodemap: add a "warn" option to the slow-path config...
r47028
nodemap: teach `hg debugformat` about the persistent nodemap option...
r45303 $ hg debugformat
format-variant repo
fncache: yes
Simon Sapin
dirstate-v2: Add `hg debugupgraderepo` command support...
r48111 dirstate-v2: no
nodemap: teach `hg debugformat` about the persistent nodemap option...
r45303 dotencode: yes
generaldelta: yes
test: enable share-safe in test-persistent-nodemap...
r48851 share-safe: yes
nodemap: teach `hg debugformat` about the persistent nodemap option...
r45303 sparserevlog: yes
persistent-nodemap: yes
copies-sdc: no
Raphaël Gomès
revlogv2: allow upgrading to v2...
r47439 revlog-v2: no
changelogv2: introduce a "changelogv2" feature...
r48037 changelog-v2: no
nodemap: teach `hg debugformat` about the persistent nodemap option...
r45303 plain-cl-delta: yes
revlog-compression: use zstd by default (if available)...
r47636 compression: zlib (no-zstd !)
compression: zstd (zstd !)
nodemap: teach `hg debugformat` about the persistent nodemap option...
r45303 compression-level: default
persistent-nodemap: add a "warn" option to the slow-path config...
r47028 $ hg debugbuilddag .+5000 --new-file
nodemap: add a flag to dump the details of the docket...
r44806 $ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: track the maximum revision tracked in the nodemap...
r44807 tip-rev: 5000
nodemap: create files in the repository used in the test...
r45289 tip-node: 6b02b8c7b96654c25e86ba69eda198d7e6ad8b3c
data-length: 121088
nodemap: track the total and unused amount of data in the rawdata file...
r44808 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: use an intermediate "docket" file to carry small metadata...
r44792 $ f --size .hg/store/00changelog.n
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog.n: size=62
nodemap: check that a simple lookup works fine...
r44954
Simple lookup works
$ ANYNODE=`hg log --template '{node|short}\n' --rev tip`
$ hg log -r "$ANYNODE" --template '{rev}\n'
5000
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000
#if rust
$ f --sha256 .hg/store/00changelog-*.nd
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: sha256=2e029d3200bd1a986b32784fc2ef1a3bd60dc331f025718bcf5ff44d93f026fd (glob)
nodemap: also use persistent nodemap for manifest...
r45290
$ f --sha256 .hg/store/00manifest-*.nd
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00manifest-????????.nd: sha256=97117b1c064ea2f86664a124589e47db0e254e8d34739b5c5cc5bf31c9da2b51 (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 $ hg debugnodemap --dump-new | f --sha256 --size
nodemap: create files in the repository used in the test...
r45289 size=121088, sha256=2e029d3200bd1a986b32784fc2ef1a3bd60dc331f025718bcf5ff44d93f026fd
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
nodemap: create files in the repository used in the test...
r45289 size=121088, sha256=2e029d3200bd1a986b32784fc2ef1a3bd60dc331f025718bcf5ff44d93f026fd
0000: 00 00 00 91 00 00 00 20 00 00 00 bb 00 00 00 e7 |....... ........|
0010: 00 00 00 66 00 00 00 a1 00 00 01 13 00 00 01 22 |...f..........."|
0020: 00 00 00 23 00 00 00 fc 00 00 00 ba 00 00 00 5e |...#...........^|
0030: 00 00 00 df 00 00 01 4e 00 00 01 65 00 00 00 ab |.......N...e....|
0040: 00 00 00 a9 00 00 00 95 00 00 00 73 00 00 00 38 |...........s...8|
0050: 00 00 00 cc 00 00 00 92 00 00 00 90 00 00 00 69 |...............i|
0060: 00 00 00 ec 00 00 00 8d 00 00 01 4f 00 00 00 12 |...........O....|
0070: 00 00 02 0c 00 00 00 77 00 00 00 9c 00 00 00 8f |.......w........|
0080: 00 00 00 d5 00 00 00 6b 00 00 00 48 00 00 00 b3 |.......k...H....|
0090: 00 00 00 e5 00 00 00 b5 00 00 00 8e 00 00 00 ad |................|
00a0: 00 00 00 7b 00 00 00 7c 00 00 00 0b 00 00 00 2b |...{...|.......+|
00b0: 00 00 00 c6 00 00 00 1e 00 00 01 08 00 00 00 11 |................|
00c0: 00 00 01 30 00 00 00 26 00 00 01 9c 00 00 00 35 |...0...&.......5|
00d0: 00 00 00 b8 00 00 01 31 00 00 00 2c 00 00 00 55 |.......1...,...U|
00e0: 00 00 00 8a 00 00 00 9a 00 00 00 0c 00 00 01 1e |................|
00f0: 00 00 00 a4 00 00 00 83 00 00 00 c9 00 00 00 8c |................|
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000
#else
nodemap: delete older raw data file when creating a new ones...
r44793 $ f --sha256 .hg/store/00changelog-*.nd
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: sha256=f544f5462ff46097432caf6d764091f6d8c46d6121be315ead8576d548c9dd79 (glob)
nodemap: add a function to read the data from disk...
r44790 $ hg debugnodemap --dump-new | f --sha256 --size
nodemap: create files in the repository used in the test...
r45289 size=121088, sha256=f544f5462ff46097432caf6d764091f6d8c46d6121be315ead8576d548c9dd79
nodemap: add a function to read the data from disk...
r44790 $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
nodemap: create files in the repository used in the test...
r45289 size=121088, sha256=f544f5462ff46097432caf6d764091f6d8c46d6121be315ead8576d548c9dd79
nodemap: have some python code writing a nodemap in persistent binary form...
r44788 0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
nodemap: create files in the repository used in the test...
r45289 0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
0020: ff ff ff ff ff ff f5 06 ff ff ff ff ff ff f3 e7 |................|
0030: ff ff ef ca ff ff ff ff ff ff ff ff ff ff ff ff |................|
0040: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
0050: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ed 08 |................|
0060: ff ff ed 66 ff ff ff ff ff ff ff ff ff ff ff ff |...f............|
nodemap: have some python code writing a nodemap in persistent binary form...
r44788 0070: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
nodemap: create files in the repository used in the test...
r45289 0080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
0090: ff ff ff ff ff ff ff ff ff ff ff ff ff ff f6 ed |................|
00a0: ff ff ff ff ff ff fe 61 ff ff ff ff ff ff ff ff |.......a........|
00b0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
00c0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
nodemap: have some python code writing a nodemap in persistent binary form...
r44788 00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
nodemap: create files in the repository used in the test...
r45289 00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff f1 02 |................|
00f0: ff ff ff ff ff ff ed 1b ff ff ff ff ff ff ff ff |................|
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000
#endif
nodemap: add basic checking of the on disk nodemap content...
r44799 $ hg debugnodemap --check
revision in index: 5001
revision in nodemap: 5001
nodemap: delete older raw data file when creating a new ones...
r44793
add a new commit
$ hg up
nodemap: create files in the repository used in the test...
r45289 5001 files updated, 0 files merged, 0 files removed, 0 files unresolved
nodemap: delete older raw data file when creating a new ones...
r44793 $ echo foo > foo
$ hg add foo
nodemap: add a new mode value, "strict"...
r45293
persistent-node: check the value of the slow-path config...
r47027
Check slow-path config value handling
-------------------------------------
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 #if no-pure no-rust
persistent-node: check the value of the slow-path config...
r47027 $ hg id --config "storage.revlog.persistent-nodemap.slow-path=invalid-value"
unknown value for config "storage.revlog.persistent-nodemap.slow-path": "invalid-value"
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 falling back to default value: abort
abort: accessing `persistent-nodemap` repository without associated fast implementation.
(check `hg help config.format.use-persistent-nodemap` for details)
[255]
nodemap: add a new mode value, "strict"...
r45293
persistent-nodemap: add a "warn" option to the slow-path config...
r47028 $ hg log -r . --config "storage.revlog.persistent-nodemap.slow-path=warn"
warning: accessing `persistent-nodemap` repository without associated fast implementation.
(check `hg help config.format.use-persistent-nodemap` for details)
changeset: 5000:6b02b8c7b966
tag: tip
user: debugbuilddag
date: Thu Jan 01 01:23:20 1970 +0000
summary: r5000
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 $ hg ci -m 'foo' --config "storage.revlog.persistent-nodemap.slow-path=abort"
abort: accessing `persistent-nodemap` repository without associated fast implementation.
(check `hg help config.format.use-persistent-nodemap` for details)
nodemap: add a new mode value, "strict"...
r45293 [255]
persistent-nodemap: add a "abort" option to the slow-path config...
r47029 #else
$ hg id --config "storage.revlog.persistent-nodemap.slow-path=invalid-value"
unknown value for config "storage.revlog.persistent-nodemap.slow-path": "invalid-value"
falling back to default value: abort
6b02b8c7b966+ tip
nodemap: add a new mode value, "strict"...
r45293 #endif
nodemap: delete older raw data file when creating a new ones...
r44793 $ hg ci -m 'foo'
nodemap: track the total and unused amount of data in the rawdata file...
r44808
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #if no-pure no-rust
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 tip-rev: 5001
nodemap: create files in the repository used in the test...
r45289 tip-node: 16395c3cf7e231394735e6b1717823ada303fb0c
data-length: 121088
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #else
nodemap: add a flag to dump the details of the docket...
r44806 $ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: track the maximum revision tracked in the nodemap...
r44807 tip-rev: 5001
nodemap: create files in the repository used in the test...
r45289 tip-node: 16395c3cf7e231394735e6b1717823ada303fb0c
data-length: 121344
data-unused: 256
data-unused: 0.211%
nodemap: track the total and unused amount of data in the rawdata file...
r44808 #endif
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000
nodemap: delete older raw data file when creating a new ones...
r44793 $ f --size .hg/store/00changelog.n
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog.n: size=62
nodemap: introduce append-only incremental update of the persistent data...
r44805
(The pure code use the debug code that perform incremental update, the C code reencode from scratch)
#if pure
$ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121344, sha256=cce54c5da5bde3ad72a4938673ed4064c86231b9c64376b082b163fdb20f8f66 (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #endif
nodemap: introduce append-only incremental update of the persistent data...
r44805
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #if rust
$ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121344, sha256=952b042fcf614ceb37b542b1b723e04f18f83efe99bee4e0f5ccd232ef470e58 (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #endif
#if no-pure no-rust
nodemap: delete older raw data file when creating a new ones...
r44793 $ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121088, sha256=df7c06a035b96cb28c7287d349d603baef43240be7736fe34eea419a49702e17 (glob)
nodemap: introduce append-only incremental update of the persistent data...
r44805 #endif
nodemap: add basic checking of the on disk nodemap content...
r44799 $ hg debugnodemap --check
revision in index: 5002
revision in nodemap: 5002
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843
Test code path without mmap
---------------------------
$ echo bar > bar
$ hg add bar
persistent-nodemap: rename the storage.revlog.nodemap.mmap option...
r47024 $ hg ci -m 'bar' --config storage.revlog.persistent-nodemap.mmap=no
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843
persistent-nodemap: rename the storage.revlog.nodemap.mmap option...
r47024 $ hg debugnodemap --check --config storage.revlog.persistent-nodemap.mmap=yes
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 revision in index: 5003
revision in nodemap: 5003
persistent-nodemap: rename the storage.revlog.nodemap.mmap option...
r47024 $ hg debugnodemap --check --config storage.revlog.persistent-nodemap.mmap=no
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 revision in index: 5003
revision in nodemap: 5003
#if pure
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121600
data-unused: 512
data-unused: 0.421%
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 $ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121600, sha256=def52503d049ccb823974af313a98a935319ba61f40f3aa06a8be4d35c215054 (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #endif
#if rust
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121600
data-unused: 512
data-unused: 0.421%
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 $ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121600, sha256=dacf5b5f1d4585fee7527d0e67cad5b1ba0930e6a0928f650f779aefb04ce3fb (glob)
rust-nodemap: automatically use the rust index for persistent nodemap...
r45000 #endif
#if no-pure no-rust
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 $ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121088
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 $ f --sha256 .hg/store/00changelog-*.nd --size
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121088, sha256=59fcede3e3cc587755916ceed29e3c33748cd1aa7d2f91828ac83e7979d935e8 (glob)
nodemap: introduce an option to use mmap to read the nodemap mapping...
r44843 #endif
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932
Test force warming the cache
$ rm .hg/store/00changelog.n
$ hg debugnodemap --metadata
$ hg debugupdatecache
#if pure
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121088
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 #else
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121088
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: warm the persistent nodemap on disk with debugupdatecache...
r44932 #endif
nodemap: test that an outdated nodemap can catch up...
r44984
Check out of sync nodemap
=========================
First copy old data on the side.
$ mkdir ../tmp-copies
persistent-nodemap: use the intended uuid size...
r48092 $ cp .hg/store/00changelog-????????.nd .hg/store/00changelog.n ../tmp-copies
nodemap: test that an outdated nodemap can catch up...
r44984
Nodemap lagging behind
----------------------
make a new commit
$ echo bar2 > bar
$ hg ci -m 'bar2'
$ NODE=`hg log -r tip -T '{node}\n'`
$ hg log -r "$NODE" -T '{rev}\n'
5003
If the nodemap is lagging behind, it can catch up fine
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: test that an outdated nodemap can catch up...
r44984 tip-rev: 5003
nodemap: create files in the repository used in the test...
r45289 tip-node: c9329770f979ade2d16912267c38ba5f82fd37b3
data-length: 121344 (pure !)
data-length: 121344 (rust !)
data-length: 121152 (no-rust no-pure !)
data-unused: 192 (pure !)
data-unused: 192 (rust !)
nodemap: test that an outdated nodemap can catch up...
r44984 data-unused: 0 (no-rust no-pure !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.158% (pure !)
data-unused: 0.158% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-rust no-pure !)
nodemap: test that an outdated nodemap can catch up...
r44984 $ cp -f ../tmp-copies/* .hg/store/
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: test that an outdated nodemap can catch up...
r44984 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121088
nodemap: test that an outdated nodemap can catch up...
r44984 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: test that an outdated nodemap can catch up...
r44984 $ hg log -r "$NODE" -T '{rev}\n'
5003
nodemap: track the tip_node for validation...
r45002
changelog altered
-----------------
If the nodemap is not gated behind a requirements, an unaware client can alter
the repository so the revlog used to generate the nodemap is not longer
compatible with the persistent nodemap. We need to detect that.
$ hg up "$NODE~5"
nodemap: create files in the repository used in the test...
r45289 0 files updated, 0 files merged, 4 files removed, 0 files unresolved
nodemap: track the tip_node for validation...
r45002 $ echo bar > babar
$ hg add babar
$ hg ci -m 'babar'
created new head
$ OTHERNODE=`hg log -r tip -T '{node}\n'`
$ hg log -r "$OTHERNODE" -T '{rev}\n'
5004
$ hg --config extensions.strip= strip --rev "$NODE~1" --no-backup
the nodemap should detect the changelog have been tampered with and recover.
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: track the tip_node for validation...
r45002 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: b355ef8adce0949b8bdf6afc72ca853740d65944
data-length: 121536 (pure !)
data-length: 121088 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: track the tip_node for validation...
r45002 data-unused: 448 (pure !)
nodemap: automatically "vacuum" the persistent nodemap when too sparse...
r45126 data-unused: 0 (rust !)
nodemap: track the tip_node for validation...
r45002 data-unused: 0 (no-pure no-rust !)
nodemap: automatically "vacuum" the persistent nodemap when too sparse...
r45126 data-unused: 0.000% (rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.369% (pure !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: track the tip_node for validation...
r45002
$ cp -f ../tmp-copies/* .hg/store/
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: track the tip_node for validation...
r45002 tip-rev: 5002
nodemap: create files in the repository used in the test...
r45289 tip-node: 880b18d239dfa9f632413a2071bfdbcc4806a4fd
data-length: 121088
nodemap: track the tip_node for validation...
r45002 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: track the tip_node for validation...
r45002 $ hg log -r "$OTHERNODE" -T '{rev}\n'
5002
nodemap: make sure hooks have access to an up-to-date version...
r45003
persistent-nodemap: catch the right exception on python...
r47039 missing data file
-----------------
$ UUID=`hg debugnodemap --metadata| grep 'uid:' | \
> sed 's/uid: //'`
$ FILE=.hg/store/00changelog-"${UUID}".nd
$ mv $FILE ../tmp-data-file
$ cp .hg/store/00changelog.n ../tmp-docket
mercurial don't crash
$ hg log -r .
changeset: 5002:b355ef8adce0
tag: tip
parent: 4998:d918ad6d18d3
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: babar
$ hg debugnodemap --metadata
$ hg debugupdatecache
$ hg debugnodemap --metadata
uid: * (glob)
tip-rev: 5002
tip-node: b355ef8adce0949b8bdf6afc72ca853740d65944
data-length: 121088
data-unused: 0
data-unused: 0.000%
$ mv ../tmp-data-file $FILE
$ mv ../tmp-docket .hg/store/00changelog.n
nodemap: make sure hooks have access to an up-to-date version...
r45003 Check transaction related property
==================================
An up to date nodemap should be available to shell hooks,
$ echo dsljfl > a
$ hg add a
$ hg ci -m a
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: make sure hooks have access to an up-to-date version...
r45003 tip-rev: 5003
nodemap: create files in the repository used in the test...
r45289 tip-node: a52c5079765b5865d97b993b303a18740113bbb2
data-length: 121088
nodemap: make sure hooks have access to an up-to-date version...
r45003 data-unused: 0
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000%
nodemap: make sure hooks have access to an up-to-date version...
r45003 $ echo babar2 > babar
$ hg ci -m 'babar2' --config "hooks.pretxnclose.nodemap-test=hg debugnodemap --metadata"
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: make sure hooks have access to an up-to-date version...
r45003 tip-rev: 5004
nodemap: create files in the repository used in the test...
r45289 tip-node: 2f5fb1c06a16834c5679d672e90da7c5f3b1a984
data-length: 121280 (pure !)
data-length: 121280 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: make sure hooks have access to an up-to-date version...
r45003 data-unused: 192 (pure !)
data-unused: 192 (rust !)
data-unused: 0 (no-pure no-rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.158% (pure !)
data-unused: 0.158% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: make sure hooks have access to an up-to-date version...
r45003 $ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: make sure hooks have access to an up-to-date version...
r45003 tip-rev: 5004
nodemap: create files in the repository used in the test...
r45289 tip-node: 2f5fb1c06a16834c5679d672e90da7c5f3b1a984
data-length: 121280 (pure !)
data-length: 121280 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: make sure hooks have access to an up-to-date version...
r45003 data-unused: 192 (pure !)
data-unused: 192 (rust !)
data-unused: 0 (no-pure no-rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.158% (pure !)
data-unused: 0.158% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: test that concurrent process don't see the pending transaction...
r45123
Another process does not see the pending nodemap content during run.
$ echo qpoasp > a
$ hg ci -m a2 \
Matt Harbison
tests: stabilize test-persistent-nodemap.t on Windows...
r47861 > --config "hooks.pretxnclose=sh \"$RUNTESTDIR/testlib/wait-on-file\" 20 sync-repo-read sync-txn-pending" \
nodemap: test that concurrent process don't see the pending transaction...
r45123 > --config "hooks.txnclose=touch sync-txn-close" > output.txt 2>&1 &
(read the repository while the commit transaction is pending)
Matt Harbison
tests: stabilize test-persistent-nodemap.t on Windows...
r47861 $ sh "$RUNTESTDIR/testlib/wait-on-file" 20 sync-txn-pending && \
nodemap: test that concurrent process don't see the pending transaction...
r45123 > hg debugnodemap --metadata && \
Matt Harbison
tests: stabilize test-persistent-nodemap.t on Windows...
r47861 > sh "$RUNTESTDIR/testlib/wait-on-file" 20 sync-txn-close sync-repo-read
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: test that concurrent process don't see the pending transaction...
r45123 tip-rev: 5004
nodemap: create files in the repository used in the test...
r45289 tip-node: 2f5fb1c06a16834c5679d672e90da7c5f3b1a984
data-length: 121280 (pure !)
data-length: 121280 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: test that concurrent process don't see the pending transaction...
r45123 data-unused: 192 (pure !)
data-unused: 192 (rust !)
data-unused: 0 (no-pure no-rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.158% (pure !)
data-unused: 0.158% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: test that concurrent process don't see the pending transaction...
r45123 $ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: test that concurrent process don't see the pending transaction...
r45123 tip-rev: 5005
nodemap: create files in the repository used in the test...
r45289 tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121536 (pure !)
data-length: 121536 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: test that concurrent process don't see the pending transaction...
r45123 data-unused: 448 (pure !)
data-unused: 448 (rust !)
data-unused: 0 (no-pure no-rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.369% (pure !)
data-unused: 0.369% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: test that concurrent process don't see the pending transaction...
r45123
$ cat output.txt
nodemap: make sure on disk change get rolled back with the transaction...
r45124 Check that a failing transaction will properly revert the data
$ echo plakfe > a
$ f --size --sha256 .hg/store/00changelog-*.nd
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121536, sha256=bb414468d225cf52d69132e1237afba34d4346ee2eb81b505027e6197b107f03 (glob) (pure !)
.hg/store/00changelog-????????.nd: size=121536, sha256=909ac727bc4d1c0fda5f7bff3c620c98bd4a2967c143405a1503439e33b377da (glob) (rust !)
.hg/store/00changelog-????????.nd: size=121088, sha256=342d36d30d86dde67d3cb6c002606c4a75bcad665595d941493845066d9c8ee0 (glob) (no-pure no-rust !)
nodemap: make sure on disk change get rolled back with the transaction...
r45124 $ hg ci -m a3 --config "extensions.abort=$RUNTESTDIR/testlib/crash_transaction_late.py"
transaction abort!
rollback completed
abort: This is a late abort
[255]
$ hg debugnodemap --metadata
persistent-nodemap: use the intended uuid size...
r48092 uid: ???????? (glob)
nodemap: make sure on disk change get rolled back with the transaction...
r45124 tip-rev: 5005
nodemap: create files in the repository used in the test...
r45289 tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121536 (pure !)
data-length: 121536 (rust !)
data-length: 121088 (no-pure no-rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 448 (pure !)
data-unused: 448 (rust !)
data-unused: 0 (no-pure no-rust !)
nodemap: create files in the repository used in the test...
r45289 data-unused: 0.369% (pure !)
data-unused: 0.369% (rust !)
nodemap: display percentage of unused in `hg debugnodemap`...
r45125 data-unused: 0.000% (no-pure no-rust !)
nodemap: make sure on disk change get rolled back with the transaction...
r45124 $ f --size --sha256 .hg/store/00changelog-*.nd
persistent-nodemap: use the intended uuid size...
r48092 .hg/store/00changelog-????????.nd: size=121536, sha256=bb414468d225cf52d69132e1237afba34d4346ee2eb81b505027e6197b107f03 (glob) (pure !)
.hg/store/00changelog-????????.nd: size=121536, sha256=909ac727bc4d1c0fda5f7bff3c620c98bd4a2967c143405a1503439e33b377da (glob) (rust !)
.hg/store/00changelog-????????.nd: size=121088, sha256=342d36d30d86dde67d3cb6c002606c4a75bcad665595d941493845066d9c8ee0 (glob) (no-pure no-rust !)
nodemap: fix validity checking when revlog is too short...
r45481
Check that removing content does not confuse the nodemap
--------------------------------------------------------
removing data with rollback
$ echo aso > a
$ hg ci -m a4
$ hg rollback
repository tip rolled back to revision 5005 (undo commit)
working directory now based on revision 5005
$ hg id -r .
Augie Fackler
merge with stable
r45487 90d5d3ba2fc4 tip
nodemap: fix validity checking when revlog is too short...
r45481
persistent-nodemap: fix a typo in a test comment...
r48848 removing data with strip
nodemap: fix validity checking when revlog is too short...
r45481
$ echo aso > a
$ hg ci -m a4
$ hg --config extensions.strip= strip -r . --no-backup
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg id -r . --traceback
Augie Fackler
merge with stable
r45487 90d5d3ba2fc4 tip
persistent-nodemap: introduce a test to highlight possible race...
r48852 (be a good citizen and regenerate the nodemap)
$ hg debugupdatecaches
$ hg debugnodemap --metadata
uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
Check race condition when multiple process write new data to the repository
---------------------------------------------------------------------------
In this test, we check that two writers touching the repositories will not
overwrite each other data. This test is prompted by the existent of issue6554.
Where a writer ended up using and outdated docket to update the repository. See
the dedicated extension for details on the race windows and read/write schedule
necessary to end up in this situation: testlib/persistent-nodemap-race-ext.py
The issue was initially observed on a server with a high push trafic, but it
can be reproduced using a share and two commiting process which seems simpler.
The test is Rust only as the other implementation does not use the same
read/write patterns.
$ cd ..
#if rust
$ cp -R test-repo race-repo
$ hg share race-repo ./other-wc --config format.use-share-safe=yes
updating working directory
5001 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg debugformat -R ./race-repo | egrep 'share-safe|persistent-nodemap'
share-safe: yes
persistent-nodemap: yes
$ hg debugformat -R ./other-wc/ | egrep 'share-safe|persistent-nodemap'
share-safe: yes
persistent-nodemap: yes
$ hg -R ./other-wc update 'min(head())'
3 files updated, 0 files merged, 2 files removed, 0 files unresolved
$ hg -R ./race-repo debugnodemap --metadata
uid: 43c37dde
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
$ hg -R ./race-repo log -G -r 'head()'
@ changeset: 5005:90d5d3ba2fc4
| tag: tip
~ user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a2
o changeset: 5001:16395c3cf7e2
| user: test
~ date: Thu Jan 01 00:00:00 1970 +0000
summary: foo
$ hg -R ./other-wc log -G -r 'head()'
o changeset: 5005:90d5d3ba2fc4
| tag: tip
~ user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a2
@ changeset: 5001:16395c3cf7e2
| user: test
~ date: Thu Jan 01 00:00:00 1970 +0000
summary: foo
$ echo left-side-race > race-repo/left-side-race
$ hg -R ./race-repo/ add race-repo/left-side-race
$ echo right-side-race > ./other-wc/right-side-race
$ hg -R ./other-wc/ add ./other-wc/right-side-race
$ mkdir sync-files
$ mkdir outputs
$ (
> hg -R ./race-repo/ commit -m left-side-commit \
> --config "extensions.race=${RUNTESTDIR}/testlib/persistent-nodemap-race-ext.py" \
> --config 'devel.nodemap-race.role=left';
> touch sync-files/left-done
> ) > outputs/left.txt 2>&1 &
$ (
> hg -R ./other-wc/ commit -m right-side-commit \
> --config "extensions.race=${RUNTESTDIR}/testlib/persistent-nodemap-race-ext.py" \
> --config 'devel.nodemap-race.role=right';
> touch sync-files/right-done
> ) > outputs/right.txt 2>&1 &
$ (
> hg -R ./race-repo/ check-nodemap-race \
> --config "extensions.race=${RUNTESTDIR}/testlib/persistent-nodemap-race-ext.py" \
> --config 'devel.nodemap-race.role=reader';
> touch sync-files/reader-done
> ) > outputs/reader.txt 2>&1 &
$ sh "$RUNTESTDIR"/testlib/wait-on-file 10 sync-files/left-done
$ cat outputs/left.txt
docket-details:
uid: 43c37dde
actual-tip: 5005
tip-rev: 5005
data-length: 121088
nodemap-race: left side locked and ready to commit
docket-details:
uid: 43c37dde
actual-tip: 5005
tip-rev: 5005
data-length: 121088
finalized changelog write
persisting changelog nodemap
new data start at 121088
persisted changelog nodemap
docket-details:
uid: 43c37dde
actual-tip: 5006
tip-rev: 5006
data-length: 121280
$ sh "$RUNTESTDIR"/testlib/wait-on-file 10 sync-files/right-done
$ cat outputs/right.txt
nodemap-race: right side start of the locking sequence
nodemap-race: right side reading changelog
nodemap-race: right side reading of changelog is done
docket-details:
uid: 43c37dde
actual-tip: 5006
tip-rev: 5005
data-length: 121088
nodemap-race: right side ready to wait for the lock
nodemap-race: right side locked and ready to commit
docket-details:
uid: 43c37dde
actual-tip: 5006
changelog: also monitor `00changelog.n` when applicable (issue6554)...
r48853 tip-rev: 5006
data-length: 121280
persistent-nodemap: introduce a test to highlight possible race...
r48852 right ready to write, waiting for reader
right proceeding with writing its changelog index and nodemap
finalized changelog write
persisting changelog nodemap
changelog: also monitor `00changelog.n` when applicable (issue6554)...
r48853 new data start at 121280
persistent-nodemap: introduce a test to highlight possible race...
r48852 persisted changelog nodemap
docket-details:
uid: 43c37dde
actual-tip: 5007
tip-rev: 5007
changelog: also monitor `00changelog.n` when applicable (issue6554)...
r48853 data-length: 121536
persistent-nodemap: introduce a test to highlight possible race...
r48852 $ sh "$RUNTESTDIR"/testlib/wait-on-file 10 sync-files/reader-done
$ cat outputs/reader.txt
reader: reading changelog
reader ready to read the changelog, waiting for right
reader: nodemap docket read
record-data-length: 121280
actual-data-length: 121280
changelog: also monitor `00changelog.n` when applicable (issue6554)...
r48853 file-actual-length: 121536
persistent-nodemap: introduce a test to highlight possible race...
r48852 reader: changelog read
docket-details:
uid: 43c37dde
actual-tip: 5006
tip-rev: 5006
data-length: 121280
tip-rev: 5006
tip-node: 492901161367
node-rev: 5006
$ hg -R ./race-repo log -G -r 'head()'
o changeset: 5007:ac4a2abde241
| tag: tip
~ parent: 5001:16395c3cf7e2
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: right-side-commit
@ changeset: 5006:492901161367
| user: test
~ date: Thu Jan 01 00:00:00 1970 +0000
summary: left-side-commit
$ hg -R ./other-wc log -G -r 'head()'
@ changeset: 5007:ac4a2abde241
| tag: tip
~ parent: 5001:16395c3cf7e2
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: right-side-commit
o changeset: 5006:492901161367
| user: test
~ date: Thu Jan 01 00:00:00 1970 +0000
summary: left-side-commit
#endif
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 Test upgrade / downgrade
========================
persistent-nodemap: introduce a test to highlight possible race...
r48852 $ cd ./test-repo/
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 downgrading
$ cat << EOF >> .hg/hgrc
> [format]
> use-persistent-nodemap=no
> EOF
$ hg debugformat -v
format-variant repo config default
fncache: yes yes yes
Simon Sapin
dirstate-v2: Add `hg debugupgraderepo` command support...
r48111 dirstate-v2: no no no
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 dotencode: yes yes yes
generaldelta: yes yes yes
share-safe: enable by default (BC)...
r49515 share-safe: yes yes yes
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 sparserevlog: yes yes yes
persistent-nodemap: yes no no
copies-sdc: no no no
Raphaël Gomès
revlogv2: allow upgrading to v2...
r47439 revlog-v2: no no no
changelogv2: introduce a "changelogv2" feature...
r48037 changelog-v2: no no no
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 plain-cl-delta: yes yes yes
revlog-compression: use zstd by default (if available)...
r47636 compression: zlib zlib zlib (no-zstd !)
compression: zstd zstd zstd (zstd !)
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 compression-level: default default default
persistent-nodemap: use quiet upgrade in tests...
r48849 $ hg debugupgraderepo --run --no-backup --quiet
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 upgrade will perform the following actions:
requirements
test: enable share-safe in test-persistent-nodemap...
r48851 preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd no-dirstate-v2 !)
preserved: dotencode, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd no-dirstate-v2 !)
dirstate-v2: freeze the on-disk format...
r49116 preserved: dotencode, exp-rc-dirstate-v2, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd dirstate-v2 !)
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 removed: persistent-nodemap
upgrade: display the list of processed revlog before proceeding...
r46649 processed revlogs:
- all-filelogs
- changelog
- manifest
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 $ ls -1 .hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
Pulkit Goyal
upgrade: speed up when we have only nodemap to downgrade...
r47276 00changelog-*.nd (glob)
00manifest-*.nd (glob)
undo.backup.00changelog.n
undo.backup.00manifest.n
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 $ hg debugnodemap --metadata
upgrading
$ cat << EOF >> .hg/hgrc
> [format]
> use-persistent-nodemap=yes
> EOF
$ hg debugformat -v
format-variant repo config default
fncache: yes yes yes
Simon Sapin
dirstate-v2: Add `hg debugupgraderepo` command support...
r48111 dirstate-v2: no no no
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 dotencode: yes yes yes
generaldelta: yes yes yes
share-safe: enable by default (BC)...
r49515 share-safe: yes yes yes
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 sparserevlog: yes yes yes
persistent-nodemap: no yes no
copies-sdc: no no no
Raphaël Gomès
revlogv2: allow upgrading to v2...
r47439 revlog-v2: no no no
changelogv2: introduce a "changelogv2" feature...
r48037 changelog-v2: no no no
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 plain-cl-delta: yes yes yes
revlog-compression: use zstd by default (if available)...
r47636 compression: zlib zlib zlib (no-zstd !)
compression: zstd zstd zstd (zstd !)
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 compression-level: default default default
persistent-nodemap: use quiet upgrade in tests...
r48849 $ hg debugupgraderepo --run --no-backup --quiet
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 upgrade will perform the following actions:
requirements
test: enable share-safe in test-persistent-nodemap...
r48851 preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd no-dirstate-v2 !)
preserved: dotencode, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd no-dirstate-v2 !)
dirstate-v2: freeze the on-disk format...
r49116 preserved: dotencode, exp-rc-dirstate-v2, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd dirstate-v2 !)
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 added: persistent-nodemap
upgrade: display the list of processed revlog before proceeding...
r46649 processed revlogs:
- all-filelogs
- changelog
- manifest
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 $ ls -1 .hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
00changelog-*.nd (glob)
00changelog.n
Pulkit Goyal
upgrade: write nodemap for manifests too...
r47275 00manifest-*.nd (glob)
00manifest.n
Pulkit Goyal
upgrade: speed up when we have only nodemap to downgrade...
r47276 undo.backup.00changelog.n
undo.backup.00manifest.n
upgrade: support upgrade and downgrade from persistent nodemap...
r45356
$ hg debugnodemap --metadata
uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
Running unrelated upgrade
$ hg debugupgraderepo --run --no-backup --quiet --optimize re-delta-all
upgrade will perform the following actions:
requirements
test: enable share-safe in test-persistent-nodemap...
r48851 preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, share-safe, sparserevlog, store (no-zstd no-dirstate-v2 !)
preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd no-dirstate-v2 !)
dirstate-v2: freeze the on-disk format...
r49116 preserved: dotencode, exp-rc-dirstate-v2, fncache, generaldelta, persistent-nodemap, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd dirstate-v2 !)
upgrade: support upgrade and downgrade from persistent nodemap...
r45356
optimisations: re-delta-all
upgrade: display the list of processed revlog before proceeding...
r46649 processed revlogs:
- all-filelogs
- changelog
- manifest
upgrade: support upgrade and downgrade from persistent nodemap...
r45356 $ ls -1 .hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
00changelog-*.nd (glob)
00changelog.n
00manifest-*.nd (glob)
00manifest.n
$ hg debugnodemap --metadata
uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
persistent-nodemap: test that is it present after a clone...
r46865
Persistent nodemap and local/streaming clone
============================================
$ cd ..
standard clone
--------------
The persistent nodemap should exist after a streaming clone
$ hg clone --pull --quiet -U test-repo standard-clone
$ ls -1 standard-clone/.hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
00changelog-*.nd (glob)
00changelog.n
00manifest-*.nd (glob)
00manifest.n
$ hg -R standard-clone debugnodemap --metadata
uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
persistent-nodemap: test it (does not) exist after a local clone...
r46866
local clone
------------
The persistent nodemap should exist after a streaming clone
$ hg clone -U test-repo local-clone
$ ls -1 local-clone/.hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
clone: make sure we warm the cache after a clone...
r47032 00changelog-*.nd (glob)
00changelog.n
00manifest-*.nd (glob)
00manifest.n
persistent-nodemap: test it (does not) exist after a local clone...
r46866 $ hg -R local-clone debugnodemap --metadata
clone: make sure we warm the cache after a clone...
r47032 uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
persistent-nodemap: test it (does not) exist after a stream clone...
r46867
nodemap: test various corruption scenario for the persistent nodemap...
r47732 Test various corruption case
============================
Missing datafile
----------------
Test behavior with a missing datafile
$ hg clone --quiet --pull test-repo corruption-test-repo
$ ls -1 corruption-test-repo/.hg/store/00changelog*
corruption-test-repo/.hg/store/00changelog-*.nd (glob)
corruption-test-repo/.hg/store/00changelog.d
corruption-test-repo/.hg/store/00changelog.i
corruption-test-repo/.hg/store/00changelog.n
$ rm corruption-test-repo/.hg/store/00changelog*.nd
$ hg log -R corruption-test-repo -r .
changeset: 5005:90d5d3ba2fc4
tag: tip
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a2
$ ls -1 corruption-test-repo/.hg/store/00changelog*
corruption-test-repo/.hg/store/00changelog.d
corruption-test-repo/.hg/store/00changelog.i
corruption-test-repo/.hg/store/00changelog.n
Truncated data file
-------------------
Test behavior with a too short datafile
rebuild the missing data
$ hg -R corruption-test-repo debugupdatecache
$ ls -1 corruption-test-repo/.hg/store/00changelog*
corruption-test-repo/.hg/store/00changelog-*.nd (glob)
corruption-test-repo/.hg/store/00changelog.d
corruption-test-repo/.hg/store/00changelog.i
corruption-test-repo/.hg/store/00changelog.n
truncate the file
$ datafilepath=`ls corruption-test-repo/.hg/store/00changelog*.nd`
$ f -s $datafilepath
corruption-test-repo/.hg/store/00changelog-*.nd: size=121088 (glob)
Thomas Klausner
tests: dd status=noxfer is not portable (issue6523)...
r49147 $ dd if=$datafilepath bs=1000 count=10 of=$datafilepath-tmp
Matt Harbison
tests: stabilize test-persistent-nodemap.t on Windows...
r47861 10+0 records in
10+0 records out
Thomas Klausner
tests: dd status=noxfer is not portable (issue6523)...
r49147 * bytes * (glob)
nodemap: test various corruption scenario for the persistent nodemap...
r47732 $ mv $datafilepath-tmp $datafilepath
$ f -s $datafilepath
corruption-test-repo/.hg/store/00changelog-*.nd: size=10000 (glob)
Check that Mercurial reaction to this event
nodemap: deal with data mmap error...
r47733 $ hg -R corruption-test-repo log -r . --traceback
changeset: 5005:90d5d3ba2fc4
tag: tip
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a2
nodemap: test various corruption scenario for the persistent nodemap...
r47732
persistent-nodemap: test it (does not) exist after a stream clone...
r46867 stream clone
nodemap: add a test about racy commit during stream clone...
r47749 ============
persistent-nodemap: test it (does not) exist after a stream clone...
r46867
The persistent nodemap should exist after a streaming clone
nodemap: add a test about racy commit during stream clone...
r47749 Simple case
-----------
No race condition
Valentin Gatien-Baron
tests: rely on dummyssh being the default...
r48732 $ hg clone -U --stream ssh://user@dummy/test-repo stream-clone --debug | egrep '00(changelog|manifest)'
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00manifest.n (62 bytes)
persistent-nodemap: also exchange the nodemap data over the wire...
r47040 adding [s] 00manifest-*.nd (118 KB) (glob)
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00changelog.n (62 bytes)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00changelog-*.nd (118 KB) (glob)
adding [s] 00manifest.d (452 KB) (no-zstd !)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 adding [s] 00manifest.d (491 KB) (zstd no-bigendian !)
adding [s] 00manifest.d (492 KB) (zstd bigendian !)
revlog-compression: use zstd by default (if available)...
r47636 adding [s] 00changelog.d (360 KB) (no-zstd !)
adding [s] 00changelog.d (368 KB) (zstd !)
store: also return some information about the type of file `walk` found...
r47657 adding [s] 00manifest.i (313 KB)
adding [s] 00changelog.i (313 KB)
persistent-nodemap: test it (does not) exist after a stream clone...
r46867 $ ls -1 stream-clone/.hg/store/ | egrep '00(changelog|manifest)(\.n|-.*\.nd)'
clone: make sure we warm the cache after a clone...
r47032 00changelog-*.nd (glob)
00changelog.n
00manifest-*.nd (glob)
00manifest.n
persistent-nodemap: test it (does not) exist after a stream clone...
r46867 $ hg -R stream-clone debugnodemap --metadata
clone: make sure we warm the cache after a clone...
r47032 uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
nodemap: add a test about racy commit during stream clone...
r47749
new data appened
-----------------
Other commit happening on the server during the stream clone
setup the step-by-step stream cloning
$ HG_TEST_STREAM_WALKED_FILE_1="$TESTTMP/sync_file_walked_1"
$ export HG_TEST_STREAM_WALKED_FILE_1
$ HG_TEST_STREAM_WALKED_FILE_2="$TESTTMP/sync_file_walked_2"
$ export HG_TEST_STREAM_WALKED_FILE_2
$ HG_TEST_STREAM_WALKED_FILE_3="$TESTTMP/sync_file_walked_3"
$ export HG_TEST_STREAM_WALKED_FILE_3
$ cat << EOF >> test-repo/.hg/hgrc
> [extensions]
> steps=$RUNTESTDIR/testlib/ext-stream-clone-steps.py
> EOF
Check and record file state beforehand
$ f --size test-repo/.hg/store/00changelog*
test-repo/.hg/store/00changelog-*.nd: size=121088 (glob)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 test-repo/.hg/store/00changelog.d: size=376891 (zstd no-bigendian !)
test-repo/.hg/store/00changelog.d: size=376889 (zstd bigendian !)
nodemap: add a test about racy commit during stream clone...
r47749 test-repo/.hg/store/00changelog.d: size=368890 (no-zstd !)
test-repo/.hg/store/00changelog.i: size=320384
persistent-nodemap: use the intended uuid size...
r48092 test-repo/.hg/store/00changelog.n: size=62
nodemap: add a test about racy commit during stream clone...
r47749 $ hg -R test-repo debugnodemap --metadata | tee server-metadata.txt
uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
Prepare a commit
$ echo foo >> test-repo/foo
$ hg -R test-repo/ add test-repo/foo
Do a mix of clone and commit at the same time so that the file listed on disk differ at actual transfer time.
Valentin Gatien-Baron
tests: rely on dummyssh being the default...
r48732 $ (hg clone -U --stream ssh://user@dummy/test-repo stream-clone-race-1 --debug 2>> clone-output | egrep '00(changelog|manifest)' >> clone-output; touch $HG_TEST_STREAM_WALKED_FILE_3) &
nodemap: add a test about racy commit during stream clone...
r47749 $ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_1
$ hg -R test-repo/ commit -m foo
$ touch $HG_TEST_STREAM_WALKED_FILE_2
$ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_3
$ cat clone-output
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00manifest.n (62 bytes)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00manifest-*.nd (118 KB) (glob)
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00changelog.n (62 bytes)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00changelog-*.nd (118 KB) (glob)
nodemap: add a test about racy commit during stream clone...
r47749 adding [s] 00manifest.d (452 KB) (no-zstd !)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 adding [s] 00manifest.d (491 KB) (zstd no-bigendian !)
adding [s] 00manifest.d (492 KB) (zstd bigendian !)
nodemap: add a test about racy commit during stream clone...
r47749 adding [s] 00changelog.d (360 KB) (no-zstd !)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00changelog.d (368 KB) (zstd !)
adding [s] 00manifest.i (313 KB)
adding [s] 00changelog.i (313 KB)
nodemap: add a test about racy commit during stream clone...
r47749
Check the result state
$ f --size stream-clone-race-1/.hg/store/00changelog*
streamclone: treat volatile file as "fullfile"...
r47751 stream-clone-race-1/.hg/store/00changelog-*.nd: size=121088 (glob)
stream-clone-race-1/.hg/store/00changelog.d: size=368890 (no-zstd !)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 stream-clone-race-1/.hg/store/00changelog.d: size=376891 (zstd no-bigendian !)
stream-clone-race-1/.hg/store/00changelog.d: size=376889 (zstd bigendian !)
streamclone: treat volatile file as "fullfile"...
r47751 stream-clone-race-1/.hg/store/00changelog.i: size=320384
persistent-nodemap: use the intended uuid size...
r48092 stream-clone-race-1/.hg/store/00changelog.n: size=62
nodemap: add a test about racy commit during stream clone...
r47749
$ hg -R stream-clone-race-1 debugnodemap --metadata | tee client-metadata.txt
streamclone: treat volatile file as "fullfile"...
r47751 uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
data-unused: 0
data-unused: 0.000%
nodemap: add a test about racy commit during stream clone...
r47749
We get a usable nodemap, so no rewrite would be needed and the metadata should be identical
(ie: the following diff should be empty)
streamclone: treat volatile file as "fullfile"...
r47751 This isn't the case for the `no-rust` `no-pure` implementation as it use a very minimal nodemap implementation that unconditionnaly rewrite the nodemap "all the time".
#if no-rust no-pure
nodemap: add a test about racy commit during stream clone...
r47749 $ diff -u server-metadata.txt client-metadata.txt
streamclone: treat volatile file as "fullfile"...
r47751 --- server-metadata.txt * (glob)
+++ client-metadata.txt * (glob)
@@ -1,4 +1,4 @@
-uid: * (glob)
+uid: * (glob)
tip-rev: 5005
tip-node: 90d5d3ba2fc47db50f712570487cb261a68c8ffe
data-length: 121088
nodemap: add a test about racy commit during stream clone...
r47749 [1]
streamclone: treat volatile file as "fullfile"...
r47751 #else
$ diff -u server-metadata.txt client-metadata.txt
#endif
nodemap: add a test about racy commit during stream clone...
r47749
Clean up after the test.
$ rm -f "$HG_TEST_STREAM_WALKED_FILE_1"
$ rm -f "$HG_TEST_STREAM_WALKED_FILE_2"
$ rm -f "$HG_TEST_STREAM_WALKED_FILE_3"
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750
full regeneration
-----------------
A full nodemap is generated
(ideally this test would append enough data to make sure the nodemap data file
get changed, however to make thing simpler we will force the regeneration for
this test.
Check the initial state
$ f --size test-repo/.hg/store/00changelog*
test-repo/.hg/store/00changelog-*.nd: size=121344 (glob) (rust !)
test-repo/.hg/store/00changelog-*.nd: size=121344 (glob) (pure !)
test-repo/.hg/store/00changelog-*.nd: size=121152 (glob) (no-rust no-pure !)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 test-repo/.hg/store/00changelog.d: size=376950 (zstd no-bigendian !)
test-repo/.hg/store/00changelog.d: size=376948 (zstd bigendian !)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 test-repo/.hg/store/00changelog.d: size=368949 (no-zstd !)
test-repo/.hg/store/00changelog.i: size=320448
persistent-nodemap: use the intended uuid size...
r48092 test-repo/.hg/store/00changelog.n: size=62
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 $ hg -R test-repo debugnodemap --metadata | tee server-metadata-2.txt
uid: * (glob)
tip-rev: 5006
tip-node: ed2ec1eef9aa2a0ec5057c51483bc148d03e810b
data-length: 121344 (rust !)
streamclone: treat volatile file as "fullfile"...
r47751 data-length: 121344 (pure !)
data-length: 121152 (no-rust no-pure !)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 data-unused: 192 (rust !)
streamclone: treat volatile file as "fullfile"...
r47751 data-unused: 192 (pure !)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 data-unused: 0 (no-rust no-pure !)
streamclone: treat volatile file as "fullfile"...
r47751 data-unused: 0.158% (rust !)
data-unused: 0.158% (pure !)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 data-unused: 0.000% (no-rust no-pure !)
Performe the mix of clone and full refresh of the nodemap, so that the files
(and filenames) are different between listing time and actual transfer time.
Valentin Gatien-Baron
tests: rely on dummyssh being the default...
r48732 $ (hg clone -U --stream ssh://user@dummy/test-repo stream-clone-race-2 --debug 2>> clone-output-2 | egrep '00(changelog|manifest)' >> clone-output-2; touch $HG_TEST_STREAM_WALKED_FILE_3) &
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 $ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_1
$ rm test-repo/.hg/store/00changelog.n
$ rm test-repo/.hg/store/00changelog-*.nd
$ hg -R test-repo/ debugupdatecache
$ touch $HG_TEST_STREAM_WALKED_FILE_2
$ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_3
walk: no longer ignore revlogs of files starting with `undo.` (issue6542)...
r48459
(note: the stream clone code wronly pick the `undo.` files)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 $ cat clone-output-2
walk: no longer ignore revlogs of files starting with `undo.` (issue6542)...
r48459 adding [s] undo.backup.00manifest.n (62 bytes) (known-bad-output !)
adding [s] undo.backup.00changelog.n (62 bytes) (known-bad-output !)
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00manifest.n (62 bytes)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00manifest-*.nd (118 KB) (glob)
persistent-nodemap: use the intended uuid size...
r48092 adding [s] 00changelog.n (62 bytes)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00changelog-*.nd (118 KB) (glob)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 adding [s] 00manifest.d (492 KB) (zstd !)
adding [s] 00manifest.d (452 KB) (no-zstd !)
adding [s] 00changelog.d (360 KB) (no-zstd !)
streamclone: treat volatile file as "fullfile"...
r47751 adding [s] 00changelog.d (368 KB) (zstd !)
adding [s] 00manifest.i (313 KB)
adding [s] 00changelog.i (313 KB)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750
Check the result.
$ f --size stream-clone-race-2/.hg/store/00changelog*
streamclone: treat volatile file as "fullfile"...
r47751 stream-clone-race-2/.hg/store/00changelog-*.nd: size=121344 (glob) (rust !)
stream-clone-race-2/.hg/store/00changelog-*.nd: size=121344 (glob) (pure !)
stream-clone-race-2/.hg/store/00changelog-*.nd: size=121152 (glob) (no-rust no-pure !)
Julien Cristau
tests: allow for variation in zstd output as seen on s390x and powerpc
r49353 stream-clone-race-2/.hg/store/00changelog.d: size=376950 (zstd no-bigendian !)
stream-clone-race-2/.hg/store/00changelog.d: size=376948 (zstd bigendian !)
streamclone: treat volatile file as "fullfile"...
r47751 stream-clone-race-2/.hg/store/00changelog.d: size=368949 (no-zstd !)
stream-clone-race-2/.hg/store/00changelog.i: size=320448
persistent-nodemap: use the intended uuid size...
r48092 stream-clone-race-2/.hg/store/00changelog.n: size=62
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750
$ hg -R stream-clone-race-2 debugnodemap --metadata | tee client-metadata-2.txt
streamclone: treat volatile file as "fullfile"...
r47751 uid: * (glob)
tip-rev: 5006
tip-node: ed2ec1eef9aa2a0ec5057c51483bc148d03e810b
data-length: 121344 (rust !)
data-unused: 192 (rust !)
data-unused: 0.158% (rust !)
data-length: 121152 (no-rust no-pure !)
data-unused: 0 (no-rust no-pure !)
data-unused: 0.000% (no-rust no-pure !)
data-length: 121344 (pure !)
data-unused: 192 (pure !)
data-unused: 0.158% (pure !)
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750
We get a usable nodemap, so no rewrite would be needed and the metadata should be identical
(ie: the following diff should be empty)
streamclone: treat volatile file as "fullfile"...
r47751 This isn't the case for the `no-rust` `no-pure` implementation as it use a very minimal nodemap implementation that unconditionnaly rewrite the nodemap "all the time".
#if no-rust no-pure
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 $ diff -u server-metadata-2.txt client-metadata-2.txt
streamclone: treat volatile file as "fullfile"...
r47751 --- server-metadata-2.txt * (glob)
+++ client-metadata-2.txt * (glob)
@@ -1,4 +1,4 @@
-uid: * (glob)
+uid: * (glob)
tip-rev: 5006
tip-node: ed2ec1eef9aa2a0ec5057c51483bc148d03e810b
data-length: 121152
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750 [1]
streamclone: treat volatile file as "fullfile"...
r47751 #else
$ diff -u server-metadata-2.txt client-metadata-2.txt
#endif
nodemap: add a test about nodemap "vacuum" during stream clone...
r47750
Clean up after the test
$ rm -f $HG_TEST_STREAM_WALKED_FILE_1
$ rm -f $HG_TEST_STREAM_WALKED_FILE_2
$ rm -f $HG_TEST_STREAM_WALKED_FILE_3