##// END OF EJS Templates
perf: add command for measuring revlog chunk operations...
perf: add command for measuring revlog chunk operations Upcoming commits will teach revlogs to leverage the new compression engine API so that new compression formats can more easily be leveraged in revlogs. We want to be sure this refactoring doesn't regress performance. So this commit introduces "perfrevchunks" to explicitly test performance of reading, decompressing, and recompressing revlog chunks. Here is output when run on the mozilla-unified repo: $ hg perfrevlogchunks -c ! read ! wall 0.346603 comb 0.350000 user 0.340000 sys 0.010000 (best of 28) ! read w/ reused fd ! wall 0.337707 comb 0.340000 user 0.320000 sys 0.020000 (best of 30) ! read batch ! wall 0.013206 comb 0.020000 user 0.000000 sys 0.020000 (best of 221) ! read batch w/ reused fd ! wall 0.013259 comb 0.030000 user 0.010000 sys 0.020000 (best of 222) ! chunk ! wall 1.909939 comb 1.910000 user 1.900000 sys 0.010000 (best of 6) ! chunk batch ! wall 1.750677 comb 1.760000 user 1.740000 sys 0.020000 (best of 6) ! compress ! wall 5.668004 comb 5.670000 user 5.670000 sys 0.000000 (best of 3) $ hg perfrevlogchunks -m ! read ! wall 0.365834 comb 0.370000 user 0.350000 sys 0.020000 (best of 26) ! read w/ reused fd ! wall 0.350160 comb 0.350000 user 0.320000 sys 0.030000 (best of 28) ! read batch ! wall 0.024777 comb 0.020000 user 0.000000 sys 0.020000 (best of 119) ! read batch w/ reused fd ! wall 0.024895 comb 0.030000 user 0.000000 sys 0.030000 (best of 118) ! chunk ! wall 2.514061 comb 2.520000 user 2.480000 sys 0.040000 (best of 4) ! chunk batch ! wall 2.380788 comb 2.380000 user 2.360000 sys 0.020000 (best of 5) ! compress ! wall 9.815297 comb 9.820000 user 9.820000 sys 0.000000 (best of 3) We already see some interesting data, such as how much slower non-batched chunk reading is and that zlib compression appears to be >2x slower than decompression. I didn't have the data when I wrote this commit message, but I ran this on Mozilla's NFS-based Mercurial server and the time for reading with a reused file descriptor was faster. So I think it is worth testing both with and without file descriptor reuse so we can make informed decisions about recycling file descriptors.

File last commit:

r28683:d0210a35 default
r30451:94ca0e13 default
Show More
test-convert-mtn.t
399 lines | 10.6 KiB | text/troff | Tads3Lexer
/ tests / test-convert-mtn.t
Matt Mackall
tests: replace exit 80 with #require
r22046 #require mtn
Matt Mackall
tests: unify test-convert-mtn
r12532
Monotone directory is called .monotone on *nix and monotone
Patrick Mezard
test-convert-mtn.t: use #if to set mtndir to the correct value...
r17449 on Windows.
#if windows
$ mtndir=monotone
#else
Matt Mackall
tests: unify test-convert-mtn
r12532
$ mtndir=.monotone
Patrick Mezard
test-convert-mtn.t: use #if to set mtndir to the correct value...
r17449
#endif
Matt Mackall
tests: unify test-convert-mtn
r12532 $ echo "[extensions]" >> $HGRCPATH
$ echo "convert=" >> $HGRCPATH
Windows version of monotone home
$ APPDATA=$HOME; export APPDATA
tedious monotone keys configuration
The /dev/null redirection is necessary under Windows, or
it complains about home directory permissions
$ mtn --quiet genkey test@selenic.com 1>/dev/null 2>&1 <<EOF
> passphrase
> passphrase
> EOF
$ cat >> $HOME/$mtndir/monotonerc <<EOF
> function get_passphrase(keypair_id)
> return "passphrase"
> end
> EOF
create monotone repository
$ mtn db init --db=repo.mtn
$ mtn --db=repo.mtn --branch=com.selenic.test setup workingdir
$ cd workingdir
$ echo a > a
$ mkdir dir
$ echo b > dir/b
$ echo d > dir/d
Augie Fackler
tests: use $PYTHON instead of hardcoding python...
r22947 $ $PYTHON -c 'file("bin", "wb").write("a\\x00b")'
Matt Mackall
tests: unify test-convert-mtn
r12532 $ echo c > c
$ mtn add a dir/b dir/d c bin
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'a' to workspace manifest
mtn: adding 'bin' to workspace manifest
mtn: adding 'c' to workspace manifest
mtn: adding 'dir' to workspace manifest
mtn: adding 'dir/b' to workspace manifest
mtn: adding 'dir/d' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m initialize
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 0f6e5e4f2e7d2a8ef312408f57618abf026afd90
update monotone working directory
$ mtn mv a dir/a
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: skipping 'dir', already accounted for in workspace
mtn: renaming 'a' to 'dir/a' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ echo a >> dir/a
$ echo b >> dir/b
$ mtn drop c
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: dropping 'c' from workspace manifest
Augie Fackler
tests: use $PYTHON instead of hardcoding python...
r22947 $ $PYTHON -c 'file("bin", "wb").write("b\\x00c")'
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m update1
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 51d0a982464573a2a2cf5ee2c9219c652aaebeff
$ cd ..
convert once
$ hg convert -s mtn repo.mtn
assuming destination repo.mtn-hg
initializing destination repo.mtn-hg repository
scanning source...
sorting...
converting...
1 initialize
0 update1
$ cd workingdir
$ echo e > e
$ mtn add e
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'e' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn drop dir/b
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: dropping 'dir/b' from workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn mv bin bin2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'bin' to 'bin2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m 'update2 "with" quotes'
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision ebe58335d85d8cb176b6d0a12be04f5314b998da
test directory move
$ mkdir -p dir1/subdir1
$ mkdir -p dir1/subdir2_other
$ echo file1 > dir1/subdir1/file1
$ echo file2 > dir1/subdir2_other/file1
$ mtn add dir1/subdir1/file1 dir1/subdir2_other/file1
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'dir1' to workspace manifest
mtn: adding 'dir1/subdir1' to workspace manifest
mtn: adding 'dir1/subdir1/file1' to workspace manifest
mtn: adding 'dir1/subdir2_other' to workspace manifest
mtn: adding 'dir1/subdir2_other/file1' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m createdir1
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision a8d62bc04fee4d2936d28e98bbcc81686dd74306
$ mtn rename dir1/subdir1 dir1/subdir2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: skipping 'dir1', already accounted for in workspace
mtn: renaming 'dir1/subdir1' to 'dir1/subdir2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m movedir1
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 2c3d241bbbfe538b1b51d910f5676407e3f4d3a6
test subdirectory move
$ mtn mv dir dir2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'dir' to 'dir2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ echo newfile > dir2/newfile
$ mtn drop dir2/d
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: dropping 'dir2/d' from workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn add dir2/newfile
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'dir2/newfile' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m movedir
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision fdb5a02dae8bfce3a79b3393680af471016e1b4c
Test directory removal with empty directory
$ mkdir dir2/dir
$ mkdir dir2/dir/subdir
$ echo f > dir2/dir/subdir/f
$ mkdir dir2/dir/emptydir
$ mtn add --quiet -R dir2/dir
$ mtn ci -m emptydir
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 8bbf76d717001d24964e4604739fdcd0f539fc88
$ mtn drop -R dir2/dir
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: dropping 'dir2/dir/subdir/f' from workspace manifest
mtn: dropping 'dir2/dir/subdir' from workspace manifest
mtn: dropping 'dir2/dir/emptydir' from workspace manifest
mtn: dropping 'dir2/dir' from workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m dropdirectory
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 2323d4bc324e6c82628dc04d47a9fd32ad24e322
test directory and file move
$ mkdir -p dir3/d1
$ echo a > dir3/a
$ mtn add dir3/a dir3/d1
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'dir3' to workspace manifest
mtn: adding 'dir3/a' to workspace manifest
mtn: adding 'dir3/d1' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m dirfilemove
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 47b192f720faa622f48c68d1eb075b26d405aa8b
$ mtn mv dir3/a dir3/d1/a
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: skipping 'dir3/d1', already accounted for in workspace
mtn: renaming 'dir3/a' to 'dir3/d1/a' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn mv dir3/d1 dir3/d2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: skipping 'dir3', already accounted for in workspace
mtn: renaming 'dir3/d1' to 'dir3/d2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m dirfilemove2
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 8b543a400d3ee7f6d4bb1835b9b9e3747c8cb632
test directory move into another directory move
$ mkdir dir4
$ mkdir dir5
$ echo a > dir4/a
$ mtn add dir4/a dir5
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'dir4' to workspace manifest
mtn: adding 'dir4/a' to workspace manifest
mtn: adding 'dir5' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m dirdirmove
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 466e0b2afc7a55aa2b4ab2f57cb240bb6cd66fc7
$ mtn mv dir5 dir6
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'dir5' to 'dir6' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn mv dir4 dir6/dir4
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: skipping 'dir6', already accounted for in workspace
mtn: renaming 'dir4' to 'dir6/dir4' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m dirdirmove2
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 3d1f77ebad0c23a5d14911be3b670f990991b749
test diverging directory moves
$ mkdir -p dir7/dir9/dir8
$ echo a > dir7/dir9/dir8/a
$ echo b > dir7/dir9/b
$ echo c > dir7/c
$ mtn add -R dir7
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'dir7' to workspace manifest
mtn: adding 'dir7/c' to workspace manifest
mtn: adding 'dir7/dir9' to workspace manifest
mtn: adding 'dir7/dir9/b' to workspace manifest
mtn: adding 'dir7/dir9/dir8' to workspace manifest
mtn: adding 'dir7/dir9/dir8/a' to workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m divergentdirmove
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 08a08511f18b428d840199b062de90d0396bc2ed
$ mtn mv dir7 dir7-2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'dir7' to 'dir7-2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn mv dir7-2/dir9 dir9-2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'dir7-2/dir9' to 'dir9-2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn mv dir9-2/dir8 dir8-2
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: renaming 'dir9-2/dir8' to 'dir8-2' in workspace manifest
Matt Mackall
tests: unify test-convert-mtn
r12532 $ mtn ci -m divergentdirmove2
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision 4a736634505795f17786fffdf2c9cbf5b11df6f6
Daniel Atallah
convert/mtn: convert suspended branches as closed branches...
r13779
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 test large file support (> 32kB)
Patrick Mezard
test-convert-mtn.t: generate files in binary mode
r17450 >>> fp = file('large-file', 'wb')
>>> for x in xrange(10000): fp.write('%d\n' % x)
>>> fp.close()
Matt Mackall
tests: drop explicit $TESTDIR from executables...
r25472 $ md5sum.py large-file
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 5d6de8a95c3b6bf9e0ffb808ba5299c1 large-file
$ mtn add large-file
Matt Mackall
tests: update monotone output for v1.0 changes
r14394 mtn: adding 'large-file' to workspace manifest
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 $ mtn ci -m largefile
mtn: beginning commit on branch 'com.selenic.test'
mtn: committed revision f0a20fecd10dc4392d18fe69a03f1f4919d3387b
Daniel Atallah
convert/mtn: convert suspended branches as closed branches...
r13779 test suspending (closing a branch)
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 $ mtn suspend f0a20fecd10dc4392d18fe69a03f1f4919d3387b 2> /dev/null
Matt Mackall
tests: unify test-convert-mtn
r12532 $ cd ..
convert incrementally
$ hg convert -s mtn repo.mtn
assuming destination repo.mtn-hg
scanning source...
sorting...
converting...
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 12 update2 "with" quotes
11 createdir1
10 movedir1
9 movedir
8 emptydir
7 dropdirectory
6 dirfilemove
5 dirfilemove2
4 dirdirmove
3 dirdirmove2
2 divergentdirmove
1 divergentdirmove2
0 largefile
Matt Mackall
tests: unify test-convert-mtn
r12532 $ glog()
> {
Martin Geisler
tests: don't load unnecessary graphlog extension...
r20117 > hg log -G --template '{rev} "{desc|firstline}" files: {files}\n' "$@"
Matt Mackall
tests: unify test-convert-mtn
r12532 > }
$ cd repo.mtn-hg
$ hg up -C
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 12 files updated, 0 files merged, 0 files removed, 0 files unresolved
FUJIWARA Katsunori
destutil: make messages at updating to the closed head usual form...
r28683 no open descendant heads on branch "com.selenic.test", updating to a closed head
(committing will reopen branch "com.selenic.test")
Matt Mackall
tests: unify test-convert-mtn
r12532 $ glog
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 @ 14 "largefile" files: large-file
|
o 13 "divergentdirmove2" files: dir7-2/c dir7/c dir7/dir9/b dir7/dir9/dir8/a dir8-2/a dir9-2/b
Matt Mackall
tests: unify test-convert-mtn
r12532 |
o 12 "divergentdirmove" files: dir7/c dir7/dir9/b dir7/dir9/dir8/a
|
o 11 "dirdirmove2" files: dir4/a dir6/dir4/a
|
o 10 "dirdirmove" files: dir4/a
|
o 9 "dirfilemove2" files: dir3/a dir3/d2/a
|
o 8 "dirfilemove" files: dir3/a
|
o 7 "dropdirectory" files: dir2/dir/subdir/f
|
o 6 "emptydir" files: dir2/dir/subdir/f
|
o 5 "movedir" files: dir/a dir/d dir2/a dir2/newfile
|
o 4 "movedir1" files: dir1/subdir1/file1 dir1/subdir2/file1
|
o 3 "createdir1" files: dir1/subdir1/file1 dir1/subdir2_other/file1
|
o 2 "update2 "with" quotes" files: bin bin2 dir/b e
|
o 1 "update1" files: a bin c dir/a dir/b
|
o 0 "initialize" files: a bin c dir/b dir/d
manifest
$ hg manifest
bin2
dir1/subdir2/file1
dir1/subdir2_other/file1
dir2/a
dir2/newfile
dir3/d2/a
dir6/dir4/a
dir7-2/c
dir8-2/a
dir9-2/b
e
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 large-file
Matt Mackall
tests: unify test-convert-mtn
r12532
contents
$ cat dir2/a
a
a
$ test -d dir2/dir && echo 'removed dir2/dir is still there!'
[1]
file move
$ hg log -v -C -r 1 | grep copies
copies: dir/a (a)
check directory move
$ hg manifest -r 4
bin2
dir/a
dir/d
dir1/subdir2/file1
dir1/subdir2_other/file1
e
$ test -d dir1/subdir2 || echo 'new dir1/subdir2 does not exist!'
$ test -d dir1/subdir1 && echo 'renamed dir1/subdir1 is still there!'
[1]
$ hg log -v -C -r 4 | grep copies
copies: dir1/subdir2/file1 (dir1/subdir1/file1)
check file remove with directory move
$ hg manifest -r 5
bin2
dir1/subdir2/file1
dir1/subdir2_other/file1
dir2/a
dir2/newfile
e
check file move with directory move
$ hg manifest -r 9
bin2
dir1/subdir2/file1
dir1/subdir2_other/file1
dir2/a
dir2/newfile
dir3/d2/a
e
check file directory directory move
$ hg manifest -r 11
bin2
dir1/subdir2/file1
dir1/subdir2_other/file1
dir2/a
dir2/newfile
dir3/d2/a
dir6/dir4/a
e
check divergent directory moves
$ hg manifest -r 13
bin2
dir1/subdir2/file1
dir1/subdir2_other/file1
dir2/a
dir2/newfile
dir3/d2/a
dir6/dir4/a
dir7-2/c
dir8-2/a
dir9-2/b
e
Daniel Atallah
convert/mtn: convert suspended branches as closed branches...
r13779
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 test large file support (> 32kB)
Matt Mackall
tests: drop explicit $TESTDIR from executables...
r25472 $ md5sum.py large-file
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 5d6de8a95c3b6bf9e0ffb808ba5299c1 large-file
Daniel Atallah
convert/mtn: convert suspended branches as closed branches...
r13779 check branch closing
$ hg branches -a
$ hg branches -c
Daniel Atallah
convert/mtn: Fix conversion of large files from mtn (broken in ed97955e0c04)...
r13792 com.selenic.test 14:* (closed) (glob)
Daniel Atallah
convert/mtn: convert suspended branches as closed branches...
r13779