##// END OF EJS Templates
store: introduce _matchtrackedpath() and use it to filter store files...
store: introduce _matchtrackedpath() and use it to filter store files This patch introduces a function to filter store files on the basis of the path which they are tracking. The function assumes that the entries can be of two types, 'meta/*' and 'data/*' which means it will just work on revlog based storage and not with another storage ways. For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the beginning and the end then pass that to matcher. For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from beginning and end then call matcher.visitdir() with it to make sure all the parent directories are also downloaded. Since the storage filtering for narrow stream clones is implemented with this patch, we remove the un-implemented error message, add some more tests and add the treemanifest case to tests too. The tests demonstrate that it works correctly. After this patch, we have now narrow stream clones working. Narrow stream clones are a very important feature for large repositories who have good internet connection because they use streamclones for cloning and if they do normal narrow clone, that takes more time then a full streamclone. Also narrow-stream clone will drastically speed up clone timings. Differential Revision: https://phab.mercurial-scm.org/D5139

File last commit:

r40462:30a7d3b6 default
r40529:9aeb9e2d default
Show More
test-narrow-widen-no-ellipsis.t
425 lines | 9.5 KiB | text/troff | Tads3Lexer
/ tests / test-narrow-widen-no-ellipsis.t
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 #testcases tree flat
$ . "$TESTDIR/narrow-library.sh"
#if tree
$ cat << EOF >> $HGRCPATH
> [experimental]
> treemanifest = 1
> EOF
#endif
$ hg init master
$ cd master
$ mkdir inside
$ echo 'inside' > inside/f
$ hg add inside/f
$ hg commit -m 'add inside'
$ mkdir widest
$ echo 'widest' > widest/f
$ hg add widest/f
$ hg commit -m 'add widest'
$ mkdir outside
$ echo 'outside' > outside/f
$ hg add outside/f
$ hg commit -m 'add outside'
$ cd ..
narrow clone the inside file
Martin von Zweigbergk
tests: add test for widening from an empty clone...
r40379 $ hg clone --narrow ssh://user@dummy/master narrow
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 requesting all changes
adding changesets
adding manifests
adding file changes
Martin von Zweigbergk
tests: add test for widening from an empty clone...
r40379 added 3 changesets with 0 changes to 0 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 new changesets *:* (glob)
updating to branch default
Martin von Zweigbergk
tests: add test for widening from an empty clone...
r40379 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ cd narrow
$ hg tracked
Martin von Zweigbergk
tests: add test for widening from an empty clone...
r40379 $ hg files
[1]
widen from an empty clone
$ hg tracked --addinclude inside
comparing with ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
adding file changes
added 0 changesets with 1 changes to 1 files
$ hg tracked
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 I path:inside
$ ls
inside
$ cat inside/f
inside
$ cd ..
add more upstream files which we will include in a wider narrow spec
$ cd master
$ mkdir wider
$ echo 'wider' > wider/f
$ hg add wider/f
$ echo 'widest v2' > widest/f
$ hg commit -m 'add wider, update widest'
$ echo 'widest v3' > widest/f
$ hg commit -m 'update widest v3'
$ echo 'inside v2' > inside/f
$ hg commit -m 'update inside'
$ mkdir outside2
$ echo 'outside2' > outside2/f
$ hg add outside2/f
$ hg commit -m 'add outside2'
$ echo 'widest v4' > widest/f
$ hg commit -m 'update widest v4'
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
7: update widest v4
6: add outside2
5: update inside
4: update widest v3
3: add wider, update widest
2: add outside
1: add widest
0: add inside
$ cd ..
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 Widen the narrow spec to see the widest file. This should not get the newly
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 added upstream revisions.
$ cd narrow
Pulkit Goyal
narrow: don't do the dirstate dance if ellipses is not enabled...
r39999 $ hg id -n
2
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 $ hg tracked --addinclude widest/f --debug
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 comparing with ssh://user@dummy/master
Matt Harbison
tests: glob over some quoting differences in test-narrow-widen-no-ellipsis.t
r39745 running python "*dummyssh" *user@dummy* *hg -R master serve --stdio* (glob)
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 sending hello command
sending between command
remote: * (glob)
remote: capabilities: * (glob)
remote: 1
sending protocaps command
query 1; heads
sending batch command
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 searching for changes
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 all local heads known remotely
Pulkit Goyal
narrow: the first version of narrow_widen wireprotocol command...
r40106 sending narrow_widen command
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 bundle2-input-bundle: with-transaction
bundle2-input-part: "changegroup" (params: * mandatory) supported (glob)
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 adding changesets
adding manifests
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 adding widest/ revisions (tree !)
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 adding file changes
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 adding widest/f revisions (tree !)
Pulkit Goyal
narrow: use diffmatcher to send only new filelogs in non-ellipses widening...
r39701 added 0 changesets with 1 changes to 1 files
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 bundle2-input-part: total payload size * (glob)
Pulkit Goyal
narrow: the first version of narrow_widen wireprotocol command...
r40106 bundle2-input-bundle: 0 parts total
Pulkit Goyal
tests: add debug output in test-narrow-widen-no-ellipsis.t...
r39707 widest/f: add from widened narrow clone -> g
getting widest/f
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg tracked
I path:inside
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 I path:widest/f
$ cat widest/f
widest
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399
Pulkit Goyal
narrow: don't do the dirstate dance if ellipses is not enabled...
r39999 $ hg id -n
2
Pulkit Goyal
tests: show that adding an already included path still calls narrow_widen()...
r40461 Test that extending already included files should not call narrow_widen
wireprotocol command
$ hg tracked --addinclude widest/f
Pulkit Goyal
narrow: rework logic to check whether we need to widen and narrow...
r40462 nothing to widen or narrow
Pulkit Goyal
tests: show that adding an already included path still calls narrow_widen()...
r40461
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 Pull down the newly added upstream revision.
$ hg pull
pulling from ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
adding file changes
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 added 5 changesets with 4 changes to 2 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 new changesets *:* (glob)
(run 'hg update' to get a working copy)
$ hg update -r 'desc("add wider")'
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502
$ cat widest/f
widest v2
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399
$ hg update -r 'desc("update inside")'
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat widest/f
widest v3
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ cat inside/f
inside v2
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
7: update widest v4
6: add outside2
5: update inside
4: update widest v3
3: add wider, update widest
2: add outside
1: add widest
0: add inside
Check that widening with a newline fails
$ hg tracked --addinclude 'widest
> '
abort: newlines are not allowed in narrowspec paths
[255]
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 widen the narrow spec to include the wider file
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 $ hg tracked --addinclude wider
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 comparing with ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
adding file changes
Pulkit Goyal
narrow: use diffmatcher to send only new filelogs in non-ellipses widening...
r39701 added 0 changesets with 1 changes to 1 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg tracked
I path:inside
Pulkit Goyal
tests: improve the widening testing in test-narrow-widen*...
r39502 I path:wider
I path:widest/f
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg update 'desc("add widest")'
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ cat widest/f
widest
$ hg update 'desc("add wider, update widest")'
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat wider/f
wider
$ cat widest/f
widest v2
$ hg update 'desc("update widest v3")'
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat widest/f
widest v3
$ hg update 'desc("update widest v4")'
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat widest/f
widest v4
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
7: update widest v4
6: add outside2
5: update inside
4: update widest v3
3: add wider, update widest
2: add outside
1: add widest
0: add inside
separate suite of tests: files from 0-10 modified in changes 0-10. This allows
more obvious precise tests tickling particular corner cases.
$ cd ..
$ hg init upstream
$ cd upstream
$ for x in `$TESTDIR/seq.py 0 10`
> do
> mkdir d$x
> echo $x > d$x/f
> hg add d$x/f
> hg commit -m "add d$x/f"
> done
$ hg log -T "{rev}: {desc}\n"
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
make narrow clone with every third node.
$ cd ..
$ hg clone --narrow ssh://user@dummy/upstream narrow2 --include d0 --include d3 --include d6 --include d9
requesting all changes
adding changesets
adding manifests
adding file changes
added 11 changesets with 4 changes to 4 files
new changesets *:* (glob)
updating to branch default
4 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd narrow2
$ hg tracked
I path:d0
I path:d3
I path:d6
I path:d9
$ hg verify
checking changesets
checking manifests
checking directory manifests (tree !)
crosschecking files in changesets and manifests
checking files
Meirambek Omyrzak
verify: make output less confusing (issue5924)...
r39525 checked 11 changesets with 4 changes to 4 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
$ hg tracked --addinclude d1
comparing with ssh://user@dummy/upstream
searching for changes
adding changesets
adding manifests
adding file changes
Pulkit Goyal
narrow: use diffmatcher to send only new filelogs in non-ellipses widening...
r39701 added 0 changesets with 1 changes to 1 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg tracked
I path:d0
I path:d1
I path:d3
I path:d6
I path:d9
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
Verify shouldn't claim the repo is corrupt after a widen.
$ hg verify
checking changesets
checking manifests
checking directory manifests (tree !)
crosschecking files in changesets and manifests
checking files
Meirambek Omyrzak
verify: make output less confusing (issue5924)...
r39525 checked 11 changesets with 5 changes to 5 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399
Widening preserves parent of local commit
$ cd ..
$ hg clone -q --narrow ssh://user@dummy/upstream narrow3 --include d2 -r 2
$ cd narrow3
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
2: add d2/f
1: add d1/f
0: add d0/f
$ hg pull -q -r 3
$ hg co -q tip
$ hg pull -q -r 4
$ echo local > d2/f
$ hg ci -m local
created new head
$ hg tracked -q --addinclude d0 --addinclude d9
Widening preserves bookmarks
$ cd ..
$ hg clone -q --narrow ssh://user@dummy/upstream narrow-bookmarks --include d4
$ cd narrow-bookmarks
$ echo local > d4/f
$ hg ci -m local
$ hg bookmarks bookmark
$ hg bookmarks
* bookmark 11:* (glob)
$ hg -q tracked --addinclude d2
$ hg bookmarks
* bookmark 11:* (glob)
$ hg log -r bookmark -T '{desc}\n'
local
Widening that fails can be recovered from
$ cd ..
$ hg clone -q --narrow ssh://user@dummy/upstream interrupted --include d0
$ cd interrupted
$ echo local > d0/f
$ hg ci -m local
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
11: local
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
$ hg bookmarks bookmark
$ hg --config hooks.pretxnchangegroup.bad=false tracked --addinclude d1
comparing with ssh://user@dummy/upstream
searching for changes
adding changesets
adding manifests
adding file changes
Pulkit Goyal
narrow: use diffmatcher to send only new filelogs in non-ellipses widening...
r39701 added 0 changesets with 1 changes to 1 files
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 $ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
11: local
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
$ hg bookmarks
* bookmark 11:* (glob)
$ hg unbundle .hg/strip-backup/*-widen.hg
Matt Harbison
tests: conditionalize narrow-widen error output for Windows
r39428 abort: .hg/strip-backup/*-widen.hg: $ENOTDIR$ (windows !)
abort: $ENOENT$: .hg/strip-backup/*-widen.hg (no-windows !)
Pulkit Goyal
tests: rename test-narrow-widen-non-ellipsis to match current names...
r39399 [255]
$ hg log -T "{if(ellipsis, '...')}{rev}: {desc}\n"
11: local
10: add d10/f
9: add d9/f
8: add d8/f
7: add d7/f
6: add d6/f
5: add d5/f
4: add d4/f
3: add d3/f
2: add d2/f
1: add d1/f
0: add d0/f
$ hg bookmarks
* bookmark 11:* (glob)