##// END OF EJS Templates
lfs: factor out a method for extracting the pointer of a single file...
lfs: factor out a method for extracting the pointer of a single file This will be useful for filesets, among other things, instead of traversing the whole context.

File last commit:

r35684:2c6ebd0c default
r35939:47e737d2 default
Show More
test-lfs-test-server.t
208 lines | 7.4 KiB | text/troff | Tads3Lexer
/ tests / test-lfs-test-server.t
Matt Harbison
hghave: add a check for lfs-test-server...
r35137 #require lfs-test-server
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097
$ LFS_LISTEN="tcp://:$HGPORT"
$ LFS_HOST="localhost:$HGPORT"
$ LFS_PUBLIC=1
$ export LFS_LISTEN LFS_HOST LFS_PUBLIC
Matt Harbison
test-lfs: allow the test server to be killed on Windows...
r35139 #if no-windows
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ lfs-test-server &> lfs-server.log &
$ echo $! >> $DAEMON_PIDS
Matt Harbison
test-lfs: allow the test server to be killed on Windows...
r35139 #else
$ cat >> $TESTTMP/spawn.py <<EOF
> import os
> import subprocess
> import sys
>
> for path in os.environ["PATH"].split(os.pathsep):
> exe = os.path.join(path, 'lfs-test-server.exe')
> if os.path.exists(exe):
> with open('lfs-server.log', 'wb') as out:
> p = subprocess.Popen(exe, stdout=out, stderr=out)
> sys.stdout.write('%s\n' % p.pid)
> sys.exit(0)
> sys.exit(1)
> EOF
$ $PYTHON $TESTTMP/spawn.py >> $DAEMON_PIDS
#endif
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097
$ cat >> $HGRCPATH <<EOF
> [extensions]
> lfs=
> [lfs]
> url=http://foo:bar@$LFS_HOST/
Matt Harbison
lfs: migrate most file filtering from threshold to custom filter...
r35636 > track=all()
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 > EOF
$ hg init repo1
$ cd repo1
$ echo THIS-IS-LFS > a
$ hg commit -m a -A a
Matt Harbison
lfs: use the local store method for opening a blob...
r35544 A push can be serviced directly from the usercache if it isn't in the local
store.
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ hg init ../repo2
Matt Harbison
lfs: use the local store method for opening a blob...
r35544 $ mv .hg/store/lfs .hg/store/lfs_
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ hg push ../repo2 -v
pushing to ../repo2
searching for changes
lfs: uploading 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b (12 bytes)
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b
Matt Harbison
lfs: emit a status message to indicate how many blobs were uploaded...
r35899 lfs: uploaded 1 files (12 bytes)
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 1 changesets found
uncompressed size of bundle content:
* (changelog) (glob)
* (manifests) (glob)
* a (glob)
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
Matt Harbison
lfs: add the 'lfs' requirement in the changegroup transaction introducing lfs...
r35520 calling hook pretxnchangegroup.lfs: hgext.lfs.checkrequireslfs
Matt Harbison
lfs: use the local store method for opening a blob...
r35544 $ mv .hg/store/lfs_ .hg/store/lfs
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097
Matt Harbison
lfs: introduce a user level cache for lfs files...
r35281 Clear the cache to force a download
$ rm -rf `hg config lfs.usercache`
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ cd ../repo2
$ hg update tip -v
resolving manifests
getting a
lfs: downloading 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b (12 bytes)
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: adding 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b to the usercache
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
When the server has some blobs already
$ hg mv a b
$ echo ANOTHER-LARGE-FILE > c
$ echo ANOTHER-LARGE-FILE2 > d
$ hg commit -m b-and-c -A b c d
$ hg push ../repo1 -v | grep -v '^ '
pushing to ../repo1
searching for changes
lfs: need to transfer 2 objects (39 bytes)
lfs: uploading 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 (20 bytes)
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 lfs: uploading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
Matt Harbison
lfs: emit a status message to indicate how many blobs were uploaded...
r35899 lfs: uploaded 2 files (39 bytes)
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 1 changesets found
uncompressed size of bundle content:
adding changesets
adding manifests
adding file changes
added 1 changesets with 3 changes to 3 files
Matt Harbison
lfs: introduce a user level cache for lfs files...
r35281 Clear the cache to force a download
$ rm -rf `hg config lfs.usercache`
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ hg --repo ../repo1 update tip -v
resolving manifests
getting b
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 getting c
lfs: downloading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: adding d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 to the usercache
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: found d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 in the local lfs store
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 getting d
lfs: downloading 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 (20 bytes)
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: adding 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 to the usercache
Wojciech Lis
lfs: using workers in lfs prefetch...
r35449 lfs: processed: 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19
Matt Harbison
lfs: add note messages indicating what store holds the lfs blob...
r35489 lfs: found 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 in the local lfs store
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 3 files updated, 0 files merged, 0 files removed, 0 files unresolved
Matt Harbison
lfs: verify lfs object content when transferring to and from the remote store...
r35492 Test a corrupt file download, but clear the cache first to force a download.
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490
$ rm -rf `hg config lfs.usercache`
$ cp $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 blob
$ echo 'damage' > $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
$ rm ../repo1/.hg/store/lfs/objects/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
$ rm ../repo1/*
Matt Harbison
lfs: verify lfs object content when transferring to and from the remote store...
r35492
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490 $ hg --repo ../repo1 update -C tip -v
resolving manifests
getting a
lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
getting b
lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
getting c
lfs: downloading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
Matt Harbison
lfs: use the localstore download method to transfer from remote stores...
r35566 abort: corrupt remote lfs object: d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490 [255]
Matt Harbison
lfs: verify lfs object content when transferring to and from the remote store...
r35492 The corrupted blob is not added to the usercache or local store
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490
Matt Harbison
lfs: verify lfs object content when transferring to and from the remote store...
r35492 $ test -f ../repo1/.hg/store/lfs/objects/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
[1]
$ test -f `hg config lfs.usercache`/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
[1]
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490 $ cp blob $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
Test a corrupted file upload
$ echo 'another lfs blob' > b
$ hg ci -m 'another blob'
$ echo 'damage' > .hg/store/lfs/objects/e6/59058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0
$ hg push -v ../repo1
pushing to ../repo1
searching for changes
lfs: uploading e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0 (17 bytes)
Matt Harbison
lfs: verify lfs object content when transferring to and from the remote store...
r35492 abort: detected corrupt lfs object: e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0
(run hg verify)
Matt Harbison
test-lfs: add tests around corrupted lfs objects...
r35490 [255]
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 Check error message when the remote missed a blob:
$ echo FFFFF > b
$ hg commit -m b -A b
$ echo FFFFF >> b
$ hg commit -m b b
$ rm -rf .hg/store/lfs
Matt Harbison
lfs: introduce a user level cache for lfs files...
r35281 $ rm -rf `hg config lfs.usercache`
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ hg update -C '.^'
Jun Wu
lfs: remove internal url in test...
r35684 abort: LFS server error. Remote object for "b" not found:(.*)! (re)
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 [255]
Check error message when object does not exist:
Jun Wu
lfs: remove internal url in test...
r35684 $ cd $TESTTMP
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 $ hg init test && cd test
$ echo "[extensions]" >> .hg/hgrc
$ echo "lfs=" >> .hg/hgrc
$ echo "[lfs]" >> .hg/hgrc
$ echo "threshold=1" >> .hg/hgrc
$ echo a > a
$ hg add a
$ hg commit -m 'test'
$ echo aaaaa > a
$ hg commit -m 'largefile'
$ hg debugdata .hg/store/data/a.i 1 # verify this is no the file content but includes "oid", the LFS "pointer".
version https://git-lfs.github.com/spec/v1
oid sha256:bdc26931acfb734b142a8d675f205becf27560dc461f501822de13274fe6fc8a
size 6
x-is-binary 0
$ cd ..
Matt Harbison
lfs: introduce a user level cache for lfs files...
r35281 $ rm -rf `hg config lfs.usercache`
Jun Wu
lfs: remove internal url in test...
r35684
(Restart the server in a different location so it no longer has the content)
$ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
$ rm $DAEMON_PIDS
$ mkdir $TESTTMP/lfs-server2
$ cd $TESTTMP/lfs-server2
#if no-windows
$ lfs-test-server &> lfs-server.log &
$ echo $! >> $DAEMON_PIDS
#else
$ $PYTHON $TESTTMP/spawn.py >> $DAEMON_PIDS
#endif
$ cd $TESTTMP
$ hg clone test test2
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 updating to branch default
Matt Harbison
lfs: improve the error message for a missing remote blob...
r35584 abort: LFS server error. Remote object for "a" not found:(.*)! (re)
Matt Harbison
lfs: import the Facebook git-lfs client extension...
r35097 [255]
Matt Harbison
test-lfs: allow the test server to be killed on Windows...
r35139
$ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS