##// END OF EJS Templates
purge: use absolute_import
purge: use absolute_import

File last commit:

r28317:8de70574 default
r28382:27996f78 default
Show More
test-run-tests.t
760 lines | 20.1 KiB | text/troff | Tads3Lexer
Pierre-Yves David
test: introduce test-run-tests.t...
r21732 This file tests the behavior of run-tests.py itself.
Pierre-Yves David
test-run-test: unset run-test specific environment variables...
r24960 Avoid interference from actual test env:
$ unset HGTEST_JOBS
$ unset HGTEST_TIMEOUT
$ unset HGTEST_PORT
$ unset HGTEST_SHELL
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 Smoke test with install
Pierre-Yves David
test: introduce test-run-tests.t...
r21732 ============
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ run-tests.py $HGTEST_RUN_TESTS_PURE -l
Pierre-Yves David
test: introduce test-run-tests.t...
r21732
# Ran 0 tests, 0 skipped, 0 warned, 0 failed.
Pierre-Yves David
test-run-tests.t: test running a passing test
r21734
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 Define a helper to avoid the install step
=============
$ rt()
> {
> run-tests.py --with-hg=`which hg` "$@"
> }
timeless
run-tests: warn about symlinks to non hg scripts...
r28037 error paths
#if symlink
$ ln -s `which true` hg
$ run-tests.py --with-hg=./hg
warning: --with-hg should specify an hg script
# Ran 0 tests, 0 skipped, 0 warned, 0 failed.
$ rm hg
#endif
#if execbit
$ touch hg
$ run-tests.py --with-hg=./hg
Usage: run-tests.py [options] [tests]
run-tests.py: error: --with-hg must specify an executable hg script
[2]
$ rm hg
#endif
Pierre-Yves David
test-run-tests.t: test running a passing test
r21734 a succesful test
=======================
$ cat > test-success.t << EOF
> $ echo babar
> babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 > $ echo xyzzy
Matt Mackall
tests: add (?) flag for optional lines...
r25388 > never happens (?)
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 > xyzzy
Matt Mackall
tests: add (?) flag for optional lines...
r25388 > nor this (?)
Matt Harbison
run-tests: defer leftover (?) cleanup until after all output is exhausted...
r28317 > $ printf 'abc\ndef\nxyz\n'
> 123 (?)
> abc
> def (?)
> 456 (?)
> xyz
Pierre-Yves David
test-run-tests.t: test running a passing test
r21734 > EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt
Pierre-Yves David
test-run-tests.t: test running a passing test
r21734 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738
failing test
==================
$ cat > test-failure.t << EOF
> $ echo babar
> rataxes
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 > This is a noop statement so that
> this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 > pad pad pad pad............................................................
> pad pad pad pad............................................................
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738 > EOF
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 >>> fh = open('test-failure-unicode.t', 'wb')
Augie Fackler
test-run-tests.t: work around file.write() returning an int...
r25054 >>> fh.write(u' $ echo babar\u03b1\n'.encode('utf-8')) and None
>>> fh.write(u' l\u03b5\u03b5t\n'.encode('utf-8')) and None
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738
Matt Harbison
tests: fix globs for Windows...
r23348 --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738 $ echo babar
- rataxes
+ babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738
ERROR: test-failure.t output changed
!.
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 --- $TESTTMP/test-failure-unicode.t
+++ $TESTTMP/test-failure-unicode.t.err
@@ -1,2 +1,2 @@
$ echo babar\xce\xb1 (esc)
- l\xce\xb5\xce\xb5t (esc)
+ babar\xce\xb1 (esc)
ERROR: test-failure-unicode.t output changed
!
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738 Failed test-failure.t: output changed
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 Failed test-failure-unicode.t: output changed
# Ran 3 tests, 0 skipped, 0 warned, 2 failed.
Pierre-Yves David
test-run-tests.t: test running a failing test
r21738 python hash seed: * (glob)
[1]
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500
Augie Fackler
run-tests: add support for xunit test reports...
r22044 test --xunit support
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --xunit=xunit.xml
Augie Fackler
run-tests: add support for xunit test reports...
r22044
--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Augie Fackler
run-tests: add support for xunit test reports...
r22044 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Augie Fackler
run-tests: add support for xunit test reports...
r22044
ERROR: test-failure.t output changed
!.
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 --- $TESTTMP/test-failure-unicode.t
+++ $TESTTMP/test-failure-unicode.t.err
@@ -1,2 +1,2 @@
$ echo babar\xce\xb1 (esc)
- l\xce\xb5\xce\xb5t (esc)
+ babar\xce\xb1 (esc)
ERROR: test-failure-unicode.t output changed
!
Augie Fackler
run-tests: add support for xunit test reports...
r22044 Failed test-failure.t: output changed
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 Failed test-failure-unicode.t: output changed
# Ran 3 tests, 0 skipped, 0 warned, 2 failed.
Augie Fackler
run-tests: add support for xunit test reports...
r22044 python hash seed: * (glob)
[1]
$ cat xunit.xml
<?xml version="1.0" encoding="utf-8"?>
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
Augie Fackler
run-tests: add support for xunit test reports...
r22044 <testcase name="test-success.t" time="*"/> (glob)
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 <testcase name="test-failure-unicode.t" time="*"> (glob)
<![CDATA[--- $TESTTMP/test-failure-unicode.t
+++ $TESTTMP/test-failure-unicode.t.err
@@ -1,2 +1,2 @@
$ echo babar\xce\xb1 (esc)
- l\xce\xb5\xce\xb5t (esc)
+ babar\xce\xb1 (esc)
]]> </testcase>
Augie Fackler
run-tests: add support for xunit test reports...
r22044 <testcase name="test-failure.t" time="*"> (glob)
<![CDATA[--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Augie Fackler
run-tests: add support for xunit test reports...
r22044 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Augie Fackler
run-tests: add support for xunit test reports...
r22044 ]]> </testcase>
</testsuite>
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741
Gregory Szorc
run-tests: explicitly handle unicode when writing xunit file...
r24500 $ rm test-failure-unicode.t
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741 test for --retest
====================
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --retest
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741
Matt Harbison
tests: fix globs for Windows...
r23348 --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741 $ echo babar
- rataxes
+ babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741
ERROR: test-failure.t output changed
!
Failed test-failure.t: output changed
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
Pierre-Yves David
test-run-tests.t: tests the --retest option
r21741 python hash seed: * (glob)
[1]
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742
Selecting Tests To Run
======================
successful
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-success.t
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 success w/ keyword
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt -k xyzzy
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 .
# Ran 2 tests, 1 skipped, 0 warned, 0 failed.
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742 failed
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-failure.t
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742
Matt Harbison
tests: fix globs for Windows...
r23348 --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742 $ echo babar
- rataxes
+ babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Pierre-Yves David
test-run-tests.t: test selection of testfile from the command line
r21742
ERROR: test-failure.t output changed
!
Failed test-failure.t: output changed
# Ran 1 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
Pierre-Yves David
test-run-tests.t: test --debug option
r21743
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 failure w/ keyword
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt -k rataxes
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997
ERROR: test-failure.t output changed
!
Failed test-failure.t: output changed
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 python hash seed: * (glob)
[1]
Augie Fackler
test-run-tests: add a test for detection of failure to start a server...
r22840 Verify that when a process fails to start we show a useful message
==================================================================
$ cat > test-serve-fail.t <<EOF
> $ echo 'abort: child process failed to start blah'
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-serve-fail.t
Augie Fackler
test-run-tests: add a test for detection of failure to start a server...
r22840
ERROR: test-serve-fail.t output changed
!
Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
timeless
run-tests: avoid double counting server fails
r27567 # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
Augie Fackler
test-run-tests: add a test for detection of failure to start a server...
r22840 python hash seed: * (glob)
[1]
$ rm test-serve-fail.t
timeless
run-tests: fix get port to try differing ports...
r27602 Verify that we can try other ports
===================================
$ hg init inuse
$ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
$ cat blocks.pid >> $DAEMON_PIDS
$ cat > test-serve-inuse.t <<EOF
> $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
> $ cat hg.pid >> \$DAEMON_PIDS
> EOF
$ rt test-serve-inuse.t
.
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
$ rm test-serve-inuse.t
Pierre-Yves David
test-run-tests.t: test --debug option
r21743 Running In Debug Mode
======================
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --debug 2>&1 | grep -v pwd
Thomas Klausner
tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
r23676 + echo *SALT* 0 0 (glob)
*SALT* 0 0 (glob)
Pierre-Yves David
test-run-tests.t: test --debug option
r21743 + echo babar
babar
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 + echo *SALT* 6 0 (glob)
*SALT* 6 0 (glob)
Matt Harbison
test-run-tests: glob away a --debug run difference on Windows...
r27509 *+ echo *SALT* 0 0 (glob)
Thomas Klausner
tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
r23676 *SALT* 0 0 (glob)
Pierre-Yves David
test-run-tests.t: test --debug option
r21743 + echo babar
babar
Thomas Klausner
tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
r23676 + echo *SALT* 2 0 (glob)
*SALT* 2 0 (glob)
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 + echo xyzzy
xyzzy
Matt Mackall
tests: add (?) flag for optional lines...
r25388 + echo *SALT* 6 0 (glob)
*SALT* 6 0 (glob)
Matt Harbison
run-tests: defer leftover (?) cleanup until after all output is exhausted...
r28317 + printf *abc\ndef\nxyz\n* (glob)
abc
def
xyz
+ echo *SALT* 12 0 (glob)
*SALT* 12 0 (glob)
Pierre-Yves David
test-run-tests.t: test --debug option
r21743 .
# Ran 2 tests, 0 skipped, 0 warned, 0 failed.
Pierre-Yves David
test-run-tests.t: tests --jobs option
r21744
Parallel runs
==============
(duplicate the failing test to get predictable output)
$ cp test-failure.t test-failure-copy.t
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --jobs 2 test-failure*.t -n
Matt Mackall
tests: silence output race in test-run-tests.t
r23107 !!
Pierre-Yves David
test-run-tests.t: tests --jobs option
r21744 Failed test-failure*.t: output changed (glob)
Failed test-failure*.t: output changed (glob)
# Ran 2 tests, 0 skipped, 0 warned, 2 failed.
python hash seed: * (glob)
[1]
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838 failures in parallel with --first should only print one failure
>>> f = open('test-nothing.t', 'w')
Augie Fackler
test-run-tests.t: work around file.write() returning an int...
r25054 >>> f.write('foo\n' * 1024) and None
>>> f.write(' $ sleep 1') and None
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --jobs 2 --first
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838
--- $TESTTMP/test-failure*.t (glob)
+++ $TESTTMP/test-failure*.t.err (glob)
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838
Failed test-failure*.t: output changed (glob)
Gregory Szorc
run-tests: wait for test threads after first error...
r24507 Failed test-nothing.t: output changed
# Ran 2 tests, 0 skipped, 0 warned, 2 failed.
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838 python hash seed: * (glob)
[1]
Pierre-Yves David
test-run-tests.t: tests --jobs option
r21744 (delete the duplicated test file)
Augie Fackler
run-tests: handle --jobs and --first gracefully...
r22838 $ rm test-failure-copy.t test-nothing.t
Pierre-Yves David
test-run-tests.t: tests --jobs option
r21744
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
Interactive run
===============
(backup the failing test)
$ cp test-failure.t backup
Refuse the fix
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ echo 'n' | rt -i
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 $ echo babar
- rataxes
+ babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Matt Mackall
run-tests: hold iolock across diff/prompt when interactive...
r21763 Accept this change? [n]
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 ERROR: test-failure.t output changed
Matt Mackall
run-tests: hold iolock across diff/prompt when interactive...
r21763 !.
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
$ cat test-failure.t
$ echo babar
rataxes
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
pad pad pad pad............................................................
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
Gregory Szorc
run-tests: make --interactive work with --view
r22361 Interactive with custom view
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ echo 'n' | rt -i --view echo
Matt Harbison
tests: fix globs for Windows...
r23348 $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
Gregory Szorc
run-tests: make --interactive work with --view
r22361 Accept this change? [n]* (glob)
ERROR: test-failure.t output changed
!.
Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
Matt Mackall
test-run-tests: test --view
r22108 View the fix
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ echo 'y' | rt --view echo
Matt Harbison
tests: fix globs for Windows...
r23348 $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
Matt Mackall
test-run-tests: test --view
r22108
ERROR: test-failure.t output changed
!.
Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
Accept the fix
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ echo " $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
$ echo " saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
$ echo " $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
$ echo " saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
$ echo " $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
$ echo " saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ echo 'y' | rt -i 2>&1
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,11 +1,11 @@
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 $ echo babar
- rataxes
+ babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
pad pad pad pad............................................................
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ echo 'saved backup bundle to $TESTTMP/foo.hg'
- saved backup bundle to $TESTTMP/foo.hg
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 + saved backup bundle to $TESTTMP/foo.hg* (glob)
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ echo 'saved backup bundle to $TESTTMP/foo.hg'
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 saved backup bundle to $TESTTMP/foo.hg* (glob)
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ echo 'saved backup bundle to $TESTTMP/foo.hg'
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 Accept this change? [n] ..
# Ran 2 tests, 0 skipped, 0 warned, 0 failed.
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ sed -e 's,(glob)$,&<,g' test-failure.t
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755 $ echo babar
babar
Augie Fackler
test-run-tests.t: add extra data to tests for keyword tests...
r21995 This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
pad pad pad pad............................................................
Mads Kiilerich
run-tests: automatically add (glob) to "saved backup bundle to" lines...
r23728 $ echo 'saved backup bundle to $TESTTMP/foo.hg'
saved backup bundle to $TESTTMP/foo.hg (glob)<
$ echo 'saved backup bundle to $TESTTMP/foo.hg'
saved backup bundle to $TESTTMP/foo.hg (glob)<
$ echo 'saved backup bundle to $TESTTMP/foo.hg'
saved backup bundle to $TESTTMP/*.hg (glob)<
Pierre-Yves David
test-run-tests.t: test the --interactive option
r21755
(reinstall)
$ mv backup test-failure.t
Pierre-Yves David
test-run-tests.t: test the --nodiff option
r21756
No Diff
===============
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --nodiff
Pierre-Yves David
test-run-tests.t: test the --nodiff option
r21756 !.
Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
anuraggoel
run-tests: '--time' option provide more details to Linux users...
r21977
timeless@mozdev.org
run-tests: report paths saved by --keep-tmpdir
r26422 test --tmpdir support
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --tmpdir=$TESTTMP/keep test-success.t
timeless@mozdev.org
run-tests: report paths saved by --keep-tmpdir
r26422
Matt Harbison
test-run-tests: add globs for Windows...
r26432 Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t (glob)
Keeping threadtmp dir: $TESTTMP/keep/child1 (glob)
timeless@mozdev.org
run-tests: report paths saved by --keep-tmpdir
r26422 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
timeless
run-tests: add --slowtimeout and use it for slow tests
r27141 timeouts
========
$ cat > test-timeout.t <<EOF
> $ sleep 2
> $ echo pass
> pass
> EOF
> echo '#require slow' > test-slow-timeout.t
> cat test-timeout.t >> test-slow-timeout.t
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
Matt Mackall
run-tests: report timeouts in a less alarming fashion...
r27393 st
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-slow-timeout.t: missing feature: allow slow tests
timeless
run-tests: add --slowtimeout and use it for slow tests
r27141 Failed test-timeout.t: timed out
# Ran 1 tests, 1 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --timeout=1 --slowtimeout=3 \
timeless
run-tests: add --slowtimeout and use it for slow tests
r27141 > test-timeout.t test-slow-timeout.t --allow-slow-tests
Matt Mackall
run-tests: report timeouts in a less alarming fashion...
r27393 .t
timeless
run-tests: add --slowtimeout and use it for slow tests
r27141 Failed test-timeout.t: timed out
# Ran 2 tests, 0 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
$ rm test-timeout.t test-slow-timeout.t
anuraggoel
run-tests: '--time' option provide more details to Linux users...
r21977 test for --time
==================
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-success.t --time
anuraggoel
run-tests: '--time' option provide more details to Linux users...
r21977 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
# Producing time report
Pierre-Yves David
run-tests: include 'start' and 'end' in --time output...
r25098 start end cuser csys real Test
\s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re)
anuraggoel
run-tests: '--time' option provide more details to Linux users...
r21977
test for --time with --job enabled
====================================
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-success.t --time --jobs 2
anuraggoel
run-tests: '--time' option provide more details to Linux users...
r21977 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
# Producing time report
Pierre-Yves David
run-tests: include 'start' and 'end' in --time output...
r25098 start end cuser csys real Test
\s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re)
Augie Fackler
test-run-tests.t: add tests for skips...
r21996
Skips
================
$ cat > test-skip.t <<EOF
> $ echo xyzzy
Matt Mackall
run-tests: add #require to abort full test...
r22045 > #require false
Augie Fackler
test-run-tests.t: add tests for skips...
r21996 > EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --nodiff
Augie Fackler
test-run-tests.t: add tests for skips...
r21996 !.s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-skip.t: missing feature: nail clipper
Augie Fackler
test-run-tests.t: add tests for skips...
r21996 Failed test-failure.t: output changed
# Ran 2 tests, 1 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --keyword xyzzy
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 .s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-skip.t: missing feature: nail clipper
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997
Augie Fackler
run-tests: add support for xunit test reports...
r22044 Skips with xml
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --keyword xyzzy \
Augie Fackler
run-tests: add support for xunit test reports...
r22044 > --xunit=xunit.xml
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 .s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-skip.t: missing feature: nail clipper
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
Augie Fackler
run-tests: add support for xunit test reports...
r22044 $ cat xunit.xml
<?xml version="1.0" encoding="utf-8"?>
Matt Mackall
run-tests: don't show 'i' for tests that don't match a keyword
r22107 <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
Augie Fackler
run-tests: add support for xunit test reports...
r22044 <testcase name="test-success.t" time="*"/> (glob)
</testsuite>
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 Missing skips or blacklisted skips don't count as executed:
$ echo test-failure.t > blacklist
Laurent Charignon
run-tests: fix crash when --json and --blacklist are both used (issue5050)...
r27927 $ rt --blacklist=blacklist --json\
Augie Fackler
run-tests: fix test result counts with --keyword specified or skips occurring...
r21997 > test-failure.t test-bogus.t
ss
Skipped test-bogus.t: Doesn't exist
Skipped test-failure.t: blacklisted
# Ran 0 tests, 2 skipped, 0 warned, 0 failed.
Laurent Charignon
run-tests: fix crash when --json and --blacklist are both used (issue5050)...
r27927 $ cat report.json
testreport ={
"test-bogus.t": {
"result": "skip"
},
"test-failure.t": {
"result": "skip"
}
} (no-eol)
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 test for --json
==================
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt --json
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391
--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391
ERROR: test-failure.t output changed
!.s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-skip.t: missing feature: nail clipper
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 Failed test-failure.t: output changed
# Ran 2 tests, 1 skipped, 0 warned, 1 failed.
python hash seed: * (glob)
[1]
$ cat report.json
testreport ={
"test-failure.t": [\{] (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "---.+\+\+\+.+", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "result": "failure", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "time": "\s*[\d\.]{4,5}" (re)
}, ? (re)
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 "test-skip.t": {
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "result": "skip", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "time": "\s*[\d\.]{4,5}" (re)
}, ? (re)
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 "test-success.t": [\{] (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "result": "success", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Augie Fackler
test-run-tests: accept more levels of precision and trailing ws (issue4440)...
r23245 "time": "\s*[\d\.]{4,5}" (re)
anuraggoel
run-tests: added '--json' functionality to store test result in json file...
r22391 }
} (no-eol)
Pierre-Yves David
test: protect the run-tests.py --json test behind an hghave rule...
r22579
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 Test that failed test accepted through interactive are properly reported:
$ cp test-failure.t backup
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ echo y | rt --json -i
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979
--- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 @@ -1,5 +1,5 @@
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 $ echo babar
- rataxes
+ babar
This is a noop statement so that
this test is still more bytes than success.
Matt Harbison
test-run-tests: pad the failure test to preserve the run order...
r28316 pad pad pad pad............................................................
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 Accept this change? [n] ..s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-skip.t: missing feature: nail clipper
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
$ cat report.json
testreport ={
"test-failure.t": [\{] (re)
"csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "result": "success", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "time": "\s*[\d\.]{4,5}" (re)
}, ? (re)
"test-skip.t": {
"csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "result": "skip", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "time": "\s*[\d\.]{4,5}" (re)
}, ? (re)
"test-success.t": [\{] (re)
"csys": "\s*[\d\.]{4,5}", ? (re)
"cuser": "\s*[\d\.]{4,5}", ? (re)
Laurent Charignon
run-tests: add 'diff' entry in json report...
r27686 "diff": "", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "end": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "result": "success", ? (re)
Pierre-Yves David
run-tests: track start and end time of tests...
r25097 "start": "\s*[\d\.]{4,5}", ? (re)
Pierre-Yves David
run-test: add a test for json output when -i is used...
r24979 "time": "\s*[\d\.]{4,5}" (re)
}
} (no-eol)
$ mv backup test-failure.t
Gregory Szorc
run-tests: don't error when glob matched line ends with backslash...
r24811 backslash on end of line with glob matching is handled properly
$ cat > test-glob-backslash.t << EOF
> $ echo 'foo bar \\'
> foo * \ (glob)
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt test-glob-backslash.t
Gregory Szorc
run-tests: don't error when glob matched line ends with backslash...
r24811 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
$ rm -f test-glob-backslash.t
FUJIWARA Katsunori
run-tests.py: execute hghave by the path relative to run-tests.py...
r25728 Test reusability for third party tools
======================================
$ mkdir "$TESTTMP"/anothertests
$ cd "$TESTTMP"/anothertests
test that `run-tests.py` can execute hghave, even if it runs not in
Mercurial source tree.
$ cat > test-hghave.t <<EOF
> #require true
> $ echo foo
> foo
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt $HGTEST_RUN_TESTS_PURE test-hghave.t
FUJIWARA Katsunori
run-tests.py: execute hghave by the path relative to run-tests.py...
r25728 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
FUJIWARA Katsunori
run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial...
r25729
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
running is placed.
$ cat > test-runtestdir.t <<EOF
> - $TESTDIR, in which test-run-tests.t is placed
> - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
> - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
>
Matt Harbison
test-run-tests: conditionalize the $TESTDIR check for Windows separator...
r27057 > #if windows
> $ test "\$TESTDIR" = "$TESTTMP\anothertests"
> #else
FUJIWARA Katsunori
run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial...
r25729 > $ test "\$TESTDIR" = "$TESTTMP"/anothertests
Matt Harbison
test-run-tests: conditionalize the $TESTDIR check for Windows separator...
r27057 > #endif
FUJIWARA Katsunori
run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial...
r25729 > $ test "\$RUNTESTDIR" = "$TESTDIR"
> $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py
> #!/usr/bin/env python
> #
> # check-code - a style and portability checker for Mercurial
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt $HGTEST_RUN_TESTS_PURE test-runtestdir.t
FUJIWARA Katsunori
run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial...
r25729 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
FUJIWARA Katsunori
run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR...
r25730
#if execbit
test that TESTDIR is referred in PATH
$ cat > custom-command.sh <<EOF
> #!/bin/sh
> echo "hello world"
> EOF
$ chmod +x custom-command.sh
$ cat > test-testdir-path.t <<EOF
> $ custom-command.sh
> hello world
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt $HGTEST_RUN_TESTS_PURE test-testdir-path.t
FUJIWARA Katsunori
run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR...
r25730 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
#endif
Augie Fackler
run-tests: add support for marking tests as very slow...
r26109
test support for --allow-slow-tests
$ cat > test-very-slow-test.t <<EOF
> #require slow
> $ echo pass
> pass
> EOF
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt $HGTEST_RUN_TESTS_PURE test-very-slow-test.t
Augie Fackler
run-tests: add support for marking tests as very slow...
r26109 s
timeless
run-tests: report missing feature for skipped tests
r27564 Skipped test-very-slow-test.t: missing feature: allow slow tests
Augie Fackler
run-tests: add support for marking tests as very slow...
r26109 # Ran 0 tests, 1 skipped, 0 warned, 0 failed.
Matt Mackall
tests: avoid duplicate install steps in test-run-tests...
r27395 $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
Augie Fackler
run-tests: add support for marking tests as very slow...
r26109 .
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
David R. MacIver
run-tests: allow run-tests.py to run tests outside current directory...
r28180
support for running a test outside the current directory
$ mkdir nonlocal
$ cat > nonlocal/test-is-not-here.t << EOF
> $ echo pass
> pass
> EOF
$ rt nonlocal/test-is-not-here.t
.
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.