##// END OF EJS Templates
run-tests: mechanism to report exceptions during test execution...
run-tests: mechanism to report exceptions during test execution Sometimes when running tests you introduce a ton of exceptions. The most extreme example of this is running Mercurial with Python 3, which currently spews thousands of exceptions when running the test harness. This commit adds an opt-in feature to run-tests.py to aggregate exceptions encountered by `hg` when running tests. When --exceptions is used, the test harness enables the "logexceptions" extension in the test environment. This extension wraps the Mercurial function to handle exceptions and writes information about the exception to a random filename in a directory defined by the test harness via an environment variable. At the end of the test harness, these files are parsed, aggregated, and a list of all unique Mercurial frames triggering exceptions is printed in order of frequency. This feature is intended to aid Python 3 development. I've only really tested it on Python 3. There is no shortage of improvements that could be made. e.g. we could write a separate file containing the exception report - maybe even an HTML report. We also don't capture which tests demonstrate the exceptions, so there's no turnkey way to test whether a code change made an exception disappear. Perfect is the enemy of good. I think the current patch is useful enough to land. Whoever uses it can send patches to imprve its usefulness. Differential Revision: https://phab.mercurial-scm.org/D1477

File last commit:

r25007:7c324f65 default
r35191:bd8875b6 default
Show More
test-bisect3.t
249 lines | 6.0 KiB | text/troff | Tads3Lexer
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 # Here we create a simple DAG which has just enough of the required
# topology to test all the bisection status labels:
#
# 13--14
# /
# 0--1--2--3---------9--10--11--12
# \ /
# 4--5--6--7--8
$ hg init
$ echo '0' >a
$ hg add a
$ hg ci -u test -d '0 0' -m '0'
$ echo '1' >a
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '1 0' -m '1'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
branch 2-3
$ echo '2' >b
$ hg add b
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '2 0' -m '2'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '3' >b
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '3 0' -m '3'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
branch 4-8
$ hg up -r 1
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo '4' >c
$ hg add c
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '4 0' -m '4'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 created new head
$ echo '5' >c
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '5 0' -m '5'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '6' >c
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '6 0' -m '6'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '7' >c
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '7 0' -m '7'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '8' >c
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '8 0' -m '8'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
merge
$ hg merge -r 3
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '9 0' -m '9=8+3'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
$ echo '10' >a
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '10 0' -m '10'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '11' >a
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '11 0' -m '11'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 $ echo '12' >a
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '12 0' -m '12'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
unrelated branch
$ hg up -r 3
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo '13' >d
$ hg add d
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '13 0' -m '13'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 created new head
$ echo '14' >d
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 $ hg ci -u test -d '14 0' -m '14'
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155
mark changesets
$ hg bisect --reset
$ hg bisect --good 4
$ hg bisect --good 6
$ hg bisect --bad 12
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 Testing changeset 9:2197c557e14c (6 changesets remaining, ~2 tests)
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg bisect --bad 10
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 Testing changeset 8:e74a86251f58 (4 changesets remaining, ~2 tests)
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg bisect --skip 7
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 Testing changeset 8:e74a86251f58 (4 changesets remaining, ~2 tests)
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
test template
$ hg log --template '{rev}:{node|short} {bisect}\n'
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 14:cbf2f3105bbf
13:e07efca37c43
12:98c6b56349c0 bad
11:03f491376e63 bad (implicit)
10:c012b15e2409 bad
9:2197c557e14c untested
8:e74a86251f58 untested
7:a5f87041c899 skipped
6:7d997bedcd8d good
5:2dd1875f1028 good (implicit)
4:2a1daef14cd4 good
3:8417d459b90c ignored
2:e1355ee1f23e ignored
1:ce7c85e06a9f good (implicit)
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 0:b4e73ffab476 good (implicit)
$ hg log --template '{bisect|shortbisect} {rev}:{node|short}\n'
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 14:cbf2f3105bbf
13:e07efca37c43
B 12:98c6b56349c0
B 11:03f491376e63
B 10:c012b15e2409
U 9:2197c557e14c
U 8:e74a86251f58
S 7:a5f87041c899
G 6:7d997bedcd8d
G 5:2dd1875f1028
G 4:2a1daef14cd4
I 3:8417d459b90c
I 2:e1355ee1f23e
G 1:ce7c85e06a9f
"Yann E. MORIN"
templates: add 'bisect' keyword to return a cset's bisect status...
r15155 G 0:b4e73ffab476
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156
test style
$ hg log --style bisect
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 14:cbf2f3105bbf
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect:
tag: tip
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:14 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 14
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 13:e07efca37c43
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect:
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 parent: 3:8417d459b90c
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:13 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 13
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 12:98c6b56349c0
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: bad
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:12 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 12
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 11:03f491376e63
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: bad (implicit)
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:11 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 11
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 10:c012b15e2409
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: bad
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:10 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 10
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 9:2197c557e14c
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: untested
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 parent: 8:e74a86251f58
parent: 3:8417d459b90c
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:09 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 9=8+3
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 8:e74a86251f58
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: untested
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:08 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 8
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 7:a5f87041c899
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: skipped
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:07 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 7
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 6:7d997bedcd8d
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: good
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:06 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 6
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 5:2dd1875f1028
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: good (implicit)
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:05 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 5
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 4:2a1daef14cd4
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: good
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 parent: 1:ce7c85e06a9f
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:04 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 4
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 3:8417d459b90c
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: ignored
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:03 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 3
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 2:e1355ee1f23e
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: ignored
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:02 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 2
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 changeset: 1:ce7c85e06a9f
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 bisect: good (implicit)
user: test
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 date: Thu Jan 01 00:00:01 1970 +0000
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 summary: 1
changeset: 0:b4e73ffab476
bisect: good (implicit)
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: 0
$ hg log --quiet --style bisect
Mads Kiilerich
tests: don't let time go back before 0 o'clock in bisect3.t...
r15438 14:cbf2f3105bbf
13:e07efca37c43
B 12:98c6b56349c0
B 11:03f491376e63
B 10:c012b15e2409
U 9:2197c557e14c
U 8:e74a86251f58
S 7:a5f87041c899
G 6:7d997bedcd8d
G 5:2dd1875f1028
G 4:2a1daef14cd4
I 3:8417d459b90c
I 2:e1355ee1f23e
G 1:ce7c85e06a9f
"Yann E. MORIN"
styles: add new 'bisect' style that prints the bisection status...
r15156 G 0:b4e73ffab476
Jordi Gutiérrez Hermoso
map-cmdline.bisect: rewrite to just %include the default template...
r25007
$ hg --config extensions.color= --color=debug log --quiet --style bisect
[log.bisect| ] 14:cbf2f3105bbf
[log.bisect| ] 13:e07efca37c43
[log.bisect bisect.bad|B] 12:98c6b56349c0
[log.bisect bisect.bad|B] 11:03f491376e63
[log.bisect bisect.bad|B] 10:c012b15e2409
[log.bisect bisect.untested|U] 9:2197c557e14c
[log.bisect bisect.untested|U] 8:e74a86251f58
[log.bisect bisect.skipped|S] 7:a5f87041c899
[log.bisect bisect.good|G] 6:7d997bedcd8d
[log.bisect bisect.good|G] 5:2dd1875f1028
[log.bisect bisect.good|G] 4:2a1daef14cd4
[log.bisect bisect.ignored|I] 3:8417d459b90c
[log.bisect bisect.ignored|I] 2:e1355ee1f23e
[log.bisect bisect.good|G] 1:ce7c85e06a9f
[log.bisect bisect.good|G] 0:b4e73ffab476