##// END OF EJS Templates
match: match explicit file using a set...
match: match explicit file using a set The matcher as all the logic to do quick comparison against explicit patterns, however the pattern matcher was shadowing the code using that set and used the compiled regex pattern in all cases, which is quite slow. We restore the usage of the set based matching to boost performance. Building the regexp is still consuming a large amount of time (actually, the majority of the time), which is still silly. Maybe using re2 would help that, but this is a quest for another adventure. Another path to improve this is to have a pattern type dedicated to match the exact path to a file only (not a directory). This pattern could use the set matching only and be skipped in the regex all together. Benchmarks ========== In the following benchmark we are comparing the `hg cat` and `hg files` run time when matching against all files in the repository. They are run: - without the rust extensions - with the standard python engine (so without re2) Performance improvement in this series -------------------------------------- ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.230092 seconds prev-changeset: 0.230069 seconds this-changeset: 0.211425 seconds (-8.36%) ### mercurial-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.234235 seconds prev-changeset: 0.231165 seconds (-1.38%) this-changeset: 0.212300 seconds (-9.43%) ### pypy-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.613567 seconds prev-changeset: 0.616799 seconds this-changeset: 0.510852 seconds (-16.82%) ### pypy-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.801880 seconds prev-changeset: 0.616393 seconds (-23.22%) this-changeset: 0.511903 seconds (-36.23%) ### netbeans-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 21.541828 seconds prev-changeset: 21.586773 seconds this-changeset: 13.648347 seconds (-36.76%) ### netbeans-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 172.759857 seconds prev-changeset: 21.908197 seconds (-87.32%) this-changeset: 13.945110 seconds (-91.93%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 62.474221 seconds prev-changeset: 61.279490 seconds (-1.22%) this-changeset: 29.529469 seconds (-52.40%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 1364.180218 seconds prev-changeset: 62.473549 seconds (-95.40%) this-changeset: 30.625249 seconds (-97.75%) ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.764407 seconds prev-changeset: 0.763883 seconds this-changeset: 0.737326 seconds (-3.68%) ### mercurial-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.768924 seconds prev-changeset: 0.765848 seconds this-changeset: 0.174d0b seconds (-4.44%) ### pypy-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 2.065220 seconds prev-changeset: 2.070498 seconds this-changeset: 1.939482 seconds (-6.08%) ### pypy-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 2.276388 seconds prev-changeset: 2.069197 seconds (-9.15%) this-changeset: 1.931746 seconds (-15.19%) ### netbeans-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 40.967983 seconds prev-changeset: 41.392423 seconds this-changeset: 32.181681 seconds (-22.20%) ### netbeans-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 216.388709 seconds prev-changeset: 41.648689 seconds (-80.88%) this-changeset: 32.580817 seconds (-85.04%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 105.228510 seconds prev-changeset: 103.315670 seconds (-1.23%) this-changeset: 69.416118 seconds (-33.64%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 1448.722784 seconds prev-changeset: 104.369358 seconds (-92.80%) this-changeset: 70.554789 seconds (-95.13%) Different way to list the same data with this revision ------------------------------------------------------ ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog root: 0.119182 seconds glob: 0.120697 seconds (+1.27%) sorted: 0.211425 seconds (+77.40%) shuffled: 0.212300 seconds (+78.13%) ### pypy-2018-08-01-zstd-sparse-revlog root: 0.121986 seconds glob: 0.124822 seconds (+2.32%) sorted: 0.510852 seconds (+318.78%) shuffled: 0.511903 seconds (+319.64%) ### netbeans-2018-08-01-zstd-sparse-revlog root: 0.173984 seconds glob: 0.227203 seconds (+30.59%) sorted: 13.648347 seconds (+7744.59%) shuffled: 13.945110 seconds (+7915.16%) ### mozilla-central-2018-08-01-zstd-sparse-revlog root: 0.366463 seconds glob: 0.491030 seconds (+33.99%) sorted: 29.529469 seconds (+7957.96%) shuffled: 30.625249 seconds (+8256.97%) ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog glob: 0.647471 seconds root: 0.643120 seconds shuffled: 0.174d0b seconds (+13.92%) sorted: 0.737326 seconds (+13.88%) ### mozilla-central-2018-08-01-zstd-sparse-revlog glob: 40.596983 seconds root: 40.129136 seconds shuffled: 70.554789 seconds (+73.79%) sorted: 69.416118 seconds (+70.99%) ### netbeans-2018-08-01-zstd-sparse-revlog glob: 18.777924 seconds root: 18.613905 seconds shuffled: 32.580817 seconds (+73.51%) sorted: 32.181681 seconds (+71.38%) ### pypy-2018-08-01-zstd-sparse-revlog glob: 1.555319 seconds root: 1.536534 seconds shuffled: 1.931746 seconds (+24.20%) sorted: 1.939482 seconds (+24.70%)

File last commit:

r49768:f254fc73 default
r51286:81c7d04f stable
Show More
wsgicgi.py
93 lines | 2.9 KiB | text/x-python | PythonLexer
Eric Hopper
This patch make several WSGI related alterations....
r2506 # hgweb/wsgicgi.py - CGI->WSGI translator
#
# Copyright 2006 Eric Hopper <hopper@omnifarious.org>
#
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Eric Hopper
This patch make several WSGI related alterations....
r2506 #
# This was originally copied from the public domain code at
# http://www.python.org/dev/peps/pep-0333/#the-server-gateway-side
Yuya Nishihara
hgweb: use absolute_import
r27046
Augie Fackler
wsgicgi: un-do some prior porting work that is now wrong...
r37765 import os
Gregory Szorc
py3: manually import getattr where it is needed...
r43359 from ..pycompat import getattr
Augie Fackler
formatting: blacken the codebase...
r43346 from .. import pycompat
Yuya Nishihara
hgweb: use absolute_import
r27046
Augie Fackler
formatting: blacken the codebase...
r43346 from ..utils import procutil
Yuya Nishihara
procutil: bulk-replace util.std* to point to new module
r37137
Augie Fackler
formatting: blacken the codebase...
r43346 from . import common
Eric Hopper
This patch make several WSGI related alterations....
r2506
def launch(application):
Yuya Nishihara
procutil: bulk-replace function calls to point to new module
r37138 procutil.setbinary(procutil.stdin)
procutil.setbinary(procutil.stdout)
Eric Hopper
This patch make several WSGI related alterations....
r2506
Gregory Szorc
global: bulk replace simple pycompat.iteritems(x) with x.items()...
r49768 environ = dict(os.environ.items()) # re-exports
Matt Harbison
wsgicgi: switch the default PATH_INFO back to str...
r47529 environ.setdefault('PATH_INFO', '')
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 if environ.get('SERVER_SOFTWARE', '').startswith('Microsoft-IIS'):
Mads Kiilerich
fix trivial spelling errors
r17424 # IIS includes script_name in PATH_INFO
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 scriptname = environ['SCRIPT_NAME']
if environ['PATH_INFO'].startswith(scriptname):
environ['PATH_INFO'] = environ['PATH_INFO'][len(scriptname) :]
Dirkjan Ochtman
hgweb: support broken IIS 5 behavior with .cgi in PATH_INFO
r7406
Yuya Nishihara
procutil: bulk-replace util.std* to point to new module
r37137 stdin = procutil.stdin
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 if environ.get('HTTP_EXPECT', '').lower() == '100-continue':
Yuya Nishihara
procutil: bulk-replace util.std* to point to new module
r37137 stdin = common.continuereader(stdin, procutil.stdout.write)
Augie Fackler
hgweb: add support for 100-continue as recommended by PEP 333.
r13570
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 environ['wsgi.input'] = stdin
environ['wsgi.errors'] = procutil.stderr
environ['wsgi.version'] = (1, 0)
environ['wsgi.multithread'] = False
environ['wsgi.multiprocess'] = True
environ['wsgi.run_once'] = True
Eric Hopper
This patch make several WSGI related alterations....
r2506
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 if environ.get('HTTPS', 'off').lower() in ('on', '1', 'yes'):
environ['wsgi.url_scheme'] = 'https'
Eric Hopper
This patch make several WSGI related alterations....
r2506 else:
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 environ['wsgi.url_scheme'] = 'http'
Eric Hopper
This patch make several WSGI related alterations....
r2506
headers_set = []
headers_sent = []
Yuya Nishihara
procutil: bulk-replace util.std* to point to new module
r37137 out = procutil.stdout
Eric Hopper
This patch make several WSGI related alterations....
r2506
def write(data):
if not headers_set:
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 raise AssertionError(b"write() before start_response()")
Eric Hopper
This patch make several WSGI related alterations....
r2506
elif not headers_sent:
Thomas Arendsen Hein
white space and line break cleanups
r3673 # Before the first output, send the stored headers
status, response_headers = headers_sent[:] = headers_set
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 out.write(b'Status: %s\r\n' % pycompat.bytesurl(status))
Augie Fackler
wsgicgi: un-do some prior porting work that is now wrong...
r37765 for hk, hv in response_headers:
Augie Fackler
formatting: blacken the codebase...
r43346 out.write(
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 b'%s: %s\r\n'
Augie Fackler
formatting: blacken the codebase...
r43346 % (pycompat.bytesurl(hk), pycompat.bytesurl(hv))
)
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 out.write(b'\r\n')
Eric Hopper
This patch make several WSGI related alterations....
r2506
Alexis S. L. Carvalho
hgweb: fix unbundle....
r2558 out.write(data)
out.flush()
Eric Hopper
This patch make several WSGI related alterations....
r2506
Thomas Arendsen Hein
white space and line break cleanups
r3673 def start_response(status, response_headers, exc_info=None):
Eric Hopper
This patch make several WSGI related alterations....
r2506 if exc_info:
try:
if headers_sent:
# Re-raise original exception if headers sent
Peter Ruibal
use Exception(args)-style raising consistently (py3k compatibility)
r7008 raise exc_info[0](exc_info[1], exc_info[2])
Eric Hopper
This patch make several WSGI related alterations....
r2506 finally:
Matt Harbison
hgweb: delete a local variable instead of setting to `None`...
r44445 del exc_info # avoid dangling circular ref
Eric Hopper
This patch make several WSGI related alterations....
r2506 elif headers_set:
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 raise AssertionError(b"Headers already set!")
Eric Hopper
This patch make several WSGI related alterations....
r2506
Thomas Arendsen Hein
white space and line break cleanups
r3673 headers_set[:] = [status, response_headers]
Eric Hopper
This patch make several WSGI related alterations....
r2506 return write
Dirkjan Ochtman
hgweb: fix WSGI iterators handling in CGI adapter (issue1254)
r6922 content = application(environ, start_response)
Konstantin Zemlyak
wsgicgi: call close() on iterable to avoid resource leaks...
r10753 try:
for chunk in content:
write(chunk)
Mads Kiilerich
hgweb.cgi: fix internal WSGI emulation (issue3804)...
r18552 if not headers_sent:
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 write(b'') # send headers now if body was empty
Konstantin Zemlyak
wsgicgi: call close() on iterable to avoid resource leaks...
r10753 finally:
Alex Gaynor
style: never use a space before a colon or comma...
r34487 getattr(content, 'close', lambda: None)()