##// END OF EJS Templates
demandimport: replace more references to _demandmod instances...
demandimport: replace more references to _demandmod instances _demandmod instances may be referenced by multiple importing modules. Before this patch, the _demandmod instance only maintained a reference to its first consumer when using the "from X import Y" syntax. This is because we only created a single _demandmod instance (attached to the parent X module). If multiple modules A and B performed "from X import Y", we'd produce a single _demandmod instance "demandmod" with the following references: X.Y = <demandmod> A.Y = <demandmod> B.Y = <demandmod> The locals from the first consumer (A) would be stored in <demandmod1>. When <demandmod1> was loaded, we'd look at the locals for the first consumer and replace the symbol, if necessary. This resulted in state: X.Y = <module> A.Y = <module> B.Y = <demandmod> B's reference to Y wasn't updated and was still using the proxy object because we just didn't record that B had a reference to <demandmod> that needed updating! With this patch, we add support for tracking which modules in addition to the initial importer have a reference to the _demandmod instance and we replace those references at module load time. In the case of posix.py, this fixes an issue where the "encoding" module was being proxied, resulting in hundreds of thousands of __getattribute__ lookups on the _demandmod instance during dirstate operations on mozilla-central, speeding up execution by many milliseconds. There are likely several other operation that benefit from this change as well. The new mechanism isn't perfect: references in locals (not globals) may likely linger. So, if there is an import inside a function and a symbol from that module is used in a hot loop, we could have unwanted overhead from proxying through _demandmod. Non-global imports are discouraged anyway. So hopefully this isn't a big deal in practice. We could potentially deploy a code checker that bans use of attribute lookups of function-level-imported modules inside loops. This deficiency in theory could be avoided by storing the set of globals and locals dicts to update in the _demandmod instance. However, I tried this and it didn't work. One reason is that some globals are _demandmod instances. We could work around this, but it's a bit more work. There also might be other module import foo at play. The solution as implemented is better than what we had and IMO is good enough for the time being. It's worth noting that this sub-optimal behavior was made worse by the introduction of absolute_import and its recommended "from . import X" syntax for importing modules from the "mercurial" package. If we ever wrote performance tests, measuring the amount of module imports and __getattribute__ proxy calls through _demandmod instances would be something I'd have it check.

File last commit:

r26200:461e7b70 default
r26457:7e813050 default
Show More
request.py
140 lines | 5.2 KiB | text/x-python | PythonLexer
Eric Hopper
Fixing up comment headers for split up code.
r2391 # hgweb/request.py - An http request from either CGI or the standalone server.
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355 #
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
Vadim Gelfer
update copyrights.
r2859 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355 #
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355
Benoit Boissinot
remove various unused import
r3963 import socket, cgi, errno
Dirkjan Ochtman
hgweb: be sure to drain request data even in early error conditions...
r7180 from mercurial import util
Augie Fackler
hgweb: don't send a body or illegal headers during 304 response...
r12739 from common import ErrorResponse, statusmessage, HTTP_NOT_MODIFIED
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355
Dirkjan Ochtman
hgweb: move shortcut expansion to request instantiation
r6774 shortcuts = {
'cl': [('cmd', ['changelog']), ('rev', None)],
'sl': [('cmd', ['shortlog']), ('rev', None)],
'cs': [('cmd', ['changeset']), ('node', None)],
'f': [('cmd', ['file']), ('filenode', None)],
'fl': [('cmd', ['filelog']), ('filenode', None)],
'fd': [('cmd', ['filediff']), ('node', None)],
'fa': [('cmd', ['annotate']), ('filenode', None)],
'mf': [('cmd', ['manifest']), ('manifest', None)],
'ca': [('cmd', ['archive']), ('node', None)],
'tags': [('cmd', ['tags'])],
'tip': [('cmd', ['changeset']), ('node', ['tip'])],
'static': [('cmd', ['static']), ('file', None)]
}
Nicolas Dumazet
hgweb: request: strip() form values...
r10261 def normalize(form):
# first expand the shortcuts
Dirkjan Ochtman
hgweb: move shortcut expansion to request instantiation
r6774 for k in shortcuts.iterkeys():
if k in form:
for name, value in shortcuts[k]:
if value is None:
value = form[k]
form[name] = value
del form[k]
Nicolas Dumazet
hgweb: request: strip() form values...
r10261 # And strip the values
for k, v in form.iteritems():
form[k] = [i.strip() for i in v]
Dirkjan Ochtman
hgweb: move shortcut expansion to request instantiation
r6774 return form
Dirkjan Ochtman
Less indirection in the WSGI web interface. This simplifies some code, and makes it more compliant with WSGI.
r5566 class wsgirequest(object):
Gregory Szorc
hgweb: add some documentation...
r26132 """Higher-level API for a WSGI request.
WSGI applications are invoked with 2 arguments. They are used to
instantiate instances of this class, which provides higher-level APIs
for obtaining request parameters, writing HTTP output, etc.
"""
Dirkjan Ochtman
Less indirection in the WSGI web interface. This simplifies some code, and makes it more compliant with WSGI.
r5566 def __init__(self, wsgienv, start_response):
Eric Hopper
This patch make several WSGI related alterations....
r2506 version = wsgienv['wsgi.version']
Thomas Arendsen Hein
white space and line break cleanups
r3673 if (version < (1, 0)) or (version >= (2, 0)):
Thomas Arendsen Hein
Cleanup of whitespace, indentation and line continuation.
r4633 raise RuntimeError("Unknown and unsupported WSGI version %d.%d"
Eric Hopper
This patch make several WSGI related alterations....
r2506 % version)
self.inp = wsgienv['wsgi.input']
self.err = wsgienv['wsgi.errors']
self.threaded = wsgienv['wsgi.multithread']
self.multiprocess = wsgienv['wsgi.multiprocess']
self.run_once = wsgienv['wsgi.run_once']
self.env = wsgienv
Nicolas Dumazet
hgweb: request: strip() form values...
r10261 self.form = normalize(cgi.parse(self.inp,
self.env,
keep_blank_values=1))
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888 self._start_response = start_response
Dirkjan Ochtman
hgweb: explicit response status
r5993 self.server_write = None
Eric Hopper
This patch make several WSGI related alterations....
r2506 self.headers = []
def __iter__(self):
return iter([])
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355
Vadim Gelfer
push over http: server support....
r2464 def read(self, count=-1):
return self.inp.read(count)
Dirkjan Ochtman
hgweb: be sure to drain request data even in early error conditions...
r7180 def drain(self):
'''need to read all data from request, httplib is half-duplex'''
Dirkjan Ochtman
hgweb: pmezard thinks one default is enough
r13600 length = int(self.env.get('CONTENT_LENGTH') or 0)
Dirkjan Ochtman
hgweb: be sure to drain request data even in early error conditions...
r7180 for s in util.filechunkiter(self.inp, limit=length):
pass
Mads Kiilerich
hgweb: pass the actual response body to request.response, not just the length...
r18352 def respond(self, status, type, filename=None, body=None):
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888 if self._start_response is not None:
Mads Kiilerich
hgweb: simplify wsgirequest header handling...
r18348 self.headers.append(('Content-Type', type))
if filename:
filename = (filename.split('/')[-1]
.replace('\\', '\\\\').replace('"', '\\"'))
self.headers.append(('Content-Disposition',
'inline; filename="%s"' % filename))
Mads Kiilerich
hgweb: pass the actual response body to request.response, not just the length...
r18352 if body is not None:
self.headers.append(('Content-Length', str(len(body))))
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888
Dirkjan Ochtman
hgweb: be sure to send a valid content-type for raw files
r5926 for k, v in self.headers:
if not isinstance(v, str):
Mads Kiilerich
hgweb: simplify wsgirequest header handling...
r18348 raise TypeError('header value must be string: %r' % (v,))
Dirkjan Ochtman
hgweb: be sure to send a valid content-type for raw files
r5926
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888 if isinstance(status, ErrorResponse):
Mads Kiilerich
hgweb: simplify wsgirequest header handling...
r18348 self.headers.extend(status.headers)
Augie Fackler
hgweb: don't send a body or illegal headers during 304 response...
r12739 if status.code == HTTP_NOT_MODIFIED:
# RFC 2616 Section 10.3.5: 304 Not Modified has cases where
# it MUST NOT include any headers other than these and no
# body
self.headers = [(k, v) for (k, v) in self.headers if
k in ('Date', 'ETag', 'Expires',
'Cache-Control', 'Vary')]
timeless@mozdev.org
hgweb: remove ErrorResponse.message...
r26200 status = statusmessage(status.code, str(status))
Dirkjan Ochtman
hgweb: explicit response status
r5993 elif status == 200:
status = '200 Script output follows'
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888 elif isinstance(status, int):
status = statusmessage(status)
self.server_write = self._start_response(status, self.headers)
self._start_response = None
self.headers = []
Mads Kiilerich
hgweb: pass the actual response body to request.response, not just the length...
r18352 if body is not None:
self.write(body)
self.server_write = None
Dirkjan Ochtman
hgweb: separate out start_response() calling
r5888
Dirkjan Ochtman
hgweb: explicit response status
r5993 def write(self, thing):
Mads Kiilerich
hgweb: don't pass empty response chunks on...
r18351 if thing:
try:
self.server_write(thing)
Gregory Szorc
global: mass rewrite to use modern exception syntax...
r25660 except socket.error as inst:
Mads Kiilerich
hgweb: don't pass empty response chunks on...
r18351 if inst[0] != errno.ECONNRESET:
raise
Eric Hopper
Splitting up hgweb so it's easier to change.
r2355
Alexis S. L. Carvalho
avoid _wsgioutputfile <-> _wsgirequest circular reference...
r4246 def writelines(self, lines):
for line in lines:
self.write(line)
def flush(self):
return None
def close(self):
return None
Dirkjan Ochtman
Less indirection in the WSGI web interface. This simplifies some code, and makes it more compliant with WSGI.
r5566 def wsgiapplication(app_maker):
Dirkjan Ochtman
hgweb: return iterable, add deprecation note
r5887 '''For compatibility with old CGI scripts. A plain hgweb() or hgwebdir()
can and should now be used as a WSGI application.'''
Thomas Arendsen Hein
Removed tabs and trailing whitespace in python files
r5760 application = app_maker()
def run_wsgi(env, respond):
Dirkjan Ochtman
hgweb: return iterable, add deprecation note
r5887 return application(env, respond)
Thomas Arendsen Hein
Removed tabs and trailing whitespace in python files
r5760 return run_wsgi