##// END OF EJS Templates
demandimport: replace more references to _demandmod instances...
demandimport: replace more references to _demandmod instances _demandmod instances may be referenced by multiple importing modules. Before this patch, the _demandmod instance only maintained a reference to its first consumer when using the "from X import Y" syntax. This is because we only created a single _demandmod instance (attached to the parent X module). If multiple modules A and B performed "from X import Y", we'd produce a single _demandmod instance "demandmod" with the following references: X.Y = <demandmod> A.Y = <demandmod> B.Y = <demandmod> The locals from the first consumer (A) would be stored in <demandmod1>. When <demandmod1> was loaded, we'd look at the locals for the first consumer and replace the symbol, if necessary. This resulted in state: X.Y = <module> A.Y = <module> B.Y = <demandmod> B's reference to Y wasn't updated and was still using the proxy object because we just didn't record that B had a reference to <demandmod> that needed updating! With this patch, we add support for tracking which modules in addition to the initial importer have a reference to the _demandmod instance and we replace those references at module load time. In the case of posix.py, this fixes an issue where the "encoding" module was being proxied, resulting in hundreds of thousands of __getattribute__ lookups on the _demandmod instance during dirstate operations on mozilla-central, speeding up execution by many milliseconds. There are likely several other operation that benefit from this change as well. The new mechanism isn't perfect: references in locals (not globals) may likely linger. So, if there is an import inside a function and a symbol from that module is used in a hot loop, we could have unwanted overhead from proxying through _demandmod. Non-global imports are discouraged anyway. So hopefully this isn't a big deal in practice. We could potentially deploy a code checker that bans use of attribute lookups of function-level-imported modules inside loops. This deficiency in theory could be avoided by storing the set of globals and locals dicts to update in the _demandmod instance. However, I tried this and it didn't work. One reason is that some globals are _demandmod instances. We could work around this, but it's a bit more work. There also might be other module import foo at play. The solution as implemented is better than what we had and IMO is good enough for the time being. It's worth noting that this sub-optimal behavior was made worse by the introduction of absolute_import and its recommended "from . import X" syntax for importing modules from the "mercurial" package. If we ever wrote performance tests, measuring the amount of module imports and __getattribute__ proxy calls through _demandmod instances would be something I'd have it check.

File last commit:

r26200:461e7b70 default
r26457:7e813050 default
Show More
common.py
189 lines | 6.4 KiB | text/x-python | PythonLexer
Eric Hopper
Fixing up comment headers for split up code.
r2391 # hgweb/common.py - Utility functions needed by hgweb_mod and hgwebdir_mod
Eric Hopper
Final stage of the hgweb split up....
r2356 #
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
Vadim Gelfer
update copyrights.
r2859 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
Eric Hopper
Final stage of the hgweb split up....
r2356 #
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Eric Hopper
Final stage of the hgweb split up....
r2356
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 import errno, mimetypes, os
Dirkjan Ochtman
hgweb: explicit response status
r5993 HTTP_OK = 200
Dirkjan Ochtman
hgweb: support very simple caching model (issue1845)
r12183 HTTP_NOT_MODIFIED = 304
Dirkjan Ochtman
hgweb: explicit response status
r5993 HTTP_BAD_REQUEST = 400
Dirkjan Ochtman
hgweb: raise ErrorResponses to communicate protocol errors
r6926 HTTP_UNAUTHORIZED = 401
Rocco Rutte
hgweb: Respond with HTTP 403 for disabled archive types instead of 404...
r7029 HTTP_FORBIDDEN = 403
Dirkjan Ochtman
hgweb: explicit response status
r5993 HTTP_NOT_FOUND = 404
Dirkjan Ochtman
hgweb: raise ErrorResponses to communicate protocol errors
r6926 HTTP_METHOD_NOT_ALLOWED = 405
Dirkjan Ochtman
hgweb: explicit response status
r5993 HTTP_SERVER_ERROR = 500
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 def ismember(ui, username, userlist):
"""Check if username is a member of userlist.
If userlist has a single '*' member, all users are considered members.
Mads Kiilerich
spelling: random spell checker fixes
r19951 Can be overridden by extensions to provide more complex authorization
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 schemes.
"""
return userlist == ['*'] or username in userlist
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910 def checkauthz(hgweb, req, op):
'''Check permission for operation based on request data (including
authentication info). Return if op allowed, else raise an ErrorResponse
exception.'''
user = req.env.get('REMOTE_USER')
deny_read = hgweb.configlist('web', 'deny_read')
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 if deny_read and (not user or ismember(hgweb.repo.ui, user, deny_read)):
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910 raise ErrorResponse(HTTP_UNAUTHORIZED, 'read not authorized')
allow_read = hgweb.configlist('web', 'allow_read')
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 if allow_read and (not ismember(hgweb.repo.ui, user, allow_read)):
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910 raise ErrorResponse(HTTP_UNAUTHORIZED, 'read not authorized')
if op == 'pull' and not hgweb.allowpull:
raise ErrorResponse(HTTP_UNAUTHORIZED, 'pull not authorized')
elif op == 'pull' or op is None: # op is None for interface requests
return
# enforce that you can only push using POST requests
if req.env['REQUEST_METHOD'] != 'POST':
msg = 'push requires POST request'
raise ErrorResponse(HTTP_METHOD_NOT_ALLOWED, msg)
# require ssl by default for pushing, auth info cannot be sniffed
# and replayed
scheme = req.env.get('wsgi.url_scheme')
if hgweb.configbool('web', 'push_ssl', True) and scheme != 'https':
Yuya Nishihara
hgweb: respond 403 forbidden for ssl required error...
r17456 raise ErrorResponse(HTTP_FORBIDDEN, 'ssl required')
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910
deny = hgweb.configlist('web', 'deny_push')
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 if deny and (not user or ismember(hgweb.repo.ui, user, deny)):
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910 raise ErrorResponse(HTTP_UNAUTHORIZED, 'push not authorized')
allow = hgweb.configlist('web', 'allow_push')
Wagner Bruna
hgweb: refactor checks for granting and revoking user permissions...
r19032 if not (allow and ismember(hgweb.repo.ui, user, allow)):
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910 raise ErrorResponse(HTTP_UNAUTHORIZED, 'push not authorized')
Martin Geisler
hgweb: initialize permhooks at definition time...
r14058 # Hooks for hgweb permission checks; extensions can add hooks here.
# Each hook is invoked like this: hook(hgweb, request, operation),
# where operation is either read, pull or push. Hooks should either
# raise an ErrorResponse exception, or just return.
#
# It is possible to do both authentication and authorization through
# this.
permhooks = [checkauthz]
Sune Foldager
hgweb: add support for extension-provided permission hooks...
r9910
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 class ErrorResponse(Exception):
Sune Foldager
hgweb: support custom http headers in ErrorResponse
r7741 def __init__(self, code, message=None, headers=[]):
Mads Kiilerich
hgweb: give ErrorResponse a descriptive string/Exception representation...
r13444 if message is None:
message = _statusmessage(code)
timeless@mozdev.org
hgweb: remove ErrorResponse.message...
r26200 Exception.__init__(self, message)
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 self.code = code
Sune Foldager
hgweb: support custom http headers in ErrorResponse
r7741 self.headers = headers
Bryan O'Sullivan
hgweb: fix breaking tests on Python < 2.5
r5563
Augie Fackler
hgweb: add support for 100-continue as recommended by PEP 333.
r13570 class continuereader(object):
def __init__(self, f, write):
self.f = f
self._write = write
self.continued = False
def read(self, amt=-1):
if not self.continued:
self.continued = True
self._write('HTTP/1.1 100 Continue\r\n\r\n')
return self.f.read(amt)
def __getattr__(self, attr):
if attr in ('close', 'readline', 'readlines', '__iter__'):
return getattr(self.f, attr)
Brodie Rao
cleanup: "raise SomeException()" -> "raise SomeException"
r16687 raise AttributeError
Bryan O'Sullivan
hgweb: fix breaking tests on Python < 2.5
r5563
def _statusmessage(code):
from BaseHTTPServer import BaseHTTPRequestHandler
responses = BaseHTTPRequestHandler.responses
return responses.get(code, ('Error', 'Unknown error'))[0]
Thomas Arendsen Hein
Removed tabs and trailing whitespace in python files
r5760
Sune Foldager
hgweb: send proper error messages to the client...
r9694 def statusmessage(code, message=None):
return '%d %s' % (code, message or _statusmessage(code))
Eric Hopper
Final stage of the hgweb split up....
r2356
Pierre-Yves David
hgweb: drop the default argument for get_stat...
r25717 def get_stat(spath, fn):
"""stat fn if it exists, spath otherwise"""
Anton Shestakov
hgweb: refresh hgweb.repo on phase change (issue4061)...
r22577 cl_path = os.path.join(spath, fn)
Benoit Boissinot
switch to the .hg/store layout, fix the tests
r3853 if os.path.exists(cl_path):
Martin Geisler
hgweb: detect change based on changelog size too...
r13958 return os.stat(cl_path)
Eric Hopper
Final stage of the hgweb split up....
r2356 else:
Martin Geisler
hgweb: detect change based on changelog size too...
r13958 return os.stat(spath)
def get_mtime(spath):
Pierre-Yves David
hgweb: drop the default argument for get_stat...
r25717 return get_stat(spath, "00changelog.i").st_mtime
Eric Hopper
Final stage of the hgweb split up....
r2356
Eric Hopper
Really fix http headers for web UI and issue 254....
r2514 def staticfile(directory, fname, req):
Dirkjan Ochtman
send conservatively capitalized HTTP headers
r5930 """return a file inside directory with guessed Content-Type header
Eric Hopper
Final stage of the hgweb split up....
r2356
fname always uses '/' as directory separator and isn't allowed to
contain unusual path components.
Dirkjan Ochtman
send conservatively capitalized HTTP headers
r5930 Content-Type is guessed using the mimetypes module.
Eric Hopper
Final stage of the hgweb split up....
r2356 Return an empty string if fname is illegal or file not found.
"""
parts = fname.split('/')
for part in parts:
if (part in ('', os.curdir, os.pardir) or
os.sep in part or os.altsep is not None and os.altsep in part):
Mads Kiilerich
hgweb: simplify internal staticfile return codes
r18645 return
Brendan Cully
Allow per-file shadowing of static directory in templatepath
r7288 fpath = os.path.join(*parts)
if isinstance(directory, str):
directory = [directory]
for d in directory:
path = os.path.join(d, fpath)
if os.path.exists(path):
break
Eric Hopper
Final stage of the hgweb split up....
r2356 try:
os.stat(path)
ct = mimetypes.guess_type(path)[0] or "text/plain"
Dan Villiom Podlaski Christiansen
explicitly close files...
r13400 fp = open(path, 'rb')
data = fp.read()
fp.close()
Mads Kiilerich
hgweb: pass the actual response body to request.response, not just the length...
r18352 req.respond(HTTP_OK, ct, body=data)
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 except TypeError:
timeless
Generally replace "file name" with "filename" in help and comments.
r8761 raise ErrorResponse(HTTP_SERVER_ERROR, 'illegal filename')
Gregory Szorc
global: mass rewrite to use modern exception syntax...
r25660 except OSError as err:
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 if err.errno == errno.ENOENT:
Dirkjan Ochtman
hgweb: explicit response status
r5993 raise ErrorResponse(HTTP_NOT_FOUND)
Bryan O'Sullivan
hgweb: return meaningful HTTP status codes instead of nonsense
r5561 else:
Dirkjan Ochtman
hgweb: explicit response status
r5993 raise ErrorResponse(HTTP_SERVER_ERROR, err.strerror)
Thomas Arendsen Hein
hgweb: Search templates in templatepath/style/map, too, using a common function....
r3276
Thomas Arendsen Hein
hgweb: use generator to count parity of horizontal stripes for easier reading....
r4462 def paritygen(stripecount, offset=0):
"""count parity of horizontal stripes for easier reading"""
if stripecount and offset:
# account for offset, e.g. due to building the list in reverse
count = (stripecount + offset) % stripecount
parity = (stripecount + offset) / stripecount & 1
else:
count = 0
parity = 0
while True:
yield parity
count += 1
if stripecount and count >= stripecount:
parity = 1 - parity
count = 0
Thomas Arendsen Hein
Don't let ui.username override web.contact (issue900)...
r5779 def get_contact(config):
"""Return repo contact information or empty string.
web.contact is the primary source, but if that is not set, try
ui.username or $EMAIL as a fallback to display something useful.
"""
return (config("web", "contact") or
config("ui", "username") or
os.environ.get("EMAIL") or "")
Dirkjan Ochtman
hgweb: support very simple caching model (issue1845)
r12183
def caching(web, req):
tag = str(web.mtime)
if req.env.get('HTTP_IF_NONE_MATCH') == tag:
raise ErrorResponse(HTTP_NOT_MODIFIED)
req.headers.append(('ETag', tag))