##// END OF EJS Templates
pyoxidizer: produce working Python 3 Windows installers (issue6366)...
pyoxidizer: produce working Python 3 Windows installers (issue6366) While we've had code to produce Python 3 Windows installers with PyOxidizer, we haven't been advertising them on the web site due to a bug in making TLS connections and issues around resource handling. This commit upgrades our PyOxidizer install and configuration to use a recent Git commit of PyOxidizer. This new version of PyOxidizer contains a *ton* of changes, improvements, and bug fixes. Notably, Windows shared distributions now mostly "just work" and the TLS bug and random problems with Python extension modules in the standard library go away. And Python has been upgraded from 3.7 to 3.8.6. The price we pay for this upgrade is a ton of backwards incompatible changes to Starlark. I applied this commit (the overall series actually) on stable to produce Windows installers for Mercurial 5.5.2, which I published shortly before submitting this commit for review. In order to get the stable branch working, I decided to take a less aggressive approach to Python resource management. Previously, we were attempting to load all Python modules from memory and were performing some hacks to copy Mercurial's non-module resources into additional directories in Starlark. This commit implements a resource callback function in Starlark (a new feature since PyOxidizer 0.7) to dynamically assign standard library resources to in-memory loading and all other resources to filesystem loading. This means that Mercurial's files and all the other packages we ship in the Windows installers (e.g. certifi and pygments) are loaded from the filesystem instead of from memory. This avoids issues due to lack of __file__ and enables us to ship a working Python 3 installer on Windows. The end state of the install layout after this patch is not ideal for @: we still copy resource files like templates and help text to directories next to the hg.exe executable. There is code in @ to use importlib.resources to load these files and we could likely remove these copies once this lands on @. But for now, the install layout mimics what we've shipped for seemingly forever and is backwards compatible. It allows us to achieve the milestone of working Python 3 Windows installers and gets us a giant step closer to deleting Python 2. Differential Revision: https://phab.mercurial-scm.org/D9148

File last commit:

r43812:2fe6121c default
r46277:57b5452a default
Show More
lsprof.py
145 lines | 4.2 KiB | text/x-python | PythonLexer
Gregory Szorc
lsprof: use print function
r27617 from __future__ import absolute_import, print_function
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061
import _lsprof
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 import sys
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061
Gregory Szorc
py3: manually import getattr where it is needed...
r43359 from .pycompat import getattr
Gregory Szorc
py3: finish porting iteritems() to pycompat and remove source transformer...
r43376 from . import pycompat
Gregory Szorc
py3: manually import getattr where it is needed...
r43359
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061 Profiler = _lsprof.Profiler
# PyPy doesn't expose profiler_entry from the module.
profiler_entry = getattr(_lsprof, 'profiler_entry', None)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 __all__ = [b'profile', b'Stats']
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
Augie Fackler
formatting: blacken the codebase...
r43346
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 def profile(f, *args, **kwds):
"""XXX docstring"""
p = Profiler()
Dirkjan Ochtman
updating lsprof.py from remote repository
r5992 p.enable(subcalls=True, builtins=True)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 try:
Dirkjan Ochtman
updating lsprof.py from remote repository
r5992 f(*args, **kwds)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 finally:
p.disable()
Dirkjan Ochtman
updating lsprof.py from remote repository
r5992 return Stats(p.getstats())
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
class Stats(object):
"""XXX docstring"""
def __init__(self, data):
self.data = data
Augie Fackler
cleanup: remove pointless r-prefixes on double-quoted strings...
r43809 def sort(self, crit="inlinetime"):
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 """XXX docstring"""
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061 # profiler_entries isn't defined when running under PyPy.
if profiler_entry:
if crit not in profiler_entry.__dict__:
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 raise ValueError(b"Can't sort by %s" % crit)
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061 elif self.data and not getattr(self.data[0], crit, None):
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 raise ValueError(b"Can't sort by %s" % crit)
Gregory Szorc
lsprof: support PyPy (issue4573)...
r27061
Alejandro Santos
compat: use 'key' argument instead of 'cmp' when sorting a list
r9032 self.data.sort(key=lambda x: getattr(x, crit), reverse=True)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 for e in self.data:
if e.calls:
Alejandro Santos
compat: use 'key' argument instead of 'cmp' when sorting a list
r9032 e.calls.sort(key=lambda x: getattr(x, crit), reverse=True)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
def pprint(self, top=None, file=None, limit=None, climit=None):
"""XXX docstring"""
if file is None:
file = sys.stdout
d = self.data
if top is not None:
d = d[:top]
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 cols = b"% 12d %12d %11.4f %11.4f %s\n"
hcols = b"% 12s %12s %12s %12s %s\n"
Augie Fackler
formatting: blacken the codebase...
r43346 file.write(
hcols
% (
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 b"CallCount",
b"Recursive",
b"Total(s)",
b"Inline(s)",
b"module:lineno(function)",
Augie Fackler
formatting: blacken the codebase...
r43346 )
)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 count = 0
for e in d:
Augie Fackler
formatting: blacken the codebase...
r43346 file.write(
cols
% (
e.callcount,
e.reccallcount,
e.totaltime,
e.inlinetime,
label(e.code),
)
)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 count += 1
if limit is not None and count == limit:
return
ccount = 0
Matt Mackall
profile: add undocumented config options for profiler output
r16263 if climit and e.calls:
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 for se in e.calls:
Augie Fackler
formatting: blacken the codebase...
r43346 file.write(
cols
% (
se.callcount,
se.reccallcount,
se.totaltime,
se.inlinetime,
Augie Fackler
formatting: byteify all mercurial/ and hgext/ string literals...
r43347 b" %s" % label(se.code),
Augie Fackler
formatting: blacken the codebase...
r43346 )
)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 count += 1
ccount += 1
if limit is not None and count == limit:
return
if climit is not None and ccount == climit:
break
def freeze(self):
"""Replace all references to code objects with string
descriptions; this makes it possible to pickle the instance."""
# this code is probably rather ickier than it needs to be!
for i in range(len(self.data)):
e = self.data[i]
if not isinstance(e.code, str):
self.data[i] = type(e)((label(e.code),) + e[1:])
Dirkjan Ochtman
updating lsprof.py from remote repository
r5992 if e.calls:
for j in range(len(e.calls)):
se = e.calls[j]
if not isinstance(se.code, str):
e.calls[j] = type(se)((label(se.code),) + se[1:])
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
Augie Fackler
formatting: blacken the codebase...
r43346
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 _fn2mod = {}
Augie Fackler
formatting: blacken the codebase...
r43346
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 def label(code):
if isinstance(code, str):
Gregory Szorc
py3: encode str to bytes...
r40238 if sys.version_info.major >= 3:
code = code.encode('latin-1')
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 return code
try:
mname = _fn2mod[code.co_filename]
except KeyError:
Gregory Szorc
py3: finish porting iteritems() to pycompat and remove source transformer...
r43376 for k, v in list(pycompat.iteritems(sys.modules)):
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 if v is None:
continue
Augie Fackler
lsprof: use getattr instead of hasattr
r14959 if not isinstance(getattr(v, '__file__', None), str):
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422 continue
if v.__file__.startswith(code.co_filename):
mname = _fn2mod[code.co_filename] = k
break
else:
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 mname = _fn2mod[code.co_filename] = '<%s>' % code.co_filename
Gregory Szorc
py3: encode str to bytes...
r40238
Augie Fackler
cleanup: remove pointless r-prefixes on single-quoted strings...
r43906 res = '%s:%d(%s)' % (mname, code.co_firstlineno, code.co_name)
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
Gregory Szorc
py3: encode str to bytes...
r40238 if sys.version_info.major >= 3:
res = res.encode('latin-1')
Vadim Gelfer
add --lsprof option. 3x faster than --profile, more useful output....
r2422
Gregory Szorc
py3: encode str to bytes...
r40238 return res