##// END OF EJS Templates
zstd: vendor zstd 1.1.1...
zstd: vendor zstd 1.1.1 zstd is a new compression format and it is awesome, yielding higher compression ratios and significantly faster compression and decompression operations compared to zlib (our current compression engine of choice) across the board. We want zstd to be a 1st class citizen in Mercurial and to eventually be the preferred compression format for various operations. This patch starts the formal process of supporting zstd by vendoring a copy of zstd. Why do we need to vendor zstd? Good question. First, zstd is relatively new and not widely available yet. If we didn't vendor zstd or distribute it with Mercurial, most users likely wouldn't have zstd installed or even available to install. What good is a feature if you can't use it? Vendoring and distributing the zstd sources gives us the highest liklihood that zstd will be available to Mercurial installs. Second, the Python bindings to zstd (which will be vendored in a separate changeset) make use of zstd APIs that are only available via static linking. One reason they are only available via static linking is that they are unstable and could change at any time. While it might be possible for the Python bindings to attempt to talk to different versions of the zstd C library, the safest thing to do is link against a specific, known-working version of zstd. This is why the Python zstd bindings themselves vendor zstd and why we must as well. This also explains why the added files are in a "python-zstandard" directory. The added files are from the 1.1.1 release of zstd (Git commit 4c0b44f8ced84c4c8edfa07b564d31e4fa3e8885 from https://github.com/facebook/zstd) and are added without modifications. Not all files from the zstd "distribution" have been added. Notably missing are files to support interacting with "legacy," pre-1.0 versions of zstd. The decision of which files to include is made by the upstream python-zstandard project (which I'm the author of). The files in this commit are a snapshot of the files from the 0.5.0 release of that project, Git commit e637c1b214d5f869cf8116c550dcae23ec13b677 from https://github.com/indygreg/python-zstandard.

File last commit:

r30117:b85fa6bf default
r30434:2e484bde default
Show More
check-py3-compat.py
95 lines | 3.2 KiB | text/x-python | PythonLexer
#!/usr/bin/env python
#
# check-py3-compat - check Python 3 compatibility of Mercurial files
#
# Copyright 2015 Gregory Szorc <gregory.szorc@gmail.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function
import ast
import os
import sys
import traceback
def check_compat_py2(f):
"""Check Python 3 compatibility for a file with Python 2"""
with open(f, 'rb') as fh:
content = fh.read()
root = ast.parse(content)
# Ignore empty files.
if not root.body:
return
futures = set()
haveprint = False
for node in ast.walk(root):
if isinstance(node, ast.ImportFrom):
if node.module == '__future__':
futures |= set(n.name for n in node.names)
elif isinstance(node, ast.Print):
haveprint = True
if 'absolute_import' not in futures:
print('%s not using absolute_import' % f)
if haveprint and 'print_function' not in futures:
print('%s requires print_function' % f)
def check_compat_py3(f):
"""Check Python 3 compatibility of a file with Python 3."""
import importlib # not available on Python 2.6
with open(f, 'rb') as fh:
content = fh.read()
try:
ast.parse(content)
except SyntaxError as e:
print('%s: invalid syntax: %s' % (f, e))
return
# Try to import the module.
# For now we only support mercurial.* and hgext.* modules because figuring
# out module paths for things not in a package can be confusing.
if f.startswith(('hgext/', 'mercurial/')) and not f.endswith('__init__.py'):
assert f.endswith('.py')
name = f.replace('/', '.')[:-3].replace('.pure.', '.')
try:
importlib.import_module(name)
except Exception as e:
exc_type, exc_value, tb = sys.exc_info()
# We walk the stack and ignore frames from our custom importer,
# import mechanisms, and stdlib modules. This kinda/sorta
# emulates CPython behavior in import.c while also attempting
# to pin blame on a Mercurial file.
for frame in reversed(traceback.extract_tb(tb)):
if frame.name == '_call_with_frames_removed':
continue
if 'importlib' in frame.filename:
continue
if 'mercurial/__init__.py' in frame.filename:
continue
if frame.filename.startswith(sys.prefix):
continue
break
if frame.filename:
filename = os.path.basename(frame.filename)
print('%s: error importing: <%s> %s (error at %s:%d)' % (
f, type(e).__name__, e, filename, frame.lineno))
else:
print('%s: error importing module: <%s> %s (line %d)' % (
f, type(e).__name__, e, frame.lineno))
if __name__ == '__main__':
if sys.version_info[0] == 2:
fn = check_compat_py2
else:
fn = check_compat_py3
for f in sys.argv[1:]:
fn(f)
sys.exit(0)