##// END OF EJS Templates
deltas: set estimated compression upper bound to "3x" instead of "10x"...
deltas: set estimated compression upper bound to "3x" instead of "10x" In pratice, we very rarely observer compression better than "3x" on manifest deltas. Having a more aggressive estimate significantly helps our pathological use case on a private repository. Here are a comparison of timings using different upper bound. Estimated compression | ø | ×10 | ×5 | ×3 | timing | 14.11 | 2.61 | 1.96 | 1.53 | We also tested the impact of this series on an array of public repositories. This shown no impact in either size nor timing. Full data set below for those interested. Size ---- Regarding size, not significant impact have been noticed on neither public nor private repositories. Here are the number we gathered on public repositories: zlib/upperbound | no | 10x | 5x | 3x mercurial | 5 875 730 | 5 875 730 | 5 875 730 | 5 875 730 pypy | 27 782 913 | 27 782 913 | 27 782 913 | 27 782 913 netbeans | 159 161 207 | 159 161 207 | 159 161 207 | 159 959 879 (+0.5%) mozilla-central | 323 841 642 | 323 841 642 | 323 841 642 | 319 867 519 (-2.5%) mozilla-try | 746 649 123 | 746 649 123 | 746 649 123 | 741 155 568 (-0.7%) private-repo | 1 485 287 294 | 1 485 287 294 | 1 485 287 294 | 1 409 248 382 (-5.1%) zstd/upperbound | no | 10x | 5x | 3x mercurial | 5 895 206 | 5 895 206 | 5 895 206 | 5 895 206 pypy | 28 689 230 | 28 689 230 | 28 689 230 | 28 689 230 netbeans | 157 636 387 | 157 636 387 | 157 636 387 | 159 692 678 (+1.3%) mozilla-central | 317 650 281 | 317 650 281 | 317 650 281 | 319 613 603 (+0.6%) mozilla-try | 737 555 275 | 737 555 275 | 737 555 275 | 738 079 473 (+0.1%) private-repo | 1 352 362 982 | 1 352 362 982 | 1 346 961 880 | 1 361 327 384 (+0.7%) Speed ------ Timing gathered using `hg perfrevlogwrite -m`. Value are in seconds. mercurial zlib | no | 10x | 5x | 3x | total | 65.551783 | 65.388887 | 65.260658 | 65.321199 | max | 0.034544 | 0.034571 | 0.034659 | 0.034521 | 99.99% | 0.034544 | 0.034571 | 0.034659 | 0.034521 | zstd | no | 10x | 5x | 3x | total | 49.118449 | 49.054062 | 48.753588 | 48.740230 | max | 0.009338 | 0.009239 | 0.009202 | 0.009178 | 99.99% | 0.007618 | 0.007639 | 0.007626 | 0.007621 | pypy zlib | no | 10x | 5x | 3x | total | 560.865984 | 558.983817 | 559.083815 | 559.349152 | max | 0.219614 | 0.215922 | 0.218112 | 0.218107 | 99.99% | 0.219614 | 0.215922 | 0.218112 | 0.218107 | zstd | no | 10x | 5x | 3x | total | 349.393280 | 347.395819 | 347.185407 | 345.643985 | max | 0.084143 | 0.083536 | 0.081834 | 0.082178 | 99.99% | 0.039445 | 0.039639 | 0.039612 | 0.039175 | netbeans zlib | no | 10x | 5x | 3x | total | 33103.327727 | 33314.932260 | 33211.745233 | 33345.891778 | max | 2.666852 | 2.672059 | 2.662453 | 2.662936 | 99.99% | 2.058772 | 2.070429 | 2.069569 | 2.064653 | zstd | no | 10x | 5x | 3x | total | 20112.102708 | 20095.879719 | 20083.390300 | 20123.221859 | max | 2.063482 | 2.062851 | 2.065229 | 2.060147 | 99.99% | 1.146647 | 1.143794 | 1.142933 | 1.146529 | mozilla zlib | no | 10x | 5x | 3x | total | 41374.102138 | 41418.816773 | 41381.956370 | 41334.280732 | max | 3.383474 | 3.387400 | 3.405711 | 3.387316 | 99.99% | 1.006755 | 1.005954 | 1.007700 | 1.007373 | zstd | no | 10x | 5x | 3x | total | 24689.691520 | 24643.939662 | 24664.630027 | 24664.512714 | max | 1.460822 | 1.449640 | 1.439747 | 1.465304 | 99.99% | 0.527111 | 0.527377 | 0.527807 | 0.527226 |

File last commit:

r33368:ffb30661 default
r42669:4a3abb33 default
Show More
transport.py
132 lines | 4.9 KiB | text/x-python | PythonLexer
Daniel Holth
convert extension: stripped-down svn transport module
r4764 # -*- coding: utf-8 -*-
# Copyright (C) 2007 Daniel Holth <dholth@fastmail.fm>
# This is a stripped-down version of the original bzr-svn transport.py,
# Copyright (C) 2006 Jelmer Vernooij <jelmer@samba.org>
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
Martin Geisler
Remove FSF mailing address from GPL headers...
r15782 # along with this program; if not, see <http://www.gnu.org/licenses/>.
timeless
convert: transport use absolute_import
r28412 from __future__ import absolute_import
Daniel Holth
convert extension: stripped-down svn transport module
r4764
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 import svn.client
Daniel Holth
convert extension: stripped-down svn transport module
r4764 import svn.core
timeless
convert: transport use absolute_import
r28412 import svn.ra
Pool = svn.core.Pool
SubversionException = svn.core.SubversionException
Daniel Holth
convert extension: stripped-down svn transport module
r4764
FUJIWARA Katsunori
convert: fix "stdlib import follows local import" problem in transport...
r28461 from mercurial import (
util,
)
Thomas Arendsen Hein
removed trailing whitespace
r4957 # Some older versions of the Python bindings need to be
Daniel Holth
convert extension: stripped-down svn transport module
r4764 # explicitly initialized. But what we want to do probably
# won't work worth a darn against those libraries anyway!
svn.ra.initialize()
Durham Goode
convert: move svn config initializer out of the module level...
r29668 svn_config = None
Daniel Holth
convert extension: stripped-down svn transport module
r4764
def _create_auth_baton(pool):
"""Create a Subversion authentication baton. """
import svn.client
# Give the client context baton a suite of authentication
# providers.h
providers = [
svn.client.get_simple_provider(pool),
svn.client.get_username_provider(pool),
svn.client.get_ssl_client_cert_file_provider(pool),
svn.client.get_ssl_client_cert_pw_file_provider(pool),
svn.client.get_ssl_server_trust_file_provider(pool),
]
timeless@mozdev.org
spelling: dependent
r17481 # Platform-dependent authentication methods
Patrick Mezard
convert/svn: support more OS specific auth providers via svn 1.6 API
r8120 getprovider = getattr(svn.core, 'svn_auth_get_platform_specific_provider',
None)
if getprovider:
# Available in svn >= 1.6
for name in ('gnome_keyring', 'keychain', 'kwallet', 'windows'):
for type in ('simple', 'ssl_client_cert_pw', 'ssl_server_trust'):
p = getprovider(name, type, pool)
if p:
providers.append(p)
else:
Augie Fackler
hgext: replace uses of hasattr with util.safehasattr
r14945 if util.safehasattr(svn.client, 'get_windows_simple_provider'):
Patrick Mezard
convert/svn: support more OS specific auth providers via svn 1.6 API
r8120 providers.append(svn.client.get_windows_simple_provider(pool))
Patrick Mezard
convert: support windows SVN simple auth provider
r5126
Daniel Holth
convert extension: stripped-down svn transport module
r4764 return svn.core.svn_auth_open(providers, pool)
class NotBranchError(SubversionException):
pass
class SvnRaTransport(object):
"""
Open an ra connection to a Subversion repository.
"""
def __init__(self, url="", ra=None):
self.pool = Pool()
self.svn_url = url
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 self.username = ''
self.password = ''
Daniel Holth
convert extension: stripped-down svn transport module
r4764
# Only Subversion 1.4 has reparent()
Augie Fackler
hgext: replace uses of hasattr with util.safehasattr
r14945 if ra is None or not util.safehasattr(svn.ra, 'reparent'):
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 self.client = svn.client.create_context(self.pool)
ab = _create_auth_baton(self.pool)
self.client.auth_baton = ab
Durham Goode
convert: move svn config initializer out of the module level...
r29668 global svn_config
if svn_config is None:
svn_config = svn.core.svn_config_get_config(None)
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 self.client.config = svn_config
Daniel Holth
convert extension: stripped-down svn transport module
r4764 try:
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 self.ra = svn.client.open_ra_session(
Patrick Mezard
convert/svn: fix URL quoting issue with svn 1.7...
r15599 self.svn_url,
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 self.client, self.pool)
Gregory Szorc
global: mass rewrite to use modern exception syntax...
r25660 except SubversionException as xxx_todo_changeme:
(inst, num) = xxx_todo_changeme.args
Bryan O'Sullivan
convert/subversion: get converter working against plain HTTP.
r4938 if num in (svn.core.SVN_ERR_RA_ILLEGAL_URL,
svn.core.SVN_ERR_RA_LOCAL_REPOS_OPEN_FAILED,
svn.core.SVN_ERR_BAD_URL):
Daniel Holth
convert extension: stripped-down svn transport module
r4764 raise NotBranchError(url)
raise
else:
self.ra = ra
svn.ra.reparent(self.ra, self.svn_url.encode('utf8'))
Benoit Boissinot
use new style classes
r8778 class Reporter(object):
Renato Cunha
convert: tuple parameter unpacking is deprecated in py3k
r11498 def __init__(self, reporter_data):
self._reporter, self._baton = reporter_data
Daniel Holth
convert extension: stripped-down svn transport module
r4764
def set_path(self, path, revnum, start_empty, lock_token, pool=None):
svn.ra.reporter2_invoke_set_path(self._reporter, self._baton,
path, revnum, start_empty, lock_token, pool)
def delete_path(self, path, pool=None):
svn.ra.reporter2_invoke_delete_path(self._reporter, self._baton,
path, pool)
def link_path(self, path, url, revision, start_empty, lock_token,
pool=None):
svn.ra.reporter2_invoke_link_path(self._reporter, self._baton,
path, url, revision, start_empty, lock_token,
pool)
def finish_report(self, pool=None):
svn.ra.reporter2_invoke_finish_report(self._reporter,
self._baton, pool)
def abort_report(self, pool=None):
svn.ra.reporter2_invoke_abort_report(self._reporter,
self._baton, pool)
def do_update(self, revnum, path, *args, **kwargs):
Matt Mackall
many, many trivial check-code fixups
r10282 return self.Reporter(svn.ra.do_update(self.ra, revnum, path,
*args, **kwargs))