##// END OF EJS Templates
changegroup: port to emitrevisions() (issue5976)...
changegroup: port to emitrevisions() (issue5976) We now have a unified API for emitting revision data from a storage backend. It handles sorting nodes and the complicated delta versus revision decisions for us. This commit ports changegroup to that API. There should be no behavior changes for changegroups not using ellipsis. And lack of test changes seems to confirm that. There are some changes for ellipsis mode, however. Before, when sending an ellipsis revision, we would always send a fulltext revision (as opposed to a delta). There was a TODO tracking this open item. One of the things the emitrevisions() API does for us is figure out whether we can safely emit a delta. So, it is now possible for ellipsis revisions to be sent as deltas! (It does this by not assuming parent/ancestor revisions are available and tracking which revisions have been sent out.) Because we eliminated the list of revision delta request objects, performance has improved substantially: $ hg perfchangegroupchangelog before: ! wall 24.348077 comb 24.330000 user 24.140000 sys 0.190000 (best of 3) after: ! wall 18.245911 comb 18.240000 user 18.100000 sys 0.140000 (best of 3) That's a lot of overhead for creating a few hundred thousand Python objects! This is still a little slower than 4.7. Probably due to 23d582ca introducing a type for the revision/delta results. There is potentially room to optimize. But at some point we need to abstract storage in order to support alternate storage backends. Unfortunately that means using a Python data structure to represent results. And unfortunately there is overhead with every new Python object created. Differential Revision: https://phab.mercurial-scm.org/D4725

File last commit:

r37471:2735d08e default
r39901:31b7e8e7 default
Show More
bundleparts.py
115 lines | 3.5 KiB | text/x-python | PythonLexer
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import
from mercurial.i18n import _
from mercurial import (
bundle2,
changegroup,
error,
extensions,
revsetlang,
util,
)
from . import common
isremotebooksenabled = common.isremotebooksenabled
scratchbranchparttype = 'b2x:infinitepush'
def getscratchbranchparts(repo, peer, outgoing, ui, bookmark):
if not outgoing.missing:
raise error.Abort(_('no commits to push'))
if scratchbranchparttype not in bundle2.bundle2caps(peer):
raise error.Abort(_('no server support for %r') % scratchbranchparttype)
_validaterevset(repo, revsetlang.formatspec('%ln', outgoing.missing),
bookmark)
supportedversions = changegroup.supportedoutgoingversions(repo)
# Explicitly avoid using '01' changegroup version in infinitepush to
# support general delta
supportedversions.discard('01')
cgversion = min(supportedversions)
_handlelfs(repo, outgoing.missing)
cg = changegroup.makestream(repo, outgoing, cgversion, 'push')
params = {}
params['cgversion'] = cgversion
if bookmark:
params['bookmark'] = bookmark
# 'prevbooknode' is necessary for pushkey reply part
params['bookprevnode'] = ''
bookmarks = repo._bookmarks
if bookmark in bookmarks:
params['bookprevnode'] = bookmarks.changectx(bookmark).hex()
# Do not send pushback bundle2 part with bookmarks if remotenames extension
# is enabled. It will be handled manually in `_push()`
if not isremotebooksenabled(ui):
params['pushbackbookmarks'] = '1'
parts = []
# .upper() marks this as a mandatory part: server will abort if there's no
# handler
parts.append(bundle2.bundlepart(
scratchbranchparttype.upper(),
advisoryparams=params.iteritems(),
data=cg))
return parts
def _validaterevset(repo, revset, bookmark):
"""Abort if the revs to be pushed aren't valid for a scratch branch."""
if not repo.revs(revset):
raise error.Abort(_('nothing to push'))
if bookmark:
# Allow bundle with many heads only if no bookmark is specified
heads = repo.revs('heads(%r)', revset)
if len(heads) > 1:
raise error.Abort(
_('cannot push more than one head to a scratch branch'))
def _handlelfs(repo, missing):
'''Special case if lfs is enabled
If lfs is enabled then we need to call prepush hook
to make sure large files are uploaded to lfs
'''
try:
lfsmod = extensions.find('lfs')
lfsmod.wrapper.uploadblobsfromrevs(repo, missing)
except KeyError:
# Ignore if lfs extension is not enabled
return
class copiedpart(object):
"""a copy of unbundlepart content that can be consumed later"""
def __init__(self, part):
# copy "public properties"
self.type = part.type
self.id = part.id
self.mandatory = part.mandatory
self.mandatoryparams = part.mandatoryparams
self.advisoryparams = part.advisoryparams
self.params = part.params
self.mandatorykeys = part.mandatorykeys
# copy the buffer
self._io = util.stringio(part.read())
def consume(self):
return
def read(self, size=None):
if size is None:
return self._io.read()
else:
return self._io.read(size)