##// END OF EJS Templates
revlog: linearize created changegroups in generaldelta revlogs...
revlog: linearize created changegroups in generaldelta revlogs This greatly improves the speed of the bundling process, and often reduces the bundle size considerably. (Although if the repository is already ordered, this has little effect on both time and bundle size.) For non-generaldelta clients, the reduced bundle size translates to a reduced repository size, similar to shrinking the revlogs (which uses the exact same algorithm). For generaldelta clients the difference is minor. When the new bundle format comes, reordering will not be necessary since we can then store the deltaparent relationsships directly. The eventual default behavior for clients and servers is presented in the table below, where "new" implies support for GD as well as the new bundle format: old client new client old server old bundle, no reorder old bundle, no reorder new server, non-GD old bundle, no reorder[1] old bundle, no reorder[2] new server, GD old bundle, reorder[3] new bundle, no reorder[4] [1] reordering is expensive on the server in this case, skip it [2] client can choose to do its own redelta here [3] reordering is needed because otherwise the pull does a lot of extra work on the server [4] reordering isn't needed because client can get deltabase in bundle format Currently, the default is to reorder on GD-servers, and not otherwise. A new setting, bundle.reorder, has been added to override the default reordering behavior. It can be set to either 'auto' (the default), or any true or false value as a standard boolean setting, to either force the reordering on or off regardless of generaldelta. Some timing data from a relatively branch test repository follows. All bundling is done with --all --type none options. Non-generaldelta, non-shrunk repo: ----------------------------------- Size: 276M Without reorder (default): Bundle time: 14.4 seconds Bundle size: 939M With reorder: Bundle time: 1 minute, 29.3 seconds Bundle size: 381M Generaldelta, non-shrunk repo: ----------------------------------- Size: 87M Without reorder: Bundle time: 2 minutes, 1.4 seconds Bundle size: 939M With reorder (default): Bundle time: 25.5 seconds Bundle size: 381M

File last commit:

r14094:d10c6835 default
r14365:a8e3931e default
Show More
protocol.py
88 lines | 2.8 KiB | text/x-python | PythonLexer
Dirkjan Ochtman
separate the wire protocol commands from the user interface commands
r5598 #
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
#
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Dirkjan Ochtman
separate the wire protocol commands from the user interface commands
r5598
Matt Mackall
http: minor tweaks to long arg handling...
r14094 import cgi, cStringIO, zlib, sys, urllib
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 from mercurial import util, wireproto
from common import HTTP_OK
Dirkjan Ochtman
hgweb: explicitly check if requested command exists
r5963
Dirkjan Ochtman
hgweb: explicit response status
r5993 HGTYPE = 'application/mercurial-0.1'
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595
class webproto(object):
def __init__(self, req):
self.req = req
self.response = ''
def getargs(self, args):
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 knownargs = self._args()
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 data = {}
keys = args.split()
for k in keys:
if k == '*':
star = {}
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 for key in knownargs.keys():
Peter Arrenbrecht
wireproto: fix handling of '*' args for HTTP and SSH
r13721 if key != 'cmd' and key not in keys:
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 star[key] = knownargs[key][0]
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 data['*'] = star
else:
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 data[k] = knownargs[k][0]
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 return [data[k] for k in keys]
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 def _args(self):
args = self.req.form.copy()
chunks = []
Matt Mackall
http: minor tweaks to long arg handling...
r14094 i = 1
while 1:
h = self.req.env.get('HTTP_X_HGARG_' + str(i))
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 if h is None:
break
chunks += [h]
Matt Mackall
http: minor tweaks to long arg handling...
r14094 i += 1
Steven Brown
httprepo: long arguments support (issue2126)...
r14093 args.update(cgi.parse_qs(''.join(chunks), keep_blank_values=True))
return args
Dirkjan Ochtman
protocol: shuffle server methods to group send methods
r11621 def getfile(self, fp):
length = int(self.req.env['CONTENT_LENGTH'])
for s in util.filechunkiter(self.req, limit=length):
fp.write(s)
def redirect(self):
self.oldio = sys.stdout, sys.stderr
sys.stderr = sys.stdout = cStringIO.StringIO()
Dirkjan Ochtman
protocol: extract compression from streaming mechanics
r11623 def groupchunks(self, cg):
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 z = zlib.compressobj()
while 1:
chunk = cg.read(4096)
if not chunk:
break
Dirkjan Ochtman
protocol: extract compression from streaming mechanics
r11623 yield z.compress(chunk)
yield z.flush()
Matt Mackall
protocol: move hgweb protocol support back into protocol.py...
r11595 def _client(self):
return 'remote:%s:%s:%s' % (
self.req.env.get('wsgi.url_scheme') or 'http',
urllib.quote(self.req.env.get('REMOTE_HOST', '')),
urllib.quote(self.req.env.get('REMOTE_USER', '')))
def iscmd(cmd):
return cmd in wireproto.commands
def call(repo, req, cmd):
p = webproto(req)
Dirkjan Ochtman
protocol: wrap non-string protocol responses in classes
r11625 rsp = wireproto.dispatch(repo, p, cmd)
Dirkjan Ochtman
protocol: use generators instead of req.write() for hgweb stream responses
r11626 if isinstance(rsp, str):
req.respond(HTTP_OK, HGTYPE, length=len(rsp))
return [rsp]
elif isinstance(rsp, wireproto.streamres):
req.respond(HTTP_OK, HGTYPE)
return rsp.gen
elif isinstance(rsp, wireproto.pushres):
val = sys.stdout.getvalue()
sys.stdout, sys.stderr = p.oldio
req.respond(HTTP_OK, HGTYPE)
return ['%d\n%s' % (rsp.res, val)]
Benoit Boissinot
wireproto: introduce pusherr() to deal with "unsynced changes" error...
r12703 elif isinstance(rsp, wireproto.pusherr):
Benoit Boissinot
wireproto/http: drain the incoming bundle in case of errors
r12704 # drain the incoming bundle
req.drain()
Benoit Boissinot
wireproto: introduce pusherr() to deal with "unsynced changes" error...
r12703 sys.stdout, sys.stderr = p.oldio
rsp = '0\n%s\n' % rsp.res
req.respond(HTTP_OK, HGTYPE, length=len(rsp))
return [rsp]