##// END OF EJS Templates
phases: really fix native phase computation...
phases: really fix native phase computation For some reason (probably rebase issue, leprechaun or badly resolved .rej) 1635579f9baf contains only half of the emailed patches and do not fix the bug. This patch adds the other half and enable the sweet native computation for real. As expected this provide massive speedup along the board. revset #0: not public() plain first 0) 0.011960 0.010523 1) 0.000465 3% 0.000492 4% revset #1: (tip~1000::) - public() plain first 0) 0.025700 0.025169 1) 0.002864 11% 0.001899 7% revset #2: not public() and branch("default") plain first 0) 0.022842 0.020863 1) 0.011418 49% 0.010948 52% However, it has a less impact (even bad) on first result time in simple situation. This comes from the overhead of building the set and filtering it. This is especially true on my Mercurial repository (used here) where about 1/3 of the changesets are non public and hidden. This could be mitigated by a caching of the set and a better usage of smartset in '_notpublic'. (But this won't happen in this patch because the win is massive everywhere else). revset #0: not public() last 0) 0.000081 1) 0.000493 x6.1 <-- bad impact revset #1: (tip~1000::) - public() last 0) 0.013966 1) 0.002737 19% revset #2: not public() and branch("default") last 0) 0.011021 1) 0.011038 The effect mostly disappear when the number of non-public changesets is small and/or the repo get bigger. Result for Mozilla central: Mozilla revset #0: not public() plain first last 0) 0.092787 0.084094 0.000080 1) 0.000054 0% 0.000083 0% 0.000083 revset #1: (tip~1000::) - public() plain first last 0) 0.215607 0.183996 0.124962 1) 0.031620 14% 0.006616 3% 0.031168 24% revset #2: not public() and branch("default") plain first last 0) 0.092626 0.082687 0.000162 1) 0.000139 0% 0.000165 0% 0.000167

File last commit:

r16683:525fdb73 default
r25527:262e6ad9 default
Show More
mpatch.py
118 lines | 3.2 KiB | text/x-python | PythonLexer
Martin Geisler
pure Python implementation of mpatch.c
r7699 # mpatch.py - Python implementation of mpatch.c
#
# Copyright 2009 Matt Mackall <mpm@selenic.com> and others
#
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Martin Geisler
pure Python implementation of mpatch.c
r7699
Martin Geisler
pure/mpatch: use StringIO instead of mmap (issue1493)...
r7775 import struct
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
Martin Geisler
pure Python implementation of mpatch.c
r7699
# This attempts to apply a series of patches in time proportional to
# the total size of the patches, rather than patches * len(text). This
# means rather than shuffling strings around, we shuffle around
# pointers to fragments with fragment lists.
#
# When the fragment lists get too long, we collapse them. To do this
# efficiently, we do all our operations inside a buffer created by
# mmap and simply use memmove. This avoids creating a bunch of large
# temporary string buffers.
def patches(a, bins):
Matt Mackall
many, many trivial check-code fixups
r10282 if not bins:
return a
Martin Geisler
pure Python implementation of mpatch.c
r7699
plens = [len(x) for x in bins]
pl = sum(plens)
bl = len(a) + pl
tl = bl + bl + pl # enough for the patches and two working texts
b1, b2 = 0, bl
Matt Mackall
many, many trivial check-code fixups
r10282 if not tl:
return a
Martin Geisler
pure Python implementation of mpatch.c
r7699
Martin Geisler
pure/mpatch: use StringIO instead of mmap (issue1493)...
r7775 m = StringIO()
def move(dest, src, count):
"""move count bytes from src to dest
The file pointer is left at the end of dest.
"""
m.seek(src)
buf = m.read(count)
m.seek(dest)
m.write(buf)
Martin Geisler
pure Python implementation of mpatch.c
r7699
# load our original text
m.write(a)
frags = [(len(a), b1)]
# copy all the patches into our segment so we can memmove from them
pos = b2 + bl
m.seek(pos)
for p in bins: m.write(p)
def pull(dst, src, l): # pull l bytes from src
while l:
Dan Villiom Podlaski Christiansen
pure mpatch: avoid using list.insert(0, ...)...
r14065 f = src.pop()
Martin Geisler
pure Python implementation of mpatch.c
r7699 if f[0] > l: # do we need to split?
Dan Villiom Podlaski Christiansen
pure mpatch: avoid using list.insert(0, ...)...
r14065 src.append((f[0] - l, f[1] + l))
Martin Geisler
pure Python implementation of mpatch.c
r7699 dst.append((l, f[1]))
return
dst.append(f)
l -= f[0]
def collect(buf, list):
start = buf
Dan Villiom Podlaski Christiansen
pure mpatch: avoid using list.insert(0, ...)...
r14065 for l, p in reversed(list):
Martin Geisler
pure/mpatch: use StringIO instead of mmap (issue1493)...
r7775 move(buf, p, l)
Martin Geisler
pure Python implementation of mpatch.c
r7699 buf += l
return (buf - start, start)
for plen in plens:
# if our list gets too long, execute it
if len(frags) > 128:
b2, b1 = b1, b2
frags = [collect(b1, frags)]
new = []
end = pos + plen
last = 0
while pos < end:
Martin Geisler
pure/mpatch: use StringIO instead of mmap (issue1493)...
r7775 m.seek(pos)
p1, p2, l = struct.unpack(">lll", m.read(12))
Martin Geisler
pure Python implementation of mpatch.c
r7699 pull(new, frags, p1 - last) # what didn't change
pull([], frags, p2 - p1) # what got deleted
Brodie Rao
cleanup: eradicate long lines
r16683 new.append((l, pos + 12)) # what got added
Martin Geisler
pure Python implementation of mpatch.c
r7699 pos += l + 12
last = p2
Brodie Rao
cleanup: eradicate long lines
r16683 frags.extend(reversed(new)) # what was left at the end
Martin Geisler
pure Python implementation of mpatch.c
r7699
t = collect(b2, frags)
Martin Geisler
pure/mpatch: use StringIO instead of mmap (issue1493)...
r7775 m.seek(t[1])
return m.read(t[0])
Martin Geisler
pure Python implementation of mpatch.c
r7699
def patchedsize(orig, delta):
outlen, last, bin = 0, 0, 0
binend = len(delta)
data = 12
while data <= binend:
decode = delta[bin:bin + 12]
start, end, length = struct.unpack(">lll", decode)
if start > end:
break
bin = data + length
data = bin + 12
outlen += start - last
last = end
outlen += length
if bin != binend:
Matt Mackall
clean up remaining generic exceptions
r11122 raise ValueError("patch cannot be decoded")
Martin Geisler
pure Python implementation of mpatch.c
r7699
outlen += orig - last
return outlen