##// END OF EJS Templates
setdiscovery: make progress on most connected groups each roundtrip...
setdiscovery: make progress on most connected groups each roundtrip Consider history like this: o | o | | | o | | | o |/ o | o | | | o | | | o |/ o | o | | | o | | | o |/ o ~ Assume the left mainline is available in the remote repo and the other commits are only in the local repo. Also imagine that instead of 3 local branches with 3 commits on each, there are 1000 branches (the number of commits on each doesn't matter much here). In such a scenario, the current setdiscovery code will pick a sample size of 200 among these branches and ask the remote which of them it has. However, the discovery for each such branch is completely independent of the discovery for the others -- knowing whether the remote has a commit in one branch doesn't give us any information about the other branches. The discovery will therefore take at least 5 roundtrips (maybe more depending on which commit in each linear chain was sampled). Since the discovery for each branch is independent, there is no reason to let one branch wait for another, so this patch makes it so we sample at least as many commits as there are branches. It may still happen (it's very likely, even) that we get multiple samples from one branch and none from another, but that will even out over a few rounds and I think this is still a big improvement. Because of http header size limits, we still use the old behavior unless experimental.httppostargs=true. I've timed this by running `hg debugdiscovery mozilla-unified --debug` in the mozilla-try repo. Both repos were local. Before this patch, last part of the output was: 2249 total queries in 5276.4859s elapsed time: 5276.652634 seconds heads summary: total common heads: 13 also local heads: 4 also remote heads: 8 both: 4 local heads: 28317 common: 4 missing: 28313 remote heads: 12 common: 8 unknown: 4 local changesets: 2014901 common: 530373 missing: 1484528 common heads: 1dad417c28ad 4a108e94d3e2 4d7ef530fffb 5350524bb654 777e60ca8853 7d97fafba271 9cd2ab4d0029 a55ce37217da d38398e5144e dcc6d7a0dc00 e09297892ada e24ec6070d7b fd559328eaf3 After this patch, the output was (including all the samples, since there were so few now): taking initial sample query 2; still undecided: 1599476, sample size is: 108195 sampling from both directions query 3; still undecided: 810922, sample size is: 194158 sampling from both directions query 4; still undecided: 325882, sample size is: 137302 sampling from both directions query 5; still undecided: 111459, sample size is: 74586 sampling from both directions query 6; still undecided: 26805, sample size is: 23960 sampling from both directions query 7; still undecided: 2549, sample size is: 2528 sampling from both directions query 8; still undecided: 21, sample size is: 21 8 total queries in 24.5064s elapsed time: 24.670051 seconds heads summary: total common heads: 13 also local heads: 4 also remote heads: 8 both: 4 local heads: 28317 common: 4 missing: 28313 remote heads: 12 common: 8 unknown: 4 local changesets: 2014901 common: 530373 missing: 1484528 common heads: 1dad417c28ad 4a108e94d3e2 4d7ef530fffb 5350524bb654 777e60ca8853 7d97fafba271 9cd2ab4d0029 a55ce37217da d38398e5144e dcc6d7a0dc00 e09297892ada e24ec6070d7b fd559328eaf3 Differential Revision: https://phab.mercurial-scm.org/D2647

File last commit:

r34398:765eb17a default
r42594:5b34972a default
Show More
_compat.py
90 lines | 2.8 KiB | text/x-python | PythonLexer
from __future__ import absolute_import, division, print_function
import sys
import types
PY2 = sys.version_info[0] == 2
if PY2:
from UserDict import IterableUserDict
# We 'bundle' isclass instead of using inspect as importing inspect is
# fairly expensive (order of 10-15 ms for a modern machine in 2016)
def isclass(klass):
return isinstance(klass, (type, types.ClassType))
# TYPE is used in exceptions, repr(int) is different on Python 2 and 3.
TYPE = "type"
def iteritems(d):
return d.iteritems()
def iterkeys(d):
return d.iterkeys()
# Python 2 is bereft of a read-only dict proxy, so we make one!
class ReadOnlyDict(IterableUserDict):
"""
Best-effort read-only dict wrapper.
"""
def __setitem__(self, key, val):
# We gently pretend we're a Python 3 mappingproxy.
raise TypeError("'mappingproxy' object does not support item "
"assignment")
def update(self, _):
# We gently pretend we're a Python 3 mappingproxy.
raise AttributeError("'mappingproxy' object has no attribute "
"'update'")
def __delitem__(self, _):
# We gently pretend we're a Python 3 mappingproxy.
raise TypeError("'mappingproxy' object does not support item "
"deletion")
def clear(self):
# We gently pretend we're a Python 3 mappingproxy.
raise AttributeError("'mappingproxy' object has no attribute "
"'clear'")
def pop(self, key, default=None):
# We gently pretend we're a Python 3 mappingproxy.
raise AttributeError("'mappingproxy' object has no attribute "
"'pop'")
def popitem(self):
# We gently pretend we're a Python 3 mappingproxy.
raise AttributeError("'mappingproxy' object has no attribute "
"'popitem'")
def setdefault(self, key, default=None):
# We gently pretend we're a Python 3 mappingproxy.
raise AttributeError("'mappingproxy' object has no attribute "
"'setdefault'")
def __repr__(self):
# Override to be identical to the Python 3 version.
return "mappingproxy(" + repr(self.data) + ")"
def metadata_proxy(d):
res = ReadOnlyDict()
res.data.update(d) # We blocked update, so we have to do it like this.
return res
else:
def isclass(klass):
return isinstance(klass, type)
TYPE = "class"
def iteritems(d):
return d.items()
def iterkeys(d):
return d.keys()
def metadata_proxy(d):
return types.MappingProxyType(dict(d))