##// END OF EJS Templates
revlog: change generaldelta delta parent heuristic...
revlog: change generaldelta delta parent heuristic The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text, use it, otherwise use prev". This was problematic when a repo contained multiple branches that were very different. If commits to branch A were pushed, and the last full text was branch B, it would generate a fulltext. Then if branch B was pushed, it would generate another fulltext. The problem is that the last fulltext (and delta'ing against `prev` in general) has no correlation with the contents of the incoming revision, and therefore will always have degenerate cases. According to the blame, that algorithm was chosen to minimize the chain length. Since there is already code that protects against that (the delta-vs-fulltext code), and since it has been improved since the original generaldelta algorithm went in (2011), I believe the chain length criteria will still be preserved. The new algorithm always diffs against p1 (or p2 if it's closer), unless the resulting delta will fail the delta-vs-fulltext check, in which case we delta against prev. Some before and after stats on manifest.d size. internal large repo old heuristic - 2.0 GB new heuristic - 1.2 GB mozilla-central old heuristic - 242 MB new heuristic - 261 MB The regression in mozilla central is due to the new heuristic choosing p2r as the delta when it's closer to the tip. Switching the algorithm to always prefer p1r brings the size back down (242 MB). This is result of the way in which mozilla does merges and pushes, and the result could easily swing the other direction in other repos (depending on if they merge X into Y or Y into X), but will never be as degenerate as before. I future patch will address the regression by introducing an optional, even more aggressive delta heuristic which will knock the mozilla manifest size down dramatically.

File last commit:

r24972:56c64c91 default
r26117:4dc5b51f default
Show More
buildrpm
147 lines | 3.8 KiB | text/plain | TextLexer
Mads Kiilerich
buildrpm: various minor cleanup
r21638 #!/bin/sh -e
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564 #
Mads Kiilerich
buildrpm: various minor cleanup
r21638 # Build a Mercurial RPM from the current repo
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564 #
Mads Kiilerich
contrib/buildrpm: Support python 2.4 and 2.6
r8867 # Tested on
Mads Kiilerich
buildrpm: various minor cleanup
r21638 # - Fedora 20
# - CentOS 5
# - centOS 6
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564
Augie Fackler
packaging: extract packagelib for common code from builddeb and buildrpm
r24972 . $(dirname $0)/packagelib.sh
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 BUILD=1
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 RPMBUILDDIR="$PWD/rpmbuild"
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 while [ "$1" ]; do
case "$1" in
--prepare )
shift
BUILD=
;;
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 --withpython | --with-python)
shift
Mads Kiilerich
rpm: make Python 2.7.9 the default Python to include in rpms for EL 5...
r23881 PYTHONVER=2.7.9
PYTHONMD5=5eebcaa0030dc4061156d3429657fb83
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 ;;
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 --rpmbuilddir )
shift
RPMBUILDDIR="$1"
shift
;;
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 * )
echo "Invalid parameter $1!" 1>&2
exit 1
;;
esac
done
Gilles Moris
buildrpm: enable to start the script from anywhere...
r9811 cd "`dirname $0`/.."
Mads Kiilerich
buildrpm: complain when hg command isn't available...
r7431
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 specfile=$PWD/contrib/mercurial.spec
Gilles Moris
buildrpm: cleanup script
r9812 if [ ! -f $specfile ]; then
echo "Cannot find $specfile!" 1>&2
exit 1
fi
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564
Gilles Moris
buildrpm: cleanup script
r9812 if [ ! -d .hg ]; then
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564 echo 'You are not inside a Mercurial repository!' 1>&2
exit 1
fi
Augie Fackler
packaging: extract packagelib for common code from builddeb and buildrpm
r24972 gethgversion
Gilles Moris
buildrpm: build from working dir parent and use hg version for RPM versioning...
r9809
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 if [ "$PYTHONVER" ]; then
release=$release+$PYTHONVER
RPMPYTHONVER=$PYTHONVER
else
RPMPYTHONVER=%{nil}
fi
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 $HG archive -t tgz $RPMBUILDDIR/SOURCES/mercurial-$version-$release.tar.gz
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 if [ "$PYTHONVER" ]; then
(
Mads Kiilerich
rpms: create missing builds dir if it doesn't exist
r24730 mkdir -p build
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 cd build
PYTHON_SRCFILE=Python-$PYTHONVER.tgz
[ -f $PYTHON_SRCFILE ] || curl -Lo $PYTHON_SRCFILE http://www.python.org/ftp/python/$PYTHONVER/$PYTHON_SRCFILE
Mads Kiilerich
contrib: buildrpm checking of md5 checksums of downloaded Python and Docutils
r23141 if [ "$PYTHONMD5" ]; then
echo "$PYTHONMD5 $PYTHON_SRCFILE" | md5sum -w -c
fi
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 ln -f $PYTHON_SRCFILE $RPMBUILDDIR/SOURCES/$PYTHON_SRCFILE
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436
DOCUTILSVER=`sed -ne "s/^%global docutilsname docutils-//p" $specfile`
DOCUTILS_SRCFILE=docutils-$DOCUTILSVER.tar.gz
[ -f $DOCUTILS_SRCFILE ] || curl -Lo $DOCUTILS_SRCFILE http://downloads.sourceforge.net/project/docutils/docutils/$DOCUTILSVER/$DOCUTILS_SRCFILE
Mads Kiilerich
contrib: buildrpm checking of md5 checksums of downloaded Python and Docutils
r23141 DOCUTILSMD5=`sed -ne "s/^%global docutilsmd5 //p" $specfile`
if [ "$DOCUTILSMD5" ]; then
echo "$DOCUTILSMD5 $DOCUTILS_SRCFILE" | md5sum -w -c
fi
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 ln -f $DOCUTILS_SRCFILE $RPMBUILDDIR/SOURCES/$DOCUTILS_SRCFILE
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 )
fi
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 rpmspec=$RPMBUILDDIR/SPECS/mercurial.spec
Gilles Moris
buildrpm: build full RPM package including sources
r9813
Gilles Moris
buildrpm: cleanup script
r9812 sed -e "s,^Version:.*,Version: $version," \
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564 -e "s,^Release:.*,Release: $release," \
Gilles Moris
buildrpm: build full RPM package including sources
r9813 $specfile > $rpmspec
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564
Gilles Moris
buildrpm: enhance changelog of the RPM file...
r9814 echo >> $rpmspec
echo "%changelog" >> $rpmspec
if echo $version | grep '+' > /dev/null 2>&1; then
latesttag="`echo $version | sed -e 's/+.*//'`"
$HG log -r .:"$latesttag" -fM \
--template '{date|hgdate}\t{author}\t{desc|firstline}\n' | python -c '
import sys, time
def datestr(date, format):
return time.strftime(format, time.gmtime(float(date[0]) - date[1]))
Adam Spiers
buildrpm: auto-generate %changelog in .spec file...
r4754
Gilles Moris
buildrpm: enhance changelog of the RPM file...
r9814 changelog = []
for l in sys.stdin.readlines():
tok = l.split("\t")
hgdate = tuple(int(v) for v in tok[0].split())
changelog.append((datestr(hgdate, "%F"), tok[1], hgdate, tok[2]))
prevtitle = ""
for l in sorted(changelog, reverse=True):
title = "* %s %s" % (datestr(l[2], "%a %b %d %Y"), l[1])
if prevtitle != title:
prevtitle = title
print
print title
print "- %s" % l[3].strip()
' >> $rpmspec
else
$HG log \
--template '{date|hgdate}\t{author}\t{desc|firstline}\n' \
.hgtags | python -c '
import sys, time
def datestr(date, format):
return time.strftime(format, time.gmtime(float(date[0]) - date[1]))
for l in sys.stdin.readlines():
tok = l.split("\t")
hgdate = tuple(int(v) for v in tok[0].split())
print "* %s %s\n- %s" % (datestr(hgdate, "%a %b %d %Y"), tok[1], tok[2])
' >> $rpmspec
fi
Adam Spiers
buildrpm: auto-generate %changelog in .spec file...
r4754
Mads Kiilerich
buildrpm: introduce --withpython for building rpms that includes Python 2.7
r22436 sed -i \
-e "s/^%define withpython.*$/%define withpython $RPMPYTHONVER/" \
$rpmspec
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 if [ "$BUILD" ]; then
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 rpmbuild --define "_topdir $RPMBUILDDIR" -ba $rpmspec --clean
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 if [ $? = 0 ]; then
echo
echo "Built packages for $version-$release:"
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 find $RPMBUILDDIR/*RPMS/ -type f -newer $rpmspec
Mads Kiilerich
buildrpm: introduce --prepare for preparing without actually building rpms
r22435 fi
else
Mads Kiilerich
buildrpm: introduce --rpmdir instead of using hardcoded rpmbuild dir...
r22437 echo "Prepared sources for $version-$release $rpmspec are in $RPMBUILDDIR/SOURCES/ - use like:"
echo "rpmbuild --define '_topdir $RPMBUILDDIR' -ba $rpmspec --clean"
mpm@selenic.com
[PATCH] Add contrib/buildrpm script...
r564 fi