##// END OF EJS Templates
changegroup: remove reordering control (BC)...
changegroup: remove reordering control (BC) This logic - including the experimental bundle.reorder option - was originally added in a8e3931e3fb5 in 2011 and then later ported to changegroup.py. The intent of this option and associated logic is to control the ordering of revisions in deltagroups in changegroups. At the time it was implemented, only changegroup version 1 existed and generaldelta revlogs were just coming into the world. Changegroup version 1 requires that deltas be made against the last revision sent over the wire. Used with generaldelta, this created an impedance mismatch of sorts and resulted in changegroup producers spending a lot of time recomputing deltas. Revision reordering was introduced so outgoing revisions would be sent in "generaldelta order" and producers would be able to reuse internal deltas from storage. Later on, we introduced changegroup version 2. It supported denoting which revision a delta was against. So we no longer needed to sort outgoing revisions to ensure optimal delta generation from the producer. So, subsequent changegroup versions disabled reordering. We also later made the changelog not store deltas by default. And we also made the changelog send out deltas in storage order. Why we do this for changelog, I'm not sure. Maybe we want to preserve revision order across clones? It doesn't really matter for this commit. Fast forward to 2018. We want to abstract storage backends. And having changegroup code require knowledge about how deltas are stored internally interferes with that goal. This commit removes reordering control from changegroup generation. After this commit, the reordering behavior is: * The changelog is always sent out in storage order (no behavior change). * Non-changelog generaldelta revlogs are reordered to always be in DAG topological order (previously, generaldelta revlogs would be emitted in storage order for version 2 and 3 changegroups). * Non-changelog non-generaldelta revlogs are sent in storage order (no behavior change). * There exists no config option to override behavior. The big difference here is that generaldelta revlogs now *always* have their revisions sorted in DAG order before going out over the wire. This behavior was previously only done for changegroup version 1. Version 2 and version 3 changegroups disabled reordering because the interchange format supported encoding arbitrary delta parents, so reordering wasn't strictly necessary. I can think of a few significant implications for this change. Because changegroup receivers will now see non-changelog revisions in DAG order instead of storage order, the internal storage order of manifests and files may differ substantially between producer and consumer. I don't think this matters that much, since the storage order of manifests and files is largely hidden from users. Only the storage order of changelog matters (because `hg log` shows the changelog in storage order). I don't think there should be any controversy here. The reordering of revisions has implications for changegroup producers. Previously, generaldelta revlogs would be emitted in storage order. And in the common case, the internally-stored delta could effectively be copied from disk into the deltagroup delta. This meant that emitting delta groups for generaldelta revlogs would be mostly linear read I/O. This is desirable for performance. With us now reordering generaldelta revlog revisions in DAG order, the read operations may use more random I/O instead of sequential I/O. This could result in performance loss. But with the prevalence of SSDs and fast random I/O, I'm not too worried. (Note: the optimal emission order for revlogs is actually delta encoding order. But the changegroup code wasn't doing that before or after this change. We could potentially implement that in a later commit.) Changegroups in DAG order will have implications for receivers. Previously, receiving storage order might mean seeing a number of interleaved branches. This would mean long delta chains, sparse I/O, and possibly more fulltext revisions instead of deltas, blowing up storage storage. (This is the same set of problems that sparse revlogs aims to address.) With the producer now sending revisions in DAG order, the receiver also stores revisions in DAG order. That means revisions for the same DAG branch are all grouped together. And this should yield better storage outcomes. In other words, sending the reordered changegroup allows the receiver to have better storage order and for the producer to not propagate its (possibly sub-optimal) internal storage order. On the mozilla-unified repository, this change influences bundle generation: $ hg bundle -t none-v2 -a before: time: real 355.680 secs (user 256.790+0.000 sys 16.820+0.000) after: time: real 382.950 secs (user 281.700+0.000 sys 17.690+0.000) before: 7,150,228,967 bytes (uncompressed) after: 7,041,556,273 bytes (uncompressed) before: 1,669,063,234 bytes (zstd l=3) after: 1,628,598,830 bytes (zstd l=3) $ hg unbundle before: time: real 511.910 secs (user 466.750+0.000 sys 32.680+0.000) after: time: real 487.790 secs (user 443.940+0.000 sys 30.840+0.000) 00manifest.d size: source: 274,924,292 bytes before: 304,741,626 bytes after: 245,252,087 bytes .hg/store total file size: source: 2,649,133,490 before: 2,680,888,130 after: 2,627,875,673 We see the bundle size drop. That's probably because if a revlog internally isn't storing a delta, it will choose to delta against the last emitted revision. And on repos with interleaved branches (like mozilla-unified), the previous revision could be an unrelated branch and therefore be a large delta. But with this patch, the previous revision is likely p1 or p2 and a delta should be small. We also see the manifest size drop by ~50 MB. It's worth noting that the manifest actually *increased* in size by ~25 MB in the old strategy and decreased ~25 MB from its source in the new strategy. Again, my explanation for this is that the DAG ordering in the changegroup is resulting in better grouping of revisions in the receiver, which results in more compact delta chains and higher storage efficiency. Unbundle time also dropped. I suspect this is due to the revlog having to work less to compute deltas since the incoming deltas are more optimal. i.e. the receiver spends less time resolving fulltext revisions as incoming deltas bounce around between DAG branches and delta chains. We also see bundle generation time increase. This is not desirable. However, the regression is only significant on the original repository: if we generate a bundle from the repository created from the new, always reordered bundles, we're close to baseline (if not at it with expected noise): $ hg bundle -t none-v2 -a before (original): time: real 355.680 secs (user 256.790+0.000 sys 16.820+0.000) after (original): time: real 382.950 secs (user 281.700+0.000 sys 17.690+0.000) after (new repo): time: real 362.280 secs (user 260.300+0.000 sys 17.700+0.000) This regression is a bit worrying because it will impact serving canonical repositories (that don't have optimal internal storage unless they are reordered - possibly as part of running `hg debugupgraderepo`). However, this regression will only be noticed by very large changegroups. And I'm guessing/hoping that any repository that large is using clonebundles to mitigate server load. Again, sending DAG order isn't the optimal send order for servers: sending in storage-delta order is. But in order to enable storage-optimal send order, we'll need a storage API that handles sorting. Future commits will introduce such an API. Differential Revision: https://phab.mercurial-scm.org/D4721

File last commit:

r37509:685ad41f default
r39897:db5501d9 default
Show More
followlines.js
286 lines | 10.6 KiB | application/javascript | JavascriptLexer
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 // followlines.js - JavaScript utilities for followlines UI
//
// Copyright 2017 Logilab SA <contact@logilab.fr>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
//** Install event listeners for line block selection and followlines action */
document.addEventListener('DOMContentLoaded', function() {
var sourcelines = document.getElementsByClassName('sourcelines')[0];
if (typeof sourcelines === 'undefined') {
return;
}
// URL to complement with "linerange" query parameter
var targetUri = sourcelines.dataset.logurl;
av6
hgweb: use strict equals, remove non-breaking space in followlines.js...
r35157 if (typeof targetUri === 'undefined') {
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 return;
}
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 // Tag of children of "sourcelines" element on which to add "line
// selection" style.
var selectableTag = sourcelines.dataset.selectabletag;
if (typeof selectableTag === 'undefined') {
return;
}
Denis Laxalde
hgweb: do not show "descending" link in followlines UI for filelog heads...
r32070 var isHead = parseInt(sourcelines.dataset.ishead || "0");
Denis Laxalde
hgweb: position the "followlines" box close to latest cursor position
r31849 //* position "element" on top-right of cursor */
function positionTopRight(element, event) {
var x = (event.clientX + 10) + 'px',
y = (event.clientY - 20) + 'px';
element.style.top = y;
element.style.left = x;
}
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 // retrieve all direct *selectable* children of class="sourcelines"
// element
var selectableElements = Array.prototype.filter.call(
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 sourcelines.children,
av6
hgweb: add missing semicolons to followlines.js...
r35027 function(x) { return x.tagName === selectableTag; });
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 var btnTitleStart = 'start following lines history from here';
var btnTitleEnd = 'terminate line block selection here';
//** return a <button> element with +/- spans */
function createButton() {
var btn = document.createElement('button');
btn.title = btnTitleStart;
btn.classList.add('btn-followlines');
var plusSpan = document.createElement('span');
plusSpan.classList.add('followlines-plus');
plusSpan.textContent = '+';
btn.appendChild(plusSpan);
var br = document.createElement('br');
btn.appendChild(br);
var minusSpan = document.createElement('span');
minusSpan.classList.add('followlines-minus');
minusSpan.textContent = '−';
btn.appendChild(minusSpan);
return btn;
}
// extend DOM with CSS class for selection highlight and action buttons
av6
hgweb: add missing semicolons to followlines.js...
r35027 var followlinesButtons = [];
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 for (var i = 0; i < selectableElements.length; i++) {
selectableElements[i].classList.add('followlines-select');
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 var btn = createButton();
followlinesButtons.push(btn);
// insert the <button> as child of `selectableElements[i]` unless the
// latter has itself a child with a "followlines-btn-parent" class
// (annotate view)
var btnSupportElm = selectableElements[i];
var childSupportElms = btnSupportElm.getElementsByClassName(
'followlines-btn-parent');
if ( childSupportElms.length > 0 ) {
btnSupportElm = childSupportElms[0];
}
av6
hgweb: insert followlines buttons before any children, including text nodes...
r37509 var refNode = btnSupportElm.childNodes[0]; // node to insert <button> before
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 btnSupportElm.insertBefore(btn, refNode);
}
// ** re-initialize followlines buttons */
function resetButtons() {
for (var i = 0; i < followlinesButtons.length; i++) {
var btn = followlinesButtons[i];
btn.title = btnTitleStart;
btn.classList.remove('btn-followlines-end');
btn.classList.remove('btn-followlines-hidden');
}
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 }
var lineSelectedCSSClass = 'followlines-selected';
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 //** add CSS class on selectable elements in `from`-`to` line range */
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 function addSelectedCSSClass(from, to) {
for (var i = from; i <= to; i++) {
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 selectableElements[i].classList.add(lineSelectedCSSClass);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 }
}
//** remove CSS class from previously selected lines */
function removeSelectedCSSClass() {
var elements = sourcelines.getElementsByClassName(
lineSelectedCSSClass);
while (elements.length) {
elements[0].classList.remove(lineSelectedCSSClass);
}
}
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 // ** return the element of type "selectableTag" parent of `element` */
function selectableParent(element) {
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 var parent = element.parentElement;
if (parent === null) {
return null;
}
av6
hgweb: use strict equals, remove non-breaking space in followlines.js...
r35157 if (element.tagName === selectableTag && parent.isSameNode(sourcelines)) {
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 return element;
}
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 return selectableParent(parent);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 }
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 // ** update buttons title and style upon first click */
function updateButtons(selectable) {
for (var i = 0; i < followlinesButtons.length; i++) {
var btn = followlinesButtons[i];
btn.title = btnTitleEnd;
btn.classList.add('btn-followlines-end');
}
// on clicked button, change title to "cancel"
var clicked = selectable.getElementsByClassName('btn-followlines')[0];
clicked.title = 'cancel';
clicked.classList.remove('btn-followlines-end');
}
//** add `listener` on "click" event for all `followlinesButtons` */
function buttonsAddEventListener(listener) {
for (var i = 0; i < followlinesButtons.length; i++) {
followlinesButtons[i].addEventListener('click', listener);
}
}
//** remove `listener` on "click" event for all `followlinesButtons` */
function buttonsRemoveEventListener(listener) {
for (var i = 0; i < followlinesButtons.length; i++) {
followlinesButtons[i].removeEventListener('click', listener);
}
}
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 //** event handler for "click" on the first line of a block */
function lineSelectStart(e) {
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 var startElement = selectableParent(e.target.parentElement);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 if (startElement === null) {
Denis Laxalde
hgweb: parameterize the tag name of elements holding followlines selection...
r32993 // not a "selectable" element (maybe <a>): abort, keeping event
// listener registered for other click with a "selectable" target
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 return;
}
Denis Laxalde
hgweb: add a floating tooltip to invite on followlines action...
r31848
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 // update button tooltip text and CSS
updateButtons(startElement);
Denis Laxalde
hgweb: add a floating tooltip to invite on followlines action...
r31848
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 var startId = parseInt(startElement.id.slice(1));
startElement.classList.add(lineSelectedCSSClass); // CSS
// remove this event listener
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsRemoveEventListener(lineSelectStart);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
//** event handler for "click" on the last line of the block */
function lineSelectEnd(e) {
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 var endElement = selectableParent(e.target.parentElement);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 if (endElement === null) {
// not a <span> (maybe <a>): abort, keeping event listener
// registered for other click with <span> target
return;
}
// remove this event listener
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsRemoveEventListener(lineSelectEnd);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 // reset button tooltip text
resetButtons();
Denis Laxalde
hgweb: add a floating tooltip to invite on followlines action...
r31848
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 // compute line range (startId, endId)
var endId = parseInt(endElement.id.slice(1));
av6
hgweb: use strict equals, remove non-breaking space in followlines.js...
r35157 if (endId === startId) {
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 // clicked twice the same line, cancel and reset initial state
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 // (CSS, event listener for selection start)
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 removeSelectedCSSClass();
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsAddEventListener(lineSelectStart);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 return;
}
var inviteElement = endElement;
if (endId < startId) {
var tmp = endId;
endId = startId;
startId = tmp;
inviteElement = startElement;
}
addSelectedCSSClass(startId - 1, endId -1); // CSS
// append the <div id="followlines"> element to last line of the
// selection block
Denis Laxalde
hgweb: do not show "descending" link in followlines UI for filelog heads...
r32070 var divAndButton = followlinesBox(targetUri, startId, endId, isHead);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 var div = divAndButton[0],
button = divAndButton[1];
inviteElement.appendChild(div);
Denis Laxalde
hgweb: position the "followlines" box close to latest cursor position
r31849 // set position close to cursor (top-right)
positionTopRight(div, e);
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 // hide all buttons
for (var i = 0; i < followlinesButtons.length; i++) {
followlinesButtons[i].classList.add('btn-followlines-hidden');
}
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
//** event handler for cancelling selection */
function cancel() {
// remove invite box
div.parentNode.removeChild(div);
// restore initial event listeners
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsAddEventListener(lineSelectStart);
buttonsRemoveEventListener(cancel);
for (var i = 0; i < followlinesButtons.length; i++) {
followlinesButtons[i].classList.remove('btn-followlines-hidden');
}
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 // remove styles on selected lines
removeSelectedCSSClass();
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 resetButtons();
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 }
// bind cancel event to click on <button>
button.addEventListener('click', cancel);
// as well as on an click on any source line
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsAddEventListener(cancel);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 }
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsAddEventListener(lineSelectEnd);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
}
Denis Laxalde
hgweb: re-implement followlines UI selection using buttons...
r33390 buttonsAddEventListener(lineSelectStart);
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785
//** return a <div id="followlines"> and inner cancel <button> elements */
Denis Laxalde
hgweb: do not show "descending" link in followlines UI for filelog heads...
r32070 function followlinesBox(targetUri, fromline, toline, isHead) {
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 // <div id="followlines">
var div = document.createElement('div');
div.id = 'followlines';
// <div class="followlines-cancel">
var buttonDiv = document.createElement('div');
buttonDiv.classList.add('followlines-cancel');
// <button>x</button>
var button = document.createElement('button');
button.textContent = 'x';
buttonDiv.appendChild(button);
div.appendChild(buttonDiv);
// <div class="followlines-link">
var aDiv = document.createElement('div');
aDiv.classList.add('followlines-link');
Denis Laxalde
hgweb: add a link to followlines in descending direction...
r31940 aDiv.textContent = 'follow history of lines ' + fromline + ':' + toline + ':';
var linesep = document.createElement('br');
aDiv.appendChild(linesep);
// link to "ascending" followlines
var aAsc = document.createElement('a');
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 var url = targetUri + '?patch=&linerange=' + fromline + ':' + toline;
Denis Laxalde
hgweb: add a link to followlines in descending direction...
r31940 aAsc.setAttribute('href', url);
Denis Laxalde
hgweb: change text of followlines links to "older / newer"...
r32071 aAsc.textContent = 'older';
Denis Laxalde
hgweb: add a link to followlines in descending direction...
r31940 aDiv.appendChild(aAsc);
Denis Laxalde
hgweb: do not show "descending" link in followlines UI for filelog heads...
r32070
if (!isHead) {
var sep = document.createTextNode(' / ');
aDiv.appendChild(sep);
// link to "descending" followlines
var aDesc = document.createElement('a');
aDesc.setAttribute('href', url + '&descend=');
Denis Laxalde
hgweb: change text of followlines links to "older / newer"...
r32071 aDesc.textContent = 'newer';
Denis Laxalde
hgweb: do not show "descending" link in followlines UI for filelog heads...
r32070 aDiv.appendChild(aDesc);
}
Denis Laxalde
hgweb: add a link to followlines in descending direction...
r31940
Denis Laxalde
hgweb: rename linerangelog.js as followlines.js...
r31785 div.appendChild(aDiv);
return [div, button];
}
}, false);