##// END OF EJS Templates
sshpeer: initial definition and implementation of new SSH protocol...
sshpeer: initial definition and implementation of new SSH protocol The existing SSH protocol has several design flaws. Future commits will elaborate on these flaws as new features are introduced to combat these flaws. For now, hopefully you can take me for my word that a ground up rewrite of the SSH protocol is needed. This commit lays the foundation for a new SSH protocol by defining a mechanism to upgrade the SSH transport channel away from the default (version 1) protocol to something modern (which we'll call "version 2" for now). This upgrade process is detailed in the internals documentation for the wire protocol. The gist of it is the client sends a request line preceding the "hello" command/line which basically says "I'm requesting an upgrade: here's what I support." If the server recognizes that line, it processes the upgrade request and the transport channel is switched to use the new version of the protocol. If not, it sends an empty response, which is how all Mercurial SSH servers from the beginning of time reacted to unknown commands. The upgrade request is effectively ignored and the client continues to use the existing version of the protocol as if nothing happened. The new version of the SSH protocol is completely identical to version 1 aside from the upgrade dance and the bytes that follow. The immediate bytes that follow the protocol switch are defined to be a length framed "capabilities: " line containing the remote's advertised capabilities. In reality, this looks very similar to what the "hello" response would look like. But it will evolve quickly. The methodology by which the protocol will evolve is important. I'm not going to introduce the new protocol all at once. That would likely lead to endless bike shedding and forward progress would stall. Instead, I intend to tricle out new features and diversions from the existing protocol in small, incremental changes. To support the gradual evolution of the protocol, the on-the-wire advertised protocol name contains an "exp" to denote "experimental" and a 4 digit field to capture the sub-version of the protocol. Whenever we make a BC change to the wire protocol, we can increment this version and lock out all older clients because it will appear as a completely different protocol version. This means we can incur as many breaking changes as we want. We don't have to commit to supporting any one feature or idea for a long period of time. We can even evolve the handshake mechanism, because that is defined as being an implementation detail of the negotiated protocol version! Hopefully this lowers the barrier to accepting changes to the protocol and for experimenting with "radical" ideas during its development. In core, sshpeer received most of the attention. We haven't even implemented the server bits for the new protocol in core yet. Instead, we add very primitive support to our test server, mainly just to exercise the added code paths in sshpeer. Differential Revision: https://phab.mercurial-scm.org/D2061 # no-check-commit because of required foo_bar naming

File last commit:

r34802:1f4249c7 default
r35994:48a3a928 default
Show More
mpatch.c
279 lines | 6.0 KiB | text/x-c | CLexer
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 /*
mpatch.c - efficient binary patching for Mercurial
This implements a patch algorithm that's O(m + nlog n) where m is the
size of the output and n is the number of patches.
Given a list of binary patches, it unpacks each into a hunk list,
then combines the hunk lists with a treewise recursion to form a
single hunk list. This hunk list is then applied to the original
text.
The text (or binary) fragments are copied directly from their source
Python objects into a preallocated output string to avoid the
allocation of intermediate Python objects. Working memory is about 2x
the total number of hunks.
Vadim Gelfer
update copyrights.
r2859 Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
This software may be used and distributed according to the terms
of the GNU General Public License, incorporated herein by reference.
*/
#include <stdlib.h>
#include <string.h>
Vadim Gelfer
mac os x: fixes for 10.2 from chris monson <monpublic@gmail.com>
r2468
Maciej Fijalkowski
internals: move the bitmanipulation routines into its own file...
r29444 #include "bitmanipulation.h"
Maciej Fijalkowski
mpatch: change Py_ssize_t to ssize_t in places that will be later copied
r29691 #include "compat.h"
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 #include "mpatch.h"
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
Yuya Nishihara
mpatch: change lalloc() to local function...
r29741 static struct mpatch_flist *lalloc(ssize_t size)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_flist *a = NULL;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
Matt Mackall
mpatch: Fix for malloc corner case on AIX
r3138 if (size < 1)
size = 1;
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 a = (struct mpatch_flist *)malloc(sizeof(struct mpatch_flist));
mpm@selenic.com
Add safety checking to mpatch
r128 if (a) {
Augie Fackler
mpatch: re-wrap wide line with clang-format...
r34634 a->base = (struct mpatch_frag *)malloc(
sizeof(struct mpatch_frag) * size);
Thomas Arendsen Hein
Set correct exception for another possible malloc error in mpatch.c
r2048 if (a->base) {
mpm@selenic.com
Add safety checking to mpatch
r128 a->head = a->tail = a->base;
Thomas Arendsen Hein
Set correct exception for another possible malloc error in mpatch.c
r2048 return a;
}
free(a);
mpm@selenic.com
Add safety checking to mpatch
r128 }
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 return NULL;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
Maciej Fijalkowski
mpatch: split mpatch into two files
r29693 void mpatch_lfree(struct mpatch_flist *a)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
mpm@selenic.com
Add safety checking to mpatch
r128 if (a) {
free(a->base);
free(a);
}
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 static ssize_t lsize(struct mpatch_flist *a)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
return a->tail - a->head;
}
/* move hunks in source that are less cut to dest, compensating
for changes in offset. the last hunk may be split if necessary.
*/
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 static int gather(struct mpatch_flist *dest, struct mpatch_flist *src, int cut,
Augie Fackler
mpatch: reformat function prototypes with clang-format...
r34801 int offset)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_frag *d = dest->tail, *s = src->head;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 int postend, c, l;
while (s != src->tail) {
if (s->start + offset >= cut)
mpm@selenic.com
Gotos are embarrassing.
r82 break; /* we've gone far enough */
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
postend = offset + s->start + s->len;
if (postend <= cut) {
/* save this hunk */
offset += s->start + s->len - s->end;
*d++ = *s++;
Augie Fackler
mpatch: reflow two oddly formatted else blocks with clang-format...
r34635 } else {
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 /* break up this hunk */
c = cut - offset;
if (s->end < c)
c = s->end;
l = cut - offset - s->start;
if (s->len < l)
l = s->len;
offset += s->start + l - c;
d->start = s->start;
d->end = c;
d->len = l;
d->data = s->data;
d++;
s->start = c;
s->len = s->len - l;
s->data = s->data + l;
mpm@selenic.com
Gotos are embarrassing.
r82 break;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
}
dest->tail = d;
src->head = s;
return offset;
}
/* like gather, but with no output list */
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 static int discard(struct mpatch_flist *src, int cut, int offset)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_frag *s = src->head;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 int postend, c, l;
while (s != src->tail) {
if (s->start + offset >= cut)
mpm@selenic.com
Gotos are embarrassing.
r82 break;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
postend = offset + s->start + s->len;
if (postend <= cut) {
offset += s->start + s->len - s->end;
s++;
Augie Fackler
mpatch: reflow two oddly formatted else blocks with clang-format...
r34635 } else {
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 c = cut - offset;
if (s->end < c)
c = s->end;
l = cut - offset - s->start;
if (s->len < l)
l = s->len;
offset += s->start + l - c;
s->start = c;
s->len = s->len - l;
s->data = s->data + l;
mpm@selenic.com
Gotos are embarrassing.
r82 break;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
}
src->head = s;
return offset;
}
/* combine hunk lists a and b, while adjusting b for offset changes in a/
this deletes a and b and returns the resultant list. */
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 static struct mpatch_flist *combine(struct mpatch_flist *a,
Augie Fackler
mpatch: reformat function prototypes with clang-format...
r34801 struct mpatch_flist *b)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_flist *c = NULL;
struct mpatch_frag *bh, *ct;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 int offset = 0, post;
mpm@selenic.com
Add safety checking to mpatch
r128 if (a && b)
c = lalloc((lsize(a) + lsize(b)) * 2);
if (c) {
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
mpm@selenic.com
Add safety checking to mpatch
r128 for (bh = b->head; bh != b->tail; bh++) {
/* save old hunks */
offset = gather(c, a, bh->start, offset);
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
mpm@selenic.com
Add safety checking to mpatch
r128 /* discard replaced hunks */
post = discard(a, bh->end, offset);
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
mpm@selenic.com
Add safety checking to mpatch
r128 /* insert new hunk */
ct = c->tail;
ct->start = bh->start - offset;
ct->end = bh->end - post;
ct->len = bh->len;
ct->data = bh->data;
c->tail++;
offset = post;
}
/* hold on to tail from a */
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 memcpy(c->tail, a->head, sizeof(struct mpatch_frag) * lsize(a));
mpm@selenic.com
Add safety checking to mpatch
r128 c->tail += lsize(a);
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 mpatch_lfree(a);
mpatch_lfree(b);
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 return c;
}
/* decode a binary patch into a hunk list */
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 int mpatch_decode(const char *bin, ssize_t len, struct mpatch_flist **res)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_flist *l;
struct mpatch_frag *lt;
Matt Mackall
mpatch: rewrite pointer overflow checks
r20167 int pos = 0;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
/* assume worst case size, we won't have many of these lists */
Matt Mackall
parsers: fix list sizing rounding error (SEC)...
r28656 l = lalloc(len / 12 + 1);
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 if (!l)
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return MPATCH_ERR_NO_MEM;
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 lt = l->tail;
Matt Mackall
mpatch: rewrite pointer overflow checks
r20167 while (pos >= 0 && pos < len) {
lt->start = getbe32(bin + pos);
lt->end = getbe32(bin + pos + 4);
lt->len = getbe32(bin + pos + 8);
lt->data = bin + pos + 12;
pos += 12 + lt->len;
Matt Mackall
parsers: detect short records (SEC)...
r28657 if (lt->start > lt->end || lt->len < 0)
break; /* sanity check */
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 lt++;
}
Matt Mackall
mpatch: rewrite pointer overflow checks
r20167 if (pos != len) {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 mpatch_lfree(l);
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return MPATCH_ERR_CANNOT_BE_DECODED;
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 }
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 l->tail = lt;
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 *res = l;
return 0;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
/* calculate the size of resultant text */
Maciej Fijalkowski
mpatch: split mpatch into two files
r29693 ssize_t mpatch_calcsize(ssize_t len, struct mpatch_flist *l)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: change Py_ssize_t to ssize_t in places that will be later copied
r29691 ssize_t outlen = 0, last = 0;
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_frag *f = l->head;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
while (f != l->tail) {
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 if (f->start < last || f->end > len) {
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return MPATCH_ERR_INVALID_PATCH;
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 }
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 outlen += f->start - last;
last = f->end;
outlen += f->len;
f++;
}
outlen += len - last;
return outlen;
}
Maciej Fijalkowski
mpatch: split mpatch into two files
r29693 int mpatch_apply(char *buf, const char *orig, ssize_t len,
Augie Fackler
mpatch: reformat function prototypes with clang-format...
r34801 struct mpatch_flist *l)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: provide things that will be exported later with a prefixed name...
r29692 struct mpatch_frag *f = l->head;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 int last = 0;
char *p = buf;
while (f != l->tail) {
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 if (f->start < last || f->end > len) {
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return MPATCH_ERR_INVALID_PATCH;
Benoit Boissinot
catch errors and throw exception with invalid binary patch data
r1722 }
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 memcpy(p, orig + last, f->start - last);
p += f->start - last;
memcpy(p, f->data, f->len);
last = f->end;
p += f->len;
f++;
}
memcpy(p, orig + last, len - last);
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return 0;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
/* recursively generate a patch of all bins between start and end */
Augie Fackler
mpatch: reformat function prototypes with clang-format...
r34801 struct mpatch_flist *
mpatch_fold(void *bins, struct mpatch_flist *(*get_next_item)(void *, ssize_t),
ssize_t start, ssize_t end)
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 {
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 ssize_t len;
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72
if (start + 1 == end) {
/* trivial case, output a decoded list */
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return get_next_item(bins, start);
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }
/* divide and conquer, memory management is elsewhere */
len = (end - start) / 2;
Maciej Fijalkowski
mpatch: remove dependency on Python.h in mpatch.c...
r29694 return combine(mpatch_fold(bins, get_next_item, start, start + len),
Augie Fackler
mpatch: switch alignment of wrapped line from tab to spaces with clang-format...
r34802 mpatch_fold(bins, get_next_item, start + len, end));
mpm@selenic.com
Add an O(m + nlog n) patching extension
r72 }