wireproto: define content negotiation for HTTPv2

HTTP messages communicate their media types and what media types
they can understand via the Content-Type and Accept header,
respectively.

While I don't want the wire protocol to lean too heavily on HTTP
because I'm aiming for the wire protocol to be as transport
agnostic as possible, it is nice to play by the spec if possible.

This commit defines our media negotiation mechanism for version
2 of the HTTP protocol. Essentially, we mandate the use of a
new media type and how clients and servers should react to
various headers or lack thereof.

The name of the media type is a placeholder. We purposefully don't
yet define the format of the new media type because that's a lot
of work.

I feel pretty strongly that we should use Content-Type. I feel
less strongly about Accept. I think it is reasonable for servers
to return the media type that was submitted to them. So we may
strike this header before the protocol is finished...

Differential Revision: https://phab.mercurial-scm.org/D2850

mercurial/help/internals/wireprotocol.txt

@@ -187,6 +187,19 b' exchange takes place. This provides a be'
 Requests to unknown commands or URLS result in an HTTP 404.
 TODO formally define response type, how error is communicated, etc.
 
+HTTP request and response bodies use the *TBD Protocol* for media exchange.
+
+Clients and servers MUST advertise the ``TBD`` media type via the
+``Content-Type`` request and response headers. In addition, clients MUST
+advertise this media type value in their ``Accept`` request header in all
+requests.
+
+Servers receiving requests without an ``Accept`` header SHOULD respond with
+an HTTP 406.
+
+Servers receiving requests with an invalid ``Content-Type`` header SHOULD
+respond with an HTTP 415.
+
 SSH Protocol
 ============
 

mercurial/wireprotoserver.py

@@ -32,6 +32,7 b' HTTP_OK = 200'
 HGTYPE = 'application/mercurial-0.1'
 HGTYPE2 = 'application/mercurial-0.2'
 HGERRTYPE = 'application/hg-error'
+HTTPV2TYPE = 'application/mercurial-tbd'
 
 HTTPV2 = wireprototypes.HTTPV2
 SSHV1 = wireprototypes.SSHV1
@@ -335,6 +336,23 b' def _handlehttpv2request(rctx, req, res,'
         res.setbodybytes(_('invalid wire protocol command: %s') % command)
         return
 
+    if req.headers.get(b'Accept') != HTTPV2TYPE:
+        res.status = b'406 Not Acceptable'
+        res.headers[b'Content-Type'] = b'text/plain'
+        res.setbodybytes(_('client MUST specify Accept header with value: %s\n')
+                           % HTTPV2TYPE)
+        return
+
+    if (b'Content-Type' in req.headers
+        and req.headers[b'Content-Type'] != HTTPV2TYPE):
+        res.status = b'415 Unsupported Media Type'
+        # TODO we should send a response with appropriate media type,
+        # since client does Accept it.
+        res.headers[b'Content-Type'] = b'text/plain'
+        res.setbodybytes(_('client MUST send Content-Type header with '
+                           'value: %s\n') % HTTPV2TYPE)
+        return
+
     # We don't do anything meaningful yet.
     res.status = b'200 OK'
     res.headers[b'Content-Type'] = b'text/plain'

tests/test-http-api-httpv2.t

@@ -1,4 +1,5 b''
   $ HTTPV2=exp-http-v2-0001
+  $ MEDIATYPE=application/mercurial-tbd
 
   $ send() {
   >   hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT/
@@ -60,27 +61,6 b' Restart server with support for HTTP v2 '
   $ hg -R server serve -p $HGPORT -d --pid-file hg.pid
   $ cat hg.pid > $DAEMON_PIDS
 
-Request to read-only command works out of the box
-
-  $ send << EOF
-  > httprequest POST api/$HTTPV2/ro/customreadonly
-  >     user-agent: test
-  > EOF
-  using raw connection to peer
-  s>     POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
-  s>     Accept-Encoding: identity\r\n
-  s>     user-agent: test\r\n
-  s>     host: $LOCALIP:$HGPORT\r\n (glob)
-  s>     \r\n
-  s> makefile('rb', None)
-  s>     HTTP/1.1 200 OK\r\n
-  s>     Server: testing stub value\r\n
-  s>     Date: $HTTP_DATE$\r\n
-  s>     Content-Type: text/plain\r\n
-  s>     Content-Length: 18\r\n
-  s>     \r\n
-  s>     ro/customreadonly\n
-
 Request to unknown command yields 404
 
   $ send << EOF
@@ -123,6 +103,100 b' GET to read-only command yields a 405'
   s>     \r\n
   s>     commands require POST requests
 
+Missing Accept header results in 406
+
+  $ send << EOF
+  > httprequest POST api/$HTTPV2/ro/customreadonly
+  >     user-agent: test
+  > EOF
+  using raw connection to peer
+  s>     POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     user-agent: test\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     \r\n
+  s> makefile('rb', None)
+  s>     HTTP/1.1 406 Not Acceptable\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: text/plain\r\n
+  s>     Content-Length: 72\r\n
+  s>     \r\n
+  s>     client MUST specify Accept header with value: application/mercurial-tbd\n
+
+Bad Accept header results in 406
+
+  $ send << EOF
+  > httprequest POST api/$HTTPV2/ro/customreadonly
+  >     accept: invalid
+  >     user-agent: test
+  > EOF
+  using raw connection to peer
+  s>     POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: invalid\r\n
+  s>     user-agent: test\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     \r\n
+  s> makefile('rb', None)
+  s>     HTTP/1.1 406 Not Acceptable\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: text/plain\r\n
+  s>     Content-Length: 72\r\n
+  s>     \r\n
+  s>     client MUST specify Accept header with value: application/mercurial-tbd\n
+
+Bad Content-Type header results in 415
+
+  $ send << EOF
+  > httprequest POST api/$HTTPV2/ro/customreadonly
+  >     accept: $MEDIATYPE
+  >     user-agent: test
+  >     content-type: badmedia
+  > EOF
+  using raw connection to peer
+  s>     POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-tbd\r\n
+  s>     content-type: badmedia\r\n
+  s>     user-agent: test\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     \r\n
+  s> makefile('rb', None)
+  s>     HTTP/1.1 415 Unsupported Media Type\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: text/plain\r\n
+  s>     Content-Length: 75\r\n
+  s>     \r\n
+  s>     client MUST send Content-Type header with value: application/mercurial-tbd\n
+
+Request to read-only command works out of the box
+
+  $ send << EOF
+  > httprequest POST api/$HTTPV2/ro/customreadonly
+  >     accept: $MEDIATYPE
+  >     content-type: $MEDIATYPE
+  >     user-agent: test
+  > EOF
+  using raw connection to peer
+  s>     POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
+  s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-tbd\r\n
+  s>     content-type: application/mercurial-tbd\r\n
+  s>     user-agent: test\r\n
+  s>     host: $LOCALIP:$HGPORT\r\n (glob)
+  s>     \r\n
+  s> makefile('rb', None)
+  s>     HTTP/1.1 200 OK\r\n
+  s>     Server: testing stub value\r\n
+  s>     Date: $HTTP_DATE$\r\n
+  s>     Content-Type: text/plain\r\n
+  s>     Content-Length: 18\r\n
+  s>     \r\n
+  s>     ro/customreadonly\n
+
 Request to read-write command fails because server is read-only by default
 
 GET to read-write request yields 405
@@ -207,10 +281,14 b' Authorized request for valid read-write '
   $ send << EOF
   > httprequest POST api/$HTTPV2/rw/customreadonly
   >     user-agent: test
+  >     accept: $MEDIATYPE
+  >     content-type: $MEDIATYPE
   > EOF
   using raw connection to peer
   s>     POST /api/exp-http-v2-0001/rw/customreadonly HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-tbd\r\n
+  s>     content-type: application/mercurial-tbd\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n
@@ -228,10 +306,12 b' Authorized request for unknown command i'
   $ send << EOF
   > httprequest POST api/$HTTPV2/rw/badcommand
   >     user-agent: test
+  >     accept: $MEDIATYPE
   > EOF
   using raw connection to peer
   s>     POST /api/exp-http-v2-0001/rw/badcommand HTTP/1.1\r\n
   s>     Accept-Encoding: identity\r\n
+  s>     accept: application/mercurial-tbd\r\n
   s>     user-agent: test\r\n
   s>     host: $LOCALIP:$HGPORT\r\n (glob)
   s>     \r\n