hgweb: support Content Security Policy

Content-Security-Policy (CSP) is a web security feature that allows
servers to declare what loaded content is allowed to do. For example,
a policy can prevent loading of images, JavaScript, CSS, etc unless
the source of that content is whitelisted (by hostname, URI scheme,
hashes of content, etc). It's a nifty security feature that provides
extra mitigation against some attacks, notably XSS.

Mitigation against these attacks is important for Mercurial because
hgweb renders repository data, which is commonly untrusted. While we
make attempts to escape things, etc, there's the possibility that
malicious data could be injected into the site content. If this happens
today, the full power of the web browser is available to that
malicious content. A restrictive CSP policy (defined by the server
operator and sent in an HTTP header which is outside the control of
malicious content), could restrict browser capabilities and mitigate
security problems posed by malicious data.

CSP works by emitting an HTTP header declaring the policy that browsers
should apply. Ideally, this header would be emitted by a layer above
Mercurial (likely the HTTP server doing the WSGI "proxying"). This
works for some CSP policies, but not all.

For example, policies to allow inline JavaScript may require setting
a "nonce" attribute on <script>. This attribute value must be unique
and non-guessable. And, the value must be present in the HTTP header
and the HTML body. This means that coordinating the value between
Mercurial and another HTTP server could be difficult: it is much
easier to generate and emit the nonce in a central location.

This commit introduces support for emitting a
Content-Security-Policy header from hgweb. A config option defines
the header value. If present, the header is emitted. A special
"%nonce%" syntax in the value triggers generation of a nonce and
inclusion in <script> elements in templates. The inclusion of a
nonce does not occur unless "%nonce%" is present. This makes this
commit completely backwards compatible and the feature opt-in.

The nonce is a type 4 UUID, which is the flavor that is randomly
generated. It has 122 random bits, which should be plenty to satisfy
the guarantees of a nonce.

tests/test-hgweb-csp.t

@@ -0,0 +1,129 b''
+#require serve
+
+  $ cat > web.conf << EOF
+  > [paths]
+  > / = $TESTTMP/*
+  > EOF
+
+  $ hg init repo1
+  $ cd repo1
+  $ touch foo
+  $ hg -q commit -A -m initial
+  $ cd ..
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid >> $DAEMON_PIDS
+
+repo index should not send Content-Security-Policy header by default
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+
+static page should not send CSP by default
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+
+repo page should not send CSP by default, should send ETag
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  etag: W/"*" (glob)
+
+  $ killdaemons.py
+
+Configure CSP without nonce
+
+  $ cat >> web.conf << EOF
+  > [web]
+  > csp = script-src https://example.com/ 'unsafe-inline'
+  > EOF
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid > $DAEMON_PIDS
+
+repo index should send Content-Security-Policy header when enabled
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+
+static page should send CSP when enabled
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+
+repo page should send CSP by default, include etag w/o nonce
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+  etag: W/"*" (glob)
+
+nonce should not be added to html if CSP doesn't use it
+
+  $ get-with-headers.py localhost:$HGPORT repo1/graph/tip | egrep 'content-security-policy|<script'
+  <script type="text/javascript" src="/repo1/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/repo1/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript">
+  <script type="text/javascript">
+
+Configure CSP with nonce
+
+  $ killdaemons.py
+  $ cat >> web.conf << EOF
+  > csp = image-src 'self'; script-src https://example.com/ 'nonce-%nonce%'
+  > EOF
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid > $DAEMON_PIDS
+
+nonce should be substituted in CSP header
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce should be included in CSP for static pages
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+repo page should have nonce, no ETag
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce should be added to html when used
+
+  $ get-with-headers.py localhost:$HGPORT repo1/graph/tip content-security-policy | egrep 'content-security-policy|<script'
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+  <script type="text/javascript" src="/repo1/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/repo1/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript" nonce="*"> (glob)
+  <script type="text/javascript" nonce="*"> (glob)
+
+hgweb_mod w/o hgwebdir works as expected
+
+  $ killdaemons.py
+
+  $ hg -R repo1 serve -p $HGPORT -d --pid-file=hg.pid --config "web.csp=image-src 'self'; script-src https://example.com/ 'nonce-%nonce%'"
+  $ cat hg.pid > $DAEMON_PIDS
+
+static page sends CSP
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce included in <script> and headers
+
+  $ get-with-headers.py localhost:$HGPORT graph/tip content-security-policy  | egrep 'content-security-policy|<script'
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+  <script type="text/javascript" src="/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript" nonce="*"> (glob)
+  <script type="text/javascript" nonce="*"> (glob)

mercurial/help/config.txt

@@ -2111,6 +2111,20 b' The full set of options is:'
     Name or email address of the person in charge of the repository.
     (default: ui.username or ``$EMAIL`` or "unknown" if unset or empty)
 
+``csp``
+    Send a ``Content-Security-Policy`` HTTP header with this value.
+
+    The value may contain a special string ``%nonce%``, which will be replaced
+    by a randomly-generated one-time use value. If the value contains
+    ``%nonce%``, ``web.cache`` will be disabled, as caching undermines the
+    one-time property of the nonce. This nonce will also be inserted into
+    ``<script>`` elements containing inline JavaScript.
+
+    Note: lots of HTML content sent by the server is derived from repository
+    data. Please consider the potential for malicious repository data to
+    "inject" itself into generated HTML content as part of your security
+    threat model.
+
 ``deny_push``
     Whether to deny pushing to the repository. If empty or not set,
     push is not denied. If the special value ``*``, all remote users are

mercurial/hgweb/common.py

@@ -8,9 +8,11 b''
 
 from __future__ import absolute_import
 
+import base64
 import errno
 import mimetypes
 import os
+import uuid
 
 from .. import (
     encoding,
@@ -199,3 +201,22 b' def caching(web, req):'
     if req.env.get('HTTP_IF_NONE_MATCH') == tag:
         raise ErrorResponse(HTTP_NOT_MODIFIED)
     req.headers.append(('ETag', tag))
+
+def cspvalues(ui):
+    """Obtain the Content-Security-Policy header and nonce value.
+
+    Returns a 2-tuple of the CSP header value and the nonce value.
+
+    First value is ``None`` if CSP isn't enabled. Second value is ``None``
+    if CSP isn't enabled or if the CSP header doesn't need a nonce.
+    """
+    # Don't allow untrusted CSP setting since it be disable protections
+    # from a trusted/global source.
+    csp = ui.config('web', 'csp', untrusted=False)
+    nonce = None
+
+    if csp and '%nonce%' in csp:
+        nonce = base64.urlsafe_b64encode(uuid.uuid4().bytes).rstrip('=')
+        csp = csp.replace('%nonce%', nonce)
+
+    return csp, nonce

mercurial/hgweb/hgweb_mod.py

@@ -19,6 +19,7 b' from .common import ('
     HTTP_OK,
     HTTP_SERVER_ERROR,
     caching,
+    cspvalues,
     permhooks,
 )
 from .request import wsgirequest
@@ -115,6 +116,8 b' class requestcontext(object):'
         # of the request.
         self.websubtable = app.websubtable
 
+        self.csp, self.nonce = cspvalues(self.repo.ui)
+
     # Trust the settings from the .hg/hgrc files by default.
     def config(self, section, name, default=None, untrusted=True):
         return self.repo.ui.config(section, name, default,
@@ -201,6 +204,7 b' class requestcontext(object):'
             'sessionvars': sessionvars,
             'pathdef': makebreadcrumb(req.url),
             'style': style,
+            'nonce': self.nonce,
         }
         tmpl = templater.templater.frommapfile(mapfile,
                                                filters={'websub': websubfilter},
@@ -318,6 +322,13 b' class hgweb(object):'
         encoding.encoding = rctx.config('web', 'encoding', encoding.encoding)
         rctx.repo.ui.environ = req.env
 
+        if rctx.csp:
+            # hgwebdir may have added CSP header. Since we generate our own,
+            # replace it.
+            req.headers = [h for h in req.headers
+                           if h[0] != 'Content-Security-Policy']
+            req.headers.append(('Content-Security-Policy', rctx.csp))
+
         # work with CGI variables to create coherent structure
         # use SCRIPT_NAME, PATH_INFO and QUERY_STRING as well as our REPO_NAME
 
@@ -414,7 +425,9 b' class hgweb(object):'
                 req.form['cmd'] = [tmpl.cache['default']]
                 cmd = req.form['cmd'][0]
 
-            if rctx.configbool('web', 'cache', True):
+            # Don't enable caching if using a CSP nonce because then it wouldn't
+            # be a nonce.
+            if rctx.configbool('web', 'cache', True) and not rctx.nonce:
                 caching(self, req) # sets ETag header or raises NOT_MODIFIED
             if cmd not in webcommands.__all__:
                 msg = 'no such method: %s' % cmd

mercurial/hgweb/hgwebdir_mod.py

@@ -19,6 +19,7 b' from .common import ('
     HTTP_NOT_FOUND,
     HTTP_OK,
     HTTP_SERVER_ERROR,
+    cspvalues,
     get_contact,
     get_mtime,
     ismember,
@@ -227,8 +228,12 b' class hgwebdir(object):'
         try:
             self.refresh()
 
+            csp, nonce = cspvalues(self.ui)
+            if csp:
+                req.headers.append(('Content-Security-Policy', csp))
+
             virtual = req.env.get("PATH_INFO", "").strip('/')
-            tmpl = self.templater(req)
+            tmpl = self.templater(req, nonce)
             ctype = tmpl('mimetype', encoding=encoding.encoding)
             ctype = templater.stringify(ctype)
 
@@ -466,7 +471,7 b' class hgwebdir(object):'
                     sortcolumn=sortcolumn, descending=descending,
                     **dict(sort))
 
-    def templater(self, req):
+    def templater(self, req, nonce):
 
         def motd(**map):
             if self.motd is not None:
@@ -510,6 +515,7 b' class hgwebdir(object):'
             "staticurl": staticurl,
             "sessionvars": sessionvars,
             "style": style,
+            "nonce": nonce,
         }
         tmpl = templater.templater.frommapfile(mapfile, defaults=defaults)
         return tmpl

mercurial/templates/gitweb/graph.tmpl

@@ -45,7 +45,7 b' graph |'
 <ul id="graphnodes"></ul>
 </div>
 
-<script>
+<script{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};
@@ -108,7 +108,7 b' graph.render(data);'
 | {changenav%navgraph}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,

mercurial/templates/gitweb/shortlog.tmpl

@@ -40,7 +40,7 b' shortlog |'
 {changenav%navshort}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH

mercurial/templates/monoblue/graph.tmpl

@@ -40,7 +40,7 b''
         <ul id="graphnodes"></ul>
     </div>
 
-    <script>
+    <script{if(nonce, ' nonce="{nonce}"')}>
     <!-- hide script content
 
     document.getElementById('noscript').style.display = 'none';
@@ -104,7 +104,7 b''
         | {changenav%navgraph}
     </div>
 
-    <script type="text/javascript">
+    <script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,

mercurial/templates/monoblue/shortlog.tmpl

@@ -41,7 +41,7 b''
     {changenav%navshort}
     </div>
 
-    <script type="text/javascript">
+    <script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH

mercurial/templates/paper/graph.tmpl

@@ -59,7 +59,7 b''
 <ul id="graphnodes"></ul>
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};
@@ -121,7 +121,7 b' graph.render(data);'
 | rev {rev}: {changenav%navgraph}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,

mercurial/templates/paper/shortlog.tmpl

@@ -72,7 +72,7 b''
 | rev {rev}: {changenav%navshort}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH

mercurial/templates/spartan/graph.tmpl

@@ -36,7 +36,7 b' navigate: <small class="navigate">{chang'
 <ul id="graphnodes"></ul>
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};