py3: make setup.py's hgcommand() consistently return bytes

Before this patch, it returned unicode when the command failed. That
made e.g. `make local PYTHON=python3` fail on an obsolete commit.

Differential Revision: https://phab.mercurial-scm.org/D8331

setup.py

@@ -274,7 +274,7 b' class hgcommand(object):'
         if err or returncode != 0:
             printf("stderr from '%s':" % (' '.join(cmd)), file=sys.stderr)
             printf(err, file=sys.stderr)
-            return ''
+            return b''
         return out