revsetlang: do not pass in non-bytes to parse()

Since parse() isn't a simple function, we shouldn't expect it would raise
TypeError or ValueError for invalid inputs. Before, TypeError was raised
at 'if pos != len(spec)', which was quite late to report an error.

This patch also makes tokenize() detect invalid object before converting
it to a py3-safe bytes.

Spotted while adding the 'revset(...)' hack to _parsewith().

mercurial/revsetlang.py

@@ -89,6 +89,9 b' def tokenize(program, lookup=None, symin'
     [('symbol', '@', 0), ('::', None, 1), ('end', None, 3)]
 
     '''
+    if not isinstance(program, bytes):
+        raise error.ProgrammingError('revset statement must be bytes, got %r'
+                                     % program)
     program = pycompat.bytestr(program)
     if syminitletters is None:
         syminitletters = _syminitletters
@@ -581,6 +584,8 b' def _formatargtype(c, arg):'
     elif c == 's':
         return _quote(arg)
     elif c == 'r':
+        if not isinstance(arg, bytes):
+            raise TypeError
         parse(arg) # make sure syntax errors are confined
         return '(%s)' % arg
     elif c == 'n':