windows: add a method to convert Unix style command lines to Windows style

This started as a copy/paste of `os.path.expandvars()`, but limited to a given
dictionary of variables, converting `foo = foo + bar` to `foo += bar`, and
adding 'b' string prefixes.  Then code was added to make sure that a value being
substituted in wouldn't itself be expanded by cmd.exe.  But that left
inconsistent results between `$var1` and `%var1%` when its value was '%foo%'-
since neither were touched, `$var1` wouldn't expand but `%var1%` would.  So
instead, this just converts the Unix style to Windows style (if the variable
exists, because Windows will leave `%missing%` as-is), and lets cmd.exe do its
thing.

I then dropped the %% -> % conversion (because Windows doesn't do this), and
added the ability to escape the '$' with '\'.  The escape character is dropped,
for consistency with shell handling.

After everything seemed stable and working, running the whole test suite flagged
a problem near the end of test-bookmarks.t:1069.  The problem is cmd.exe won't
pass empty variables to its child, so defined but empty variables are now
skipped.  I can't think of anything better, and it seems like a pre-existing
violation of the documentation, which calls out that HG_OLDNODE is empty on
bookmark creation.

Future additions could potentially be replacing strong quotes with double quotes
(cmd.exe doesn't know what to do with the former), escaping a double quote, and
some tilde expansion via os.path.expanduser().  I've got some doubts about
replacing the strong quotes in case sh.exe is run, but it seems like the right
thing to do the vast majority of the time.  The original form of this was
discussed about a year ago[1].

[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-July/100735.html

mercurial/windows.py

@@ -12,6 +12,7 b' import msvcrt'
 import os
 import re
 import stat
+import string
 import sys
 
 from .i18n import _
@@ -253,6 +254,108 b' normcasefallback = encoding.upperfallbac'
 def samestat(s1, s2):
     return False
 
+def shelltocmdexe(path, env):
+    r"""Convert shell variables in the form $var and ${var} inside ``path``
+    to %var% form.  Existing Windows style variables are left unchanged.
+
+    The variables are limited to the given environment.  Unknown variables are
+    left unchanged.
+
+    >>> e = {b'var1': b'v1', b'var2': b'v2', b'var3': b'v3'}
+    >>> # Only valid values are expanded
+    >>> shelltocmdexe(b'cmd $var1 ${var2} %var3% $missing ${missing} %missing%',
+    ...               e)
+    'cmd %var1% %var2% %var3% $missing ${missing} %missing%'
+    >>> # Single quote prevents expansion, as does \$ escaping
+    >>> shelltocmdexe(b"cmd '$var1 ${var2} %var3%' \$var1 \${var2} \\", e)
+    "cmd '$var1 ${var2} %var3%' $var1 ${var2} \\"
+    >>> # $$ -> $, %% is not special, but can be the end and start of variables
+    >>> shelltocmdexe(b"cmd $$ %% %var1%%var2%", e)
+    'cmd $ %% %var1%%var2%'
+    >>> # No double substitution
+    >>> shelltocmdexe(b"$var1 %var1%", {b'var1': b'%var2%', b'var2': b'boom'})
+    '%var1% %var1%'
+    """
+    if b'$' not in path:
+        return path
+
+    varchars = pycompat.sysbytes(string.ascii_letters + string.digits) + b'_-'
+
+    res = b''
+    index = 0
+    pathlen = len(path)
+    while index < pathlen:
+        c = path[index]
+        if c == b'\'':   # no expansion within single quotes
+            path = path[index + 1:]
+            pathlen = len(path)
+            try:
+                index = path.index(b'\'')
+                res += b'\'' + path[:index + 1]
+            except ValueError:
+                res += c + path
+                index = pathlen - 1
+        elif c == b'%':  # variable
+            path = path[index + 1:]
+            pathlen = len(path)
+            try:
+                index = path.index(b'%')
+            except ValueError:
+                res += b'%' + path
+                index = pathlen - 1
+            else:
+                var = path[:index]
+                res += b'%' + var + b'%'
+        elif c == b'$':  # variable or '$$'
+            if path[index + 1:index + 2] == b'$':
+                res += c
+                index += 1
+            elif path[index + 1:index + 2] == b'{':
+                path = path[index + 2:]
+                pathlen = len(path)
+                try:
+                    index = path.index(b'}')
+                    var = path[:index]
+
+                    # See below for why empty variables are handled specially
+                    if env.get(var, '') != '':
+                        res += b'%' + var + b'%'
+                    else:
+                        res += b'${' + var + b'}'
+                except ValueError:
+                    res += b'${' + path
+                    index = pathlen - 1
+            else:
+                var = b''
+                index += 1
+                c = path[index:index + 1]
+                while c != b'' and c in varchars:
+                    var += c
+                    index += 1
+                    c = path[index:index + 1]
+                # Some variables (like HG_OLDNODE) may be defined, but have an
+                # empty value.  Those need to be skipped because when spawning
+                # cmd.exe to run the hook, it doesn't replace %VAR% for an empty
+                # VAR, and that really confuses things like revset expressions.
+                # OTOH, if it's left in Unix format and the hook runs sh.exe, it
+                # will substitute to an empty string, and everything is happy.
+                if env.get(var, '') != '':
+                    res += b'%' + var + b'%'
+                else:
+                    res += b'$' + var
+
+                if c != '':
+                    index -= 1
+        elif c == b'\\' and index + 1 < pathlen and path[index + 1] == b'$':
+            # Skip '\', but only if it is escaping $
+            res += b'$'
+            index += 1
+        else:
+            res += c
+
+        index += 1
+    return res
+
 # A sequence of backslashes is special iff it precedes a double quote:
 # - if there's an even number of backslashes, the double quote is not
 #   quoted (i.e. it ends the quoted region)