encoding: default ambiguous character to narrow

The current implementation of colwidth was treating 'A'mbiguous
characters as wide, which was incorrect in a non-East Asian context.
As per http://unicode.org/reports/tr11/#Recommendations, we should
instead default to 'narrow' if we don't know better. As character
width is dependent on the particular font used and we have no idea
what fonts are in use, this recommendation applies.

This introduces HGENCODINGAMBIGUOUS to get the old behavior back.

mercurial/encoding.py

@@ -87,11 +87,17 b' def fromlocal(s):'
     except LookupError, k:
         raise error.Abort("%s, please check your locale settings" % k)
 
+# How to treat ambiguous-width characters. Set to 'wide' to treat as wide.
+ambiguous = os.environ.get("HGENCODINGAMBIGUOUS", "narrow")
+
 def colwidth(s):
     "Find the column width of a UTF-8 string for display"
     d = s.decode(encoding, 'replace')
     if hasattr(unicodedata, 'east_asian_width'):
+        wide = "WF"
+        if ambiguous == "wide":
+            wide = "WFA"
         w = unicodedata.east_asian_width
-        return sum([w(c) in 'WFA' and 2 or 1 for c in d])
+        return sum([w(c) in wide and 2 or 1 for c in d])
     return len(d)
 

mercurial/help/environment.txt

@@ -24,6 +24,13 b' HGENCODINGMODE'
     "ignore", which drops them. This setting can be overridden with
     the --encodingmode command-line option.
 
+HGENCODINGAMBIGUOUS
+    This sets Mercurial's behavior for handling characters with
+    "ambiguous" widths like accented Latin characters with East Asian
+    fonts. By default, Mercurial assumes ambiguous characters are
+    narrow, set this variable to "wide" if such characters cause
+    formatting problems.
+
 HGMERGE
     An executable to use for resolving merge conflicts. The program
     will be executed with three arguments: local file, remote file,

tests/test-encoding.t

@@ -166,13 +166,13 b' hg tags (latin-1)'
 
   $ HGENCODING=latin-1 hg tags
   tip                                5:093c6077d1c8
-  �                                 3:ca661e7520de
+  �                                  3:ca661e7520de
 
 hg tags (utf-8)
 
   $ HGENCODING=utf-8 hg tags
   tip                                5:093c6077d1c8
-  é                                 3:ca661e7520de
+  é                                  3:ca661e7520de
 
 hg branches (ascii)
 
@@ -183,13 +183,13 b' hg branches (ascii)'
 hg branches (latin-1)
 
   $ HGENCODING=latin-1 hg branches
-  �                             5:093c6077d1c8
+  �                              5:093c6077d1c8
   default                        4:94db611b4196 (inactive)
 
 hg branches (utf-8)
 
   $ HGENCODING=utf-8 hg branches
-  é                             5:093c6077d1c8
+  é                              5:093c6077d1c8
   default                        4:94db611b4196 (inactive)
   $ echo '[ui]' >> .hg/hgrc
   $ echo 'fallbackencoding = koi8-r' >> .hg/hgrc