localrepo: execute appropriate actions for dirstate at releasing transaction

Before this patch, in-memory dirstate changes are still kept over a
transaction scope boundary regardless of the result of it.

For "all or nothing" policy of the transaction, in-memory dirstate
changes should be:

  - written out at successful closing a transaction, because
    subsequent 'dirstate.invalidate()' can lose them

  - discarded at failure of a transaction, because outer
    'wlock.release()' or so may write them out

To discard all changes in a transaction completely, this patch also
restores '.hg/dirstate' by '.hg/journal.dirstate' at failure, because
'transaction' itself does nothing for files related to '.hg/journal.*'
in such case (therefore, renaming in this patch is safe enough).

This is a part of preparations for "transactional dirstate". See also
the wiki page below for detail about it.

    https://mercurial.selenic.com/wiki/DirstateTransactionPlan

This patch also removes redundant 'dirstate.invalidate()' just before
aborting a transaction for shelve/unshelve.

hgext/shelve.py

@@ -209,9 +209,6 b' def _aborttransaction(repo):'
         tr = repo.currenttransaction()
         tr.abort()
 
-        # TODO: this should be done via transaction.abort()
-        repo.dirstate.invalidate() # prevent wlock from writing changes out
-
         # restore to backuped dirstate
         repo.vfs.rename(dirstatebackup, 'dirstate')
         dirstatebackup = None

mercurial/localrepo.py

@@ -997,13 +997,26 b' class localrepository(object):'
             pending = lambda: tr.writepending() and self.root or ""
             reporef().hook('pretxnclose', throw=True, pending=pending,
                            txnname=desc, **tr.hookargs)
+        def releasefn(tr, success):
+            repo = reporef()
+            if success:
+                repo.dirstate.write()
+            else:
+                # prevent in-memory changes from being written out at
+                # the end of outer wlock scope or so
+                repo.dirstate.invalidate()
+
+                # discard all changes (including ones already written
+                # out) in this transaction
+                repo.vfs.rename('journal.dirstate', 'dirstate')
 
         tr = transaction.transaction(rp, self.svfs, vfsmap,
                                      "journal",
                                      "undo",
                                      aftertrans(renames),
                                      self.store.createmode,
-                                     validator=validate)
+                                     validator=validate,
+                                     releasefn=releasefn)
 
         tr.hookargs['txnid'] = txnid
         # note: writing the fncache only during finalize mean that the file is