rebase: don't run IMM if running rebase in a transaction

Some callers to rebase.rebase(), like `_moverelative` in `fbamend/movement.py`,
wrap the entire rebase call in a transaction. This raises havoc when IMM tries
to retry the rebase when it hits merge conflicts, because the abort will fail
the whole transaction, not the subset. It also fails at the end, losing any
conflict resolution, as @sid0 noticed.

The right long-term fix that @quark and I have discussed is to change the
restarting logic such that it doesn't abort at all, but simply switches between
IMM and non-IMM fluidly for each commit, which has other nice properties. In
the meantime this will do for now.

Differential Revision: https://phab.mercurial-scm.org/D1782

hgext/rebase.py

@@ -771,8 +771,11 b' def rebase(ui, repo, **opts):'
 
     """
     inmemory = ui.configbool('rebase', 'experimental.inmemory')
-    if opts.get('continue') or opts.get('abort'):
+    if (opts.get('continue') or opts.get('abort') or
+        repo.currenttransaction() is not None):
         # in-memory rebase is not compatible with resuming rebases.
+        # (Or if it is run within a transaction, since the restart logic can
+        # fail the entire transaction.)
         inmemory = False
 
     if inmemory: