scmutil: consistently return subrepos relative to ctx1 from itersubrepos()

Previously, if a subrepo was added in ctx2 and then compared to another without
it (ctx1), the subrepo for ctx2 was returned amongst all of the ctx1 based
subrepos, since no subrepo exists in ctx1 to replace it in the 'subpaths' dict.
The two callers of this, basectx.status() and cmdutil.diffordiffstat(), both
compare the yielded subrepo against ctx2, and thus saw no changes when ctx2's
subrepo was returned.  The tests here previously didn't mention 's/a' for the
'p1()' case.

This appears to have been a known issue, because some diffordiffstat() comments
mention that the subpath disappeared, and "the best we can do is ignore it".  I
originally ran into the issue with some custom convert code to flatten a tree of
subrepos causing hg.putcommit() to abort, but this new behavior seems like the
correct status and diff behavior regardless.  (The abort in convert isn't
something users will see, because convert doesn't currently support subrepos in
the official repo.)

mercurial/scmutil.py

@@ -80,9 +80,24 b' def itersubrepos(ctx1, ctx2):'
     # has been modified (in ctx2) but not yet committed (in ctx1).
     subpaths = dict.fromkeys(ctx2.substate, ctx2)
     subpaths.update(dict.fromkeys(ctx1.substate, ctx1))
+
+    missing = set()
+
+    for subpath in ctx2.substate:
+        if subpath not in ctx1.substate:
+            del subpaths[subpath]
+            missing.add(subpath)
+
     for subpath, ctx in sorted(subpaths.iteritems()):
         yield subpath, ctx.sub(subpath)
 
+    # Yield an empty subrepo based on ctx1 for anything only in ctx2.  That way,
+    # status and diff will have an accurate result when it does
+    # 'sub.{status|diff}(rev2)'.  Otherwise, the ctx2 subrepo is compared
+    # against itself.
+    for subpath in missing:
+        yield subpath, ctx2.nullsub(subpath, ctx1)
+
 def nochangesfound(ui, repo, excluded=None):
     '''Report no changes for push/pull, excluded is None or a list of
     nodes excluded from the push/pull.

tests/test-mq-subrepo.t

@@ -106,6 +106,7 b' handle subrepos safely on qnew'
   [255]
   % update substate when adding .hgsub w/clean updated subrepo
   A .hgsub
+  A sub/a
   % qnew -X path:no-effect -m0 0.diff
   path sub
    source   sub
@@ -121,6 +122,7 b' handle subrepos safely on qnew'
   [255]
   % update substate when modifying .hgsub w/clean updated subrepo
   M .hgsub
+  A sub2/a
   % qnew --cwd .. -R repo-2499-qnew -X path:no-effect -m1 1.diff
   path sub
    source   sub
@@ -165,6 +167,7 b' handle subrepos safely on qrefresh'
   [255]
   % update substate when adding .hgsub w/clean updated subrepo
   A .hgsub
+  A sub/a
   % qrefresh
   path sub
    source   sub
@@ -181,6 +184,7 b' handle subrepos safely on qrefresh'
   [255]
   % update substate when modifying .hgsub w/clean updated subrepo
   M .hgsub
+  A sub2/a
   % qrefresh
   path sub
    source   sub
@@ -304,6 +308,7 b' handle subrepos safely on qrecord'
   [255]
   % update substate when adding .hgsub w/clean updated subrepo
   A .hgsub
+  A sub/a
   % qrecord --config ui.interactive=1 -m0 0.diff
   diff --git a/.hgsub b/.hgsub
   new file mode 100644
@@ -339,6 +344,7 b' handle subrepos safely on qrecord'
   [255]
   % update substate when modifying .hgsub w/clean updated subrepo
   M .hgsub
+  A sub2/a
   % qrecord --config ui.interactive=1 -m1 1.diff
   diff --git a/.hgsub b/.hgsub
   1 hunks, 1 lines changed

tests/test-subrepo.t

@@ -1703,4 +1703,37 b" Test that '[paths]' is configured correc"
   [paths]
   default = $TESTTMP/t/t
   default-push = /foo/bar/t
+
+  $ cd $TESTTMP/t
+  $ hg up -qC 0
+  $ echo 'bar' > bar.txt
+  $ hg ci -Am 'branch before subrepo add'
+  adding bar.txt
+  created new head
+  $ hg merge -r "first(subrepo('s'))"
+  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ hg status -S -X '.hgsub*'
+  A s/a
+  ? s/b
+  ? s/c
+  ? s/f1
+  $ hg status -S --rev 'p2()'
+  A bar.txt
+  ? s/b
+  ? s/c
+  ? s/f1
+  $ hg diff -S -X '.hgsub*' --nodates
+  diff -r 000000000000 s/a
+  --- /dev/null
+  +++ b/s/a
+  @@ -0,0 +1,1 @@
+  +a
+  $ hg diff -S --rev 'p2()' --nodates
+  diff -r 7cf8cfea66e4 bar.txt
+  --- /dev/null
+  +++ b/bar.txt
+  @@ -0,0 +1,1 @@
+  +bar
+
   $ cd ..