dirstate: walk returns None for files that have a symlink in their path

Previously dirstate.walk would return a stat object for files in the dmap
that have a symlink to a directory in their path.  Now it will return None
to indicate that they are no longer considered part of the repository. This
currently only affects walks that traverse the entire directory tree (ex:
hg status) and not walks that only list the contents of the dmap (ex: hg diff).

In a situation like this:
  mkdir foo && touch foo/a && hg commit -Am "a"
  mv foo bar
  ln -s bar foo

'hg status' will now show '! foo/a', whereas before it incorrectly considered
'foo/a' to be unchanged.

In addition to making 'hg status' report the correct information, this will
allow callers to dirstate.walk to not have to detect symlinks themselves,
which can be very expensive.

mercurial/dirstate.py

@@ -677,9 +677,26 b' class dirstate(object):'
         # step 3: report unseen items in the dmap hash
         if not skipstep3 and not exact:
             visit = sorted([f for f in dmap if f not in results and matchfn(f)])
-            nf = iter(visit).next
-            for st in util.statfiles([join(i) for i in visit]):
-                results[nf()] = st
+            if unknown:
+                # unknown == True means we walked the full directory tree above.
+                # So if a file is not seen it was either a) not matching matchfn
+                # b) ignored, c) missing, or d) under a symlink directory.
+                audit_path = scmutil.pathauditor(self._root)
+
+                for nf in iter(visit):
+                    # Report ignored items in the dmap as long as they are not
+                    # under a symlink directory.
+                    if ignore(nf) and audit_path.check(nf):
+                        results[nf] = util.statfiles([join(nf)])[0]
+                    else:
+                        # It's either missing or under a symlink directory
+                        results[nf] = None
+            else:
+                # We may not have walked the full directory tree above,
+                # so stat everything we missed.
+                nf = iter(visit).next
+                for st in util.statfiles([join(i) for i in visit]):
+                    results[nf()] = st
         for s in subrepos:
             del results[s]
         del results['.hg']

tests/test-symlinks.t

@@ -149,6 +149,10 b' directory moved and symlinked'
   adding foo/a
   $ mv foo bar
   $ ln -s bar foo
+  $ hg status
+  ! foo/a
+  ? bar/a
+  ? foo
 
 now addremove should remove old files