Document the code execution semantics much more carefully.

Also renamed user var/expression functions to match the fields in the
execute request.

IPython/core/interactiveshell.py

@@ -1964,10 +1964,17 b' class InteractiveShell(Configurable, Magic):'
         etype, value = sys.exc_info()[:2]
         return u'[ERROR] {e.__name__}: {v}'.format(e=etype, v=value)
 
-    def get_user_variables(self, names):
+    def user_variables(self, names):
         """Get a list of variable names from the user's namespace.
 
-        The return value is a dict with the repr() of each value.
+        Parameters
+        ----------
+        names : list of strings
+          A list of names of variables to be read from the user namespace.
+
+        Returns
+        -------
+        A dict, keyed by the input names and with the repr() of each value.
         """
         out = {}
         user_ns = self.user_ns
@@ -1979,10 +1986,20 b' class InteractiveShell(Configurable, Magic):'
             out[varname] = value
         return out
         
-    def eval_expressions(self, expressions):
+    def user_expressions(self, expressions):
         """Evaluate a dict of expressions in the user's namespace.
 
-        The return value is a dict with the repr() of each value.
+        Parameters
+        ----------
+        expressions : dict
+          A dict with string keys and string values.  The expression values
+          should be valid Python expressions, each of which will be evaluated
+          in the user namespace.
+        
+        Returns
+        -------
+        A dict, keyed like the input expressions dict, with the repr() of each
+        value.
         """
         out = {}
         user_ns = self.user_ns
@@ -2121,10 +2138,10 b' class InteractiveShell(Configurable, Magic):'
 
         If there's more than one block, it depends:
 
-        - if the last one is a single line long, run all but the last in
-        'exec' mode and the very last one in 'single' mode.  This makes it
-        easy to type simple expressions at the end to see computed values.
-        - otherwise (last one is also multiline), run all in 'exec' mode
+        - if the last one is no more than two lines long, run all but the last
+        in 'exec' mode and the very last one in 'single' mode.  This makes it
+        easy to type simple expressions at the end to see computed values.  -
+        otherwise (last one is also multiline), run all in 'exec' mode
 
         When code is executed in 'single' mode, :func:`sys.displayhook` fires,
         results are displayed and output prompts are computed.  In 'exec' mode,

IPython/zmq/ipkernel.py

@@ -250,9 +250,9 b' class Kernel(Configurable):'
         # or not.  If it did, we proceed to evaluate user_variables/expressions
         if reply_content['status'] == 'ok':
             reply_content[u'user_variables'] = \
-                         shell.get_user_variables(content[u'user_variables'])
+                         shell.user_variables(content[u'user_variables'])
             reply_content[u'user_expressions'] = \
-                         shell.eval_expressions(content[u'user_expressions'])
+                         shell.user_expressions(content[u'user_expressions'])
         else:
             # If there was an error, don't even try to compute variables or
             # expressions

docs/source/development/messaging.txt

@@ -160,30 +160,23 b' Message type: ``execute_request``::'
     'user_expressions' : dict,
     }
 
-The ``code`` field contains a single string, but this may be a multiline
-string.  The kernel is responsible for splitting this into possibly more than
-one block and deciding whether to compile these in 'single' or 'exec' mode.
-We're still sorting out this policy.  The current inputsplitter is capable of
-splitting the input for blocks that can all be run as 'single', but in the long
-run it may prove cleaner to only use 'single' mode for truly single-line
-inputs, and run all multiline input in 'exec' mode.  This would preserve the
-natural behavior of single-line inputs while allowing long cells to behave more
-likea a script.  This design will be refined as we complete the implementation.
+The ``code`` field contains a single string (possibly multiline).  The kernel
+is responsible for splitting this into one or more independent execution blocks
+and deciding whether to compile these in 'single' or 'exec' mode (see below for
+detailed execution semantics).
 
 The ``user_`` fields deserve a detailed explanation.  In the past, IPython had
 the notion of a prompt string that allowed arbitrary code to be evaluated, and
 this was put to good use by many in creating prompts that displayed system
 status, path information, and even more esoteric uses like remote instrument
 status aqcuired over the network.  But now that IPython has a clean separation
-between the kernel and the clients, the notion of embedding 'prompt'
-maninpulations into the kernel itself feels awkward.  Prompts should be a
-frontend-side feature, and it should be even possible for different frontends
-to display different prompts while interacting with the same kernel.
+between the kernel and the clients, the kernel has no prompt knowledge; prompts
+are a frontend-side feature, and it should be even possible for different
+frontends to display different prompts while interacting with the same kernel.
 
-We have therefore abandoned the idea of a 'prompt string' to be evaluated by
-the kernel, and instead provide the ability to retrieve from the user's
-namespace information after the execution of the main ``code``, with two fields
-of the execution request:
+The kernel now provides the ability to retrieve data from the user's namespace
+after the execution of the main ``code``, thanks to two fields in the
+``execute_request`` message:
 
 - ``user_variables``: If only variables from the user's namespace are needed, a
   list of variable names can be passed and a dict with these names as keys and
@@ -205,32 +198,106 b' terminal, etc).'
    empty code string and ``silent=True``.
 
 Execution semantics
-    Upon completion of the execution request, the kernel *always* sends a
-    reply, with a status code indicating what happened and additional data
-    depending on the outcome.
+~~~~~~~~~~~~~~~~~~~
 
-    The ``code`` field is executed first, and then the ``user_variables`` and
-    ``user_expressions`` are computed.  This ensures that any error in the
-    latter don't harm the main code execution.
+When the silent flag is false, the execution of use code consists of the
+following phases (in silent mode, only the ``code`` field is executed):
 
-    Any error in retrieving the ``user_variables`` or evaluating the
-    ``user_expressions`` will result in a simple error message in the return
-    fields of the form::
+1. Run the ``pre_runcode_hook``.
 
-       [ERROR] ExceptionType: Exception message
+2. Execute the ``code`` field, see below for details.
+
+3. If #2 succeeds, compute ``user_variables`` and ``user_expressions`` are
+   computed.  This ensures that any error in the latter don't harm the main
+   code execution.
+
+4. Call any method registered with :meth:`register_post_execute`.
+
+.. warning::
+
+   The API for running code before/after the main code block is likely to
+   change soon.  Both the ``pre_runcode_hook`` and the
+   :meth:`register_post_execute` are susceptible to modification, as we find a
+   consistent model for both.
+
+To understand how the ``code`` field is executed, one must know that Python
+code can be compiled in one of three modes (controlled by the ``mode`` argument
+to the :func:`compile` builtin):
+
+*single*
+  Valid for a single interactive statement (though the source can contain
+  multiple lines, such as a for loop).  When compiled in this mode, the
+  generated bytecode contains special instructions that trigger the calling of
+  :func:`sys.displayhook` for any expression in the block that returns a value.
+  This means that a single statement can actually produce multiple calls to
+  :func:`sys.displayhook`, if for example it contains a loop where each
+  iteration computes an unassigned expression would generate 10 calls::
+
+      for i in range(10):
+          i**2
+
+*exec*
+  An arbitrary amount of source code, this is how modules are compiled.
+  :func:`sys.displayhook` is *never* implicitly called.
+
+*eval*
+  A single expression that returns a value.  :func:`sys.displayhook` is *never*
+  implicitly called.
+
+
+The ``code`` field is split into individual blocks each of which is valid for
+execution in 'single' mode, and then:
+
+- If there is only a single block: it is executed in 'single' mode.
+
+- If there is more than one block:
+
+  * if the last one is a single line long, run all but the last in 'exec' mode
+    and the very last one in 'single' mode.  This makes it easy to type simple
+    expressions at the end to see computed values.
+
+  * if the last one is no more than two lines long, run all but the last in
+    'exec' mode and the very last one in 'single' mode.  This makes it easy to
+    type simple expressions at the end to see computed values.  - otherwise
+    (last one is also multiline), run all in 'exec' mode
+
+  * otherwise (last one is also multiline), run all in 'exec' mode as a single
+    unit.
+
+Any error in retrieving the ``user_variables`` or evaluating the
+``user_expressions`` will result in a simple error message in the return fields
+of the form::
+
+   [ERROR] ExceptionType: Exception message
+
+The user can simply send the same variable name or expression for evaluation to
+see a regular traceback.
+
+Errors in any registered post_execute functions are also reported similarly,
+and the failing function is removed from the post_execution set so that it does
+not continue triggering failures.
+
+Upon completion of the execution request, the kernel *always* sends a reply,
+with a status code indicating what happened and additional data depending on
+the outcome.  See :ref:`below <execution_results>` for the possible return
+codes and associated data.
 
-    The user can simply send the same variable name or expression for
-    evaluation to see a regular traceback.
 
 Execution counter (old prompt number)
-    The kernel has a single, monotonically increasing counter of all execution
-    requests that are made with ``silent=False``.  This counter is used to
-    populate the ``In[n]``, ``Out[n]`` and ``_n`` variables, so clients will
-    likely want to display it in some form to the user, which will typically
-    (but not necessarily) be done in the prompts.  The value of this counter
-    will be returned as the ``execution_count`` field of all ``execute_reply```
-    messages.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The kernel has a single, monotonically increasing counter of all execution
+requests that are made with ``silent=False``.  This counter is used to populate
+the ``In[n]``, ``Out[n]`` and ``_n`` variables, so clients will likely want to
+display it in some form to the user, which will typically (but not necessarily)
+be done in the prompts.  The value of this counter will be returned as the
+``execution_count`` field of all ``execute_reply`` messages.
+
+.. _execution_results:
 
+Execution results
+~~~~~~~~~~~~~~~~~
+    
 Message type: ``execute_reply``::
 
     content = {
@@ -303,6 +370,11 b' happens when the kernel was interrupted by a signal.'
 Kernel attribute access
 -----------------------
 
+.. warning::
+
+   This part of the messaging spec is not actually implemented in the kernel
+   yet.
+ 
 While this protocol does not specify full RPC access to arbitrary methods of
 the kernel object, the kernel does allow read (and in some cases write) access
 to certain attributes.
@@ -361,6 +433,7 b' Message type: ``setattr_reply``::'
     }
 
 
+    
 Object information
 ------------------