Merge pull request #5666 from takluyver/inputhook-extensible

Refactor inputhook to allow easy extension

docs/source/config/eventloops.rst

@@ -0,0 +1,103 b''
+================================
+Integrating with GUI event loops
+================================
+
+When the user types ``%gui qt``, IPython integrates itself with the Qt event
+loop, so you can use both a GUI and an interactive prompt together. IPython
+supports a number of common GUI toolkits, but from IPython 3.0, it is possible
+to integrate other event loops without modifying IPython itself.
+
+Terminal IPython handles event loops very differently from the IPython kernel,
+so different steps are needed to integrate with each.
+
+Event loops in the terminal
+---------------------------
+
+In the terminal, IPython uses a blocking Python function to wait for user input.
+However, the Python C API provides a hook, :c:func:`PyOS_InputHook`, which is
+called frequently while waiting for input. This can be set to a function which
+briefly runs the event loop and then returns.
+
+IPython provides Python level wrappers for setting and resetting this hook. To
+use them, subclass :class:`IPython.lib.inputhook.InputHookBase`, and define
+an ``enable(app=None)`` method, which initialises the event loop and calls
+``self.manager.set_inputhook(f)`` with a function which will briefly run the
+event loop before exiting. Decorate the class with a call to
+:func:`IPython.lib.inputhook.register`::
+
+    from IPython.lib.inputhook import register, InputHookBase
+
+    @register('clutter')
+    class ClutterInputHook(InputHookBase):
+        def enable(self, app=None):
+            self.manager.set_inputhook(inputhook_clutter)
+
+You can also optionally define a ``disable()`` method, taking no arguments, if
+there are extra steps needed to clean up. IPython will take care of resetting
+the hook, whether or not you provide a disable method.
+
+The simplest way to define the hook function is just to run one iteration of the
+event loop, or to run until no events are pending. Most event loops provide some
+mechanism to do one of these things. However, the GUI may lag slightly,
+because the hook is only called every 0.1 seconds. Alternatively, the hook can
+keep running the event loop until there is input ready on stdin. IPython
+provides a function to facilitate this:
+
+.. currentmodule:: IPython.lib.inputhook
+
+.. function:: stdin_ready()
+
+   Returns True if there is something ready to read on stdin.
+   
+   If this is the case, the hook function should return immediately.
+   
+   This is implemented for Windows and POSIX systems - on other platforms, it
+   always returns True, so that the hook always gives Python a chance to check
+   for input.
+
+
+Event loops in the kernel
+-------------------------
+
+The kernel runs its own event loop, so it's simpler to integrate with others.
+IPython allows the other event loop to take control, but it must call
+:meth:`IPython.kernel.zmq.kernelbase.Kernel.do_one_iteration` periodically.
+
+To integrate with this, write a function that takes a single argument,
+the IPython kernel instance, arranges for your event loop to call
+``kernel.do_one_iteration()`` at least every ``kernel._poll_interval`` seconds,
+and starts the event loop.
+
+Decorate this function with :func:`IPython.kernel.zmq.eventloops.register_integration`,
+passing in the names you wish to register it for. Here is a slightly simplified
+version of the Tkinter integration already included in IPython::
+
+    @register_integration('tk')
+    def loop_tk(kernel):
+        """Start a kernel with the Tk event loop."""
+        from tkinter import Tk
+
+        # Tk uses milliseconds
+        poll_interval = int(1000*kernel._poll_interval)
+        # For Tkinter, we create a Tk object and call its withdraw method.
+        class Timer(object):
+            def __init__(self, func):
+                self.app = Tk()
+                self.app.withdraw()
+                self.func = func
+
+            def on_timer(self):
+                self.func()
+                self.app.after(poll_interval, self.on_timer)
+
+            def start(self):
+                self.on_timer()  # Call it once to get things going.
+                self.app.mainloop()
+
+        kernel.timer = Timer(kernel.do_one_iteration)
+        kernel.timer.start()
+
+Some event loops can go one better, and integrate checking for messages on the
+kernel's ZMQ sockets, making the kernel more responsive than plain polling. How
+to do this is outside the scope of this document; if you are interested, look at
+the integration with Qt in :mod:`IPython.kernel.zmq.eventloops`.

docs/source/whatsnew/pr/inputhook_extensible.rst

@@ -0,0 +1,7 b''
+* It's now possible to provide mechanisms to integrate IPython with other event
+  loops, in addition to the ones we already support. This lets you run GUI code
+  in IPython with an interactive prompt, and to embed the IPython
+  kernel in GUI applications. See :doc:`/config/eventloops` for details. As part
+  of this, the direct ``enable_*`` and ``disable_*`` functions for various GUIs
+  in :mod:`IPython.lib.inputhook` have been deprecated in favour of
+  :meth:`~.InputHookManager.enable_gui` and :meth:`~.InputHookManager.disable_gui`.

IPython/kernel/zmq/eventloops.py

@@ -51,6 +51,34 b' def _notify_stream_qt(kernel, stream):'
     notifier = QtCore.QSocketNotifier(fd, QtCore.QSocketNotifier.Read, kernel.app)
     notifier.activated.connect(process_stream_events)
 
+# mapping of keys to loop functions
+loop_map = {
+    'inline': None,
+    None : None,
+}
+
+def register_integration(*toolkitnames):
+    """Decorator to register an event loop to integrate with the IPython kernel
+    
+    The decorator takes names to register the event loop as for the %gui magic.
+    You can provide alternative names for the same toolkit.
+    
+    The decorated function should take a single argument, the IPython kernel
+    instance, arrange for the event loop to call ``kernel.do_one_iteration()``
+    at least every ``kernel._poll_interval`` seconds, and start the event loop.
+    
+    :mod:`IPython.kernel.zmq.eventloops` provides and registers such functions
+    for a few common event loops.
+    """
+    def decorator(func):
+        for name in toolkitnames:
+            loop_map[name] = func
+        return func
+    
+    return decorator
+
+
+@register_integration('qt', 'qt4')
 def loop_qt4(kernel):
     """Start a kernel with PyQt4 event loop integration."""
 
@@ -65,6 +93,7 b' def loop_qt4(kernel):'
     start_event_loop_qt4(kernel.app)
 
 
+@register_integration('wx')
 def loop_wx(kernel):
     """Start a kernel with wx event loop support."""
 
@@ -117,6 +146,7 b' def loop_wx(kernel):'
     start_event_loop_wx(kernel.app)
 
 
+@register_integration('tk')
 def loop_tk(kernel):
     """Start a kernel with the Tk event loop."""
 
@@ -146,6 +176,7 b' def loop_tk(kernel):'
     kernel.timer.start()
 
 
+@register_integration('gtk')
 def loop_gtk(kernel):
     """Start the kernel, coordinating with the GTK event loop"""
     from .gui.gtkembed import GTKEmbed
@@ -154,6 +185,7 b' def loop_gtk(kernel):'
     gtk_kernel.start()
 
 
+@register_integration('gtk3')
 def loop_gtk3(kernel):
     """Start the kernel, coordinating with the GTK event loop"""
     from .gui.gtk3embed import GTKEmbed
@@ -162,6 +194,7 b' def loop_gtk3(kernel):'
     gtk_kernel.start()
 
 
+@register_integration('osx')
 def loop_cocoa(kernel):
     """Start the kernel, coordinating with the Cocoa CFRunLoop event loop
     via the matplotlib MacOSX backend.
@@ -232,18 +265,6 b' def loop_cocoa(kernel):'
             # ensure excepthook is restored
             sys.excepthook = real_excepthook
 
-# mapping of keys to loop functions
-loop_map = {
-    'qt' : loop_qt4,
-    'qt4': loop_qt4,
-    'inline': None,
-    'osx': loop_cocoa,
-    'wx' : loop_wx,
-    'tk' : loop_tk,
-    'gtk': loop_gtk,
-    'gtk3': loop_gtk3,
-    None : None,
-}
 
 
 def enable_gui(gui, kernel=None):

IPython/lib/inputhook.py

@@ -110,7 +110,9 b' class InputHookManager(object):'
             warn("IPython GUI event loop requires ctypes, %gui will not be available")
             return
         self.PYFUNC = ctypes.PYFUNCTYPE(ctypes.c_int)
-        self._apps = {}
+        self.guihooks = {}
+        self.aliases = {}
+        self.apps = {}
         self._reset()
 
     def _reset(self):
@@ -177,11 +179,104 b' class InputHookManager(object):'
             as those toolkits don't have the notion of an app.
         """
         if gui is None:
-            self._apps = {}
-        elif gui in self._apps:
-            del self._apps[gui]
+            self.apps = {}
+        elif gui in self.apps:
+            del self.apps[gui]
 
-    def enable_wx(self, app=None):
+    def register(self, toolkitname, *aliases):
+        """Register a class to provide the event loop for a given GUI.
+        
+        This is intended to be used as a class decorator. It should be passed
+        the names with which to register this GUI integration. The classes
+        themselves should subclass :class:`InputHookBase`.
+        
+        ::
+        
+            @inputhook_manager.register('qt')
+            class QtInputHook(InputHookBase):
+                def enable(self, app=None):
+                    ...
+        """
+        def decorator(cls):
+            inst = cls(self)
+            self.guihooks[toolkitname] = inst
+            for a in aliases:
+                self.aliases[a] = toolkitname
+            return cls
+        return decorator
+
+    def current_gui(self):
+        """Return a string indicating the currently active GUI or None."""
+        return self._current_gui
+
+    def enable_gui(self, gui=None, app=None):
+        """Switch amongst GUI input hooks by name.
+
+        This is a higher level method than :meth:`set_inputhook` - it uses the
+        GUI name to look up a registered object which enables the input hook
+        for that GUI.
+
+        Parameters
+        ----------
+        gui : optional, string or None
+          If None (or 'none'), clears input hook, otherwise it must be one
+          of the recognized GUI names (see ``GUI_*`` constants in module).
+
+        app : optional, existing application object.
+          For toolkits that have the concept of a global app, you can supply an
+          existing one.  If not given, the toolkit will be probed for one, and if
+          none is found, a new one will be created.  Note that GTK does not have
+          this concept, and passing an app if ``gui=="GTK"`` will raise an error.
+
+        Returns
+        -------
+        The output of the underlying gui switch routine, typically the actual
+        PyOS_InputHook wrapper object or the GUI toolkit app created, if there was
+        one.
+        """
+        if gui in (None, GUI_NONE):
+            return self.disable_gui()
+        
+        if gui in self.aliases:
+            return self.enable_gui(self.aliases[gui], app)
+        
+        try:
+            gui_hook = self.guihooks[gui]
+        except KeyError:
+            e = "Invalid GUI request {!r}, valid ones are: {}"
+            raise ValueError(e.format(gui, ', '.join(self.guihooks)))
+        self._current_gui = gui
+        return gui_hook.enable(app)
+
+    def disable_gui(self):
+        """Disable GUI event loop integration.
+        
+        If an application was registered, this sets its ``_in_event_loop``
+        attribute to False. It then calls :meth:`clear_inputhook`.
+        """
+        gui = self._current_gui
+        if gui in self.apps:
+            self.apps[gui]._in_event_loop = False
+        return self.clear_inputhook()
+
+class InputHookBase(object):
+    """Base class for input hooks for specific toolkits.
+    
+    Subclasses should define an :meth:`enable` method with one argument, ``app``,
+    which will either be an instance of the toolkit's application class, or None.
+    They may also define a :meth:`disable` method with no arguments.
+    """
+    def __init__(self, manager):
+        self.manager = manager
+
+    def disable(self):
+        pass
+
+inputhook_manager = InputHookManager()
+
+@inputhook_manager.register('wx')
+class WxInputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with wxPython.
 
         Parameters
@@ -212,30 +307,29 b' class InputHookManager(object):'
         
         from IPython.lib.inputhookwx import inputhook_wx
         from IPython.external.appnope import nope
-        self.set_inputhook(inputhook_wx)
+        self.manager.set_inputhook(inputhook_wx)
         nope()
-        self._current_gui = GUI_WX
+
         import wx
         if app is None:
             app = wx.GetApp()
         if app is None:
             app = wx.App(redirect=False, clearSigInt=False)
         app._in_event_loop = True
-        self._apps[GUI_WX] = app
+        self.manager.apps[GUI_WX] = app
         return app
 
-    def disable_wx(self):
+    def disable(self):
         """Disable event loop integration with wxPython.
 
-        This merely sets PyOS_InputHook to NULL.
+        This restores appnapp on OS X
         """
         from IPython.external.appnope import nap
-        if GUI_WX in self._apps:
-            self._apps[GUI_WX]._in_event_loop = False
-        self.clear_inputhook()
         nap()
 
-    def enable_qt4(self, app=None):
+@inputhook_manager.register('qt', 'qt4')
+class Qt4InputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with PyQt4.
         
         Parameters
@@ -260,26 +354,24 b' class InputHookManager(object):'
         from IPython.lib.inputhookqt4 import create_inputhook_qt4
         from IPython.external.appnope import nope
         app, inputhook_qt4 = create_inputhook_qt4(self, app)
-        self.set_inputhook(inputhook_qt4)
+        self.manager.set_inputhook(inputhook_qt4)
         nope()
 
-        self._current_gui = GUI_QT4
         app._in_event_loop = True
-        self._apps[GUI_QT4] = app
+        self.manager.apps[GUI_QT4] = app
         return app
 
     def disable_qt4(self):
         """Disable event loop integration with PyQt4.
 
-        This merely sets PyOS_InputHook to NULL.
+        This restores appnapp on OS X
         """
         from IPython.external.appnope import nap
-        if GUI_QT4 in self._apps:
-            self._apps[GUI_QT4]._in_event_loop = False
-        self.clear_inputhook()
         nap()
 
-    def enable_gtk(self, app=None):
+@inputhook_manager.register('gtk')
+class GtkInputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with PyGTK.
 
         Parameters
@@ -298,21 +390,15 b' class InputHookManager(object):'
         import gtk
         try:
             gtk.set_interactive(True)
-            self._current_gui = GUI_GTK
         except AttributeError:
             # For older versions of gtk, use our own ctypes version
             from IPython.lib.inputhookgtk import inputhook_gtk
-            self.set_inputhook(inputhook_gtk)
-            self._current_gui = GUI_GTK
+            self.manager.set_inputhook(inputhook_gtk)
 
-    def disable_gtk(self):
-        """Disable event loop integration with PyGTK.
-        
-        This merely sets PyOS_InputHook to NULL.
-        """
-        self.clear_inputhook()
 
-    def enable_tk(self, app=None):
+@inputhook_manager.register('tk')
+class TkInputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with Tk.
 
         Parameters
@@ -328,7 +414,6 b' class InputHookManager(object):'
         :class:`InputHookManager`, since creating that object automatically
         sets ``PyOS_InputHook``.
         """
-        self._current_gui = GUI_TK
         if app is None:
             try:
                 from tkinter import Tk  # Py 3
@@ -336,19 +421,14 b' class InputHookManager(object):'
                 from Tkinter import Tk  # Py 2
             app = Tk()
             app.withdraw()
-            self._apps[GUI_TK] = app
+            self.manager.apps[GUI_TK] = app
             return app
 
-    def disable_tk(self):
-        """Disable event loop integration with Tkinter.
-        
-        This merely sets PyOS_InputHook to NULL.
-        """
-        self.clear_inputhook()
-
 
-    def enable_glut(self, app=None):
-        """ Enable event loop integration with GLUT.
+@inputhook_manager.register('glut')
+class GlutInputHook(InputHookBase):
+    def enable(self, app=None):
+        """Enable event loop integration with GLUT.
 
         Parameters
         ----------
@@ -377,7 +457,7 b' class InputHookManager(object):'
                                               glut_close, glut_display, \
                                               glut_idle, inputhook_glut
 
-        if GUI_GLUT not in self._apps:
+        if GUI_GLUT not in self.manager.apps:
             glut.glutInit( sys.argv )
             glut.glutInitDisplayMode( glut_display_mode )
             # This is specific to freeglut
@@ -394,12 +474,11 b' class InputHookManager(object):'
             glut.glutWMCloseFunc( glut_close )
             glut.glutDisplayFunc( glut_display )
             glut.glutIdleFunc( glut_idle)
-        self.set_inputhook( inputhook_glut )
-        self._current_gui = GUI_GLUT
-        self._apps[GUI_GLUT] = True
+        self.manager.set_inputhook( inputhook_glut )
+        self.manager.apps[GUI_GLUT] = True
 
 
-    def disable_glut(self):
+    def disable(self):
         """Disable event loop integration with glut.
         
         This sets PyOS_InputHook to NULL and set the display function to a
@@ -411,9 +490,11 b' class InputHookManager(object):'
 
         glut.glutHideWindow() # This is an event to be processed below
         glutMainLoopEvent()
-        self.clear_inputhook()
+        super(GlutInputHook, self).disable()
 
-    def enable_pyglet(self, app=None):
+@inputhook_manager.register('pyglet')
+class PygletInputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with pyglet.
 
         Parameters
@@ -431,18 +512,13 b' class InputHookManager(object):'
 
         """
         from IPython.lib.inputhookpyglet import inputhook_pyglet
-        self.set_inputhook(inputhook_pyglet)
-        self._current_gui = GUI_PYGLET
+        self.manager.set_inputhook(inputhook_pyglet)
         return app
 
-    def disable_pyglet(self):
-        """Disable event loop integration with pyglet.
 
-        This merely sets PyOS_InputHook to NULL.
-        """
-        self.clear_inputhook()
-
-    def enable_gtk3(self, app=None):
+@inputhook_manager.register('gtk3')
+class Gtk3InputHook(InputHookBase):
+    def enable(self, app=None):
         """Enable event loop integration with Gtk3 (gir bindings).
 
         Parameters
@@ -459,84 +535,35 b' class InputHookManager(object):'
         IPython.
         """
         from IPython.lib.inputhookgtk3 import inputhook_gtk3
-        self.set_inputhook(inputhook_gtk3)
-        self._current_gui = GUI_GTK
+        self.manager.set_inputhook(inputhook_gtk3)
+        self.manager._current_gui = GUI_GTK
 
-    def disable_gtk3(self):
-        """Disable event loop integration with PyGTK.
 
-        This merely sets PyOS_InputHook to NULL.
-        """
-        self.clear_inputhook()
-
-    def current_gui(self):
-        """Return a string indicating the currently active GUI or None."""
-        return self._current_gui
-
-inputhook_manager = InputHookManager()
-
-enable_wx = inputhook_manager.enable_wx
-disable_wx = inputhook_manager.disable_wx
-enable_qt4 = inputhook_manager.enable_qt4
-disable_qt4 = inputhook_manager.disable_qt4
-enable_gtk = inputhook_manager.enable_gtk
-disable_gtk = inputhook_manager.disable_gtk
-enable_tk = inputhook_manager.enable_tk
-disable_tk = inputhook_manager.disable_tk
-enable_glut = inputhook_manager.enable_glut
-disable_glut = inputhook_manager.disable_glut
-enable_pyglet = inputhook_manager.enable_pyglet
-disable_pyglet = inputhook_manager.disable_pyglet
-enable_gtk3 = inputhook_manager.enable_gtk3
-disable_gtk3 = inputhook_manager.disable_gtk3
 clear_inputhook = inputhook_manager.clear_inputhook
 set_inputhook = inputhook_manager.set_inputhook
 current_gui = inputhook_manager.current_gui
 clear_app_refs = inputhook_manager.clear_app_refs
-
-guis = {None: clear_inputhook,
-        GUI_NONE: clear_inputhook,
-        GUI_OSX: lambda app=False: None,
-        GUI_TK: enable_tk,
-        GUI_GTK: enable_gtk,
-        GUI_WX: enable_wx,
-        GUI_QT: enable_qt4, # qt3 not supported
-        GUI_QT4: enable_qt4,
-        GUI_GLUT: enable_glut,
-        GUI_PYGLET: enable_pyglet,
-        GUI_GTK3: enable_gtk3,
-}
-
-
-# Convenience function to switch amongst them
-def enable_gui(gui=None, app=None):
-    """Switch amongst GUI input hooks by name.
-
-    This is just a utility wrapper around the methods of the InputHookManager
-    object.
-
-    Parameters
-    ----------
-    gui : optional, string or None
-      If None (or 'none'), clears input hook, otherwise it must be one
-      of the recognized GUI names (see ``GUI_*`` constants in module).
-
-    app : optional, existing application object.
-      For toolkits that have the concept of a global app, you can supply an
-      existing one.  If not given, the toolkit will be probed for one, and if
-      none is found, a new one will be created.  Note that GTK does not have
-      this concept, and passing an app if ``gui=="GTK"`` will raise an error.
-
-    Returns
-    -------
-    The output of the underlying gui switch routine, typically the actual
-    PyOS_InputHook wrapper object or the GUI toolkit app created, if there was
-    one.
-    """
-    try:
-        gui_hook = guis[gui]
-    except KeyError:
-        e = "Invalid GUI request %r, valid ones are:%s" % (gui, guis.keys())
-        raise ValueError(e)
-    return gui_hook(app)
-
+enable_gui = inputhook_manager.enable_gui
+disable_gui = inputhook_manager.disable_gui
+register = inputhook_manager.register
+guis = inputhook_manager.guihooks
+
+# Deprecated methods: kept for backwards compatibility, do not use in new code
+def _make_deprecated_enable(name):
+    def enable_toolkit(app=None):
+        warn("This function is deprecated - use enable_gui(%r) instead" % name)
+        inputhook_manager.enable_gui(name, app)
+    
+enable_wx = _make_deprecated_enable('wx')
+enable_qt4 = _make_deprecated_enable('qt4')
+enable_gtk = _make_deprecated_enable('gtk')
+enable_tk = _make_deprecated_enable('tk')
+enable_glut = _make_deprecated_enable('glut')
+enable_pyglet = _make_deprecated_enable('pyglet')
+enable_gtk3 = _make_deprecated_enable('gtk3')
+
+def _deprecated_disable():
+    warn("This function is deprecated: use disable_gui() instead")
+    inputhook_manager.disable_gui()
+disable_wx = disable_qt4 = disable_gtk = disable_gtk3 = disable_glut = \
+        disable_pyglet = _deprecated_disable

docs/source/config/index.rst

@@ -30,3 +30,4 b' Extending and integrating with IPython'
    custommagics
    inputtransforms
    callbacks
+   eventloops

examples/IPython Kernel/gui/gui-gtk.py

@@ -33,7 +33,7 b' button.show()'
 window.show()
 
 try:
-    from IPython.lib.inputhook import enable_gtk
-    enable_gtk()
+    from IPython.lib.inputhook import enable_gui
+    enable_gui('gtk')
 except ImportError:
     gtk.main()

examples/IPython Kernel/gui/gui-gtk3.py

@@ -31,7 +31,7 b' button.show()'
 window.show()
 
 try:
-    from IPython.lib.inputhook import enable_gtk3
-    enable_gtk3()
+    from IPython.lib.inputhook import enable_gui
+    enable_gui('gtk3')
 except ImportError:
     Gtk.main()

examples/IPython Kernel/gui/gui-pyglet.py

@@ -27,7 +27,7 b' def on_draw():'
     label.draw()
 
 try:
-    from IPython.lib.inputhook import enable_pyglet
-    enable_pyglet()
+    from IPython.lib.inputhook import enable_gui
+    enable_gui('pyglet')
 except ImportError:
     pyglet.app.run()

examples/IPython Kernel/gui/gui-tk.py

@@ -30,6 +30,7 b' root = Tk()'
 app = MyApp(root)
 
 try:
-    from IPython.lib.inputhook import enable_tk; enable_tk(root)
+    from IPython.lib.inputhook import enable_gui
+    enable_gui('tk', root)
 except ImportError:
     root.mainloop()

examples/IPython Kernel/gui/gui-wx.py

@@ -100,7 +100,7 b" if __name__ == '__main__':"
         frame.Show(True)
 
     try:
-        from IPython.lib.inputhook import enable_wx
-        enable_wx(app)
+        from IPython.lib.inputhook import enable_gui
+        enable_gui('wx', app)
     except ImportError:
         app.MainLoop()