.. _qtconsole: ========================= A Qt Console for IPython ========================= We now have a version of IPython, using the new two-process :ref:`ZeroMQ Kernel `, running in a PyQt_ GUI. This is a very lightweight widget that largely feels like a terminal, but provides a number of enhancements only possible in a GUI, such as inline figures, proper multiline editing with syntax highlighting, graphical calltips, and much more. .. figure:: ../_static/qtconsole.png :width: 400px :alt: IPython Qt console with embedded plots :align: center :target: ../_static/qtconsole.png The Qt console for IPython, using inline matplotlib plots. To get acquainted with the Qt console, type `%guiref` to see a quick introduction of its main features. The Qt frontend has hand-coded emacs-style bindings for text navigation. This is not yet configurable. .. tip:: Since the Qt console tries hard to behave like a terminal, by default it immediately executes single lines of input that are complete. If you want to force multiline input, hit :kbd:`Ctrl-Enter` at the end of the first line instead of :kbd:`Enter`, and it will open a new line for input. At any point in a multiline block, you can force its execution (without having to go to the bottom) with :kbd:`Shift-Enter`. ``%loadpy`` =========== The new ``%loadpy`` magic takes any python script (must end in '.py'), and pastes its contents as your next input, so you can edit it before executing. The script may be on your machine, but you can also specify a url, and it will download the script from the web. This is particularly useful for playing with examples from documentation, such as matplotlib. .. sourcecode:: ipython In [6]: %loadpy http://matplotlib.sourceforge.net/plot_directive/mpl_examples/mplot3d/contour3d_demo.py In [7]: from mpl_toolkits.mplot3d import axes3d ...: import matplotlib.pyplot as plt ...: ...: fig = plt.figure() ...: ax = fig.add_subplot(111, projection='3d') ...: X, Y, Z = axes3d.get_test_data(0.05) ...: cset = ax.contour(X, Y, Z) ...: ax.clabel(cset, fontsize=9, inline=1) ...: ...: plt.show() Pylab ===== One of the most exciting features of the new console is embedded matplotlib figures. You can use any standard matplotlib GUI backend (Except native MacOSX) to draw the figures, and since there is now a two-process model, there is no longer a conflict between user input and the drawing eventloop. .. image:: figs/besselj.png :width: 519px .. display: :func:`display` *************** An additional function, :func:`display`, will be added to the global namespace if you specify the ``--pylab`` option at the command line. The IPython display system provides a mechanism for specifying PNG or SVG (and more) representations of objects for GUI frontends. By default, IPython registers convenient PNG and SVG renderers for matplotlib figures, so you can embed them in your document by calling :func:`display` on one or more of them. This is especially useful for saving_ your work. .. sourcecode:: ipython In [5]: plot(range(5)) # plots in the matplotlib window In [6]: display(gcf()) # embeds the current figure in the qtconsole In [7]: display(*getfigs()) # embeds all active figures in the qtconsole If you have a reference to a matplotlib figure object, you can always display that specific figure: .. sourcecode:: ipython In [1]: f = figure() In [2]: plot(rand(100)) Out[2]: [] In [3]: display(f) # Plot is shown here In [4]: title('A title') Out[4]: In [5]: display(f) # Updated plot with title is shown here. .. _inline: ``--pylab=inline`` ****************** If you want to have all of your figures embedded in your session, instead of calling :func:`display`, you can specify ``--pylab=inline`` when you start the console, and each time you make a plot, it will show up in your document, as if you had called :func:`display(fig)`. .. _saving: Saving and Printing =================== IPythonQt has the ability to save your current session, as either HTML or XHTML. If you have been using :func:`display` or inline_ pylab, your figures will be PNG in HTML, or inlined as SVG in XHTML. PNG images have the option to be either in an external folder, as in many browsers' "Webpage, Complete" option, or inlined as well, for a larger, but more portable file. The widget also exposes the ability to print directly, via the default print shortcut or context menu. .. Note:: Saving is only available to richtext Qt widgets, which are used by default, but if you pass the ``--plain`` flag, saving will not be available to you. See these examples of :download:`png/html` and :download:`svg/xhtml ` output. Note that syntax highlighting does not survive export. This is a known issue, and is being investigated. Colors and Highlighting ======================= Terminal IPython has always had some coloring, but never syntax highlighting. There are a few simple color choices, specified by the ``colors`` flag or ``%colors`` magic: * LightBG for light backgrounds * Linux for dark backgrounds * NoColor for a simple colorless terminal The Qt widget has full support for the ``colors`` flag used in the terminal shell. The Qt widget, however, has full syntax highlighting as you type, handled by the `pygments`_ library. The ``style`` argument exposes access to any style by name that can be found by pygments, and there are several already installed. The ``colors`` argument, if unspecified, will be guessed based on the chosen style. Similarly, there are default styles associated with each ``colors`` option. Screenshot of ``ipython qtconsole --colors=linux``, which uses the 'monokai' theme by default: .. image:: figs/colors_dark.png :width: 627px .. Note:: Calling ``ipython qtconsole -h`` will show all the style names that pygments can find on your system. You can also pass the filename of a custom CSS stylesheet, if you want to do your own coloring, via the ``stylesheet`` argument. The default LightBG stylesheet: .. sourcecode:: css QPlainTextEdit, QTextEdit { background-color: white; color: black ; selection-background-color: #ccc} .error { color: red; } .in-prompt { color: navy; } .in-prompt-number { font-weight: bold; } .out-prompt { color: darkred; } .out-prompt-number { font-weight: bold; } Fonts ===== The QtConsole has configurable via the ConsoleWidget. To change these, set the ``font_family`` or ``font_size`` traits of the ConsoleWidget. For instance, to use 9pt Anonymous Pro:: $> ipython qtconsole --ConsoleWidget.font_family="Anonymous Pro" --ConsoleWidget.font_size=9 Process Management ================== With the two-process ZMQ model, the frontend does not block input during execution. This means that actions can be taken by the frontend while the Kernel is executing, or even after it crashes. The most basic such command is via 'Ctrl-.', which restarts the kernel. This can be done in the middle of a blocking execution. The frontend can also know, via a heartbeat mechanism, that the kernel has died. This means that the frontend can safely restart the kernel. .. _multiple_consoles: Multiple Consoles ***************** Since the Kernel listens on the network, multiple frontends can connect to it. These do not have to all be qt frontends - any IPython frontend can connect and run code. When you start ipython qtconsole, there will be an output line, like:: [IPKernelApp] To connect another client to this kernel, use: [IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797 Other frontends can connect to your kernel, and share in the execution. This is great for collaboration. The ``--existing`` flag means connect to a kernel that already exists. Starting other consoles with that flag will not try to start their own, but rather connect to yours. Ultimately, you will not have to specify each port individually, but for now this copy-paste method is best. You can even launch a standalone kernel, and connect and disconnect Qt Consoles from various machines. This lets you keep the same running IPython session on your work machine (with matplotlib plots and everything), logging in from home, cafés, etc.:: $> ipython kernel [IPKernelApp] To connect another client to this kernel, use: [IPKernelApp] --existing --shell=60690 --iopub=44045 --stdin=38323 --hb=41797 This is actually exactly the same as the subprocess launched by the qtconsole, so all the information about connecting to a standalone kernel is identical to that of connecting to the kernel attached to a running console. .. _kernel_security: Security -------- .. warning:: Since the ZMQ code currently has no security, listening on an external-facing IP is dangerous. You are giving any computer that can see you on the network the ability to issue arbitrary shell commands as you on your machine. Read the rest of this section before listening on external ports or running an IPython kernel on a shared machine. By default (for security reasons), the kernel only listens on localhost, so you can only connect multiple frontends to the kernel from your local machine. You can specify to listen on an external interface by specifying the ``ip`` argument:: $> ipython qtconsole --ip=192.168.1.123 If you specify the ip as 0.0.0.0, that refers to all interfaces, so any computer that can see yours on the network can connect to the kernel. Messages are not encrypted, so users with access to the ports your kernel is using will be able to see any output of the kernel. They will also be able to issue shell commands as you, unless you enable HMAC digests, which are **DISABLED** by default. The one security feature IPython does provide is protection from unauthorized execution. IPython's messaging system can sign messages with HMAC digests using a shared-key. The key is never sent over the network, it is only used to generate a unique hash for each message, based on its content. When IPython receives a message, it will check that the digest matches. You can use any file that only you have access to to generate this key. One logical choice would be to use your own SSH private key. Or you can generate a new random private key with:: # generate 1024b of random data, and store in a file only you can read: # (assumes IPYTHON_DIR is defined, otherwise use your IPython directory) $> python -c "import os; print os.urandom(128).encode('base64')" > $IPYTHON_DIR/sessionkey $> chmod 600 $IPYTHON_DIR/sessionkey To enable HMAC digests, simply specify the ``Session.keyfile`` configurable in :file:`ipython_config.py` or at the command-line, as in:: # instruct IPython to sign messages with that key: $> ipython qtconsole --Session.keyfile=$IPYTHON_DIR/sessionkey You must use the same key you used to start the kernel with all frontends, or they will be treated as an unauthorized peer (all messages will be ignored). .. note:: IPython will move to using files to store connection information, as is done in :mod:`IPython.parallel`, at which point HMAC signatures will be enabled *by default*. .. _ssh_tunnels: SSH Tunnels ----------- Sometimes you want to connect to machines across the internet, or just across a LAN that either doesn't permit open ports or you don't trust the other machines on the network. To do this, you can use SSH tunnels. SSH tunnels are a way to securely forward ports on your local machine to ports on another machine, to which you have SSH access. In simple cases, IPython's tools can forward ports over ssh by simply adding the ``--ssh=remote`` argument to the usual ``--existing...`` set of flags for connecting to a running kernel. .. warning:: Using SSH tunnels does *not* increase localhost security. In fact, when tunneling from one machine to another *both* machines have open ports on localhost available for connections. There are two primary models for using SSH tunnels with IPython. The first is to have the Kernel listen only on localhost, and connect to it from another machine on the same LAN. First, let's start a kernel on machine **worker**, listening only on loopback:: user@worker $> ipython kernel [IPKernelApp] To connect another client to this kernel, use: [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511 In this case, the IP that you would connect to would still be 127.0.0.1, but you want to specify the additional ``--ssh`` argument with the hostname of the kernel (in this example, it's 'worker'):: user@client $> ipython qtconsole --ssh=worker --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511 Note again that this opens ports on the *client* machine that point to your kernel. Be sure to use a Session key, as described above, if localhost on *either* the client or kernel machines is untrusted. .. note:: the ssh argument is simply passed to openssh, so it can be fully specified ``user@host:port`` but it will also respect your aliases, etc. in :file:`.ssh/config` if you have any. The second pattern is for connecting to a machine behind a firewall across the internet (or otherwise wide network). This time, we have a machine **login** that you have ssh access to, which can see **kernel**, but **client** is on another network. The important difference now is that **client** can see **login**, but *not* **worker**. So we need to forward ports from client to worker *via* login. This means that the kernel must be started listening on external interfaces, so that its ports are visible to `login`:: user@worker $> ipython kernel --ip=0.0.0.0 [IPKernelApp] To connect another client to this kernel, use: [IPKernelApp] --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511 Which we can connect to from the client with:: user@client $> ipython qtconsole --ssh=login --ip=192.168.1.123 --existing --shell=59480 --iopub=62199 --stdin=64898 --hb=56511 Note that now the IP is the address of worker as seen from login. Manual SSH tunnels ------------------ It's possible that IPython's ssh helper functions won't work for you, for various reasons. You can still connect to remote machines, as long as you set up the tunnels yourself. The basic format of forwarding a local port to a remote one is:: [client] $> ssh :: -f -N This will forward local connections to **localport** on client to **remoteip:remoteport** *via* **server**. Note that remoteip is interpreted relative to *server*, not the client. So if you have direct ssh access to the machine to which you want to forward connections, then the server *is* the remote machine, and remoteip should be server's IP as seen from the server itself, i.e. 127.0.0.1. Thus, to forward local port 12345 to remote port 54321 on a machine you can see, do:: [client] $> ssh machine 12345:127.0.0.1:54321 -f -N But if your target is actually on a LAN at 192.168.1.123, behind another machine called **login**, then you would do:: [client] $> ssh login 12345:192.168.1.16:54321 -f -N The ``-f -N`` on the end are flags that tell ssh to run in the background, and don't actually run any commands beyond creating the tunnel. .. seealso:: A short discussion of ssh tunnels: http://www.revsys.com/writings/quicktips/ssh-tunnel.html Stopping Kernels and Consoles ***************************** Since there can be many consoles per kernel, the shutdown mechanism and dialog are probably more complicated than you are used to. Since you don't always want to shutdown a kernel when you close a window, you are given the option to just close the console window or also close the Kernel and *all other windows*. Note that this only refers to all other *local* windows, as remote Consoles are not allowed to shutdown the kernel, and shutdowns do not close Remote consoles (to allow for saving, etc.). Rules: * Restarting the kernel automatically clears all *local* Consoles, and prompts remote Consoles about the reset. * Shutdown closes all *local* Consoles, and notifies remotes that the Kernel has been shutdown. * Remote Consoles may not restart or shutdown the kernel. Qt and the QtConsole ==================== An important part of working with the QtConsole when you are writing your own Qt code is to remember that user code (in the kernel) is *not* in the same process as the frontend. This means that there is not necessarily any Qt code running in the kernel, and under most normal circumstances there isn't. If, however, you specify ``--pylab=qt`` at the command-line, then there *will* be a :class:`QCoreApplication` instance running in the kernel process along with user-code. To get a reference to this application, do: .. sourcecode:: python from PyQt4 import QtCore app = QtCore.QCoreApplication.instance() # app will be None if there is no such instance A common problem listed in the PyQt4 Gotchas_ is the fact that Python's garbage collection will destroy Qt objects (Windows, etc.) once there is no longer a Python reference to them, so you have to hold on to them. For instance, in: .. sourcecode:: python def make_window(): win = QtGui.QMainWindow() def make_and_return_window(): win = QtGui.QMainWindow() return win :func:`make_window` will never draw a window, because garbage collection will destroy it before it is drawn, whereas :func:`make_and_return_window` lets the caller decide when the window object should be destroyed. If, as a developer, you know that you always want your objects to last as long as the process, you can attach them to the QApplication instance itself: .. sourcecode:: python # do this just once: app = QtCore.QCoreApplication.instance() app.references = set() # then when you create Windows, add them to the set def make_window(): win = QtGui.QMainWindow() app.references.add(win) Now the QApplication itself holds a reference to ``win``, so it will never be garbage collected until the application itself is destroyed. .. _Gotchas: http://www.riverbankcomputing.co.uk/static/Docs/PyQt4/html/gotchas.html#garbage-collection Regressions =========== There are some features, where the qt console lags behind the Terminal frontend: * !cmd input: Due to our use of pexpect, we cannot pass input to subprocesses launched using the '!' escape, so you should never call a command that requires interactive input. For such cases, use the terminal IPython. This will not be fixed, as abandoning pexpect would significantly degrade the console experience. * Use of ``\b`` and ``\r`` characters in the console: these are control characters that allow the cursor to move backwards on a line, and are used to display things like in-place progress bars in a terminal. We currently do not support this, but it is being tracked as issue 629_. .. _629: https://github.com/ipython/ipython/issues/629 .. [PyQt] PyQt4 http://www.riverbankcomputing.co.uk/software/pyqt/download .. [pygments] Pygments http://pygments.org/