Show More
@@ -1,145 +1,152 | |||
|
1 | 1 | ========================== |
|
2 | 2 | Making kernels for IPython |
|
3 | 3 | ========================== |
|
4 | 4 | |
|
5 | 5 | A 'kernel' is a program that runs and introspects the user's code. IPython |
|
6 | 6 | includes a kernel for Python code, and people have written kernels for |
|
7 | 7 | `several other languages <https://github.com/ipython/ipython/wiki/Projects-using-IPython#list-of-some-ipython-compatible-kernels>`_. |
|
8 | 8 | |
|
9 | 9 | When IPython starts a kernel, it passes it a connection file. This specifies |
|
10 | 10 | how to set up communications with the frontend. |
|
11 | 11 | |
|
12 | 12 | There are two options for writing a kernel: |
|
13 | 13 | |
|
14 | 14 | 1. You can reuse the IPython kernel machinery to handle the communications, and |
|
15 | 15 | just describe how to execute your code. This is much simpler if the target |
|
16 | 16 | language can be driven from Python. See :doc:`wrapperkernels` for details. |
|
17 | 17 | 2. You can implement the kernel machinery in your target language. This is more |
|
18 | 18 | work initially, but the people using your kernel might be more likely to |
|
19 | 19 | contribute to it if it's in the language they know. |
|
20 | 20 | |
|
21 | 21 | Connection files |
|
22 | 22 | ================ |
|
23 | 23 | |
|
24 | 24 | Your kernel will be given the path to a connection file when it starts (see |
|
25 | 25 | :ref:`kernelspecs` for how to specify the command line arguments for your kernel). |
|
26 | 26 | This file, which is accessible only to the current user, will contain a JSON |
|
27 | 27 | dictionary looking something like this:: |
|
28 | 28 | |
|
29 | 29 | { |
|
30 | 30 | "control_port": 50160, |
|
31 | 31 | "shell_port": 57503, |
|
32 | 32 | "transport": "tcp", |
|
33 | 33 | "signature_scheme": "hmac-sha256", |
|
34 | 34 | "stdin_port": 52597, |
|
35 | 35 | "hb_port": 42540, |
|
36 | 36 | "ip": "127.0.0.1", |
|
37 | 37 | "iopub_port": 40885, |
|
38 | 38 | "key": "a0436f6c-1916-498b-8eb9-e81ab9368e84" |
|
39 | 39 | } |
|
40 | 40 | |
|
41 | 41 | The ``transport``, ``ip`` and five ``_port`` fields specify five ports which the |
|
42 | 42 | kernel should bind to using `ZeroMQ <http://zeromq.org/>`_. For instance, the |
|
43 | 43 | address of the shell socket in the example above would be:: |
|
44 | 44 | |
|
45 | 45 | tcp://127.0.0.1:57503 |
|
46 | 46 | |
|
47 | 47 | New ports are chosen at random for each kernel started. |
|
48 | 48 | |
|
49 | 49 | ``signature_scheme`` and ``key`` are used to cryptographically sign messages, so |
|
50 | 50 | that other users on the system can't send code to run in this kernel. See |
|
51 | 51 | :ref:`wire_protocol` for the details of how this signature is calculated. |
|
52 | 52 | |
|
53 | 53 | Handling messages |
|
54 | 54 | ================= |
|
55 | 55 | |
|
56 | 56 | After reading the connection file and binding to the necessary sockets, the |
|
57 | 57 | kernel should go into an event loop, listening on the hb (heartbeat), control |
|
58 | 58 | and shell sockets. |
|
59 | 59 | |
|
60 | 60 | :ref:`Heartbeat <kernel_heartbeat>` messages should be echoed back immediately |
|
61 | 61 | on the same socket - the frontend uses this to check that the kernel is still |
|
62 | 62 | alive. |
|
63 | 63 | |
|
64 | 64 | Messages on the control and shell sockets should be parsed, and their signature |
|
65 | 65 | validated. See :ref:`wire_protocol` for how to do this. |
|
66 | 66 | |
|
67 | 67 | The kernel will send messages on the iopub socket to display output, and on the |
|
68 | 68 | stdin socket to prompt the user for textual input. |
|
69 | 69 | |
|
70 | 70 | .. seealso:: |
|
71 | 71 | |
|
72 | 72 | :doc:`messaging` |
|
73 |
Details of the different sockets and the messages that come over them |
|
|
73 | Details of the different sockets and the messages that come over them | |
|
74 | ||
|
75 | `Creating Language Kernels for IPython <http://andrew.gibiansky.com/blog/ipython/ipython-kernels/>`_ | |
|
76 | A blog post by the author of `IHaskell <https://github.com/gibiansky/IHaskell>`_, | |
|
77 | a Haskell kernel | |
|
78 | ||
|
79 | `simple_kernel <https://github.com/dsblank/simple_kernel>`_ | |
|
80 | A simple example implementation of the kernel machinery in Python | |
|
74 | 81 | |
|
75 | 82 | |
|
76 | 83 | .. _kernelspecs: |
|
77 | 84 | |
|
78 | 85 | Kernel specs |
|
79 | 86 | ============ |
|
80 | 87 | |
|
81 | 88 | A kernel identifies itself to IPython by creating a directory, the name of which |
|
82 | 89 | is used as an identifier for the kernel. These may be created in a number of |
|
83 | 90 | locations: |
|
84 | 91 | |
|
85 | 92 | +--------+--------------------------------------+-----------------------------------+ |
|
86 | 93 | | | Unix | Windows | |
|
87 | 94 | +========+======================================+===================================+ |
|
88 | 95 | | System | ``/usr/share/ipython/kernels`` | ``%PROGRAMDATA%\ipython\kernels`` | |
|
89 | 96 | | | | | |
|
90 | 97 | | | ``/usr/local/share/ipython/kernels`` | | |
|
91 | 98 | +--------+--------------------------------------+-----------------------------------+ |
|
92 | 99 | | User | ``~/.ipython/kernels`` | |
|
93 | 100 | +--------+--------------------------------------+-----------------------------------+ |
|
94 | 101 | |
|
95 | 102 | The user location takes priority over the system locations, and the case of the |
|
96 | 103 | names is ignored, so selecting kernels works the same way whether or not the |
|
97 | 104 | filesystem is case sensitive. |
|
98 | 105 | |
|
99 | 106 | Inside the directory, the most important file is *kernel.json*. This should be a |
|
100 | 107 | JSON serialised dictionary containing the following keys and values: |
|
101 | 108 | |
|
102 | 109 | - **argv**: A list of command line arguments used to start the kernel. The text |
|
103 | 110 | ``{connection_file}`` in any argument will be replaced with the path to the |
|
104 | 111 | connection file. |
|
105 | 112 | - **display_name**: The kernel's name as it should be displayed in the UI. |
|
106 | 113 | Unlike the kernel name used in the API, this can contain arbitrary unicode |
|
107 | 114 | characters. |
|
108 | 115 | - **language**: The programming language which this kernel runs. This will be |
|
109 | 116 | stored in notebook metadata. This may be used by syntax highlighters to guess |
|
110 | 117 | how to parse code in a notebook, and frontends may eventually use it to |
|
111 | 118 | identify alternative kernels that can run some code. |
|
112 | 119 | - **codemirror_mode** (optional): The `codemirror mode <http://codemirror.net/mode/index.html>`_ |
|
113 | 120 | to use for code in this language. This can be a string or a dictionary, as |
|
114 | 121 | passed to codemirror config. The string from *language* will be used if this is |
|
115 | 122 | not provided. |
|
116 | 123 | - **env** (optional): A dictionary of environment variables to set for the kernel. |
|
117 | 124 | These will be added to the current environment variables before the kernel is |
|
118 | 125 | started. |
|
119 | 126 | - **help_links** (optional): A list of dictionaries, each with keys 'text' and |
|
120 | 127 | 'url'. These will be displayed in the help menu in the notebook UI. |
|
121 | 128 | |
|
122 | 129 | For example, the kernel.json file for IPython looks like this:: |
|
123 | 130 | |
|
124 | 131 | { |
|
125 | 132 | "argv": ["python3", "-c", "from IPython.kernel.zmq.kernelapp import main; main()", |
|
126 | 133 | "-f", "{connection_file}"], |
|
127 | 134 | "codemirror_mode": { |
|
128 | 135 | "version": 3, |
|
129 | 136 | "name": "ipython" |
|
130 | 137 | }, |
|
131 | 138 | "display_name": "IPython (Python 3)", |
|
132 | 139 | "language": "python" |
|
133 | 140 | } |
|
134 | 141 | |
|
135 | 142 | To see the available kernel specs, run:: |
|
136 | 143 | |
|
137 | 144 | ipython kernelspec list |
|
138 | 145 | |
|
139 | 146 | To start the terminal console or the Qt console with a specific kernel:: |
|
140 | 147 | |
|
141 | 148 | ipython console --kernel bash |
|
142 | 149 | ipython qtconsole --kernel bash |
|
143 | 150 | |
|
144 | 151 | To use different kernels in the notebook, select a different kernel from the |
|
145 | 152 | dropdown menu in the top-right of the UI. |
@@ -1,148 +1,153 | |||
|
1 | 1 | Making simple Python wrapper kernels |
|
2 | 2 | ==================================== |
|
3 | 3 | |
|
4 | 4 | .. versionadded:: 3.0 |
|
5 | 5 | |
|
6 | 6 | You can now re-use the kernel machinery in IPython to easily make new kernels. |
|
7 | 7 | This is useful for languages that have Python bindings, such as `Octave |
|
8 | 8 | <http://www.gnu.org/software/octave/>`_ (via |
|
9 | 9 | `Oct2Py <http://blink1073.github.io/oct2py/docs/index.html>`_), or languages |
|
10 | 10 | where the REPL can be controlled in a tty using `pexpect <http://pexpect.readthedocs.org/en/latest/>`_, |
|
11 | 11 | such as bash. |
|
12 | 12 | |
|
13 | .. seealso:: | |
|
14 | ||
|
15 | `bash_kernel <https://github.com/takluyver/bash_kernel>`_ | |
|
16 | A simple kernel for bash, written using this machinery | |
|
17 | ||
|
13 | 18 | Required steps |
|
14 | 19 | -------------- |
|
15 | 20 | |
|
16 | 21 | Subclass :class:`IPython.kernel.zmq.kernelbase.Kernel`, and implement the |
|
17 | 22 | following methods and attributes: |
|
18 | 23 | |
|
19 | 24 | .. class:: MyKernel |
|
20 | 25 | |
|
21 | 26 | .. attribute:: implementation |
|
22 | 27 | implementation_version |
|
23 | 28 | language |
|
24 | 29 | language_version |
|
25 | 30 | banner |
|
26 | 31 | |
|
27 | 32 | Information for :ref:`msging_kernel_info` replies. 'Implementation' refers |
|
28 | 33 | to the kernel (e.g. IPython), and 'language' refers to the language it |
|
29 | 34 | interprets (e.g. Python). The 'banner' is displayed to the user in console |
|
30 | 35 | UIs before the first prompt. All of these values are strings. |
|
31 | 36 | |
|
32 | 37 | .. method:: do_execute(code, silent, store_history=True, user_expressions=None, allow_stdin=False) |
|
33 | 38 | |
|
34 | 39 | Execute user code. |
|
35 | 40 | |
|
36 | 41 | :param str code: The code to be executed. |
|
37 | 42 | :param bool silent: Whether to display output. |
|
38 | 43 | :param bool store_history: Whether to record this code in history and |
|
39 | 44 | increase the execution count. If silent is True, this is implicitly |
|
40 | 45 | False. |
|
41 | 46 | :param dict user_expressions: Mapping of names to expressions to evaluate |
|
42 | 47 | after the code has run. You can ignore this if you need to. |
|
43 | 48 | :param bool allow_stdin: Whether the frontend can provide input on request |
|
44 | 49 | (e.g. for Python's :func:`raw_input`). |
|
45 | 50 | |
|
46 | 51 | Your method should return a dict containing the fields described in |
|
47 | 52 | :ref:`execution_results`. To display output, it can send messages |
|
48 | 53 | using :meth:`~IPython.kernel.zmq.kernelbase.Kernel.send_response`. |
|
49 | 54 | See :doc:`messaging` for details of the different message types. |
|
50 | 55 | |
|
51 | 56 | To launch your kernel, add this at the end of your module:: |
|
52 | 57 | |
|
53 | 58 | if __name__ == '__main__': |
|
54 | 59 | from IPython.kernel.zmq.kernelapp import IPKernelApp |
|
55 | 60 | IPKernelApp.launch_instance(kernel_class=MyKernel) |
|
56 | 61 | |
|
57 | 62 | Example |
|
58 | 63 | ------- |
|
59 | 64 | |
|
60 | 65 | ``echokernel.py`` will simply echo any input it's given to stdout:: |
|
61 | 66 | |
|
62 | 67 | from IPython.kernel.zmq.kernelbase import Kernel |
|
63 | 68 | |
|
64 | 69 | class EchoKernel(Kernel): |
|
65 | 70 | implementation = 'Echo' |
|
66 | 71 | implementation_version = '1.0' |
|
67 | 72 | language = 'no-op' |
|
68 | 73 | language_version = '0.1' |
|
69 | 74 | banner = "Echo kernel - as useful as a parrot" |
|
70 | 75 | |
|
71 | 76 | def do_execute(self, code, silent, store_history=True, user_experssions=None, |
|
72 | 77 | allow_stdin=False): |
|
73 | 78 | if not silent: |
|
74 | 79 | stream_content = {'name': 'stdout', 'data':code} |
|
75 | 80 | self.send_response(self.iopub_socket, 'stream', stream_content) |
|
76 | 81 | |
|
77 | 82 | return {'status': 'ok', |
|
78 | 83 | # The base class increments the execution count |
|
79 | 84 | 'execution_count': self.execution_count, |
|
80 | 85 | 'payload': [], |
|
81 | 86 | 'user_expressions': {}, |
|
82 | 87 | } |
|
83 | 88 | |
|
84 | 89 | if __name__ == '__main__': |
|
85 | 90 | from IPython.kernel.zmq.kernelapp import IPKernelApp |
|
86 | 91 | IPKernelApp.launch_instance(kernel_class=EchoKernel) |
|
87 | 92 | |
|
88 | 93 | Here's the Kernel spec ``kernel.json`` file for this:: |
|
89 | 94 | |
|
90 | 95 | {"argv":["python","-m","echokernel", "-f", "{connection_file}"], |
|
91 | 96 | "display_name":"Echo", |
|
92 | 97 | "language":"no-op" |
|
93 | 98 | } |
|
94 | 99 | |
|
95 | 100 | |
|
96 | 101 | Optional steps |
|
97 | 102 | -------------- |
|
98 | 103 | |
|
99 | 104 | You can override a number of other methods to improve the functionality of your |
|
100 | 105 | kernel. All of these methods should return a dictionary as described in the |
|
101 | 106 | relevant section of the :doc:`messaging spec <messaging>`. |
|
102 | 107 | |
|
103 | 108 | .. class:: MyKernel |
|
104 | 109 | |
|
105 | 110 | .. method:: do_complete(code, cusor_pos) |
|
106 | 111 | |
|
107 | 112 | Code completion |
|
108 | 113 | |
|
109 | 114 | :param str code: The code already present |
|
110 | 115 | :param int cursor_pos: The position in the code where completion is requested |
|
111 | 116 | |
|
112 | 117 | .. seealso:: |
|
113 | 118 | |
|
114 | 119 | :ref:`msging_completion` messages |
|
115 | 120 | |
|
116 | 121 | .. method:: do_inspect(code, cusor_pos, detail_level=0) |
|
117 | 122 | |
|
118 | 123 | Object introspection |
|
119 | 124 | |
|
120 | 125 | :param str code: The code |
|
121 | 126 | :param int cursor_pos: The position in the code where introspection is requested |
|
122 | 127 | :param int detail_level: 0 or 1 for more or less detail. In IPython, 1 gets |
|
123 | 128 | the source code. |
|
124 | 129 | |
|
125 | 130 | .. seealso:: |
|
126 | 131 | |
|
127 | 132 | :ref:`msging_inspection` messages |
|
128 | 133 | |
|
129 | 134 | .. method:: do_history(hist_access_type, output, raw, session=None, start=None, stop=None, n=None, pattern=None, unique=False) |
|
130 | 135 | |
|
131 | 136 | History access. Only the relevant parameters for the type of history |
|
132 | 137 | request concerned will be passed, so your method definition must have defaults |
|
133 | 138 | for all the arguments shown with defaults here. |
|
134 | 139 | |
|
135 | 140 | .. seealso:: |
|
136 | 141 | |
|
137 | 142 | :ref:`msging_history` messages |
|
138 | 143 | |
|
139 | 144 | .. method:: do_shutdown(restart) |
|
140 | 145 | |
|
141 | 146 | Shutdown the kernel. You only need to handle your own clean up - the kernel |
|
142 | 147 | machinery will take care of cleaning up its own things before stopping. |
|
143 | 148 | |
|
144 | 149 | :param bool restart: Whether the kernel will be started again afterwards |
|
145 | 150 | |
|
146 | 151 | .. seealso:: |
|
147 | 152 | |
|
148 | 153 | :ref:`msging_shutdown` messages |
General Comments 0
You need to be logged in to leave comments.
Login now