Show More
@@ -1,95 +1,99 b'' | |||||
1 | # -*- coding: utf-8 -*- |
|
1 | # -*- coding: utf-8 -*- | |
2 | """Release data for the IPython project. |
|
2 | """Release data for the IPython project. | |
3 |
|
3 | |||
4 | $Id: Release.py 3002 2008-02-01 07:17:00Z fperez $""" |
|
4 | $Id: Release.py 3002 2008-02-01 07:17:00Z fperez $""" | |
5 |
|
5 | |||
6 | #***************************************************************************** |
|
6 | #***************************************************************************** | |
7 | # Copyright (C) 2001-2006 Fernando Perez <fperez@colorado.edu> |
|
7 | # Copyright (C) 2001-2006 Fernando Perez <fperez@colorado.edu> | |
8 | # |
|
8 | # | |
9 | # Copyright (c) 2001 Janko Hauser <jhauser@zscout.de> and Nathaniel Gray |
|
9 | # Copyright (c) 2001 Janko Hauser <jhauser@zscout.de> and Nathaniel Gray | |
10 | # <n8gray@caltech.edu> |
|
10 | # <n8gray@caltech.edu> | |
11 | # |
|
11 | # | |
12 | # Distributed under the terms of the BSD License. The full license is in |
|
12 | # Distributed under the terms of the BSD License. The full license is in | |
13 | # the file COPYING, distributed as part of this software. |
|
13 | # the file COPYING, distributed as part of this software. | |
14 | #***************************************************************************** |
|
14 | #***************************************************************************** | |
15 |
|
15 | |||
16 | # Name of the package for release purposes. This is the name which labels |
|
16 | # Name of the package for release purposes. This is the name which labels | |
17 | # the tarballs and RPMs made by distutils, so it's best to lowercase it. |
|
17 | # the tarballs and RPMs made by distutils, so it's best to lowercase it. | |
18 | name = 'ipython' |
|
18 | name = 'ipython' | |
19 |
|
19 | |||
20 | # For versions with substrings (like 0.6.16.svn), use an extra . to separate |
|
20 | # For versions with substrings (like 0.6.16.svn), use an extra . to separate | |
21 | # the new substring. We have to avoid using either dashes or underscores, |
|
21 | # the new substring. We have to avoid using either dashes or underscores, | |
22 | # because bdist_rpm does not accept dashes (an RPM) convention, and |
|
22 | # because bdist_rpm does not accept dashes (an RPM) convention, and | |
23 | # bdist_deb does not accept underscores (a Debian convention). |
|
23 | # bdist_deb does not accept underscores (a Debian convention). | |
24 |
|
24 | |||
25 | revision = '1016' |
|
25 | development = True # change this to False to do a release | |
|
26 | version_base = '0.9.0' | |||
26 | branch = 'ipython' |
|
27 | branch = 'ipython' | |
|
28 | revision = '1016' | |||
27 |
|
29 | |||
28 | if branch == 'ipython': |
|
30 | if development: | |
29 | version = '0.9.0.bzr.r' + revision |
|
31 | if branch == 'ipython': | |
|
32 | version = '%s.bzr.r%s' % (version_base, revision) | |||
|
33 | else: | |||
|
34 | version = '%s.bzr.r%s.%s' % (version_base, revision, branch) | |||
30 | else: |
|
35 | else: | |
31 | version = '0.9.0.bzr.r%s.%s' % (revision,branch) |
|
36 | version = version_base | |
32 |
|
37 | |||
33 | # version = '0.8.4' |
|
|||
34 |
|
38 | |||
35 | description = "Tools for interactive development in Python." |
|
39 | description = "Tools for interactive development in Python." | |
36 |
|
40 | |||
37 | long_description = \ |
|
41 | long_description = \ | |
38 | """ |
|
42 | """ | |
39 | IPython provides a replacement for the interactive Python interpreter with |
|
43 | IPython provides a replacement for the interactive Python interpreter with | |
40 | extra functionality. |
|
44 | extra functionality. | |
41 |
|
45 | |||
42 | Main features: |
|
46 | Main features: | |
43 |
|
47 | |||
44 | * Comprehensive object introspection. |
|
48 | * Comprehensive object introspection. | |
45 |
|
49 | |||
46 | * Input history, persistent across sessions. |
|
50 | * Input history, persistent across sessions. | |
47 |
|
51 | |||
48 | * Caching of output results during a session with automatically generated |
|
52 | * Caching of output results during a session with automatically generated | |
49 | references. |
|
53 | references. | |
50 |
|
54 | |||
51 | * Readline based name completion. |
|
55 | * Readline based name completion. | |
52 |
|
56 | |||
53 | * Extensible system of 'magic' commands for controlling the environment and |
|
57 | * Extensible system of 'magic' commands for controlling the environment and | |
54 | performing many tasks related either to IPython or the operating system. |
|
58 | performing many tasks related either to IPython or the operating system. | |
55 |
|
59 | |||
56 | * Configuration system with easy switching between different setups (simpler |
|
60 | * Configuration system with easy switching between different setups (simpler | |
57 | than changing $PYTHONSTARTUP environment variables every time). |
|
61 | than changing $PYTHONSTARTUP environment variables every time). | |
58 |
|
62 | |||
59 | * Session logging and reloading. |
|
63 | * Session logging and reloading. | |
60 |
|
64 | |||
61 | * Extensible syntax processing for special purpose situations. |
|
65 | * Extensible syntax processing for special purpose situations. | |
62 |
|
66 | |||
63 | * Access to the system shell with user-extensible alias system. |
|
67 | * Access to the system shell with user-extensible alias system. | |
64 |
|
68 | |||
65 | * Easily embeddable in other Python programs. |
|
69 | * Easily embeddable in other Python programs. | |
66 |
|
70 | |||
67 | * Integrated access to the pdb debugger and the Python profiler. |
|
71 | * Integrated access to the pdb debugger and the Python profiler. | |
68 |
|
72 | |||
69 | The latest development version is always available at the IPython subversion |
|
73 | The latest development version is always available at the IPython subversion | |
70 | repository_. |
|
74 | repository_. | |
71 |
|
75 | |||
72 | .. _repository: http://ipython.scipy.org/svn/ipython/ipython/trunk#egg=ipython-dev |
|
76 | .. _repository: http://ipython.scipy.org/svn/ipython/ipython/trunk#egg=ipython-dev | |
73 | """ |
|
77 | """ | |
74 |
|
78 | |||
75 | license = 'BSD' |
|
79 | license = 'BSD' | |
76 |
|
80 | |||
77 | authors = {'Fernando' : ('Fernando Perez','fperez@colorado.edu'), |
|
81 | authors = {'Fernando' : ('Fernando Perez','fperez@colorado.edu'), | |
78 | 'Janko' : ('Janko Hauser','jhauser@zscout.de'), |
|
82 | 'Janko' : ('Janko Hauser','jhauser@zscout.de'), | |
79 | 'Nathan' : ('Nathaniel Gray','n8gray@caltech.edu'), |
|
83 | 'Nathan' : ('Nathaniel Gray','n8gray@caltech.edu'), | |
80 | 'Ville' : ('Ville Vainio','vivainio@gmail.com'), |
|
84 | 'Ville' : ('Ville Vainio','vivainio@gmail.com'), | |
81 | 'Brian' : ('Brian E Granger', 'ellisonbg@gmail.com'), |
|
85 | 'Brian' : ('Brian E Granger', 'ellisonbg@gmail.com'), | |
82 | 'Min' : ('Min Ragan-Kelley', 'benjaminrk@gmail.com') |
|
86 | 'Min' : ('Min Ragan-Kelley', 'benjaminrk@gmail.com') | |
83 | } |
|
87 | } | |
84 |
|
88 | |||
85 | author = 'The IPython Development Team' |
|
89 | author = 'The IPython Development Team' | |
86 |
|
90 | |||
87 | author_email = 'ipython-dev@scipy.org' |
|
91 | author_email = 'ipython-dev@scipy.org' | |
88 |
|
92 | |||
89 | url = 'http://ipython.scipy.org' |
|
93 | url = 'http://ipython.scipy.org' | |
90 |
|
94 | |||
91 | download_url = 'http://ipython.scipy.org/dist' |
|
95 | download_url = 'http://ipython.scipy.org/dist' | |
92 |
|
96 | |||
93 | platforms = ['Linux','Mac OSX','Windows XP/2000/NT','Windows 95/98/ME'] |
|
97 | platforms = ['Linux','Mac OSX','Windows XP/2000/NT','Windows 95/98/ME'] | |
94 |
|
98 | |||
95 | keywords = ['Interactive','Interpreter','Shell','Parallel','Distributed'] |
|
99 | keywords = ['Interactive','Interpreter','Shell','Parallel','Distributed'] |
@@ -1,323 +1,323 b'' | |||||
1 | #!/usr/bin/env python |
|
1 | #!/usr/bin/env python | |
2 | # encoding: utf-8 |
|
2 | # encoding: utf-8 | |
3 |
|
3 | |||
4 | """Start an IPython cluster conveniently, either locally or remotely. |
|
4 | """Start an IPython cluster conveniently, either locally or remotely. | |
5 |
|
5 | |||
6 | Basic usage |
|
6 | Basic usage | |
7 | ----------- |
|
7 | ----------- | |
8 |
|
8 | |||
9 | For local operation, the simplest mode of usage is: |
|
9 | For local operation, the simplest mode of usage is: | |
10 |
|
10 | |||
11 | %prog -n N |
|
11 | %prog -n N | |
12 |
|
12 | |||
13 | where N is the number of engines you want started. |
|
13 | where N is the number of engines you want started. | |
14 |
|
14 | |||
15 | For remote operation, you must call it with a cluster description file: |
|
15 | For remote operation, you must call it with a cluster description file: | |
16 |
|
16 | |||
17 | %prog -f clusterfile.py |
|
17 | %prog -f clusterfile.py | |
18 |
|
18 | |||
19 | The cluster file is a normal Python script which gets run via execfile(). You |
|
19 | The cluster file is a normal Python script which gets run via execfile(). You | |
20 | can have arbitrary logic in it, but all that matters is that at the end of the |
|
20 | can have arbitrary logic in it, but all that matters is that at the end of the | |
21 | execution, it declares the variables 'controller', 'engines', and optionally |
|
21 | execution, it declares the variables 'controller', 'engines', and optionally | |
22 | 'sshx'. See the accompanying examples for details on what these variables must |
|
22 | 'sshx'. See the accompanying examples for details on what these variables must | |
23 | contain. |
|
23 | contain. | |
24 |
|
24 | |||
25 |
|
25 | |||
26 | Notes |
|
26 | Notes | |
27 | ----- |
|
27 | ----- | |
28 |
|
28 | |||
29 | WARNING: this code is still UNFINISHED and EXPERIMENTAL! It is incomplete, |
|
29 | WARNING: this code is still UNFINISHED and EXPERIMENTAL! It is incomplete, | |
30 | some listed options are not really implemented, and all of its interfaces are |
|
30 | some listed options are not really implemented, and all of its interfaces are | |
31 | subject to change. |
|
31 | subject to change. | |
32 |
|
32 | |||
33 | When operating over SSH for a remote cluster, this program relies on the |
|
33 | When operating over SSH for a remote cluster, this program relies on the | |
34 | existence of a particular script called 'sshx'. This script must live in the |
|
34 | existence of a particular script called 'sshx'. This script must live in the | |
35 | target systems where you'll be running your controller and engines, and is |
|
35 | target systems where you'll be running your controller and engines, and is | |
36 | needed to configure your PATH and PYTHONPATH variables for further execution of |
|
36 | needed to configure your PATH and PYTHONPATH variables for further execution of | |
37 | python code at the other end of an SSH connection. The script can be as simple |
|
37 | python code at the other end of an SSH connection. The script can be as simple | |
38 | as: |
|
38 | as: | |
39 |
|
39 | |||
40 | #!/bin/sh |
|
40 | #!/bin/sh | |
41 | . $HOME/.bashrc |
|
41 | . $HOME/.bashrc | |
42 | "$@" |
|
42 | "$@" | |
43 |
|
43 | |||
44 | which is the default one provided by IPython. You can modify this or provide |
|
44 | which is the default one provided by IPython. You can modify this or provide | |
45 | your own. Since it's quite likely that for different clusters you may need |
|
45 | your own. Since it's quite likely that for different clusters you may need | |
46 | this script to configure things differently or that it may live in different |
|
46 | this script to configure things differently or that it may live in different | |
47 | locations, its full path can be set in the same file where you define the |
|
47 | locations, its full path can be set in the same file where you define the | |
48 | cluster setup. IPython's order of evaluation for this variable is the |
|
48 | cluster setup. IPython's order of evaluation for this variable is the | |
49 | following: |
|
49 | following: | |
50 |
|
50 | |||
51 | a) Internal default: 'sshx'. This only works if it is in the default system |
|
51 | a) Internal default: 'sshx'. This only works if it is in the default system | |
52 | path which SSH sets up in non-interactive mode. |
|
52 | path which SSH sets up in non-interactive mode. | |
53 |
|
53 | |||
54 | b) Environment variable: if $IPYTHON_SSHX is defined, this overrides the |
|
54 | b) Environment variable: if $IPYTHON_SSHX is defined, this overrides the | |
55 | internal default. |
|
55 | internal default. | |
56 |
|
56 | |||
57 | c) Variable 'sshx' in the cluster configuration file: finally, this will |
|
57 | c) Variable 'sshx' in the cluster configuration file: finally, this will | |
58 | override the previous two values. |
|
58 | override the previous two values. | |
59 |
|
59 | |||
60 | This code is Unix-only, with precious little hope of any of this ever working |
|
60 | This code is Unix-only, with precious little hope of any of this ever working | |
61 | under Windows, since we need SSH from the ground up, we background processes, |
|
61 | under Windows, since we need SSH from the ground up, we background processes, | |
62 | etc. Ports of this functionality to Windows are welcome. |
|
62 | etc. Ports of this functionality to Windows are welcome. | |
63 |
|
63 | |||
64 |
|
64 | |||
65 | Call summary |
|
65 | Call summary | |
66 | ------------ |
|
66 | ------------ | |
67 |
|
67 | |||
68 | %prog [options] |
|
68 | %prog [options] | |
69 | """ |
|
69 | """ | |
70 |
|
70 | |||
71 | __docformat__ = "restructuredtext en" |
|
71 | __docformat__ = "restructuredtext en" | |
72 |
|
72 | |||
73 | #------------------------------------------------------------------------------- |
|
73 | #------------------------------------------------------------------------------- | |
74 | # Copyright (C) 2008 The IPython Development Team |
|
74 | # Copyright (C) 2008 The IPython Development Team | |
75 | # |
|
75 | # | |
76 | # Distributed under the terms of the BSD License. The full license is in |
|
76 | # Distributed under the terms of the BSD License. The full license is in | |
77 | # the file COPYING, distributed as part of this software. |
|
77 | # the file COPYING, distributed as part of this software. | |
78 | #------------------------------------------------------------------------------- |
|
78 | #------------------------------------------------------------------------------- | |
79 |
|
79 | |||
80 | #------------------------------------------------------------------------------- |
|
80 | #------------------------------------------------------------------------------- | |
81 | # Stdlib imports |
|
81 | # Stdlib imports | |
82 | #------------------------------------------------------------------------------- |
|
82 | #------------------------------------------------------------------------------- | |
83 |
|
83 | |||
84 | import os |
|
84 | import os | |
85 | import signal |
|
85 | import signal | |
86 | import sys |
|
86 | import sys | |
87 | import time |
|
87 | import time | |
88 |
|
88 | |||
89 | from optparse import OptionParser |
|
89 | from optparse import OptionParser | |
90 | from subprocess import Popen,call |
|
90 | from subprocess import Popen,call | |
91 |
|
91 | |||
92 | #--------------------------------------------------------------------------- |
|
92 | #--------------------------------------------------------------------------- | |
93 | # IPython imports |
|
93 | # IPython imports | |
94 | #--------------------------------------------------------------------------- |
|
94 | #--------------------------------------------------------------------------- | |
95 | from IPython.tools import utils |
|
95 | from IPython.tools import utils | |
96 | from IPython.config import cutils |
|
96 | from IPython.config import cutils | |
97 |
|
97 | |||
98 | #--------------------------------------------------------------------------- |
|
98 | #--------------------------------------------------------------------------- | |
99 | # Normal code begins |
|
99 | # Normal code begins | |
100 | #--------------------------------------------------------------------------- |
|
100 | #--------------------------------------------------------------------------- | |
101 |
|
101 | |||
102 | def parse_args(): |
|
102 | def parse_args(): | |
103 | """Parse command line and return opts,args.""" |
|
103 | """Parse command line and return opts,args.""" | |
104 |
|
104 | |||
105 | parser = OptionParser(usage=__doc__) |
|
105 | parser = OptionParser(usage=__doc__) | |
106 | newopt = parser.add_option # shorthand |
|
106 | newopt = parser.add_option # shorthand | |
107 |
|
107 | |||
108 | newopt("--controller-port", type="int", dest="controllerport", |
|
108 | newopt("--controller-port", type="int", dest="controllerport", | |
109 | help="the TCP port the controller is listening on") |
|
109 | help="the TCP port the controller is listening on") | |
110 |
|
110 | |||
111 | newopt("--controller-ip", type="string", dest="controllerip", |
|
111 | newopt("--controller-ip", type="string", dest="controllerip", | |
112 | help="the TCP ip address of the controller") |
|
112 | help="the TCP ip address of the controller") | |
113 |
|
113 | |||
114 | newopt("-n", "--num", type="int", dest="n",default=2, |
|
114 | newopt("-n", "--num", type="int", dest="n",default=2, | |
115 | help="the number of engines to start") |
|
115 | help="the number of engines to start") | |
116 |
|
116 | |||
117 | newopt("--engine-port", type="int", dest="engineport", |
|
117 | newopt("--engine-port", type="int", dest="engineport", | |
118 | help="the TCP port the controller will listen on for engine " |
|
118 | help="the TCP port the controller will listen on for engine " | |
119 | "connections") |
|
119 | "connections") | |
120 |
|
120 | |||
121 | newopt("--engine-ip", type="string", dest="engineip", |
|
121 | newopt("--engine-ip", type="string", dest="engineip", | |
122 | help="the TCP ip address the controller will listen on " |
|
122 | help="the TCP ip address the controller will listen on " | |
123 | "for engine connections") |
|
123 | "for engine connections") | |
124 |
|
124 | |||
125 | newopt("--mpi", type="string", dest="mpi", |
|
125 | newopt("--mpi", type="string", dest="mpi", | |
126 | help="use mpi with package: for instance --mpi=mpi4py") |
|
126 | help="use mpi with package: for instance --mpi=mpi4py") | |
127 |
|
127 | |||
128 | newopt("-l", "--logfile", type="string", dest="logfile", |
|
128 | newopt("-l", "--logfile", type="string", dest="logfile", | |
129 | help="log file name") |
|
129 | help="log file name") | |
130 |
|
130 | |||
131 | newopt('-f','--cluster-file',dest='clusterfile', |
|
131 | newopt('-f','--cluster-file',dest='clusterfile', | |
132 | help='file describing a remote cluster') |
|
132 | help='file describing a remote cluster') | |
133 |
|
133 | |||
134 | return parser.parse_args() |
|
134 | return parser.parse_args() | |
135 |
|
135 | |||
136 | def numAlive(controller,engines): |
|
136 | def numAlive(controller,engines): | |
137 | """Return the number of processes still alive.""" |
|
137 | """Return the number of processes still alive.""" | |
138 | retcodes = [controller.poll()] + \ |
|
138 | retcodes = [controller.poll()] + \ | |
139 | [e.poll() for e in engines] |
|
139 | [e.poll() for e in engines] | |
140 | return retcodes.count(None) |
|
140 | return retcodes.count(None) | |
141 |
|
141 | |||
142 | stop = lambda pid: os.kill(pid,signal.SIGINT) |
|
142 | stop = lambda pid: os.kill(pid,signal.SIGINT) | |
143 | kill = lambda pid: os.kill(pid,signal.SIGTERM) |
|
143 | kill = lambda pid: os.kill(pid,signal.SIGTERM) | |
144 |
|
144 | |||
145 | def cleanup(clean,controller,engines): |
|
145 | def cleanup(clean,controller,engines): | |
146 | """Stop the controller and engines with the given cleanup method.""" |
|
146 | """Stop the controller and engines with the given cleanup method.""" | |
147 |
|
147 | |||
148 | for e in engines: |
|
148 | for e in engines: | |
149 | if e.poll() is None: |
|
149 | if e.poll() is None: | |
150 | print 'Stopping engine, pid',e.pid |
|
150 | print 'Stopping engine, pid',e.pid | |
151 | clean(e.pid) |
|
151 | clean(e.pid) | |
152 | if controller.poll() is None: |
|
152 | if controller.poll() is None: | |
153 | print 'Stopping controller, pid',controller.pid |
|
153 | print 'Stopping controller, pid',controller.pid | |
154 | clean(controller.pid) |
|
154 | clean(controller.pid) | |
155 |
|
155 | |||
156 |
|
156 | |||
157 | def ensureDir(path): |
|
157 | def ensureDir(path): | |
158 | """Ensure a directory exists or raise an exception.""" |
|
158 | """Ensure a directory exists or raise an exception.""" | |
159 | if not os.path.isdir(path): |
|
159 | if not os.path.isdir(path): | |
160 | os.makedirs(path) |
|
160 | os.makedirs(path) | |
161 |
|
161 | |||
162 |
|
162 | |||
163 | def startMsg(control_host,control_port=10105): |
|
163 | def startMsg(control_host,control_port=10105): | |
164 | """Print a startup message""" |
|
164 | """Print a startup message""" | |
165 |
|
165 | |||
166 | print 'Your cluster is up and running.' |
|
166 | print 'Your cluster is up and running.' | |
167 |
|
167 | |||
168 | print 'For interactive use, you can make a MultiEngineClient with:' |
|
168 | print 'For interactive use, you can make a MultiEngineClient with:' | |
169 |
|
169 | |||
170 | print 'from IPython.kernel import client' |
|
170 | print 'from IPython.kernel import client' | |
171 |
print "mec = client.MultiEngineClient( |
|
171 | print "mec = client.MultiEngineClient()" | |
172 | (control_host,control_port) |
|
|||
173 |
|
172 | |||
174 | print 'You can then cleanly stop the cluster from IPython using:' |
|
173 | print 'You can then cleanly stop the cluster from IPython using:' | |
175 |
|
174 | |||
176 | print 'mec.kill(controller=True)' |
|
175 | print 'mec.kill(controller=True)' | |
177 |
|
176 | |||
178 |
|
177 | |||
179 |
|
178 | |||
180 | def clusterLocal(opt,arg): |
|
179 | def clusterLocal(opt,arg): | |
181 | """Start a cluster on the local machine.""" |
|
180 | """Start a cluster on the local machine.""" | |
182 |
|
181 | |||
183 | # Store all logs inside the ipython directory |
|
182 | # Store all logs inside the ipython directory | |
184 | ipdir = cutils.get_ipython_dir() |
|
183 | ipdir = cutils.get_ipython_dir() | |
185 | pjoin = os.path.join |
|
184 | pjoin = os.path.join | |
186 |
|
185 | |||
187 | logfile = opt.logfile |
|
186 | logfile = opt.logfile | |
188 | if logfile is None: |
|
187 | if logfile is None: | |
189 | logdir_base = pjoin(ipdir,'log') |
|
188 | logdir_base = pjoin(ipdir,'log') | |
190 | ensureDir(logdir_base) |
|
189 | ensureDir(logdir_base) | |
191 | logfile = pjoin(logdir_base,'ipcluster-') |
|
190 | logfile = pjoin(logdir_base,'ipcluster-') | |
192 |
|
191 | |||
193 | print 'Starting controller:', |
|
192 | print 'Starting controller:', | |
194 | controller = Popen(['ipcontroller','--logfile',logfile]) |
|
193 | controller = Popen(['ipcontroller','--logfile',logfile]) | |
195 | print 'Controller PID:',controller.pid |
|
194 | print 'Controller PID:',controller.pid | |
196 |
|
195 | |||
197 | print 'Starting engines: ', |
|
196 | print 'Starting engines: ', | |
198 |
time.sleep( |
|
197 | time.sleep(5) | |
199 |
|
198 | |||
200 | englogfile = '%s%s-' % (logfile,controller.pid) |
|
199 | englogfile = '%s%s-' % (logfile,controller.pid) | |
201 | mpi = opt.mpi |
|
200 | mpi = opt.mpi | |
202 | if mpi: # start with mpi - killing the engines with sigterm will not work if you do this |
|
201 | if mpi: # start with mpi - killing the engines with sigterm will not work if you do this | |
203 | engines = [Popen(['mpirun', '-np', str(opt.n), 'ipengine', '--mpi', mpi, '--logfile',englogfile])] |
|
202 | engines = [Popen(['mpirun', '-np', str(opt.n), 'ipengine', '--mpi', mpi, '--logfile',englogfile])] | |
|
203 | # engines = [Popen(['mpirun', '-np', str(opt.n), 'ipengine', '--mpi', mpi])] | |||
204 | else: # do what we would normally do |
|
204 | else: # do what we would normally do | |
205 | engines = [ Popen(['ipengine','--logfile',englogfile]) |
|
205 | engines = [ Popen(['ipengine','--logfile',englogfile]) | |
206 | for i in range(opt.n) ] |
|
206 | for i in range(opt.n) ] | |
207 | eids = [e.pid for e in engines] |
|
207 | eids = [e.pid for e in engines] | |
208 | print 'Engines PIDs: ',eids |
|
208 | print 'Engines PIDs: ',eids | |
209 | print 'Log files: %s*' % englogfile |
|
209 | print 'Log files: %s*' % englogfile | |
210 |
|
210 | |||
211 | proc_ids = eids + [controller.pid] |
|
211 | proc_ids = eids + [controller.pid] | |
212 | procs = engines + [controller] |
|
212 | procs = engines + [controller] | |
213 |
|
213 | |||
214 | grpid = os.getpgrp() |
|
214 | grpid = os.getpgrp() | |
215 | try: |
|
215 | try: | |
216 | startMsg('127.0.0.1') |
|
216 | startMsg('127.0.0.1') | |
217 | print 'You can also hit Ctrl-C to stop it, or use from the cmd line:' |
|
217 | print 'You can also hit Ctrl-C to stop it, or use from the cmd line:' | |
218 |
|
218 | |||
219 | print 'kill -INT',grpid |
|
219 | print 'kill -INT',grpid | |
220 |
|
220 | |||
221 | try: |
|
221 | try: | |
222 | while True: |
|
222 | while True: | |
223 | time.sleep(5) |
|
223 | time.sleep(5) | |
224 | except: |
|
224 | except: | |
225 | pass |
|
225 | pass | |
226 | finally: |
|
226 | finally: | |
227 | print 'Stopping cluster. Cleaning up...' |
|
227 | print 'Stopping cluster. Cleaning up...' | |
228 | cleanup(stop,controller,engines) |
|
228 | cleanup(stop,controller,engines) | |
229 | for i in range(4): |
|
229 | for i in range(4): | |
230 | time.sleep(i+2) |
|
230 | time.sleep(i+2) | |
231 | nZombies = numAlive(controller,engines) |
|
231 | nZombies = numAlive(controller,engines) | |
232 | if nZombies== 0: |
|
232 | if nZombies== 0: | |
233 | print 'OK: All processes cleaned up.' |
|
233 | print 'OK: All processes cleaned up.' | |
234 | break |
|
234 | break | |
235 | print 'Trying again, %d processes did not stop...' % nZombies |
|
235 | print 'Trying again, %d processes did not stop...' % nZombies | |
236 | cleanup(kill,controller,engines) |
|
236 | cleanup(kill,controller,engines) | |
237 | if numAlive(controller,engines) == 0: |
|
237 | if numAlive(controller,engines) == 0: | |
238 | print 'OK: All processes cleaned up.' |
|
238 | print 'OK: All processes cleaned up.' | |
239 | break |
|
239 | break | |
240 | else: |
|
240 | else: | |
241 | print '*'*75 |
|
241 | print '*'*75 | |
242 | print 'ERROR: could not kill some processes, try to do it', |
|
242 | print 'ERROR: could not kill some processes, try to do it', | |
243 | print 'manually.' |
|
243 | print 'manually.' | |
244 | zombies = [] |
|
244 | zombies = [] | |
245 | if controller.returncode is None: |
|
245 | if controller.returncode is None: | |
246 | print 'Controller is alive: pid =',controller.pid |
|
246 | print 'Controller is alive: pid =',controller.pid | |
247 | zombies.append(controller.pid) |
|
247 | zombies.append(controller.pid) | |
248 | liveEngines = [ e for e in engines if e.returncode is None ] |
|
248 | liveEngines = [ e for e in engines if e.returncode is None ] | |
249 | for e in liveEngines: |
|
249 | for e in liveEngines: | |
250 | print 'Engine is alive: pid =',e.pid |
|
250 | print 'Engine is alive: pid =',e.pid | |
251 | zombies.append(e.pid) |
|
251 | zombies.append(e.pid) | |
252 |
|
252 | |||
253 | print 'Zombie summary:',' '.join(map(str,zombies)) |
|
253 | print 'Zombie summary:',' '.join(map(str,zombies)) | |
254 |
|
254 | |||
255 | def clusterRemote(opt,arg): |
|
255 | def clusterRemote(opt,arg): | |
256 | """Start a remote cluster over SSH""" |
|
256 | """Start a remote cluster over SSH""" | |
257 |
|
257 | |||
258 | # Load the remote cluster configuration |
|
258 | # Load the remote cluster configuration | |
259 | clConfig = {} |
|
259 | clConfig = {} | |
260 | execfile(opt.clusterfile,clConfig) |
|
260 | execfile(opt.clusterfile,clConfig) | |
261 | contConfig = clConfig['controller'] |
|
261 | contConfig = clConfig['controller'] | |
262 | engConfig = clConfig['engines'] |
|
262 | engConfig = clConfig['engines'] | |
263 | # Determine where to find sshx: |
|
263 | # Determine where to find sshx: | |
264 | sshx = clConfig.get('sshx',os.environ.get('IPYTHON_SSHX','sshx')) |
|
264 | sshx = clConfig.get('sshx',os.environ.get('IPYTHON_SSHX','sshx')) | |
265 |
|
265 | |||
266 | # Store all logs inside the ipython directory |
|
266 | # Store all logs inside the ipython directory | |
267 | ipdir = cutils.get_ipython_dir() |
|
267 | ipdir = cutils.get_ipython_dir() | |
268 | pjoin = os.path.join |
|
268 | pjoin = os.path.join | |
269 |
|
269 | |||
270 | logfile = opt.logfile |
|
270 | logfile = opt.logfile | |
271 | if logfile is None: |
|
271 | if logfile is None: | |
272 | logdir_base = pjoin(ipdir,'log') |
|
272 | logdir_base = pjoin(ipdir,'log') | |
273 | ensureDir(logdir_base) |
|
273 | ensureDir(logdir_base) | |
274 | logfile = pjoin(logdir_base,'ipcluster') |
|
274 | logfile = pjoin(logdir_base,'ipcluster') | |
275 |
|
275 | |||
276 | # Append this script's PID to the logfile name always |
|
276 | # Append this script's PID to the logfile name always | |
277 | logfile = '%s-%s' % (logfile,os.getpid()) |
|
277 | logfile = '%s-%s' % (logfile,os.getpid()) | |
278 |
|
278 | |||
279 | print 'Starting controller:' |
|
279 | print 'Starting controller:' | |
280 | # Controller data: |
|
280 | # Controller data: | |
281 | xsys = os.system |
|
281 | xsys = os.system | |
282 |
|
282 | |||
283 | contHost = contConfig['host'] |
|
283 | contHost = contConfig['host'] | |
284 | contLog = '%s-con-%s-' % (logfile,contHost) |
|
284 | contLog = '%s-con-%s-' % (logfile,contHost) | |
285 | cmd = "ssh %s '%s' 'ipcontroller --logfile %s' &" % \ |
|
285 | cmd = "ssh %s '%s' 'ipcontroller --logfile %s' &" % \ | |
286 | (contHost,sshx,contLog) |
|
286 | (contHost,sshx,contLog) | |
287 | #print 'cmd:<%s>' % cmd # dbg |
|
287 | #print 'cmd:<%s>' % cmd # dbg | |
288 | xsys(cmd) |
|
288 | xsys(cmd) | |
289 | time.sleep(2) |
|
289 | time.sleep(2) | |
290 |
|
290 | |||
291 | print 'Starting engines: ' |
|
291 | print 'Starting engines: ' | |
292 | for engineHost,engineData in engConfig.iteritems(): |
|
292 | for engineHost,engineData in engConfig.iteritems(): | |
293 | if isinstance(engineData,int): |
|
293 | if isinstance(engineData,int): | |
294 | numEngines = engineData |
|
294 | numEngines = engineData | |
295 | else: |
|
295 | else: | |
296 | raise NotImplementedError('port configuration not finished for engines') |
|
296 | raise NotImplementedError('port configuration not finished for engines') | |
297 |
|
297 | |||
298 | print 'Sarting %d engines on %s' % (numEngines,engineHost) |
|
298 | print 'Sarting %d engines on %s' % (numEngines,engineHost) | |
299 | engLog = '%s-eng-%s-' % (logfile,engineHost) |
|
299 | engLog = '%s-eng-%s-' % (logfile,engineHost) | |
300 | for i in range(numEngines): |
|
300 | for i in range(numEngines): | |
301 | cmd = "ssh %s '%s' 'ipengine --controller-ip %s --logfile %s' &" % \ |
|
301 | cmd = "ssh %s '%s' 'ipengine --controller-ip %s --logfile %s' &" % \ | |
302 | (engineHost,sshx,contHost,engLog) |
|
302 | (engineHost,sshx,contHost,engLog) | |
303 | #print 'cmd:<%s>' % cmd # dbg |
|
303 | #print 'cmd:<%s>' % cmd # dbg | |
304 | xsys(cmd) |
|
304 | xsys(cmd) | |
305 | # Wait after each host a little bit |
|
305 | # Wait after each host a little bit | |
306 | time.sleep(1) |
|
306 | time.sleep(1) | |
307 |
|
307 | |||
308 | startMsg(contConfig['host']) |
|
308 | startMsg(contConfig['host']) | |
309 |
|
309 | |||
310 | def main(): |
|
310 | def main(): | |
311 | """Main driver for the two big options: local or remote cluster.""" |
|
311 | """Main driver for the two big options: local or remote cluster.""" | |
312 |
|
312 | |||
313 | opt,arg = parse_args() |
|
313 | opt,arg = parse_args() | |
314 |
|
314 | |||
315 | clusterfile = opt.clusterfile |
|
315 | clusterfile = opt.clusterfile | |
316 | if clusterfile: |
|
316 | if clusterfile: | |
317 | clusterRemote(opt,arg) |
|
317 | clusterRemote(opt,arg) | |
318 | else: |
|
318 | else: | |
319 | clusterLocal(opt,arg) |
|
319 | clusterLocal(opt,arg) | |
320 |
|
320 | |||
321 |
|
321 | |||
322 | if __name__=='__main__': |
|
322 | if __name__=='__main__': | |
323 | main() |
|
323 | main() |
@@ -1,169 +1,171 b'' | |||||
1 | #!/usr/bin/env python |
|
1 | #!/usr/bin/env python | |
2 | # encoding: utf-8 |
|
2 | # encoding: utf-8 | |
3 |
|
3 | |||
4 | """Start the IPython Engine.""" |
|
4 | """Start the IPython Engine.""" | |
5 |
|
5 | |||
6 | __docformat__ = "restructuredtext en" |
|
6 | __docformat__ = "restructuredtext en" | |
7 |
|
7 | |||
8 | #------------------------------------------------------------------------------- |
|
8 | #------------------------------------------------------------------------------- | |
9 | # Copyright (C) 2008 The IPython Development Team |
|
9 | # Copyright (C) 2008 The IPython Development Team | |
10 | # |
|
10 | # | |
11 | # Distributed under the terms of the BSD License. The full license is in |
|
11 | # Distributed under the terms of the BSD License. The full license is in | |
12 | # the file COPYING, distributed as part of this software. |
|
12 | # the file COPYING, distributed as part of this software. | |
13 | #------------------------------------------------------------------------------- |
|
13 | #------------------------------------------------------------------------------- | |
14 |
|
14 | |||
15 | #------------------------------------------------------------------------------- |
|
15 | #------------------------------------------------------------------------------- | |
16 | # Imports |
|
16 | # Imports | |
17 | #------------------------------------------------------------------------------- |
|
17 | #------------------------------------------------------------------------------- | |
18 |
|
18 | |||
19 | # Python looks for an empty string at the beginning of sys.path to enable |
|
19 | # Python looks for an empty string at the beginning of sys.path to enable | |
20 | # importing from the cwd. |
|
20 | # importing from the cwd. | |
21 | import sys |
|
21 | import sys | |
22 | sys.path.insert(0, '') |
|
22 | sys.path.insert(0, '') | |
23 |
|
23 | |||
24 | import sys, os |
|
24 | import sys, os | |
25 | from optparse import OptionParser |
|
25 | from optparse import OptionParser | |
26 |
|
26 | |||
27 | from twisted.application import service |
|
27 | from twisted.application import service | |
28 | from twisted.internet import reactor |
|
28 | from twisted.internet import reactor | |
29 | from twisted.python import log |
|
29 | from twisted.python import log | |
30 |
|
30 | |||
31 | from IPython.kernel.fcutil import Tub, UnauthenticatedTub |
|
31 | from IPython.kernel.fcutil import Tub, UnauthenticatedTub | |
32 |
|
32 | |||
33 | from IPython.kernel.core.config import config_manager as core_config_manager |
|
33 | from IPython.kernel.core.config import config_manager as core_config_manager | |
34 | from IPython.config.cutils import import_item |
|
34 | from IPython.config.cutils import import_item | |
35 | from IPython.kernel.engineservice import EngineService |
|
35 | from IPython.kernel.engineservice import EngineService | |
36 | from IPython.kernel.config import config_manager as kernel_config_manager |
|
36 | from IPython.kernel.config import config_manager as kernel_config_manager | |
37 | from IPython.kernel.engineconnector import EngineConnector |
|
37 | from IPython.kernel.engineconnector import EngineConnector | |
38 |
|
38 | |||
39 |
|
39 | |||
40 | #------------------------------------------------------------------------------- |
|
40 | #------------------------------------------------------------------------------- | |
41 | # Code |
|
41 | # Code | |
42 | #------------------------------------------------------------------------------- |
|
42 | #------------------------------------------------------------------------------- | |
43 |
|
43 | |||
44 | def start_engine(): |
|
44 | def start_engine(): | |
45 | """ |
|
45 | """ | |
46 | Start the engine, by creating it and starting the Twisted reactor. |
|
46 | Start the engine, by creating it and starting the Twisted reactor. | |
47 |
|
47 | |||
48 | This method does: |
|
48 | This method does: | |
49 |
|
49 | |||
50 | * If it exists, runs the `mpi_import_statement` to call `MPI_Init` |
|
50 | * If it exists, runs the `mpi_import_statement` to call `MPI_Init` | |
51 | * Starts the engine logging |
|
51 | * Starts the engine logging | |
52 | * Creates an IPython shell and wraps it in an `EngineService` |
|
52 | * Creates an IPython shell and wraps it in an `EngineService` | |
53 | * Creates a `foolscap.Tub` to use in connecting to a controller. |
|
53 | * Creates a `foolscap.Tub` to use in connecting to a controller. | |
54 | * Uses the tub and the `EngineService` along with a Foolscap URL |
|
54 | * Uses the tub and the `EngineService` along with a Foolscap URL | |
55 | (or FURL) to connect to the controller and register the engine |
|
55 | (or FURL) to connect to the controller and register the engine | |
56 | with the controller |
|
56 | with the controller | |
57 | """ |
|
57 | """ | |
58 | kernel_config = kernel_config_manager.get_config_obj() |
|
58 | kernel_config = kernel_config_manager.get_config_obj() | |
59 | core_config = core_config_manager.get_config_obj() |
|
59 | core_config = core_config_manager.get_config_obj() | |
60 |
|
60 | |||
|
61 | ||||
61 | # Execute the mpi import statement that needs to call MPI_Init |
|
62 | # Execute the mpi import statement that needs to call MPI_Init | |
|
63 | global mpi | |||
62 | mpikey = kernel_config['mpi']['default'] |
|
64 | mpikey = kernel_config['mpi']['default'] | |
63 | mpi_import_statement = kernel_config['mpi'].get(mpikey, None) |
|
65 | mpi_import_statement = kernel_config['mpi'].get(mpikey, None) | |
64 | if mpi_import_statement is not None: |
|
66 | if mpi_import_statement is not None: | |
65 | try: |
|
67 | try: | |
66 |
exec mpi_import_statement in |
|
68 | exec mpi_import_statement in globals() | |
67 | except: |
|
69 | except: | |
68 | mpi = None |
|
70 | mpi = None | |
69 | else: |
|
71 | else: | |
70 | mpi = None |
|
72 | mpi = mpi_namespace.get('mpi') | |
71 |
|
73 | |||
72 | # Start logging |
|
74 | # Start logging | |
73 | logfile = kernel_config['engine']['logfile'] |
|
75 | logfile = kernel_config['engine']['logfile'] | |
74 | if logfile: |
|
76 | if logfile: | |
75 | logfile = logfile + str(os.getpid()) + '.log' |
|
77 | logfile = logfile + str(os.getpid()) + '.log' | |
76 | try: |
|
78 | try: | |
77 | openLogFile = open(logfile, 'w') |
|
79 | openLogFile = open(logfile, 'w') | |
78 | except: |
|
80 | except: | |
79 | openLogFile = sys.stdout |
|
81 | openLogFile = sys.stdout | |
80 | else: |
|
82 | else: | |
81 | openLogFile = sys.stdout |
|
83 | openLogFile = sys.stdout | |
82 | log.startLogging(openLogFile) |
|
84 | log.startLogging(openLogFile) | |
83 |
|
85 | |||
84 | # Create the underlying shell class and EngineService |
|
86 | # Create the underlying shell class and EngineService | |
85 | shell_class = import_item(core_config['shell']['shell_class']) |
|
87 | shell_class = import_item(core_config['shell']['shell_class']) | |
86 | engine_service = EngineService(shell_class, mpi=mpi) |
|
88 | engine_service = EngineService(shell_class, mpi=mpi) | |
87 | shell_import_statement = core_config['shell']['import_statement'] |
|
89 | shell_import_statement = core_config['shell']['import_statement'] | |
88 | if shell_import_statement: |
|
90 | if shell_import_statement: | |
89 | try: |
|
91 | try: | |
90 | engine_service.execute(shell_import_statement) |
|
92 | engine_service.execute(shell_import_statement) | |
91 | except: |
|
93 | except: | |
92 | log.msg("Error running import_statement: %s" % sis) |
|
94 | log.msg("Error running import_statement: %s" % sis) | |
93 |
|
95 | |||
94 | # Create the service hierarchy |
|
96 | # Create the service hierarchy | |
95 | main_service = service.MultiService() |
|
97 | main_service = service.MultiService() | |
96 | engine_service.setServiceParent(main_service) |
|
98 | engine_service.setServiceParent(main_service) | |
97 | tub_service = Tub() |
|
99 | tub_service = Tub() | |
98 | tub_service.setServiceParent(main_service) |
|
100 | tub_service.setServiceParent(main_service) | |
99 | # This needs to be called before the connection is initiated |
|
101 | # This needs to be called before the connection is initiated | |
100 | main_service.startService() |
|
102 | main_service.startService() | |
101 |
|
103 | |||
102 | # This initiates the connection to the controller and calls |
|
104 | # This initiates the connection to the controller and calls | |
103 | # register_engine to tell the controller we are ready to do work |
|
105 | # register_engine to tell the controller we are ready to do work | |
104 | engine_connector = EngineConnector(tub_service) |
|
106 | engine_connector = EngineConnector(tub_service) | |
105 | furl_file = kernel_config['engine']['furl_file'] |
|
107 | furl_file = kernel_config['engine']['furl_file'] | |
106 | d = engine_connector.connect_to_controller(engine_service, furl_file) |
|
108 | d = engine_connector.connect_to_controller(engine_service, furl_file) | |
107 | d.addErrback(lambda _: reactor.stop()) |
|
109 | d.addErrback(lambda _: reactor.stop()) | |
108 |
|
110 | |||
109 | reactor.run() |
|
111 | reactor.run() | |
110 |
|
112 | |||
111 |
|
113 | |||
112 | def init_config(): |
|
114 | def init_config(): | |
113 | """ |
|
115 | """ | |
114 | Initialize the configuration using default and command line options. |
|
116 | Initialize the configuration using default and command line options. | |
115 | """ |
|
117 | """ | |
116 |
|
118 | |||
117 | parser = OptionParser() |
|
119 | parser = OptionParser() | |
118 |
|
120 | |||
119 | parser.add_option( |
|
121 | parser.add_option( | |
120 | "--furl-file", |
|
122 | "--furl-file", | |
121 | type="string", |
|
123 | type="string", | |
122 | dest="furl_file", |
|
124 | dest="furl_file", | |
123 | help="The filename containing the FURL of the controller" |
|
125 | help="The filename containing the FURL of the controller" | |
124 | ) |
|
126 | ) | |
125 | parser.add_option( |
|
127 | parser.add_option( | |
126 | "--mpi", |
|
128 | "--mpi", | |
127 | type="string", |
|
129 | type="string", | |
128 | dest="mpi", |
|
130 | dest="mpi", | |
129 | help="How to enable MPI (mpi4py, pytrilinos, or empty string to disable)" |
|
131 | help="How to enable MPI (mpi4py, pytrilinos, or empty string to disable)" | |
130 | ) |
|
132 | ) | |
131 | parser.add_option( |
|
133 | parser.add_option( | |
132 | "-l", |
|
134 | "-l", | |
133 | "--logfile", |
|
135 | "--logfile", | |
134 | type="string", |
|
136 | type="string", | |
135 | dest="logfile", |
|
137 | dest="logfile", | |
136 | help="log file name (default is stdout)" |
|
138 | help="log file name (default is stdout)" | |
137 | ) |
|
139 | ) | |
138 | parser.add_option( |
|
140 | parser.add_option( | |
139 | "--ipythondir", |
|
141 | "--ipythondir", | |
140 | type="string", |
|
142 | type="string", | |
141 | dest="ipythondir", |
|
143 | dest="ipythondir", | |
142 | help="look for config files and profiles in this directory" |
|
144 | help="look for config files and profiles in this directory" | |
143 | ) |
|
145 | ) | |
144 |
|
146 | |||
145 | (options, args) = parser.parse_args() |
|
147 | (options, args) = parser.parse_args() | |
146 |
|
148 | |||
147 | kernel_config_manager.update_config_obj_from_default_file(options.ipythondir) |
|
149 | kernel_config_manager.update_config_obj_from_default_file(options.ipythondir) | |
148 | core_config_manager.update_config_obj_from_default_file(options.ipythondir) |
|
150 | core_config_manager.update_config_obj_from_default_file(options.ipythondir) | |
149 |
|
151 | |||
150 | kernel_config = kernel_config_manager.get_config_obj() |
|
152 | kernel_config = kernel_config_manager.get_config_obj() | |
151 | # Now override with command line options |
|
153 | # Now override with command line options | |
152 | if options.furl_file is not None: |
|
154 | if options.furl_file is not None: | |
153 | kernel_config['engine']['furl_file'] = options.furl_file |
|
155 | kernel_config['engine']['furl_file'] = options.furl_file | |
154 | if options.logfile is not None: |
|
156 | if options.logfile is not None: | |
155 | kernel_config['engine']['logfile'] = options.logfile |
|
157 | kernel_config['engine']['logfile'] = options.logfile | |
156 | if options.mpi is not None: |
|
158 | if options.mpi is not None: | |
157 | kernel_config['mpi']['default'] = options.mpi |
|
159 | kernel_config['mpi']['default'] = options.mpi | |
158 |
|
160 | |||
159 |
|
161 | |||
160 | def main(): |
|
162 | def main(): | |
161 | """ |
|
163 | """ | |
162 | After creating the configuration information, start the engine. |
|
164 | After creating the configuration information, start the engine. | |
163 | """ |
|
165 | """ | |
164 | init_config() |
|
166 | init_config() | |
165 | start_engine() |
|
167 | start_engine() | |
166 |
|
168 | |||
167 |
|
169 | |||
168 | if __name__ == "__main__": |
|
170 | if __name__ == "__main__": | |
169 | main() No newline at end of file |
|
171 | main() |
@@ -1,162 +1,164 b'' | |||||
1 | .. _changes: |
|
1 | .. _changes: | |
2 |
|
2 | |||
3 | ========== |
|
3 | ========== | |
4 | What's new |
|
4 | What's new | |
5 | ========== |
|
5 | ========== | |
6 |
|
6 | |||
7 | .. contents:: |
|
7 | .. contents:: | |
8 |
|
8 | |||
9 | Release 0.9 |
|
9 | Release 0.9 | |
10 | =========== |
|
10 | =========== | |
11 |
|
11 | |||
12 | New features |
|
12 | New features | |
13 | ------------ |
|
13 | ------------ | |
14 |
|
14 | |||
15 | * All of the parallel computing capabilities from `ipython1-dev` have been merged into |
|
15 | * All of the parallel computing capabilities from `ipython1-dev` have been merged into | |
16 | IPython proper. This resulted in the following new subpackages: |
|
16 | IPython proper. This resulted in the following new subpackages: | |
17 | :mod:`IPython.kernel`, :mod:`IPython.kernel.core`, :mod:`IPython.config`, |
|
17 | :mod:`IPython.kernel`, :mod:`IPython.kernel.core`, :mod:`IPython.config`, | |
18 | :mod:`IPython.tools` and :mod:`IPython.testing`. |
|
18 | :mod:`IPython.tools` and :mod:`IPython.testing`. | |
19 | * As part of merging in the `ipython1-dev` stuff, the `setup.py` script and friends |
|
19 | * As part of merging in the `ipython1-dev` stuff, the `setup.py` script and friends | |
20 | have been completely refactored. Now we are checking for dependencies using |
|
20 | have been completely refactored. Now we are checking for dependencies using | |
21 | the approach that matplotlib uses. |
|
21 | the approach that matplotlib uses. | |
22 | * The documentation has been completely reorganized to accept the documentation |
|
22 | * The documentation has been completely reorganized to accept the documentation | |
23 | from `ipython1-dev`. |
|
23 | from `ipython1-dev`. | |
24 | * We have switched to using Foolscap for all of our network protocols in |
|
24 | * We have switched to using Foolscap for all of our network protocols in | |
25 | :mod:`IPython.kernel`. This gives us secure connections that are both encrypted |
|
25 | :mod:`IPython.kernel`. This gives us secure connections that are both encrypted | |
26 | and authenticated. |
|
26 | and authenticated. | |
27 | * We have a brand new `COPYING.txt` files that describes the IPython license |
|
27 | * We have a brand new `COPYING.txt` files that describes the IPython license | |
28 | and copyright. The biggest change is that we are putting "The IPython |
|
28 | and copyright. The biggest change is that we are putting "The IPython | |
29 | Development Team" as the copyright holder. We give more details about exactly |
|
29 | Development Team" as the copyright holder. We give more details about exactly | |
30 | what this means in this file. All developer should read this and use the new |
|
30 | what this means in this file. All developer should read this and use the new | |
31 | banner in all IPython source code files. |
|
31 | banner in all IPython source code files. | |
32 |
|
|
32 | * sh profile: ./foo runs foo as system command, no need to do !./foo anymore | |
33 |
|
33 | |||
34 | Bug fixes |
|
34 | Bug fixes | |
35 | --------- |
|
35 | --------- | |
36 |
|
36 | |||
|
37 | * The :mod:`IPython.kernel.scripts.ipengine` script was exec'ing mpi_import_statement | |||
|
38 | incorrectly, which was leading the engine to crash when mpi was enabled. | |||
37 | * A few subpackages has missing `__init__.py` files. |
|
39 | * A few subpackages has missing `__init__.py` files. | |
38 | * The documentation is only created is Sphinx is found. Previously, the `setup.py` |
|
40 | * The documentation is only created is Sphinx is found. Previously, the `setup.py` | |
39 | script would fail if it was missing. |
|
41 | script would fail if it was missing. | |
40 |
|
42 | |||
41 | Backwards incompatible changes |
|
43 | Backwards incompatible changes | |
42 | ------------------------------ |
|
44 | ------------------------------ | |
43 |
|
45 | |||
44 | * IPython has a larger set of dependencies if you want all of its capabilities. |
|
46 | * IPython has a larger set of dependencies if you want all of its capabilities. | |
45 | See the `setup.py` script for details. |
|
47 | See the `setup.py` script for details. | |
46 | * The constructors for :class:`IPython.kernel.client.MultiEngineClient` and |
|
48 | * The constructors for :class:`IPython.kernel.client.MultiEngineClient` and | |
47 | :class:`IPython.kernel.client.TaskClient` no longer take the (ip,port) tuple. |
|
49 | :class:`IPython.kernel.client.TaskClient` no longer take the (ip,port) tuple. | |
48 | Instead they take the filename of a file that contains the FURL for that |
|
50 | Instead they take the filename of a file that contains the FURL for that | |
49 | client. If the FURL file is in your IPYTHONDIR, it will be found automatically |
|
51 | client. If the FURL file is in your IPYTHONDIR, it will be found automatically | |
50 | and the constructor can be left empty. |
|
52 | and the constructor can be left empty. | |
51 | * The asynchronous clients in :mod:`IPython.kernel.asyncclient` are now created |
|
53 | * The asynchronous clients in :mod:`IPython.kernel.asyncclient` are now created | |
52 | using the factory functions :func:`get_multiengine_client` and |
|
54 | using the factory functions :func:`get_multiengine_client` and | |
53 | :func:`get_task_client`. These return a `Deferred` to the actual client. |
|
55 | :func:`get_task_client`. These return a `Deferred` to the actual client. | |
54 | * The command line options to `ipcontroller` and `ipengine` have changed to |
|
56 | * The command line options to `ipcontroller` and `ipengine` have changed to | |
55 | reflect the new Foolscap network protocol and the FURL files. Please see the |
|
57 | reflect the new Foolscap network protocol and the FURL files. Please see the | |
56 | help for these scripts for details. |
|
58 | help for these scripts for details. | |
57 | * The configuration files for the kernel have changed because of the Foolscap stuff. |
|
59 | * The configuration files for the kernel have changed because of the Foolscap stuff. | |
58 | If you were using custom config files before, you should delete them and regenerate |
|
60 | If you were using custom config files before, you should delete them and regenerate | |
59 | new ones. |
|
61 | new ones. | |
60 |
|
62 | |||
61 | Changes merged in from IPython1 |
|
63 | Changes merged in from IPython1 | |
62 | ------------------------------- |
|
64 | ------------------------------- | |
63 |
|
65 | |||
64 | New features |
|
66 | New features | |
65 | ............ |
|
67 | ............ | |
66 |
|
68 | |||
67 | * Much improved ``setup.py`` and ``setupegg.py`` scripts. Because Twisted |
|
69 | * Much improved ``setup.py`` and ``setupegg.py`` scripts. Because Twisted | |
68 | and zope.interface are now easy installable, we can declare them as dependencies |
|
70 | and zope.interface are now easy installable, we can declare them as dependencies | |
69 | in our setupegg.py script. |
|
71 | in our setupegg.py script. | |
70 | * IPython is now compatible with Twisted 2.5.0 and 8.x. |
|
72 | * IPython is now compatible with Twisted 2.5.0 and 8.x. | |
71 | * Added a new example of how to use :mod:`ipython1.kernel.asynclient`. |
|
73 | * Added a new example of how to use :mod:`ipython1.kernel.asynclient`. | |
72 | * Initial draft of a process daemon in :mod:`ipython1.daemon`. This has not |
|
74 | * Initial draft of a process daemon in :mod:`ipython1.daemon`. This has not | |
73 | been merged into IPython and is still in `ipython1-dev`. |
|
75 | been merged into IPython and is still in `ipython1-dev`. | |
74 | * The ``TaskController`` now has methods for getting the queue status. |
|
76 | * The ``TaskController`` now has methods for getting the queue status. | |
75 | * The ``TaskResult`` objects not have information about how long the task |
|
77 | * The ``TaskResult`` objects not have information about how long the task | |
76 | took to run. |
|
78 | took to run. | |
77 | * We are attaching additional attributes to exceptions ``(_ipython_*)`` that |
|
79 | * We are attaching additional attributes to exceptions ``(_ipython_*)`` that | |
78 | we use to carry additional info around. |
|
80 | we use to carry additional info around. | |
79 | * New top-level module :mod:`asyncclient` that has asynchronous versions (that |
|
81 | * New top-level module :mod:`asyncclient` that has asynchronous versions (that | |
80 | return deferreds) of the client classes. This is designed to users who want |
|
82 | return deferreds) of the client classes. This is designed to users who want | |
81 | to run their own Twisted reactor |
|
83 | to run their own Twisted reactor | |
82 | * All the clients in :mod:`client` are now based on Twisted. This is done by |
|
84 | * All the clients in :mod:`client` are now based on Twisted. This is done by | |
83 | running the Twisted reactor in a separate thread and using the |
|
85 | running the Twisted reactor in a separate thread and using the | |
84 | :func:`blockingCallFromThread` function that is in recent versions of Twisted. |
|
86 | :func:`blockingCallFromThread` function that is in recent versions of Twisted. | |
85 | * Functions can now be pushed/pulled to/from engines using |
|
87 | * Functions can now be pushed/pulled to/from engines using | |
86 | :meth:`MultiEngineClient.push_function` and :meth:`MultiEngineClient.pull_function`. |
|
88 | :meth:`MultiEngineClient.push_function` and :meth:`MultiEngineClient.pull_function`. | |
87 | * Gather/scatter are now implemented in the client to reduce the work load |
|
89 | * Gather/scatter are now implemented in the client to reduce the work load | |
88 | of the controller and improve performance. |
|
90 | of the controller and improve performance. | |
89 | * Complete rewrite of the IPython docuementation. All of the documentation |
|
91 | * Complete rewrite of the IPython docuementation. All of the documentation | |
90 | from the IPython website has been moved into docs/source as restructured |
|
92 | from the IPython website has been moved into docs/source as restructured | |
91 | text documents. PDF and HTML documentation are being generated using |
|
93 | text documents. PDF and HTML documentation are being generated using | |
92 | Sphinx. |
|
94 | Sphinx. | |
93 | * New developer oriented documentation: development guidelines and roadmap. |
|
95 | * New developer oriented documentation: development guidelines and roadmap. | |
94 | * Traditional ``ChangeLog`` has been changed to a more useful ``changes.txt`` file |
|
96 | * Traditional ``ChangeLog`` has been changed to a more useful ``changes.txt`` file | |
95 | that is organized by release and is meant to provide something more relevant |
|
97 | that is organized by release and is meant to provide something more relevant | |
96 | for users. |
|
98 | for users. | |
97 |
|
99 | |||
98 | Bug fixes |
|
100 | Bug fixes | |
99 | ......... |
|
101 | ......... | |
100 |
|
102 | |||
101 | * Created a proper ``MANIFEST.in`` file to create source distributions. |
|
103 | * Created a proper ``MANIFEST.in`` file to create source distributions. | |
102 | * Fixed a bug in the ``MultiEngine`` interface. Previously, multi-engine |
|
104 | * Fixed a bug in the ``MultiEngine`` interface. Previously, multi-engine | |
103 | actions were being collected with a :class:`DeferredList` with |
|
105 | actions were being collected with a :class:`DeferredList` with | |
104 | ``fireononeerrback=1``. This meant that methods were returning |
|
106 | ``fireononeerrback=1``. This meant that methods were returning | |
105 | before all engines had given their results. This was causing extremely odd |
|
107 | before all engines had given their results. This was causing extremely odd | |
106 | bugs in certain cases. To fix this problem, we have 1) set |
|
108 | bugs in certain cases. To fix this problem, we have 1) set | |
107 | ``fireononeerrback=0`` to make sure all results (or exceptions) are in |
|
109 | ``fireononeerrback=0`` to make sure all results (or exceptions) are in | |
108 | before returning and 2) introduced a :exc:`CompositeError` exception |
|
110 | before returning and 2) introduced a :exc:`CompositeError` exception | |
109 | that wraps all of the engine exceptions. This is a huge change as it means |
|
111 | that wraps all of the engine exceptions. This is a huge change as it means | |
110 | that users will have to catch :exc:`CompositeError` rather than the actual |
|
112 | that users will have to catch :exc:`CompositeError` rather than the actual | |
111 | exception. |
|
113 | exception. | |
112 |
|
114 | |||
113 | Backwards incompatible changes |
|
115 | Backwards incompatible changes | |
114 | .............................. |
|
116 | .............................. | |
115 |
|
117 | |||
116 | * All names have been renamed to conform to the lowercase_with_underscore |
|
118 | * All names have been renamed to conform to the lowercase_with_underscore | |
117 | convention. This will require users to change references to all names like |
|
119 | convention. This will require users to change references to all names like | |
118 | ``queueStatus`` to ``queue_status``. |
|
120 | ``queueStatus`` to ``queue_status``. | |
119 | * Previously, methods like :meth:`MultiEngineClient.push` and |
|
121 | * Previously, methods like :meth:`MultiEngineClient.push` and | |
120 | :meth:`MultiEngineClient.push` used ``*args`` and ``**kwargs``. This was |
|
122 | :meth:`MultiEngineClient.push` used ``*args`` and ``**kwargs``. This was | |
121 | becoming a problem as we weren't able to introduce new keyword arguments into |
|
123 | becoming a problem as we weren't able to introduce new keyword arguments into | |
122 | the API. Now these methods simple take a dict or sequence. This has also allowed |
|
124 | the API. Now these methods simple take a dict or sequence. This has also allowed | |
123 | us to get rid of the ``*All`` methods like :meth:`pushAll` and :meth:`pullAll`. |
|
125 | us to get rid of the ``*All`` methods like :meth:`pushAll` and :meth:`pullAll`. | |
124 | These things are now handled with the ``targets`` keyword argument that defaults |
|
126 | These things are now handled with the ``targets`` keyword argument that defaults | |
125 | to ``'all'``. |
|
127 | to ``'all'``. | |
126 | * The :attr:`MultiEngineClient.magicTargets` has been renamed to |
|
128 | * The :attr:`MultiEngineClient.magicTargets` has been renamed to | |
127 | :attr:`MultiEngineClient.targets`. |
|
129 | :attr:`MultiEngineClient.targets`. | |
128 | * All methods in the MultiEngine interface now accept the optional keyword argument |
|
130 | * All methods in the MultiEngine interface now accept the optional keyword argument | |
129 | ``block``. |
|
131 | ``block``. | |
130 | * Renamed :class:`RemoteController` to :class:`MultiEngineClient` and |
|
132 | * Renamed :class:`RemoteController` to :class:`MultiEngineClient` and | |
131 | :class:`TaskController` to :class:`TaskClient`. |
|
133 | :class:`TaskController` to :class:`TaskClient`. | |
132 | * Renamed the top-level module from :mod:`api` to :mod:`client`. |
|
134 | * Renamed the top-level module from :mod:`api` to :mod:`client`. | |
133 | * Most methods in the multiengine interface now raise a :exc:`CompositeError` exception |
|
135 | * Most methods in the multiengine interface now raise a :exc:`CompositeError` exception | |
134 | that wraps the user's exceptions, rather than just raising the raw user's exception. |
|
136 | that wraps the user's exceptions, rather than just raising the raw user's exception. | |
135 | * Changed the ``setupNS`` and ``resultNames`` in the ``Task`` class to ``push`` |
|
137 | * Changed the ``setupNS`` and ``resultNames`` in the ``Task`` class to ``push`` | |
136 | and ``pull``. |
|
138 | and ``pull``. | |
137 |
|
139 | |||
138 | Release 0.8.4 |
|
140 | Release 0.8.4 | |
139 | ============= |
|
141 | ============= | |
140 |
|
142 | |||
141 | Someone needs to describe what went into 0.8.4. |
|
143 | Someone needs to describe what went into 0.8.4. | |
142 |
|
144 | |||
143 | Release 0.8.2 |
|
145 | Release 0.8.2 | |
144 | ============= |
|
146 | ============= | |
145 |
|
147 | |||
146 | * %pushd/%popd behave differently; now "pushd /foo" pushes CURRENT directory |
|
148 | * %pushd/%popd behave differently; now "pushd /foo" pushes CURRENT directory | |
147 | and jumps to /foo. The current behaviour is closer to the documented |
|
149 | and jumps to /foo. The current behaviour is closer to the documented | |
148 | behaviour, and should not trip anyone. |
|
150 | behaviour, and should not trip anyone. | |
149 |
|
151 | |||
150 | Release 0.8.3 |
|
152 | Release 0.8.3 | |
151 | ============= |
|
153 | ============= | |
152 |
|
154 | |||
153 | * pydb is now disabled by default (due to %run -d problems). You can enable |
|
155 | * pydb is now disabled by default (due to %run -d problems). You can enable | |
154 | it by passing -pydb command line argument to IPython. Note that setting |
|
156 | it by passing -pydb command line argument to IPython. Note that setting | |
155 | it in config file won't work. |
|
157 | it in config file won't work. | |
156 |
|
158 | |||
157 | Older releases |
|
159 | Older releases | |
158 | ============== |
|
160 | ============== | |
159 |
|
161 | |||
160 | Changes in earlier releases of IPython are described in the older file ``ChangeLog``. |
|
162 | Changes in earlier releases of IPython are described in the older file ``ChangeLog``. | |
161 | Please refer to this document for details. |
|
163 | Please refer to this document for details. | |
162 |
|
164 |
General Comments 0
You need to be logged in to leave comments.
Login now