##// END OF EJS Templates
update ipcluster docs w/ latest PBS/SGE/LSF changes
Justin Riley -
Show More
@@ -54,7 +54,8 b' The :command:`ipcluster` command provides a simple way of starting a controller '
54 with most MPI [MPI]_ implementations
54 with most MPI [MPI]_ implementations
55 3. When engines are started using the PBS [PBS]_ batch system.
55 3. When engines are started using the PBS [PBS]_ batch system.
56 4. When engines are started using the SGE [SGE]_ batch system.
56 4. When engines are started using the SGE [SGE]_ batch system.
57 5. When the controller is started on localhost and the engines are started on
57 5. When engines are started using the LSF [LSF]_ batch system.
58 6. When the controller is started on localhost and the engines are started on
58 remote nodes using :command:`ssh`.
59 remote nodes using :command:`ssh`.
59
60
60 .. note::
61 .. note::
@@ -127,44 +128,21 b' More details on using MPI with IPython can be found :ref:`here <parallelmpi>`.'
127 Using :command:`ipcluster` in PBS mode
128 Using :command:`ipcluster` in PBS mode
128 --------------------------------------
129 --------------------------------------
129
130
130 The PBS mode uses the Portable Batch System [PBS]_ to start the engines. To use this mode, you first need to create a PBS script template that will be used to start the engines. Here is a sample PBS script template:
131 The PBS mode uses the Portable Batch System [PBS]_ to start the engines.
131
132
132 .. sourcecode:: bash
133 To start an ipcluster using the Portable Batch System::
133
134 #PBS -N ipython
135 #PBS -j oe
136 #PBS -l walltime=00:10:00
137 #PBS -l nodes=${n/4}:ppn=4
138 #PBS -q parallel
139
140 cd $$PBS_O_WORKDIR
141 export PATH=$$HOME/usr/local/bin
142 export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages
143 /usr/local/bin/mpiexec -n ${n} ipengine --logfile=$$PBS_O_WORKDIR/ipengine
144
145 There are a few important points about this template:
146
147 1. This template will be rendered at runtime using IPython's :mod:`Itpl`
148 template engine.
149
134
150 2. Instead of putting in the actual number of engines, use the notation
135 $ ipcluster pbs -n 12
151 ``${n}`` to indicate the number of engines to be started. You can also uses
152 expressions like ``${n/4}`` in the template to indicate the number of
153 nodes.
154
136
155 3. Because ``$`` is a special character used by the template engine, you must
137 The above command will launch a PBS job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option:
156 escape any ``$`` by using ``$$``. This is important when referring to
157 environment variables in the template.
158
138
159 4. Any options to :command:`ipengine` should be given in the batch script
139 $ ipcluster pbs -n 12 -q hpcqueue
160 template.
161
140
162 5. Depending on the configuration of you system, you may have to set
141 By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option:
163 environment variables in the script template.
164
142
165 Once you have created such a script, save it with a name like :file:`pbs.template`. Now you are ready to start your job::
143 $ ipcluster pbs -n 12 -q hpcqueue -s mypbscript.sh
166
144
167 $ ipcluster pbs -n 128 --pbs-script=pbs.template
145 NOTE: ipcluster relies on using PBS job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#PBS -t 1-N) to your script.
168
146
169 Additional command line options for this mode can be found by doing::
147 Additional command line options for this mode can be found by doing::
170
148
@@ -173,46 +151,48 b' Additional command line options for this mode can be found by doing::'
173 Using :command:`ipcluster` in SGE mode
151 Using :command:`ipcluster` in SGE mode
174 --------------------------------------
152 --------------------------------------
175
153
176 The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines. To use this mode, you first need to create a SGE script template that will be used to start the engines. Here is a sample SGE script template:
154 The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines.
177
155
178 .. sourcecode:: bash
156 To start an ipcluster using Sun Grid Engine::
179
157
180 #!/bin/bash
158 $ ipcluster sge -n 12
181 #$ -V
182 #$ -m n
183 #$ -N ipengine-${eid}
184 #$ -r y
185 #$ -q sub
186 #$ -S /bin/bash
187
159
188 cd $$HOME/sge
160 The above command will launch an SGE job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option:
189 ipengine --logfile=ipengine${eid}
190
161
191 There are a few important points about this template:
162 $ ipcluster sge -n 12 -q hpcqueue
192
163
193 1. This template will be rendered at runtime using IPython's :mod:`Itpl`
164 By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option:
194 template engine.
195
165
196 2. Instead of putting in the actual id of engines, use the notation
166 $ ipcluster sge -n 12 -q hpcqueue -s mysgescript.sh
197 ``${eid}`` to indicate where engine id should be inserted.
198
167
199 3. Because ``$`` is a special character used by the template engine, you must
168 NOTE: ipcluster relies on using SGE job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#$ -t 1-N) to your script.
200 escape any ``$`` by using ``$$``. This is important when referring to
201 environment variables in the template.
202
169
203 4. Any options to :command:`ipengine` should be given in the batch script
170 Additional command line options for this mode can be found by doing::
204 template.
205
171
206 5. Depending on the configuration of you system, you may have to set
172 $ ipcluster sge -h
207 environment variables in the script template.
208
173
209 Once you have created such a script, save it with a name like :file:`sge.template`. Now you are ready to start your job::
174 Using :command:`ipcluster` in LSF mode
175 --------------------------------------
210
176
211 $ ipcluster sge -n 12 --sge-script=sge.template
177 The LSF mode uses the Load Sharing Facility [LSF]_ to start the engines.
178
179 To start an ipcluster using the Load Sharing Facility::
180
181 $ ipcluster lsf -n 12
182
183 The above command will launch an LSF job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option:
184
185 $ ipcluster lsf -n 12 -q hpcqueue
186
187 By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option:
188
189 $ ipcluster lsf -n 12 -q hpcqueue -s mysgescript.sh
190
191 NOTE: ipcluster relies on using LSF job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#BSUB -J ipengine[1-N]) to your script.
212
192
213 Additional command line options for this mode can be found by doing::
193 Additional command line options for this mode can be found by doing::
214
194
215 $ ipcluster sge -h
195 $ ipcluster lsf -h
216
196
217 Using :command:`ipcluster` in SSH mode
197 Using :command:`ipcluster` in SSH mode
218 --------------------------------------
198 --------------------------------------
@@ -394,4 +374,5 b' the log files to us will often help us to debug any problems.'
394
374
395 .. [PBS] Portable Batch System. http://www.openpbs.org/
375 .. [PBS] Portable Batch System. http://www.openpbs.org/
396 .. [SGE] Sun Grid Engine. http://www.sun.com/software/sge/
376 .. [SGE] Sun Grid Engine. http://www.sun.com/software/sge/
377 .. [LSF] Load Sharing Facility. http://www.platform.com/
397 .. [SSH] SSH-Agent http://en.wikipedia.org/wiki/Ssh-agent
378 .. [SSH] SSH-Agent http://en.wikipedia.org/wiki/Ssh-agent
General Comments 0
You need to be logged in to leave comments. Login now