Show More
@@ -54,7 +54,8 b' The :command:`ipcluster` command provides a simple way of starting a controller ' | |||||
54 | with most MPI [MPI]_ implementations |
|
54 | with most MPI [MPI]_ implementations | |
55 | 3. When engines are started using the PBS [PBS]_ batch system. |
|
55 | 3. When engines are started using the PBS [PBS]_ batch system. | |
56 | 4. When engines are started using the SGE [SGE]_ batch system. |
|
56 | 4. When engines are started using the SGE [SGE]_ batch system. | |
57 | 5. When the controller is started on localhost and the engines are started on |
|
57 | 5. When engines are started using the LSF [LSF]_ batch system. | |
|
58 | 6. When the controller is started on localhost and the engines are started on | |||
58 | remote nodes using :command:`ssh`. |
|
59 | remote nodes using :command:`ssh`. | |
59 |
|
60 | |||
60 | .. note:: |
|
61 | .. note:: | |
@@ -127,44 +128,21 b' More details on using MPI with IPython can be found :ref:`here <parallelmpi>`.' | |||||
127 | Using :command:`ipcluster` in PBS mode |
|
128 | Using :command:`ipcluster` in PBS mode | |
128 | -------------------------------------- |
|
129 | -------------------------------------- | |
129 |
|
130 | |||
130 | The PBS mode uses the Portable Batch System [PBS]_ to start the engines. To use this mode, you first need to create a PBS script template that will be used to start the engines. Here is a sample PBS script template: |
|
131 | The PBS mode uses the Portable Batch System [PBS]_ to start the engines. | |
131 |
|
132 | |||
132 | .. sourcecode:: bash |
|
133 | To start an ipcluster using the Portable Batch System:: | |
133 |
|
||||
134 | #PBS -N ipython |
|
|||
135 | #PBS -j oe |
|
|||
136 | #PBS -l walltime=00:10:00 |
|
|||
137 | #PBS -l nodes=${n/4}:ppn=4 |
|
|||
138 | #PBS -q parallel |
|
|||
139 |
|
||||
140 | cd $$PBS_O_WORKDIR |
|
|||
141 | export PATH=$$HOME/usr/local/bin |
|
|||
142 | export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages |
|
|||
143 | /usr/local/bin/mpiexec -n ${n} ipengine --logfile=$$PBS_O_WORKDIR/ipengine |
|
|||
144 |
|
||||
145 | There are a few important points about this template: |
|
|||
146 |
|
||||
147 | 1. This template will be rendered at runtime using IPython's :mod:`Itpl` |
|
|||
148 | template engine. |
|
|||
149 |
|
134 | |||
150 | 2. Instead of putting in the actual number of engines, use the notation |
|
135 | $ ipcluster pbs -n 12 | |
151 | ``${n}`` to indicate the number of engines to be started. You can also uses |
|
|||
152 | expressions like ``${n/4}`` in the template to indicate the number of |
|
|||
153 | nodes. |
|
|||
154 |
|
136 | |||
155 | 3. Because ``$`` is a special character used by the template engine, you must |
|
137 | The above command will launch a PBS job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: | |
156 | escape any ``$`` by using ``$$``. This is important when referring to |
|
|||
157 | environment variables in the template. |
|
|||
158 |
|
138 | |||
159 | 4. Any options to :command:`ipengine` should be given in the batch script |
|
139 | $ ipcluster pbs -n 12 -q hpcqueue | |
160 | template. |
|
|||
161 |
|
140 | |||
162 | 5. Depending on the configuration of you system, you may have to set |
|
141 | By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: | |
163 | environment variables in the script template. |
|
|||
164 |
|
142 | |||
165 | Once you have created such a script, save it with a name like :file:`pbs.template`. Now you are ready to start your job:: |
|
143 | $ ipcluster pbs -n 12 -q hpcqueue -s mypbscript.sh | |
166 |
|
144 | |||
167 | $ ipcluster pbs -n 128 --pbs-script=pbs.template |
|
145 | NOTE: ipcluster relies on using PBS job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#PBS -t 1-N) to your script. | |
168 |
|
146 | |||
169 | Additional command line options for this mode can be found by doing:: |
|
147 | Additional command line options for this mode can be found by doing:: | |
170 |
|
148 | |||
@@ -173,46 +151,48 b' Additional command line options for this mode can be found by doing::' | |||||
173 | Using :command:`ipcluster` in SGE mode |
|
151 | Using :command:`ipcluster` in SGE mode | |
174 | -------------------------------------- |
|
152 | -------------------------------------- | |
175 |
|
153 | |||
176 | The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines. To use this mode, you first need to create a SGE script template that will be used to start the engines. Here is a sample SGE script template: |
|
154 | The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines. | |
177 |
|
155 | |||
178 | .. sourcecode:: bash |
|
156 | To start an ipcluster using Sun Grid Engine:: | |
179 |
|
157 | |||
180 | #!/bin/bash |
|
158 | $ ipcluster sge -n 12 | |
181 | #$ -V |
|
|||
182 | #$ -m n |
|
|||
183 | #$ -N ipengine-${eid} |
|
|||
184 | #$ -r y |
|
|||
185 | #$ -q sub |
|
|||
186 | #$ -S /bin/bash |
|
|||
187 |
|
159 | |||
188 | cd $$HOME/sge |
|
160 | The above command will launch an SGE job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: | |
189 | ipengine --logfile=ipengine${eid} |
|
|||
190 |
|
161 | |||
191 | There are a few important points about this template: |
|
162 | $ ipcluster sge -n 12 -q hpcqueue | |
192 |
|
163 | |||
193 | 1. This template will be rendered at runtime using IPython's :mod:`Itpl` |
|
164 | By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: | |
194 | template engine. |
|
|||
195 |
|
165 | |||
196 | 2. Instead of putting in the actual id of engines, use the notation |
|
166 | $ ipcluster sge -n 12 -q hpcqueue -s mysgescript.sh | |
197 | ``${eid}`` to indicate where engine id should be inserted. |
|
|||
198 |
|
167 | |||
199 | 3. Because ``$`` is a special character used by the template engine, you must |
|
168 | NOTE: ipcluster relies on using SGE job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#$ -t 1-N) to your script. | |
200 | escape any ``$`` by using ``$$``. This is important when referring to |
|
|||
201 | environment variables in the template. |
|
|||
202 |
|
169 | |||
203 | 4. Any options to :command:`ipengine` should be given in the batch script |
|
170 | Additional command line options for this mode can be found by doing:: | |
204 | template. |
|
|||
205 |
|
171 | |||
206 | 5. Depending on the configuration of you system, you may have to set |
|
172 | $ ipcluster sge -h | |
207 | environment variables in the script template. |
|
|||
208 |
|
173 | |||
209 | Once you have created such a script, save it with a name like :file:`sge.template`. Now you are ready to start your job:: |
|
174 | Using :command:`ipcluster` in LSF mode | |
|
175 | -------------------------------------- | |||
210 |
|
176 | |||
211 | $ ipcluster sge -n 12 --sge-script=sge.template |
|
177 | The LSF mode uses the Load Sharing Facility [LSF]_ to start the engines. | |
|
178 | ||||
|
179 | To start an ipcluster using the Load Sharing Facility:: | |||
|
180 | ||||
|
181 | $ ipcluster lsf -n 12 | |||
|
182 | ||||
|
183 | The above command will launch an LSF job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: | |||
|
184 | ||||
|
185 | $ ipcluster lsf -n 12 -q hpcqueue | |||
|
186 | ||||
|
187 | By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: | |||
|
188 | ||||
|
189 | $ ipcluster lsf -n 12 -q hpcqueue -s mysgescript.sh | |||
|
190 | ||||
|
191 | NOTE: ipcluster relies on using LSF job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#BSUB -J ipengine[1-N]) to your script. | |||
212 |
|
192 | |||
213 | Additional command line options for this mode can be found by doing:: |
|
193 | Additional command line options for this mode can be found by doing:: | |
214 |
|
194 | |||
215 |
$ ipcluster |
|
195 | $ ipcluster lsf -h | |
216 |
|
196 | |||
217 | Using :command:`ipcluster` in SSH mode |
|
197 | Using :command:`ipcluster` in SSH mode | |
218 | -------------------------------------- |
|
198 | -------------------------------------- | |
@@ -394,4 +374,5 b' the log files to us will often help us to debug any problems.' | |||||
394 |
|
374 | |||
395 | .. [PBS] Portable Batch System. http://www.openpbs.org/ |
|
375 | .. [PBS] Portable Batch System. http://www.openpbs.org/ | |
396 | .. [SGE] Sun Grid Engine. http://www.sun.com/software/sge/ |
|
376 | .. [SGE] Sun Grid Engine. http://www.sun.com/software/sge/ | |
|
377 | .. [LSF] Load Sharing Facility. http://www.platform.com/ | |||
397 | .. [SSH] SSH-Agent http://en.wikipedia.org/wiki/Ssh-agent |
|
378 | .. [SSH] SSH-Agent http://en.wikipedia.org/wiki/Ssh-agent |
General Comments 0
You need to be logged in to leave comments.
Login now