diff --git a/docs/source/parallel/parallel_process.txt b/docs/source/parallel/parallel_process.txt index 3d35254..d5d77ca 100644 --- a/docs/source/parallel/parallel_process.txt +++ b/docs/source/parallel/parallel_process.txt @@ -54,7 +54,8 @@ The :command:`ipcluster` command provides a simple way of starting a controller with most MPI [MPI]_ implementations 3. When engines are started using the PBS [PBS]_ batch system. 4. When engines are started using the SGE [SGE]_ batch system. -5. When the controller is started on localhost and the engines are started on +5. When engines are started using the LSF [LSF]_ batch system. +6. When the controller is started on localhost and the engines are started on remote nodes using :command:`ssh`. .. note:: @@ -127,44 +128,21 @@ More details on using MPI with IPython can be found :ref:`here `. Using :command:`ipcluster` in PBS mode -------------------------------------- -The PBS mode uses the Portable Batch System [PBS]_ to start the engines. To use this mode, you first need to create a PBS script template that will be used to start the engines. Here is a sample PBS script template: +The PBS mode uses the Portable Batch System [PBS]_ to start the engines. -.. sourcecode:: bash - - #PBS -N ipython - #PBS -j oe - #PBS -l walltime=00:10:00 - #PBS -l nodes=${n/4}:ppn=4 - #PBS -q parallel - - cd $$PBS_O_WORKDIR - export PATH=$$HOME/usr/local/bin - export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages - /usr/local/bin/mpiexec -n ${n} ipengine --logfile=$$PBS_O_WORKDIR/ipengine - -There are a few important points about this template: - -1. This template will be rendered at runtime using IPython's :mod:`Itpl` - template engine. +To start an ipcluster using the Portable Batch System:: -2. Instead of putting in the actual number of engines, use the notation - ``${n}`` to indicate the number of engines to be started. You can also uses - expressions like ``${n/4}`` in the template to indicate the number of - nodes. + $ ipcluster pbs -n 12 -3. Because ``$`` is a special character used by the template engine, you must - escape any ``$`` by using ``$$``. This is important when referring to - environment variables in the template. +The above command will launch a PBS job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: -4. Any options to :command:`ipengine` should be given in the batch script - template. + $ ipcluster pbs -n 12 -q hpcqueue -5. Depending on the configuration of you system, you may have to set - environment variables in the script template. +By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: -Once you have created such a script, save it with a name like :file:`pbs.template`. Now you are ready to start your job:: + $ ipcluster pbs -n 12 -q hpcqueue -s mypbscript.sh - $ ipcluster pbs -n 128 --pbs-script=pbs.template +NOTE: ipcluster relies on using PBS job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#PBS -t 1-N) to your script. Additional command line options for this mode can be found by doing:: @@ -173,46 +151,48 @@ Additional command line options for this mode can be found by doing:: Using :command:`ipcluster` in SGE mode -------------------------------------- -The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines. To use this mode, you first need to create a SGE script template that will be used to start the engines. Here is a sample SGE script template: +The SGE mode uses the Sun Grid Engine [SGE]_ to start the engines. -.. sourcecode:: bash +To start an ipcluster using Sun Grid Engine:: - #!/bin/bash - #$ -V - #$ -m n - #$ -N ipengine-${eid} - #$ -r y - #$ -q sub - #$ -S /bin/bash + $ ipcluster sge -n 12 - cd $$HOME/sge - ipengine --logfile=ipengine${eid} +The above command will launch an SGE job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: -There are a few important points about this template: + $ ipcluster sge -n 12 -q hpcqueue -1. This template will be rendered at runtime using IPython's :mod:`Itpl` - template engine. +By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: -2. Instead of putting in the actual id of engines, use the notation - ``${eid}`` to indicate where engine id should be inserted. + $ ipcluster sge -n 12 -q hpcqueue -s mysgescript.sh -3. Because ``$`` is a special character used by the template engine, you must - escape any ``$`` by using ``$$``. This is important when referring to - environment variables in the template. +NOTE: ipcluster relies on using SGE job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#$ -t 1-N) to your script. -4. Any options to :command:`ipengine` should be given in the batch script - template. +Additional command line options for this mode can be found by doing:: -5. Depending on the configuration of you system, you may have to set - environment variables in the script template. + $ ipcluster sge -h -Once you have created such a script, save it with a name like :file:`sge.template`. Now you are ready to start your job:: +Using :command:`ipcluster` in LSF mode +-------------------------------------- - $ ipcluster sge -n 12 --sge-script=sge.template +The LSF mode uses the Load Sharing Facility [LSF]_ to start the engines. + +To start an ipcluster using the Load Sharing Facility:: + + $ ipcluster lsf -n 12 + +The above command will launch an LSF job array with 12 tasks using the default queue. If you would like to submit the job to a different queue use the -q option: + + $ ipcluster lsf -n 12 -q hpcqueue + +By default, ipcluster will generate and submit a job script to launch the engines. However, if you need to use your own job script use the -s option: + + $ ipcluster lsf -n 12 -q hpcqueue -s mysgescript.sh + +NOTE: ipcluster relies on using LSF job arrays to start the engines. If you specify your own job script without specifying the job array settings ipcluster will automatically add the job array settings (#BSUB -J ipengine[1-N]) to your script. Additional command line options for this mode can be found by doing:: - $ ipcluster sge -h + $ ipcluster lsf -h Using :command:`ipcluster` in SSH mode -------------------------------------- @@ -394,4 +374,5 @@ the log files to us will often help us to debug any problems. .. [PBS] Portable Batch System. http://www.openpbs.org/ .. [SGE] Sun Grid Engine. http://www.sun.com/software/sge/ +.. [LSF] Load Sharing Facility. http://www.platform.com/ .. [SSH] SSH-Agent http://en.wikipedia.org/wiki/Ssh-agent