upstream/ipython Files · docs/examples/parallel/taskmap.py

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE)...

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE) I have setup ipcluster_config.py to start with LSF: ``` c.IPClusterStart.controller_launcher_class = 'LSF' c.IPClusterStart.engine_launcher_class = 'LSF' ``` But the ipcluster command fails to start the engines: ``` ipcluster start --profile=lsf -n 10 ``` The problem is fixed if I add quotes to the launch command string ```cmd``` in ```launcher.py```. ``` diff --git a/IPython/parallel/apps/launcher.py b/IPython/parallel/apps/launcher.py index e752d2a..6035303 100644 --- a/IPython/parallel/apps/launcher.py +++ b/IPython/parallel/apps/launcher.py @@ -73,7 +73,7 @@ WINDOWS = os.name == 'nt' # Paths to the kernel apps #----------------------------------------------------------------------------- -cmd = "from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()" +cmd = "\"from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()\"" ipcluster_cmd_argv = [sys.executable, "-c", cmd % "ipclusterapp"] ```

Thomas Kluyver - - Load All Authors

File last commit:

r6455:15863dc1


                r7995:061632b4

Download file

             taskmap.py
        
                    36 lines
            
             | 636 B
            
                | text/x-python
            
             |
                PythonLexer
            
             / docs / examples / parallel / taskmap.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
      # <nbformat>2</nbformat>

      # <markdowncell>

      # # Load balanced map and parallel function decorator

      # <codecell>

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      from __future__ import print_function

        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
        MinRK
    
translate last remaining old parallel examples

              r3690
            
      from IPython.parallel import Client

        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
      # <codecell>

        MinRK
    
translate last remaining old parallel examples

              r3690
            
      rc = Client()

      v = rc.load_balanced_view()

        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
      # <codecell>

        MinRK
    
translate last remaining old parallel examples

              r3690
            
      result = v.map(lambda x: 2*x, range(10))

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print("Simple, default map: ", list(result))

        MinRK
    
translate last remaining old parallel examples

              r3690
            
        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
      # <codecell>

        MinRK
    
translate last remaining old parallel examples

              r3690
            
      ar = v.map_async(lambda x: 2*x, range(10))

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print("Submitted tasks, got ids: ", ar.msg_ids)

        MinRK
    
translate last remaining old parallel examples

              r3690
            
      result = ar.get()

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print("Using a mapper: ", result)

        MinRK
    
translate last remaining old parallel examples

              r3690
            
        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581
            
      # <codecell>

        MinRK
    
translate last remaining old parallel examples

              r3690
            
      @v.parallel(block=True)

      def f(x): return 2*x

      result = f.map(range(10))

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print("Using a parallel function: ", result)

        Brian E. Granger
    
Reworking parallel examples as notebooks.

              r4581

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

Brian E. Granger Reworking parallel examples as notebooks.	r4581	# <nbformat>2</nbformat>

		# <markdowncell>

		# # Load balanced map and parallel function decorator

		# <codecell>
Thomas Kluyver Update print syntax in parallel examples.	r6455	from __future__ import print_function
Brian E. Granger Reworking parallel examples as notebooks.	r4581
MinRK translate last remaining old parallel examples	r3690	from IPython.parallel import Client

Brian E. Granger Reworking parallel examples as notebooks.	r4581	# <codecell>

MinRK translate last remaining old parallel examples	r3690	rc = Client()
		v = rc.load_balanced_view()

Brian E. Granger Reworking parallel examples as notebooks.	r4581	# <codecell>

MinRK translate last remaining old parallel examples	r3690	result = v.map(lambda x: 2*x, range(10))
Thomas Kluyver Update print syntax in parallel examples.	r6455	print("Simple, default map: ", list(result))
MinRK translate last remaining old parallel examples	r3690
Brian E. Granger Reworking parallel examples as notebooks.	r4581	# <codecell>

MinRK translate last remaining old parallel examples	r3690	ar = v.map_async(lambda x: 2*x, range(10))
Thomas Kluyver Update print syntax in parallel examples.	r6455	print("Submitted tasks, got ids: ", ar.msg_ids)
MinRK translate last remaining old parallel examples	r3690	result = ar.get()
Thomas Kluyver Update print syntax in parallel examples.	r6455	print("Using a mapper: ", result)
MinRK translate last remaining old parallel examples	r3690
Brian E. Granger Reworking parallel examples as notebooks.	r4581	# <codecell>

MinRK translate last remaining old parallel examples	r3690	@v.parallel(block=True)
		def f(x): return 2*x

		result = f.map(range(10))
Thomas Kluyver Update print syntax in parallel examples.	r6455	print("Using a parallel function: ", result)
Brian E. Granger Reworking parallel examples as notebooks.	r4581