upstream/ipython Files · docs/examples/parallel/demo/noncopying.py

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE)...

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE) I have setup ipcluster_config.py to start with LSF: ``` c.IPClusterStart.controller_launcher_class = 'LSF' c.IPClusterStart.engine_launcher_class = 'LSF' ``` But the ipcluster command fails to start the engines: ``` ipcluster start --profile=lsf -n 10 ``` The problem is fixed if I add quotes to the launch command string ```cmd``` in ```launcher.py```. ``` diff --git a/IPython/parallel/apps/launcher.py b/IPython/parallel/apps/launcher.py index e752d2a..6035303 100644 --- a/IPython/parallel/apps/launcher.py +++ b/IPython/parallel/apps/launcher.py @@ -73,7 +73,7 @@ WINDOWS = os.name == 'nt' # Paths to the kernel apps #----------------------------------------------------------------------------- -cmd = "from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()" +cmd = "\"from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()\"" ipcluster_cmd_argv = [sys.executable, "-c", cmd % "ipclusterapp"] ```

Thomas Kluyver - - Load All Authors

File last commit:

r6455:15863dc1


                r7995:061632b4

Download file

             noncopying.py
        
                    44 lines
            
             | 866 B
            
                | text/x-python
            
             |
                PythonLexer
            
             / docs / examples / parallel / demo / noncopying.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        MinRK
    
added py4science demos as examples + NetworkX DAG dependencies

              r3564
            
      """non-copying sends"""

      import zmq

      import numpy

      n = 10

      iface = 'inproc://pub'

      ctx = zmq.Context()

      p = ctx.socket(zmq.PUB)

      p.bind(iface)

      # connect 2 subs

      s1 = ctx.socket(zmq.SUB)

      s1.connect(iface)

      s1.setsockopt(zmq.SUBSCRIBE, '')

      s2 = ctx.socket(zmq.SUB)

      s2.connect(iface)

      s2.setsockopt(zmq.SUBSCRIBE, '')

      A = numpy.random.random((1024,1024))

      # send

      p.send(A, copy=False)

      # recv on 1 non-copy

      msg1 = s1.recv(copy=False)

      B1 = numpy.frombuffer(msg1.buffer, dtype=A.dtype).reshape(A.shape)

      # recv on 2 copy

      msg2 = s2.recv(copy=False)

      B2 = numpy.frombuffer(buffer(msg2.bytes), dtype=A.dtype).reshape(A.shape)

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print((B1==B2).all())

      print((B1==A).all())

        MinRK
    
added py4science demos as examples + NetworkX DAG dependencies

              r3564
            
      A[0][0] += 10

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print("~")

        MinRK
    
added py4science demos as examples + NetworkX DAG dependencies

              r3564
            
      # after changing A in-place, B1 changes too, proving non-copying sends

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print((B1==A).all())

        MinRK
    
added py4science demos as examples + NetworkX DAG dependencies

              r3564
            
      # but B2 is fixed, since it called the msg.bytes attr, which copies

        Thomas Kluyver
    
Update print syntax in parallel examples.

              r6455
            
      print((B1==B2).all())

        MinRK
    
added py4science demos as examples + NetworkX DAG dependencies

              r3564

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

MinRK added py4science demos as examples + NetworkX DAG dependencies	r3564	"""non-copying sends"""
		import zmq
		import numpy

		n = 10
		iface = 'inproc://pub'

		ctx = zmq.Context()

		p = ctx.socket(zmq.PUB)
		p.bind(iface)

		# connect 2 subs
		s1 = ctx.socket(zmq.SUB)
		s1.connect(iface)
		s1.setsockopt(zmq.SUBSCRIBE, '')

		s2 = ctx.socket(zmq.SUB)
		s2.connect(iface)
		s2.setsockopt(zmq.SUBSCRIBE, '')

		A = numpy.random.random((1024,1024))

		# send
		p.send(A, copy=False)
		# recv on 1 non-copy
		msg1 = s1.recv(copy=False)
		B1 = numpy.frombuffer(msg1.buffer, dtype=A.dtype).reshape(A.shape)
		# recv on 2 copy
		msg2 = s2.recv(copy=False)
		B2 = numpy.frombuffer(buffer(msg2.bytes), dtype=A.dtype).reshape(A.shape)

Thomas Kluyver Update print syntax in parallel examples.	r6455	print((B1==B2).all())
		print((B1==A).all())
MinRK added py4science demos as examples + NetworkX DAG dependencies	r3564	A[0][0] += 10
Thomas Kluyver Update print syntax in parallel examples.	r6455	print("~")
MinRK added py4science demos as examples + NetworkX DAG dependencies	r3564	# after changing A in-place, B1 changes too, proving non-copying sends
Thomas Kluyver Update print syntax in parallel examples.	r6455	print((B1==A).all())
MinRK added py4science demos as examples + NetworkX DAG dependencies	r3564	# but B2 is fixed, since it called the msg.bytes attr, which copies
Thomas Kluyver Update print syntax in parallel examples.	r6455	print((B1==B2).all())
MinRK added py4science demos as examples + NetworkX DAG dependencies	r3564