upstream/ipython Files · docs/examples/parallel/demo/noncopying.py

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE)...

Backport PR : ipcluster broken with any batch (PBS/LSF/SGE) I have setup ipcluster_config.py to start with LSF: ``` c.IPClusterStart.controller_launcher_class = 'LSF' c.IPClusterStart.engine_launcher_class = 'LSF' ``` But the ipcluster command fails to start the engines: ``` ipcluster start --profile=lsf -n 10 ``` The problem is fixed if I add quotes to the launch command string ```cmd``` in ```launcher.py```. ``` diff --git a/IPython/parallel/apps/launcher.py b/IPython/parallel/apps/launcher.py index e752d2a..6035303 100644 --- a/IPython/parallel/apps/launcher.py +++ b/IPython/parallel/apps/launcher.py @@ -73,7 +73,7 @@ WINDOWS = os.name == 'nt' # Paths to the kernel apps #----------------------------------------------------------------------------- -cmd = "from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()" +cmd = "\"from IPython.parallel.apps.%s import launch_new_instance; launch_new_instance()\"" ipcluster_cmd_argv = [sys.executable, "-c", cmd % "ipclusterapp"] ```

Thomas Kluyver - - Load All Authors

File last commit:

r6455:15863dc1


                r7995:061632b4

Download file

             noncopying.py
        
                    44 lines
            
             | 866 B
            
                | text/x-python
            
             |
                PythonLexer
            
             / docs / examples / parallel / demo / noncopying.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      """non-copying sends"""

      import zmq

      import numpy

      n = 10

      iface = 'inproc://pub'

      ctx = zmq.Context()

      p = ctx.socket(zmq.PUB)

      p.bind(iface)

      # connect 2 subs

      s1 = ctx.socket(zmq.SUB)

      s1.connect(iface)

      s1.setsockopt(zmq.SUBSCRIBE, '')

      s2 = ctx.socket(zmq.SUB)

      s2.connect(iface)

      s2.setsockopt(zmq.SUBSCRIBE, '')

      A = numpy.random.random((1024,1024))

      # send

      p.send(A, copy=False)

      # recv on 1 non-copy

      msg1 = s1.recv(copy=False)

      B1 = numpy.frombuffer(msg1.buffer, dtype=A.dtype).reshape(A.shape)

      # recv on 2 copy

      msg2 = s2.recv(copy=False)

      B2 = numpy.frombuffer(buffer(msg2.bytes), dtype=A.dtype).reshape(A.shape)

      print((B1==B2).all())

      print((B1==A).all())

      A[0][0] += 10

      print("~")

      # after changing A in-place, B1 changes too, proving non-copying sends

      print((B1==A).all())

      # but B2 is fixed, since it called the msg.bytes attr, which copies

      print((B1==B2).all())

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				"""non-copying sends"""
				import zmq
				import numpy

				n = 10
				iface = 'inproc://pub'

				ctx = zmq.Context()

				p = ctx.socket(zmq.PUB)
				p.bind(iface)

				# connect 2 subs
				s1 = ctx.socket(zmq.SUB)
				s1.connect(iface)
				s1.setsockopt(zmq.SUBSCRIBE, '')

				s2 = ctx.socket(zmq.SUB)
				s2.connect(iface)
				s2.setsockopt(zmq.SUBSCRIBE, '')

				A = numpy.random.random((1024,1024))

				# send
				p.send(A, copy=False)
				# recv on 1 non-copy
				msg1 = s1.recv(copy=False)
				B1 = numpy.frombuffer(msg1.buffer, dtype=A.dtype).reshape(A.shape)
				# recv on 2 copy
				msg2 = s2.recv(copy=False)
				B2 = numpy.frombuffer(buffer(msg2.bytes), dtype=A.dtype).reshape(A.shape)

				print((B1==B2).all())
				print((B1==A).all())
				A[0][0] += 10
				print("~")
				# after changing A in-place, B1 changes too, proving non-copying sends
				print((B1==A).all())
				# but B2 is fixed, since it called the msg.bytes attr, which copies
				print((B1==B2).all())