upstream/ipython Files · IPython/parallel/apps/win32support.py

Merge pull request from minrk/ar_sugar...

Merge pull request from minrk/ar_sugar Add sugar methods/properties to AsyncResult that are generically useful: * `ar.wall_time` = received - submitted * `ar.serial_time` = sum of serial computation time * `ar.elapsed` = time since submission (wall_time if done) * `ar.progress` = (int) number of sub-tasks that have completed * `len(ar)` = # of tasks * `ar.wait_interactive()`: prints progress These are simple methods derived from the metadata/timestamps already created. But I've been persuaded by @wesm's practice of including simple methods that do useful (and/or cool) things. This also required/revealed some minor fixes/cleanup to clear_output in some cases: * dedent base `core.displaypub.clear_output`, so it's actually defined in the class * clear_output publishes `'\r\b'`, so it will clear terminal-like frontends that don't understand full clear_output behavior. * `core.display.clear_output()` still works, even outside an IPython session. Added a new notebook that shows how to use these new methods and how to do simple animations/progress bars using `clear_output()`. Added `Client.spin_thread(interval)` / `stop_spin_thread()` for running spin in a background thread, to keep zmq queue clear. This can be used to ensure that timing information is as accurate as possible (at the cost of having a background thread active).

Thomas Spura - - Load All Authors

File last commit:

r4574:a8c54759


                r6485:e0b43119

Download file

             win32support.py
        
                    72 lines
            
             | 2.1 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / IPython / parallel / apps / win32support.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        MinRK
    
forward subprocess IO over zmq on Windows...

              r3771
            
      """Utility for forwarding file read events over a zmq socket.

        MinRK
    
code updates per review of PR #454

              r4021
            
      This is necessary because select on Windows only supports sockets, not FDs.

        MinRK
    
update recently changed modules with Authors in docstring

              r4018
            
      Authors:

      * MinRK

      """

        MinRK
    
forward subprocess IO over zmq on Windows...

              r3771
            
      #-----------------------------------------------------------------------------

      #  Copyright (C) 2011  The IPython Development Team

      #

      #  Distributed under the terms of the BSD License.  The full license is in

      #  the file COPYING, distributed as part of this software.

      #-----------------------------------------------------------------------------

      #-----------------------------------------------------------------------------

      # Imports

      #-----------------------------------------------------------------------------

      import uuid

      import zmq

      from threading import Thread

      #-----------------------------------------------------------------------------

      # Code

      #-----------------------------------------------------------------------------

      class ForwarderThread(Thread):

          def __init__(self, sock, fd):

              Thread.__init__(self)

              self.daemon=True

              self.sock = sock

              self.fd = fd

          def run(self):

        MinRK
    
code updates per review of PR #454

              r4021
            
              """Loop through lines in self.fd, and send them over self.sock."""

        MinRK
    
forward subprocess IO over zmq on Windows...

              r3771
            
              line = self.fd.readline()

              # allow for files opened in unicode mode

              if isinstance(line, unicode):

                  send = self.sock.send_unicode

              else:

                  send = self.sock.send

              while line:

                  send(line)

                  line = self.fd.readline()

              # line == '' means EOF

              self.fd.close()

              self.sock.close()

      def forward_read_events(fd, context=None):

        MinRK
    
code updates per review of PR #454

              r4021
            
          """Forward read events from an FD over a socket.

        MinRK
    
forward subprocess IO over zmq on Windows...

              r3771
            
          This method wraps a file in a socket pair, so it can

          be polled for read events by select (specifically zmq.eventloop.ioloop)

          """

          if context is None:

              context = zmq.Context.instance()

          push = context.socket(zmq.PUSH)

          push.setsockopt(zmq.LINGER, -1)

          pull = context.socket(zmq.PULL)

          addr='inproc://%s'%uuid.uuid4()

          push.bind(addr)

          pull.connect(addr)

          forwarder = ForwarderThread(push, fd)

          forwarder.start()

          return pull

        MinRK
    
code updates per review of PR #454

              r4021
            
      __all__ = ['forward_read_events']

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

MinRK forward subprocess IO over zmq on Windows...	r3771	"""Utility for forwarding file read events over a zmq socket.

MinRK code updates per review of PR #454	r4021	This is necessary because select on Windows only supports sockets, not FDs.
MinRK update recently changed modules with Authors in docstring	r4018
		Authors:

		* MinRK

		"""
MinRK forward subprocess IO over zmq on Windows...	r3771
		#-----------------------------------------------------------------------------
		# Copyright (C) 2011 The IPython Development Team
		#
		# Distributed under the terms of the BSD License. The full license is in
		# the file COPYING, distributed as part of this software.
		#-----------------------------------------------------------------------------

		#-----------------------------------------------------------------------------
		# Imports
		#-----------------------------------------------------------------------------

		import uuid
		import zmq

		from threading import Thread

		#-----------------------------------------------------------------------------
		# Code
		#-----------------------------------------------------------------------------

		class ForwarderThread(Thread):
		def __init__(self, sock, fd):
		Thread.__init__(self)
		self.daemon=True
		self.sock = sock
		self.fd = fd

		def run(self):
MinRK code updates per review of PR #454	r4021	"""Loop through lines in self.fd, and send them over self.sock."""
MinRK forward subprocess IO over zmq on Windows...	r3771	line = self.fd.readline()
		# allow for files opened in unicode mode
		if isinstance(line, unicode):
		send = self.sock.send_unicode
		else:
		send = self.sock.send
		while line:
		send(line)
		line = self.fd.readline()
		# line == '' means EOF
		self.fd.close()
		self.sock.close()

		def forward_read_events(fd, context=None):
MinRK code updates per review of PR #454	r4021	"""Forward read events from an FD over a socket.
MinRK forward subprocess IO over zmq on Windows...	r3771
		This method wraps a file in a socket pair, so it can
		be polled for read events by select (specifically zmq.eventloop.ioloop)
		"""
		if context is None:
		context = zmq.Context.instance()
		push = context.socket(zmq.PUSH)
		push.setsockopt(zmq.LINGER, -1)
		pull = context.socket(zmq.PULL)
		addr='inproc://%s'%uuid.uuid4()
		push.bind(addr)
		pull.connect(addr)
		forwarder = ForwarderThread(push, fd)
		forwarder.start()
		return pull


MinRK code updates per review of PR #454	r4021	__all__ = ['forward_read_events']