upstream/ipython Files · IPython/parallel/factory.py

Merge pull request from minrk/ar_sugar...

Merge pull request from minrk/ar_sugar Add sugar methods/properties to AsyncResult that are generically useful: * `ar.wall_time` = received - submitted * `ar.serial_time` = sum of serial computation time * `ar.elapsed` = time since submission (wall_time if done) * `ar.progress` = (int) number of sub-tasks that have completed * `len(ar)` = # of tasks * `ar.wait_interactive()`: prints progress These are simple methods derived from the metadata/timestamps already created. But I've been persuaded by @wesm's practice of including simple methods that do useful (and/or cool) things. This also required/revealed some minor fixes/cleanup to clear_output in some cases: * dedent base `core.displaypub.clear_output`, so it's actually defined in the class * clear_output publishes `'\r\b'`, so it will clear terminal-like frontends that don't understand full clear_output behavior. * `core.display.clear_output()` still works, even outside an IPython session. Added a new notebook that shows how to use these new methods and how to do simple animations/progress bars using `clear_output()`. Added `Client.spin_thread(interval)` / `stop_spin_thread()` for running spin in a background thread, to keep zmq queue clear. This can be used to ensure that timing information is as accurate as possible (at the cost of having a background thread active).

MinRK - - Load All Authors

File last commit:

r5344:293d3eed


                r6485:e0b43119

Download file

             factory.py
        
                    77 lines
            
             | 2.9 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / IPython / parallel / factory.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        MinRK
    
update recently changed modules with Authors in docstring

              r4018
            
      """Base config factories.

      Authors:

      * Min RK

      """

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
      #-----------------------------------------------------------------------------

        MinRK
    
reorganize Factory classes to follow relocation of Session object

              r4007
            
      #  Copyright (C) 2010-2011  The IPython Development Team

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
      #

      #  Distributed under the terms of the BSD License.  The full license is in

      #  the file COPYING, distributed as part of this software.

      #-----------------------------------------------------------------------------

      #-----------------------------------------------------------------------------

      # Imports

      #-----------------------------------------------------------------------------

      import logging

        MinRK
    
resort imports in a cleaner order

              r3631
            
      import os

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
        MinRK
    
use Context.instance() for default in SessionFactory

              r3998
            
      import zmq

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
      from zmq.eventloop.ioloop import IOLoop

      from IPython.config.configurable import Configurable

        MinRK
    
add Integer traitlet...

              r5344
            
      from IPython.utils.traitlets import Integer, Instance, Unicode

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
        MinRK
    
organize IPython.parallel into subpackages

              r3673
            
      from IPython.parallel.util import select_random_ports

        MinRK
    
reorganize Factory classes to follow relocation of Session object

              r4007
            
      from IPython.zmq.session import Session, SessionFactory

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
      #-----------------------------------------------------------------------------

      # Classes

      #-----------------------------------------------------------------------------

      class RegistrationFactory(SessionFactory):

          """The Base Configurable for objects that involve registration."""

        MinRK
    
cleanup parallel traits...

              r3988
            
          url = Unicode('', config=True,

        MinRK
    
all ipcluster scripts in some degree of working order with new config

              r3985
            
              help="""The 0MQ url used for registration. This sets transport, ip, and port

              in one variable. For example: url='tcp://127.0.0.1:12345' or

              url='epgm://*:90210'""") # url takes precedence over ip,regport,transport

        MinRK
    
cleanup parallel traits...

              r3988
            
          transport = Unicode('tcp', config=True,

        MinRK
    
all ipcluster scripts in some degree of working order with new config

              r3985
            
              help="""The 0MQ transport for communications.  This will likely be

              the default of 'tcp', but other values include 'ipc', 'epgm', 'inproc'.""")

        MinRK
    
cleanup parallel traits...

              r3988
            
          ip = Unicode('127.0.0.1', config=True,

        MinRK
    
all ipcluster scripts in some degree of working order with new config

              r3985
            
              help="""The IP address for registration.  This is generally either

              '127.0.0.1' for loopback only or '*' for all interfaces.

              [default: '127.0.0.1']""")

        MinRK
    
add Integer traitlet...

              r5344
            
          regport = Integer(config=True,

        MinRK
    
all ipcluster scripts in some degree of working order with new config

              r3985
            
              help="""The port on which the Hub listens for registration.""")

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
          def _regport_default(self):

        MinRK
    
persist connection data to disk as json

              r3614
            
              return select_random_ports(1)[0]

        MinRK
    
Refactor newparallel to use Config system...

              r3604
            
          def __init__(self, **kwargs):

              super(RegistrationFactory, self).__init__(**kwargs)

              self._propagate_url()

              self._rebuild_url()

              self.on_trait_change(self._propagate_url, 'url')

              self.on_trait_change(self._rebuild_url, 'ip')

              self.on_trait_change(self._rebuild_url, 'transport')

              self.on_trait_change(self._rebuild_url, 'regport')

          def _rebuild_url(self):

              self.url = "%s://%s:%i"%(self.transport, self.ip, self.regport)

          def _propagate_url(self):

              """Ensure self.url contains full transport://interface:port"""

              if self.url:

                  iface = self.url.split('://',1)

                  if len(iface) == 2:

                      self.transport,iface = iface

                  iface = iface.split(':')

                  self.ip = iface[0]

                  if iface[1]:

                      self.regport = int(iface[1])

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

MinRK update recently changed modules with Authors in docstring	r4018	"""Base config factories.

		Authors:

		* Min RK
		"""
MinRK Refactor newparallel to use Config system...	r3604
		#-----------------------------------------------------------------------------
MinRK reorganize Factory classes to follow relocation of Session object	r4007	# Copyright (C) 2010-2011 The IPython Development Team
MinRK Refactor newparallel to use Config system...	r3604	#
		# Distributed under the terms of the BSD License. The full license is in
		# the file COPYING, distributed as part of this software.
		#-----------------------------------------------------------------------------

		#-----------------------------------------------------------------------------
		# Imports
		#-----------------------------------------------------------------------------


		import logging
MinRK resort imports in a cleaner order	r3631	import os
MinRK Refactor newparallel to use Config system...	r3604
MinRK use Context.instance() for default in SessionFactory	r3998	import zmq
MinRK Refactor newparallel to use Config system...	r3604	from zmq.eventloop.ioloop import IOLoop

		from IPython.config.configurable import Configurable
MinRK add Integer traitlet...	r5344	from IPython.utils.traitlets import Integer, Instance, Unicode
MinRK Refactor newparallel to use Config system...	r3604
MinRK organize IPython.parallel into subpackages	r3673	from IPython.parallel.util import select_random_ports
MinRK reorganize Factory classes to follow relocation of Session object	r4007	from IPython.zmq.session import Session, SessionFactory
MinRK Refactor newparallel to use Config system...	r3604
		#-----------------------------------------------------------------------------
		# Classes
		#-----------------------------------------------------------------------------


		class RegistrationFactory(SessionFactory):
		"""The Base Configurable for objects that involve registration."""

MinRK cleanup parallel traits...	r3988	url = Unicode('', config=True,
MinRK all ipcluster scripts in some degree of working order with new config	r3985	help="""The 0MQ url used for registration. This sets transport, ip, and port
		in one variable. For example: url='tcp://127.0.0.1:12345' or
		url='epgm://*:90210'""") # url takes precedence over ip,regport,transport
MinRK cleanup parallel traits...	r3988	transport = Unicode('tcp', config=True,
MinRK all ipcluster scripts in some degree of working order with new config	r3985	help="""The 0MQ transport for communications. This will likely be
		the default of 'tcp', but other values include 'ipc', 'epgm', 'inproc'.""")
MinRK cleanup parallel traits...	r3988	ip = Unicode('127.0.0.1', config=True,
MinRK all ipcluster scripts in some degree of working order with new config	r3985	help="""The IP address for registration. This is generally either
		'127.0.0.1' for loopback only or '*' for all interfaces.
		[default: '127.0.0.1']""")
MinRK add Integer traitlet...	r5344	regport = Integer(config=True,
MinRK all ipcluster scripts in some degree of working order with new config	r3985	help="""The port on which the Hub listens for registration.""")
MinRK Refactor newparallel to use Config system...	r3604	def _regport_default(self):
MinRK persist connection data to disk as json	r3614	return select_random_ports(1)[0]
MinRK Refactor newparallel to use Config system...	r3604
		def __init__(self, **kwargs):
		super(RegistrationFactory, self).__init__(**kwargs)
		self._propagate_url()
		self._rebuild_url()
		self.on_trait_change(self._propagate_url, 'url')
		self.on_trait_change(self._rebuild_url, 'ip')
		self.on_trait_change(self._rebuild_url, 'transport')
		self.on_trait_change(self._rebuild_url, 'regport')

		def _rebuild_url(self):
		self.url = "%s://%s:%i"%(self.transport, self.ip, self.regport)

		def _propagate_url(self):
		"""Ensure self.url contains full transport://interface:port"""
		if self.url:
		iface = self.url.split('://',1)
		if len(iface) == 2:
		self.transport,iface = iface
		iface = iface.split(':')
		self.ip = iface[0]
		if iface[1]:
		self.regport = int(iface[1])