##// END OF EJS Templates
Merge pull request #1768 from minrk/parallelmagics...
Merge pull request #1768 from minrk/parallelmagics Update parallel magics They now display all output, so you can do parallel plotting or other actions with complex display. The `px` magic has now both line and cell modes, and in cell mode finer control has been added about how to collate output from multiple engines. Tests, docs and example notebook added.

File last commit:

r7053:30a685a4
r7060:a1360828 merge
Show More
Parallel Magics.ipynb
227 lines | 4.9 KiB | text/plain | TextLexer

Using Parallel Magics

IPython has a few magics for working with your engines.

This assumes you have started an IPython cluster, either with the notebook interface, or the ipcluster/controller/engine commands.

In [ ]:
from IPython import parallel
rc = parallel.Client()
dv = rc[:]
dv.block = True
dv

The parallel magics come from the parallelmagics IPython extension. The magics are set to work with a particular View object, so to activate them, you call the activate() method on a particular view:

In [ ]:
dv.activate()

Now we can execute code remotely with %px:

In [ ]:
%px a=5
In [ ]:
%px print a
In [ ]:
%px a

You don't have to wait for results:

In [ ]:
dv.block = False
In [ ]:
%px import time
%px time.sleep(5)
%px time.time()

But you will notice that this didn't output the result of the last command. For this, we have %result, which displays the output of the latest request:

In [ ]:
%result

Remember, an IPython engine is IPython, so you can do magics remotely as well!

In [ ]:
dv.block = True
%px %pylab inline

%%px can also be used as a cell magic, for submitting whole blocks. This one acceps --block and --noblock flags to specify the blocking behavior, though the default is unchanged.

In [ ]:
dv.scatter('id', dv.targets, flatten=True)
dv['stride'] = len(dv)
In [ ]:
%%px --noblock
x = linspace(0,pi,1000)
for n in range(id,12, stride):
    print n
    plt.plot(x,sin(n*x))
plt.title("Plot %i" % id)
In [ ]:
%result

It also lets you choose some amount of the grouping of the outputs with --group-outputs:

The choices are:

  • engine - all of an engine's output is collected together
  • type - where stdout of each engine is grouped, etc. (the default)
  • order - same as type, but individual displaypub outputs are interleaved. That is, it will output the first plot from each engine, then the second from each, etc.
In [ ]:
%%px --group-outputs=engine
x = linspace(0,pi,1000)
for n in range(id,12, stride):
    print n
    plt.plot(x,sin(n*x))
plt.title("Plot %i" % id)
In [ ]: