Show More
@@ -2,11 +2,19 b'' | |||||
2 | """ |
|
2 | """ | |
3 | Script for setting up and using [all]reduce with a binary-tree engine interconnect. |
|
3 | Script for setting up and using [all]reduce with a binary-tree engine interconnect. | |
4 |
|
4 | |||
5 | Binary trees allow large data communications to be highly scalable, because even |
|
|||
6 | in a global communication, never more than two messages are occupant on a single node. |
|
|||
7 |
|
||||
8 | usage: `python bintree_script.py` |
|
5 | usage: `python bintree_script.py` | |
9 |
|
6 | |||
|
7 | This spanning tree strategy ensures that a single node node mailbox will never | |||
|
8 | receive more that 2 messages at once. This is very important to scale to large | |||
|
9 | clusters (e.g. 1000 nodes) since if you have many incoming messages of a couple | |||
|
10 | of megabytes you might saturate the network interface of a single node and | |||
|
11 | potentially its memory buffers if the messages are not consumed in a streamed | |||
|
12 | manner. | |||
|
13 | ||||
|
14 | Note that the AllReduce scheme implemented with the spanning tree strategy | |||
|
15 | impose the aggregation function to be commutative and distributive. It might | |||
|
16 | not be the case if you implement the naive gather / reduce / broadcast strategy | |||
|
17 | where you can reorder the partial data before performing the reduce. | |||
10 | """ |
|
18 | """ | |
11 |
|
19 | |||
12 | from IPython.parallel import Client, Reference |
|
20 | from IPython.parallel import Client, Reference |
General Comments 0
You need to be logged in to leave comments.
Login now