##// END OF EJS Templates
add bintree paragraphs by @ogrisel...
MinRK -
Show More
@@ -2,11 +2,19 b''
2 """
2 """
3 Script for setting up and using [all]reduce with a binary-tree engine interconnect.
3 Script for setting up and using [all]reduce with a binary-tree engine interconnect.
4
4
5 Binary trees allow large data communications to be highly scalable, because even
6 in a global communication, never more than two messages are occupant on a single node.
7
8 usage: `python bintree_script.py`
5 usage: `python bintree_script.py`
9
6
7 This spanning tree strategy ensures that a single node node mailbox will never
8 receive more that 2 messages at once. This is very important to scale to large
9 clusters (e.g. 1000 nodes) since if you have many incoming messages of a couple
10 of megabytes you might saturate the network interface of a single node and
11 potentially its memory buffers if the messages are not consumed in a streamed
12 manner.
13
14 Note that the AllReduce scheme implemented with the spanning tree strategy
15 impose the aggregation function to be commutative and distributive. It might
16 not be the case if you implement the naive gather / reduce / broadcast strategy
17 where you can reorder the partial data before performing the reduce.
10 """
18 """
11
19
12 from IPython.parallel import Client, Reference
20 from IPython.parallel import Client, Reference
General Comments 0
You need to be logged in to leave comments. Login now