##// END OF EJS Templates
Merge pull request #2405 from minrk/paralleldoc...
Min RK -
r8447:48dd8a34 merge
parent child Browse files
Show More
@@ -386,19 +386,20 b' weighted: Weighted Two-Bin Random'
386 Greedy Assignment
386 Greedy Assignment
387 -----------------
387 -----------------
388
388
389 Tasks are assigned greedily as they are submitted. If their dependencies are
389 Tasks can be assigned greedily as they are submitted. If their dependencies are
390 met, they will be assigned to an engine right away, and multiple tasks can be
390 met, they will be assigned to an engine right away, and multiple tasks can be
391 assigned to an engine at a given time. This limit is set with the
391 assigned to an engine at a given time. This limit is set with the
392 ``TaskScheduler.hwm`` (high water mark) configurable:
392 ``TaskScheduler.hwm`` (high water mark) configurable in your
393 :file:`ipcontroller_config.py` config file, with:
393
394
394 .. sourcecode:: python
395 .. sourcecode:: python
395
396
396 # the most common choices are:
397 # the most common choices are:
397 c.TaskSheduler.hwm = 0 # (minimal latency, default in IPython ≤ 0.12)
398 c.TaskSheduler.hwm = 0 # (minimal latency, default in IPython < 0.13)
398 # or
399 # or
399 c.TaskScheduler.hwm = 1 # (most-informed balancing, default in > 0.12)
400 c.TaskScheduler.hwm = 1 # (most-informed balancing, default in ≥ 0.13)
400
401
401 In IPython ≤ 0.12,the default is 0, or no-limit. That is, there is no limit to the number of
402 In IPython < 0.13, the default is 0, or no-limit. That is, there is no limit to the number of
402 tasks that can be outstanding on a given engine. This greatly benefits the
403 tasks that can be outstanding on a given engine. This greatly benefits the
403 latency of execution, because network traffic can be hidden behind computation.
404 latency of execution, because network traffic can be hidden behind computation.
404 However, this means that workload is assigned without knowledge of how long
405 However, this means that workload is assigned without knowledge of how long
@@ -406,10 +407,10 b' each task might take, and can result in poor load-balancing, particularly for'
406 submitting a collection of heterogeneous tasks all at once. You can limit this
407 submitting a collection of heterogeneous tasks all at once. You can limit this
407 effect by setting hwm to a positive integer, 1 being maximum load-balancing (a
408 effect by setting hwm to a positive integer, 1 being maximum load-balancing (a
408 task will never be waiting if there is an idle engine), and any larger number
409 task will never be waiting if there is an idle engine), and any larger number
409 being a compromise between load-balance and latency-hiding.
410 being a compromise between load-balancing and latency-hiding.
410
411
411 In practice, some users have been confused by having this optimization on by
412 In practice, some users have been confused by having this optimization on by
412 default, and the default value has been changed to 1. This can be slower,
413 default, so the default value has been changed to 1 in IPython 0.13. This can be slower,
413 but has more obvious behavior and won't result in assigning too many tasks to
414 but has more obvious behavior and won't result in assigning too many tasks to
414 some engines in heterogeneous cases.
415 some engines in heterogeneous cases.
415
416
General Comments 0
You need to be logged in to leave comments. Login now