##// END OF EJS Templates
Adding the argparse module to externals.
Brian Granger -
Show More
@@ -32,19 +32,15 b' IPython is implemented using a distributed set of processes that communicate usi'
32
32
33 We need to build a system that makes it trivial for users to start and manage IPython processes. This system should have the following properties:
33 We need to build a system that makes it trivial for users to start and manage IPython processes. This system should have the following properties:
34
34
35 * It should possible to do everything through an extremely simple API that users
35 * It should possible to do everything through an extremely simple API that
36 can call from their own Python script. No shell commands should be needed.
36 users can call from their own Python script. No shell commands should be
37
37 needed.
38 * This simple API should be configured using standard .ini files.
38 * This simple API should be configured using standard .ini files.
39
39 * The system should make it possible to start processes using a number of
40 * The system should make it possible to start processes using a number of different
40 different approaches: SSH, PBS/Torque, Xgrid, Windows Server, mpirun, etc.
41 approaches: SSH, PBS/Torque, Xgrid, Windows Server, mpirun, etc.
41 * The controller and engine processes should each have a daemon for
42
42 monitoring, signaling and clean up.
43 * The controller and engine processes should each have a daemon for monitoring,
44 signaling and clean up.
45
46 * The system should be secure.
43 * The system should be secure.
47
48 * The system should work under all the major operating systems, including
44 * The system should work under all the major operating systems, including
49 Windows.
45 Windows.
50
46
@@ -63,12 +59,10 b' Security'
63 Currently, IPython has no built in security or security model. Because we would like IPython to be usable on public computer systems and over wide area networks, we need to come up with a robust solution for security. Here are some of the specific things that need to be included:
59 Currently, IPython has no built in security or security model. Because we would like IPython to be usable on public computer systems and over wide area networks, we need to come up with a robust solution for security. Here are some of the specific things that need to be included:
64
60
65 * User authentication between all processes (engines, controller and clients).
61 * User authentication between all processes (engines, controller and clients).
66
67 * Optional TSL/SSL based encryption of all communication channels.
62 * Optional TSL/SSL based encryption of all communication channels.
68
69 * A good way of picking network ports so multiple users on the same system can
63 * A good way of picking network ports so multiple users on the same system can
70 run their own controller and engines without interfering with those of others.
64 run their own controller and engines without interfering with those of
71
65 others.
72 * A clear model for security that enables users to evaluate the security risks
66 * A clear model for security that enables users to evaluate the security risks
73 associated with using IPython in various manners.
67 associated with using IPython in various manners.
74
68
@@ -86,22 +80,25 b' Latent performance issues'
86
80
87 Currently, we have a number of performance issues that are waiting to bite users:
81 Currently, we have a number of performance issues that are waiting to bite users:
88
82
89 * The controller store a large amount of state in Python dictionaries. Under heavy
83 * The controller store a large amount of state in Python dictionaries. Under
90 usage, these dicts with get very large, causing memory usage problems. We need to
84 heavy usage, these dicts with get very large, causing memory usage problems.
91 develop more scalable solutions to this problem, such as using a sqlite database
85 We need to develop more scalable solutions to this problem, such as using a
92 to store this state. This will also help the controller to be more fault tolerant.
86 sqlite database to store this state. This will also help the controller to
93 * Currently, the client to controller connections are done through XML-RPC using
87 be more fault tolerant.
94 HTTP 1.0. This is very inefficient as XML-RPC is a very verbose protocol and
88 * Currently, the client to controller connections are done through XML-RPC
95 each request must be handled with a new connection. We need to move these network
89 using HTTP 1.0. This is very inefficient as XML-RPC is a very verbose
96 connections over to PB or Foolscap. Done!
90 protocol and each request must be handled with a new connection. We need to
97 * We currently don't have a good way of handling large objects in the controller.
91 move these network connections over to PB or Foolscap. Done!
98 The biggest problem is that because we don't have any way of streaming objects,
92 * We currently don't have a good way of handling large objects in the
99 we get lots of temporary copies in the low-level buffers. We need to implement
93 controller. The biggest problem is that because we don't have any way of
100 a better serialization approach and true streaming support.
94 streaming objects, we get lots of temporary copies in the low-level buffers.
101 * The controller currently unpickles and repickles objects. We need to use the
95 We need to implement a better serialization approach and true streaming
102 [push|pull]_serialized methods instead.
96 support.
103 * Currently the controller is a bottleneck. We need the ability to scale the
97 * The controller currently unpickles and repickles objects. We need to use the
104 controller by aggregating multiple controllers into one effective controller.
98 [push|pull]_serialized methods instead.
99 * Currently the controller is a bottleneck. We need the ability to scale the
100 controller by aggregating multiple controllers into one effective
101 controller.
105
102
106
103
107
104
General Comments 0
You need to be logged in to leave comments. Login now