##// END OF EJS Templates
Adding the argparse module to externals.
Brian Granger -
Show More
@@ -32,19 +32,15 b' IPython is implemented using a distributed set of processes that communicate usi'
32 32
33 33 We need to build a system that makes it trivial for users to start and manage IPython processes. This system should have the following properties:
34 34
35 * It should possible to do everything through an extremely simple API that users
36 can call from their own Python script. No shell commands should be needed.
37
35 * It should possible to do everything through an extremely simple API that
36 users can call from their own Python script. No shell commands should be
37 needed.
38 38 * This simple API should be configured using standard .ini files.
39
40 * The system should make it possible to start processes using a number of different
41 approaches: SSH, PBS/Torque, Xgrid, Windows Server, mpirun, etc.
42
43 * The controller and engine processes should each have a daemon for monitoring,
44 signaling and clean up.
45
39 * The system should make it possible to start processes using a number of
40 different approaches: SSH, PBS/Torque, Xgrid, Windows Server, mpirun, etc.
41 * The controller and engine processes should each have a daemon for
42 monitoring, signaling and clean up.
46 43 * The system should be secure.
47
48 44 * The system should work under all the major operating systems, including
49 45 Windows.
50 46
@@ -63,12 +59,10 b' Security'
63 59 Currently, IPython has no built in security or security model. Because we would like IPython to be usable on public computer systems and over wide area networks, we need to come up with a robust solution for security. Here are some of the specific things that need to be included:
64 60
65 61 * User authentication between all processes (engines, controller and clients).
66
67 62 * Optional TSL/SSL based encryption of all communication channels.
68
69 63 * A good way of picking network ports so multiple users on the same system can
70 run their own controller and engines without interfering with those of others.
71
64 run their own controller and engines without interfering with those of
65 others.
72 66 * A clear model for security that enables users to evaluate the security risks
73 67 associated with using IPython in various manners.
74 68
@@ -86,22 +80,25 b' Latent performance issues'
86 80
87 81 Currently, we have a number of performance issues that are waiting to bite users:
88 82
89 * The controller store a large amount of state in Python dictionaries. Under heavy
90 usage, these dicts with get very large, causing memory usage problems. We need to
91 develop more scalable solutions to this problem, such as using a sqlite database
92 to store this state. This will also help the controller to be more fault tolerant.
93 * Currently, the client to controller connections are done through XML-RPC using
94 HTTP 1.0. This is very inefficient as XML-RPC is a very verbose protocol and
95 each request must be handled with a new connection. We need to move these network
96 connections over to PB or Foolscap. Done!
97 * We currently don't have a good way of handling large objects in the controller.
98 The biggest problem is that because we don't have any way of streaming objects,
99 we get lots of temporary copies in the low-level buffers. We need to implement
100 a better serialization approach and true streaming support.
101 * The controller currently unpickles and repickles objects. We need to use the
102 [push|pull]_serialized methods instead.
103 * Currently the controller is a bottleneck. We need the ability to scale the
104 controller by aggregating multiple controllers into one effective controller.
83 * The controller store a large amount of state in Python dictionaries. Under
84 heavy usage, these dicts with get very large, causing memory usage problems.
85 We need to develop more scalable solutions to this problem, such as using a
86 sqlite database to store this state. This will also help the controller to
87 be more fault tolerant.
88 * Currently, the client to controller connections are done through XML-RPC
89 using HTTP 1.0. This is very inefficient as XML-RPC is a very verbose
90 protocol and each request must be handled with a new connection. We need to
91 move these network connections over to PB or Foolscap. Done!
92 * We currently don't have a good way of handling large objects in the
93 controller. The biggest problem is that because we don't have any way of
94 streaming objects, we get lots of temporary copies in the low-level buffers.
95 We need to implement a better serialization approach and true streaming
96 support.
97 * The controller currently unpickles and repickles objects. We need to use the
98 [push|pull]_serialized methods instead.
99 * Currently the controller is a bottleneck. We need the ability to scale the
100 controller by aggregating multiple controllers into one effective
101 controller.
105 102
106 103
107 104
General Comments 0
You need to be logged in to leave comments. Login now