Show More
@@ -1,99 +1,112 b'' | |||
|
1 | 1 | .. _performance: |
|
2 | 2 | |
|
3 | 3 | ================================ |
|
4 | 4 | Optimizing Kallithea performance |
|
5 | 5 | ================================ |
|
6 | 6 | |
|
7 | When serving a large amount of big repositories, Kallithea can start | |
|
8 |
|
|
|
9 |
|
|
|
10 |
|
|
|
7 | When serving a large amount of big repositories, Kallithea can start performing | |
|
8 | slower than expected. Because of the demanding nature of handling large amounts | |
|
9 | of data from version control systems, here are some tips on how to get the best | |
|
10 | performance. | |
|
11 | 11 | |
|
12 | Follow these few steps to improve performance of Kallithea system. | |
|
12 | ||
|
13 | Fast storage | |
|
14 | ------------ | |
|
13 | 15 | |
|
14 |
|
|
|
15 |
|
|
|
16 | Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) and plenty of RAM | |
|
17 | is usually more important than a fast CPU. | |
|
16 | 18 | |
|
17 | 2. Increase cache | |
|
19 | ||
|
20 | Caching | |
|
21 | ------- | |
|
18 | 22 | |
|
19 |
|
|
|
20 |
|
|
|
23 | Tweak beaker cache settings in the ini file. The actual effect of that is | |
|
24 | questionable. | |
|
21 | 25 | |
|
22 | 3. Switch from SQLite to PostgreSQL or MySQL | |
|
26 | ||
|
27 | Database | |
|
28 | -------- | |
|
23 | 29 | |
|
24 | 30 |
|
|
25 | 31 |
|
|
26 | deployments. Switching to MySQL or PostgreSQL will result in an immediate | |
|
27 | performance increase. A tool like SQLAlchemyGrate_ can be used for | |
|
28 | migrating to another database platform. | |
|
32 | deployments. | |
|
33 | ||
|
34 | Switching to MySQL or PostgreSQL will result in an immediate performance | |
|
35 | increase. A tool like SQLAlchemyGrate_ can be used for migrating to another | |
|
36 | database platform. | |
|
37 | ||
|
29 | 38 | |
|
30 | 4. Scale Kallithea horizontally | |
|
39 | Horizontal scaling | |
|
40 | ------------------ | |
|
31 | 41 | |
|
32 | Scaling horizontally can give huge performance benefits when dealing with | |
|
33 | large amounts of traffic (many users, CI servers, etc.). Kallithea can be | |
|
34 | scaled horizontally on one (recommended) or multiple machines. | |
|
42 | Scaling horizontally means running several Kallithea instances and let them | |
|
43 | share the load. That can give huge performance benefits when dealing with large | |
|
44 | amounts of traffic (many users, CI servers, etc.). Kallithea can be scaled | |
|
45 | horizontally on one (recommended) or multiple machines. | |
|
35 | 46 | |
|
36 | 47 |
|
|
37 |
|
|
|
38 |
|
|
|
39 |
|
|
|
48 | several HTTP requests are served from the same Python process at once. That can | |
|
49 | in principle give better utilization of internal caches and less process | |
|
50 | overhead. | |
|
40 | 51 | |
|
41 |
|
|
|
42 |
|
|
|
43 |
|
|
|
52 | One danger of running multithreaded is that program execution becomes much more | |
|
53 | complex; programs must be written to consider all combinations of events and | |
|
54 | problems might depend on timing and be impossible to reproduce. | |
|
44 | 55 | |
|
45 | 56 |
|
|
46 | 57 |
|
|
47 | 58 |
|
|
48 | 59 | |
|
49 | 60 |
|
|
50 | 61 |
|
|
51 | 62 | |
|
52 | 63 |
|
|
53 | 64 |
|
|
54 | 65 | |
|
55 | 66 | - Each instance's ``data`` storage needs to be configured to be stored on a |
|
56 | 67 | shared disk storage, preferably together with repositories. This ``data`` |
|
57 | 68 | dir contains template caches, sessions, whoosh index and is used for |
|
58 | 69 | task locking (so it is safe across multiple instances). Set the |
|
59 | 70 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` |
|
60 | 71 | variables in each .ini file to a shared location across Kallithea instances |
|
61 | 72 | - If using several Celery instances, |
|
62 | 73 | the message broker should be common to all of them (e.g., one |
|
63 | 74 | shared RabbitMQ server) |
|
64 | 75 | - Load balance using round robin or IP hash, recommended is writing LB rules |
|
65 | 76 | that will separate regular user traffic from automated processes like CI |
|
66 | 77 | servers or build bots. |
|
67 | 78 | |
|
68 | 5. Serve static files directly from the web server | |
|
79 | ||
|
80 | Serve static files directly from the web server | |
|
81 | ----------------------------------------------- | |
|
69 | 82 | |
|
70 | 83 | With the default ``static_files`` ini setting, the Kallithea WSGI application |
|
71 | 84 | will take care of serving the static files found in ``kallithea/public`` from |
|
72 | 85 | the root of the application URL. While doing that, it will currently also |
|
73 | 86 | apply buffering and compression of all the responses it is serving. |
|
74 | 87 | |
|
75 | 88 | The actual serving of the static files is unlikely to be a problem in a |
|
76 | 89 | Kallithea setup. The buffering of responses is more likely to be a problem; |
|
77 | 90 | large responses (clones or pulls) will have to be fully processed and spooled |
|
78 | 91 | to disk or memory before the client will see any response. |
|
79 | 92 | |
|
80 | 93 | To serve static files from the web server, use something like this Apache config |
|
81 | 94 | snippet:: |
|
82 | 95 | |
|
83 | 96 | Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/ |
|
84 | 97 | Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/ |
|
85 | 98 | Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/ |
|
86 | 99 | Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/ |
|
87 | 100 | Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/ |
|
88 | 101 | |
|
89 | 102 | Then disable serving of static files in the ``.ini`` ``app:main`` section:: |
|
90 | 103 | |
|
91 | 104 | static_files = false |
|
92 | 105 | |
|
93 | 106 | If using Kallithea installed as a package, you should be able to find the files |
|
94 | 107 | under site-packages/kallithea, either in your Python installation or in your |
|
95 | 108 | virtualenv. When upgrading, make sure to update the web server configuration |
|
96 | 109 | too if necessary. |
|
97 | 110 | |
|
98 | 111 | |
|
99 | 112 | .. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate |
General Comments 0
You need to be logged in to leave comments.
Login now