Show More
@@ -1,63 +1,66 b'' | |||||
1 | .. _performance: |
|
1 | .. _performance: | |
2 |
|
2 | |||
3 | ================================ |
|
3 | ================================ | |
4 | Optimizing Kallithea Performance |
|
4 | Optimizing Kallithea Performance | |
5 | ================================ |
|
5 | ================================ | |
6 |
|
6 | |||
7 | When serving a large amount of big repositories, Kallithea can start |
|
7 | When serving a large amount of big repositories, Kallithea can start | |
8 | performing slower than expected. Because of the demanding nature of handling large |
|
8 | performing slower than expected. Because of the demanding nature of handling large | |
9 | amounts of data from version control systems, here are some tips on how to get |
|
9 | amounts of data from version control systems, here are some tips on how to get | |
10 | the best performance. |
|
10 | the best performance. | |
11 |
|
11 | |||
12 | * Kallithea will perform better on machines with faster disks (SSD/SAN). It's |
|
12 | * Kallithea will perform better on machines with faster disks (SSD/SAN). It's | |
13 | more important to have a faster disk than a faster CPU. |
|
13 | more important to have a faster disk than a faster CPU. | |
14 |
|
14 | |||
15 | * Slowness on initial page can be easily fixed by grouping repositories, and/or |
|
15 | * Slowness on initial page can be easily fixed by grouping repositories, and/or | |
16 | increasing cache size (see below). This includes using the lightweight dashboard |
|
16 | increasing cache size (see below). This includes using the lightweight dashboard | |
17 | option and ``vcs_full_cache`` setting in .ini file |
|
17 | option and ``vcs_full_cache`` setting in .ini file | |
18 |
|
18 | |||
19 |
|
19 | |||
20 | Follow these few steps to improve performance of Kallithea system. |
|
20 | Follow these few steps to improve performance of Kallithea system. | |
21 |
|
21 | |||
22 |
|
22 | |||
23 | 1. Increase cache |
|
23 | 1. Increase cache | |
24 |
|
24 | |||
25 | In the .ini file:: |
|
25 | In the .ini file:: | |
26 |
|
26 | |||
27 | beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number |
|
27 | beaker.cache.sql_cache_long.expire=3600 <-- set this to higher number | |
28 |
|
28 | |||
29 | This option affects the cache expiration time for the main |
|
29 | This option affects the cache expiration time for the main | |
30 | page. Having several hundreds of repositories on main page can |
|
30 | page. Having several hundreds of repositories on main page can | |
31 | sometimes make the system behave slowly when the cache expires for |
|
31 | sometimes make the system behave slowly when the cache expires for | |
32 | all of them. Increasing the ``expire`` option to a day (86400) or a |
|
32 | all of them. Increasing the ``expire`` option to a day (86400) or a | |
33 | week (604800) will improve general response times for the main |
|
33 | week (604800) will improve general response times for the main | |
34 | page. Kallithea has an intelligent cache expiration system and it |
|
34 | page. Kallithea has an intelligent cache expiration system and it | |
35 | will expire the cache for repositories that have been changed. |
|
35 | will expire the cache for repositories that have been changed. | |
36 |
|
36 | |||
37 | 2. Switch from sqlite to postgres or mysql |
|
37 | 2. Switch from sqlite to postgres or mysql | |
38 |
|
38 | |||
39 | sqlite is a good option when having a small load on the system. But due to |
|
39 | sqlite is a good option when having a small load on the system. But due to | |
40 | locking issues with sqlite, it is not recommended to use it for larger |
|
40 | locking issues with sqlite, it is not recommended to use it for larger | |
41 | deployments. Switching to mysql or postgres will result in an immediate |
|
41 | deployments. Switching to mysql or postgres will result in an immediate | |
42 | performance increase. |
|
42 | performance increase. A tool like SQLAlchemyGrate_ can be used for | |
|
43 | migrating to another database platform. | |||
43 |
|
44 | |||
44 | 3. Scale Kallithea horizontally |
|
45 | 3. Scale Kallithea horizontally | |
45 |
|
46 | |||
46 | Scaling horizontally can give huge performance increases when dealing with |
|
47 | Scaling horizontally can give huge performance increases when dealing with | |
47 | large traffic (large amount of users, CI servers etc). Kallithea can be |
|
48 | large traffic (large amount of users, CI servers etc). Kallithea can be | |
48 | scaled horizontally on one (recommended) or multiple machines. In order |
|
49 | scaled horizontally on one (recommended) or multiple machines. In order | |
49 | to scale horizontally you need to do the following: |
|
50 | to scale horizontally you need to do the following: | |
50 |
|
51 | |||
51 | - Each instance needs its own .ini file and unique ``instance_id`` set. |
|
52 | - Each instance needs its own .ini file and unique ``instance_id`` set. | |
52 | - Each instance's ``data`` storage needs to be configured to be stored on a |
|
53 | - Each instance's ``data`` storage needs to be configured to be stored on a | |
53 | shared disk storage, preferably together with repositories. This ``data`` |
|
54 | shared disk storage, preferably together with repositories. This ``data`` | |
54 | dir contains template caches, sessions, whoosh index and is used for |
|
55 | dir contains template caches, sessions, whoosh index and is used for | |
55 | task locking (so it is safe across multiple instances). Set the |
|
56 | task locking (so it is safe across multiple instances). Set the | |
56 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` |
|
57 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` | |
57 | variables in each .ini file to a shared location across Kallithea instances |
|
58 | variables in each .ini file to a shared location across Kallithea instances | |
58 | - If celery is used each instance should run a separate Celery instance, but |
|
59 | - If celery is used each instance should run a separate Celery instance, but | |
59 | the message broker should be common to all of them (e.g., one |
|
60 | the message broker should be common to all of them (e.g., one | |
60 | shared RabbitMQ server) |
|
61 | shared RabbitMQ server) | |
61 | - Load balance using round robin or IP hash, recommended is writing LB rules |
|
62 | - Load balance using round robin or IP hash, recommended is writing LB rules | |
62 | that will separate regular user traffic from automated processes like CI |
|
63 | that will separate regular user traffic from automated processes like CI | |
63 | servers or build bots. |
|
64 | servers or build bots. | |
|
65 | ||||
|
66 | .. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate |
General Comments 0
You need to be logged in to leave comments.
Login now