gearbox: replace paster with something TurboGears2-ish that still works with the Pylons stack

This is a step towards moving away from the Pylons stack to TurboGears2, but
still independent of it.


Some notes from the porting - it could perhaps be the missing(?) documentation
for migrating from paster to gearbox:

Note: 'gearbox' without parameters will crash - specify '-h' to get started
testing.

Replace paster
    summary = 'yada yada'
with the first line of the docstring of the Command class ... or override
get_description.

Note: All newlines in the docstring will be collapsed and mangle the long help
text.

Grouping of commands is not possible. Standard commands (for development) can't
be customized under the same name or hidden. (Like for paster, the conceptual
model also assumes that the sub-command naming is namespaced so commands from
other packages won't conflict.)

The usage help is fully automated from the declared options.

For all deprecated Commands, replace paster
    hidden = True
with gearbox
    deprecated = True

Note: config_file, takes_config_file, min_args and max_args are not available /
relevant.

The gearbox parser is customized by overriding get_parser - there is nothing
like paster update_parser.

Gearbox is using argparse instead of optparse ... but argparse add_argument is
mostly backwards compatible with optparse add_option.

Instead of overriding command or run as in paster, override take_action in
gearbox. The parsed arguments are passed to take_action, not available on the
command instance.

Paster BadCommand is not available and must be handled manually, terminating
with sys.exit(1).

There is no standard make-config command in gearbox.

Paster appinstall has been replaced by the somewhat different setup_app module
in gearbox. There is still no clean way to pass parameters to SetupAppCommand
and it relies on websetup and other apparently unnecessary complexity. Instead,
implement setup-db from scratch.


Minor change by Thomas De Schampheleire: add gearbox logging configuration.
Because we use logging.config.fileConfig(.inifile) during gearbox command
execution, the logging settings need to be correct and contain a block for
gearbox logging itself. Otherwise, errors in command processing are not even
visible and the command exits silently.

development.ini

@@ -1,601 +1,610 b''
 ################################################################################
 ################################################################################
 # Kallithea - Development config:                                              #
 # listening on *:5000                                                          #
 # sqlite and kallithea.db                                                      #
 # initial_repo_scan = true                                                     #
 # set debug = true                                                             #
 # verbose and colorful logging                                                 #
 #                                                                              #
 # The %(here)s variable will be replaced with the parent directory of this file#
 ################################################################################
 ################################################################################
 
 [DEFAULT]
 debug = true
 pdebug = false
 
 ################################################################################
 ## Email settings                                                             ##
 ##                                                                            ##
 ## Refer to the documentation ("Email settings") for more details.            ##
 ##                                                                            ##
 ## It is recommended to use a valid sender address that passes access         ##
 ## validation and spam filtering in mail servers.                             ##
 ################################################################################
 
 ## 'From' header for application emails. You can optionally add a name.
 ## Default:
 #app_email_from = Kallithea
 ## Examples:
 #app_email_from = Kallithea <[e-mail]>
 #app_email_from = [e-mail]
 
 ## Subject prefix for application emails.
 ## A space between this prefix and the real subject is automatically added.
 ## Default:
 #email_prefix =
 ## Example:
 #email_prefix = [Kallithea]
 
 ## Recipients for error emails and fallback recipients of application mails.
 ## Multiple addresses can be specified, space-separated.
 ## Only addresses are allowed, do not add any name part.
 ## Default:
 #email_to =
 ## Examples:
 #email_to = [e-mail]
 #email_to = [e-mail] [e-mail]
 
 ## 'From' header for error emails. You can optionally add a name.
 ## Default:
 #error_email_from = [e-mail]
 ## Examples:
 #error_email_from = Kallithea Errors <[e-mail]>
 #error_email_from = [e-mail]
 
 ## SMTP server settings
 ## If specifying credentials, make sure to use secure connections.
 ## Default: Send unencrypted unauthenticated mails to the specified smtp_server.
 ## For "SSL", use smtp_use_ssl = true and smtp_port = 465.
 ## For "STARTTLS", use smtp_use_tls = true and smtp_port = 587.
 #smtp_server = smtp.example.com
 #smtp_username =
 #smtp_password =
 #smtp_port = 25
 #smtp_use_ssl = false
 #smtp_use_tls = false
 
 [server:main]
-## PASTE ##
-#use = egg:Paste#http
+## Gearbox default web server ##
+#use = egg:gearbox#wsgiref
 ## nr of worker threads to spawn
 #threadpool_workers = 1
 ## max request before thread respawn
 #threadpool_max_requests = 100
 ## option to use threads of process
 #use_threadpool = true
 
+## Gearbox gevent web server ##
+#use = egg:gearbox#gevent
+
 ## WAITRESS ##
 use = egg:waitress#main
 ## number of worker threads
 threads = 1
 ## MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
 ## use poll instead of select, fixes fd limits, may not work on old
 ## windows systems.
 #asyncore_use_poll = True
 
 ## GUNICORN ##
 #use = egg:gunicorn#main
 ## number of process workers. You must set `instance_id = *` when this option
 ## is set to more than one worker
 #workers = 1
 ## process name
 #proc_name = kallithea
 ## type of worker class, one of sync, eventlet, gevent, tornado
 ## recommended for bigger setup is using of of other than sync one
 #worker_class = sync
 #max_requests = 1000
 ## amount of time a worker can handle request before it gets killed and
 ## restarted
 #timeout = 3600
 
 ## UWSGI ##
 ## run with uwsgi --ini-paste-logged <inifile.ini>
 #[uwsgi]
 #socket = /tmp/uwsgi.sock
 #master = true
 #http = 127.0.0.1:5000
 
 ## set as deamon and redirect all output to file
 #daemonize = ./uwsgi_kallithea.log
 
 ## master process PID
 #pidfile = ./uwsgi_kallithea.pid
 
 ## stats server with workers statistics, use uwsgitop
 ## for monitoring, `uwsgitop 127.0.0.1:1717`
 #stats = 127.0.0.1:1717
 #memory-report = true
 
 ## log 5XX errors
 #log-5xx = true
 
 ## Set the socket listen queue size.
 #listen = 256
 
 ## Gracefully Reload workers after the specified amount of managed requests
 ## (avoid memory leaks).
 #max-requests = 1000
 
 ## enable large buffers
 #buffer-size = 65535
 
 ## socket and http timeouts ##
 #http-timeout = 3600
 #socket-timeout = 3600
 
 ## Log requests slower than the specified number of milliseconds.
 #log-slow = 10
 
 ## Exit if no app can be loaded.
 #need-app = true
 
 ## Set lazy mode (load apps in workers instead of master).
 #lazy = true
 
 ## scaling ##
 ## set cheaper algorithm to use, if not set default will be used
 #cheaper-algo = spare
 
 ## minimum number of workers to keep at all times
 #cheaper = 1
 
 ## number of workers to spawn at startup
 #cheaper-initial = 1
 
 ## maximum number of workers that can be spawned
 #workers = 4
 
 ## how many workers should be spawned at a time
 #cheaper-step = 1
 
 ## COMMON ##
 #host = 127.0.0.1
 host = 0.0.0.0
 port = 5000
 
 ## middleware for hosting the WSGI application under a URL prefix
 #[filter:proxy-prefix]
 #use = egg:PasteDeploy#prefix
 #prefix = /<your-prefix>
 
 [app:main]
 use = egg:kallithea
 ## enable proxy prefix middleware
 #filter-with = proxy-prefix
 
 full_stack = true
 static_files = true
 ## Available Languages:
 ## cs de fr hu ja nl_BE pl pt_BR ru sk zh_CN zh_TW
 lang =
 cache_dir = %(here)s/data
 index_dir = %(here)s/data/index
 
 ## perform a full repository scan on each server start, this should be
 ## set to false after first startup, to allow faster server restarts.
 #initial_repo_scan = false
 initial_repo_scan = true
 
 ## uncomment and set this path to use archive download cache
 archive_cache_dir = %(here)s/tarballcache
 
 ## change this to unique ID for security
 app_instance_uuid = development-not-secret
 
 ## cut off limit for large diffs (size in bytes)
 cut_off_limit = 256000
 
 ## force https in Kallithea, fixes https redirects, assumes it's always https
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## number of commits stats will parse on each iteration
 commit_parse_limit = 25
 
 ## path to git executable
 git_path = git
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 #git_rev_filter = --branches --tags
 
 ## RSS feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## options for showing and identifying changesets
 show_sha_length = 12
 show_revision_number = false
 
 ## Canonical URL to use when creating full URLs in UI and texts.
 ## Useful when the site is available under different names or protocols.
 ## Defaults to what is provided in the WSGI environment.
 #canonical_url = https://kallithea.example.com/repos
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/<gistid>.
 ## example: http://gist.example.com/{gistid}. Empty means use the internal
 ## Kallithea url, ie. http[s]://kallithea.example.com/_admin/gists/<gistid>
 gist_alias_url =
 
 ## white list of API enabled controllers. This allows to add list of
 ## controllers to which access will be enabled by api_key. eg: to enable
 ## api access to raw_files put `FilesController:raw`, to enable access to patches
 ## add `ChangesetController:changeset_patch`. This list should be "," separated
 ## Syntax is <ControllerClass>:<function>. Check debug logs for generated names
 ## Recommended settings below are commented out:
 api_access_controllers_whitelist =
 #    ChangesetController:changeset_patch,
 #    ChangesetController:changeset_raw,
 #    FilesController:raw,
 #    FilesController:archivefile
 
 ## default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = utf8
 
 ## issue tracker for Kallithea (leave blank to disable, absent for default)
 #bugtracker = https://bitbucket.org/conservancy/kallithea/issues
 
 ## issue tracking mapping for commits messages
 ## comment out issue_pat, issue_server, issue_prefix to enable
 
 ## pattern to get the issues from commit messages
 ## default one used here is #<numbers> with a regex passive group for `#`
 ## {id} will be all groups matched from this pattern
 
 issue_pat = (?:\s*#)(\d+)
 
 ## server url to the issue, each {id} will be replaced with match
 ## fetched from the regex and {repo} is replaced with full repository name
 ## including groups {repo_name} is replaced with just name of repo
 
 issue_server_link = https://issues.example.com/{repo}/issue/{id}
 
 ## prefix to add to link to indicate it's an url
 ## #314 will be replaced by <issue_prefix><id>
 
 issue_prefix = #
 
 ## issue_pat, issue_server_link, issue_prefix can have suffixes to specify
 ## multiple patterns, to other issues server, wiki or others
 ## below an example how to create a wiki pattern
 # wiki-some-id -> https://wiki.example.com/some-id
 
 #issue_pat_wiki = (?:wiki-)(.+)
 #issue_server_link_wiki = https://wiki.example.com/{id}
 #issue_prefix_wiki = WIKI-
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently Mercurial clients have trouble with
 ## handling that. Set this variable to 403 to return HTTPForbidden
 auth_ret_code =
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = True
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = True
 
 ## extra extensions for indexing, space separated and without the leading '.'.
 # index.extensions =
 #    gemfile
 #    lock
 
 ## extra filenames for indexing, space separated
 # index.filenames =
 #    .dockerignore
 #    .editorconfig
 #    INSTALL
 #    CHANGELOG
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 
 use_celery = false
 
 ## Example: connect to the virtual host 'rabbitmqhost' on localhost as rabbitmq:
 broker.url = amqp://rabbitmq:qewqew@localhost:5672/rabbitmqhost
 
 celery.imports = kallithea.lib.celerylib.tasks
 celery.accept.content = pickle
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 
 celeryd.concurrency = 2
 celeryd.max.tasks.per.child = 1
 
 ## If true, tasks will never be sent to the queue, but executed locally instead.
 celery.always.eager = false
 
 ####################################
 ###         BEAKER CACHE        ####
 ####################################
 
 beaker.cache.data_dir = %(here)s/data/cache/data
 beaker.cache.lock_dir = %(here)s/data/cache/lock
 
 beaker.cache.regions = short_term,long_term,sql_cache_short
 
 beaker.cache.short_term.type = memory
 beaker.cache.short_term.expire = 60
 beaker.cache.short_term.key_length = 256
 
 beaker.cache.long_term.type = memory
 beaker.cache.long_term.expire = 36000
 beaker.cache.long_term.key_length = 256
 
 beaker.cache.sql_cache_short.type = memory
 beaker.cache.sql_cache_short.expire = 10
 beaker.cache.sql_cache_short.key_length = 256
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## Name of session cookie. Should be unique for a given host and path, even when running
 ## on different ports. Otherwise, cookie sessions will be shared and messed up.
 beaker.session.key = kallithea
 ## Sessions should always only be accessible by the browser, not directly by JavaScript.
 beaker.session.httponly = true
 ## Session lifetime. 2592000 seconds is 30 days.
 beaker.session.timeout = 2592000
 
 ## Server secret used with HMAC to ensure integrity of cookies.
 beaker.session.secret = development-not-secret
 ## Further, encrypt the data with AES.
 #beaker.session.encrypt_key = <key_for_encryption>
 #beaker.session.validate_key = <validation_key>
 
 ## Type of storage used for the session, current types are
 ## dbm, file, memcached, database, and memory.
 
 ## File system storage of session data. (default)
 #beaker.session.type = file
 
 ## Cookie only, store all session data inside the cookie. Requires secure secrets.
 #beaker.session.type = cookie
 
 ## Database storage of session data.
 #beaker.session.type = ext:database
 #beaker.session.sa.url = postgresql://postgres:qwe@localhost/kallithea
 #beaker.session.table_name = db_session
 
 ############################
 ## ERROR HANDLING SYSTEMS ##
 ############################
 
 ####################
 ### [appenlight] ###
 ####################
 
 ## AppEnlight is tailored to work with Kallithea, see
 ## http://appenlight.com for details how to obtain an account
 ## you must install python package `appenlight_client` to make it work
 
 ## appenlight enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 #appenlight.logging = true
 
 ## minimum log level for log capture
 #appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
 ## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 ################
 ### [sentry] ###
 ################
 
 ## sentry is a alternative open source error aggregator
 ## you must install python packages `sentry` and `raven` to enable
 
 sentry.dsn = YOUR_DNS
 sentry.servers =
 sentry.name =
 sentry.key =
 sentry.public_key =
 sentry.secret_key =
 sentry.project =
 sentry.site =
 sentry.include_paths =
 sentry.exclude_paths =
 
 ################################################################################
 ## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 ## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 ## execute malicious code after an exception is raised.                       ##
 ################################################################################
 #set debug = false
 set debug = true
 
 ##################################
 ###       LOGVIEW CONFIG       ###
 ##################################
 
 logview.sqlalchemy = #faa
 logview.pylons.templating = #bfb
 logview.pylons.util = #eee
 
 #########################################################
 ### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 #########################################################
 
 # SQLITE [default]
 sqlalchemy.url = sqlite:///%(here)s/kallithea.db?timeout=60
 
 # POSTGRESQL
 #sqlalchemy.url = postgresql://user:pass@localhost/kallithea
 
 # MySQL
 #sqlalchemy.url = mysql://user:pass@localhost/kallithea?charset=utf8
 
 # see sqlalchemy docs for others
 
 sqlalchemy.echo = false
 sqlalchemy.pool_recycle = 3600
 
 ################################
 ### ALEMBIC CONFIGURATION   ####
 ################################
 
 [alembic]
 script_location = kallithea:alembic
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 
 [loggers]
-keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer
+keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_routes]
 level = DEBUG
 handlers =
 qualname = routes.middleware
 ## "level = DEBUG" logs the route matched and routing variables.
 propagate = 1
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_templates]
 level = INFO
 handlers =
 qualname = pylons.templating
 propagate = 1
 
 [logger_kallithea]
 level = DEBUG
 handlers =
 qualname = kallithea
 propagate = 1
 
+[logger_gearbox]
+level = DEBUG
+handlers =
+qualname = gearbox
+propagate = 1
+
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_whoosh_indexer]
 level = DEBUG
 handlers =
 qualname = whoosh_indexer
 propagate = 1
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 #level = INFO
 level = DEBUG
 #formatter = generic
 formatter = color_formatter
 
 [handler_console_sql]
 class = StreamHandler
 args = (sys.stderr,)
 #level = WARN
 level = DEBUG
 #formatter = generic
 formatter = color_formatter_sql
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = kallithea.lib.colored_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = kallithea.lib.colored_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

docs/contributing.rst

@@ -1,227 +1,227 b''
 .. _contributing:
 
 =========================
 Contributing to Kallithea
 =========================
 
 Kallithea is developed and maintained by its users. Please join us and scratch
 your own itch.
 
 
 Infrastructure
 --------------
 
 The main repository is hosted on Our Own Kallithea (aka OOK) at
 https://kallithea-scm.org/repos/kallithea/, our self-hosted instance
 of Kallithea.
 
 For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The
 issue tracker is for tracking bugs, not for support, discussion, or ideas --
 please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.
 
 We use Weblate_ to translate the user interface messages into languages other
 than English. Join our project on `Hosted Weblate`_ to help us.
 To register, you can use your Bitbucket or GitHub account. See :ref:`translations`
 for more details.
 
 
 Getting started
 ---------------
 
 To get started with development::
 
         hg clone https://kallithea-scm.org/repos/kallithea
         cd kallithea
         virtualenv ../kallithea-venv
         source ../kallithea-venv/bin/activate
         pip install --upgrade pip setuptools
         pip install -e .
-        paster make-config Kallithea my.ini
-        paster setup-db my.ini --user=user --email=[e-mail] --password=password --repos=/tmp
-        paster serve my.ini --reload &
+        TODO make-config Kallithea my.ini
+        gearbox setup-db -c my.ini --user=user --email=[e-mail] --password=password --repos=/tmp
+        gearbox serve -c my.ini --reload &
         firefox http://127.0.0.1:5000/
 
 You can also start out by forking https://bitbucket.org/conservancy/kallithea
 on Bitbucket_ and create a local clone of your own fork.
 
 
 Running tests
 -------------
 
 After finishing your changes make sure all tests pass cleanly. Install the test
 dependencies, then run the testsuite by invoking ``py.test`` from the
 project root::
 
     pip install -r dev_requirements.txt
     py.test
 
 Note that testing on Python 2.6 also requires ``unittest2``.
 
 You can also use ``tox`` to run the tests with all supported Python versions
 (currently Python 2.6--2.7).
 
 When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the
 SQLite database specified there.
 
 It is possible to avoid recreating the full test database on each invocation of
 the tests, thus eliminating the initial delay. To achieve this, run the tests as::
 
-    paster serve kallithea/tests/test.ini --pid-file=test.pid --daemon
+    gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon
     KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test
     kill -9 $(cat test.pid)
 
 In these commands, the following variables are used::
 
     KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests
     KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations
 
 You can run individual tests by specifying their path as argument to py.test.
 py.test also has many more options, see `py.test -h`. Some useful options
 are::
 
     -k EXPRESSION         only run tests which match the given substring
                           expression. An expression is a python evaluable
                           expression where all names are substring-matched
                           against test names and their parent classes. Example:
     -x, --exitfirst       exit instantly on first error or failed test.
     --lf                  rerun only the tests that failed at the last run (or
                           all if none failed)
     --ff                  run all tests but run the last failures first. This
                           may re-order tests and thus lead to repeated fixture
                           setup/teardown
     --pdb                 start the interactive Python debugger on errors.
     -s, --capture=no      don't capture stdout (any stdout output will be
                           printed immediately)
 
 
 Contribution guidelines
 -----------------------
 
 Kallithea is GPLv3 and we assume all contributions are made by the
 committer/contributor and under GPLv3 unless explicitly stated. We do care a
 lot about preservation of copyright and license information for existing code
 that is brought into the project.
 
 Contributions will be accepted in most formats -- such as pull requests on
 Bitbucket, something hosted on your own Kallithea instance, or patches sent by
 email to the `kallithea-general`_ mailing list.
 
 When contributing via Bitbucket, please make your fork of
 https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of
 the settings on "Repository details" page. This ensures your commits are in
 "draft" phase and makes it easier for you to address feedback and for project
 maintainers to integrate your changes.
 
 .. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository
 
 Make sure to test your changes both manually and with the automatic tests
 before posting.
 
 We care about quality and review and keeping a clean repository history. We
 might give feedback that requests polishing contributions until they are
 "perfect". We might also rebase and collapse and make minor adjustments to your
 changes when we apply them.
 
 We try to make sure we have consensus on the direction the project is taking.
 Everything non-sensitive should be discussed in public -- preferably on the
 mailing list.  We aim at having all non-trivial changes reviewed by at least
 one other core developer before pushing. Obvious non-controversial changes will
 be handled more casually.
 
 For now we just have one official branch ("default") and will keep it so stable
 that it can be (and is) used in production. Experimental changes should live
 elsewhere (for example in a pull request) until they are ready.
 
 
 Coding guidelines
 -----------------
 
 We don't have a formal coding/formatting standard. We are currently using a mix
 of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and
 consistency with existing code. Run ``scripts/run-all-cleanup`` before
 committing to ensure some basic code formatting consistency.
 
 We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care
 about Python 3 compatibility.
 
 We try to support the most common modern web browsers. IE9 is still supported
 to the extent it is feasible, IE8 is not.
 
 We primarily support Linux and OS X on the server side but Windows should also work.
 
 HTML templates should use 2 spaces for indentation ... but be pragmatic. We
 should use templates cleverly and avoid duplication. We should use reasonable
 semantic markup with element classes and IDs that can be used for styling and testing.
 We should only use inline styles in places where it really is semantic (such as
 ``display: none``).
 
 JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline
 multiline functions should be indented two levels -- one for the ``()`` and one for
 ``{}``.
 Variables holding jQuery objects should be named with a leading ``$``.
 
 Commit messages should have a leading short line summarizing the changes. For
 bug fixes, put ``(Issue #123)`` at the end of this line.
 
 Use American English grammar and spelling overall. Use `English title case`_ for
 page titles, button labels, headers, and 'labels' for fields in forms.
 
 .. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case
 
 Template helpers (that is, everything in ``kallithea.lib.helpers``)
 should only be referenced from templates. If you need to call a
 helper from the Python code, consider moving the function somewhere
 else (e.g. to the model).
 
 Notes on the SQLAlchemy session
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Each HTTP request runs inside an independent SQLAlchemy session (as well
 as in an independent database transaction). Database model objects
 (almost) always belong to a particular SQLAlchemy session, which means
 that SQLAlchemy will ensure that they're kept in sync with the database
 (but also means that they cannot be shared across requests).
 
 Objects can be added to the session using ``Session().add``, but this is
 rarely needed:
 
 * When creating a database object by calling the constructor directly,
   it must explicitly be added to the session.
 
 * When creating an object using a factory function (like
   ``create_repo``), the returned object has already (by convention)
   been added to the session, and should not be added again.
 
 * When getting an object from the session (via ``Session().query`` or
   any of the utility functions that look up objects in the database),
   it's already part of the session, and should not be added again.
   SQLAlchemy monitors attribute modifications automatically for all
   objects it knows about and syncs them to the database.
 
 SQLAlchemy also flushes changes to the database automatically; manually
 calling ``Session().flush`` is usually only necessary when the Python
 code needs the database to assign an "auto-increment" primary key ID to
 a freshly created model object (before flushing, the ID attribute will
 be ``None``).
 
 
 "Roadmap"
 ---------
 
 We do not have a road map but are waiting for your contributions. Refer to the
 wiki_ for some ideas of places we might want to go -- contributions in these
 areas are very welcome.
 
 
 Thank you for your contribution!
 --------------------------------
 
 
 .. _Weblate: http://weblate.org/
 .. _issue tracking: https://bitbucket.org/conservancy/kallithea/issues?status=new&status=open
 .. _pull requests: https://bitbucket.org/conservancy/kallithea/pull-requests
 .. _bitbucket: http://bitbucket.org/
 .. _mailing list: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
 .. _kallithea-general: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
 .. _Hosted Weblate: https://hosted.weblate.org/projects/kallithea/kallithea/
 .. _wiki: https://bitbucket.org/conservancy/kallithea/wiki/Home

docs/dev/dbmigrations.rst

@@ -1,74 +1,74 b''
 =======================
 Database schema changes
 =======================
 
 Kallithea uses Alembic for :ref:`database migrations <upgrade_db>`
 (upgrades and downgrades).
 
 If you are developing a Kallithea feature that requires database schema
 changes, you should make a matching Alembic database migration script:
 
 1. :ref:`Create a Kallithea configuration and database <setup>` for testing
    the migration script, or use existing ``development.ini`` setup.
 
    Ensure that this database is up to date with the latest database
    schema *before* the changes you're currently developing. (Do not
    create the database while your new schema changes are applied.)
 
 2. Create a separate throwaway configuration for iterating on the actual
    database changes::
 
-    paster make-config Kallithea temp.ini
+    TODO make-config Kallithea temp.ini
 
    Edit the file to change database settings. SQLite is typically fine,
    but make sure to change the path to e.g. ``temp.db``, to avoid
    clobbering any existing database file.
 
 3. Make your code changes (including database schema changes in ``db.py``).
 
 4. After every database schema change, recreate the throwaway database
    to test the changes::
 
     rm temp.db
-    paster setup-db temp.ini --repos=/var/repos --user=doe --email [e-mail] --password=123456 --no-public-access --force-yes
-    paster repo-scan temp.ini
+    gearbox setup-db -c temp.ini --repos=/var/repos --user=doe --email [e-mail] --password=123456 --no-public-access --force-yes
+    gearbox repo-scan -c temp.ini
 
 5. Once satisfied with the schema changes, auto-generate a draft Alembic
    script using the development database that has *not* been upgraded.
    (The generated script will upgrade the database to match the code.)
 
    ::
 
     alembic -c development.ini revision -m "area: add cool feature" --autogenerate
 
 6. Edit the script to clean it up and fix any problems.
 
    Note that for changes that simply add columns, it may be appropriate
    to not remove them in the downgrade script (and instead do nothing),
    to avoid the loss of data. Unknown columns will simply be ignored by
    Kallithea versions predating your changes.
 
 7. Run ``alembic -c development.ini upgrade head`` to apply changes to
    the (non-throwaway) database, and test the upgrade script. Also test
    downgrades.
 
    The included ``development.ini`` has full SQL logging enabled. If
    you're using another configuration file, you may want to enable it
    by setting ``level = DEBUG`` in section ``[handler_console_sql]``.
 
 The Alembic migration script should be committed in the same revision as
 the database schema (``db.py``) changes.
 
 See the `Alembic documentation`__ for more information, in particular
 the tutorial and the section about auto-generating migration scripts.
 
 .. __: http://alembic.zzzcomputing.com/en/latest/
 
 
 Troubleshooting
 ---------------
 
 * If ``alembic --autogenerate`` responds "Target database is not up to
   date", you need to either first use Alembic to upgrade the database
   to the most recent version (before your changes), or recreate the
   database from scratch (without your schema changes applied).

docs/installation_iis.rst

@@ -1,129 +1,129 b''
 .. _installation_iis:
 
 =====================================================================
 Installing Kallithea on Microsoft Internet Information Services (IIS)
 =====================================================================
 
 The following is documented using IIS 7/8 terminology. There should be nothing
 preventing you from applying this on IIS 6 well.
 
 .. note::
 
     Installing Kallithea under IIS can enable Single Sign-On to the Kallithea
     web interface from web browsers that can authenticate to the web server.
     (As an alternative to IIS, SSO is also possible with for example Apache and
     mod_sspi.)
 
     Mercurial and Git do however by default not support SSO on the client side
     and will still require some other kind of authentication.
     (An extension like hgssoauthentication_ might solve that.)
 
 .. note::
 
     For the best security, it is strongly recommended to only host the site over
     a secure connection, e.g. using TLS.
 
 
 Prerequisites
 -------------
 
 Apart from the normal requirements for Kallithea, it is also necessary to get an
 ISAPI-WSGI bridge module, e.g. isapi-wsgi.
 
 
 Installation
 ------------
 
 The following assumes that your Kallithea is at ``c:\inetpub\kallithea``, and
 will be served from the root of its own website. The changes to serve it in its
 own virtual folder will be noted where appropriate.
 
 Application pool
 ^^^^^^^^^^^^^^^^
 
 Make sure that there is a unique application pool for the Kallithea application
 with an identity that has read access to the Kallithea distribution.
 
 The application pool does not need to be able to run any managed code. If you
 are using a 32-bit Python installation, then you must enable 32-bit program in
 the advanced settings for the application pool; otherwise Python will not be able
 to run on the website and neither will Kallithea.
 
 .. note::
 
     The application pool can be the same as an existing application pool,
     as long as the Kallithea requirements are met by the existing pool.
 
 ISAPI handler
 ^^^^^^^^^^^^^
 
 The ISAPI handler can be generated using::
 
-    paster install-iis my.ini --virtualdir=/
+    gearbox install-iis -c my.ini --virtualdir=/
 
 This will generate a ``dispatch.py`` file in the current directory that contains
 the necessary components to finalize an installation into IIS. Once this file
 has been generated, it is necessary to run the following command due to the way
 that ISAPI-WSGI is made::
 
     python2 dispatch.py install
 
 This accomplishes two things: generating an ISAPI compliant DLL file,
 ``_dispatch.dll``, and installing a script map handler into IIS for the
 ``--virtualdir`` specified above pointing to ``_dispatch.dll``.
 
 The ISAPI handler is registered to all file extensions, so it will automatically
 be the one handling all requests to the specified virtual directory. When the website starts
-the ISAPI handler, it will start a thread pool managed wrapper around the paster
+the ISAPI handler, it will start a thread pool managed wrapper around the
 middleware WSGI handler that Kallithea runs within and each HTTP request to the
 site will be processed through this logic henceforth.
 
 Authentication with Kallithea using IIS authentication modules
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The recommended way to handle authentication with Kallithea using IIS is to let
 IIS handle all the authentication and just pass it to Kallithea.
 
 .. note::
 
     As an alternative without SSO, you can also use LDAP authentication with
     Active Directory, see :ref:`ldap-setup`.
 
 To move responsibility into IIS from Kallithea, we need to configure Kallithea
 to let external systems handle authentication and then let Kallithea create the
 user automatically. To do this, access the administration's authentication page
 and enable the ``kallithea.lib.auth_modules.auth_container`` plugin. Once it is
 added, enable it with the ``REMOTE_USER`` header and check *Clean username*.
 Finally, save the changes on this page.
 
 Switch to the administration's permissions page and disable anonymous access,
 otherwise Kallithea will not attempt to use the authenticated user name. By
 default, Kallithea will populate the list of users lazily as they log in. Either
 disable external auth account activation and ensure that you pre-populate the
 user database with an external tool, or set it to *Automatic activation of
 external account*. Finally, save the changes.
 
 The last necessary step is to enable the relevant authentication in IIS, e.g.
 Windows authentication.
 
 
 Troubleshooting
 ---------------
 
 Typically, any issues in this setup will either be entirely in IIS or entirely
-in Kallithea (or Kallithea's WSGI/paster middleware). Consequently, two
+in Kallithea (or Kallithea's WSGI middleware). Consequently, two
 different options for finding issues exist: IIS' failed request tracking which
 is great at finding issues until they exist inside Kallithea, at which point the
 ISAPI-WSGI wrapper above uses ``win32traceutil``, which is part of ``pywin32``.
 
 In order to dump output from WSGI using ``win32traceutil`` it is sufficient to
 type the following in a console window::
 
     python2 -m win32traceutil
 
 and any exceptions occurring in the WSGI layer and below (i.e. in the Kallithea
 application itself) that are uncaught, will be printed here complete with stack
 traces, making it a lot easier to identify issues.
 
 
 .. _hgssoauthentication: https://bitbucket.org/domruf/hgssoauthentication

docs/installation_win.rst

@@ -1,223 +1,223 b''
 .. _installation_win:
 
 ====================================================
 Installation on Windows (7/Server 2008 R2 and newer)
 ====================================================
 
 
 First time install
 ------------------
 
 Target OS: Windows 7 and newer or Windows Server 2008 R2 and newer
 
 Tested on Windows 8.1, Windows Server 2008 R2 and Windows Server 2012
 
 To install on an older version of Windows, see `<installation_win_old.html>`_
 
 Step 1 -- Install Python
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 Install Python 2.x.y (x = 6 or 7). Latest version is recommended. If you need another version, they can run side by side.
 
 .. warning:: Python 3.x is not supported.
 
 - Download Python 2.x.y from http://www.python.org/download/
 - Choose and click on the version
 - Click on "Windows X86-64 Installer" for x64 or "Windows x86 MSI installer" for Win32.
 - Disable UAC or run the installer with admin privileges. If you chose to disable UAC, do not forget to reboot afterwards.
 
 While writing this guide, the latest version was v2.7.9.
 Remember the specific major and minor versions installed, because they will
 be needed in the next step. In this case, it is "2.7".
 
 Step 2 -- Python BIN
 ^^^^^^^^^^^^^^^^^^^^
 
 Add Python BIN folder to the path. This can be done manually (editing
 "PATH" environment variable) or by using Windows Support Tools that
 come pre-installed in Windows Vista/7 and later.
 
 Open a CMD and type::
 
   SETX PATH "%PATH%;[your-python-path]" /M
 
 Please substitute [your-python-path] with your Python installation
 path. Typically this is ``C:\\Python27``.
 
 Step 3 -- Install pywin32 extensions
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Download pywin32 from:
 http://sourceforge.net/projects/pywin32/files/
 
 - Click on "pywin32" folder
 - Click on the first folder (in this case, Build 219, maybe newer when you try)
 - Choose the file ending with ".amd64-py2.x.exe" (".win32-py2.x.exe"
   for Win32) where x is the minor version of Python you installed.
   When writing this guide, the file was:
   http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/pywin32-219.win-amd64-py2.7.exe/download
   (x64)
   http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/pywin32-219.win32-py2.7.exe/download
   (Win32)
 
 Step 4 -- Install pip
 ^^^^^^^^^^^^^^^^^^^^^
 
 pip is a package management system for Python. You will need it to install Kallithea and its dependencies.
 
 If you installed Python 2.7.9+, you already have it (as long as you ran the installer with admin privileges or disabled UAC).
 
 If it was not installed or if you are using Python>=2.6,<2.7.9:
 
 - Go to https://bootstrap.pypa.io
 - Right-click on get-pip.py and choose Saves as...
 - Run "python2 get-pip.py" in the folder where you downloaded get-pip.py (may require admin access).
 
 .. note::
 
    See http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows
    for details and alternative methods.
 
 Note that pip.exe will be placed inside your Python installation's
 Scripts folder, which is likely not on your path. To correct this,
 open a CMD and type::
 
   SETX PATH "%PATH%;[your-python-path]\Scripts" /M
 
 Step 5 -- Kallithea folder structure
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Create a Kallithea folder structure.
 
 This is only an example to install Kallithea. Of course, you can
 change it. However, this guide will follow the proposed structure, so
 please later adapt the paths if you change them. Folders without
 spaces are recommended.
 
 Create the following folder structure::
 
   C:\Kallithea
   C:\Kallithea\Bin
   C:\Kallithea\Env
   C:\Kallithea\Repos
 
 Step 6 -- Install virtualenv
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. note::
    A python virtual environment will allow for isolation between the Python packages of your system and those used for Kallithea.
    It is strongly recommended to use it to ensure that Kallithea does not change a dependency that other software uses or vice versa.
 
 In a command prompt type::
 
   pip install virtualenv
 
 Virtualenv will now be inside your Python Scripts path (C:\\Python27\\Scripts or similar).
 
 To create a virtual environment, run::
 
   virtualenv C:\Kallithea\Env
 
 Step 7 -- Install Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In order to install Kallithea, you need to be able to run "pip install kallithea". It will use pip to install the Kallithea Python package and its dependencies.
 Some Python packages use managed code and need to be compiled.
 This can be done on Linux without any special steps. On Windows, you will need to install Microsoft Visual C++ compiler for Python 2.7.
 
 Download and install "Microsoft Visual C++ Compiler for Python 2.7" from http://aka.ms/vcpython27
 
 .. note::
   You can also install the dependencies using already compiled Windows binaries packages. A good source of compiled Python packages is http://www.lfd.uci.edu/~gohlke/pythonlibs/. However, not all of the necessary packages for Kallithea are on this site and some are hard to find, so we will stick with using the compiler.
 
 In a command prompt type (adapting paths if necessary)::
 
   cd C:\Kallithea\Env\Scripts
   activate
   pip install --upgrade pip setuptools
 
 The prompt will change into "(Env) C:\\Kallithea\\Env\\Scripts" or similar
 (depending of your folder structure). Then type::
 
   pip install kallithea
 
 .. note:: This will take some time. Please wait patiently until it is fully
           complete. Some warnings will appear. Don't worry, they are
           normal.
 
 Step 8 -- Install Git (optional)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Mercurial being a python package, was installed automatically when doing ``pip install kallithea``.
 
 You need to install Git manually if you want Kallithea to be able to host Git repositories.
 See http://git-scm.com/book/en/v2/Getting-Started-Installing-Git#Installing-on-Windows for instructions.
 The location of the Git binaries (like ``c:\path\to\git\bin``) must be
 added to the ``PATH`` environment variable so ``git.exe`` and other tools like
 ``gzip.exe`` are available.
 
 Step 9 -- Configuring Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Steps taken from `<setup.html>`_
 
 You have to use the same command prompt as in Step 7, so if you closed
 it, reopen it following the same commands (including the "activate"
 one). When ready, type::
 
   cd C:\Kallithea\Bin
-  paster make-config Kallithea production.ini
+  TODO make-config Kallithea production.ini
 
 Then you must edit production.ini to fit your needs (IP address, IP
 port, mail settings, database, etc.). `NotePad++`__ or a similar text
 editor is recommended to properly handle the newline character
 differences between Unix and Windows.
 
 __ http://notepad-plus-plus.org/
 
 For the sake of simplicity, run it with the default settings. After your edits (if any) in the previous command prompt, type::
 
-  paster setup-db production.ini
+  gearbox setup-db -c production.ini
 
 .. warning:: This time a *new* database will be installed. You must
              follow a different process to later :ref:`upgrade <upgrade>`
              to a newer Kallithea version.
 
 The script will ask you for confirmation about creating a new database, answer yes (y)
 
 The script will ask you for the repository path, answer C:\\Kallithea\\Repos (or similar).
 
 The script will ask you for the admin username and password, answer "admin" + "123456" (or whatever you want)
 
 The script will ask you for admin mail, answer "[e-mail]" (or whatever you want).
 
 If you make a mistake and the script doesn't end, don't worry: start it again.
 
 If you decided not to install Git, you will get errors about it that you can ignore.
 
 Step 10 -- Running Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In the previous command prompt, being in the C:\\Kallithea\\Bin folder, type::
 
-  paster serve production.ini
+  gearbox serve -c production.ini
 
 Open your web server, and go to http://127.0.0.1:5000
 
 It works!! :-)
 
 Remark:
 If it does not work the first time, Ctrl-C the CMD process and start it again. Don't forget the "http://" in Internet Explorer.
 
 What this guide does not cover:
 
 - Installing Celery
 - Running Kallithea as a Windows Service. You can investigate here:
 
   - http://pypi.python.org/pypi/wsgisvc
   - http://ryrobes.com/python/running-python-scripts-as-a-windows-service/
   - http://wiki.pylonshq.com/display/pylonscookbook/How+to+run+Pylons+as+a+Windows+service
 
 - Using Apache. You can investigate here:
 
   - https://groups.google.com/group/rhodecode/msg/c433074e813ffdc4

docs/installation_win_old.rst

@@ -1,263 +1,263 b''
 .. _installation_win_old:
 
 ==========================================================
 Installation on Windows (XP/Vista/Server 2003/Server 2008)
 ==========================================================
 
 
 First-time install
 ------------------
 
 Target OS: Windows XP SP3 32-bit English (Clean installation)
 + All Windows Updates until 24-may-2012
 
 .. note::
 
    This installation is for 32-bit systems, for 64-bit Windows you might need
    to download proper 64-bit versions of the different packages (Windows Installer, Win32py extensions)
    plus some extra tweaks.
    These extra steps haven been marked as "64-bit".
    Tested on Windows Server 2008 R2 SP1, 9-feb-2013.
    If you run into any 64-bit related problems, please check these pages:
 
    - http://blog.victorjabur.com/2011/06/05/compiling-python-2-7-modules-on-windows-32-and-64-using-msvc-2008-express/
    - http://bugs.python.org/issue7511
 
 Step 1 -- Install Visual Studio 2008 Express
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Optional: You can also install MinGW, but VS2008 installation is easier.
 
 Download "Visual C++ 2008 Express Edition with SP1" from:
 http://download.microsoft.com/download/E/8/E/E8EEB394-7F42-4963-A2D8-29559B738298/VS2008ExpressWithSP1ENUX1504728.iso
 (if not found or relocated, google for "visual studio 2008 express" for updated link. This link was taken from http://stackoverflow.com/questions/15318560/visual-c-2008-express-download-link-dead)
 
 You can also download full ISO file for offline installation, just
 choose "All -- Offline Install ISO image file" in the previous page and
 choose "Visual C++ 2008 Express" when installing.
 
 .. note::
 
    Using other versions of Visual Studio will lead to random crashes.
    You must use Visual Studio 2008!"
 
 .. note::
 
    Silverlight Runtime and SQL Server 2008 Express Edition are not
    required, you can uncheck them
 
 .. note::
 
    64-bit: You also need to install the Microsoft Windows SDK for .NET 3.5 SP1 (.NET 4.0 won't work).
    Download from: http://www.microsoft.com/en-us/download/details.aspx?id=3138
 
 .. note::
 
    64-bit: You also need to copy and rename a .bat file to make the Visual C++ compiler work.
    I am not sure why this is not necessary for 32-bit.
    Copy C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat to C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat
 
 Step 2 -- Install Python
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 Install Python 2.x.y (x = 6 or 7) x86 version (32-bit). DO NOT USE A 3.x version.
 Download Python 2.x.y from:
 http://www.python.org/download/
 
 Choose "Windows Installer" (32-bit version) not "Windows X86-64
 Installer". While writing this guide, the latest version was v2.7.3.
 Remember the specific major and minor version installed, because it will
 be needed in the next step. In this case, it is "2.7".
 
 .. note::
 
    64-bit: Just download and install the 64-bit version of python.
 
 Step 3 -- Install Win32py extensions
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Download pywin32 from:
 http://sourceforge.net/projects/pywin32/files/
 
 - Click on "pywin32" folder
 - Click on the first folder (in this case, Build 217, maybe newer when you try)
 - Choose the file ending with ".win32-py2.x.exe" -> x being the minor
   version of Python you installed (in this case, 7)
   When writing this guide, the file was:
   http://sourceforge.net/projects/pywin32/files/pywin32/Build%20217/pywin32-217.win32-py2.7.exe/download
 
   .. note::
 
      64-bit: Download and install the 64-bit version.
      At the time of writing you can find this at:
      http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/pywin32-218.win-amd64-py2.7.exe/download
 
 Step 4 -- Python BIN
 ^^^^^^^^^^^^^^^^^^^^
 
 Add Python BIN folder to the path
 
 You have to add the Python folder to the path, you can do it manually
 (editing "PATH" environment variable) or using Windows Support Tools
 that came preinstalled in Vista/7 and can be installed in Windows XP.
 
 - Using support tools on WINDOWS XP:
   If you use Windows XP you can install them using Windows XP CD and
   navigating to \SUPPORT\TOOLS. There, execute Setup.EXE (not MSI).
   Afterwards, open a CMD and type::
 
     SETX PATH "%PATH%;[your-python-path]" -M
 
   Close CMD (the path variable will be updated then)
 
 - Using support tools on WINDOWS Vista/7:
 
   Open a CMD and type::
 
     SETX PATH "%PATH%;[your-python-path]" /M
 
   Please substitute [your-python-path] with your Python installation path.
   Typically: C:\\Python27
 
 Step 5 -- Kallithea folder structure
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Create a Kallithea folder structure
 
 This is only a example to install Kallithea, you can of course change
 it. However, this guide will follow the proposed structure, so please
 later adapt the paths if you change them. My recommendation is to use
 folders with NO SPACES. But you can try if you are brave...
 
 Create the following folder structure::
 
   C:\Kallithea
   C:\Kallithea\Bin
   C:\Kallithea\Env
   C:\Kallithea\Repos
 
 Step 6 -- Install virtualenv
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Install Virtual Env for Python
 
 Navigate to: http://www.virtualenv.org/en/latest/index.html#installation
 Right click on "virtualenv.py" file and choose "Save link as...".
 Download to C:\\Kallithea (or whatever you want)
 (the file is located at
 https://raw.github.com/pypa/virtualenv/master/virtualenv.py)
 
 Create a virtual Python environment in C:\\Kallithea\\Env (or similar). To
 do so, open a CMD (Python Path should be included in Step3), navigate
 where you downloaded "virtualenv.py", and write::
 
   python2 virtualenv.py C:\Kallithea\Env
 
 (--no-site-packages is now the default behaviour of virtualenv, no need
 to include it)
 
 Step 7 -- Install Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Finally, install Kallithea
 
 Close previously opened command prompt/s, and open a Visual Studio 2008
 Command Prompt (**IMPORTANT!!**). To do so, go to Start Menu, and then open
 "Microsoft Visual C++ 2008 Express Edition" -> "Visual Studio Tools" ->
 "Visual Studio 2008 Command Prompt"
 
 .. note::
 
    64-bit: For 64-bit you need to modify the shortcut that is used to start the
    Visual Studio 2008 Command Prompt. Use right-mouse click to open properties.
 
 Change commandline from::
 
 %comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" x86
 
 to::
 
 %comspec% /k ""C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"" amd64
 
 In that CMD (loaded with VS2008 PATHs) type::
 
   cd C:\Kallithea\Env\Scripts (or similar)
   activate
   pip install --upgrade pip setuptools
 
 The prompt will change into "(Env) C:\\Kallithea\\Env\\Scripts" or similar
 (depending of your folder structure). Then type::
 
  pip install kallithea
 
 (long step, please wait until fully complete)
 
 Some warnings will appear, don't worry as they are normal.
 
 Step 8 -- Configuring Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 steps taken from http://packages.python.org/Kallithea/setup.html
 
 You have to use the same Visual Studio 2008 command prompt as Step7, so
 if you closed it reopen it following the same commands (including the
 "activate" one). When ready, just type::
 
   cd C:\Kallithea\Bin
-  paster make-config Kallithea production.ini
+  TODO make-config Kallithea production.ini
 
 Then, you must edit production.ini to fit your needs (network address and
 port, mail settings, database, whatever). I recommend using NotePad++
 (free) or similar text editor, as it handles well the EndOfLine
 character differences between Unix and Windows
 (http://notepad-plus-plus.org/)
 
 For the sake of simplicity lets run it with the default settings. After
 your edits (if any), in the previous Command Prompt, type::
 
- paster setup-db production.ini
+ gearbox setup-db -c production.ini
 
 .. warning:: This time a *new* database will be installed. You must
              follow a different process to later :ref:`upgrade <upgrade>`
              to a newer Kallithea version.
 
 The script will ask you for confirmation about creating a NEW database,
 answer yes (y)
 The script will ask you for repository path, answer C:\\Kallithea\\Repos
 (or similar)
 The script will ask you for admin username and password, answer "admin"
 + "123456" (or whatever you want)
 The script will ask you for admin mail, answer "[e-mail]" (or
 whatever you want)
 
 If you make some mistake and the script does not end, don't worry, start
 it again.
 
 Step 9 -- Running Kallithea
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In the previous command prompt, being in the C:\\Kallithea\\Bin folder,
 just type::
 
- paster serve production.ini
+ gearbox serve -c production.ini
 
 Open yout web server, and go to http://127.0.0.1:5000
 
 It works!! :-)
 
 Remark:
 If it does not work first time, just Ctrl-C the CMD process and start it
 again. Don't forget the "http://" in Internet Explorer
 
 What this Guide does not cover:
 
 - Installing Celery
 - Running Kallithea as Windows Service. You can investigate here:
 
   - http://pypi.python.org/pypi/wsgisvc
   - http://ryrobes.com/python/running-python-scripts-as-a-windows-service/
   - http://wiki.pylonshq.com/display/pylonscookbook/How+to+run+Pylons+as+a+Windows+service
 
 - Using Apache. You can investigate here:
 
   - https://groups.google.com/group/rhodecode/msg/c433074e813ffdc4

docs/overview.rst

@@ -1,140 +1,140 b''
 .. _overview:
 
 =====================
 Installation overview
 =====================
 
 Some overview and some details that can help understanding the options when
 installing Kallithea.
 
 
 Python environment
 ------------------
 
 **Kallithea** is written entirely in Python_ and requires Python version
 2.6 or higher. Python 3.x is currently not supported.
 
 Given a Python installation, there are different ways of providing the
 environment for running Python applications. Each of them pretty much
 corresponds to a ``site-packages`` directory somewhere where packages can be
 installed.
 
 Kallithea itself can be run from source or be installed, but even when running
 from source, there are some dependencies that must be installed in the Python
 environment used for running Kallithea.
 
 - Packages *could* be installed in Python's ``site-packages`` directory ... but
   that would require running pip_ as root and it would be hard to uninstall or
   upgrade and is probably not a good idea unless using a package manager.
 
 - Packages could also be installed in ``~/.local`` ... but that is probably
   only a good idea if using a dedicated user per application or instance.
 
 - Finally, it can be installed in a virtualenv_. That is a very lightweight
   "container" where each Kallithea instance can get its own dedicated and
   self-contained virtual environment.
 
 We recommend using virtualenv for installing Kallithea.
 
 
 Installation methods
 --------------------
 
 Kallithea must be installed on a server. Kallithea is installed in a Python
 environment so it can use packages that are installed there and make itself
 available for other packages.
 
 Two different cases will pretty much cover the options for how it can be
 installed.
 
 - The Kallithea source repository can be cloned and used -- it is kept stable and
   can be used in production. The Kallithea maintainers use the development
   branch in production. The advantage of installation from source and regularly
   updating it is that you take advantage of the most recent improvements. Using
   it directly from a DVCS also means that it is easy to track local customizations.
 
   Running ``pip install -e .`` in the source will use pip to install the
   necessary dependencies in the Python environment and create a
   ``.../site-packages/Kallithea.egg-link`` file there that points at the Kallithea
   source.
 
 - Kallithea can also be installed from ready-made packages using a package manager.
   The official released versions are available on PyPI_ and can be downloaded and
   installed with all dependencies using ``pip install kallithea``.
 
   With this method, Kallithea is installed in the Python environment as any
   other package, usually as a ``.../site-packages/Kallithea-X-py2.7.egg/``
   directory with Python files and everything else that is needed.
 
   (``pip install kallithea`` from a source tree will do pretty much the same
   but build the Kallithea package itself locally instead of downloading it.)
 
 
 Web server
 ----------
 
 Kallithea is (primarily) a WSGI_ application that must be run from a web
 server that serves WSGI applications over HTTP.
 
 Kallithea itself is not serving HTTP (or HTTPS); that is the web server's
 responsibility. Kallithea does however need to know its own user facing URL
 (protocol, address, port and path) for each HTTP request. Kallithea will
 usually use its own HTML/cookie based authentication but can also be configured
 to use web server authentication.
 
 There are several web server options:
 
-- Kallithea uses the Paste_ tool as command line interface. Paste provides
-  ``paster serve`` as a convenient way to launch a Python WSGI / web server
+- Kallithea uses the Gearbox_ tool as command line interface. Gearbox provides
+  ``gearbox serve`` as a convenient way to launch a Python WSGI / web server
   from the command line. That is perfect for development and evaluation.
   Actual use in production might have different requirements and need extra
   work to make it manageable as a scalable system service.
 
-  Paste comes with its own built-in web server but Kallithea defaults to use
+  Gearbox comes with its own built-in web server but Kallithea defaults to use
   Waitress_. Gunicorn_ is also an option. These web servers have different
   limited feature sets.
 
-  The web server used by ``paster`` is configured in the ``.ini`` file passed
+  The web server used by ``gearbox`` is configured in the ``.ini`` file passed
   to it. The entry point for the WSGI application is configured
   in ``setup.py`` as ``kallithea.config.middleware:make_app``.
 
 - `Apache httpd`_ can serve WSGI applications directly using mod_wsgi_ and a
   simple Python file with the necessary configuration. This is a good option if
   Apache is an option.
 
 - uWSGI_ is also a full web server with built-in WSGI module.
 
 - IIS_ can also server WSGI applications directly using isapi-wsgi_.
 
 - A `reverse HTTP proxy <https://en.wikipedia.org/wiki/Reverse_proxy>`_
   can be put in front of another web server which has WSGI support.
   Such a layered setup can be complex but might in some cases be the right
   option, for example to standardize on one internet-facing web server, to add
   encryption or special authentication or for other security reasons, to
   provide caching of static files, or to provide load balancing or fail-over.
   Nginx_, Varnish_ and HAProxy_ are often used for this purpose, often in front
-  of a ``paster`` server that somehow is wrapped as a service.
+  of a ``gearbox serve`` that somehow is wrapped as a service.
 
 The best option depends on what you are familiar with and the requirements for
 performance and stability. Also, keep in mind that Kallithea mainly is serving
 dynamically generated pages from a relatively slow Python process. Kallithea is
 also often used inside organizations with a limited amount of users and thus no
 continuous hammering from the internet.
 
 
 .. _Python: http://www.python.org/
 .. _Gunicorn: http://gunicorn.org/
 .. _Waitress: http://waitress.readthedocs.org/en/latest/
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
-.. _Paste: http://pythonpaste.org/
+.. _Gearbox: http://turbogears.readthedocs.io/en/latest/turbogears/gearbox.html
 .. _PyPI: https://pypi.python.org/pypi
 .. _Apache httpd: http://httpd.apache.org/
 .. _mod_wsgi: https://code.google.com/p/modwsgi/
 .. _isapi-wsgi: https://github.com/hexdump42/isapi-wsgi
 .. _uWSGI: https://uwsgi-docs.readthedocs.org/en/latest/
 .. _nginx: http://nginx.org/en/
 .. _iis: http://en.wikipedia.org/wiki/Internet_Information_Services
 .. _pip: http://en.wikipedia.org/wiki/Pip_%28package_manager%29
 .. _WSGI: http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
 .. _HAProxy: http://www.haproxy.org/
 .. _Varnish: https://www.varnish-cache.org/

docs/setup.rst

@@ -1,874 +1,877 b''
 .. _setup:
 
 =====
 Setup
 =====
 
 
 Setting up Kallithea
 --------------------
 
 First, you will need to create a Kallithea configuration file. Run the
 following command to do so::
 
-    paster make-config Kallithea my.ini
+    TODO make-config Kallithea my.ini
 
 This will create the file ``my.ini`` in the current directory. This
 configuration file contains the various settings for Kallithea, e.g.
 proxy port, email settings, usage of static files, cache, Celery
 settings, and logging.
 
 Next, you need to create the databases used by Kallithea. It is recommended to
 use PostgreSQL or SQLite (default). If you choose a database other than the
 default, ensure you properly adjust the database URL in your ``my.ini``
 configuration file to use this other database. Kallithea currently supports
 PostgreSQL, SQLite and MySQL databases. Create the database by running
 the following command::
 
-    paster setup-db my.ini
+    gearbox setup-db -c my.ini
 
 This will prompt you for a "root" path. This "root" path is the location where
 Kallithea will store all of its repositories on the current machine. After
 entering this "root" path ``setup-db`` will also prompt you for a username
 and password for the initial admin account which ``setup-db`` sets
 up for you.
 
 The ``setup-db`` values can also be given on the command line.
 Example::
 
-    paster setup-db my.ini --user=nn --password=secret --email=[e-mail] --repos=/srv/repos
+    gearbox setup-db -c my.ini --user=nn --password=secret --email=[e-mail] --repos=/srv/repos
 
 The ``setup-db`` command will create all needed tables and an
 admin account. When choosing a root path you can either use a new
 empty location, or a location which already contains existing
 repositories. If you choose a location which contains existing
 repositories Kallithea will add all of the repositories at the chosen
 location to its database.  (Note: make sure you specify the correct
 path to the root).
 
 .. note:: the given path for Mercurial_ repositories **must** be write
           accessible for the application. It's very important since
           the Kallithea web interface will work without write access,
           but when trying to do a push it will fail with permission
           denied errors unless it has write access.
 
 You are now ready to use Kallithea. To run it simply execute::
 
-    paster serve my.ini
+    gearbox serve -c my.ini
 
 - This command runs the Kallithea server. The web app should be available at
   http://127.0.0.1:5000. The IP address and port is configurable via the
   configuration file created in the previous step.
 - Log in to Kallithea using the admin account created when running ``setup-db``.
 - The default permissions on each repository is read, and the owner is admin.
   Remember to update these if needed.
 - In the admin panel you can toggle LDAP, anonymous, and permissions
   settings, as well as edit more advanced options on users and
   repositories.
 
 
 Using Kallithea with SSH
 ------------------------
 
 Kallithea currently only hosts repositories using http and https. (The addition
 of ssh hosting is a planned future feature.) However you can easily use ssh in
 parallel with Kallithea. (Repository access via ssh is a standard "out of
 the box" feature of Mercurial_ and you can use this to access any of the
 repositories that Kallithea is hosting. See PublishingRepositories_)
 
 Kallithea repository structures are kept in directories with the same name
 as the project. When using repository groups, each group is a subdirectory.
 This allows you to easily use ssh for accessing repositories.
 
 In order to use ssh you need to make sure that your web server and the users'
 login accounts have the correct permissions set on the appropriate directories.
 
 .. note:: These permissions are independent of any permissions you
           have set up using the Kallithea web interface.
 
 If your main directory (the same as set in Kallithea settings) is for
 example set to ``/srv/repos`` and the repository you are using is
 named ``kallithea``, then to clone via ssh you should run::
 
     hg clone ssh://[e-mail]/srv/repos/kallithea
 
 Using other external tools such as mercurial-server_ or using ssh key-based
 authentication is fully supported.
 
 .. note:: In an advanced setup, in order for your ssh access to use
           the same permissions as set up via the Kallithea web
           interface, you can create an authentication hook to connect
           to the Kallithea db and run check functions for permissions
           against that.
 
 
 Setting up Whoosh full text search
 ----------------------------------
 
 Kallithea provides full text search of repositories using `Whoosh`__.
 
 .. __: https://pythonhosted.org/Whoosh/
 
 For an incremental index build, run::
 
-    paster make-index my.ini
+    gearbox make-index -c my.ini
 
 For a full index rebuild, run::
 
-    paster make-index my.ini -f
+    gearbox make-index -c my.ini -f
 
 The ``--repo-location`` option allows the location of the repositories to be overridden;
 usually, the location is retrieved from the Kallithea database.
 
 The ``--index-only`` option can be used to limit the indexed repositories to a comma-separated list::
 
-    paster make-index my.ini --index-only=vcs,kallithea
+    gearbox make-index -c my.ini --index-only=vcs,kallithea
 
 To keep your index up-to-date it is necessary to do periodic index builds;
 for this, it is recommended to use a crontab entry. Example::
 
-    0  3  *  *  *  /path/to/virtualenv/bin/paster make-index /path/to/kallithea/my.ini
+    0  3  *  *  *  /path/to/virtualenv/bin/gearbox make-index -c /path/to/kallithea/my.ini
 
 When using incremental mode (the default), Whoosh will check the last
 modification date of each file and add it to be reindexed if a newer file is
 available. The indexing daemon checks for any removed files and removes them
 from index.
 
 If you want to rebuild the index from scratch, you can use the ``-f`` flag as above,
 or in the admin panel you can check the "build from scratch" checkbox.
 
 .. _ldap-setup:
 
 
 Setting up LDAP support
 -----------------------
 
 Kallithea supports LDAP authentication. In order
 to use LDAP, you have to install the python-ldap_ package. This package is
 available via PyPI, so you can install it by running::
 
     pip install python-ldap
 
 .. note:: ``python-ldap`` requires some libraries to be installed on
           your system, so before installing it check that you have at
           least the ``openldap`` and ``sasl`` libraries.
 
 Choose *Admin > Authentication*, click the ``kallithea.lib.auth_modules.auth_ldap`` button
 and then *Save*, to enable the LDAP plugin and configure its settings.
 
 Here's a typical LDAP setup::
 
  Connection settings
  Enable LDAP          = checked
  Host                 = host.example.com
  Account              = <account>
  Password             = <password>
  Connection Security  = LDAPS
  Certificate Checks   = DEMAND
 
  Search settings
  Base DN              = CN=users,DC=host,DC=example,DC=org
  LDAP Filter          = (&(objectClass=user)(!(objectClass=computer)))
  LDAP Search Scope    = SUBTREE
 
  Attribute mappings
  Login Attribute      = uid
  First Name Attribute = firstName
  Last Name Attribute  = lastName
  Email Attribute      = mail
 
 If your user groups are placed in an Organisation Unit (OU) structure, the Search Settings configuration differs::
 
  Search settings
  Base DN              = DC=host,DC=example,DC=org
  LDAP Filter          = (&(memberOf=CN=your user group,OU=subunit,OU=unit,DC=host,DC=example,DC=org)(objectClass=user))
  LDAP Search Scope    = SUBTREE
 
 .. _enable_ldap:
 
 Enable LDAP : required
     Whether to use LDAP for authenticating users.
 
 .. _ldap_host:
 
 Host : required
     LDAP server hostname or IP address. Can be also a comma separated
     list of servers to support LDAP fail-over.
 
 .. _Port:
 
 Port : optional
     Defaults to 389 for PLAIN un-encrypted LDAP and START_TLS.
     Defaults to 636 for LDAPS.
 
 .. _ldap_account:
 
 Account : optional
     Only required if the LDAP server does not allow anonymous browsing of
     records.  This should be a special account for record browsing.  This
     will require `LDAP Password`_ below.
 
 .. _LDAP Password:
 
 Password : optional
     Only required if the LDAP server does not allow anonymous browsing of
     records.
 
 .. _Enable LDAPS:
 
 Connection Security : required
     Defines the connection to LDAP server
 
     PLAIN
         Plain unencrypted LDAP connection.
         This will by default use `Port`_ 389.
 
     LDAPS
         Use secure LDAPS connections according to `Certificate
         Checks`_ configuration.
         This will by default use `Port`_ 636.
 
     START_TLS
         Use START TLS according to `Certificate Checks`_ configuration on an
         apparently "plain" LDAP connection.
         This will by default use `Port`_ 389.
 
 .. _Certificate Checks:
 
 Certificate Checks : optional
     How SSL certificates verification is handled -- this is only useful when
     `Enable LDAPS`_ is enabled.  Only DEMAND or HARD offer full SSL security
     with mandatory certificate validation, while the other options are
     susceptible to man-in-the-middle attacks.
 
     NEVER
         A serve certificate will never be requested or checked.
 
     ALLOW
         A server certificate is requested.  Failure to provide a
         certificate or providing a bad certificate will not terminate the
         session.
 
     TRY
         A server certificate is requested.  Failure to provide a
         certificate does not halt the session; providing a bad certificate
         halts the session.
 
     DEMAND
         A server certificate is requested and must be provided and
         authenticated for the session to proceed.
 
     HARD
         The same as DEMAND.
 
 .. _Custom CA Certificates:
 
 Custom CA Certificates : optional
     Directory used by OpenSSL to find CAs for validating the LDAP server certificate.
     Python 2.7.10 and later default to using the system certificate store, and
     this should thus not be necessary when using certificates signed by a CA
     trusted by the system.
     It can be set to something like `/etc/openldap/cacerts` on older systems or
     if using self-signed certificates.
 
 .. _Base DN:
 
 Base DN : required
     The Distinguished Name (DN) where searches for users will be performed.
     Searches can be controlled by `LDAP Filter`_ and `LDAP Search Scope`_.
 
 .. _LDAP Filter:
 
 LDAP Filter : optional
     A LDAP filter defined by RFC 2254.  This is more useful when `LDAP
     Search Scope`_ is set to SUBTREE.  The filter is useful for limiting
     which LDAP objects are identified as representing Users for
     authentication.  The filter is augmented by `Login Attribute`_ below.
     This can commonly be left blank.
 
 .. _LDAP Search Scope:
 
 LDAP Search Scope : required
     This limits how far LDAP will search for a matching object.
 
     BASE
         Only allows searching of `Base DN`_ and is usually not what you
         want.
 
     ONELEVEL
         Searches all entries under `Base DN`_, but not Base DN itself.
 
     SUBTREE
         Searches all entries below `Base DN`_, but not Base DN itself.
         When using SUBTREE `LDAP Filter`_ is useful to limit object
         location.
 
 .. _Login Attribute:
 
 Login Attribute : required
     The LDAP record attribute that will be matched as the USERNAME or
     ACCOUNT used to connect to Kallithea.  This will be added to `LDAP
     Filter`_ for locating the User object.  If `LDAP Filter`_ is specified as
     "LDAPFILTER", `Login Attribute`_ is specified as "uid" and the user has
     connected as "jsmith" then the `LDAP Filter`_ will be augmented as below
     ::
 
         (&(LDAPFILTER)(uid=jsmith))
 
 .. _ldap_attr_firstname:
 
 First Name Attribute : required
     The LDAP record attribute which represents the user's first name.
 
 .. _ldap_attr_lastname:
 
 Last Name Attribute : required
     The LDAP record attribute which represents the user's last name.
 
 .. _ldap_attr_email:
 
 Email Attribute : required
     The LDAP record attribute which represents the user's email address.
 
 If all data are entered correctly, and python-ldap_ is properly installed
 users should be granted access to Kallithea with LDAP accounts.  At this
 time user information is copied from LDAP into the Kallithea user database.
 This means that updates of an LDAP user object may not be reflected as a
 user update in Kallithea.
 
 If You have problems with LDAP access and believe You entered correct
 information check out the Kallithea logs, any error messages sent from LDAP
 will be saved there.
 
 Active Directory
 ^^^^^^^^^^^^^^^^
 
 Kallithea can use Microsoft Active Directory for user authentication.  This
 is done through an LDAP or LDAPS connection to Active Directory.  The
 following LDAP configuration settings are typical for using Active
 Directory ::
 
  Base DN              = OU=SBSUsers,OU=Users,OU=MyBusiness,DC=v3sys,DC=local
  Login Attribute      = sAMAccountName
  First Name Attribute = givenName
  Last Name Attribute  = sn
  Email Attribute     = mail
 
 All other LDAP settings will likely be site-specific and should be
 appropriately configured.
 
 
 Authentication by container or reverse-proxy
 --------------------------------------------
 
 Kallithea supports delegating the authentication
 of users to its WSGI container, or to a reverse-proxy server through which all
 clients access the application.
 
 When these authentication methods are enabled in Kallithea, it uses the
 username that the container/proxy (Apache or Nginx, etc.) provides and doesn't
 perform the authentication itself. The authorization, however, is still done by
 Kallithea according to its settings.
 
 When a user logs in for the first time using these authentication methods,
 a matching user account is created in Kallithea with default permissions. An
 administrator can then modify it using Kallithea's admin interface.
 
 It's also possible for an administrator to create accounts and configure their
 permissions before the user logs in for the first time, using the :ref:`create-user` API.
 
 Container-based authentication
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In a container-based authentication setup, Kallithea reads the user name from
 the ``REMOTE_USER`` server variable provided by the WSGI container.
 
 After setting up your container (see `Apache with mod_wsgi`_), you'll need
 to configure it to require authentication on the location configured for
 Kallithea.
 
 Proxy pass-through authentication
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In a proxy pass-through authentication setup, Kallithea reads the user name
 from the ``X-Forwarded-User`` request header, which should be configured to be
 sent by the reverse-proxy server.
 
 After setting up your proxy solution (see `Apache virtual host reverse proxy example`_,
 `Apache as subdirectory`_ or `Nginx virtual host example`_), you'll need to
 configure the authentication and add the username in a request header named
 ``X-Forwarded-User``.
 
 For example, the following config section for Apache sets a subdirectory in a
 reverse-proxy setup with basic auth:
 
 .. code-block:: apache
 
     <Location /someprefix>
       ProxyPass http://127.0.0.1:5000/someprefix
       ProxyPassReverse http://127.0.0.1:5000/someprefix
       SetEnvIf X-Url-Scheme https HTTPS=1
 
       AuthType Basic
       AuthName "Kallithea authentication"
       AuthUserFile /srv/kallithea/.htpasswd
       Require valid-user
 
       RequestHeader unset X-Forwarded-User
 
       RewriteEngine On
       RewriteCond %{LA-U:REMOTE_USER} (.+)
       RewriteRule .* - [E=RU:%1]
       RequestHeader set X-Forwarded-User %{RU}e
     </Location>
 
 Setting metadata in container/reverse-proxy
 """""""""""""""""""""""""""""""""""""""""""
 When a new user account is created on the first login, Kallithea has no information about
 the user's email and full name. So you can set some additional request headers like in the
 example below. In this example the user is authenticated via Kerberos and an Apache
 mod_python fixup handler is used to get the user information from a LDAP server. But you
 could set the request headers however you want.
 
 .. code-block:: apache
 
     <Location /someprefix>
       ProxyPass http://127.0.0.1:5000/someprefix
       ProxyPassReverse http://127.0.0.1:5000/someprefix
       SetEnvIf X-Url-Scheme https HTTPS=1
 
       AuthName "Kerberos Login"
       AuthType Kerberos
       Krb5Keytab /etc/apache2/http.keytab
       KrbMethodK5Passwd off
       KrbVerifyKDC on
       Require valid-user
 
       PythonFixupHandler ldapmetadata
 
       RequestHeader set X_REMOTE_USER %{X_REMOTE_USER}e
       RequestHeader set X_REMOTE_EMAIL %{X_REMOTE_EMAIL}e
       RequestHeader set X_REMOTE_FIRSTNAME %{X_REMOTE_FIRSTNAME}e
       RequestHeader set X_REMOTE_LASTNAME %{X_REMOTE_LASTNAME}e
     </Location>
 
 .. code-block:: python
 
     from mod_python import apache
     import ldap
 
     LDAP_SERVER = "ldaps://server.mydomain.com:636"
     LDAP_USER = ""
     LDAP_PASS = ""
     LDAP_ROOT = "dc=mydomain,dc=com"
     LDAP_FILTER = "sAMAccountName=%s"
     LDAP_ATTR_LIST = ['sAMAccountName','givenname','sn','mail']
 
     def fixuphandler(req):
         if req.user is None:
             # no user to search for
             return apache.OK
         else:
             try:
                 if('\\' in req.user):
                     username = req.user.split('\\')[1]
                 elif('@' in req.user):
                     username = req.user.split('@')[0]
                 else:
                     username = req.user
                 l = ldap.initialize(LDAP_SERVER)
                 l.simple_bind_s(LDAP_USER, LDAP_PASS)
                 r = l.search_s(LDAP_ROOT, ldap.SCOPE_SUBTREE, LDAP_FILTER % username, attrlist=LDAP_ATTR_LIST)
 
                 req.subprocess_env['X_REMOTE_USER'] = username
                 req.subprocess_env['X_REMOTE_EMAIL'] = r[0][1]['mail'][0].lower()
                 req.subprocess_env['X_REMOTE_FIRSTNAME'] = "%s" % r[0][1]['givenname'][0]
                 req.subprocess_env['X_REMOTE_LASTNAME'] = "%s" % r[0][1]['sn'][0]
             except Exception, e:
                 apache.log_error("error getting data from ldap %s" % str(e), apache.APLOG_ERR)
 
             return apache.OK
 
 .. note::
    If you enable proxy pass-through authentication, make sure your server is
    only accessible through the proxy. Otherwise, any client would be able to
    forge the authentication header and could effectively become authenticated
    using any account of their liking.
 
 
 Integration with issue trackers
 -------------------------------
 
 Kallithea provides a simple integration with issue trackers. It's possible
 to define a regular expression that will match an issue ID in commit messages,
 and have that replaced with a URL to the issue. To enable this simply
 uncomment the following variables in the ini file::
 
     issue_pat = (?:^#|\s#)(\w+)
     issue_server_link = https://issues.example.com/{repo}/issue/{id}
     issue_prefix = #
 
 ``issue_pat`` is the regular expression describing which strings in
 commit messages will be treated as issue references. A match group in
 parentheses should be used to specify the actual issue id.
 
 The default expression matches issues in the format ``#<number>``, e.g., ``#300``.
 
 Matched issue references are replaced with the link specified in
 ``issue_server_link``. ``{id}`` is replaced with the issue ID, and
 ``{repo}`` with the repository name.  Since the # is stripped away,
 ``issue_prefix`` is prepended to the link text.  ``issue_prefix`` doesn't
 necessarily need to be ``#``: if you set issue prefix to ``ISSUE-`` this will
 generate a URL in the format:
 
 .. code-block:: html
 
   <a href="https://issues.example.com/example_repo/issue/300">ISSUE-300</a>
 
 If needed, more than one pattern can be specified by appending a unique suffix to
 the variables. For example::
 
     issue_pat_wiki = (?:wiki-)(.+)
     issue_server_link_wiki = https://wiki.example.com/{id}
     issue_prefix_wiki = WIKI-
 
 With these settings, wiki pages can be referenced as wiki-some-id, and every
 such reference will be transformed into:
 
 .. code-block:: html
 
   <a href="https://wiki.example.com/some-id">WIKI-some-id</a>
 
 
 Hook management
 ---------------
 
 Hooks can be managed in similar way to that used in ``.hgrc`` files.
 To manage hooks, choose *Admin > Settings > Hooks*.
 
 The built-in hooks cannot be modified, though they can be enabled or disabled in the *VCS* section.
 
 To add another custom hook simply fill in the first textbox with
 ``<name>.<hook_type>`` and the second with the hook path. Example hooks
 can be found in ``kallithea.lib.hooks``.
 
 
 Changing default encoding
 -------------------------
 
 By default, Kallithea uses UTF-8 encoding.
 This is configurable as ``default_encoding`` in the .ini file.
 This affects many parts in Kallithea including user names, filenames, and
 encoding of commit messages. In addition Kallithea can detect if the ``chardet``
 library is installed. If ``chardet`` is detected Kallithea will fallback to it
 when there are encode/decode errors.
 
 
 Celery configuration
 --------------------
 
 Kallithea can use the distributed task queue system Celery_ to run tasks like
 cloning repositories or sending emails.
 
 Kallithea will in most setups work perfectly fine out of the box (without
 Celery), executing all tasks in the web server process. Some tasks can however
 take some time to run and it can be better to run such tasks asynchronously in
 a separate process so the web server can focus on serving web requests.
 
 For installation and configuration of Celery, see the `Celery documentation`_.
 Note that Celery requires a message broker service like RabbitMQ_ (recommended)
 or Redis_.
 
 The use of Celery is configured in the Kallithea ini configuration file.
 To enable it, simply set::
 
   use_celery = true
 
 and add or change the ``celery.*`` and ``broker.*`` configuration variables.
 
 Remember that the ini files use the format with '.' and not with '_' like
 Celery. So for example setting `BROKER_HOST` in Celery means setting
 `broker.host` in the configuration file.
 
 To start the Celery process, run::
 
- paster celeryd <configfile.ini>
+ gearbox celeryd -c <configfile.ini>
+
+Extra options to the Celery worker can be passed after ``--`` - see ``-- -h``
+for more info.
 
 .. note::
    Make sure you run this command from the same virtualenv, and with the same
    user that Kallithea runs.
 
 
 HTTPS support
 -------------
 
 Kallithea will by default generate URLs based on the WSGI environment.
 
 Alternatively, you can use some special configuration settings to control
 directly which scheme/protocol Kallithea will use when generating URLs:
 
 - With ``https_fixup = true``, the scheme will be taken from the
   ``X-Url-Scheme``, ``X-Forwarded-Scheme`` or ``X-Forwarded-Proto`` HTTP header
   (default ``http``).
 - With ``force_https = true`` the default will be ``https``.
 - With ``use_htsts = true``, Kallithea will set ``Strict-Transport-Security`` when using https.
 
 
 Nginx virtual host example
 --------------------------
 
 Sample config for Nginx using proxy:
 
 .. code-block:: nginx
 
     upstream kallithea {
         server 127.0.0.1:5000;
         # add more instances for load balancing
         #server 127.0.0.1:5001;
         #server 127.0.0.1:5002;
     }
 
     ## gist alias
     server {
        listen          443;
        server_name     gist.example.com;
        access_log      /var/log/nginx/gist.access.log;
        error_log       /var/log/nginx/gist.error.log;
 
        ssl on;
        ssl_certificate     gist.your.kallithea.server.crt;
        ssl_certificate_key gist.your.kallithea.server.key;
 
        ssl_session_timeout 5m;
 
        ssl_protocols SSLv3 TLSv1;
        ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;
        ssl_prefer_server_ciphers on;
 
        rewrite ^/(.+)$ https://kallithea.example.com/_admin/gists/$1;
        rewrite (.*)    https://kallithea.example.com/_admin/gists;
     }
 
     server {
        listen          443;
        server_name     kallithea.example.com
        access_log      /var/log/nginx/kallithea.access.log;
        error_log       /var/log/nginx/kallithea.error.log;
 
        ssl on;
        ssl_certificate     your.kallithea.server.crt;
        ssl_certificate_key your.kallithea.server.key;
 
        ssl_session_timeout 5m;
 
        ssl_protocols SSLv3 TLSv1;
        ssl_ciphers DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA:DES-CBC3-SHA:AES128-SHA:RC4-SHA:RC4-MD5;
        ssl_prefer_server_ciphers on;
 
        ## uncomment root directive if you want to serve static files by nginx
        ## requires static_files = false in .ini file
        #root /srv/kallithea/kallithea/kallithea/public;
        include         /etc/nginx/proxy.conf;
        location / {
             try_files $uri @kallithea;
        }
 
        location @kallithea {
             proxy_pass      http://127.0.0.1:5000;
        }
 
     }
 
 Here's the proxy.conf. It's tuned so it will not timeout on long
 pushes or large pushes::
 
     proxy_redirect              off;
     proxy_set_header            Host $host;
     ## needed for container auth
     #proxy_set_header            REMOTE_USER $remote_user;
     #proxy_set_header            X-Forwarded-User $remote_user;
     proxy_set_header            X-Url-Scheme $scheme;
     proxy_set_header            X-Host $http_host;
     proxy_set_header            X-Real-IP $remote_addr;
     proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header            Proxy-host $proxy_host;
     proxy_buffering             off;
     proxy_connect_timeout       7200;
     proxy_send_timeout          7200;
     proxy_read_timeout          7200;
     proxy_buffers               8 32k;
     client_max_body_size        1024m;
     client_body_buffer_size     128k;
     large_client_header_buffers 8 64k;
 
 
 Apache virtual host reverse proxy example
 -----------------------------------------
 
 Here is a sample configuration file for Apache using proxy:
 
 .. code-block:: apache
 
     <VirtualHost *:80>
             ServerName kallithea.example.com
 
             <Proxy *>
               # For Apache 2.4 and later:
               Require all granted
 
               # For Apache 2.2 and earlier, instead use:
               # Order allow,deny
               # Allow from all
             </Proxy>
 
             #important !
             #Directive to properly generate url (clone url) for Kallithea
             ProxyPreserveHost On
 
             #kallithea instance
             ProxyPass / http://127.0.0.1:5000/
             ProxyPassReverse / http://127.0.0.1:5000/
 
             #to enable https use line below
             #SetEnvIf X-Url-Scheme https HTTPS=1
     </VirtualHost>
 
 Additional tutorial
 http://pylonsbook.com/en/1.1/deployment.html#using-apache-to-proxy-requests-to-pylons
 
 
 Apache as subdirectory
 ----------------------
 
 Apache subdirectory part:
 
 .. code-block:: apache
 
     <Location /<someprefix> >
       ProxyPass http://127.0.0.1:5000/<someprefix>
       ProxyPassReverse http://127.0.0.1:5000/<someprefix>
       SetEnvIf X-Url-Scheme https HTTPS=1
     </Location>
 
 Besides the regular apache setup you will need to add the following line
 into ``[app:main]`` section of your .ini file::
 
     filter-with = proxy-prefix
 
 Add the following at the end of the .ini file::
 
     [filter:proxy-prefix]
     use = egg:PasteDeploy#prefix
     prefix = /<someprefix>
 
 then change ``<someprefix>`` into your chosen prefix
 
 
 Apache with mod_wsgi
 --------------------
 
 Alternatively, Kallithea can be set up with Apache under mod_wsgi. For
 that, you'll need to:
 
 - Install mod_wsgi. If using a Debian-based distro, you can install
   the package libapache2-mod-wsgi::
 
     aptitude install libapache2-mod-wsgi
 
 - Enable mod_wsgi::
 
     a2enmod wsgi
 
 - Add global Apache configuration to tell mod_wsgi that Python only will be
   used in the WSGI processes and shouldn't be initialized in the Apache
   processes::
 
     WSGIRestrictEmbedded On
 
 - Create a wsgi dispatch script, like the one below. Make sure you
   check that the paths correctly point to where you installed Kallithea
   and its Python Virtual Environment.
 - Enable the ``WSGIScriptAlias`` directive for the WSGI dispatch script,
   as in the following example. Once again, check the paths are
   correctly specified.
 
 Here is a sample excerpt from an Apache Virtual Host configuration file:
 
 .. code-block:: apache
 
     WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 \
         python-home=/srv/kallithea/venv
     WSGIProcessGroup kallithea
     WSGIScriptAlias / /srv/kallithea/dispatch.wsgi
     WSGIPassAuthorization On
 
 Or if using a dispatcher WSGI script with proper virtualenv activation:
 
 .. code-block:: apache
 
     WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100
     WSGIProcessGroup kallithea
     WSGIScriptAlias / /srv/kallithea/dispatch.wsgi
     WSGIPassAuthorization On
 
 Apache will by default run as a special Apache user, on Linux systems
 usually ``www-data`` or ``apache``. If you need to have the repositories
 directory owned by a different user, use the user and group options to
 WSGIDaemonProcess to set the name of the user and group.
 
 Example WSGI dispatch script:
 
 .. code-block:: python
 
     import os
     os.environ["HGENCODING"] = "UTF-8"
     os.environ['PYTHON_EGG_CACHE'] = '/srv/kallithea/.egg-cache'
 
     # sometimes it's needed to set the current dir
     os.chdir('/srv/kallithea/')
 
     import site
     site.addsitedir("/srv/kallithea/venv/lib/python2.7/site-packages")
 
     ini = '/srv/kallithea/my.ini'
     from paste.script.util.logging_config import fileConfig
     fileConfig(ini)
     from paste.deploy import loadapp
     application = loadapp('config:' + ini)
 
 Or using proper virtualenv activation:
 
 .. code-block:: python
 
     activate_this = '/srv/kallithea/venv/bin/activate_this.py'
     execfile(activate_this, dict(__file__=activate_this))
 
     import os
     os.environ['HOME'] = '/srv/kallithea'
 
     ini = '/srv/kallithea/kallithea.ini'
     from paste.script.util.logging_config import fileConfig
     fileConfig(ini)
     from paste.deploy import loadapp
     application = loadapp('config:' + ini)
 
 
 Other configuration files
 -------------------------
 
 A number of `example init.d scripts`__ can be found in
 the ``init.d`` directory of the Kallithea source.
 
 .. __: https://kallithea-scm.org/repos/kallithea/files/tip/init.d/ .
 
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _Mercurial: https://www.mercurial-scm.org/
 .. _Celery: http://celeryproject.org/
 .. _Celery documentation: http://docs.celeryproject.org/en/latest/getting-started/index.html
 .. _RabbitMQ: http://www.rabbitmq.com/
 .. _Redis: http://redis.io/
 .. _python-ldap: http://www.python-ldap.org/
 .. _mercurial-server: http://www.lshift.net/mercurial-server.html
 .. _PublishingRepositories: https://www.mercurial-scm.org/wiki/PublishingRepositories

docs/upgrade.rst

@@ -1,187 +1,187 b''
 .. _upgrade:
 
 ===================
 Upgrading Kallithea
 ===================
 
 This describes the process for upgrading Kallithea, independently of the
 Kallithea installation method.
 
 .. note::
     If you are upgrading from a RhodeCode installation, you must first
     install Kallithea 0.3.2 and follow the instructions in the 0.3.2
     README to perform a one-time conversion of the database from
     RhodeCode to Kallithea, before upgrading to the latest version
     of Kallithea.
 
 
 1. Stop the Kallithea web application
 -------------------------------------
 
 This step depends entirely on the web server software used to serve
 Kallithea, but in any case, Kallithea should not be running during
 the upgrade.
 
 .. note::
     If you're using Celery, make sure you stop all instances during the
     upgrade.
 
 
 2. Create a backup of both database and configuration
 -----------------------------------------------------
 
 You are of course strongly recommended to make backups regularly, but it
 is *especially* important to make a full database and configuration
 backup before performing a Kallithea upgrade.
 
 Back up your configuration
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Make a copy of your Kallithea configuration (``.ini``) file.
 
 If you are using :ref:`rcextensions <customization>`, you should also
 make a copy of the entire ``rcextensions`` directory.
 
 Back up your database
 ^^^^^^^^^^^^^^^^^^^^^
 
 If using SQLite, simply make a copy of the Kallithea database (``.db``)
 file.
 
 If using PostgreSQL, please consult the documentation for the ``pg_dump``
 utility.
 
 If using MySQL, please consult the documentation for the ``mysqldump``
 utility.
 
 Look for ``sqlalchemy.url`` in your configuration file to determine
 database type, settings, location, etc.
 
 
 3. Activate the Kallithea virtual environment (if any)
 ------------------------------------------------------
 
 Verify that you are using the Python environment that you originally
 installed Kallithea in by running::
 
     pip freeze
 
 This will list all packages installed in the current environment. If
 Kallithea isn't listed, activate the correct virtual environment.
 See the appropriate installation page for details.
 
 
 4. Install new version of Kallithea
 -----------------------------------
 
 Please refer to the instructions for the installation method you
 originally used to install Kallithea.
 
 If you originally installed using pip, it is as simple as::
 
     pip install --upgrade kallithea
 
 If you originally installed from version control, it is as simple as::
 
     cd my-kallithea-clone
     hg pull -u
     pip install -e .
 
 
 5. Upgrade your configuration
 -----------------------------
 
 Run the following command to upgrade your configuration (``.ini``) file::
 
-    paster make-config Kallithea my.ini
+    TODO make-config Kallithea my.ini
 
 This will display any changes made by the new version of Kallithea to your
 current configuration, and attempt an automatic merge. It is recommended
 that you check the contents after the merge.
 
 .. note::
     Please always make sure your ``.ini`` files are up to date. Errors
     can often be caused by missing parameters added in new versions.
 
 .. _upgrade_db:
 
 
 6. Upgrade your database
 ------------------------
 
 .. note::
     If you are *downgrading* Kallithea, you should perform the database
     migration step *before* installing the older version. (That is,
     always perform migrations using the most recent of the two versions
     you're migrating between.)
 
 First, run the following command to see your current database version::
 
     alembic -c my.ini current
 
 Typical output will be something like "9358dc3d6828 (head)", which is
 the current Alembic database "revision ID". Write down the entire output
 for troubleshooting purposes.
 
 The output will be empty if you're upgrading from Kallithea 0.3.x or
 older. That's expected. If you get an error that the config file was not
 found or has no ``[alembic]`` section, see the next section.
 
 Next, if you are performing an *upgrade*: Run the following command to
 upgrade your database to the current Kallithea version::
 
     alembic -c my.ini upgrade head
 
 If you are performing a *downgrade*: Run the following command to
 downgrade your database to the given version::
 
     alembic -c my.ini downgrade 0.4
 
 Alembic will show the necessary migrations (if any) as it executes them.
 If no "ERROR" is displayed, the command was successful.
 
 Should an error occur, the database may be "stranded" half-way
 through the migration, and you should restore it from backup.
 
 Enabling old Kallithea config files for Alembic use
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Kallithea configuration files created before the introduction of Alembic
 (i.e. predating Kallithea 0.4) need to be updated for use with Alembic.
 Without this, Alembic will fail with an error like this::
 
     FAILED: No config file 'my.ini' found, or file has no '[alembic]' section
 
 If Alembic complains specifically about a missing ``alembic.ini``, it is
 likely because you did not specify a config file using the ``-c`` option.
 On the other hand, if the mentioned config file actually exists, you
 need to append the following lines to it::
 
     [alembic]
     script_location = kallithea:alembic
 
 Your config file should now work with Alembic.
 
 
 7. Rebuild the Whoosh full-text index
 -------------------------------------
 
 It is recommended that you rebuild the Whoosh index after upgrading since
 new Whoosh versions can introduce incompatible index changes.
 
 
 8. Start the Kallithea web application
 --------------------------------------
 
 This step once again depends entirely on the web server software used to
 serve Kallithea.
 
 Before starting the new version of Kallithea, you may find it helpful to
 clear out your log file so that new errors are readily apparent.
 
 .. note::
     If you're using Celery, make sure you restart all instances of it after
     upgrade.
 
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv

docs/usage/customization.rst

@@ -1,50 +1,50 b''
 .. _customization:
 
 =============
 Customization
 =============
 
 There are several ways to customize Kallithea to your needs depending on what
 you want to achieve.
 
 
 HTML/JavaScript/CSS customization
 ---------------------------------
 
 To customize the look-and-feel of the web interface (for example to add a
 company banner or some JavaScript widget or to tweak the CSS style definitions)
 you can enter HTML code (possibly with JavaScript and/or CSS) directly via the
 *Admin > Settings > Global > HTML/JavaScript customization
 block*.
 
 
 Behavioral customization: rcextensions
 --------------------------------------
 
 Some behavioral customization can be done in Python using ``rcextensions``, a
 custom Python package that can extend Kallithea functionality.
 
 With ``rcextensions`` it's possible to add additional mappings for Whoosh
 indexing and statistics, to add additional code into the push/pull/create/delete
 repository hooks (for example to send signals to build bots such as Jenkins) and
 even to monkey-patch certain parts of the Kallithea source code (for example
 overwrite an entire function, change a global variable, ...).
 
 To generate a skeleton extensions package, run::
 
-    paster make-rcext my.ini
+    gearbox make-rcext -c my.ini
 
 This will create an ``rcextensions`` package next to the specified ``ini`` file.
 See the ``__init__.py`` file inside the generated ``rcextensions`` package
 for more details.
 
 
 Behavioral customization: code changes
 --------------------------------------
 
 As Kallithea is open-source software, you can make any changes you like directly
 in the source code.
 
 We encourage you to send generic improvements back to the
 community so that Kallithea can become better. See :ref:`contributing` for more
 details.

docs/usage/general.rst

@@ -1,181 +1,181 b''
 .. _general:
 
 =======================
 General Kallithea usage
 =======================
 
 
 Repository deletion
 -------------------
 
 Currently when an admin or owner deletes a repository, Kallithea does
 not physically delete said repository from the filesystem, but instead
 renames it in a special way so that it is not possible to push, clone
 or access the repository.
 
 There is a special command for cleaning up such archived repositories::
 
-    paster cleanup-repos --older-than=30d my.ini
+    gearbox cleanup-repos --older-than=30d -c my.ini
 
 This command scans for archived repositories that are older than
 30 days, displays them, and asks if you want to delete them (unless given
 the ``--dont-ask`` flag). If you host a large amount of repositories with
 forks that are constantly being deleted, it is recommended that you run this
 command via crontab.
 
 It is worth noting that even if someone is given administrative access to
 Kallithea and deletes a repository, you can easily restore such an action by
 renaming the repository directory, removing the ``rm__<date>`` prefix.
 
 
 File view: follow current branch
 --------------------------------
 
 In the file view, left and right arrows allow to jump to the previous and next
 revision. Depending on the way revisions were created in the repository, this
 could jump to a different branch.  When the checkbox ``Follow current branch``
 is checked, these arrows will only jump to revisions on the same branch as the
 currently visible revision.  So for example, if someone is viewing files in the
 ``beta`` branch and marks the `Follow current branch` checkbox, the < and >
 arrows will only show revisions on the ``beta`` branch.
 
 
 Changelog features
 ------------------
 
 The core feature of a repository's ``changelog`` page is to show the revisions
 in a repository. However, there are several other features available from the
 changelog.
 
 Branch filter
   By default, the changelog shows revisions from all branches in the
   repository. Use the branch filter to restrict to a given branch.
 
 Viewing a changeset
   A particular changeset can be opened by clicking on either the changeset
   hash or the commit message, or by ticking the checkbox and clicking the
   ``Show selected changeset`` button at the top.
 
 Viewing all changes between two changesets
   To get a list of all changesets between two selected changesets, along with
   the changes in each one of them, tick the checkboxes of the first and
   last changeset in the desired range and click the ``Show selected changesets``
   button at the top. You can only show the range between the first and last
   checkbox (no cherry-picking).
 
   From that page, you can proceed to viewing the overall delta between the
   selected changesets, by clicking the ``Compare revisions`` button.
 
 Creating a pull request
   You can create a new pull request for the changes of a particular changeset
   (and its ancestors) by selecting it and clicking the ``Open new pull request
   for selected changesets`` button.
 
 
 Permanent repository URLs
 -------------------------
 
 Due to the complicated nature of repository grouping, URLs of repositories
 can often change. For example, a repository originally accessible from::
 
   http://kallithea.example.com/repo_name
 
 would get a new URL after moving it to test_group::
 
   http://kallithea.example.com/test_group/repo_name
 
 Such moving of a repository to a group can be an issue for build systems and
 other scripts where the repository paths are hardcoded. To mitigate this,
 Kallithea provides permanent URLs using the repository ID prefixed with an
 underscore. In all Kallithea URLs, for example those for the changelog and the
 file view, a repository name can be replaced by this ``_ID`` string. Since IDs
 are always the same, moving the repository to a different group will not affect
 such URLs.
 
 In the example, the repository could also be accessible as::
 
   http://kallithea.example.com/_<ID>
 
 The ID of a given repository can be shown from the repository ``Summary`` page,
 by selecting the ``Show by ID`` button next to ``Clone URL``.
 
 
 Email notifications
 -------------------
 
 With email settings properly configured in the Kallithea
 configuration file, Kallithea will send emails on user registration and when
 errors occur.
 
 Emails are also sent for comments on changesets. In this case, an email is sent
 to the committer of the changeset (if known to Kallithea), to all reviewers of
 the pull request (if applicable) and to all people mentioned in the comment
 using @mention notation.
 
 
 Trending source files
 ---------------------
 
 Trending source files are calculated based on a predefined dictionary of known
 types and extensions. If an extension is missing or you would like to scan
 custom files, it is possible to extend the ``LANGUAGES_EXTENSIONS_MAP``
 dictionary located in ``kallithea/config/conf.py`` with new types.
 
 
 Cloning remote repositories
 ---------------------------
 
 Kallithea has the ability to clone repositories from given remote locations.
 Currently it supports the following options:
 
 - hg  -> hg clone
 - svn -> hg clone
 - git -> git clone
 
 .. note:: svn -> hg cloning requires the ``hgsubversion`` library to be
    installed.
 
 If you need to clone repositories that are protected via basic authentication,
 you can pass the credentials in the URL, e.g.
 ``http://user:[e-mail]/repo``. Kallithea will then try to login and
 clone using the given credentials. Please note that the given credentials will
 be stored as plaintext inside the database. However, the authentication
 information will not be shown in the clone URL on the summary page.
 
 
 Specific features configurable in the Admin settings
 ----------------------------------------------------
 
 In general, the Admin settings should be self-explanatory and will not be
 described in more detail in this documentation. However, there are a few
 features that merit further explanation.
 
 Repository extra fields
 ^^^^^^^^^^^^^^^^^^^^^^^
 
 In the *Visual* tab, there is an option "Use repository extra
 fields", which allows to set custom fields for each repository in the system.
 
 Once enabled site-wide, the custom fields can be edited per-repository under
 *Options* | *Settings* | *Extra Fields*.
 
 Example usage of such fields would be to define company-specific information
 into repositories, e.g., defining a ``repo_manager`` key that would give info
 about a manager of each repository.  There's no limit for adding custom fields.
 Newly created fields are accessible via the API.
 
 Meta tagging
 ^^^^^^^^^^^^
 
 In the *Visual* tab, option "Stylify recognised meta tags" will cause Kallithea
 to turn certain text fragments in repository and repository group
 descriptions into colored tags. Currently recognised tags are::
 
     [featured]
     [stale]
     [dead]
     [lang => lang]
     [license => License]
     [requires => Repo]
     [recommends => Repo]
     [see => URI]

docs/usage/vcs_support.rst

@@ -1,87 +1,87 b''
 .. _vcs_support:
 
 ===============================
 Version control systems support
 ===============================
 
 Kallithea supports Git and Mercurial repositories out-of-the-box.
 For Git, you do need the ``git`` command line client installed on the server.
 
 You can always disable Git or Mercurial support by editing the
 file ``kallithea/__init__.py`` and commenting out the backend.
 
 .. code-block:: python
 
    BACKENDS = {
        'hg': 'Mercurial repository',
        #'git': 'Git repository',
    }
 
 
 Git support
 -----------
 
 
 Web server with chunked encoding
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Large Git pushes require an HTTP server with support for
 chunked encoding for POST. The Python web servers waitress_ and
 gunicorn_ (Linux only) can be used. By default, Kallithea uses
-waitress_ for `paster serve` instead of the built-in `paste` WSGI
+waitress_ for `gearbox serve` instead of the built-in `paste` WSGI
 server.
 
-The paster server is controlled in the .ini file::
+The web server used by gearbox is controlled in the .ini file::
 
     use = egg:waitress#main
 
 or::
 
     use = egg:gunicorn#main
 
 Also make sure to comment out the following options::
 
     threadpool_workers =
     threadpool_max_requests =
     use_threadpool =
 
 
 Mercurial support
 -----------------
 
 
 Working with Mercurial subrepositories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 This section explains how to use Mercurial subrepositories_ in Kallithea.
 
 Example usage::
 
     ## init a simple repo
     hg init mainrepo
     cd mainrepo
     echo "file" > file
     hg add file
     hg ci --message "initial file"
 
     # clone subrepo we want to add from Kallithea
     hg clone http://kallithea.local/subrepo
 
     ## specify URL to existing repo in Kallithea as subrepository path
     echo "subrepo = http://kallithea.local/subrepo" > .hgsub
     hg add .hgsub
     hg ci --message "added remote subrepo"
 
 In the file list of a clone of ``mainrepo`` you will see a connected
 subrepository at the revision it was cloned with. Clicking on the
 subrepository link sends you to the proper repository in Kallithea.
 
 Cloning ``mainrepo`` will also clone the attached subrepository.
 
 Next we can edit the subrepository data, and push back to Kallithea. This will
 update both repositories.
 
 
 .. _waitress: http://pypi.python.org/pypi/waitress
 .. _gunicorn: http://pypi.python.org/pypi/gunicorn
 .. _subrepositories: http://mercurial.aragost.com/kick-start/en/subrepositories/

init.d/celeryd-upstart.conf

@@ -1,34 +1,34 b''
 # celeryd - run the celeryd daemon as an upstart job for kallithea
 # Change variables/paths as necessary and place file /etc/init/celeryd.conf
 # start/stop/restart as normal upstart job (ie: $ start celeryd)
 
 description     "Celery for Kallithea Mercurial Server"
 author          "Matt Zuba <[e-mail]"
 
 start on starting kallithea
 stop on stopped kallithea
 
 respawn
 
 umask 0022
 
 env PIDFILE=/tmp/celeryd.pid
 env APPINI=/var/hg/kallithea/production.ini
 env HOME=/var/hg
 env USER=hg
 # To use group (if different from user), you must edit sudoers file and change
 # root's entry from (ALL) to (ALL:ALL)
 # env GROUP=hg
 
 script
-    COMMAND="/var/hg/.virtualenvs/kallithea/bin/paster celeryd $APPINI --pidfile=$PIDFILE"
+    COMMAND="/var/hg/.virtualenvs/kallithea/bin/gearbox celeryd -c $APPINI -- --pidfile=$PIDFILE"
     if [ -z "$GROUP" ]; then
         exec sudo -u $USER $COMMAND
     else
         exec sudo -u $USER -g $GROUP $COMMAND
     fi
 end script
 
 post-stop script
     rm -f $PIDFILE
 end script

init.d/kallithea-daemon-arch

@@ -1,70 +1,70 b''
 #!/bin/bash
 ###########################################
 #### THIS IS AN ARCH LINUX RC.D SCRIPT ####
 ###########################################
 
 . /etc/rc.conf
 . /etc/rc.d/functions
 
 DAEMON=kallithea
 APP_HOMEDIR="/srv"
 APP_PATH="$APP_HOMEDIR/$DAEMON"
 CONF_NAME="production.ini"
 LOG_FILE="/var/log/$DAEMON.log"
 PID_FILE="/run/daemons/$DAEMON"
-APPL=/usr/bin/paster
+APPL=/usr/bin/gearbox
 RUN_AS="*****"
 
 ARGS="serve --daemon \
 --user=$RUN_AS \
 --group=$RUN_AS \
 --pid-file=$PID_FILE \
 --log-file=$LOG_FILE \
-$APP_PATH/$CONF_NAME"
+-c $APP_PATH/$CONF_NAME"
 
 [ -r /etc/conf.d/$DAEMON ] && . /etc/conf.d/$DAEMON
 
 if [[ -r $PID_FILE ]]; then
     read -r PID < "$PID_FILE"
     if [[ $PID && ! -d /proc/$PID ]]; then
         unset PID
         rm_daemon $DAEMON
     fi
 fi
 
 case "$1" in
 start)
     stat_busy "Starting $DAEMON"
     export HOME=$APP_PATH
     [ -z "$PID" ] && $APPL $ARGS &>/dev/null
     if [ $? = 0 ]; then
         add_daemon $DAEMON
         stat_done
     else
         stat_fail
         exit 1
     fi
     ;;
 stop)
     stat_busy "Stopping $DAEMON"
     [ -n "$PID" ] && kill $PID &>/dev/null
     if [ $? = 0 ]; then
         rm_daemon $DAEMON
         stat_done
     else
         stat_fail
         exit 1
     fi
     ;;
 restart)
     $0 stop
     sleep 1
     $0 start
     ;;
 status)
     stat_busy "Checking $name status";
     ck_status $name
     ;;
 *)
     echo "usage: $0 {start|stop|restart|status}"
 esac

init.d/kallithea-daemon-debian

@@ -1,90 +1,90 b''
 #!/bin/sh -e
 ########################################
 #### THIS IS A DEBIAN INIT.D SCRIPT ####
 ########################################
 
 ### BEGIN INIT INFO
 # Provides:          kallithea
 # Required-Start:    $all
 # Required-Stop:     $all
 # Default-Start:     2 3 4 5
 # Default-Stop:      0 1 6
 # Short-Description: starts instance of kallithea
 # Description:       starts instance of kallithea using start-stop-daemon
 ### END INIT INFO
 
 APP_NAME="kallithea"
 APP_HOMEDIR="opt"
 APP_PATH="/$APP_HOMEDIR/$APP_NAME"
 
 CONF_NAME="production.ini"
 
 PID_PATH="$APP_PATH/$APP_NAME.pid"
 LOG_PATH="$APP_PATH/$APP_NAME.log"
 
 PYTHON_PATH="/$APP_HOMEDIR/$APP_NAME-venv"
 
 RUN_AS="root"
 
-DAEMON="$PYTHON_PATH/bin/paster"
+DAEMON="$PYTHON_PATH/bin/gearbox"
 
 DAEMON_OPTS="serve --daemon \
  --user=$RUN_AS \
  --group=$RUN_AS \
  --pid-file=$PID_PATH \
- --log-file=$LOG_PATH  $APP_PATH/$CONF_NAME"
+ --log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME"
 
 
 start() {
   echo "Starting $APP_NAME"
   PYTHON_EGG_CACHE="/tmp" start-stop-daemon -d $APP_PATH \
       --start --quiet \
       --pidfile $PID_PATH \
       --user $RUN_AS \
       --exec $DAEMON -- $DAEMON_OPTS
 }
 
 stop() {
   echo "Stopping $APP_NAME"
   start-stop-daemon -d $APP_PATH \
       --stop --quiet \
       --pidfile $PID_PATH || echo "$APP_NAME - Not running!"
 
   if [ -f $PID_PATH ]; then
     rm $PID_PATH
   fi
 }
 
 status() {
   echo -n "Checking status of $APP_NAME ... "
   pid=`cat $PID_PATH`
   status=`ps ax | grep $pid | grep -ve grep`
   if [ "$?" -eq 0 ]; then
     echo "running"
   else
     echo "NOT running"
   fi
 }
 
 case "$1" in
   status)
    status
     ;;
   start)
     start
     ;;
   stop)
     stop
     ;;
   restart)
     echo "Restarting $APP_NAME"
     ### stop ###
     stop
     wait
     ### start ###
     start
     ;;
   *)
     echo "Usage: $0 {start|stop|restart}"
     exit 1
 esac

init.d/kallithea-daemon-gentoo

@@ -1,61 +1,61 b''
 #!/sbin/runscript
 ########################################
 #### THIS IS AN GENTOO INIT.D SCRIPT####
 ########################################
 
 APP_NAME="kallithea"
 APP_HOMEDIR="username/python_workspace"
 APP_PATH="/home/$APP_HOMEDIR/$APP_NAME"
 
 CONF_NAME="production.ini"
 
 PID_PATH="$APP_PATH/$APP_NAME.pid"
 LOG_PATH="$APP_PATH/$APP_NAME.log"
 
 PYTHON_PATH="/home/$APP_HOMEDIR/v-env"
 
 RUN_AS="username"
 
-DAEMON="$PYTHON_PATH/bin/paster"
+DAEMON="$PYTHON_PATH/bin/gearbox"
 
 DAEMON_OPTS="serve --daemon \
 --user=$RUN_AS \
 --group=$RUN_AS \
 --pid-file=$PID_PATH \
---log-file=$LOG_PATH  $APP_PATH/$CONF_NAME"
+--log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME"
 
 #extra options
 opts="${opts} restartdelay"
 
 depend() {
     need nginx
 }
 
 start() {
     ebegin "Starting $APP_NAME"
     start-stop-daemon -d $APP_PATH -e PYTHON_EGG_CACHE="/tmp" \
         --start --quiet \
         --pidfile $PID_PATH \
         --user $RUN_AS \
         --exec $DAEMON -- $DAEMON_OPTS
     eend $?
 }
 
 stop() {
     ebegin "Stopping $APP_NAME"
     start-stop-daemon -d $APP_PATH \
         --stop --quiet \
         --pidfile $PID_PATH || echo "$APP_NAME - Not running!"
     if [ -f $PID_PATH ]; then
         rm $PID_PATH
     fi
     eend $?
 }
 
 restartdelay() {
     #stop()
     echo "sleep3"
     sleep 3
 
     #start()
 }

init.d/kallithea-daemon-redhat

@@ -1,132 +1,132 b''
 #!/bin/sh
 ########################################
 #### THIS IS A REDHAT INIT.D SCRIPT ####
 ########################################
 
 ##################################################
 #
 # Kallithea server startup script
 # Recommended default-startup: 2 3 4 5
 # Recommended default-stop: 0 1 6
 #
 ##################################################
 
 
 APP_NAME="kallithea"
 # the location of your app
 # since this is a web app, it should go in /var/www
 APP_PATH="/var/www/$APP_NAME"
 
 CONF_NAME="production.ini"
 
 # write to wherever the PID should be stored, just ensure
-# that the user you run paster as has the appropriate permissions
+# that the user you run gearbox as has the appropriate permissions
 # same goes for the log file
 PID_PATH="/var/run/kallithea/pid"
 LOG_PATH="/var/log/kallithea/kallithea.log"
 
 # replace this with the path to the virtual environment you
 # made for Kallithea
 PYTHON_PATH="/opt/python_virtualenvironments/kallithea-venv"
 
 RUN_AS="kallithea"
 
-DAEMON="$PYTHON_PATH/bin/paster"
+DAEMON="$PYTHON_PATH/bin/gearbox"
 
 DAEMON_OPTS="serve --daemon \
     --user=$RUN_AS \
     --group=$RUN_AS \
     --pid-file=$PID_PATH \
-    --log-file=$LOG_PATH $APP_PATH/$CONF_NAME"
+    --log-file=$LOG_PATH -c $APP_PATH/$CONF_NAME"
 
 DESC="kallithea-server"
 LOCK_FILE="/var/lock/subsys/$APP_NAME"
 
 # source CentOS init functions
 . /etc/init.d/functions
 
 RETVAL=0
 
 remove_pid () {
   rm -f ${PID_PATH}
   rmdir `dirname ${PID_PATH}`
 }
 
 ensure_pid_dir () {
   PID_DIR=`dirname ${PID_PATH}`
   if [ ! -d ${PID_DIR} ] ; then
     mkdir -p ${PID_DIR}
     chown -R ${RUN_AS}:${RUN_AS} ${PID_DIR}
     chmod 755 ${PID_DIR}
   fi
 }
 
 start_kallithea () {
     ensure_pid_dir
     PYTHON_EGG_CACHE="/tmp" daemon --pidfile $PID_PATH \
         --user $RUN_AS "$DAEMON $DAEMON_OPTS"
     RETVAL=$?
     [ $RETVAL -eq 0 ] && touch $LOCK_FILE
     return $RETVAL
 }
 
 stop_kallithea () {
     if [ -e $LOCK_FILE ]; then
       killproc -p $PID_PATH
       RETVAL=$?
       rm -f $LOCK_FILE
       rm -f $PID_PATH
     else
       RETVAL=1
     fi
     return $RETVAL
 }
 
 status_kallithea() {
   if [ -e $LOCK_FILE ]; then
     # exit with non-zero to indicate failure
     RETVAL=1
   else
     RETVAL=0
   fi
   return $RETVAL
 }
 
 restart_kallithea () {
     stop_kallithea
     start_kallithea
     RETVAL=$?
 }
 
 case "$1" in
   start)
     echo -n $"Starting $DESC: "
     start_kallithea
     echo
     ;;
   stop)
     echo -n $"Stopping $DESC: "
     stop_kallithea
     echo
     ;;
   status)
     status_kallithea
     RETVAL=$?
     if [ ! $RETVAL -eq 0 ]; then
       echo "Kallithea server is running..."
     else
       echo "Kallithea server is stopped."
     fi
     ;;
   restart)
     echo -n $"Restarting $DESC: "
     restart_kallithea
     echo
     ;;
   *)
     echo $"Usage: $0 {start|stop|restart|status}"
     RETVAL=1
     ;;
 esac
 
 exit $RETVAL

init.d/kallithea-upstart.conf

@@ -1,26 +1,26 b''
 # kallithea - run the kallithea daemon as an upstart job
 # Change variables/paths as necessary and place file /etc/init/kallithea.conf
 # start/stop/restart as normal upstart job (ie: $ start kallithea)
 
 description     "Kallithea Mercurial Server"
 author          "Matt Zuba <[e-mail]"
 
 start on (local-filesystems and runlevel [2345])
 stop on runlevel [!2345]
 
 respawn
 
 umask 0022
 
 env PIDFILE=/var/hg/kallithea/kallithea.pid
 env LOGFILE=/var/hg/kallithea/log/kallithea.log
 env APPINI=/var/hg/kallithea/production.ini
 env HOME=/var/hg
 env USER=hg
 env GROUP=hg
 
-exec /var/hg/.virtualenvs/kallithea/bin/paster serve --user=$USER --group=$GROUP --pid-file=$PIDFILE --log-file=$LOGFILE $APPINI
+exec /var/hg/.virtualenvs/kallithea/bin/gearbox serve --user=$USER --group=$GROUP --pid-file=$PIDFILE --log-file=$LOGFILE -c $APPINI
 
 post-stop script
     rm -f $PIDFILE
 end script

init.d/supervisord.conf

@@ -1,51 +1,51 b''
 ; Kallithea Supervisord
 ; ##########################
 ; for help see http://supervisord.org/configuration.html
 ; ##########################
 
 [inet_http_server]         ; inet (TCP) server disabled by default
 port=127.0.0.1:9001        ; (ip_address:port specifier, *:port for all iface)
 ;username=user              ; (default is no username (open server))
 ;password=123               ; (default is no password (open server))
 
 [supervisord]
 logfile=/%(here)s/supervisord_kallithea.log ; (main log file;default $CWD/supervisord.log)
 logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
 logfile_backups=10           ; (num of main logfile rotation backups;default 10)
 loglevel=info                ; (log level;default info; others: debug,warn,trace)
 pidfile=/%(here)s/supervisord_kallithea.pid ; (supervisord pidfile;default supervisord.pid)
 nodaemon=true               ; (start in foreground if true;default false)
 minfds=1024                  ; (min. avail startup file descriptors;default 1024)
 minprocs=200                 ; (min. avail process descriptors;default 200)
 umask=022                    ; (process file creation umask;default 022)
 user=username                  ; (default is current user, required if root)
 ;identifier=supervisor       ; (supervisord identifier, default is 'supervisor')
 ;directory=/tmp              ; (default is not to cd during start)
 ;nocleanup=true              ; (don't clean up tempfiles at start;default false)
 ;childlogdir=/tmp            ; ('AUTO' child log dir, default $TEMP)
 environment=HOME=/srv/kallithea       ; (key value pairs to add to environment)
 ;strip_ansi=false            ; (strip ansi escape codes in logs; def. false)
 
 ; the below section must remain in the config file for RPC
 ; (supervisorctl/web interface) to work, additional interfaces may be
 ; added by defining them in separate rpcinterface: sections
 [rpcinterface:supervisor]
 supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
 
 [supervisorctl]
 serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
 ;username=user               ; should be same as http_username if set
 ;password=123                ; should be same as http_password if set
 ;prompt=mysupervisor         ; cmd line prompt (default "supervisor")
 ;history_file=~/.sc_history  ; use readline history if available
 
 
 ; restart with supervisorctl restart kallithea:*
 [program:kallithea]
 numprocs = 1
 numprocs_start = 5000 # possible should match ports
 directory=/srv/kallithea
-command = /srv/kallithea/venv/bin/paster serve my.ini
+command = /srv/kallithea/venv/bin/gearbox serve -c my.ini
 process_name = %(program_name)s_%(process_num)04d
 redirect_stderr=true
 stdout_logfile=/%(here)s/kallithea.log

kallithea/alembic/env.py

@@ -1,105 +1,105 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # Alembic migration environment (configuration).
 
 import logging
 from logging.config import fileConfig
 
 from alembic import context
 from sqlalchemy import engine_from_config, pool
 
 from kallithea.model import db
 
 
 # The alembic.config.Config object, which wraps the current .ini file.
 config = context.config
 
 # Default to use the main Kallithea database string in [app:main].
 # For advanced uses, this can be overridden by specifying an explicit
 # [alembic] sqlalchemy.url.
 database_url = (
     config.get_main_option('sqlalchemy.url') or
     config.get_section_option('app:main', 'sqlalchemy.url')
 )
 
 # Configure default logging for Alembic. (This can be overriden by the
 # config file, but usually isn't.)
 logging.getLogger('alembic').setLevel(logging.INFO)
 
 # Setup Python loggers based on the config file provided to the alembic
 # command. If we're being invoked via the Alembic API (presumably for
-# stamping during "paster setup-db"), config_file_name is not available,
+# stamping during "gearbox setup-db"), config_file_name is not available,
 # and loggers are assumed to already have been configured.
 if config.config_file_name:
     fileConfig(config.config_file_name, disable_existing_loggers=False)
 
 
 def include_in_autogeneration(object, name, type, reflected, compare_to):
     """Filter changes subject to autogeneration of migrations. """
 
     # Don't include changes to sqlite_sequence.
     if type == 'table' and name == 'sqlite_sequence':
         return False
 
     return True
 
 
 def run_migrations_offline():
     """Run migrations in 'offline' (--sql) mode.
 
     This produces an SQL script instead of directly applying the changes.
     Some migrations may not run in offline mode.
     """
     context.configure(
         url=database_url,
         literal_binds=True,
     )
 
     with context.begin_transaction():
         context.run_migrations()
 
 
 def run_migrations_online():
     """Run migrations in 'online' mode.
 
     Connects to the database and directly applies the necessary
     migrations.
     """
     cfg = config.get_section(config.config_ini_section)
     cfg['sqlalchemy.url'] = database_url
     connectable = engine_from_config(
         cfg,
         prefix='sqlalchemy.',
         poolclass=pool.NullPool)
 
     with connectable.connect() as connection:
         context.configure(
             connection=connection,
 
             # Support autogeneration of migration scripts based on "diff" between
             # current database schema and kallithea.model.db schema.
             target_metadata=db.Base.metadata,
             include_object=include_in_autogeneration,
             render_as_batch=True, # batch mode is needed for SQLite support
         )
 
         with context.begin_transaction():
             context.run_migrations()
 
 
 if context.is_offline_mode():
     run_migrations_offline()
 else:
     run_migrations_online()

kallithea/bin/template.ini.mako

@@ -1,600 +1,610 b''
 ## -*- coding: utf-8 -*-
 <%text>################################################################################</%text>
 <%text>################################################################################</%text>
 # Kallithea - config file generated with kallithea-config                      #
 <%text>################################################################################</%text>
 <%text>################################################################################</%text>
 
 [DEFAULT]
 debug = true
 pdebug = false
 
 <%text>################################################################################</%text>
 <%text>## Email settings                                                             ##</%text>
 <%text>##                                                                            ##</%text>
 <%text>## Refer to the documentation ("Email settings") for more details.            ##</%text>
 <%text>##                                                                            ##</%text>
 <%text>## It is recommended to use a valid sender address that passes access         ##</%text>
 <%text>## validation and spam filtering in mail servers.                             ##</%text>
 <%text>################################################################################</%text>
 
 <%text>## 'From' header for application emails. You can optionally add a name.</%text>
 <%text>## Default:</%text>
 #app_email_from = Kallithea
 <%text>## Examples:</%text>
 #app_email_from = Kallithea <[e-mail]>
 #app_email_from = [e-mail]
 
 <%text>## Subject prefix for application emails.</%text>
 <%text>## A space between this prefix and the real subject is automatically added.</%text>
 <%text>## Default:</%text>
 #email_prefix =
 <%text>## Example:</%text>
 #email_prefix = [Kallithea]
 
 <%text>## Recipients for error emails and fallback recipients of application mails.</%text>
 <%text>## Multiple addresses can be specified, space-separated.</%text>
 <%text>## Only addresses are allowed, do not add any name part.</%text>
 <%text>## Default:</%text>
 #email_to =
 <%text>## Examples:</%text>
 #email_to = [e-mail]
 #email_to = [e-mail] [e-mail]
 
 <%text>## 'From' header for error emails. You can optionally add a name.</%text>
 <%text>## Default:</%text>
 #error_email_from = [e-mail]
 <%text>## Examples:</%text>
 #error_email_from = Kallithea Errors <[e-mail]>
 #error_email_from = [e-mail]
 
 <%text>## SMTP server settings</%text>
 <%text>## If specifying credentials, make sure to use secure connections.</%text>
 <%text>## Default: Send unencrypted unauthenticated mails to the specified smtp_server.</%text>
 <%text>## For "SSL", use smtp_use_ssl = true and smtp_port = 465.</%text>
 <%text>## For "STARTTLS", use smtp_use_tls = true and smtp_port = 587.</%text>
 #smtp_server = smtp.example.com
 #smtp_username =
 #smtp_password =
 #smtp_port = 25
 #smtp_use_ssl = false
 #smtp_use_tls = false
 
 [server:main]
-%if http_server == 'paste':
-<%text>## PASTE ##</%text>
-use = egg:Paste#http
+%if http_server == 'gearbox':
+<%text>## Gearbox default web server ##</%text>
+use = egg:gearbox#wsgiref
 <%text>## nr of worker threads to spawn</%text>
 threadpool_workers = 1
 <%text>## max request before thread respawn</%text>
 threadpool_max_requests = 100
 <%text>## option to use threads of process</%text>
 use_threadpool = true
 
+%elif http_server == 'gevent':
+<%text>## Gearbox gevent web server ##</%text>
+use = egg:gearbox#gevent
+
 %elif http_server == 'waitress':
 <%text>## WAITRESS ##</%text>
 use = egg:waitress#main
 <%text>## number of worker threads</%text>
 threads = 1
 <%text>## MAX BODY SIZE 100GB</%text>
 max_request_body_size = 107374182400
 <%text>## use poll instead of select, fixes fd limits, may not work on old</%text>
 <%text>## windows systems.</%text>
 #asyncore_use_poll = True
 
 %elif http_server == 'gunicorn':
 <%text>## GUNICORN ##</%text>
 use = egg:gunicorn#main
 <%text>## number of process workers. You must set `instance_id = *` when this option</%text>
 <%text>## is set to more than one worker</%text>
 workers = 1
 <%text>## process name</%text>
 proc_name = kallithea
 <%text>## type of worker class, one of sync, eventlet, gevent, tornado</%text>
 <%text>## recommended for bigger setup is using of of other than sync one</%text>
 worker_class = sync
 max_requests = 1000
 <%text>## amount of time a worker can handle request before it gets killed and</%text>
 <%text>## restarted</%text>
 timeout = 3600
 
 %elif http_server == 'uwsgi':
 <%text>## UWSGI ##</%text>
 <%text>## run with uwsgi --ini-paste-logged <inifile.ini></%text>
 [uwsgi]
 socket = /tmp/uwsgi.sock
 master = true
 http = 127.0.0.1:5000
 
 <%text>## set as deamon and redirect all output to file</%text>
 #daemonize = ./uwsgi_kallithea.log
 
 <%text>## master process PID</%text>
 pidfile = ./uwsgi_kallithea.pid
 
 <%text>## stats server with workers statistics, use uwsgitop</%text>
 <%text>## for monitoring, `uwsgitop 127.0.0.1:1717`</%text>
 stats = 127.0.0.1:1717
 memory-report = true
 
 <%text>## log 5XX errors</%text>
 log-5xx = true
 
 <%text>## Set the socket listen queue size.</%text>
 listen = 256
 
 <%text>## Gracefully Reload workers after the specified amount of managed requests</%text>
 <%text>## (avoid memory leaks).</%text>
 max-requests = 1000
 
 <%text>## enable large buffers</%text>
 buffer-size = 65535
 
 <%text>## socket and http timeouts ##</%text>
 http-timeout = 3600
 socket-timeout = 3600
 
 <%text>## Log requests slower than the specified number of milliseconds.</%text>
 log-slow = 10
 
 <%text>## Exit if no app can be loaded.</%text>
 need-app = true
 
 <%text>## Set lazy mode (load apps in workers instead of master).</%text>
 lazy = true
 
 <%text>## scaling ##</%text>
 <%text>## set cheaper algorithm to use, if not set default will be used</%text>
 cheaper-algo = spare
 
 <%text>## minimum number of workers to keep at all times</%text>
 cheaper = 1
 
 <%text>## number of workers to spawn at startup</%text>
 cheaper-initial = 1
 
 <%text>## maximum number of workers that can be spawned</%text>
 workers = 4
 
 <%text>## how many workers should be spawned at a time</%text>
 cheaper-step = 1
 
 %endif
 <%text>## COMMON ##</%text>
 host = ${host}
 port = ${port}
 
 <%text>## middleware for hosting the WSGI application under a URL prefix</%text>
 #[filter:proxy-prefix]
 #use = egg:PasteDeploy#prefix
 #prefix = /<your-prefix>
 
 [app:main]
 use = egg:kallithea
 <%text>## enable proxy prefix middleware</%text>
 #filter-with = proxy-prefix
 
 full_stack = true
 static_files = true
 <%text>## Available Languages:</%text>
 <%text>## cs de fr hu ja nl_BE pl pt_BR ru sk zh_CN zh_TW</%text>
 lang =
 cache_dir = ${here}/data
 index_dir = ${here}/data/index
 
 <%text>## perform a full repository scan on each server start, this should be</%text>
 <%text>## set to false after first startup, to allow faster server restarts.</%text>
 initial_repo_scan = false
 
 <%text>## uncomment and set this path to use archive download cache</%text>
 archive_cache_dir = ${here}/tarballcache
 
 <%text>## change this to unique ID for security</%text>
 app_instance_uuid = ${uuid()}
 
 <%text>## cut off limit for large diffs (size in bytes)</%text>
 cut_off_limit = 256000
 
 <%text>## force https in Kallithea, fixes https redirects, assumes it's always https</%text>
 force_https = false
 
 <%text>## use Strict-Transport-Security headers</%text>
 use_htsts = false
 
 <%text>## number of commits stats will parse on each iteration</%text>
 commit_parse_limit = 25
 
 <%text>## path to git executable</%text>
 git_path = git
 
 <%text>## git rev filter option, --all is the default filter, if you need to</%text>
 <%text>## hide all refs in changelog switch this to --branches --tags</%text>
 #git_rev_filter = --branches --tags
 
 <%text>## RSS feed options</%text>
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 <%text>## options for showing and identifying changesets</%text>
 show_sha_length = 12
 show_revision_number = false
 
 <%text>## Canonical URL to use when creating full URLs in UI and texts.</%text>
 <%text>## Useful when the site is available under different names or protocols.</%text>
 <%text>## Defaults to what is provided in the WSGI environment.</%text>
 #canonical_url = https://kallithea.example.com/repos
 
 <%text>## gist URL alias, used to create nicer urls for gist. This should be an</%text>
 <%text>## url that does rewrites to _admin/gists/<gistid>.</%text>
 <%text>## example: http://gist.example.com/{gistid}. Empty means use the internal</%text>
 <%text>## Kallithea url, ie. http[s]://kallithea.example.com/_admin/gists/<gistid></%text>
 gist_alias_url =
 
 <%text>## white list of API enabled controllers. This allows to add list of</%text>
 <%text>## controllers to which access will be enabled by api_key. eg: to enable</%text>
 <%text>## api access to raw_files put `FilesController:raw`, to enable access to patches</%text>
 <%text>## add `ChangesetController:changeset_patch`. This list should be "," separated</%text>
 <%text>## Syntax is <ControllerClass>:<function>. Check debug logs for generated names</%text>
 <%text>## Recommended settings below are commented out:</%text>
 api_access_controllers_whitelist =
 #    ChangesetController:changeset_patch,
 #    ChangesetController:changeset_raw,
 #    FilesController:raw,
 #    FilesController:archivefile
 
 <%text>## default encoding used to convert from and to unicode</%text>
 <%text>## can be also a comma separated list of encoding in case of mixed encodings</%text>
 default_encoding = utf8
 
 <%text>## issue tracker for Kallithea (leave blank to disable, absent for default)</%text>
 #bugtracker = https://bitbucket.org/conservancy/kallithea/issues
 
 <%text>## issue tracking mapping for commits messages</%text>
 <%text>## comment out issue_pat, issue_server, issue_prefix to enable</%text>
 
 <%text>## pattern to get the issues from commit messages</%text>
 <%text>## default one used here is #<numbers> with a regex passive group for `#`</%text>
 <%text>## {id} will be all groups matched from this pattern</%text>
 
 issue_pat = (?:\s*#)(\d+)
 
 <%text>## server url to the issue, each {id} will be replaced with match</%text>
 <%text>## fetched from the regex and {repo} is replaced with full repository name</%text>
 <%text>## including groups {repo_name} is replaced with just name of repo</%text>
 
 issue_server_link = https://issues.example.com/{repo}/issue/{id}
 
 <%text>## prefix to add to link to indicate it's an url</%text>
 <%text>## #314 will be replaced by <issue_prefix><id></%text>
 
 issue_prefix = #
 
 <%text>## issue_pat, issue_server_link, issue_prefix can have suffixes to specify</%text>
 <%text>## multiple patterns, to other issues server, wiki or others</%text>
 <%text>## below an example how to create a wiki pattern</%text>
 # wiki-some-id -> https://wiki.example.com/some-id
 
 #issue_pat_wiki = (?:wiki-)(.+)
 #issue_server_link_wiki = https://wiki.example.com/{id}
 #issue_prefix_wiki = WIKI-
 
 <%text>## alternative return HTTP header for failed authentication. Default HTTP</%text>
 <%text>## response is 401 HTTPUnauthorized. Currently Mercurial clients have trouble with</%text>
 <%text>## handling that. Set this variable to 403 to return HTTPForbidden</%text>
 auth_ret_code =
 
 <%text>## locking return code. When repository is locked return this HTTP code. 2XX</%text>
 <%text>## codes don't break the transactions while 4XX codes do</%text>
 lock_ret_code = 423
 
 <%text>## allows to change the repository location in settings page</%text>
 allow_repo_location_change = True
 
 <%text>## allows to setup custom hooks in settings page</%text>
 allow_custom_hooks_settings = True
 
 <%text>## extra extensions for indexing, space separated and without the leading '.'.</%text>
 # index.extensions =
 #    gemfile
 #    lock
 
 <%text>## extra filenames for indexing, space separated</%text>
 # index.filenames =
 #    .dockerignore
 #    .editorconfig
 #    INSTALL
 #    CHANGELOG
 
 <%text>####################################</%text>
 <%text>###        CELERY CONFIG        ####</%text>
 <%text>####################################</%text>
 
 use_celery = false
 
 <%text>## Example: connect to the virtual host 'rabbitmqhost' on localhost as rabbitmq:</%text>
 broker.url = amqp://rabbitmq:qewqew@localhost:5672/rabbitmqhost
 
 celery.imports = kallithea.lib.celerylib.tasks
 celery.accept.content = pickle
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 
 celeryd.concurrency = 2
 celeryd.max.tasks.per.child = 1
 
 <%text>## If true, tasks will never be sent to the queue, but executed locally instead.</%text>
 celery.always.eager = false
 
 <%text>####################################</%text>
 <%text>###         BEAKER CACHE        ####</%text>
 <%text>####################################</%text>
 
 beaker.cache.data_dir = ${here}/data/cache/data
 beaker.cache.lock_dir = ${here}/data/cache/lock
 
 beaker.cache.regions = short_term,long_term,sql_cache_short
 
 beaker.cache.short_term.type = memory
 beaker.cache.short_term.expire = 60
 beaker.cache.short_term.key_length = 256
 
 beaker.cache.long_term.type = memory
 beaker.cache.long_term.expire = 36000
 beaker.cache.long_term.key_length = 256
 
 beaker.cache.sql_cache_short.type = memory
 beaker.cache.sql_cache_short.expire = 10
 beaker.cache.sql_cache_short.key_length = 256
 
 <%text>####################################</%text>
 <%text>###       BEAKER SESSION        ####</%text>
 <%text>####################################</%text>
 
 <%text>## Name of session cookie. Should be unique for a given host and path, even when running</%text>
 <%text>## on different ports. Otherwise, cookie sessions will be shared and messed up.</%text>
 beaker.session.key = kallithea
 <%text>## Sessions should always only be accessible by the browser, not directly by JavaScript.</%text>
 beaker.session.httponly = true
 <%text>## Session lifetime. 2592000 seconds is 30 days.</%text>
 beaker.session.timeout = 2592000
 
 <%text>## Server secret used with HMAC to ensure integrity of cookies.</%text>
 beaker.session.secret = ${uuid()}
 <%text>## Further, encrypt the data with AES.</%text>
 #beaker.session.encrypt_key = <key_for_encryption>
 #beaker.session.validate_key = <validation_key>
 
 <%text>## Type of storage used for the session, current types are</%text>
 <%text>## dbm, file, memcached, database, and memory.</%text>
 
 <%text>## File system storage of session data. (default)</%text>
 #beaker.session.type = file
 
 <%text>## Cookie only, store all session data inside the cookie. Requires secure secrets.</%text>
 #beaker.session.type = cookie
 
 <%text>## Database storage of session data.</%text>
 #beaker.session.type = ext:database
 #beaker.session.sa.url = postgresql://postgres:qwe@localhost/kallithea
 #beaker.session.table_name = db_session
 
 %if error_aggregation_service == 'appenlight':
 <%text>############################</%text>
 <%text>## ERROR HANDLING SYSTEMS ##</%text>
 <%text>############################</%text>
 
 <%text>####################</%text>
 <%text>### [appenlight] ###</%text>
 <%text>####################</%text>
 
 <%text>## AppEnlight is tailored to work with Kallithea, see</%text>
 <%text>## http://appenlight.com for details how to obtain an account</%text>
 <%text>## you must install python package `appenlight_client` to make it work</%text>
 
 <%text>## appenlight enabled</%text>
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 
 <%text>## TWEAK AMOUNT OF INFO SENT HERE</%text>
 
 <%text>## enables 404 error logging (default False)</%text>
 appenlight.report_404 = false
 
 <%text>## time in seconds after request is considered being slow (default 1)</%text>
 appenlight.slow_request_time = 1
 
 <%text>## record slow requests in application</%text>
 <%text>## (needs to be enabled for slow datastore recording and time tracking)</%text>
 appenlight.slow_requests = true
 
 <%text>## enable hooking to application loggers</%text>
 #appenlight.logging = true
 
 <%text>## minimum log level for log capture</%text>
 #appenlight.logging.level = WARNING
 
 <%text>## send logs only from erroneous/slow requests</%text>
 <%text>## (saves API quota for intensive logging)</%text>
 appenlight.logging_on_error = false
 
 <%text>## list of additional keywords that should be grabbed from environ object</%text>
 <%text>## can be string with comma separated list of words in lowercase</%text>
 <%text>## (by default client will always send following info:</%text>
 <%text>## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that</%text>
 <%text>## start with HTTP* this list be extended with additional keywords here</%text>
 appenlight.environ_keys_whitelist =
 
 <%text>## list of keywords that should be blanked from request object</%text>
 <%text>## can be string with comma separated list of words in lowercase</%text>
 <%text>## (by default client will always blank keys that contain following words</%text>
 <%text>## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'</%text>
 <%text>## this list be extended with additional keywords set here</%text>
 appenlight.request_keys_blacklist =
 
 <%text>## list of namespaces that should be ignores when gathering log entries</%text>
 <%text>## can be string with comma separated list of namespaces</%text>
 <%text>## (by default the client ignores own entries: appenlight_client.client)</%text>
 appenlight.log_namespace_blacklist =
 
 %elif error_aggregation_service == 'sentry':
 <%text>################</%text>
 <%text>### [sentry] ###</%text>
 <%text>################</%text>
 
 <%text>## sentry is a alternative open source error aggregator</%text>
 <%text>## you must install python packages `sentry` and `raven` to enable</%text>
 
 sentry.dsn = YOUR_DNS
 sentry.servers =
 sentry.name =
 sentry.key =
 sentry.public_key =
 sentry.secret_key =
 sentry.project =
 sentry.site =
 sentry.include_paths =
 sentry.exclude_paths =
 
 %endif
 <%text>################################################################################</%text>
 <%text>## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##</%text>
 <%text>## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##</%text>
 <%text>## execute malicious code after an exception is raised.                       ##</%text>
 <%text>################################################################################</%text>
 set debug = false
 
 <%text>##################################</%text>
 <%text>###       LOGVIEW CONFIG       ###</%text>
 <%text>##################################</%text>
 
 logview.sqlalchemy = #faa
 logview.pylons.templating = #bfb
 logview.pylons.util = #eee
 
 <%text>#########################################################</%text>
 <%text>### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###</%text>
 <%text>#########################################################</%text>
 
 %if database_engine == 'sqlite':
 # SQLITE [default]
 sqlalchemy.url = sqlite:///${here}/kallithea.db?timeout=60
 
 %elif database_engine == 'postgres':
 # POSTGRESQL
 sqlalchemy.url = postgresql://user:pass@localhost/kallithea
 
 %elif database_engine == 'mysql':
 # MySQL
 sqlalchemy.url = mysql://user:pass@localhost/kallithea?charset=utf8
 
 %endif
 # see sqlalchemy docs for others
 
 sqlalchemy.echo = false
 sqlalchemy.pool_recycle = 3600
 
 <%text>################################</%text>
 <%text>### ALEMBIC CONFIGURATION   ####</%text>
 <%text>################################</%text>
 
 [alembic]
 script_location = kallithea:alembic
 
 <%text>################################</%text>
 <%text>### LOGGING CONFIGURATION   ####</%text>
 <%text>################################</%text>
 
 [loggers]
-keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer
+keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 <%text>#############</%text>
 <%text>## LOGGERS ##</%text>
 <%text>#############</%text>
 
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_routes]
 level = DEBUG
 handlers =
 qualname = routes.middleware
 <%text>## "level = DEBUG" logs the route matched and routing variables.</%text>
 propagate = 1
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_templates]
 level = INFO
 handlers =
 qualname = pylons.templating
 propagate = 1
 
 [logger_kallithea]
 level = DEBUG
 handlers =
 qualname = kallithea
 propagate = 1
 
+[logger_gearbox]
+level = DEBUG
+handlers =
+qualname = gearbox
+propagate = 1
+
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_whoosh_indexer]
 level = DEBUG
 handlers =
 qualname = whoosh_indexer
 propagate = 1
 
 <%text>##############</%text>
 <%text>## HANDLERS ##</%text>
 <%text>##############</%text>
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 level = INFO
 formatter = generic
 
 [handler_console_sql]
 class = StreamHandler
 args = (sys.stderr,)
 level = WARN
 formatter = generic
 
 <%text>################</%text>
 <%text>## FORMATTERS ##</%text>
 <%text>################</%text>
 
 [formatter_generic]
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = kallithea.lib.colored_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = kallithea.lib.colored_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

kallithea/config/deployment.ini_tmpl

@@ -1,589 +1,598 b''
 ################################################################################
 ################################################################################
 # Kallithea - Example config                                                   #
 #                                                                              #
 # The %(here)s variable will be replaced with the parent directory of this file#
 ################################################################################
 ################################################################################
 
 [DEFAULT]
 debug = true
 pdebug = false
 
 ################################################################################
 ## Email settings                                                             ##
 ##                                                                            ##
 ## Refer to the documentation ("Email settings") for more details.            ##
 ##                                                                            ##
 ## It is recommended to use a valid sender address that passes access         ##
 ## validation and spam filtering in mail servers.                             ##
 ################################################################################
 
 ## 'From' header for application emails. You can optionally add a name.
 ## Default:
 #app_email_from = Kallithea
 ## Examples:
 #app_email_from = Kallithea <[e-mail]>
 #app_email_from = [e-mail]
 
 ## Subject prefix for application emails.
 ## A space between this prefix and the real subject is automatically added.
 ## Default:
 #email_prefix =
 ## Example:
 #email_prefix = [Kallithea]
 
 ## Recipients for error emails and fallback recipients of application mails.
 ## Multiple addresses can be specified, space-separated.
 ## Only addresses are allowed, do not add any name part.
 ## Default:
 #email_to =
 ## Examples:
 #email_to = [e-mail]
 #email_to = [e-mail] [e-mail]
 
 ## 'From' header for error emails. You can optionally add a name.
 ## Default:
 #error_email_from = [e-mail]
 ## Examples:
 #error_email_from = Kallithea Errors <[e-mail]>
 #error_email_from = [e-mail]
 
 ## SMTP server settings
 ## If specifying credentials, make sure to use secure connections.
 ## Default: Send unencrypted unauthenticated mails to the specified smtp_server.
 ## For "SSL", use smtp_use_ssl = true and smtp_port = 465.
 ## For "STARTTLS", use smtp_use_tls = true and smtp_port = 587.
 #smtp_server = smtp.example.com
 #smtp_username =
 #smtp_password =
 #smtp_port = 25
 #smtp_use_ssl = false
 #smtp_use_tls = false
 
 [server:main]
-## PASTE ##
-#use = egg:Paste#http
+## Gearbox default web server ##
+#use = egg:gearbox#wsgiref
 ## nr of worker threads to spawn
 #threadpool_workers = 1
 ## max request before thread respawn
 #threadpool_max_requests = 100
 ## option to use threads of process
 #use_threadpool = true
 
+## Gearbox gevent web server ##
+#use = egg:gearbox#gevent
+
 ## WAITRESS ##
 use = egg:waitress#main
 ## number of worker threads
 threads = 1
 ## MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
 ## use poll instead of select, fixes fd limits, may not work on old
 ## windows systems.
 #asyncore_use_poll = True
 
 ## GUNICORN ##
 #use = egg:gunicorn#main
 ## number of process workers. You must set `instance_id = *` when this option
 ## is set to more than one worker
 #workers = 1
 ## process name
 #proc_name = kallithea
 ## type of worker class, one of sync, eventlet, gevent, tornado
 ## recommended for bigger setup is using of of other than sync one
 #worker_class = sync
 #max_requests = 1000
 ## amount of time a worker can handle request before it gets killed and
 ## restarted
 #timeout = 3600
 
 ## UWSGI ##
 ## run with uwsgi --ini-paste-logged <inifile.ini>
 #[uwsgi]
 #socket = /tmp/uwsgi.sock
 #master = true
 #http = 127.0.0.1:5000
 
 ## set as deamon and redirect all output to file
 #daemonize = ./uwsgi_kallithea.log
 
 ## master process PID
 #pidfile = ./uwsgi_kallithea.pid
 
 ## stats server with workers statistics, use uwsgitop
 ## for monitoring, `uwsgitop 127.0.0.1:1717`
 #stats = 127.0.0.1:1717
 #memory-report = true
 
 ## log 5XX errors
 #log-5xx = true
 
 ## Set the socket listen queue size.
 #listen = 256
 
 ## Gracefully Reload workers after the specified amount of managed requests
 ## (avoid memory leaks).
 #max-requests = 1000
 
 ## enable large buffers
 #buffer-size = 65535
 
 ## socket and http timeouts ##
 #http-timeout = 3600
 #socket-timeout = 3600
 
 ## Log requests slower than the specified number of milliseconds.
 #log-slow = 10
 
 ## Exit if no app can be loaded.
 #need-app = true
 
 ## Set lazy mode (load apps in workers instead of master).
 #lazy = true
 
 ## scaling ##
 ## set cheaper algorithm to use, if not set default will be used
 #cheaper-algo = spare
 
 ## minimum number of workers to keep at all times
 #cheaper = 1
 
 ## number of workers to spawn at startup
 #cheaper-initial = 1
 
 ## maximum number of workers that can be spawned
 #workers = 4
 
 ## how many workers should be spawned at a time
 #cheaper-step = 1
 
 ## COMMON ##
 host = 127.0.0.1
 port = 5000
 
 ## middleware for hosting the WSGI application under a URL prefix
 #[filter:proxy-prefix]
 #use = egg:PasteDeploy#prefix
 #prefix = /<your-prefix>
 
 [app:main]
 use = egg:kallithea
 ## enable proxy prefix middleware
 #filter-with = proxy-prefix
 
 full_stack = true
 static_files = true
 ## Available Languages:
 ## cs de fr hu ja nl_BE pl pt_BR ru sk zh_CN zh_TW
 lang =
 cache_dir = %(here)s/data
 index_dir = %(here)s/data/index
 
 ## perform a full repository scan on each server start, this should be
 ## set to false after first startup, to allow faster server restarts.
 initial_repo_scan = false
 
 ## uncomment and set this path to use archive download cache
 archive_cache_dir = %(here)s/tarballcache
 
 ## change this to unique ID for security
 app_instance_uuid = ${app_instance_uuid}
 
 ## cut off limit for large diffs (size in bytes)
 cut_off_limit = 256000
 
 ## force https in Kallithea, fixes https redirects, assumes it's always https
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## number of commits stats will parse on each iteration
 commit_parse_limit = 25
 
 ## path to git executable
 git_path = git
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 #git_rev_filter = --branches --tags
 
 ## RSS feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## options for showing and identifying changesets
 show_sha_length = 12
 show_revision_number = false
 
 ## Canonical URL to use when creating full URLs in UI and texts.
 ## Useful when the site is available under different names or protocols.
 ## Defaults to what is provided in the WSGI environment.
 #canonical_url = https://kallithea.example.com/repos
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/<gistid>.
 ## example: http://gist.example.com/{gistid}. Empty means use the internal
 ## Kallithea url, ie. http[s]://kallithea.example.com/_admin/gists/<gistid>
 gist_alias_url =
 
 ## white list of API enabled controllers. This allows to add list of
 ## controllers to which access will be enabled by api_key. eg: to enable
 ## api access to raw_files put `FilesController:raw`, to enable access to patches
 ## add `ChangesetController:changeset_patch`. This list should be "," separated
 ## Syntax is <ControllerClass>:<function>. Check debug logs for generated names
 ## Recommended settings below are commented out:
 api_access_controllers_whitelist =
 #    ChangesetController:changeset_patch,
 #    ChangesetController:changeset_raw,
 #    FilesController:raw,
 #    FilesController:archivefile
 
 ## default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = utf8
 
 ## issue tracker for Kallithea (leave blank to disable, absent for default)
 #bugtracker = https://bitbucket.org/conservancy/kallithea/issues
 
 ## issue tracking mapping for commits messages
 ## comment out issue_pat, issue_server, issue_prefix to enable
 
 ## pattern to get the issues from commit messages
 ## default one used here is #<numbers> with a regex passive group for `#`
 ## {id} will be all groups matched from this pattern
 
 issue_pat = (?:\s*#)(\d+)
 
 ## server url to the issue, each {id} will be replaced with match
 ## fetched from the regex and {repo} is replaced with full repository name
 ## including groups {repo_name} is replaced with just name of repo
 
 issue_server_link = https://issues.example.com/{repo}/issue/{id}
 
 ## prefix to add to link to indicate it's an url
 ## #314 will be replaced by <issue_prefix><id>
 
 issue_prefix = #
 
 ## issue_pat, issue_server_link, issue_prefix can have suffixes to specify
 ## multiple patterns, to other issues server, wiki or others
 ## below an example how to create a wiki pattern
 # wiki-some-id -> https://wiki.example.com/some-id
 
 #issue_pat_wiki = (?:wiki-)(.+)
 #issue_server_link_wiki = https://wiki.example.com/{id}
 #issue_prefix_wiki = WIKI-
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently Mercurial clients have trouble with
 ## handling that. Set this variable to 403 to return HTTPForbidden
 auth_ret_code =
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = True
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = True
 
 ## extra extensions for indexing, space separated and without the leading '.'.
 # index.extensions =
 #    gemfile
 #    lock
 
 ## extra filenames for indexing, space separated
 # index.filenames =
 #    .dockerignore
 #    .editorconfig
 #    INSTALL
 #    CHANGELOG
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 
 use_celery = false
 
 ## Example: connect to the virtual host 'rabbitmqhost' on localhost as rabbitmq:
 broker.url = amqp://rabbitmq:qewqew@localhost:5672/rabbitmqhost
 
 celery.imports = kallithea.lib.celerylib.tasks
 celery.accept.content = pickle
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 
 celeryd.concurrency = 2
 celeryd.max.tasks.per.child = 1
 
 ## If true, tasks will never be sent to the queue, but executed locally instead.
 celery.always.eager = false
 
 ####################################
 ###         BEAKER CACHE        ####
 ####################################
 
 beaker.cache.data_dir = %(here)s/data/cache/data
 beaker.cache.lock_dir = %(here)s/data/cache/lock
 
 beaker.cache.regions = short_term,long_term,sql_cache_short
 
 beaker.cache.short_term.type = memory
 beaker.cache.short_term.expire = 60
 beaker.cache.short_term.key_length = 256
 
 beaker.cache.long_term.type = memory
 beaker.cache.long_term.expire = 36000
 beaker.cache.long_term.key_length = 256
 
 beaker.cache.sql_cache_short.type = memory
 beaker.cache.sql_cache_short.expire = 10
 beaker.cache.sql_cache_short.key_length = 256
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## Name of session cookie. Should be unique for a given host and path, even when running
 ## on different ports. Otherwise, cookie sessions will be shared and messed up.
 beaker.session.key = kallithea
 ## Sessions should always only be accessible by the browser, not directly by JavaScript.
 beaker.session.httponly = true
 ## Session lifetime. 2592000 seconds is 30 days.
 beaker.session.timeout = 2592000
 
 ## Server secret used with HMAC to ensure integrity of cookies.
 beaker.session.secret = ${app_instance_uuid}
 ## Further, encrypt the data with AES.
 #beaker.session.encrypt_key = <key_for_encryption>
 #beaker.session.validate_key = <validation_key>
 
 ## Type of storage used for the session, current types are
 ## dbm, file, memcached, database, and memory.
 
 ## File system storage of session data. (default)
 #beaker.session.type = file
 
 ## Cookie only, store all session data inside the cookie. Requires secure secrets.
 #beaker.session.type = cookie
 
 ## Database storage of session data.
 #beaker.session.type = ext:database
 #beaker.session.sa.url = postgresql://postgres:qwe@localhost/kallithea
 #beaker.session.table_name = db_session
 
 ############################
 ## ERROR HANDLING SYSTEMS ##
 ############################
 
 ####################
 ### [appenlight] ###
 ####################
 
 ## AppEnlight is tailored to work with Kallithea, see
 ## http://appenlight.com for details how to obtain an account
 ## you must install python package `appenlight_client` to make it work
 
 ## appenlight enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 #appenlight.logging = true
 
 ## minimum log level for log capture
 #appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
 ## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 ################
 ### [sentry] ###
 ################
 
 ## sentry is a alternative open source error aggregator
 ## you must install python packages `sentry` and `raven` to enable
 
 sentry.dsn = YOUR_DNS
 sentry.servers =
 sentry.name =
 sentry.key =
 sentry.public_key =
 sentry.secret_key =
 sentry.project =
 sentry.site =
 sentry.include_paths =
 sentry.exclude_paths =
 
 ################################################################################
 ## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 ## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 ## execute malicious code after an exception is raised.                       ##
 ################################################################################
 set debug = false
 
 ##################################
 ###       LOGVIEW CONFIG       ###
 ##################################
 
 logview.sqlalchemy = #faa
 logview.pylons.templating = #bfb
 logview.pylons.util = #eee
 
 #########################################################
 ### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 #########################################################
 
 # SQLITE [default]
 sqlalchemy.url = sqlite:///%(here)s/kallithea.db?timeout=60
 
 # POSTGRESQL
 #sqlalchemy.url = postgresql://user:pass@localhost/kallithea
 
 # MySQL
 #sqlalchemy.url = mysql://user:pass@localhost/kallithea?charset=utf8
 
 # see sqlalchemy docs for others
 
 sqlalchemy.echo = false
 sqlalchemy.pool_recycle = 3600
 
 ################################
 ### ALEMBIC CONFIGURATION   ####
 ################################
 
 [alembic]
 script_location = kallithea:alembic
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 
 [loggers]
-keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer
+keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_routes]
 level = DEBUG
 handlers =
 qualname = routes.middleware
 ## "level = DEBUG" logs the route matched and routing variables.
 propagate = 1
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_templates]
 level = INFO
 handlers =
 qualname = pylons.templating
 propagate = 1
 
 [logger_kallithea]
 level = DEBUG
 handlers =
 qualname = kallithea
 propagate = 1
 
+[logger_gearbox]
+level = DEBUG
+handlers =
+qualname = gearbox
+propagate = 1
+
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_whoosh_indexer]
 level = DEBUG
 handlers =
 qualname = whoosh_indexer
 propagate = 1
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 level = INFO
 formatter = generic
 
 [handler_console_sql]
 class = StreamHandler
 args = (sys.stderr,)
 level = WARN
 formatter = generic
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = kallithea.lib.colored_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = kallithea.lib.colored_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

kallithea/lib/dbmigrate/__init__.py

@@ -1,11 +1,12 b''
-from paste.script.command import Command
+from gearbox.command import Command
 
 class UpgradeDb(Command):
-    hidden = True
-    summary = '(removed)'
+    '''(removed)'''
+
+    deprecated = True
 
     def run(self, args):
         raise SystemExit(
             'The "paster upgrade-db" command has been removed; please see the docs:\n'
             '    https://kallithea.readthedocs.io/en/default/upgrade.html'
         )

kallithea/lib/paster_commands/cache_keys.py

@@ -1,79 +1,72 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.cache_keys
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-cleanup-keys paster command for Kallithea
+cleanup-keys gearbox command for Kallithea
 
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: mar 27, 2013
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 
 from kallithea.lib.paster_commands.common import BasePasterCommand
 from kallithea.model.meta import Session
 from kallithea.lib.utils2 import safe_str
 from kallithea.model.db import CacheInvalidation
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    "Kallithea: Utilities for managing caching of database content"
 
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Cache keys utils"
-
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-
+    def take_action(self, args):
         _caches = CacheInvalidation.query().order_by(CacheInvalidation.cache_key).all()
-        if self.options.show:
+        if args.show:
             for c_obj in _caches:
                 print 'key:%s active:%s' % (safe_str(c_obj.cache_key), c_obj.cache_active)
-        elif self.options.cleanup:
+        elif args.cleanup:
             for c_obj in _caches:
                 Session().delete(c_obj)
                 print 'Removing key: %s' % (safe_str(c_obj.cache_key))
             Session().commit()
         else:
             print 'Nothing done, exiting...'
 
-    def update_parser(self):
-        self.parser.add_option(
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument(
             '--show',
             action='store_true',
             dest='show',
-            help=("show existing cache keys with together with status")
+            help="show existing cache keys with together with status",
         )
 
-        self.parser.add_option(
+        parser.add_argument(
             '--cleanup',
             action="store_true",
             dest="cleanup",
-            help="cleanup existing cache keys"
+            help="cleanup existing cache keys",
         )
+
+        return parser

kallithea/lib/paster_commands/celeryd.py

@@ -1,46 +1,50 b''
 # -*- coding: utf-8 -*-
 
+import argparse
+
 import kallithea
 from kallithea.lib.paster_commands.common import BasePasterCommand
 from kallithea.lib.utils import load_rcextensions
 from kallithea.lib.utils2 import str2bool
 
 __all__ = ['Command']
 
 
 class Command(BasePasterCommand):
-    """Start the celery worker
-
-    Starts the celery worker that uses a paste.deploy configuration
-    file.
-    """
+    """Kallithea: Celery worker for asynchronous tasks"""
 
-    usage = 'CONFIG_FILE [celeryd options...]'
-    summary = __doc__.splitlines()[0]
-    description = "".join(__doc__.splitlines()[2:])
-    group_name = "Kallithea"
-
-    parser = BasePasterCommand.standard_parser(quiet=True)
+    # Starts the celery worker using configuration from a paste.deploy
+    # configuration file.
 
-    def update_parser(self):
-        from kallithea.lib import celerypylons
-        cmd = celerypylons.worker.worker(celerypylons.app.app_or_default())
-        for x in cmd.get_options():
-            self.parser.add_option(x)
+    requires_db_session = False # will start session on demand
 
-    def command(self):
+    def take_action(self, args):
         from kallithea.lib import celerypylons
         from tg import config
         try:
             CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
         except KeyError:
             CELERY_ON = False
 
         if not CELERY_ON:
             raise Exception('Please set use_celery = true in .ini config '
                             'file before running celeryd')
         kallithea.CELERY_ON = CELERY_ON
 
         load_rcextensions(config['here'])
         cmd = celerypylons.worker.worker(celerypylons.app.app_or_default())
-        return cmd.run(**vars(self.options))
+
+        celery_args = args.celery_args
+        if '--' in celery_args:
+            celery_args.remove('--')
+
+        return cmd.run_from_argv('kallithea celery worker', celery_args)
+
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument('celery_args', nargs=argparse.REMAINDER,
+            help="Pass extra options to Celery after a '--' separator",
+            )
+
+        return parser

kallithea/lib/paster_commands/cleanup.py

@@ -1,149 +1,142 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.cleanup
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-cleanup-repos paster command for Kallithea
+cleanup-repos gearbox command for Kallithea
 
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Jul 14, 2012
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 import re
 import shutil
 import datetime
 
 from kallithea.lib.paster_commands.common import ask_ok, BasePasterCommand
 from kallithea.lib.utils import REMOVED_REPO_PAT
 from kallithea.lib.utils2 import safe_str
 from kallithea.model.db import Ui
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
-
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Cleanup deleted repos"
+    """Kallithea: Cleanup of backup files of deleted repositories"""
 
     def _parse_older_than(self, val):
         regex = re.compile(r'((?P<days>\d+?)d)?((?P<hours>\d+?)h)?((?P<minutes>\d+?)m)?((?P<seconds>\d+?)s)?')
         parts = regex.match(val)
         if not parts:
             return
         parts = parts.groupdict()
         time_params = {}
         for (name, param) in parts.iteritems():
             if param:
                 time_params[name] = int(param)
         return datetime.timedelta(**time_params)
 
     def _extract_date(self, name):
         """
         Extract the date part from rm__<date> pattern of removed repos,
         and convert it to datetime object
 
         :param name:
         """
         date_part = name[4:19]  # 4:19 since we don't parse milliseconds
         return datetime.datetime.strptime(date_part, '%Y%m%d_%H%M%S')
 
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-
+    def take_action(self, args):
         repos_location = Ui.get_repos_location()
         to_remove = []
         for dn_, dirs, f in os.walk(safe_str(repos_location)):
             alldirs = list(dirs)
             del dirs[:]
             if ('.hg' in alldirs or
                 '.git' in alldirs or
                 '.svn' in alldirs or
                 'objects' in alldirs and ('refs' in alldirs or 'packed-refs' in f)):
                 continue
             for loc in alldirs:
                 if REMOVED_REPO_PAT.match(loc):
                     to_remove.append([os.path.join(dn_, loc),
                                       self._extract_date(loc)])
                 else:
                     dirs.append(loc)
             if dirs:
                 print 'Scanning: %s' % dn_
 
         #filter older than (if present)!
         now = datetime.datetime.now()
-        older_than = self.options.older_than
+        older_than = args.older_than
         if older_than:
             to_remove_filtered = []
             older_than_date = self._parse_older_than(older_than)
             for name, date_ in to_remove:
                 repo_age = now - date_
                 if repo_age > older_than_date:
                     to_remove_filtered.append([name, date_])
 
             to_remove = to_remove_filtered
             print 'Removing %s deleted repos older than %s (%s)' \
                 % (len(to_remove), older_than, older_than_date)
         else:
             print 'Removing all %s deleted repos' % len(to_remove)
-        if self.options.dont_ask or not to_remove:
+        if args.dont_ask or not to_remove:
             # don't ask just remove !
             remove = True
         else:
             remove = ask_ok('the following repositories will be deleted completely:\n%s\n'
                             'are you sure you want to remove them [y/n]?'
                             % '\n'.join(['%s removed on %s' % (safe_str(x[0]), safe_str(x[1]))
                                          for x in to_remove]))
 
         if remove:
             for path, date_ in to_remove:
                 print 'Removing repository %s' % path
                 shutil.rmtree(path)
         else:
             print 'Nothing done, exiting...'
 
-    def update_parser(self):
-        self.parser.add_option(
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument(
             '--older-than',
             action='store',
             dest='older_than',
             help=("only remove repos that have been removed "
                  "at least given time ago. "
                  "The default is to remove all removed repositories. "
                  "Possible suffixes: "
                  "d (days), h (hours), m (minutes), s (seconds). "
                  "For example --older-than=30d deletes repositories "
                  "removed more than 30 days ago.")
             )
 
-        self.parser.add_option(
+        parser.add_argument(
             '--dont-ask',
             action="store_true",
             dest="dont_ask",
             help="remove repositories without asking for confirmation."
         )
+
+        return parser

kallithea/lib/paster_commands/common.py

@@ -1,107 +1,108 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.common
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Common code for Paster commands.
+Common code for gearbox commands.
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Apr 18, 2010
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 import os
-import logging
+import sys
+import logging.config
 
-import paste
-from paste.script.command import Command, BadCommand
-
-from kallithea.lib.utils import setup_cache_regions
+import paste.deploy
+import gearbox.command
 
 
 def ask_ok(prompt, retries=4, complaint='Yes or no please!'):
     while True:
         ok = raw_input(prompt)
         if ok in ('y', 'ye', 'yes'):
             return True
         if ok in ('n', 'no', 'nop', 'nope'):
             return False
         retries = retries - 1
         if retries < 0:
             raise IOError
         print complaint
 
 
-class BasePasterCommand(Command):
+class BasePasterCommand(gearbox.command.Command):
     """
-    Abstract Base Class for paster commands.
+    Abstract Base Class for gearbox commands.
     """
-    min_args = 1
-    min_args_error = "Please provide a paster config file as an argument."
-    takes_config_file = 1
-    requires_config_file = True
+
+    # override to control how much get_parser and run should do:
+    takes_config_file = True
+    requires_db_session = True
 
     def run(self, args):
         """
         Overrides Command.run
 
         Checks for a config file argument and loads it.
         """
-        if len(args) < self.min_args:
-            raise BadCommand(
-                self.min_args_error % {'min_args': self.min_args,
-                                       'actual_args': len(args)})
+        if self.takes_config_file:
+             self._bootstrap_config(args.config_file)
+             if self.requires_db_session:
+                  self._init_session()
 
-        # Decrement because we're going to lob off the first argument.
-        # @@ This is hacky
-        self.min_args -= 1
-        self.bootstrap_config(args[0])
-        self.update_parser()
-        return super(BasePasterCommand, self).run(args[1:])
+        return super(BasePasterCommand, self).run(args)
+
+    def get_parser(self, prog_name):
+        parser = super(BasePasterCommand, self).get_parser(prog_name)
 
-    def update_parser(self):
+        if self.takes_config_file:
+            parser.add_argument("-c", "--config",
+                help='Kallithea .ini file with configuration of database etc',
+                dest='config_file', required=True)
+
+        return parser
+
+    def _bootstrap_config(self, config_file):
         """
-        Abstract method.  Allows for the class's parser to be updated
-        before the superclass's `run` method is called.  Necessary to
-        allow options/arguments to be passed through to the underlying
-        celery command.
-        """
-        raise NotImplementedError("Abstract Method.")
-
-    def bootstrap_config(self, conf):
-        """
-        Loads the app configuration.
+        Read the config file and initialize logging and the application.
         """
         from tg import config as pylonsconfig
 
-        self.path_to_ini_file = os.path.realpath(conf)
-        conf = paste.deploy.appconfig('config:' + self.path_to_ini_file)
+        path_to_ini_file = os.path.realpath(config_file)
+        conf = paste.deploy.appconfig('config:' + path_to_ini_file)
+        logging.config.fileConfig(path_to_ini_file)
         pylonsconfig.init_app(conf.global_conf, conf.local_conf)
 
     def _init_session(self):
         """
-        Inits SqlAlchemy Session
+        Initialize SqlAlchemy Session from global config.
         """
-        logging.config.fileConfig(self.path_to_ini_file)
 
         from tg import config
         from kallithea.model.base import init_model
         from kallithea.lib.utils2 import engine_from_config
+        from kallithea.lib.utils import setup_cache_regions
         setup_cache_regions(config)
         engine = engine_from_config(config, 'sqlalchemy.')
         init_model(engine)
+
+    def error(self, msg, exitcode=1):
+        """Write error message and exit"""
+        sys.stderr.write('%s\n' % msg)
+        raise SystemExit(exitcode)

kallithea/lib/paster_commands/install_iis.py

@@ -1,104 +1,99 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.install_iis
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 IIS installation tools for Kallithea
 """
 
 
 import os
 import sys
 from paste.script.appinstall import AbstractInstallCommand
-from paste.script.command import BadCommand
+
+from kallithea.lib.paster_commands.common import BasePasterCommand
 
 
-class Command(AbstractInstallCommand):
-    default_verbosity = 1
-    max_args = 1
-    min_args = 1
-    summary = 'Setup IIS given a config file'
-    usage = 'CONFIG_FILE'
+class Command(BasePasterCommand):
+    '''Kallithea: Install into IIS using isapi-wsgi'''
 
-    description = '''
-    Script for installing into IIS using isapi-wsgi.
-    '''
-    parser = AbstractInstallCommand.standard_parser(
-        simulate=True, quiet=True, interactive=True)
-    parser.add_option('--virtualdir',
-                      action='store',
-                      dest='virtualdir',
-                      default='/',
-                      help='The virtual folder to install into on IIS')
+    requires_db_session = False
 
-    def command(self):
-        config_spec = self.args[0]
-        if not config_spec.startswith('config:'):
-            config_spec = 'config:' + config_spec
-        config_file = config_spec[len('config:'):].split('#', 1)[0]
-        config_file = os.path.join(os.getcwd(), config_file)
+    def take_action(self, args):
+        config_file = os.path.abspath(args.config_file)
         try:
             import isapi_wsgi
         except ImportError:
-            raise BadCommand('missing requirement: isapi-wsgi not installed')
+            self.error('missing requirement: isapi-wsgi not installed')
 
         file = '''\
 # Created by Kallithea install_iis
 import sys
 
 if hasattr(sys, "isapidllhandle"):
     import win32traceutil
 
 import isapi_wsgi
 import os
 
 def __ExtensionFactory__():
     from paste.deploy import loadapp
     from paste.script.util.logging_config import fileConfig
     fileConfig('%(inifile)s')
     application = loadapp('config:%(inifile)s')
 
     def app(environ, start_response):
         user = environ.get('REMOTE_USER', None)
         if user is not None:
             os.environ['REMOTE_USER'] = user
         return application(environ, start_response)
 
     return isapi_wsgi.ISAPIThreadPoolHandler(app)
 
 if __name__=='__main__':
     from isapi.install import *
     params = ISAPIParameters()
     sm = [ScriptMapParams(Extension="*", Flags=0)]
     vd = VirtualDirParameters(Name="%(virtualdir)s",
                               Description = "Kallithea",
                               ScriptMaps = sm,
                               ScriptMapUpdate = "replace")
     params.VirtualDirs = [vd]
     HandleCommandLine(params)
 '''
 
         outdata = file % {
                 'inifile': config_file.replace('\\', '\\\\'),
-                'virtualdir': self.options.virtualdir
+                'virtualdir': args.virtualdir,
                 }
 
         dispatchfile = os.path.join(os.getcwd(), 'dispatch.py')
         self.ensure_file(dispatchfile, outdata, False)
         print 'Generating %s' % (dispatchfile,)
 
         print ('Run \'python "%s" install\' with administrative privileges '
             'to generate the _dispatch.dll file and install it into the '
             'default web site') % (dispatchfile,)
+
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument('--virtualdir',
+                      action='store',
+                      dest='virtualdir',
+                      default='/',
+                      help='The virtual folder to install into on IIS')
+
+        return parser

kallithea/lib/paster_commands/ishell.py

@@ -1,67 +1,53 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.ishell
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-interactive shell paster command for Kallithea
+interactive shell gearbox command for Kallithea
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Apr 4, 2013
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 
 # imports, used in IPython shell
 import time
 import shutil
 import datetime
 from kallithea.model.db import *
 
 from kallithea.lib.paster_commands.common import BasePasterCommand
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    "Kallithea: Interactive Python shell"
 
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Interactive shell"
-
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-
+    def take_action(self, args):
         try:
             from IPython import embed
             from IPython.config.loader import Config
             cfg = Config()
             cfg.InteractiveShellEmbed.confirm_exit = False
             embed(config=cfg, banner1="Kallithea IShell.")
         except ImportError:
             print 'Kallithea ishell requires the IPython Python package'
             sys.exit(-1)
-
-    def update_parser(self):
-        pass

kallithea/lib/paster_commands/make_index.py

@@ -1,106 +1,100 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.make_index
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-make-index paster command for Kallithea
+make-index gearbox command for Kallithea
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Aug 17, 2010
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 from os.path import dirname
 
 from string import strip
 from kallithea.model.repo import RepoModel
 from kallithea.lib.paster_commands.common import BasePasterCommand
 from kallithea.lib.utils import load_rcextensions
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    "Kallithea: Create or update full text search index"
 
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Creates or updates full text search index"
-
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-        from tg import config
+    def take_action(self, args):
+        from pylons import config
         index_location = config['index_dir']
         load_rcextensions(config['here'])
 
-        repo_location = self.options.repo_location \
-            if self.options.repo_location else RepoModel().repos_path
-        repo_list = map(strip, self.options.repo_list.split(',')) \
-            if self.options.repo_list else None
+        repo_location = args.repo_location \
+            if args.repo_location else RepoModel().repos_path
+        repo_list = map(strip, args.repo_list.split(',')) \
+            if args.repo_list else None
 
-        repo_update_list = map(strip, self.options.repo_update_list.split(',')) \
-            if self.options.repo_update_list else None
+        repo_update_list = map(strip, args.repo_update_list.split(',')) \
+            if args.repo_update_list else None
 
         #======================================================================
         # WHOOSH DAEMON
         #======================================================================
         from kallithea.lib.pidlock import LockHeld, DaemonLock
         from kallithea.lib.indexers.daemon import WhooshIndexingDaemon
         try:
             l = DaemonLock(file_=os.path.join(dirname(dirname(index_location)),
                                               'make_index.lock'))
             WhooshIndexingDaemon(index_location=index_location,
                                  repo_location=repo_location,
                                  repo_list=repo_list,
                                  repo_update_list=repo_update_list) \
-                .run(full_index=self.options.full_index)
+                .run(full_index=args.full_index)
             l.release()
         except LockHeld:
             sys.exit(1)
 
-    def update_parser(self):
-        self.parser.add_option('--repo-location',
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument('--repo-location',
                           action='store',
                           dest='repo_location',
                           help="Specifies repositories location to index OPTIONAL",
                           )
-        self.parser.add_option('--index-only',
+        parser.add_argument('--index-only',
                           action='store',
                           dest='repo_list',
                           help="Specifies a comma separated list of repositories "
                                 "to build index on. If not given all repositories "
                                 "are scanned for indexing. OPTIONAL",
                           )
-        self.parser.add_option('--update-only',
+        parser.add_argument('--update-only',
                           action='store',
                           dest='repo_update_list',
                           help="Specifies a comma separated list of repositories "
                                 "to re-build index on. OPTIONAL",
                           )
-        self.parser.add_option('-f',
+        parser.add_argument('-f',
                           action='store_true',
                           dest='full_index',
                           help="Specifies that index should be made full i.e"
                                 " destroy old and build from scratch",
                           default=False)
+
+        return parser

kallithea/lib/paster_commands/make_rcextensions.py

@@ -1,75 +1,66 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.make_rcextensions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-make-rcext paster command for Kallithea
+make-rcext gearbox command for Kallithea
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Mar 6, 2012
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 import pkg_resources
 
 from kallithea.lib.paster_commands.common import ask_ok, BasePasterCommand
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    """Kallithea: Write template file for extending Kallithea in Python
 
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Write template file for extending Kallithea in Python."
-    usage = "CONFIG_FILE"
-    description = '''\
-        A rcextensions directory with a __init__.py file will be created next to
-        the ini file. Local customizations in that file will survive upgrades.
-        The file contains instructions on how it can be customized.
-        '''
+    A rcextensions directory with a __init__.py file will be created next to
+    the ini file. Local customizations in that file will survive upgrades.
+    The file contains instructions on how it can be customized.
+    """
 
-    def command(self):
-        from tg import config
+    takes_config_file = False
+
+    def take_action(self, args):
+        from pylons import config
 
         here = config['here']
         content = pkg_resources.resource_string(
             'kallithea', os.path.join('config', 'rcextensions', '__init__.py')
         )
         ext_file = os.path.join(here, 'rcextensions', '__init__.py')
         if os.path.exists(ext_file):
             msg = ('Extension file already exists, do you want '
                    'to overwrite it ? [y/n]')
             if not ask_ok(msg):
                 print 'Nothing done, exiting...'
                 return
 
         dirname = os.path.dirname(ext_file)
         if not os.path.isdir(dirname):
             os.makedirs(dirname)
         with open(ext_file, 'wb') as f:
             f.write(content)
             print 'Wrote new extensions file to %s' % ext_file
-
-    def update_parser(self):
-        pass

kallithea/lib/paster_commands/repo_scan.py

@@ -1,70 +1,68 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.repo_scan
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-repo-scan paster command for Kallithea
+repo-scan gearbox command for Kallithea
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Feb 9, 2013
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 
 from kallithea.model.scm import ScmModel
 from kallithea.lib.paster_commands.common import BasePasterCommand
 from kallithea.lib.utils import repo2db_mapper
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    """Kallithea: Scan file system for repositories
 
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Rescan default location for new repositories"
+    Search under the repository root configured in the database,
+    all new repositories, and report missing ones with an option of removing them.
+    """
 
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-        rm_obsolete = self.options.delete_obsolete
+    def take_action(self, args):
+        rm_obsolete = args.delete_obsolete
         print 'Now scanning root location for new repos ...'
         added, removed = repo2db_mapper(ScmModel().repo_scan(),
                                         remove_obsolete=rm_obsolete)
         added = ', '.join(added) or '-'
         removed = ', '.join(removed) or '-'
         print 'Scan completed.'
         print 'Added: %s' % added
         if rm_obsolete:
             print 'Removed: %s' % removed
         else:
             print 'Missing: %s' % removed
 
-    def update_parser(self):
-        self.parser.add_option(
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument(
             '--delete-obsolete',
             action='store_true',
             help="Use this flag do delete repositories that are "
                  "present in Kallithea database but not on the filesystem",
         )
+
+        return parser

kallithea/lib/paster_commands/setup_db.py

@@ -1,126 +1,107 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.setup_db
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Databaset setup paster command for Kallithea
+Databaset setup gearbox command for Kallithea
 """
 
 
 import os
 import sys
-from paste.script.appinstall import AbstractInstallCommand
-from paste.script.command import BadCommand
-from paste.deploy import appconfig
+import paste.deploy
+
+from kallithea.lib.db_manage import DbManage
+from kallithea.lib.paster_commands.common import BasePasterCommand
+from kallithea.model.meta import Session
 
 
-class Command(AbstractInstallCommand):
+# This is almost like SetupAppCommand ... but we have to pass options and it is
+# thus simpler to drop websetup and reimplement everything
+class Command(BasePasterCommand):
+    """Kallithea: Configure the database specified in the .ini file
 
-    default_verbosity = 1
-    max_args = 1
-    min_args = 1
-    summary = "Setup an application, given a config file"
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-
-    description = """\
-        Setup Kallithea according to its configuration file.  This is
-        the second part of a two-phase web application installation
-        process (the first phase is prepare-app).  The setup process
-        consist of things like setting up databases, creating super user
+    Setup Kallithea according to its configuration file.  This is
+    the second part of a two-phase web application installation
+    process (the first phase is prepare-app). The setup process
+    consist of things like setting up databases and creating the admin user
     """
 
-    parser = AbstractInstallCommand.standard_parser(
-        simulate=True, quiet=True, interactive=True)
-    parser.add_option('--user',
-                      action='store',
-                      dest='username',
-                      default=None,
-                      help='Admin Username')
-    parser.add_option('--email',
-                      action='store',
-                      dest='email',
-                      default=None,
-                      help='Admin Email')
-    parser.add_option('--password',
-                      action='store',
-                      dest='password',
-                      default=None,
-                      help='Admin password min 6 chars')
-    parser.add_option('--repos',
-                      action='store',
-                      dest='repos_location',
-                      default=None,
-                      help='Absolute path to repositories location')
-    parser.add_option('--name',
-                      action='store',
-                      dest='section_name',
-                      default=None,
-                      help='The name of the section to set up (default: app:main)')
-    parser.add_option('--force-yes',
-                       action='store_true',
-                       dest='force_ask',
-                       default=None,
-                       help='Force yes to every question')
-    parser.add_option('--force-no',
-                       action='store_false',
-                       dest='force_ask',
-                       default=None,
-                       help='Force no to every question')
-    parser.add_option('--public-access',
-                       action='store_true',
-                       dest='public_access',
-                       default=None,
-                       help='Enable public access on this installation (default)')
-    parser.add_option('--no-public-access',
-                       action='store_false',
-                       dest='public_access',
-                       default=None,
-                       help='Disable public access on this installation ')
+    def get_description(self):
+        return self.__doc__.splitlines()[0]
+
+    requires_db_session = False # only available after this command has been run
+
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
 
-    def command(self):
-        config_spec = self.args[0]
-        section = self.options.section_name
-        if section is None:
-            if '#' in config_spec:
-                config_spec, section = config_spec.split('#', 1)
-            else:
-                section = 'main'
-        if not ':' in section:
-            plain_section = section
-            section = 'app:' + section
-        else:
-            plain_section = section.split(':', 1)[0]
-        if not config_spec.startswith('config:'):
-            config_spec = 'config:' + config_spec
-        if plain_section != 'main':
-            config_spec += '#' + plain_section
-        config_file = config_spec[len('config:'):].split('#', 1)[0]
-        config_file = os.path.join(os.getcwd(), config_file)
-        self.logging_file_config(config_file)
-        conf = appconfig(config_spec, relative_to=os.getcwd())
-        ep_name = conf.context.entry_point_name
-        ep_group = conf.context.protocol
-        dist = conf.context.distribution
-        if dist is None:
-            raise BadCommand(
-                "The section %r is not the application (probably a filter). "
-                "You should add #section_name, where section_name is the "
-                "section that configures your application" % plain_section)
-        installer = self.get_installer(dist, ep_group, ep_name)
-        installer.setup_config(
-            self, config_file, section, self.sysconfig_install_vars(installer))
-        self.call_sysconfig_functions(
-            'post_setup_hook', installer, config_file)
+        parser.add_argument('--user',
+                          action='store',
+                          dest='username',
+                          default=None,
+                          help='Admin Username')
+        parser.add_argument('--email',
+                          action='store',
+                          dest='email',
+                          default=None,
+                          help='Admin Email')
+        parser.add_argument('--password',
+                          action='store',
+                          dest='password',
+                          default=None,
+                          help='Admin password min 6 chars')
+        parser.add_argument('--repos',
+                          action='store',
+                          dest='repos_location',
+                          default=None,
+                          help='Absolute path to repositories location')
+        parser.add_argument('--force-yes',
+                           action='store_true',
+                           dest='force_ask',
+                           default=None,
+                           help='Force yes to every question')
+        parser.add_argument('--force-no',
+                           action='store_false',
+                           dest='force_ask',
+                           default=None,
+                           help='Force no to every question')
+        parser.add_argument('--public-access',
+                           action='store_true',
+                           dest='public_access',
+                           default=None,
+                           help='Enable public access on this installation (default)')
+        parser.add_argument('--no-public-access',
+                           action='store_false',
+                           dest='public_access',
+                           default=None,
+                           help='Disable public access on this installation ')
+
+        return parser
+
+    def take_action(self, opts):
+        path_to_ini_file = os.path.realpath(opts.config_file)
+        conf = paste.deploy.appconfig('config:' + path_to_ini_file)
+
+        dbconf = conf['sqlalchemy.url']
+        dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=conf['here'],
+                            tests=False, cli_args=vars(opts))
+        dbmanage.create_tables(override=True)
+        opts = dbmanage.config_prompt(None)
+        dbmanage.create_settings(opts)
+        dbmanage.create_default_user()
+        dbmanage.admin_prompt()
+        dbmanage.create_permissions()
+        dbmanage.populate_default_permissions()
+        Session().commit()

kallithea/lib/paster_commands/update_repoinfo.py

@@ -1,84 +1,76 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.paster_commands.update_repoinfo
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-update-repoinfo paster command for Kallithea
+update-repoinfo gearbox command for Kallithea
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Jul 14, 2012
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 
 import os
 import sys
 import string
 
 from kallithea.lib.paster_commands.common import BasePasterCommand
 from kallithea.lib.utils2 import safe_unicode
 from kallithea.model.db import Repository
 from kallithea.model.repo import RepoModel
 from kallithea.model.meta import Session
 
 
 class Command(BasePasterCommand):
-
-    max_args = 1
-    min_args = 1
+    "Kallithea: Update database cache of repository data"
 
-    usage = "CONFIG_FILE"
-    group_name = "Kallithea"
-    takes_config_file = -1
-    parser = BasePasterCommand.standard_parser(verbose=True)
-    summary = "Updates repositories caches for last changeset"
-
-    def command(self):
-        #get SqlAlchemy session
-        self._init_session()
-
-
-        if self.options.repo_update_list is None:
+    def take_action(self, args):
+        if args.repo_update_list is None:
             repo_list = Repository.query().all()
         else:
             repo_names = [safe_unicode(n.strip())
-                          for n in self.options.repo_update_list.split(',')]
+                          for n in args.repo_update_list.split(',')]
             repo_list = list(Repository.query()
                 .filter(Repository.repo_name.in_(repo_names)))
         for repo in repo_list:
             repo.update_changeset_cache()
         Session().commit()
 
-        if self.options.invalidate_cache:
+        if args.invalidate_cache:
             for r in repo_list:
                 r.set_invalidate()
             print 'Updated repo info and invalidated cache for %s repositories' % (len(repo_list))
         else:
             print 'Updated repo info for %s repositories' % (len(repo_list))
 
-    def update_parser(self):
-        self.parser.add_option('--update-only',
+    def get_parser(self, prog_name):
+        parser = super(Command, self).get_parser(prog_name)
+
+        parser.add_argument('--update-only',
                            action='store',
                            dest='repo_update_list',
                            help="Specifies a comma separated list of repositories "
                                 "to update last commit info for. OPTIONAL")
-        self.parser.add_option('--invalidate-cache',
+        parser.add_argument('--invalidate-cache',
                            action='store_true',
                            dest='invalidate_cache',
                            help="Trigger cache invalidation event for repos. "
                                 "OPTIONAL")
+
+        return parser

kallithea/tests/other/manual_test_vcs_operations.py

@@ -1,562 +1,562 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.tests.other.manual_test_vcs_operations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Test suite for making push/pull operations.
 
 Run it in two terminals::
- paster serve kallithea/tests/test.ini
+ gearbox serve -c kallithea/tests/test.ini
  KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test kallithea/tests/other/manual_test_vcs_operations.py
 
 You must have git > 1.8.1 for tests to work fine
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Dec 30, 2010
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 
 """
 
 import os
 import re
 import tempfile
 import time
 
 from tempfile import _RandomNameSequence
 from subprocess import Popen, PIPE
 
 from kallithea.tests.base import *
 from kallithea.tests.fixture import Fixture
 from kallithea.model.db import User, Repository, UserIpMap, CacheInvalidation
 from kallithea.model.meta import Session
 from kallithea.model.repo import RepoModel
 from kallithea.model.user import UserModel
 
 DEBUG = True
 HOST = '127.0.0.1:4999'  # test host
 
 fixture = Fixture()
 
 
 class Command(object):
 
     def __init__(self, cwd):
         self.cwd = cwd
 
     def execute(self, cmd, *args, **environ):
         """
         Runs command on the system with given ``args``.
         """
 
         command = cmd + ' ' + ' '.join(args)
         if DEBUG:
             print '*** CMD %s ***' % command
         testenv = dict(os.environ)
         testenv['LANG'] = 'en_US.UTF-8'
         testenv['LANGUAGE'] = 'en_US:en'
         testenv.update(environ)
         p = Popen(command, shell=True, stdout=PIPE, stderr=PIPE, cwd=self.cwd, env=testenv)
         stdout, stderr = p.communicate()
         if DEBUG:
             if stdout:
                 print 'stdout:', repr(stdout)
             if stderr:
                 print 'stderr:', repr(stderr)
         return stdout, stderr
 
 
 def _get_tmp_dir():
     return tempfile.mkdtemp(prefix='rc_integration_test')
 
 
 def _construct_url(repo, dest=None, **kwargs):
     if dest is None:
         #make temp clone
         dest = _get_tmp_dir()
     params = {
         'user': TEST_USER_ADMIN_LOGIN,
         'passwd': TEST_USER_ADMIN_PASS,
         'host': HOST,
         'cloned_repo': repo,
         'dest': dest
     }
     params.update(**kwargs)
     if params['user'] and params['passwd']:
         _url = 'http://%(user)s:%(passwd)s@%(host)s/%(cloned_repo)s %(dest)s' % params
     else:
         _url = 'http://(host)s/%(cloned_repo)s %(dest)s' % params
     return _url
 
 
 def _add_files_and_push(vcs, DEST, **kwargs):
     """
     Generate some files, add it to DEST repo and push back
     vcs is git or hg and defines what VCS we want to make those files for
 
     :param vcs:
     :param DEST:
     """
     # commit some stuff into this repo
     cwd = path = os.path.join(DEST)
     #added_file = os.path.join(path, '%ssetupążźć.py' % _RandomNameSequence().next())
     added_file = os.path.join(path, '%ssetup.py' % _RandomNameSequence().next())
     Command(cwd).execute('touch %s' % added_file)
     Command(cwd).execute('%s add %s' % (vcs, added_file))
 
     email = '[e-mail]'
     if os.name == 'nt':
         author_str = 'User <%s>' % email
     else:
         author_str = 'User ǝɯɐᴎ <%s>' % email
     for i in xrange(kwargs.get('files_no', 3)):
         cmd = """echo "added_line%s" >> %s""" % (i, added_file)
         Command(cwd).execute(cmd)
         if vcs == 'hg':
             cmd = """hg commit -m "committed new %s" -u "%s" "%s" """ % (
                 i, author_str, added_file
             )
         elif vcs == 'git':
             cmd = """git commit -m "committed new %s" --author "%s" "%s" """ % (
                 i, author_str, added_file
             )
         # git commit needs EMAIL on some machines
         Command(cwd).execute(cmd, EMAIL=email)
 
     # PUSH it back
     _REPO = None
     if vcs == 'hg':
         _REPO = HG_REPO
     elif vcs == 'git':
         _REPO = GIT_REPO
 
     kwargs['dest'] = ''
     clone_url = _construct_url(_REPO, **kwargs)
     if 'clone_url' in kwargs:
         clone_url = kwargs['clone_url']
     stdout = stderr = None
     if vcs == 'hg':
         stdout, stderr = Command(cwd).execute('hg push --verbose', clone_url)
     elif vcs == 'git':
         stdout, stderr = Command(cwd).execute('git push --verbose', clone_url + " master")
 
     return stdout, stderr
 
 
 def set_anonymous_access(enable=True):
     user = User.get_default_user()
     user.active = enable
     Session().commit()
     print '\tanonymous access is now:', enable
     if enable != User.get_default_user().active:
         raise Exception('Cannot set anonymous access')
 
 
 #==============================================================================
 # TESTS
 #==============================================================================
 
 
 def _check_proper_git_push(stdout, stderr):
     #WTF Git stderr is output ?!
     assert 'fatal' not in stderr
     assert 'rejected' not in stderr
     assert 'Pushing to' in stderr
     assert 'master -> master' in stderr
 
 
 class TestVCSOperations(TestController):
 
     @classmethod
     def setup_class(cls):
         #DISABLE ANONYMOUS ACCESS
         set_anonymous_access(False)
 
     def setup_method(self, method):
         r = Repository.get_by_repo_name(GIT_REPO)
         Repository.unlock(r)
         r.enable_locking = False
         Session().commit()
 
         r = Repository.get_by_repo_name(HG_REPO)
         Repository.unlock(r)
         r.enable_locking = False
         Session().commit()
 
     def test_clone_hg_repo_by_admin(self):
         clone_url = _construct_url(HG_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         assert 'requesting all changes' in stdout
         assert 'adding changesets' in stdout
         assert 'adding manifests' in stdout
         assert 'adding file changes' in stdout
 
         assert stderr == ''
 
     def test_clone_git_repo_by_admin(self):
         clone_url = _construct_url(GIT_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         assert 'Cloning into' in stdout + stderr
         assert stderr == '' or stdout == ''
 
     def test_clone_wrong_credentials_hg(self):
         clone_url = _construct_url(HG_REPO, passwd='bad!')
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
         assert 'abort: authorization failed' in stderr
 
     def test_clone_wrong_credentials_git(self):
         clone_url = _construct_url(GIT_REPO, passwd='bad!')
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
         assert 'fatal: Authentication failed' in stderr
 
     def test_clone_git_dir_as_hg(self):
         clone_url = _construct_url(GIT_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
         assert 'HTTP Error 404: Not Found' in stderr
 
     def test_clone_hg_repo_as_git(self):
         clone_url = _construct_url(HG_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
         assert 'not found' in stderr
 
     def test_clone_non_existing_path_hg(self):
         clone_url = _construct_url('trololo')
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
         assert 'HTTP Error 404: Not Found' in stderr
 
     def test_clone_non_existing_path_git(self):
         clone_url = _construct_url('trololo')
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
         assert 'not found' in stderr
 
     def test_push_new_file_hg(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(HG_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
         fixture.create_fork(HG_REPO, fork_name)
         clone_url = _construct_url(fork_name).split()[0]
         stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url)
 
         assert 'pushing to' in stdout
         assert 'Repository size' in stdout
         assert 'Last revision is now' in stdout
 
     def test_push_new_file_git(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(GIT_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         # commit some stuff into this repo
         fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
         fixture.create_fork(GIT_REPO, fork_name)
         clone_url = _construct_url(fork_name).split()[0]
         stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url)
         print [(x.repo_full_path,x.repo_path) for x in Repository.query()] # TODO: what is this for
         _check_proper_git_push(stdout, stderr)
 
     def test_push_invalidates_cache_hg(self):
         key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
                                                ==HG_REPO).scalar()
         if not key:
             key = CacheInvalidation(HG_REPO, HG_REPO)
             Session().add(key)
 
         key.cache_active = True
         Session().commit()
 
         DEST = _get_tmp_dir()
         clone_url = _construct_url(HG_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
         fixture.create_fork(HG_REPO, fork_name)
         clone_url = _construct_url(fork_name).split()[0]
         stdout, stderr = _add_files_and_push('hg', DEST, files_no=1, clone_url=clone_url)
 
         key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
                                                ==fork_name).all()
         assert key == []
 
     def test_push_invalidates_cache_git(self):
         key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
                                                ==GIT_REPO).scalar()
         if not key:
             key = CacheInvalidation(GIT_REPO, GIT_REPO)
             Session().add(key)
 
         key.cache_active = True
         Session().commit()
 
         DEST = _get_tmp_dir()
         clone_url = _construct_url(GIT_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         # commit some stuff into this repo
         fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
         fixture.create_fork(GIT_REPO, fork_name)
         clone_url = _construct_url(fork_name).split()[0]
         stdout, stderr = _add_files_and_push('git', DEST, files_no=1, clone_url=clone_url)
         _check_proper_git_push(stdout, stderr)
 
         key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
                                                ==fork_name).all()
         assert key == []
 
     def test_push_wrong_credentials_hg(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(HG_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         stdout, stderr = _add_files_and_push('hg', DEST, user='bad',
                                              passwd='name')
 
         assert 'abort: authorization failed' in stderr
 
     def test_push_wrong_credentials_git(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(GIT_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         stdout, stderr = _add_files_and_push('git', DEST, user='bad',
                                              passwd='name')
 
         assert 'fatal: Authentication failed' in stderr
 
     def test_push_back_to_wrong_url_hg(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(HG_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         stdout, stderr = _add_files_and_push('hg', DEST,
                                     clone_url='http://%s/tmp' % HOST)
 
         assert 'HTTP Error 404: Not Found' in stderr
 
     def test_push_back_to_wrong_url_git(self):
         DEST = _get_tmp_dir()
         clone_url = _construct_url(GIT_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         stdout, stderr = _add_files_and_push('git', DEST,
                                     clone_url='http://%s/tmp' % HOST)
 
         assert 'not found' in stderr
 
     def test_clone_and_create_lock_hg(self):
         # enable locking
         r = Repository.get_by_repo_name(HG_REPO)
         r.enable_locking = True
         Session().commit()
         # clone
         clone_url = _construct_url(HG_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         #check if lock was made
         r = Repository.get_by_repo_name(HG_REPO)
         assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
 
     def test_clone_and_create_lock_git(self):
         # enable locking
         r = Repository.get_by_repo_name(GIT_REPO)
         r.enable_locking = True
         Session().commit()
         # clone
         clone_url = _construct_url(GIT_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         #check if lock was made
         r = Repository.get_by_repo_name(GIT_REPO)
         assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
 
     def test_clone_after_repo_was_locked_hg(self):
         #lock repo
         r = Repository.get_by_repo_name(HG_REPO)
         Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
         #pull fails since repo is locked
         clone_url = _construct_url(HG_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
         msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
                 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
         assert msg in stderr
 
     def test_clone_after_repo_was_locked_git(self):
         #lock repo
         r = Repository.get_by_repo_name(GIT_REPO)
         Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
         #pull fails since repo is locked
         clone_url = _construct_url(GIT_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
         msg = ("""The requested URL returned error: 423""")
         assert msg in stderr
 
     def test_push_on_locked_repo_by_other_user_hg(self):
         #clone some temp
         DEST = _get_tmp_dir()
         clone_url = _construct_url(HG_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         #lock repo
         r = Repository.get_by_repo_name(HG_REPO)
         # let this user actually push !
         RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
                                           perm='repository.write')
         Session().commit()
         Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
 
         #push fails repo is locked by other user !
         stdout, stderr = _add_files_and_push('hg', DEST,
                                              user=TEST_USER_REGULAR_LOGIN,
                                              passwd=TEST_USER_REGULAR_PASS)
         msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
                 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
         assert msg in stderr
 
     def test_push_on_locked_repo_by_other_user_git(self):
         #clone some temp
         DEST = _get_tmp_dir()
         clone_url = _construct_url(GIT_REPO, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         #lock repo
         r = Repository.get_by_repo_name(GIT_REPO)
         # let this user actually push !
         RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
                                           perm='repository.write')
         Session().commit()
         Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
 
         #push fails repo is locked by other user !
         stdout, stderr = _add_files_and_push('git', DEST,
                                              user=TEST_USER_REGULAR_LOGIN,
                                              passwd=TEST_USER_REGULAR_PASS)
         err = 'Repository `%s` locked by user `%s`' % (GIT_REPO, TEST_USER_ADMIN_LOGIN)
         assert err in stderr
 
         #TODO: fix this somehow later on Git, Git is stupid and even if we throw
         #back 423 to it, it makes ANOTHER request and we fail there with 405 :/
 
         msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
                 % (GIT_REPO, TEST_USER_ADMIN_LOGIN))
         #msg = "405 Method Not Allowed"
         #assert msg in stderr
 
     def test_push_unlocks_repository_hg(self):
         # enable locking
         fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
         fixture.create_fork(HG_REPO, fork_name)
         r = Repository.get_by_repo_name(fork_name)
         r.enable_locking = True
         Session().commit()
         #clone some temp
         DEST = _get_tmp_dir()
         clone_url = _construct_url(fork_name, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         #check for lock repo after clone
         r = Repository.get_by_repo_name(fork_name)
         uid = User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
         assert r.locked[0] == uid
 
         #push is ok and repo is now unlocked
         stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url.split()[0])
         assert str('remote: Released lock on repo `%s`' % fork_name) in stdout
         #we need to cleanup the Session Here !
         Session.remove()
         r = Repository.get_by_repo_name(fork_name)
         assert r.locked == [None, None]
 
     #TODO: fix me ! somehow during tests hooks don't get called on Git
     def test_push_unlocks_repository_git(self):
         # enable locking
         fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
         fixture.create_fork(GIT_REPO, fork_name)
         r = Repository.get_by_repo_name(fork_name)
         r.enable_locking = True
         Session().commit()
         #clone some temp
         DEST = _get_tmp_dir()
         clone_url = _construct_url(fork_name, dest=DEST)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         #check for lock repo after clone
         r = Repository.get_by_repo_name(fork_name)
         assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
 
         #push is ok and repo is now unlocked
         stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url.split()[0])
         _check_proper_git_push(stdout, stderr)
 
         assert ('remote: Released lock on repo `%s`' % fork_name) in stderr
         #we need to cleanup the Session Here !
         Session.remove()
         r = Repository.get_by_repo_name(fork_name)
         assert r.locked == [None, None]
 
     def test_ip_restriction_hg(self):
         user_model = UserModel()
         try:
             user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
             Session().commit()
             clone_url = _construct_url(HG_REPO)
             stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
             assert 'abort: HTTP Error 403: Forbidden' in stderr
         finally:
             #release IP restrictions
             for ip in UserIpMap.query():
                 UserIpMap.delete(ip.ip_id)
             Session().commit()
 
         # IP permissions are cached, need to invalidate this cache explicitly
         invalidate_all_caches()
 
         clone_url = _construct_url(HG_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url)
 
         assert 'requesting all changes' in stdout
         assert 'adding changesets' in stdout
         assert 'adding manifests' in stdout
         assert 'adding file changes' in stdout
 
         assert stderr == ''
 
     def test_ip_restriction_git(self):
         user_model = UserModel()
         try:
             user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
             Session().commit()
             clone_url = _construct_url(GIT_REPO)
             stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
             # The message apparently changed in Git 1.8.3, so match it loosely.
             assert re.search(r'\b403\b', stderr)
         finally:
             #release IP restrictions
             for ip in UserIpMap.query():
                 UserIpMap.delete(ip.ip_id)
             Session().commit()
 
         # IP permissions are cached, need to invalidate this cache explicitly
         invalidate_all_caches()
 
         clone_url = _construct_url(GIT_REPO)
         stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url)
 
         assert 'Cloning into' in stdout + stderr
         assert stderr == '' or stdout == ''

kallithea/tests/scripts/create_rc.sh

@@ -1,17 +1,17 b''
 #!/bin/sh
 psql -U postgres -h localhost -c 'drop database if exists kallithea;'
 psql -U postgres -h localhost -c 'create database kallithea;'
-paster setup-db server.ini --force-yes --user=username --password=qweqwe --email=[e-mail] --repos=/home/username/repos --no-public-access
+gearbox setup-db -c server.ini --force-yes --user=username --password=qweqwe --email=[e-mail] --repos=/home/username/repos --no-public-access
 API_KEY=`psql -R " " -A -U postgres -h localhost -c "select api_key from users where admin=TRUE" -d kallithea | awk '{print $2}'`
 echo "run those after running server"
-paster serve server.ini --pid-file=server.pid --daemon
+gearbox serve -c server.ini --pid-file=server.pid --daemon
 sleep 3
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo1 password:qweqwe email:[e-mail]
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo2 password:qweqwe email:[e-mail]
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user username:demo3 password:qweqwe email:[e-mail]
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 create_user_group group_name:demo12
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo1
 kallithea-api --apikey=$API_KEY --apihost=http://127.0.0.1:5001 add_user_to_user_group usergroupid:demo12 userid:demo2
 echo "killing server"
 kill `cat server.pid`
 rm server.pid

kallithea/tests/test.ini

@@ -1,605 +1,614 b''
 ################################################################################
 ################################################################################
 # Kallithea - config for tests:                                                #
 # initial_repo_scan = true                                                     #
 # sqlalchemy and kallithea_test.sqlite                                         #
 # custom logging                                                               #
 #                                                                              #
 # The %(here)s variable will be replaced with the parent directory of this file#
 ################################################################################
 ################################################################################
 
 [DEFAULT]
 debug = true
 pdebug = false
 
 ################################################################################
 ## Email settings                                                             ##
 ##                                                                            ##
 ## Refer to the documentation ("Email settings") for more details.            ##
 ##                                                                            ##
 ## It is recommended to use a valid sender address that passes access         ##
 ## validation and spam filtering in mail servers.                             ##
 ################################################################################
 
 ## 'From' header for application emails. You can optionally add a name.
 ## Default:
 #app_email_from = Kallithea
 ## Examples:
 #app_email_from = Kallithea <[e-mail]>
 #app_email_from = [e-mail]
 
 ## Subject prefix for application emails.
 ## A space between this prefix and the real subject is automatically added.
 ## Default:
 #email_prefix =
 ## Example:
 #email_prefix = [Kallithea]
 
 ## Recipients for error emails and fallback recipients of application mails.
 ## Multiple addresses can be specified, space-separated.
 ## Only addresses are allowed, do not add any name part.
 ## Default:
 #email_to =
 ## Examples:
 #email_to = [e-mail]
 #email_to = [e-mail] [e-mail]
 
 ## 'From' header for error emails. You can optionally add a name.
 ## Default:
 #error_email_from = [e-mail]
 ## Examples:
 #error_email_from = Kallithea Errors <[e-mail]>
 #error_email_from = [e-mail]
 
 ## SMTP server settings
 ## If specifying credentials, make sure to use secure connections.
 ## Default: Send unencrypted unauthenticated mails to the specified smtp_server.
 ## For "SSL", use smtp_use_ssl = true and smtp_port = 465.
 ## For "STARTTLS", use smtp_use_tls = true and smtp_port = 587.
 #smtp_server = smtp.example.com
 #smtp_username =
 #smtp_password =
 #smtp_port = 25
 #smtp_use_ssl = false
 #smtp_use_tls = false
 
 [server:main]
-## PASTE ##
-#use = egg:Paste#http
+## Gearbox default web server ##
+#use = egg:gearbox#wsgiref
 ## nr of worker threads to spawn
 #threadpool_workers = 1
 ## max request before thread respawn
 #threadpool_max_requests = 100
 ## option to use threads of process
 #use_threadpool = true
 
+## Gearbox gevent web server ##
+#use = egg:gearbox#gevent
+
 ## WAITRESS ##
 use = egg:waitress#main
 ## number of worker threads
 threads = 1
 ## MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
 ## use poll instead of select, fixes fd limits, may not work on old
 ## windows systems.
 #asyncore_use_poll = True
 
 ## GUNICORN ##
 #use = egg:gunicorn#main
 ## number of process workers. You must set `instance_id = *` when this option
 ## is set to more than one worker
 #workers = 1
 ## process name
 #proc_name = kallithea
 ## type of worker class, one of sync, eventlet, gevent, tornado
 ## recommended for bigger setup is using of of other than sync one
 #worker_class = sync
 #max_requests = 1000
 ## amount of time a worker can handle request before it gets killed and
 ## restarted
 #timeout = 3600
 
 ## UWSGI ##
 ## run with uwsgi --ini-paste-logged <inifile.ini>
 #[uwsgi]
 #socket = /tmp/uwsgi.sock
 #master = true
 #http = 127.0.0.1:5000
 
 ## set as deamon and redirect all output to file
 #daemonize = ./uwsgi_kallithea.log
 
 ## master process PID
 #pidfile = ./uwsgi_kallithea.pid
 
 ## stats server with workers statistics, use uwsgitop
 ## for monitoring, `uwsgitop 127.0.0.1:1717`
 #stats = 127.0.0.1:1717
 #memory-report = true
 
 ## log 5XX errors
 #log-5xx = true
 
 ## Set the socket listen queue size.
 #listen = 256
 
 ## Gracefully Reload workers after the specified amount of managed requests
 ## (avoid memory leaks).
 #max-requests = 1000
 
 ## enable large buffers
 #buffer-size = 65535
 
 ## socket and http timeouts ##
 #http-timeout = 3600
 #socket-timeout = 3600
 
 ## Log requests slower than the specified number of milliseconds.
 #log-slow = 10
 
 ## Exit if no app can be loaded.
 #need-app = true
 
 ## Set lazy mode (load apps in workers instead of master).
 #lazy = true
 
 ## scaling ##
 ## set cheaper algorithm to use, if not set default will be used
 #cheaper-algo = spare
 
 ## minimum number of workers to keep at all times
 #cheaper = 1
 
 ## number of workers to spawn at startup
 #cheaper-initial = 1
 
 ## maximum number of workers that can be spawned
 #workers = 4
 
 ## how many workers should be spawned at a time
 #cheaper-step = 1
 
 ## COMMON ##
 host = 127.0.0.1
 #port = 5000
 port = 4999
 
 ## middleware for hosting the WSGI application under a URL prefix
 #[filter:proxy-prefix]
 #use = egg:PasteDeploy#prefix
 #prefix = /<your-prefix>
 
 [app:main]
 use = egg:kallithea
 ## enable proxy prefix middleware
 #filter-with = proxy-prefix
 
 full_stack = true
 static_files = true
 ## Available Languages:
 ## cs de fr hu ja nl_BE pl pt_BR ru sk zh_CN zh_TW
 lang =
 #cache_dir = %(here)s/data
 cache_dir = %(here)s/../../data/test/cache
 #index_dir = %(here)s/data/index
 index_dir = %(here)s/../../data/test/index
 
 ## perform a full repository scan on each server start, this should be
 ## set to false after first startup, to allow faster server restarts.
 #initial_repo_scan = false
 initial_repo_scan = true
 
 ## uncomment and set this path to use archive download cache
 #archive_cache_dir = %(here)s/tarballcache
 archive_cache_dir = %(here)s/../../data/test/tarballcache
 
 ## change this to unique ID for security
 app_instance_uuid = test
 
 ## cut off limit for large diffs (size in bytes)
 cut_off_limit = 256000
 
 ## force https in Kallithea, fixes https redirects, assumes it's always https
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## number of commits stats will parse on each iteration
 commit_parse_limit = 25
 
 ## path to git executable
 git_path = git
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 #git_rev_filter = --branches --tags
 
 ## RSS feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## options for showing and identifying changesets
 show_sha_length = 12
 #show_revision_number = false
 show_revision_number = true
 
 ## Canonical URL to use when creating full URLs in UI and texts.
 ## Useful when the site is available under different names or protocols.
 ## Defaults to what is provided in the WSGI environment.
 #canonical_url = https://kallithea.example.com/repos
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/<gistid>.
 ## example: http://gist.example.com/{gistid}. Empty means use the internal
 ## Kallithea url, ie. http[s]://kallithea.example.com/_admin/gists/<gistid>
 gist_alias_url =
 
 ## white list of API enabled controllers. This allows to add list of
 ## controllers to which access will be enabled by api_key. eg: to enable
 ## api access to raw_files put `FilesController:raw`, to enable access to patches
 ## add `ChangesetController:changeset_patch`. This list should be "," separated
 ## Syntax is <ControllerClass>:<function>. Check debug logs for generated names
 ## Recommended settings below are commented out:
 api_access_controllers_whitelist =
 #    ChangesetController:changeset_patch,
 #    ChangesetController:changeset_raw,
 #    FilesController:raw,
 #    FilesController:archivefile
 
 ## default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = utf8
 
 ## issue tracker for Kallithea (leave blank to disable, absent for default)
 #bugtracker = https://bitbucket.org/conservancy/kallithea/issues
 
 ## issue tracking mapping for commits messages
 ## comment out issue_pat, issue_server, issue_prefix to enable
 
 ## pattern to get the issues from commit messages
 ## default one used here is #<numbers> with a regex passive group for `#`
 ## {id} will be all groups matched from this pattern
 
 issue_pat = (?:\s*#)(\d+)
 
 ## server url to the issue, each {id} will be replaced with match
 ## fetched from the regex and {repo} is replaced with full repository name
 ## including groups {repo_name} is replaced with just name of repo
 
 issue_server_link = https://issues.example.com/{repo}/issue/{id}
 
 ## prefix to add to link to indicate it's an url
 ## #314 will be replaced by <issue_prefix><id>
 
 issue_prefix = #
 
 ## issue_pat, issue_server_link, issue_prefix can have suffixes to specify
 ## multiple patterns, to other issues server, wiki or others
 ## below an example how to create a wiki pattern
 # wiki-some-id -> https://wiki.example.com/some-id
 
 #issue_pat_wiki = (?:wiki-)(.+)
 #issue_server_link_wiki = https://wiki.example.com/{id}
 #issue_prefix_wiki = WIKI-
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently Mercurial clients have trouble with
 ## handling that. Set this variable to 403 to return HTTPForbidden
 auth_ret_code =
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = True
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = True
 
 ## extra extensions for indexing, space separated and without the leading '.'.
 # index.extensions =
 #    gemfile
 #    lock
 
 ## extra filenames for indexing, space separated
 # index.filenames =
 #    .dockerignore
 #    .editorconfig
 #    INSTALL
 #    CHANGELOG
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 
 use_celery = false
 
 ## Example: connect to the virtual host 'rabbitmqhost' on localhost as rabbitmq:
 broker.url = amqp://rabbitmq:qewqew@localhost:5672/rabbitmqhost
 
 celery.imports = kallithea.lib.celerylib.tasks
 celery.accept.content = pickle
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 
 celeryd.concurrency = 2
 celeryd.max.tasks.per.child = 1
 
 ## If true, tasks will never be sent to the queue, but executed locally instead.
 celery.always.eager = false
 
 ####################################
 ###         BEAKER CACHE        ####
 ####################################
 
 #beaker.cache.data_dir = %(here)s/data/cache/data
 beaker.cache.data_dir = %(here)s/../../data/test/cache/data
 #beaker.cache.lock_dir = %(here)s/data/cache/lock
 beaker.cache.lock_dir = %(here)s/../../data/test/cache/lock
 
 beaker.cache.regions = short_term,long_term,sql_cache_short
 
 beaker.cache.short_term.type = memory
 beaker.cache.short_term.expire = 60
 beaker.cache.short_term.key_length = 256
 
 beaker.cache.long_term.type = memory
 beaker.cache.long_term.expire = 36000
 beaker.cache.long_term.key_length = 256
 
 beaker.cache.sql_cache_short.type = memory
 #beaker.cache.sql_cache_short.expire = 10
 beaker.cache.sql_cache_short.expire = 1
 beaker.cache.sql_cache_short.key_length = 256
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## Name of session cookie. Should be unique for a given host and path, even when running
 ## on different ports. Otherwise, cookie sessions will be shared and messed up.
 beaker.session.key = kallithea
 ## Sessions should always only be accessible by the browser, not directly by JavaScript.
 beaker.session.httponly = true
 ## Session lifetime. 2592000 seconds is 30 days.
 beaker.session.timeout = 2592000
 
 ## Server secret used with HMAC to ensure integrity of cookies.
 beaker.session.secret = {74e0cd75-b339-478b-b129-07dd221def1f}
 ## Further, encrypt the data with AES.
 #beaker.session.encrypt_key = <key_for_encryption>
 #beaker.session.validate_key = <validation_key>
 
 ## Type of storage used for the session, current types are
 ## dbm, file, memcached, database, and memory.
 
 ## File system storage of session data. (default)
 #beaker.session.type = file
 
 ## Cookie only, store all session data inside the cookie. Requires secure secrets.
 #beaker.session.type = cookie
 
 ## Database storage of session data.
 #beaker.session.type = ext:database
 #beaker.session.sa.url = postgresql://postgres:qwe@localhost/kallithea
 #beaker.session.table_name = db_session
 
 ############################
 ## ERROR HANDLING SYSTEMS ##
 ############################
 
 ####################
 ### [appenlight] ###
 ####################
 
 ## AppEnlight is tailored to work with Kallithea, see
 ## http://appenlight.com for details how to obtain an account
 ## you must install python package `appenlight_client` to make it work
 
 ## appenlight enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 #appenlight.logging = true
 
 ## minimum log level for log capture
 #appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
 ## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 ################
 ### [sentry] ###
 ################
 
 ## sentry is a alternative open source error aggregator
 ## you must install python packages `sentry` and `raven` to enable
 
 sentry.dsn = YOUR_DNS
 sentry.servers =
 sentry.name =
 sentry.key =
 sentry.public_key =
 sentry.secret_key =
 sentry.project =
 sentry.site =
 sentry.include_paths =
 sentry.exclude_paths =
 
 ################################################################################
 ## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 ## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 ## execute malicious code after an exception is raised.                       ##
 ################################################################################
 set debug = false
 
 ##################################
 ###       LOGVIEW CONFIG       ###
 ##################################
 
 logview.sqlalchemy = #faa
 logview.pylons.templating = #bfb
 logview.pylons.util = #eee
 
 #########################################################
 ### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 #########################################################
 
 # SQLITE [default]
 #sqlalchemy.url = sqlite:///%(here)s/kallithea.db?timeout=60
 sqlalchemy.url = sqlite:///%(here)s/kallithea_test.sqlite
 
 # POSTGRESQL
 #sqlalchemy.url = postgresql://user:pass@localhost/kallithea
 
 # MySQL
 #sqlalchemy.url = mysql://user:pass@localhost/kallithea?charset=utf8
 
 # see sqlalchemy docs for others
 
 sqlalchemy.echo = false
 sqlalchemy.pool_recycle = 3600
 
 ################################
 ### ALEMBIC CONFIGURATION   ####
 ################################
 
 [alembic]
 script_location = kallithea:alembic
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 
 [loggers]
-keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer
+keys = root, routes, kallithea, sqlalchemy, gearbox, beaker, templates, whoosh_indexer
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_routes]
 level = DEBUG
 handlers =
 qualname = routes.middleware
 ## "level = DEBUG" logs the route matched and routing variables.
 propagate = 1
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_templates]
 level = INFO
 handlers =
 qualname = pylons.templating
 propagate = 1
 
 [logger_kallithea]
 level = DEBUG
 handlers =
 qualname = kallithea
 propagate = 1
 
+[logger_gearbox]
+level = DEBUG
+handlers =
+qualname = gearbox
+propagate = 1
+
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_whoosh_indexer]
 level = DEBUG
 handlers =
 qualname = whoosh_indexer
 propagate = 1
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 #level = INFO
 level = DEBUG
 #formatter = generic
 formatter = color_formatter
 
 [handler_console_sql]
 class = StreamHandler
 args = (sys.stderr,)
 level = WARN
 #formatter = generic
 formatter = color_formatter_sql
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = kallithea.lib.colored_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = kallithea.lib.colored_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

scripts/dbmigrate-test

@@ -1,121 +1,121 b''
 #!/bin/sh -e
 
 if [ $# -lt 2 ] || [ $# -gt 3 ]; then
     cat >&2 <<EOD
 usage: $0 CONFIG_FILE FROM_REV [TO_REV]
 
 Runs a database migration from FROM_REV to TO_REV (default: current
 working directory parent), using the specified CONFIG_FILE (.ini file).
 
 Test is run using a clean Kallithea install, in a temporary virtual
 environment. FROM_REV and (optional) TO_REV should be Mercurial revision
 identifiers (e.g. changeset hash or a version number tag). The working
 directory is not touched, but the database referenced in the config file
 will be (re)created.
 
 Only SQLite is available out of the box; for MySQL or PostgreSQL, set
 the EXTRA environment variable to the required package(s), and it'll
 be installed in the virtual environment. (E.g. EXTRA=MySQL-python or
 EXTRA=psycopg2.)
 
 The temporary directory is not removed, allowing follow-up examination
 of the upgrade results. It is, however, created in /tmp by default,
 which many Linux distributions automatically clean at regular intervals.
 EOD
     exit 1
 fi
 
 config_file=$(readlink -f "$1")
 from_rev=$2
 to_rev=$3
 source_repo=$(dirname "$(dirname "$(readlink -f "$0")")")
 
 announce() {
     echo
     echo "$1"
     echo
 }
 
 quiet_if_ok() (
     local output
     local st
     set +e
     output=$("$@" < /dev/null 2>&1)
     st=$?
     if [ $st -ne 0 ]; then
         echo "$output" >&2
         echo "Command $@ returned exit status $st." >&2
         exit 1
     fi
 )
 
 HG() {
     "${HG:-hg}" --repository "$source_repo" "$@"
 }
 
 # If upgrading to "current revision", warn if working directory is dirty.
 if [ ! "$to_rev" ] && [ "$(HG status -mard)" ]; then
     announce "Warning: Uncommitted changes in working directory will be ignored!"
 fi
 
 from_rev_hash=$(HG id --id --rev "${from_rev:-.}")
 to_rev_hash=$(HG id --id --rev "${to_rev:-.}")
 temp=$(readlink -f "$(mktemp --tmpdir -d 'dbmigrate-test.XXXXXX')")
 
 cat <<EOD
 Config file:    $config_file
 EOD
 sed -n -e 's/^sqlalchemy\.url *= */Database URL:   /p' "$config_file"
 cat <<EOD
 Working dir:    $temp
 Repository:     $source_repo
 Upgrade from:   $from_rev_hash (${from_rev:-current})
 Upgrade to:     $to_rev_hash (${to_rev:-current})
 Extra packages: ${EXTRA:-(none)}
 EOD
 
 mkdir "$temp/repos" # empty
 
 # Enable caching for old pip versions (this will cache the pip upgrade)
 # Newer pip versions cache automatically, and don't use this variable.
 if [ ! "$PIP_DOWNLOAD_CACHE" ]; then
     export PIP_DOWNLOAD_CACHE=$HOME/.cache/pip/legacy
 fi
 
 install_kallithea() {
     local prefix=$1
     local rev=$2
 
     announce "Installing Kallithea $rev in $prefix..."
 
     "${VIRTUALENV:-virtualenv}" --quiet "$prefix-env"
     HG archive --rev "$rev" "$prefix"
 
     (
         cd "$prefix"
         . "$prefix-env/bin/activate"
         pip install --quiet --upgrade pip setuptools mercurial $EXTRA
         pip install --quiet -e .
     )
 }
 
 install_kallithea "$temp/from" "$from_rev_hash"
 (
     cd "$temp/from"
     . "$temp/from-env/bin/activate"
     announce "Initializing database..."
-    quiet_if_ok paster setup-db "$config_file" --repos="$temp/repos" --user=doe --email=[e-mail] --password=123456 --no-public-access --force-yes
+    quiet_if_ok gearbox setup-db -c "$config_file" --repos="$temp/repos" --user=doe --email=[e-mail] --password=123456 --no-public-access --force-yes
     alembic -c "$config_file" current -v
 )
 
 install_kallithea "$temp/to" "$to_rev_hash"
 (
     cd "$temp/to"
     . "$temp/to-env/bin/activate"
 
     announce "Commencing database upgrade from shown Alembic revision to head..."
     alembic -c "$config_file" current -v
     alembic -c "$config_file" upgrade head
     announce "Upgrade complete, now at the shown Alembic revision:"
     alembic -c "$config_file" current -v
 )

setup.py

@@ -1,171 +1,170 b''
 #!/usr/bin/env python2
 # -*- coding: utf-8 -*-
 import os
 import sys
 import platform
 
 if sys.version_info < (2, 6) or sys.version_info >= (3,):
     raise Exception('Kallithea requires python 2.6 or 2.7')
 
 
 here = os.path.abspath(os.path.dirname(__file__))
 
 
 def _get_meta_var(name, data, callback_handler=None):
     import re
     matches = re.compile(r'(?:%s)\s*=\s*(.*)' % name).search(data)
     if matches:
         if not callable(callback_handler):
             callback_handler = lambda v: v
 
         return callback_handler(eval(matches.groups()[0]))
 
 _meta = open(os.path.join(here, 'kallithea', '__init__.py'), 'rb')
 _metadata = _meta.read()
 _meta.close()
 
 callback = lambda V: ('.'.join(map(str, V[:3])) + '.'.join(V[3:]))
 __version__ = _get_meta_var('VERSION', _metadata, callback)
 __license__ = _get_meta_var('__license__', _metadata)
 __author__ = _get_meta_var('__author__', _metadata)
 __url__ = _get_meta_var('__url__', _metadata)
 # defines current platform
 __platform__ = platform.system()
 
 is_windows = __platform__ in ['Windows']
 
 requirements = [
     "alembic>=0.8.0,<0.9",
+    "GearBox<1",
     "waitress>=0.8.8,<1.0",
     "webob>=1.7,<2",
     "Pylons>=1.0.0,<=1.0.2",
     "Beaker>=1.7.0,<2",
     "WebHelpers==1.3",
     "formencode>=1.2.4,<=1.2.6",
     "SQLAlchemy>=1.0,<1.1",
     "Mako>=0.9.0,<=1.0.0",
     "pygments>=1.5",
     "whoosh>=2.5.0,<=2.5.7",
     "celery>=3.1,<3.2",
     "babel>=0.9.6,<2.4",
     "python-dateutil>=1.5.0,<2.0.0",
     "markdown==2.2.1",
     "docutils>=0.8.1",
     "URLObject==2.3.4",
     "Routes==1.13",
     "dulwich>=0.14.1",
     "mercurial>=2.9,<4.2",
 ]
 
 if sys.version_info < (2, 7):
     requirements.append("importlib==1.0.1")
     requirements.append("argparse")
 
 if not is_windows:
     requirements.append("bcrypt>=3.1.0")
 
 dependency_links = [
 ]
 
 classifiers = [
     'Development Status :: 4 - Beta',
     'Environment :: Web Environment',
     'Framework :: Pylons',
     'Intended Audience :: Developers',
     'License :: OSI Approved :: GNU General Public License (GPL)',
     'Operating System :: OS Independent',
     'Programming Language :: Python',
     'Programming Language :: Python :: 2.6',
     'Programming Language :: Python :: 2.7',
     'Topic :: Software Development :: Version Control',
 ]
 
 
 # additional files from project that goes somewhere in the filesystem
 # relative to sys.prefix
 data_files = []
 
 description = ('Kallithea is a fast and powerful management tool '
                'for Mercurial and Git with a built in push/pull server, '
                'full text search and code-review.')
 
 keywords = ' '.join([
     'kallithea', 'mercurial', 'git', 'code review',
     'repo groups', 'ldap', 'repository management', 'hgweb replacement',
     'hgwebdir', 'gitweb replacement', 'serving hgweb',
 ])
 
 # long description
 README_FILE = 'README.rst'
 try:
     long_description = open(README_FILE).read()
 except IOError as err:
     sys.stderr.write(
         "[WARNING] Cannot find file specified as long_description (%s)\n"
         % README_FILE
     )
     long_description = description
 
 import setuptools
 
 # monkey patch setuptools to use distutils owner/group functionality
 from setuptools.command import sdist
 sdist_org = sdist.sdist
 class sdist_new(sdist_org):
     def initialize_options(self):
         sdist_org.initialize_options(self)
         self.owner = self.group = 'root'
 sdist.sdist = sdist_new
 
 packages = setuptools.find_packages(exclude=['ez_setup'])
 
 setuptools.setup(
     name='Kallithea',
     version=__version__,
     description=description,
     long_description=long_description,
     keywords=keywords,
     license=__license__,
     author=__author__,
     author_email='[e-mail]',
     dependency_links=dependency_links,
     url=__url__,
     install_requires=requirements,
     classifiers=classifiers,
-    setup_requires=['PasteScript>=1.6.3'],
     data_files=data_files,
     packages=packages,
     include_package_data=True,
     message_extractors={'kallithea': [
             ('**.py', 'python', None),
             ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
             ('templates/**.html', 'mako', {'input_encoding': 'utf-8'}),
             ('public/**', 'ignore', None)]},
     zip_safe=False,
-    paster_plugins=['PasteScript', 'Pylons'],
     entry_points="""
     [console_scripts]
     kallithea-api =    kallithea.bin.kallithea_api:main
     kallithea-gist =   kallithea.bin.kallithea_gist:main
     kallithea-config = kallithea.bin.kallithea_config:main
 
     [paste.app_factory]
     main = kallithea.config.middleware:make_app
 
     [paste.app_install]
     main = pylons.util:PylonsInstaller
 
-    [paste.global_paster_command]
+    [gearbox.commands]
     setup-db=kallithea.lib.paster_commands.setup_db:Command
     cleanup-repos=kallithea.lib.paster_commands.cleanup:Command
     update-repoinfo=kallithea.lib.paster_commands.update_repoinfo:Command
     make-rcext=kallithea.lib.paster_commands.make_rcextensions:Command
     repo-scan=kallithea.lib.paster_commands.repo_scan:Command
     cache-keys=kallithea.lib.paster_commands.cache_keys:Command
     ishell=kallithea.lib.paster_commands.ishell:Command
     make-index=kallithea.lib.paster_commands.make_index:Command
     upgrade-db=kallithea.lib.dbmigrate:UpgradeDb
     celeryd=kallithea.lib.paster_commands.celeryd:Command
     install-iis=kallithea.lib.paster_commands.install_iis:Command
     """,
 )