Fixed archivals by passing baseui to scm get method for hg.

Docs update

docs/setup.rst

@@ -1,230 +1,230 b''
 .. _setup:
 
 Setup
 =====
 
 
 Setting up the application
 --------------------------
 
 ::
  
  paster make-config RhodeCode production.ini
 
 - This will create `production.ini` config inside the directory
-  this config contains various settings for rhodecode, e.g proxy port, 
+  this config contains various settings for RhodeCode, e.g proxy port, 
   email settings,static files, cache and logging.
 
 ::
 
  paster setup-app production.ini
 
 - This command will create all needed tables and an admin account. 
   When asked for a path You can either use a new location of one with already 
   existing ones. RhodeCode will simply add all new found repositories to 
   it's database. Also make sure You specify correct path to repositories.
 - Remember that the given path for mercurial_ repositories must be write 
   accessible for the application. It's very important since RhodeCode web interface
   will work even without such an access but, when trying to do a push it'll 
   eventually fail with permission denied errors. 
 - Run 
 
 ::
  
  paster serve production.ini
  
-- This command runs the rhodecode server the app should be available at the 
+- This command runs the RhodeCode server the app should be available at the 
   127.0.0.1:5000. This ip and port is configurable via the production.ini 
   file  created in previous step
 - Use admin account you created to login.
 - Default permissions on each repository is read, and owner is admin. So 
   remember to update these if needed.
   
     
 Setting up Whoosh full text search
 ----------------------------------
 
 Index for whoosh can be build starting from version 1.1 using paster command
 passing repo locations to index, as well as Your config file that stores
 whoosh index files locations. There is possible to pass `-f` to the options
 to enable full index rebuild. Without that indexing will run always in in
 incremental mode.
 
 ::
 
  paster make-index --repo-location=<location for repos> production.ini  
 
 for full index rebuild You can use
 
 ::
 
  paster make-index -f --repo-location=<location for repos> production.ini
 
 - For full text search You can either put crontab entry for
 
 This command can be run even from crontab in order to do periodical 
 index builds and keep Your index always up to date. An example entry might 
 look like this
 
 ::
  
  /path/to/python/bin/paster --repo-location=<location for repos> /path/to/rhodecode/production.ini
   
 When using incremental(default) mode whoosh will check last modification date 
 of each file and add it to reindex if newer file is available. Also indexing 
 daemon checks for removed files and removes them from index. 
 
 Sometime You might want to rebuild index from scratch. You can do that using 
 the `-f` flag passed to paster command or, in admin panel You can check 
 `build from scratch` flag.
 
 
 Setting up LDAP support
 -----------------------
 
 
 RhodeCode starting from version 1.1 supports ldap authentication. In order
 to use ldap, You have to install ldap-python package. This package is available
 via pypi, so You can install it by running
 
 ::
 
  easy_install ldap-python
  
 ::
 
  pip install ldap-python
 
 
 ldap-python requires some certain libs on Your system, so before installing it 
 check that You have at least `openldap`, and `sasl` libraries.
 
 ldap settings are located in admin->permissions section,
 
 Here's a typical ldap setup::
 
  Enable ldap  = checked                 #controlls if ldap access is enabled
  Host         = host.domain.org         #acctuall ldap server to connect
  Port         = 389 or 689 for ldaps    #ldap server ports
  Enable LDAPS = unchecked               #enable disable ldaps
  Account      = <account>               #access for ldap server(if required)
  Password     = <password>              #password for ldap server(if required)
  Base DN      = CN=users,DC=host,DC=domain,DC=org
  
 
 `Account` and `Password` are optional, and used for two-phase ldap 
 authentication so those are credentials to access Your ldap, if it doesn't 
 support anonymous search/user lookups.
 
 If all data are entered correctly, and `ldap-python` is properly installed
-Users should be granted to access rhodecode wit theire ldap accounts. When 
-logging at the first time an special ldap account is created inside rhodecode, 
+Users should be granted to access RhodeCode wit theire ldap accounts. When 
+logging at the first time an special ldap account is created inside RhodeCode, 
 so You can control over permissions even on ldap users. If such user exists 
-already in rhodecode database ldap user with the same username would be not 
-able to access rhodecode.
+already in RhodeCode database ldap user with the same username would be not 
+able to access RhodeCode.
 
 If You have problems with ldap access and believe You entered correct 
-information check out the rhodecode logs,any error messages sent from 
+information check out the RhodeCode logs,any error messages sent from 
 ldap will be saved there.
 
 
 Nginx virtual host example
 --------------------------
 
 Sample config for nginx using proxy::
 
  server {
     listen          80;
     server_name     hg.myserver.com;
     access_log      /var/log/nginx/rhodecode.access.log;
     error_log       /var/log/nginx/rhodecode.error.log;
     location / {
             root /var/www/rhodecode/rhodecode/public/;
             if (!-f $request_filename){
                 proxy_pass      http://127.0.0.1:5000;
             }
             #this is important for https !!!
             proxy_set_header X-Url-Scheme $scheme;
             include         /etc/nginx/proxy.conf;  
     }
  }  
   
 Here's the proxy.conf. It's tuned so it'll not timeout on long
 pushes and also on large pushes::
 
     proxy_redirect              off;
     proxy_set_header            Host $host;
     proxy_set_header            X-Host $http_host;
     proxy_set_header            X-Real-IP $remote_addr;
     proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header            Proxy-host $proxy_host;
     client_max_body_size        400m;
     client_body_buffer_size     128k;
     proxy_buffering             off;
     proxy_connect_timeout       3600;
     proxy_send_timeout          3600;
     proxy_read_timeout          3600;
     proxy_buffer_size           8k;
     proxy_buffers               8 32k;
     proxy_busy_buffers_size     64k;
     proxy_temp_file_write_size  64k;
  
 Also when using root path with nginx You might set the static files to false
 in production.ini file::
 
   [app:main]
     use = egg:rhodecode
     full_stack = true
     static_files = false
     lang=en
     cache_dir = %(here)s/data
 
 To not have the statics served by the application. And improve speed.
 
 
 
 Apache's example FCGI config
 ----------------------------
 
 TODO !
 
 Other configuration files
 -------------------------
 
 Some extra configuration files and examples can be found here:
 http://hg.python-works.com/rhodecode/files/tip/init.d
 
 and also an celeryconfig file can be use from here:
 http://hg.python-works.com/rhodecode/files/tip/celeryconfig.py
 
 Troubleshooting
 ---------------
 
 - missing static files ?
 
  - make sure either to set the `static_files = true` in the .ini file or
    double check the root path for Your http setup. It should point to 
    for example:
    /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public
    
 - can't install celery/rabbitmq
 
  - don't worry RhodeCode works without them too. No extra setup required
 
 
 - long lasting push timeouts ?
 
  - make sure You set a longer timeouts in Your proxy/fcgi settings, timeouts
-   are caused by https server and not rhodecode
+   are caused by https server and not RhodeCode
 
 - large pushes timeouts ?
  
  - make sure You set a proper max_body_size for the http server
 
 
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
 .. _rabbitmq: http://www.rabbitmq.com/
  No newline at end of file

rhodecode/model/scm.py

@@ -1,242 +1,243 b''
 #!/usr/bin/env python
 # encoding: utf-8
 # Model for RhodeCode
 # Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
 # 
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 """
 Created on April 9, 2010
 Model for RhodeCode
 @author: marcink
 """
 from beaker.cache import cache_region, region_invalidate
 from mercurial import ui
 from rhodecode import BACKENDS
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import HasRepoPermissionAny
 from rhodecode.lib.utils import get_repos
 from rhodecode.model import meta
 from rhodecode.model.db import Repository, User, RhodeCodeUi, CacheInvalidation
 from rhodecode.model.caching_query import FromCache
 from sqlalchemy.orm import joinedload
 from sqlalchemy.orm.session import make_transient
 from vcs import get_backend
 from vcs.utils.helpers import get_scm
 from vcs.exceptions import RepositoryError, VCSError
 from vcs.utils.lazy import LazyProperty
 import traceback
 import logging
 import os
 import time
 
 log = logging.getLogger(__name__)
 
 class ScmModel(object):
     """
     Mercurial Model
     """
 
     def __init__(self):
         self.sa = meta.Session()
 
     @LazyProperty
     def repos_path(self):
         """
         Get's the repositories root path from database
         """
         q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
 
         return q.ui_value
 
     def repo_scan(self, repos_path, baseui, initial=False):
         """
         Listing of repositories in given path. This path should not be a 
         repository itself. Return a dictionary of repository objects
         
         :param repos_path: path to directory containing repositories
         :param baseui
         :param initial: initial scan
         """
         log.info('scanning for repositories in %s', repos_path)
 
         if not isinstance(baseui, ui.ui):
             baseui = ui.ui()
         repos_list = {}
 
         for name, path in get_repos(repos_path):
             try:
                 if repos_list.has_key(name):
                     raise RepositoryError('Duplicate repository name %s '
                                     'found in %s' % (name, path))
                 else:
 
                     klass = get_backend(path[0])
 
                     if path[0] == 'hg' and path[0] in BACKENDS.keys():
                         repos_list[name] = klass(path[1], baseui=baseui)
 
                     if path[0] == 'git' and path[0] in BACKENDS.keys():
                         repos_list[name] = klass(path[1])
             except OSError:
                 continue
 
         return repos_list
 
     def get_repos(self, all_repos=None):
         """
         Get all repos from db and for each repo create it's backend instance.
         and fill that backed with information from database
         
         :param all_repos: give specific repositories list, good for filtering
         """
         if not all_repos:
             all_repos = self.sa.query(Repository)\
                 .order_by(Repository.repo_name).all()
 
         for r in all_repos:
 
             repo = self.get(r.repo_name)
 
             if repo is not None:
                 last_change = repo.last_change
                 tip = h.get_changeset_safe(repo, 'tip')
 
                 tmp_d = {}
                 tmp_d['name'] = repo.name
                 tmp_d['name_sort'] = tmp_d['name'].lower()
                 tmp_d['description'] = repo.dbrepo.description
                 tmp_d['description_sort'] = tmp_d['description']
                 tmp_d['last_change'] = last_change
                 tmp_d['last_change_sort'] = time.mktime(last_change.timetuple())
                 tmp_d['tip'] = tip.raw_id
                 tmp_d['tip_sort'] = tip.revision
                 tmp_d['rev'] = tip.revision
                 tmp_d['contact'] = repo.dbrepo.user.full_contact
                 tmp_d['contact_sort'] = tmp_d['contact']
                 tmp_d['repo_archives'] = list(repo._get_archives())
                 tmp_d['last_msg'] = tip.message
                 tmp_d['repo'] = repo
                 yield tmp_d
 
     def get_repo(self, repo_name):
         return self.get(repo_name)
 
     def get(self, repo_name):
         """
         Get's repository from given name, creates BackendInstance and
         propagates it's data from database with all additional information
         :param repo_name:
         """
         if not HasRepoPermissionAny('repository.read', 'repository.write',
                             'repository.admin')(repo_name, 'get repo check'):
             return
 
         @cache_region('long_term')
         def _get_repo(repo_name):
 
             repo_path = os.path.join(self.repos_path, repo_name)
             alias = get_scm(repo_path)[0]
 
             log.debug('Creating instance of %s repository', alias)
             backend = get_backend(alias)
 
             #TODO: get the baseui from somewhere for this
             if alias == 'hg':
-                repo = backend(repo_path, create=False, baseui=None)
+                from pylons import app_globals as g
+                repo = backend(repo_path, create=False, baseui=g.baseui)
                 #skip hidden web repository
                 if repo._get_hidden():
                     return
             else:
                 repo = backend(repo_path, create=False)
 
             dbrepo = self.sa.query(Repository)\
                 .options(joinedload(Repository.fork))\
                 .options(joinedload(Repository.user))\
                 .filter(Repository.repo_name == repo_name)\
                 .scalar()
             make_transient(dbrepo)
             repo.dbrepo = dbrepo
             return repo
 
         invalidate = self._should_invalidate(repo_name)
         if invalidate:
             log.info('invalidating cache for repository %s', repo_name)
             region_invalidate(_get_repo, None, repo_name)
             self._mark_invalidated(invalidate)
 
         return _get_repo(repo_name)
 
 
 
     def mark_for_invalidation(self, repo_name):
         """
         Puts cache invalidation task into db for 
         further global cache invalidation
         
         :param repo_name: this repo that should invalidation take place
         """
         log.debug('marking %s for invalidation', repo_name)
         cache = self.sa.query(CacheInvalidation)\
             .filter(CacheInvalidation.cache_key == repo_name).scalar()
 
         if cache:
             #mark this cache as inactive
             cache.cache_active = False
         else:
             log.debug('cache key not found in invalidation db -> creating one')
             cache = CacheInvalidation(repo_name)
 
         try:
             self.sa.add(cache)
             self.sa.commit()
         except:
             log.error(traceback.format_exc())
             self.sa.rollback()
 
 
 
 
 
     def _should_invalidate(self, repo_name):
         """
         Looks up database for invalidation signals for this repo_name
         :param repo_name:
         """
 
         ret = self.sa.query(CacheInvalidation)\
             .options(FromCache('sql_cache_short',
                            'get_invalidation_%s' % repo_name))\
             .filter(CacheInvalidation.cache_key == repo_name)\
             .filter(CacheInvalidation.cache_active == False)\
             .scalar()
 
         return ret
 
     def _mark_invalidated(self, cache_key):
         """
         Marks all occurences of cache to invaldation as already invalidated
         @param repo_name:
         """
         if cache_key:
             log.debug('marking %s as already invalidated', cache_key)
         try:
             cache_key.cache_active = True
             self.sa.add(cache_key)
             self.sa.commit()
         except:
             log.error(traceback.format_exc())
             self.sa.rollback()