updated setup for all newest versions

EmptyChangeset can take changeset as creation param.

README.rst

@@ -1,104 +1,103 b''
 
 RhodeCode (RhodiumCode)
 =======================
 
 ``RhodeCode`` (formerly hg-app) is Pylons based repository management and 
-serving for mercurial_. It's similar to github or bitbucket, but it's suppose to run
-as standalone app, it's open source and focuses more on restricted access to repositories
-There's no default free access to RhodeCode You have to create an account in order
-to use the application. It's powered by vcs_ library that we created to handle
-many various version control systems.
+serving for mercurial_ and git_. It's similar to github or bitbucket, but 
+it's suppose to run as standalone app, it's open source and focuses more on 
+restricted access to repositories. There's no default free access to RhodeCode 
+You have to create an account in order to use the application. It's powered 
+by vcs_ library that we created to handle many various version control systems.
 
 RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 
-
 RhodeCode demo
 --------------
 
 http://hg.python-works.com
 
 The default access is
 
 - username: demo
 - password: demo
 
 Source code
 -----------
 
 Source code is along with issue tracker is available at
 http://bitbucket.org/marcinkuzminski/rhodecode
 
 Also a source codes can be obtained from demo rhodecode instance
 http://hg.python-works.com/rhodecode/summary
 
 Instalation
 -----------
 
  Please visit http://packages.python.org/RhodeCode/installation.html
 
 
 Features
 --------
 
 - Has it's own middleware to handle mercurial_ protocol request. Each request 
   can be logged and authenticated. Runs on threads unlikely to hgweb You can 
   make multiple pulls/pushes simultaneous. Supports http/https
 - Full permissions and authentication per project private/read/write/admin. 
   One account for web interface and mercurial_ push/pull/clone.
 - Mako templates let's you customize look and feel of application.
 - Beautiful diffs, annotations and source codes all colored by pygments.
 - Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 - Admin interface with user/permission management. User activity journal logs
   pulls, pushes, forks,registrations. Possible to disable built in hooks
 - Server side forks, it's possible to fork a project and hack it free without
   breaking the main.   
 - Full text search on source codes, search on file names. All powered by whoosh
   and build in indexing daemons
   (no external search servers required all in one application)
 - Rss / atom feeds, gravatar support, download sources as zip/tarballs  
 - Async tasks for speed and performance using celery_ (works without them too)  
 - Backup scripts can do backup of whole app and send it over scp to desired 
   location
 - Setup project descriptions and info inside built in db for easy, non 
   file-system operations
 - Added cache with invalidation on push/repo management for high performance and
   always up to date data. 
 - Based on pylons 1.0 / sqlalchemy 0.6 / sqlite
 
 
 Incoming
 --------
 
 - code review (probably based on hg-review)
 - full git_ support, with push/pull server
 - commit based build in wiki system
 - clone points and cloning from remote repositories into rhodecode 
   (git_ and mercurial_)
 - some cache optimizations
 - other cools stuff that i can figure out (or You can help me figure out)
 
 License
 -------
 
 ``rhodecode`` is released under GPL_ license.
 
 
 Documentation
 -------------
 
  Online documentation for current version is available at
  http://packages.python.org/RhodeCode/.
  You may also build documentation for yourself - go into ``docs/`` and run::
 
    make html
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _django: http://www.djangoproject.com/
 .. _mercurial: http://mercurial.selenic.com/
 .. _subversion: http://subversion.tigris.org/
 .. _git: http://git-scm.com/
 .. _celery: http://celeryproject.org/
 .. _Sphinx: http://sphinx.pocoo.org/
 .. _GPL: http://www.gnu.org/licenses/gpl.html
 .. _vcs: http://pypi.python.org/pypi/vcs
  No newline at end of file

rhodecode/controllers/summary.py

@@ -1,113 +1,120 b''
 #!/usr/bin/env python
 # encoding: utf-8
 # summary controller for pylons
 # Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
 # 
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 """
 Created on April 18, 2010
 summary controller for pylons
 @author: marcink
 """
 from pylons import tmpl_context as c, request, url
+from vcs.exceptions import ChangesetError
 from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 from rhodecode.lib.base import BaseController, render
-from rhodecode.lib.utils import OrderedDict
+from rhodecode.lib.utils import OrderedDict, EmptyChangeset
 from rhodecode.model.hg import HgModel
 from rhodecode.model.db import Statistics
 from webhelpers.paginate import Page
 from rhodecode.lib.celerylib import run_task
 from rhodecode.lib.celerylib.tasks import get_commits_stats
 from datetime import datetime, timedelta
 from time import mktime
 import calendar
 import logging
 try:
     import json
 except ImportError:
     #python 2.5 compatibility
     import simplejson as json
 log = logging.getLogger(__name__)
 
 class SummaryController(BaseController):
 
     @LoginRequired()
     @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
                                    'repository.admin')
     def __before__(self):
         super(SummaryController, self).__before__()
 
     def index(self):
         hg_model = HgModel()
         c.repo_info = hg_model.get_repo(c.repo_name)
         def url_generator(**kw):
             return url('shortlog_home', repo_name=c.repo_name, **kw)
 
         c.repo_changesets = Page(c.repo_info, page=1, items_per_page=10,
                                  url=url_generator)
 
         e = request.environ
 
         uri = u'%(protocol)s://%(user)s@%(host)s%(prefix)s/%(repo_name)s' % {
                                         'protocol': e.get('wsgi.url_scheme'),
                                         'user':str(c.rhodecode_user.username),
                                         'host':e.get('HTTP_HOST'),
                                         'prefix':e.get('SCRIPT_NAME'),
                                         'repo_name':c.repo_name, }
         c.clone_repo_url = uri
         c.repo_tags = OrderedDict()
         for name, hash in c.repo_info.tags.items()[:10]:
-            c.repo_tags[name] = c.repo_info.get_changeset(hash)
+            try:
+                c.repo_tags[name] = c.repo_info.get_changeset(hash)
+            except ChangesetError:
+                c.repo_tags[name] = EmptyChangeset(hash)
 
         c.repo_branches = OrderedDict()
         for name, hash in c.repo_info.branches.items()[:10]:
-            c.repo_branches[name] = c.repo_info.get_changeset(hash)
+            try:
+                c.repo_branches[name] = c.repo_info.get_changeset(hash)
+            except ChangesetError:
+                c.repo_branches[name] = EmptyChangeset(hash)
 
         td = datetime.today() + timedelta(days=1)
         y, m, d = td.year, td.month, td.day
 
         ts_min_y = mktime((y - 1, (td - timedelta(days=calendar.mdays[m])).month,
                             d, 0, 0, 0, 0, 0, 0,))
         ts_min_m = mktime((y, (td - timedelta(days=calendar.mdays[m])).month,
                             d, 0, 0, 0, 0, 0, 0,))
 
         ts_max_y = mktime((y, m, d, 0, 0, 0, 0, 0, 0,))
 
         run_task(get_commits_stats, c.repo_info.name, ts_min_y, ts_max_y)
         c.ts_min = ts_min_m
         c.ts_max = ts_max_y
 
         stats = self.sa.query(Statistics)\
             .filter(Statistics.repository == c.repo_info.dbrepo)\
             .scalar()
 
 
         if stats and stats.languages:
             lang_stats = json.loads(stats.languages)
             c.commit_data = stats.commit_activity
             c.overview_data = stats.commit_activity_combined
             c.trending_languages = json.dumps(OrderedDict(
                                        sorted(lang_stats.items(), reverse=True,
                                             key=lambda k: k[1])[:2]
                                         )
                                     )
         else:
             c.commit_data = json.dumps({})
             c.overview_data = json.dumps([[ts_min_y, 0], [ts_max_y, 0] ])
             c.trending_languages = json.dumps({})
 
         return render('summary/summary.html')
 

rhodecode/lib/helpers.py

@@ -1,398 +1,401 b''
 """Helper functions
 
 Consists of functions to typically be used within templates, but also
 available to Controllers. This module is available to both as 'h'.
 """
 from pygments.formatters import HtmlFormatter
 from pygments import highlight as code_highlight
 from pylons import url, app_globals as g
 from pylons.i18n.translation import _, ungettext
 from vcs.utils.annotate import annotate_highlight
 from webhelpers.html import literal, HTML, escape
 from webhelpers.html.tools import *
 from webhelpers.html.builder import make_tag
 from webhelpers.html.tags import auto_discovery_link, checkbox, css_classes, \
     end_form, file, form, hidden, image, javascript_link, link_to, link_to_if, \
     link_to_unless, ol, required_legend, select, stylesheet_link, submit, text, \
     password, textarea, title, ul, xml_declaration, radio
 from webhelpers.html.tools import auto_link, button_to, highlight, js_obfuscate, \
     mail_to, strip_links, strip_tags, tag_re
 from webhelpers.number import format_byte_size, format_bit_size
 from webhelpers.pylonslib import Flash as _Flash
 from webhelpers.pylonslib.secure_form import secure_form
 from webhelpers.text import chop_at, collapse, convert_accented_entities, \
     convert_misc_entities, lchop, plural, rchop, remove_formatting, \
     replace_whitespace, urlify, truncate, wrap_paragraphs
 from webhelpers.date import time_ago_in_words
 
 #Custom helpers here :)
 class _Link(object):
     '''
     Make a url based on label and url with help of url_for
     :param label:name of link    if not defined url is used
     :param url: the url for link
     '''
 
     def __call__(self, label='', *url_, **urlargs):
         if label is None or '':
             label = url
         link_fn = link_to(label, url(*url_, **urlargs))
         return link_fn
 
 link = _Link()
 
 class _GetError(object):
 
     def __call__(self, field_name, form_errors):
         tmpl = """<span class="error_msg">%s</span>"""
         if form_errors and form_errors.has_key(field_name):
             return literal(tmpl % form_errors.get(field_name))
 
 get_error = _GetError()
 
 def recursive_replace(str, replace=' '):
     """
     Recursive replace of given sign to just one instance
     :param str: given string
     :param replace:char to find and replace multiple instances
         
     Examples::
     >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
     'Mighty-Mighty-Bo-sstones'
     """
 
     if str.find(replace * 2) == -1:
         return str
     else:
         str = str.replace(replace * 2, replace)
         return recursive_replace(str, replace)
 
 class _ToolTip(object):
 
     def __call__(self, tooltip_title, trim_at=50):
         """
         Special function just to wrap our text into nice formatted autowrapped
         text
         :param tooltip_title:
         """
 
         return wrap_paragraphs(escape(tooltip_title), trim_at)\
                        .replace('\n', '<br/>')
 
     def activate(self):
         """
         Adds tooltip mechanism to the given Html all tooltips have to have 
         set class tooltip and set attribute tooltip_title.
         Then a tooltip will be generated based on that
         All with yui js tooltip
         """
 
         js = '''
         YAHOO.util.Event.onDOMReady(function(){
             function toolTipsId(){
                 var ids = [];
                 var tts = YAHOO.util.Dom.getElementsByClassName('tooltip');
                 
                 for (var i = 0; i < tts.length; i++) {
                     //if element doesn not have and id autgenerate one for tooltip
                     
                     if (!tts[i].id){
                         tts[i].id='tt'+i*100;
                     }
                     ids.push(tts[i].id);
                 }
                 return ids        
             };
             var myToolTips = new YAHOO.widget.Tooltip("tooltip", { 
                 context: toolTipsId(),
                 monitorresize:false,
                 xyoffset :[0,0],
                 autodismissdelay:300000,
                 hidedelay:5,
                 showdelay:20,
             });
             
             //Mouse Over event disabled for new repositories since they dont
             //have last commit message
             myToolTips.contextMouseOverEvent.subscribe(
                 function(type, args) {
                     var context = args[0];
                     var txt = context.getAttribute('tooltip_title');
                     if(txt){                                       
                         return true;
                     }
                     else{
                         return false;
                     }
                 });
             
                             
             // Set the text for the tooltip just before we display it. Lazy method
             myToolTips.contextTriggerEvent.subscribe( 
                  function(type, args) { 
 
                  
                         var context = args[0]; 
                         
                         var txt = context.getAttribute('tooltip_title');
                         this.cfg.setProperty("text", txt);
                         
                         
                         // positioning of tooltip
                         var tt_w = this.element.clientWidth;
                         var tt_h = this.element.clientHeight;
                         
                         var context_w = context.offsetWidth;
                         var context_h = context.offsetHeight;
                         
                         var pos_x = YAHOO.util.Dom.getX(context);
                         var pos_y = YAHOO.util.Dom.getY(context);
 
                         var display_strategy = 'top';
                         var xy_pos = [0,0];
                         switch (display_strategy){
                         
                             case 'top':
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = pos_y-tt_h-4;
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'bottom':
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = pos_y+context_h+4;
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'left':
                                 var cur_x = (pos_x-tt_w-4);
                                 var cur_y = pos_y-((tt_h/2)-context_h/2);
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'right':
                                 var cur_x = (pos_x+context_w+4);
                                 var cur_y = pos_y-((tt_h/2)-context_h/2);
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                              default:
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = pos_y-tt_h-4;
                                 xy_pos = [cur_x,cur_y];                                
                                 break;                             
                                  
                         }
 
                         this.cfg.setProperty("xy",xy_pos);
 
                   });
                   
             //Mouse out 
             myToolTips.contextMouseOutEvent.subscribe(
                 function(type, args) {
                     var context = args[0];
                     
                 });
         });
         '''
         return literal(js)
 
 tooltip = _ToolTip()
 
 class _FilesBreadCrumbs(object):
 
     def __call__(self, repo_name, rev, paths):
         url_l = [link_to(repo_name, url('files_home',
                                         repo_name=repo_name,
                                         revision=rev, f_path=''))]
         paths_l = paths.split('/')
 
         for cnt, p in enumerate(paths_l, 1):
             if p != '':
                 url_l.append(link_to(p, url('files_home',
                                             repo_name=repo_name,
                                             revision=rev,
                                             f_path='/'.join(paths_l[:cnt]))))
 
         return literal('/'.join(url_l))
 
 files_breadcrumbs = _FilesBreadCrumbs()
 class CodeHtmlFormatter(HtmlFormatter):
 
     def wrap(self, source, outfile):
         return self._wrap_div(self._wrap_pre(self._wrap_code(source)))
 
     def _wrap_code(self, source):
         for cnt, it in enumerate(source, 1):
             i, t = it
             t = '<div id="#S-%s">%s</div>' % (cnt, t)
             yield i, t
 def pygmentize(filenode, **kwargs):
     """
     pygmentize function using pygments
     :param filenode:
     """
     return literal(code_highlight(filenode.content,
                                   filenode.lexer, CodeHtmlFormatter(**kwargs)))
 
 def pygmentize_annotation(filenode, **kwargs):
     """
     pygmentize function for annotation
     :param filenode:
     """
 
     color_dict = {}
     def gen_color():
         """generator for getting 10k of evenly distibuted colors using hsv color
         and golden ratio.
         """
         import colorsys
         n = 10000
         golden_ratio = 0.618033988749895
         h = 0.22717784590367374
         #generate 10k nice web friendly colors in the same order
         for c in xrange(n):
             h += golden_ratio
             h %= 1
             HSV_tuple = [h, 0.95, 0.95]
             RGB_tuple = colorsys.hsv_to_rgb(*HSV_tuple)
             yield map(lambda x:str(int(x * 256)), RGB_tuple)
 
     cgenerator = gen_color()
 
     def get_color_string(cs):
         if color_dict.has_key(cs):
             col = color_dict[cs]
         else:
             col = color_dict[cs] = cgenerator.next()
         return "color: rgb(%s)! important;" % (', '.join(col))
 
     def url_func(changeset):
         tooltip_html = "<div style='font-size:0.8em'><b>Author:</b>" + \
         " %s<br/><b>Date:</b> %s</b><br/><b>Message:</b> %s<br/></div>"
 
         tooltip_html = tooltip_html % (changeset.author,
                                                changeset.date,
                                                tooltip(changeset.message))
         lnk_format = 'r%-5s:%s' % (changeset.revision,
                                  changeset.raw_id)
         uri = link_to(
                 lnk_format,
                 url('changeset_home', repo_name=changeset.repository.name,
                     revision=changeset.raw_id),
                 style=get_color_string(changeset.raw_id),
                 class_='tooltip',
                 tooltip_title=tooltip_html
               )
 
         uri += '\n'
         return uri
     return literal(annotate_highlight(filenode, url_func, **kwargs))
 
 def repo_name_slug(value):
     """Return slug of name of repository
     This function is called on each creation/modification
     of repository to prevent bad names in repo
     """
     slug = remove_formatting(value)
     slug = strip_tags(slug)
 
     for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:
         slug = slug.replace(c, '-')
     slug = recursive_replace(slug, '-')
     slug = collapse(slug, '-')
     return slug
 
 def get_changeset_safe(repo, rev):
     from vcs.backends.base import BaseRepository
     from vcs.exceptions import RepositoryError
     if not isinstance(repo, BaseRepository):
         raise Exception('You must pass an Repository '
                         'object as first argument got %s', type(repo))
 
     try:
         cs = repo.get_changeset(rev)
     except RepositoryError:
         from rhodecode.lib.utils import EmptyChangeset
         cs = EmptyChangeset()
     return cs
 
 
 flash = _Flash()
 
 
 #==============================================================================
 # MERCURIAL FILTERS available via h.
 #==============================================================================
 from mercurial import util
 from mercurial.templatefilters import person as _person
 
 
 
 def _age(curdate):
     """turns a datetime into an age string."""
+    if not curdate:
+        return ''
 
     from datetime import timedelta, datetime
+
     agescales = [("year", 3600 * 24 * 365),
              ("month", 3600 * 24 * 30),
              #("week", 3600 * 24 * 7),
              ("day", 3600 * 24),
              ("hour", 3600),
              ("minute", 60),
              ("second", 1)]
 
     age = datetime.now() - curdate
     age_seconds = (age.days * agescales[2][1]) + age.seconds
 
     pos = 1
     for scale in agescales:
         if scale[1] <= age_seconds:
             return time_ago_in_words(curdate, agescales[pos][0])
         pos += 1
 
 age = lambda  x:_age(x)
 capitalize = lambda x: x.capitalize()
 email = util.email
 email_or_none = lambda x: util.email(x) if util.email(x) != x else None
 person = lambda x: _person(x)
 short_id = lambda x: x[:12]
 
 #==============================================================================
 # PERMS
 #==============================================================================
 from rhodecode.lib.auth import HasPermissionAny, HasPermissionAll, \
 HasRepoPermissionAny, HasRepoPermissionAll
 
 #==============================================================================
 # GRAVATAR URL
 #==============================================================================
 import hashlib
 import urllib
 from pylons import request
 
 def gravatar_url(email_address, size=30):
     ssl_enabled = 'https' == request.environ.get('HTTP_X_URL_SCHEME')
     default = 'identicon'
     baseurl_nossl = "http://www.gravatar.com/avatar/"
     baseurl_ssl = "https://secure.gravatar.com/avatar/"
     baseurl = baseurl_ssl if ssl_enabled else baseurl_nossl
 
 
     # construct the url
     gravatar_url = baseurl + hashlib.md5(email_address.lower()).hexdigest() + "?"
     gravatar_url += urllib.urlencode({'d':default, 's':str(size)})
 
     return gravatar_url
 
 def safe_unicode(str):
     """safe unicode function. In case of UnicodeDecode error we try to return
     unicode with errors replace, if this failes we return unicode with 
     string_escape decoding """
 
     try:
         u_str = unicode(str)
     except UnicodeDecodeError:
         try:
             u_str = unicode(str, 'utf-8', 'replace')
         except UnicodeDecodeError:
             #incase we have a decode error just represent as byte string
             u_str = unicode(str(str).encode('string_escape'))
 
     return u_str

rhodecode/lib/middleware/simplegit.py

@@ -1,204 +1,204 b''
 #!/usr/bin/env python
 # encoding: utf-8
 # middleware to handle git api calls
 # Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 """
 Created on 2010-04-28
 
 @author: marcink
 SimpleGit middleware for handling git protocol request (push/clone etc.)
 It's implemented with basic auth function
 """
 
 from dulwich import server as dulserver
 
 class SimpleGitUploadPackHandler(dulserver.UploadPackHandler):
 
     def handle(self):
         write = lambda x: self.proto.write_sideband(1, x)
 
         graph_walker = dulserver.ProtocolGraphWalker(self, self.repo.object_store,
             self.repo.get_peeled)
         objects_iter = self.repo.fetch_objects(
           graph_walker.determine_wants, graph_walker, self.progress,
           get_tagged=self.get_tagged)
 
         # Do they want any objects?
         if len(objects_iter) == 0:
             return
 
         self.progress("counting objects: %d, done.\n" % len(objects_iter))
         dulserver.write_pack_data(dulserver.ProtocolFile(None, write), objects_iter,
                         len(objects_iter))
         messages = []
         messages.append('thank you for using rhodecode')
 
         for msg in messages:
             self.progress(msg + "\n")
         # we are done
         self.proto.write("0000")
 
 dulserver.DEFAULT_HANDLERS = {
   'git-upload-pack': SimpleGitUploadPackHandler,
   'git-receive-pack': dulserver.ReceivePackHandler,
 }
 
 from dulwich.repo import Repo
 from dulwich.web import HTTPGitApplication
 from paste.auth.basic import AuthBasicAuthenticator
 from paste.httpheaders import REMOTE_USER, AUTH_TYPE
 from rhodecode.lib.auth import authfunc, HasPermissionAnyMiddleware
 from rhodecode.lib.utils import action_logger, is_git, invalidate_cache, \
     check_repo_fast
 from rhodecode.model.user import UserModel
 from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError
 import logging
 import os
 import traceback
 
 
 log = logging.getLogger(__name__)
 
 class SimpleGit(object):
 
     def __init__(self, application, config):
         self.application = application
         self.config = config
         #authenticate this git request using 
         self.authenticate = AuthBasicAuthenticator('', authfunc)
 
     def __call__(self, environ, start_response):
         if not is_git(environ):
             return self.application(environ, start_response)
 
         #===================================================================
         # AUTHENTICATE THIS GIT REQUEST
         #===================================================================
         username = REMOTE_USER(environ)
         if not username:
             self.authenticate.realm = self.config['rhodecode_realm']
             result = self.authenticate(environ)
             if isinstance(result, str):
                 AUTH_TYPE.update(environ, 'basic')
                 REMOTE_USER.update(environ, result)
             else:
                 return result.wsgi_application(environ, start_response)
 
         try:
             self.repo_name = environ['PATH_INFO'].split('/')[1]
             if self.repo_name.endswith('/'):
                 self.repo_name = self.repo_name.rstrip('/')
         except:
             log.error(traceback.format_exc())
             return HTTPInternalServerError()(environ, start_response)
 
         #===================================================================
         # CHECK PERMISSIONS FOR THIS REQUEST
         #===================================================================
         action = self.__get_action(environ)
         if action:
             username = self.__get_environ_user(environ)
             try:
                 user = self.__get_user(username)
             except:
                 log.error(traceback.format_exc())
                 return HTTPInternalServerError()(environ, start_response)
 
             #check permissions for this repository
             if action == 'push':
                 if not HasPermissionAnyMiddleware('repository.write',
                                                   'repository.admin')\
                                                     (user, self.repo_name):
                     return HTTPForbidden()(environ, start_response)
 
             else:
                 #any other action need at least read permission
                 if not HasPermissionAnyMiddleware('repository.read',
                                                   'repository.write',
                                                   'repository.admin')\
                                                     (user, self.repo_name):
                     return HTTPForbidden()(environ, start_response)
 
             #log action
             if action in ('push', 'pull', 'clone'):
                 proxy_key = 'HTTP_X_REAL_IP'
                 def_key = 'REMOTE_ADDR'
                 ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))
                 self.__log_user_action(user, action, self.repo_name, ipaddr)
 
         #===================================================================
         # GIT REQUEST HANDLING
         #===================================================================
         self.basepath = self.config['base_path']
         self.repo_path = os.path.join(self.basepath, self.repo_name)
         #quick check if that dir exists...
         if check_repo_fast(self.repo_name, self.basepath):
             return HTTPNotFound()(environ, start_response)
         try:
             app = self.__make_app()
-        except Exception:
+        except:
             log.error(traceback.format_exc())
             return HTTPInternalServerError()(environ, start_response)
 
         #invalidate cache on push
         if action == 'push':
             self.__invalidate_cache(self.repo_name)
             messages = []
             messages.append('thank you for using rhodecode')
             return app(environ, start_response)
         else:
             return app(environ, start_response)
 
 
     def __make_app(self):
         backend = dulserver.DictBackend({'/' + self.repo_name: Repo(self.repo_path)})
         gitserve = HTTPGitApplication(backend)
 
         return gitserve
 
     def __get_environ_user(self, environ):
         return environ.get('REMOTE_USER')
 
     def __get_user(self, username):
         return UserModel().get_by_username(username, cache=True)
 
     def __get_action(self, environ):
         """
         Maps git request commands into a pull or push command.
         :param environ:
         """
         service = environ['QUERY_STRING'].split('=')
         if len(service) > 1:
             service_cmd = service[1]
             mapping = {'git-receive-pack': 'push',
                        'git-upload-pack': 'pull',
                        }
 
             return mapping.get(service_cmd, service_cmd if service_cmd else 'other')
         else:
             return 'other'
 
     def __log_user_action(self, user, action, repo, ipaddr):
         action_logger(user, action, repo, ipaddr)
 
     def __invalidate_cache(self, repo_name):
         """we know that some change was made to repositories and we should
         invalidate the cache to see the changes right away but only for
         push requests"""
         invalidate_cache('cached_repo_list')
         invalidate_cache('full_changelog', repo_name)

rhodecode/lib/utils.py

@@ -1,538 +1,541 b''
 #!/usr/bin/env python
 # encoding: utf-8
 # Utilities for RhodeCode
 # Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 """
 Created on April 18, 2010
 Utilities for RhodeCode
 @author: marcink
 """
 
 from UserDict import DictMixin
 from mercurial import ui, config, hg
 from mercurial.error import RepoError
 from rhodecode.model import meta
 from rhodecode.model.caching_query import FromCache
 from rhodecode.model.db import Repository, User, RhodeCodeUi, RhodeCodeSettings, \
     UserLog
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.user import UserModel
 from vcs.backends.base import BaseChangeset
 from vcs.backends.git import GitRepository
 from vcs.backends.hg import MercurialRepository
 from vcs.utils.lazy import LazyProperty
 import datetime
 import logging
 import os
 
 log = logging.getLogger(__name__)
 
 
 def get_repo_slug(request):
     return request.environ['pylons.routes_dict'].get('repo_name')
 
 def is_mercurial(environ):
     """
     Returns True if request's target is mercurial server - header
     ``HTTP_ACCEPT`` of such request would start with ``application/mercurial``.
     """
     http_accept = environ.get('HTTP_ACCEPT')
     if http_accept and http_accept.startswith('application/mercurial'):
         return True
     return False
 
 def is_git(environ):
     """
     Returns True if request's target is git server. ``HTTP_USER_AGENT`` would
     then have git client version given.
     
     :param environ:
     """
     http_user_agent = environ.get('HTTP_USER_AGENT')
     if http_user_agent.startswith('git'):
         return True
     return False
 
 def action_logger(user, action, repo, ipaddr, sa=None):
     """
     Action logger for various action made by users
     """
 
     if not sa:
         sa = meta.Session()
 
     try:
         if hasattr(user, 'user_id'):
             user_id = user.user_id
         elif isinstance(user, basestring):
             user_id = UserModel(sa).get_by_username(user, cache=False).user_id
         else:
             raise Exception('You have to provide user object or username')
 
         repo_name = repo.lstrip('/')
         user_log = UserLog()
         user_log.user_id = user_id
         user_log.action = action
         user_log.repository_name = repo_name
         user_log.repository = RepoModel(sa).get(repo_name, cache=False)
         user_log.action_date = datetime.datetime.now()
         user_log.user_ip = ipaddr
         sa.add(user_log)
         sa.commit()
 
         log.info('Adding user %s, action %s on %s',
                                         user.username, action, repo)
     except Exception, e:
         sa.rollback()
         log.error('could not log user action:%s', str(e))
 
 def get_repos(path, recursive=False, initial=False):
     """
     Scans given path for repos and return (name,(type,path)) tuple 
     :param prefix:
     :param path:
     :param recursive:
     :param initial:
     """
     from vcs.utils.helpers import get_scm
     from vcs.exceptions import VCSError
 
     try:
         scm = get_scm(path)
     except:
         pass
     else:
         raise Exception('The given path %s should not be a repository got %s',
                         path, scm)
 
     for dirpath in os.listdir(path):
         try:
             yield dirpath, get_scm(os.path.join(path, dirpath))
         except VCSError:
             pass
 
 if __name__ == '__main__':
     get_repos('', '/home/marcink/workspace-python')
 
 
 def check_repo_fast(repo_name, base_path):
     if os.path.isdir(os.path.join(base_path, repo_name)):return False
     return True
 
 def check_repo(repo_name, base_path, verify=True):
 
     repo_path = os.path.join(base_path, repo_name)
 
     try:
         if not check_repo_fast(repo_name, base_path):
             return False
         r = hg.repository(ui.ui(), repo_path)
         if verify:
             hg.verify(r)
         #here we hnow that repo exists it was verified
         log.info('%s repo is already created', repo_name)
         return False
     except RepoError:
         #it means that there is no valid repo there...
         log.info('%s repo is free for creation', repo_name)
         return True
 
 def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):
     while True:
         ok = raw_input(prompt)
         if ok in ('y', 'ye', 'yes'): return True
         if ok in ('n', 'no', 'nop', 'nope'): return False
         retries = retries - 1
         if retries < 0: raise IOError
         print complaint
 
 def get_hg_ui_cached():
     try:
         sa = meta.Session
         ret = sa.query(RhodeCodeUi)\
         .options(FromCache("sql_cache_short", "get_hg_ui_settings"))\
         .all()
     except:
         pass
     finally:
         meta.Session.remove()
     return ret
 
 
 def get_hg_settings():
     try:
         sa = meta.Session()
         ret = sa.query(RhodeCodeSettings)\
         .options(FromCache("sql_cache_short", "get_hg_settings"))\
         .all()
     except:
         pass
     finally:
         meta.Session.remove()
 
     if not ret:
         raise Exception('Could not get application settings !')
     settings = {}
     for each in ret:
         settings['rhodecode_' + each.app_settings_name] = each.app_settings_value
 
     return settings
 
 def get_hg_ui_settings():
     try:
         sa = meta.Session()
         ret = sa.query(RhodeCodeUi).all()
     except:
         pass
     finally:
         meta.Session.remove()
 
     if not ret:
         raise Exception('Could not get application ui settings !')
     settings = {}
     for each in ret:
         k = each.ui_key
         v = each.ui_value
         if k == '/':
             k = 'root_path'
 
         if k.find('.') != -1:
             k = k.replace('.', '_')
 
         if each.ui_section == 'hooks':
             v = each.ui_active
 
         settings[each.ui_section + '_' + k] = v
 
     return settings
 
 #propagated from mercurial documentation
 ui_sections = ['alias', 'auth',
                 'decode/encode', 'defaults',
                 'diff', 'email',
                 'extensions', 'format',
                 'merge-patterns', 'merge-tools',
                 'hooks', 'http_proxy',
                 'smtp', 'patch',
                 'paths', 'profiling',
                 'server', 'trusted',
                 'ui', 'web', ]
 
 def make_ui(read_from='file', path=None, checkpaths=True):
     """
     A function that will read python rc files or database
     and make an mercurial ui object from read options
     
     :param path: path to mercurial config file
     :param checkpaths: check the path
     :param read_from: read from 'file' or 'db'
     """
 
     baseui = ui.ui()
 
     if read_from == 'file':
         if not os.path.isfile(path):
             log.warning('Unable to read config file %s' % path)
             return False
         log.debug('reading hgrc from %s', path)
         cfg = config.config()
         cfg.read(path)
         for section in ui_sections:
             for k, v in cfg.items(section):
                 baseui.setconfig(section, k, v)
                 log.debug('settings ui from file[%s]%s:%s', section, k, v)
 
     elif read_from == 'db':
         hg_ui = get_hg_ui_cached()
         for ui_ in hg_ui:
             if ui_.ui_active:
                 log.debug('settings ui from db[%s]%s:%s', ui_.ui_section, ui_.ui_key, ui_.ui_value)
                 baseui.setconfig(ui_.ui_section, ui_.ui_key, ui_.ui_value)
 
 
     return baseui
 
 
 def set_rhodecode_config(config):
     hgsettings = get_hg_settings()
 
     for k, v in hgsettings.items():
         config[k] = v
 
 def invalidate_cache(name, *args):
     """Invalidates given name cache"""
 
     from beaker.cache import region_invalidate
     log.info('INVALIDATING CACHE FOR %s', name)
 
     """propagate our arguments to make sure invalidation works. First
     argument has to be the name of cached func name give to cache decorator
     without that the invalidation would not work"""
     tmp = [name]
     tmp.extend(args)
     args = tuple(tmp)
 
     if name == 'cached_repo_list':
         from rhodecode.model.hg import _get_repos_cached
         region_invalidate(_get_repos_cached, None, *args)
 
     if name == 'full_changelog':
         from rhodecode.model.hg import _full_changelog_cached
         region_invalidate(_full_changelog_cached, None, *args)
 
 class EmptyChangeset(BaseChangeset):
     """
-    An dummy empty changeset.
+    An dummy empty changeset. It's possible to pass hash when creating
+    an EmptyChangeset
     """
 
-    revision = -1
-    message = ''
-    author = ''
-    date = ''
+    def __init__(self, cs='0' * 40):
+        self._empty_cs = cs
+        self.revision = -1
+        self.message = ''
+        self.author = ''
+        self.date = ''
 
     @LazyProperty
     def raw_id(self):
         """
         Returns raw string identifying this changeset, useful for web
         representation.
         """
-        return '0' * 40
+        return self._empty_cs
 
     @LazyProperty
     def short_id(self):
         return self.raw_id[:12]
 
     def get_file_changeset(self, path):
         return self
 
     def get_file_content(self, path):
         return u''
 
     def get_file_size(self, path):
         return 0
 
 def repo2db_mapper(initial_repo_list, remove_obsolete=False):
     """
     maps all found repositories into db
     """
 
     sa = meta.Session()
     rm = RepoModel(sa)
     user = sa.query(User).filter(User.admin == True).first()
 
     for name, repo in initial_repo_list.items():
         if not rm.get(name, cache=False):
             log.info('repository %s not found creating default', name)
 
             if isinstance(repo, MercurialRepository):
                 repo_type = 'hg'
             if isinstance(repo, GitRepository):
                 repo_type = 'git'
 
             form_data = {
                          'repo_name':name,
                          'repo_type':repo_type,
                          'description':repo.description if repo.description != 'unknown' else \
                                         'auto description for %s' % name,
                          'private':False
                          }
             rm.create(form_data, user, just_db=True)
 
 
     if remove_obsolete:
         #remove from database those repositories that are not in the filesystem
         for repo in sa.query(Repository).all():
             if repo.repo_name not in initial_repo_list.keys():
                 sa.delete(repo)
                 sa.commit()
 
 
     meta.Session.remove()
 
 
 class OrderedDict(dict, DictMixin):
 
     def __init__(self, *args, **kwds):
         if len(args) > 1:
             raise TypeError('expected at most 1 arguments, got %d' % len(args))
         try:
             self.__end
         except AttributeError:
             self.clear()
         self.update(*args, **kwds)
 
     def clear(self):
         self.__end = end = []
         end += [None, end, end]         # sentinel node for doubly linked list
         self.__map = {}                 # key --> [key, prev, next]
         dict.clear(self)
 
     def __setitem__(self, key, value):
         if key not in self:
             end = self.__end
             curr = end[1]
             curr[2] = end[1] = self.__map[key] = [key, curr, end]
         dict.__setitem__(self, key, value)
 
     def __delitem__(self, key):
         dict.__delitem__(self, key)
         key, prev, next = self.__map.pop(key)
         prev[2] = next
         next[1] = prev
 
     def __iter__(self):
         end = self.__end
         curr = end[2]
         while curr is not end:
             yield curr[0]
             curr = curr[2]
 
     def __reversed__(self):
         end = self.__end
         curr = end[1]
         while curr is not end:
             yield curr[0]
             curr = curr[1]
 
     def popitem(self, last=True):
         if not self:
             raise KeyError('dictionary is empty')
         if last:
             key = reversed(self).next()
         else:
             key = iter(self).next()
         value = self.pop(key)
         return key, value
 
     def __reduce__(self):
         items = [[k, self[k]] for k in self]
         tmp = self.__map, self.__end
         del self.__map, self.__end
         inst_dict = vars(self).copy()
         self.__map, self.__end = tmp
         if inst_dict:
             return (self.__class__, (items,), inst_dict)
         return self.__class__, (items,)
 
     def keys(self):
         return list(self)
 
     setdefault = DictMixin.setdefault
     update = DictMixin.update
     pop = DictMixin.pop
     values = DictMixin.values
     items = DictMixin.items
     iterkeys = DictMixin.iterkeys
     itervalues = DictMixin.itervalues
     iteritems = DictMixin.iteritems
 
     def __repr__(self):
         if not self:
             return '%s()' % (self.__class__.__name__,)
         return '%s(%r)' % (self.__class__.__name__, self.items())
 
     def copy(self):
         return self.__class__(self)
 
     @classmethod
     def fromkeys(cls, iterable, value=None):
         d = cls()
         for key in iterable:
             d[key] = value
         return d
 
     def __eq__(self, other):
         if isinstance(other, OrderedDict):
             return len(self) == len(other) and self.items() == other.items()
         return dict.__eq__(self, other)
 
     def __ne__(self, other):
         return not self == other
 
 
 #===============================================================================
 # TEST FUNCTIONS AND CREATORS
 #===============================================================================
 def create_test_index(repo_location, full_index):
     """Makes default test index
     :param repo_location:
     :param full_index:
     """
     from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
     from rhodecode.lib.pidlock import DaemonLock, LockHeld
     from rhodecode.lib.indexers import IDX_LOCATION
     import shutil
 
     if os.path.exists(IDX_LOCATION):
         shutil.rmtree(IDX_LOCATION)
 
     try:
         l = DaemonLock()
         WhooshIndexingDaemon(repo_location=repo_location)\
             .run(full_index=full_index)
         l.release()
     except LockHeld:
         pass
 
 def create_test_env(repos_test_path, config):
     """Makes a fresh database and 
     install test repository into tmp dir
     """
     from rhodecode.lib.db_manage import DbManage
     import tarfile
     import shutil
     from os.path import dirname as dn, join as jn, abspath
 
     log = logging.getLogger('TestEnvCreator')
     # create logger
     log.setLevel(logging.DEBUG)
     log.propagate = True
     # create console handler and set level to debug
     ch = logging.StreamHandler()
     ch.setLevel(logging.DEBUG)
 
     # create formatter
     formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
 
     # add formatter to ch
     ch.setFormatter(formatter)
 
     # add ch to logger
     log.addHandler(ch)
 
     #PART ONE create db
     dbname = config['sqlalchemy.db1.url'].split('/')[-1]
     log.debug('making test db %s', dbname)
 
     dbmanage = DbManage(log_sql=True, dbname=dbname, root=config['here'],
                         tests=True)
     dbmanage.create_tables(override=True)
     dbmanage.config_prompt(repos_test_path)
     dbmanage.create_default_user()
     dbmanage.admin_prompt()
     dbmanage.create_permissions()
     dbmanage.populate_default_permissions()
 
     #PART TWO make test repo
     log.debug('making test vcs repo')
     if os.path.isdir('/tmp/vcs_test'):
         shutil.rmtree('/tmp/vcs_test')
 
     cur_dir = dn(dn(abspath(__file__)))
     tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test.tar.gz"))
     tar.extractall('/tmp')
     tar.close()

setup.py

@@ -1,88 +1,88 b''
 from rhodecode import get_version
 import sys
 py_version = sys.version_info
 
 requirements = [
         "Pylons>=1.0.0",
         "SQLAlchemy>=0.6.4",
         "Mako>=0.3.5",
-        "vcs==0.1.10",
+        "vcs>=0.1.10",
         "pygments>=1.3.0",
         "mercurial==1.6.4",
-        "whoosh==1.1.1",
-        "celery==2.1.1",
+        "whoosh>=1.2.5",
+        "celery>=2.1.2",
         "py-bcrypt",
         "babel",
     ]
 
 classifiers = ['Development Status :: 4 - Beta',
                    'Environment :: Web Environment',
                    'Framework :: Pylons',
                    'Intended Audience :: Developers',
                    'License :: OSI Approved :: BSD License',
                    'Operating System :: OS Independent',
                    'Programming Language :: Python', ]
 
 if sys.version_info < (2, 6):
     requirements.append("simplejson")
     requirements.append("pysqlite")
 
 #additional files from project that goes somewhere in the filesystem
 #relative to sys.prefix
 data_files = []
 
 #additional files that goes into package itself
 package_data = {'rhodecode': ['i18n/*/LC_MESSAGES/*.mo', ], }
 
 description = 'Mercurial repository serving and browsing app'
 #long description
 try:
     readme_file = 'README.rst'
     long_description = open(readme_file).read()
 except IOError, err:
     sys.stderr.write("[WARNING] Cannot find file specified as "
         "long_description (%s)\n skipping that file" % readme_file)
     long_description = description
 
 
 try:
     from setuptools import setup, find_packages
 except ImportError:
     from ez_setup import use_setuptools
     use_setuptools()
     from setuptools import setup, find_packages
 #packages
 packages = find_packages(exclude=['ez_setup'])
 
 setup(
     name='RhodeCode',
     version=get_version(),
     description=description,
     long_description=long_description,
     keywords='rhodiumcode mercurial web hgwebdir replacement serving hgweb rhodecode',
     license='BSD',
     author='Marcin Kuzminski',
     author_email='[e-mail]',
     url='http://hg.python-works.com',
     install_requires=requirements,
     classifiers=classifiers,
     setup_requires=["PasteScript>=1.6.3"],
     data_files=data_files,
     packages=packages,
     include_package_data=True,
     test_suite='nose.collector',
     package_data=package_data,
     message_extractors={'rhodecode': [
             ('**.py', 'python', None),
             ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
             ('public/**', 'ignore', None)]},
     zip_safe=False,
     paster_plugins=['PasteScript', 'Pylons'],
     entry_points="""
     [paste.app_factory]
     main = rhodecode.config.middleware:make_app
 
     [paste.app_install]
     main = pylons.util:PylonsInstaller
     """,
 )