u/pc/rhodecode-enterprise-ce-fork-pc Files · rhodecode/lib/index/whoosh_fallback_schema.py

caches: use individual namespaces per user to prevent beaker caching problems....

caches: use individual namespaces per user to prevent beaker caching problems. - especially for mysql in case large number of data in caches there could be critical errors storing cache, and thus preventing users from authentication. This is caused by the fact that we used single namespace for ALL users. It means it grew as number of users grew reaching mysql single column limit. This changes the behaviour and now we use namespace per-user it means that each user-id will have it's own cache namespace fragmenting maximum column data to a single user cache. Which we should never reach.

marcink - - Load All Authors

File last commit:

r2487:fcee5614 default


                r2591:36829a17

stable

Download file

             whoosh_fallback_schema.py
        
                    75 lines
            
             | 2.8 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / rhodecode / lib / index / whoosh_fallback_schema.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        marcink
    
project: added all source files and assets

              r1
            
      # -*- coding: utf-8 -*-

        marcink
    
release: update copyright year to 2018

              r2487
            
      # Copyright (C) 2012-2018 RhodeCode GmbH

        marcink
    
project: added all source files and assets

              r1
            
      #

      # This program is free software: you can redistribute it and/or modify

      # it under the terms of the GNU Affero General Public License, version 3

      # (only), as published by the Free Software Foundation.

      #

      # This program is distributed in the hope that it will be useful,

      # but WITHOUT ANY WARRANTY; without even the implied warranty of

      # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

      # GNU General Public License for more details.

      #

      # You should have received a copy of the GNU Affero General Public License

      # along with this program.  If not, see <http://www.gnu.org/licenses/>.

      #

      # This program is dual-licensed. If you wish to learn more about the

      # RhodeCode Enterprise Edition, including its added features, Support services,

      # and proprietary license terms, please see https://rhodecode.com/licenses/

      """

      Whoosh fallback schema for RhodeCode in case rhodecode_tools defined one is

      not available

      """

      from __future__ import absolute_import

      from whoosh.analysis import RegexTokenizer, LowercaseFilter

      from whoosh.formats import Characters

      from whoosh.fields import (

          TEXT, ID, STORED, NUMERIC, BOOLEAN, Schema, FieldType, DATETIME)

      # CUSTOM ANALYZER wordsplit + lowercase filter for case insensitive search

      ANALYZER = RegexTokenizer(expression=r"\w+") | LowercaseFilter()

      # FILE INDEX SCHEMA DEFINITION

      FILE_INDEX_NAME = 'FILE_INDEX'

      FILE_SCHEMA = Schema(

          fileid=ID(unique=True),  # Path

          repository=ID(stored=True),

          repository_id=NUMERIC(unique=True, stored=True),  # Numeric id of repo

          repo_name=TEXT(stored=True),

          owner=TEXT(),

          path=TEXT(stored=True),

          content=FieldType(format=Characters(), analyzer=ANALYZER,

                            scorable=True, stored=True),

          modtime=STORED(),

          md5=STORED(),

          extension=ID(stored=True),

          commit_id=TEXT(stored=True),

        marcink
    
whoosh: fallback schema uses now bigger ints, fixes #4035

              r413
            
          size=NUMERIC(int, 64, signed=False, stored=True),

        marcink
    
project: added all source files and assets

              r1
            
          mimetype=TEXT(stored=True),

        marcink
    
whoosh: fallback schema uses now bigger ints, fixes #4035

              r413
            
          lines=NUMERIC(int, 64, signed=False, stored=True),

        marcink
    
project: added all source files and assets

              r1
            
      )

      # COMMIT INDEX SCHEMA

      COMMIT_INDEX_NAME = 'COMMIT_INDEX'

      COMMIT_SCHEMA = Schema(

          commit_id=ID(unique=True, stored=True),

          repository=ID(unique=True, stored=True),

          repository_id=NUMERIC(unique=True, stored=True),

          commit_idx=NUMERIC(stored=True, sortable=True),

          commit_idx_sort=ID(),

        marcink
    
whoosh: fallback schema uses now bigger ints, fixes #4035

              r413
            
          date=NUMERIC(int, 64, signed=False, stored=True, sortable=True),

        marcink
    
project: added all source files and assets

              r1
            
          owner=TEXT(stored=True),

          author=TEXT(stored=True),

          message=FieldType(format=Characters(), analyzer=ANALYZER,

                            scorable=True, stored=True),

          parents=TEXT(stored=True),

          added=TEXT(stored=True),  # space separated names of added files

          removed=TEXT(stored=True),  # space separated names of removed files

          changed=TEXT(stored=True),  # space separated names of changed files

      )

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

marcink project: added all source files and assets	r1	# -- coding: utf-8 --

marcink release: update copyright year to 2018	r2487	# Copyright (C) 2012-2018 RhodeCode GmbH
marcink project: added all source files and assets	r1	#
		# This program is free software: you can redistribute it and/or modify
		# it under the terms of the GNU Affero General Public License, version 3
		# (only), as published by the Free Software Foundation.
		#
		# This program is distributed in the hope that it will be useful,
		# but WITHOUT ANY WARRANTY; without even the implied warranty of
		# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
		# GNU General Public License for more details.
		#
		# You should have received a copy of the GNU Affero General Public License
		# along with this program. If not, see <http://www.gnu.org/licenses/>.
		#
		# This program is dual-licensed. If you wish to learn more about the
		# RhodeCode Enterprise Edition, including its added features, Support services,
		# and proprietary license terms, please see https://rhodecode.com/licenses/

		"""
		Whoosh fallback schema for RhodeCode in case rhodecode_tools defined one is
		not available
		"""

		from __future__ import absolute_import

		from whoosh.analysis import RegexTokenizer, LowercaseFilter
		from whoosh.formats import Characters
		from whoosh.fields import (
		TEXT, ID, STORED, NUMERIC, BOOLEAN, Schema, FieldType, DATETIME)

		# CUSTOM ANALYZER wordsplit + lowercase filter for case insensitive search
		ANALYZER = RegexTokenizer(expression=r"\w+") \| LowercaseFilter()

		# FILE INDEX SCHEMA DEFINITION
		FILE_INDEX_NAME = 'FILE_INDEX'
		FILE_SCHEMA = Schema(
		fileid=ID(unique=True), # Path
		repository=ID(stored=True),
		repository_id=NUMERIC(unique=True, stored=True), # Numeric id of repo
		repo_name=TEXT(stored=True),
		owner=TEXT(),
		path=TEXT(stored=True),
		content=FieldType(format=Characters(), analyzer=ANALYZER,
		scorable=True, stored=True),
		modtime=STORED(),
		md5=STORED(),
		extension=ID(stored=True),
		commit_id=TEXT(stored=True),

marcink whoosh: fallback schema uses now bigger ints, fixes #4035	r413	size=NUMERIC(int, 64, signed=False, stored=True),
marcink project: added all source files and assets	r1	mimetype=TEXT(stored=True),
marcink whoosh: fallback schema uses now bigger ints, fixes #4035	r413	lines=NUMERIC(int, 64, signed=False, stored=True),
marcink project: added all source files and assets	r1	)


		# COMMIT INDEX SCHEMA
		COMMIT_INDEX_NAME = 'COMMIT_INDEX'
		COMMIT_SCHEMA = Schema(
		commit_id=ID(unique=True, stored=True),
		repository=ID(unique=True, stored=True),
		repository_id=NUMERIC(unique=True, stored=True),
		commit_idx=NUMERIC(stored=True, sortable=True),
		commit_idx_sort=ID(),
marcink whoosh: fallback schema uses now bigger ints, fixes #4035	r413	date=NUMERIC(int, 64, signed=False, stored=True, sortable=True),
marcink project: added all source files and assets	r1	owner=TEXT(stored=True),
		author=TEXT(stored=True),
		message=FieldType(format=Characters(), analyzer=ANALYZER,
		scorable=True, stored=True),
		parents=TEXT(stored=True),
		added=TEXT(stored=True), # space separated names of added files
		removed=TEXT(stored=True), # space separated names of removed files
		changed=TEXT(stored=True), # space separated names of changed files
		)