appenlight Pull Request !2235 · RhodeCode Free Hosting

Closed

Pull request !2235 Created on by

mahdish8

ini: added new key
registration: add a way to disable registration
Update register.jinja2
alert_channels: it should also be a pkey
alert_channels: allow binding to resources

Commit flow:

branch:master of appenlight → branch:stable of appenlight

Versions:

Pull request versions not available.

Expand 13 commits

ver	Time	Author	Commit	Description
`latest`	Mon, 25 Jun 2018 10:20:39	ergo	`r129:489ce37b1c22`	ini: added new key
`latest`	Mon, 25 Jun 2018 10:07:35	ergo	`r128:17f1d9985f3b`	registration: add a way to disable registration
`latest`	Fri, 25 May 2018 13:11:14	marcink	`r127:96df22af16d2`	Update register.jinja2 Added confirmation of approval of terms on registration
`latest`	Wed, 28 Feb 2018 19:48:16	ergo	`r126:ea01d08ee3d0`	alert_channels: it should also be a pkey
`latest`	Wed, 28 Feb 2018 19:28:31	ergo	`r125:316fd03af797`	alert_channels: allow binding to resources
`latest`	Wed, 28 Feb 2018 19:26:59	ergo	`r124:a5053823fac0`	Merge pull request #1 from pagenoare/master Option to blacklist certain resources from AlertChannels
`latest`	Wed, 28 Feb 2018 12:56:37	pagenoare	`r123:a4ed66c86f5b`	Option to blacklist certain resources from AlertChannels
`latest`	Sun, 11 Feb 2018 13:08:57	ergo	`r122:a8c5905d3906`	elasticsearch: make compatible with elasticsearch 5.6
`latest`	Fri, 13 Oct 2017 12:57:23	ergo	`r121:dfb8d8d59eca`	init: fix license type
`latest`	Thu, 21 Sep 2017 16:26:10	ergo	`r120:a8b66364c619`	readme: updates
`latest`	Thu, 21 Sep 2017 16:25:36	ergo	`r119:5de597004cf5`	ignore: added gitignore
`latest`	Wed, 13 Sep 2017 10:36:45	marcink	`r118:52cdf94b6993`	release: Bump version 1.0.0 to 1.1.0
`latest`	Wed, 13 Sep 2017 10:34:40	marcink	`r117:a2cc01f8fb81`	merge stable into default
13 commits hidden, click expand to show them.

Expand all files

The requested changes are too big and content was truncated. Show full diff

.bumpversion.cfg

0 created 644 +7 0

@@ -0,0 +1,7 b''
	1	[bumpversion]
	2	current_version = 1.1.0
	3	message = release: Bump version {current_version} to {new_version}
	4	tag_name = {new_version}
	5
	6	[bumpversion:file:backend/src/appenlight/VERSION]
	7

.hgignore

0 created 644 +161 0

@@ -0,0 +1,161 b''
	1	# Created by .ignore support plugin (hsz.mobi)
	2	syntax: glob
	3
	4	### Example user template template
	5	### Example user template
	6
	7	.venv
	8	# IntelliJ project files
	9	.idea
	10	*.iml
	11	out
	12	gen### Python template
	13	# Byte-compiled / optimized / DLL files
	14	__pycache__/
	15	*.py[cod]
	16	*$py.class
	17
	18	# C extensions
	19	*.so
	20
	21	# Distribution / packaging
	22	.Python
	23	env/
	24	build/
	25	develop-eggs/
	26	dist/
	27	downloads/
	28	eggs/
	29	.eggs/
	30	$lib
	31	lib64/
	32	parts/
	33	sdist/
	34	var/
	35	*.egg-info/
	36	.installed.cfg
	37	*.egg
	38
	39	# PyInstaller
	40	# Usually these files are written by a python script from a template
	41	# before PyInstaller builds the exe, so as to inject date/other infos into it.
	42	*.manifest
	43	*.spec
	44
	45	# Installer logs
	46	pip-log.txt
	47	pip-delete-this-directory.txt
	48
	49	# Unit test / coverage reports
	50	htmlcov/
	51	.tox/
	52	.coverage
	53	.coverage.*
	54	.cache
	55	nosetests.xml
	56	coverage.xml
	57	*,cover
	58	.hypothesis/
	59
	60	# Translations
	61	*.mo
	62	*.pot
	63
	64	# Mac stuff:
	65	*.DS_Store
	66
	67	# Django stuff:
	68	*.log
	69	local_settings.py
	70
	71	# Flask instance folder
	72	instance/
	73
	74	# Scrapy stuff:
	75	.scrapy
	76
	77	# Sphinx documentation
	78	docs/_build/
	79
	80	# PyBuilder
	81	target/
	82
	83	# IPython Notebook
	84	.ipynb_checkpoints
	85
	86	# pyenv
	87	.python-version
	88
	89	# celery beat schedule file
	90	celerybeat-schedule
	91
	92	# dotenv
	93	.env
	94
	95	# virtualenv
	96	venv/
	97	ENV/
	98
	99	# Spyder project settings
	100	.spyderproject
	101
	102	# Rope project settings
	103	.ropeproject
	104
	105
	106	syntax: regexp
	107	^\.idea$
	108	syntax: regexp
	109	^\.settings$
	110	syntax: regexp
	111	^data$
	112	syntax: regexp
	113	^webassets$
	114	syntax: regexp
	115	^dist$
	116	syntax: regexp
	117	^\.project$
	118	syntax: regexp
	119	^\.pydevproject$
	120	syntax: regexp
	121	^private$
	122	syntax: regexp
	123	^appenlight_frontend/build$
	124	syntax: regexp
	125	^appenlight_frontend/bower_components$
	126	syntax: regexp
	127	^appenlight_frontend/node_modules$
	128	^src/node_modules$
	129	syntax: regexp
	130	^\.pydevproject$
	131	syntax: regexp
	132	appenlight\.egg-info$
	133	syntax: regexp
	134	\.pyc$
	135	syntax: regexp
	136	\celerybeat.*
	137	syntax: regexp
	138	\.iml$
	139	syntax: regexp
	140	^frontend/build$
	141	syntax: regexp
	142	^frontend/bower_components$
	143	syntax: regexp
	144	^frontend/node_modules$
	145	^frontend/src/node_modules$
	146	^frontend/build$
	147
	148	syntax: regexp
	149	\.db$
	150
	151	syntax: regexp
	152	packer_cache
	153
	154	syntax: regexp
	155	packer/packer
	156
	157	syntax: regexp
	158	install_appenlight_production.yaml
	159	^docs/result$
	160	^docs/Appenlight-docs/_build$
	161	^docs/www$

backend/CHANGELOG.rst

0 created 644 +16 0

@@ -0,0 +1,16 b''
	1	.. :changelog:
	2
	3	History
	4	-------
	5
	6	0.9.1 (2016-XX-XX)
	7	++++++++++++++++++
	8
	9	* Added suppot for "NOT' operator in rule engine
	10	* Various bugfixes
	11
	12
	13	0.9.0 (2016-06-29)
	14	++++++++++++++++++
	15
	16	* first tagged public release

backend/pip2nix.ini

0 created 644 +3 0

@@ -0,0 +1,3 b''
	1	[pip2nix]
	2	requirements = ., -r ./requirements.txt
	3	output = ./python-packages.nix

backend/python-packages.nix

0 created 644 +1082 0

This diff has been collapsed as it changes many lines, (1082 lines changed) Show them Hide them
	@@ -0,0 +1,1082 b''
		1	{
		2	Jinja2 = super.buildPythonPackage {
		3	name = "Jinja2-2.8";
		4	buildInputs = with self; [];
		5	doCheck = false;
		6	propagatedBuildInputs = with self; [MarkupSafe];
		7	src = fetchurl {
		8	url = "https://pypi.python.org/packages/f2/2f/0b98b06a345a761bec91a079ccae392d282690c2d8272e708f4d10829e22/Jinja2-2.8.tar.gz";
		9	md5 = "edb51693fe22c53cee5403775c71a99e";
		10	};
		11	meta = {
		12	license = [ pkgs.lib.licenses.bsdOriginal ];
		13	};
		14	};
		15	Mako = super.buildPythonPackage {
		16	name = "Mako-1.0.4";
		17	buildInputs = with self; [];
		18	doCheck = false;
		19	propagatedBuildInputs = with self; [MarkupSafe];
		20	src = fetchurl {
		21	url = "https://pypi.python.org/packages/7a/ae/925434246ee90b42e8ef57d3b30a0ab7caf9a2de3e449b876c56dcb48155/Mako-1.0.4.tar.gz";
		22	md5 = "c5fc31a323dd4990683d2f2da02d4e20";
		23	};
		24	meta = {
		25	license = [ pkgs.lib.licenses.mit ];
		26	};
		27	};
		28	MarkupSafe = super.buildPythonPackage {
		29	name = "MarkupSafe-0.23";
		30	buildInputs = with self; [];
		31	doCheck = false;
		32	propagatedBuildInputs = with self; [];
		33	src = fetchurl {
		34	url = "https://pypi.python.org/packages/c0/41/bae1254e0396c0cc8cf1751cb7d9afc90a602353695af5952530482c963f/MarkupSafe-0.23.tar.gz";
		35	md5 = "f5ab3deee4c37cd6a922fb81e730da6e";
		36	};
		37	meta = {
		38	license = [ pkgs.lib.licenses.bsdOriginal ];
		39	};
		40	};
		41	PasteDeploy = super.buildPythonPackage {
		42	name = "PasteDeploy-1.5.2";
		43	buildInputs = with self; [];
		44	doCheck = false;
		45	propagatedBuildInputs = with self; [];
		46	src = fetchurl {
		47	url = "https://pypi.python.org/packages/0f/90/8e20cdae206c543ea10793cbf4136eb9a8b3f417e04e40a29d72d9922cbd/PasteDeploy-1.5.2.tar.gz";
		48	md5 = "352b7205c78c8de4987578d19431af3b";
		49	};
		50	meta = {
		51	license = [ pkgs.lib.licenses.mit ];
		52	};
		53	};
		54	SQLAlchemy = super.buildPythonPackage {
		55	name = "SQLAlchemy-1.0.12";
		56	buildInputs = with self; [];
		57	doCheck = false;
		58	propagatedBuildInputs = with self; [];
		59	src = fetchurl {
		60	url = "https://pypi.python.org/packages/5c/52/9b48cd58eac58cae2a27923ff34c783f390b95413ff65669a86e98f80829/SQLAlchemy-1.0.12.tar.gz";
		61	md5 = "6d19ef29883bbebdcac6613cf391cac4";
		62	};
		63	meta = {
		64	license = [ pkgs.lib.licenses.mit ];
		65	};
		66	};
		67	WebOb = super.buildPythonPackage {
		68	name = "WebOb-1.6.1";
		69	buildInputs = with self; [];
		70	doCheck = false;
		71	propagatedBuildInputs = with self; [];
		72	src = fetchurl {
		73	url = "https://pypi.python.org/packages/5d/c7/7c1565b188d8d32bf3657a24b9d71621e35ba20ec4179a0a7f9803511099/WebOb-1.6.1.tar.gz";
		74	md5 = "04049d82e9d12dd91f6f46f54cc826aa";
		75	};
		76	meta = {
		77	license = [ pkgs.lib.licenses.mit ];
		78	};
		79	};
		80	alembic = super.buildPythonPackage {
		81	name = "alembic-0.8.6";
		82	buildInputs = with self; [];
		83	doCheck = false;
		84	propagatedBuildInputs = with self; [SQLAlchemy Mako python-editor];
		85	src = fetchurl {
		86	url = "https://pypi.python.org/packages/d2/c3/fdb752aa39832d056aeac958f35f1fb9fb9397a52bdab9248adcbd9f17d9/alembic-0.8.6.tar.gz";
		87	md5 = "6517b160e576cedf9b7625a18a9bc594";
		88	};
		89	meta = {
		90	license = [ pkgs.lib.licenses.mit ];
		91	};
		92	};
		93	amqp = super.buildPythonPackage {
		94	name = "amqp-1.4.9";
		95	buildInputs = with self; [];
		96	doCheck = false;
		97	propagatedBuildInputs = with self; [];
		98	src = fetchurl {
		99	url = "https://pypi.python.org/packages/cc/a4/f265c6f9a7eb1dd45d36d9ab775520e07ff575b11ad21156f9866da047b2/amqp-1.4.9.tar.gz";
		100	md5 = "df57dde763ba2dea25b3fa92dfe43c19";
		101	};
		102	meta = {
		103	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "LGPL"; } { fullName = "GNU Library or Lesser General Public License (LGPL)"; } ];
		104	};
		105	};
		106	anyjson = super.buildPythonPackage {
		107	name = "anyjson-0.3.3";
		108	buildInputs = with self; [];
		109	doCheck = false;
		110	propagatedBuildInputs = with self; [];
		111	src = fetchurl {
		112	url = "https://pypi.python.org/packages/c3/4d/d4089e1a3dd25b46bebdb55a992b0797cff657b4477bc32ce28038fdecbc/anyjson-0.3.3.tar.gz";
		113	md5 = "2ea28d6ec311aeeebaf993cb3008b27c";
		114	};
		115	meta = {
		116	license = [ pkgs.lib.licenses.bsdOriginal ];
		117	};
		118	};
		119	appenlight = super.buildPythonPackage {
		120	name = "appenlight-0.9.0";
		121	buildInputs = with self; [];
		122	doCheck = false;
		123	propagatedBuildInputs = with self; [repoze.sendmail pyramid pyramid-tm pyramid-debugtoolbar pyramid-authstack SQLAlchemy alembic webhelpers2 transaction zope.sqlalchemy pyramid-mailer redis redlock-py pyramid-jinja2 psycopg2 wtforms celery formencode psutil ziggurat-foundations bcrypt appenlight-client markdown colander defusedxml dogpile.cache pyramid-redis-sessions simplejson waitress gunicorn requests requests-oauthlib gevent gevent-websocket pygments lxml paginate paginate-sqlalchemy pyelasticsearch six mock itsdangerous camplight jira python-dateutil authomatic cryptography webassets];
		124	src = ./.;
		125	meta = {
		126	license = [ { fullName = "AGPLv3, and Commercial License"; } ];
		127	};
		128	};
		129	appenlight-client = super.buildPythonPackage {
		130	name = "appenlight-client-0.6.17";
		131	buildInputs = with self; [];
		132	doCheck = false;
		133	propagatedBuildInputs = with self; [WebOb requests six];
		134	src = fetchurl {
		135	url = "https://pypi.python.org/packages/af/86/1075f162d6534080f7f6ed9d8a83254e8f0be90c0a3e7ead9feffbe4423f/appenlight_client-0.6.17.tar.gz";
		136	md5 = "2f4d8229ce2dba607a9077210857e0e5";
		137	};
		138	meta = {
		139	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "DFSG approved"; } ];
		140	};
		141	};
		142	authomatic = super.buildPythonPackage {
		143	name = "authomatic-0.1.0.post1";
		144	buildInputs = with self; [];
		145	doCheck = false;
		146	propagatedBuildInputs = with self; [];
		147	src = fetchurl {
		148	url = "https://pypi.python.org/packages/08/1a/8a930461e604c2d5a7a871e1ac59fa82ccf994c32e807230c8d2fb07815a/Authomatic-0.1.0.post1.tar.gz";
		149	md5 = "be3f3ce08747d776aae6d6cc8dcb49a9";
		150	};
		151	meta = {
		152	license = [ pkgs.lib.licenses.mit ];
		153	};
		154	};
		155	bcrypt = super.buildPythonPackage {
		156	name = "bcrypt-2.0.0";
		157	buildInputs = with self; [];
		158	doCheck = false;
		159	propagatedBuildInputs = with self; [cffi six];
		160	src = fetchurl {
		161	url = "https://pypi.python.org/packages/11/7d/4c7980d04314466de42ea804db71995c9b3a2a47dc79a63c51f1be0cfd50/bcrypt-2.0.0.tar.gz";
		162	md5 = "e7fb17be46904cdb2ae6a062859ee58c";
		163	};
		164	meta = {
		165	license = [ pkgs.lib.licenses.asl20 ];
		166	};
		167	};
		168	billiard = super.buildPythonPackage {
		169	name = "billiard-3.3.0.23";
		170	buildInputs = with self; [];
		171	doCheck = false;
		172	propagatedBuildInputs = with self; [];
		173	src = fetchurl {
		174	url = "https://pypi.python.org/packages/64/a6/d7b6fb7bd0a4680a41f1d4b27061c7b768c673070ba8ac116f865de4e7ca/billiard-3.3.0.23.tar.gz";
		175	md5 = "6ee416e1e7c8d8164ce29d7377cca6a4";
		176	};
		177	meta = {
		178	license = [ pkgs.lib.licenses.bsdOriginal ];
		179	};
		180	};
		181	camplight = super.buildPythonPackage {
		182	name = "camplight-0.9.6";
		183	buildInputs = with self; [];
		184	doCheck = false;
		185	propagatedBuildInputs = with self; [requests];
		186	src = fetchurl {
		187	url = "https://pypi.python.org/packages/60/df/bed89a1f1d06632b192eff09a8fa75f85e0080ff70229c8145fbc3b2afa8/camplight-0.9.6.tar.gz";
		188	md5 = "716cc7a4ea30da34ae4fcbfe2784ce59";
		189	};
		190	meta = {
		191	license = [ pkgs.lib.licenses.mit ];
		192	};
		193	};
		194	celery = super.buildPythonPackage {
		195	name = "celery-3.1.23";
		196	buildInputs = with self; [];
		197	doCheck = false;
		198	propagatedBuildInputs = with self; [pytz billiard kombu];
		199	src = fetchurl {
		200	url = "https://pypi.python.org/packages/ea/a6/6da0bac3ea8abbc2763fd2664af2955702f97f140f2d7277069445532b7c/celery-3.1.23.tar.gz";
		201	md5 = "c6f10f956a49424d553ab1391ab39ab2";
		202	};
		203	meta = {
		204	license = [ pkgs.lib.licenses.bsdOriginal ];
		205	};
		206	};
		207	certifi = super.buildPythonPackage {
		208	name = "certifi-2016.8.31";
		209	buildInputs = with self; [];
		210	doCheck = false;
		211	propagatedBuildInputs = with self; [];
		212	src = fetchurl {
		213	url = "https://pypi.python.org/packages/1c/d1/0133a5084f0d17db0270c6061e824a11b0e417d743f5ff4c594f4090ed89/certifi-2016.8.31.tar.gz";
		214	md5 = "2f22d484a36d38d98be74f9eeb2846ec";
		215	};
		216	meta = {
		217	license = [ pkgs.lib.licenses.isc ];
		218	};
		219	};
		220	cffi = super.buildPythonPackage {
		221	name = "cffi-1.8.2";
		222	buildInputs = with self; [];
		223	doCheck = false;
		224	propagatedBuildInputs = with self; [pycparser];
		225	src = fetchurl {
		226	url = "https://pypi.python.org/packages/b8/21/9d6f08d2d36a0a8c84623646b4ed5a07023d868823361a086b021fb21172/cffi-1.8.2.tar.gz";
		227	md5 = "538f307b6c5169bba41fbfda2b070762";
		228	};
		229	meta = {
		230	license = [ pkgs.lib.licenses.mit ];
		231	};
		232	};
		233	colander = super.buildPythonPackage {
		234	name = "colander-1.2";
		235	buildInputs = with self; [];
		236	doCheck = false;
		237	propagatedBuildInputs = with self; [translationstring iso8601];
		238	src = fetchurl {
		239	url = "https://pypi.python.org/packages/14/23/c9ceba07a6a1dc0eefbb215fc0dc64aabc2b22ee756bc0f0c13278fa0887/colander-1.2.tar.gz";
		240	md5 = "83db21b07936a0726e588dae1914b9ed";
		241	};
		242	meta = {
		243	license = [ { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		244	};
		245	};
		246	cryptography = super.buildPythonPackage {
		247	name = "cryptography-1.2.3";
		248	buildInputs = with self; [];
		249	doCheck = false;
		250	propagatedBuildInputs = with self; [idna pyasn1 six setuptools enum34 ipaddress cffi];
		251	src = fetchurl {
		252	url = "https://pypi.python.org/packages/8b/7d/9df253f059c8d9a9389f06df5d6301b0725a44dbf055a1f7aff8e455746a/cryptography-1.2.3.tar.gz";
		253	md5 = "5474d2b3e8c7555a60852e48d2743f85";
		254	};
		255	meta = {
		256	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "BSD or Apache License, Version 2.0"; } pkgs.lib.licenses.asl20 ];
		257	};
		258	};
		259	defusedxml = super.buildPythonPackage {
		260	name = "defusedxml-0.4.1";
		261	buildInputs = with self; [];
		262	doCheck = false;
		263	propagatedBuildInputs = with self; [];
		264	src = fetchurl {
		265	url = "https://pypi.python.org/packages/09/3b/b1afa9649f48517d027e99413fec54f387f648c90156b3cf6451c8cd45f9/defusedxml-0.4.1.tar.gz";
		266	md5 = "230a5eff64f878b392478e30376d673a";
		267	};
		268	meta = {
		269	license = [ pkgs.lib.licenses.psfl ];
		270	};
		271	};
		272	dogpile.cache = super.buildPythonPackage {
		273	name = "dogpile.cache-0.5.7";
		274	buildInputs = with self; [];
		275	doCheck = false;
		276	propagatedBuildInputs = with self; [dogpile.core];
		277	src = fetchurl {
		278	url = "https://pypi.python.org/packages/07/74/2a83bedf758156d9c95d112691bbad870d3b77ccbcfb781b4ef836ea7d96/dogpile.cache-0.5.7.tar.gz";
		279	md5 = "3e58ce41af574aab41d78e9c4190f194";
		280	};
		281	meta = {
		282	license = [ pkgs.lib.licenses.bsdOriginal ];
		283	};
		284	};
		285	dogpile.core = super.buildPythonPackage {
		286	name = "dogpile.core-0.4.1";
		287	buildInputs = with self; [];
		288	doCheck = false;
		289	propagatedBuildInputs = with self; [];
		290	src = fetchurl {
		291	url = "https://pypi.python.org/packages/0e/77/e72abc04c22aedf874301861e5c1e761231c288b5de369c18be8f4b5c9bb/dogpile.core-0.4.1.tar.gz";
		292	md5 = "01cb19f52bba3e95c9b560f39341f045";
		293	};
		294	meta = {
		295	license = [ pkgs.lib.licenses.bsdOriginal ];
		296	};
		297	};
		298	elasticsearch = super.buildPythonPackage {
		299	name = "elasticsearch-1.9.0";
		300	buildInputs = with self; [];
		301	doCheck = false;
		302	propagatedBuildInputs = with self; [urllib3];
		303	src = fetchurl {
		304	url = "https://pypi.python.org/packages/13/9b/540e311b31a10c2a904acfb08030c656047e5c7ba479d35df2799e5dccfe/elasticsearch-1.9.0.tar.gz";
		305	md5 = "3550390baea1639479f79758d66ab032";
		306	};
		307	meta = {
		308	license = [ pkgs.lib.licenses.asl20 ];
		309	};
		310	};
		311	enum34 = super.buildPythonPackage {
		312	name = "enum34-1.1.6";
		313	buildInputs = with self; [];
		314	doCheck = false;
		315	propagatedBuildInputs = with self; [];
		316	src = fetchurl {
		317	url = "https://pypi.python.org/packages/bf/3e/31d502c25302814a7c2f1d3959d2a3b3f78e509002ba91aea64993936876/enum34-1.1.6.tar.gz";
		318	md5 = "5f13a0841a61f7fc295c514490d120d0";
		319	};
		320	meta = {
		321	license = [ pkgs.lib.licenses.bsdOriginal ];
		322	};
		323	};
		324	formencode = super.buildPythonPackage {
		325	name = "formencode-1.3.0";
		326	buildInputs = with self; [];
		327	doCheck = false;
		328	propagatedBuildInputs = with self; [];
		329	src = fetchurl {
		330	url = "https://pypi.python.org/packages/99/5b/f71f36b81b42291a70f61104d0eeb1a30be856a052ebe032c37b45db840c/FormEncode-1.3.0.zip";
		331	md5 = "6df12d60bf3179402f2c2efd1129eb74";
		332	};
		333	meta = {
		334	license = [ pkgs.lib.licenses.psfl ];
		335	};
		336	};
		337	gevent = super.buildPythonPackage {
		338	name = "gevent-1.1.1";
		339	buildInputs = with self; [];
		340	doCheck = false;
		341	propagatedBuildInputs = with self; [greenlet];
		342	src = fetchurl {
		343	url = "https://pypi.python.org/packages/12/dc/0b2e57823225de86f6e111a65d212c9e3b64847dddaa19691a6cb94b0b2e/gevent-1.1.1.tar.gz";
		344	md5 = "1532f5396ab4d07a231f1935483be7c3";
		345	};
		346	meta = {
		347	license = [ pkgs.lib.licenses.mit ];
		348	};
		349	};
		350	gevent-websocket = super.buildPythonPackage {
		351	name = "gevent-websocket-0.9.5";
		352	buildInputs = with self; [];
		353	doCheck = false;
		354	propagatedBuildInputs = with self; [gevent];
		355	src = fetchurl {
		356	url = "https://pypi.python.org/packages/de/93/6bc86ddd65435a56a2f2ea7cc908d92fea894fc08e364156656e71cc1435/gevent-websocket-0.9.5.tar.gz";
		357	md5 = "03a8473b9a61426b0ef6094319141389";
		358	};
		359	meta = {
		360	license = [ { fullName = "Copyright 2011-2013 Jeffrey Gelens <jeffrey@noppo.pro>"; } pkgs.lib.licenses.asl20 ];
		361	};
		362	};
		363	greenlet = super.buildPythonPackage {
		364	name = "greenlet-0.4.10";
		365	buildInputs = with self; [];
		366	doCheck = false;
		367	propagatedBuildInputs = with self; [];
		368	src = fetchurl {
		369	url = "https://pypi.python.org/packages/67/62/ca2a95648666eaa2ffeb6a9b3964f21d419ae27f82f2e66b53da5b943fc4/greenlet-0.4.10.zip";
		370	md5 = "bed0c4b3b896702131f4d5c72f87c41d";
		371	};
		372	meta = {
		373	license = [ pkgs.lib.licenses.mit ];
		374	};
		375	};
		376	gunicorn = super.buildPythonPackage {
		377	name = "gunicorn-19.4.5";
		378	buildInputs = with self; [];
		379	doCheck = false;
		380	propagatedBuildInputs = with self; [];
		381	src = fetchurl {
		382	url = "https://pypi.python.org/packages/1e/67/95248e17050822ab436c8a43dbfc0625a8545775737e33b66508cffad278/gunicorn-19.4.5.tar.gz";
		383	md5 = "ce45c2dccba58784694dd77f23d9a677";
		384	};
		385	meta = {
		386	license = [ pkgs.lib.licenses.mit ];
		387	};
		388	};
		389	idna = super.buildPythonPackage {
		390	name = "idna-2.1";
		391	buildInputs = with self; [];
		392	doCheck = false;
		393	propagatedBuildInputs = with self; [];
		394	src = fetchurl {
		395	url = "https://pypi.python.org/packages/fb/84/8c27516fbaa8147acd2e431086b473c453c428e24e8fb99a1d89ce381851/idna-2.1.tar.gz";
		396	md5 = "f6473caa9c5e0cc1ad3fd5d04c3c114b";
		397	};
		398	meta = {
		399	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "BSD-like"; } ];
		400	};
		401	};
		402	ipaddress = super.buildPythonPackage {
		403	name = "ipaddress-1.0.17";
		404	buildInputs = with self; [];
		405	doCheck = false;
		406	propagatedBuildInputs = with self; [];
		407	src = fetchurl {
		408	url = "https://pypi.python.org/packages/bb/26/3b64955ff73f9e3155079b9ed31812afdfa5333b5c76387454d651ef593a/ipaddress-1.0.17.tar.gz";
		409	md5 = "8bbf0326719fafb1f453921ef96729fe";
		410	};
		411	meta = {
		412	license = [ pkgs.lib.licenses.psfl ];
		413	};
		414	};
		415	iso8601 = super.buildPythonPackage {
		416	name = "iso8601-0.1.11";
		417	buildInputs = with self; [];
		418	doCheck = false;
		419	propagatedBuildInputs = with self; [];
		420	src = fetchurl {
		421	url = "https://pypi.python.org/packages/c0/75/c9209ee4d1b5975eb8c2cba4428bde6b61bd55664a98290dd015cdb18e98/iso8601-0.1.11.tar.gz";
		422	md5 = "b06d11cd14a64096f907086044f0fe38";
		423	};
		424	meta = {
		425	license = [ pkgs.lib.licenses.mit ];
		426	};
		427	};
		428	itsdangerous = super.buildPythonPackage {
		429	name = "itsdangerous-0.24";
		430	buildInputs = with self; [];
		431	doCheck = false;
		432	propagatedBuildInputs = with self; [];
		433	src = fetchurl {
		434	url = "https://pypi.python.org/packages/dc/b4/a60bcdba945c00f6d608d8975131ab3f25b22f2bcfe1dab221165194b2d4/itsdangerous-0.24.tar.gz";
		435	md5 = "a3d55aa79369aef5345c036a8a26307f";
		436	};
		437	meta = {
		438	license = [ pkgs.lib.licenses.bsdOriginal ];
		439	};
		440	};
		441	jira = super.buildPythonPackage {
		442	name = "jira-1.0.7";
		443	buildInputs = with self; [];
		444	doCheck = false;
		445	propagatedBuildInputs = with self; [requests requests-oauthlib tlslite six requests-toolbelt];
		446	src = fetchurl {
		447	url = "https://pypi.python.org/packages/4e/36/4f0ab121c3510fce29743c31e2f47e99c2be68ee4441ad395366489351b0/jira-1.0.7.tar.gz";
		448	md5 = "cb1d3f1e1b7a388932ad5d961bf2c56d";
		449	};
		450	meta = {
		451	license = [ pkgs.lib.licenses.bsdOriginal ];
		452	};
		453	};
		454	kombu = super.buildPythonPackage {
		455	name = "kombu-3.0.35";
		456	buildInputs = with self; [];
		457	doCheck = false;
		458	propagatedBuildInputs = with self; [anyjson amqp];
		459	src = fetchurl {
		460	url = "https://pypi.python.org/packages/5f/4f/3859b52f6d465d0d4a767516c924ee4f0e1387498ac8d0c30d9942da3762/kombu-3.0.35.tar.gz";
		461	md5 = "6483ac8ba7109ec606f5cb9bd084b6ef";
		462	};
		463	meta = {
		464	license = [ pkgs.lib.licenses.bsdOriginal ];
		465	};
		466	};
		467	lxml = super.buildPythonPackage {
		468	name = "lxml-3.6.0";
		469	buildInputs = with self; [];
		470	doCheck = false;
		471	propagatedBuildInputs = with self; [];
		472	src = fetchurl {
		473	url = "https://pypi.python.org/packages/11/1b/fe6904151b37a0d6da6e60c13583945f8ce3eae8ebd0ec763ce546358947/lxml-3.6.0.tar.gz";
		474	md5 = "5957cc384bd6e83934be35c057ec03b6";
		475	};
		476	meta = {
		477	license = [ pkgs.lib.licenses.bsdOriginal ];
		478	};
		479	};
		480	markdown = super.buildPythonPackage {
		481	name = "markdown-2.5";
		482	buildInputs = with self; [];
		483	doCheck = false;
		484	propagatedBuildInputs = with self; [];
		485	src = fetchurl {
		486	url = "https://pypi.python.org/packages/16/7f/034572fbc66f76a626156c9500349f5b384ca1f38194318ddde32bc2fcb0/Markdown-2.5.zip";
		487	md5 = "053e5614f7efc06ac0fcd6954678096c";
		488	};
		489	meta = {
		490	license = [ pkgs.lib.licenses.bsdOriginal ];
		491	};
		492	};
		493	mock = super.buildPythonPackage {
		494	name = "mock-1.0.1";
		495	buildInputs = with self; [];
		496	doCheck = false;
		497	propagatedBuildInputs = with self; [];
		498	src = fetchurl {
		499	url = "https://pypi.python.org/packages/15/45/30273ee91feb60dabb8fbb2da7868520525f02cf910279b3047182feed80/mock-1.0.1.zip";
		500	md5 = "869f08d003c289a97c1a6610faf5e913";
		501	};
		502	meta = {
		503	license = [ pkgs.lib.licenses.bsdOriginal ];
		504	};
		505	};
		506	oauthlib = super.buildPythonPackage {
		507	name = "oauthlib-2.0.0";
		508	buildInputs = with self; [];
		509	doCheck = false;
		510	propagatedBuildInputs = with self; [];
		511	src = fetchurl {
		512	url = "https://pypi.python.org/packages/ce/92/7f07412a4f04e55c1e83a09c6fd48075b5df96c1dbd4078c3407c5be1dff/oauthlib-2.0.0.tar.gz";
		513	md5 = "79b83aa677fc45d1ea28deab7445b4ca";
		514	};
		515	meta = {
		516	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "OSI Approved"; } ];
		517	};
		518	};
		519	paginate = super.buildPythonPackage {
		520	name = "paginate-0.5.4";
		521	buildInputs = with self; [];
		522	doCheck = false;
		523	propagatedBuildInputs = with self; [];
		524	src = fetchurl {
		525	url = "https://pypi.python.org/packages/52/2e/2c3a5647d3f7583355743d73841d03c8b50b97983a478a8f82d3cb9f4a5f/paginate-0.5.4.tar.gz";
		526	md5 = "91fdb133f85ac73c6616feba38976e95";
		527	};
		528	meta = {
		529	license = [ pkgs.lib.licenses.mit ];
		530	};
		531	};
		532	paginate-sqlalchemy = super.buildPythonPackage {
		533	name = "paginate-sqlalchemy-0.2.0";
		534	buildInputs = with self; [];
		535	doCheck = false;
		536	propagatedBuildInputs = with self; [SQLAlchemy paginate];
		537	src = fetchurl {
		538	url = "https://pypi.python.org/packages/25/64/fe572514615971fc235e95798ae0e2ee3beeccf43272c623a0a6b082d2d6/paginate_sqlalchemy-0.2.0.tar.gz";
		539	md5 = "4ca097c4132f43cd72c6a1795b6bbb5d";
		540	};
		541	meta = {
		542	license = [ pkgs.lib.licenses.mit ];
		543	};
		544	};
		545	passlib = super.buildPythonPackage {
		546	name = "passlib-1.6.5";
		547	buildInputs = with self; [];
		548	doCheck = false;
		549	propagatedBuildInputs = with self; [];
		550	src = fetchurl {
		551	url = "https://pypi.python.org/packages/1e/59/d1a50836b29c87a1bde9442e1846aa11e1548491cbee719e51b45a623e75/passlib-1.6.5.tar.gz";
		552	md5 = "d2edd6c42cde136a538b48d90a06ad67";
		553	};
		554	meta = {
		555	license = [ pkgs.lib.licenses.bsdOriginal ];
		556	};
		557	};
		558	psutil = super.buildPythonPackage {
		559	name = "psutil-2.1.2";
		560	buildInputs = with self; [];
		561	doCheck = false;
		562	propagatedBuildInputs = with self; [];
		563	src = fetchurl {
		564	url = "https://pypi.python.org/packages/53/6a/8051b913b2f94eb00fd045fe9e14a7182b6e7f088b12c308edd7616a559b/psutil-2.1.2.tar.gz";
		565	md5 = "1969c9b3e256f5ce8fb90c5d0124233e";
		566	};
		567	meta = {
		568	license = [ pkgs.lib.licenses.bsdOriginal ];
		569	};
		570	};
		571	psycopg2 = super.buildPythonPackage {
		572	name = "psycopg2-2.6.1";
		573	buildInputs = with self; [];
		574	doCheck = false;
		575	propagatedBuildInputs = with self; [];
		576	src = fetchurl {
		577	url = "https://pypi.python.org/packages/86/fd/cc8315be63a41fe000cce20482a917e874cdc1151e62cb0141f5e55f711e/psycopg2-2.6.1.tar.gz";
		578	md5 = "842b44f8c95517ed5b792081a2370da1";
		579	};
		580	meta = {
		581	license = [ pkgs.lib.licenses.zpt21 { fullName = "GNU Library or Lesser General Public License (LGPL)"; } { fullName = "LGPL with exceptions or ZPL"; } ];
		582	};
		583	};
		584	pyasn1 = super.buildPythonPackage {
		585	name = "pyasn1-0.1.9";
		586	buildInputs = with self; [];
		587	doCheck = false;
		588	propagatedBuildInputs = with self; [];
		589	src = fetchurl {
		590	url = "https://pypi.python.org/packages/f7/83/377e3dd2e95f9020dbd0dfd3c47aaa7deebe3c68d3857a4e51917146ae8b/pyasn1-0.1.9.tar.gz";
		591	md5 = "f00a02a631d4016818659d1cc38d229a";
		592	};
		593	meta = {
		594	license = [ pkgs.lib.licenses.bsdOriginal ];
		595	};
		596	};
		597	pycparser = super.buildPythonPackage {
		598	name = "pycparser-2.14";
		599	buildInputs = with self; [];
		600	doCheck = false;
		601	propagatedBuildInputs = with self; [];
		602	src = fetchurl {
		603	url = "https://pypi.python.org/packages/6d/31/666614af3db0acf377876d48688c5d334b6e493b96d21aa7d332169bee50/pycparser-2.14.tar.gz";
		604	md5 = "a2bc8d28c923b4fe2b2c3b4b51a4f935";
		605	};
		606	meta = {
		607	license = [ pkgs.lib.licenses.bsdOriginal ];
		608	};
		609	};
		610	pyelasticsearch = super.buildPythonPackage {
		611	name = "pyelasticsearch-1.4";
		612	buildInputs = with self; [];
		613	doCheck = false;
		614	propagatedBuildInputs = with self; [certifi elasticsearch urllib3 simplejson six];
		615	src = fetchurl {
		616	url = "https://pypi.python.org/packages/2f/3a/7643cfcfc4cbdbb20ada800bbd54ac9705d0c047d7b8f8d5eeeb3047b4eb/pyelasticsearch-1.4.tar.gz";
		617	md5 = "ed61ebb7b253364e55b4923d11e17049";
		618	};
		619	meta = {
		620	license = [ pkgs.lib.licenses.bsdOriginal ];
		621	};
		622	};
		623	pygments = super.buildPythonPackage {
		624	name = "pygments-2.1.3";
		625	buildInputs = with self; [];
		626	doCheck = false;
		627	propagatedBuildInputs = with self; [];
		628	src = fetchurl {
		629	url = "https://pypi.python.org/packages/b8/67/ab177979be1c81bc99c8d0592ef22d547e70bb4c6815c383286ed5dec504/Pygments-2.1.3.tar.gz";
		630	md5 = "ed3fba2467c8afcda4d317e4ef2c6150";
		631	};
		632	meta = {
		633	license = [ pkgs.lib.licenses.bsdOriginal ];
		634	};
		635	};
		636	pyramid = super.buildPythonPackage {
		637	name = "pyramid-1.7.3";
		638	buildInputs = with self; [];
		639	doCheck = false;
		640	propagatedBuildInputs = with self; [setuptools WebOb repoze.lru zope.interface zope.deprecation venusian translationstring PasteDeploy];
		641	src = fetchurl {
		642	url = "https://pypi.python.org/packages/9c/6d/9b9f9acf22c5d221f25cf6756645bce9ea54ee741466197674fe77f2eee3/pyramid-1.7.3.tar.gz";
		643	md5 = "5f154c8c352ef013e6e412be02bbb576";
		644	};
		645	meta = {
		646	license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		647	};
		648	};
		649	pyramid-authstack = super.buildPythonPackage {
		650	name = "pyramid-authstack-1.0.1";
		651	buildInputs = with self; [];
		652	doCheck = false;
		653	propagatedBuildInputs = with self; [pyramid zope.interface];
		654	src = fetchurl {
		655	url = "https://pypi.python.org/packages/01/4b/e84cb8fda19f0f03f96231195fd074212b9291f732aa07f90edcfb21ff34/pyramid_authstack-1.0.1.tar.gz";
		656	md5 = "8e199862b5a5cd6385f7d5209cee2f12";
		657	};
		658	meta = {
		659	license = [ pkgs.lib.licenses.bsdOriginal pkgs.lib.licenses.mit ];
		660	};
		661	};
		662	pyramid-debugtoolbar = super.buildPythonPackage {
		663	name = "pyramid-debugtoolbar-3.0.4";
		664	buildInputs = with self; [];
		665	doCheck = false;
		666	propagatedBuildInputs = with self; [pyramid pyramid-mako repoze.lru pygments];
		667	src = fetchurl {
		668	url = "https://pypi.python.org/packages/b0/c5/aae5d99983600146875d471aab9142b925fd3596e6e637f6c35d158d09cc/pyramid_debugtoolbar-3.0.4.tar.gz";
		669	md5 = "51ff68a733ae994641027f10116e519d";
		670	};
		671	meta = {
		672	license = [ { fullName = "Repoze Public License"; } pkgs.lib.licenses.bsdOriginal ];
		673	};
		674	};
		675	pyramid-jinja2 = super.buildPythonPackage {
		676	name = "pyramid-jinja2-2.6.2";
		677	buildInputs = with self; [];
		678	doCheck = false;
		679	propagatedBuildInputs = with self; [pyramid zope.deprecation Jinja2 MarkupSafe];
		680	src = fetchurl {
		681	url = "https://pypi.python.org/packages/37/00/ac38702305dcf08fe1f1d6d882e8e2d957543bc96c62de52d99d43433c23/pyramid_jinja2-2.6.2.tar.gz";
		682	md5 = "10ca075934ebf8f52acfc9898991966d";
		683	};
		684	meta = {
		685	license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		686	};
		687	};
		688	pyramid-mailer = super.buildPythonPackage {
		689	name = "pyramid-mailer-0.14.1";
		690	buildInputs = with self; [];
		691	doCheck = false;
		692	propagatedBuildInputs = with self; [pyramid repoze.sendmail];
		693	src = fetchurl {
		694	url = "https://pypi.python.org/packages/43/02/a32823750dbdee4280090843d5788cc550ab6f24f23fcabbeb7f912bf5fe/pyramid_mailer-0.14.1.tar.gz";
		695	md5 = "a589801afdc4a3d64337e4cbd2fc7cdb";
		696	};
		697	meta = {
		698	license = [ pkgs.lib.licenses.bsdOriginal ];
		699	};
		700	};
		701	pyramid-mako = super.buildPythonPackage {
		702	name = "pyramid-mako-1.0.2";
		703	buildInputs = with self; [];
		704	doCheck = false;
		705	propagatedBuildInputs = with self; [pyramid Mako];
		706	src = fetchurl {
		707	url = "https://pypi.python.org/packages/f1/92/7e69bcf09676d286a71cb3bbb887b16595b96f9ba7adbdc239ffdd4b1eb9/pyramid_mako-1.0.2.tar.gz";
		708	md5 = "ee25343a97eb76bd90abdc2a774eb48a";
		709	};
		710	meta = {
		711	license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		712	};
		713	};
		714	pyramid-redis-sessions = super.buildPythonPackage {
		715	name = "pyramid-redis-sessions-1.0.1";
		716	buildInputs = with self; [];
		717	doCheck = false;
		718	propagatedBuildInputs = with self; [redis pyramid];
		719	src = fetchurl {
		720	url = "https://pypi.python.org/packages/45/9b/905fd70bb603b61819d525efe7626342ad5f8d033e25fbaedbc53f458c37/pyramid_redis_sessions-1.0.1.tar.gz";
		721	md5 = "a39bbfd36f61685eac32d5f4010d3fef";
		722	};
		723	meta = {
		724	license = [ { fullName = "FreeBSD"; } ];
		725	};
		726	};
		727	pyramid-tm = super.buildPythonPackage {
		728	name = "pyramid-tm-0.12";
		729	buildInputs = with self; [];
		730	doCheck = false;
		731	propagatedBuildInputs = with self; [pyramid transaction];
		732	src = fetchurl {
		733	url = "https://pypi.python.org/packages/3e/0b/a0fd3856c8ca2b30f20fcd26627b9cf9d91cd2cfabae42aee3441b2441c5/pyramid_tm-0.12.tar.gz";
		734	md5 = "6e5f4449706855fdb7c63d2190e0209b";
		735	};
		736	meta = {
		737	license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		738	};
		739	};
		740	python-dateutil = super.buildPythonPackage {
		741	name = "python-dateutil-2.5.3";
		742	buildInputs = with self; [];
		743	doCheck = false;
		744	propagatedBuildInputs = with self; [six];
		745	src = fetchurl {
		746	url = "https://pypi.python.org/packages/3e/f5/aad82824b369332a676a90a8c0d1e608b17e740bbb6aeeebca726f17b902/python-dateutil-2.5.3.tar.gz";
		747	md5 = "05ffc6d2cc85a7fd93bb245807f715ef";
		748	};
		749	meta = {
		750	license = [ pkgs.lib.licenses.bsdOriginal { fullName = "Simplified BSD"; } ];
		751	};
		752	};
		753	python-editor = super.buildPythonPackage {
		754	name = "python-editor-1.0.1";
		755	buildInputs = with self; [];
		756	doCheck = false;
		757	propagatedBuildInputs = with self; [];
		758	src = fetchurl {
		759	url = "https://pypi.python.org/packages/2b/c0/df7b87d5cf016f82eab3b05cd35f53287c1178ad8c42bfb6fa61b89b22f6/python-editor-1.0.1.tar.gz";
		760	md5 = "e1fa63535b40e022fa4fd646fd8b511a";
		761	};
		762	meta = {
		763	license = [ pkgs.lib.licenses.asl20 { fullName = "Apache"; } ];
		764	};
		765	};
		766	pytz = super.buildPythonPackage {
		767	name = "pytz-2016.6.1";
		768	buildInputs = with self; [];
		769	doCheck = false;
		770	propagatedBuildInputs = with self; [];
		771	src = fetchurl {
		772	url = "https://pypi.python.org/packages/5d/8e/6635d8f3f9f48c03bb925fab543383089858271f9cfd1216b83247e8df94/pytz-2016.6.1.tar.gz";
		773	md5 = "b6c28a3b968bc1d8badfb61b93874e03";
		774	};
		775	meta = {
		776	license = [ pkgs.lib.licenses.mit ];
		777	};
		778	};
		779	redis = super.buildPythonPackage {
		780	name = "redis-2.10.5";
		781	buildInputs = with self; [];
		782	doCheck = false;
		783	propagatedBuildInputs = with self; [];
		784	src = fetchurl {
		785	url = "https://pypi.python.org/packages/68/44/5efe9e98ad83ef5b742ce62a15bea609ed5a0d1caf35b79257ddb324031a/redis-2.10.5.tar.gz";
		786	md5 = "3b26c2b9703b4b56b30a1ad508e31083";
		787	};
		788	meta = {
		789	license = [ pkgs.lib.licenses.mit ];
		790	};
		791	};
		792	redlock-py = super.buildPythonPackage {
		793	name = "redlock-py-1.0.8";
		794	buildInputs = with self; [];
		795	doCheck = false;
		796	propagatedBuildInputs = with self; [redis];
		797	src = fetchurl {
		798	url = "https://pypi.python.org/packages/7c/40/29e1730f771b5d27e3c77b5426b6a67a3642868bf8bd592dfa6639feda98/redlock-py-1.0.8.tar.gz";
		799	md5 = "7f8fe8ddefbe35deaa64d67ebdf1c58e";
		800	};
		801	meta = {
		802	license = [ pkgs.lib.licenses.mit ];
		803	};
		804	};
		805	repoze.lru = super.buildPythonPackage {
		806	name = "repoze.lru-0.6";
		807	buildInputs = with self; [];
		808	doCheck = false;
		809	propagatedBuildInputs = with self; [];
		810	src = fetchurl {
		811	url = "https://pypi.python.org/packages/6e/1e/aa15cc90217e086dc8769872c8778b409812ff036bf021b15795638939e4/repoze.lru-0.6.tar.gz";
		812	md5 = "2c3b64b17a8e18b405f55d46173e14dd";
		813	};
		814	meta = {
		815	license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		816	};
		817	};
		818	repoze.sendmail = super.buildPythonPackage {
		819	name = "repoze.sendmail-4.1";
		820	buildInputs = with self; [];
		821	doCheck = false;
		822	propagatedBuildInputs = with self; [setuptools zope.interface transaction];
		823	src = fetchurl {
		824	url = "https://pypi.python.org/packages/6b/3a/501a897c036c7b728b02a2695998055755e9e71c7e135abdcf200958965e/repoze.sendmail-4.1.tar.gz";
		825	md5 = "81d15f1f03cc67d6f56f2091c594ef57";
		826	};
		827	meta = {
		828	license = [ pkgs.lib.licenses.zpt21 ];
		829	};
		830	};
		831	requests = super.buildPythonPackage {
		832	name = "requests-2.9.1";
		833	buildInputs = with self; [];
		834	doCheck = false;
		835	propagatedBuildInputs = with self; [];
		836	src = fetchurl {
		837	url = "https://pypi.python.org/packages/f9/6d/07c44fb1ebe04d069459a189e7dab9e4abfe9432adcd4477367c25332748/requests-2.9.1.tar.gz";
		838	md5 = "0b7f480d19012ec52bab78292efd976d";
		839	};
		840	meta = {
		841	license = [ pkgs.lib.licenses.asl20 ];
		842	};
		843	};
		844	requests-oauthlib = super.buildPythonPackage {
		845	name = "requests-oauthlib-0.6.1";
		846	buildInputs = with self; [];
		847	doCheck = false;
		848	propagatedBuildInputs = with self; [oauthlib requests];
		849	src = fetchurl {
		850	url = "https://pypi.python.org/packages/f9/98/a1aaae4bbcde0e98d6d853c4f08bd52f20b0005cefb881679bcdf7ea7a00/requests-oauthlib-0.6.1.tar.gz";
		851	md5 = "f159bc7675ebe6a2d76798f4c00c5bf8";
		852	};
		853	meta = {
		854	license = [ pkgs.lib.licenses.isc pkgs.lib.licenses.bsdOriginal ];
		855	};
		856	};
		857	requests-toolbelt = super.buildPythonPackage {
		858	name = "requests-toolbelt-0.7.0";
		859	buildInputs = with self; [];
		860	doCheck = false;
		861	propagatedBuildInputs = with self; [requests];
		862	src = fetchurl {
		863	url = "https://pypi.python.org/packages/59/78/1d391d30ebf74079a8e4de6ab66fdca5362903ef2df64496f4697e9bb626/requests-toolbelt-0.7.0.tar.gz";
		864	md5 = "bfe2009905f460f4764c32cfbbf4205f";
		865	};
		866	meta = {
		867	license = [ pkgs.lib.licenses.asl20 ];
		868	};
		869	};
		870	setuptools = super.buildPythonPackage {
		871	name = "setuptools-27.2.0";
		872	buildInputs = with self; [];
		873	doCheck = false;
		874	propagatedBuildInputs = with self; [];
		875	src = fetchurl {
		876	url = "https://pypi.python.org/packages/87/ba/54197971d107bc06f5f3fbdc0d728a7ae0b10cafca46acfddba65a0899d8/setuptools-27.2.0.tar.gz";
		877	md5 = "b39715612fdc0372dbfd7b3fcf5d4fe5";
		878	};
		879	meta = {
		880	license = [ pkgs.lib.licenses.mit ];
		881	};
		882	};
		883	simplejson = super.buildPythonPackage {
		884	name = "simplejson-3.8.2";
		885	buildInputs = with self; [];
		886	doCheck = false;
		887	propagatedBuildInputs = with self; [];
		888	src = fetchurl {
		889	url = "https://pypi.python.org/packages/f0/07/26b519e6ebb03c2a74989f7571e6ae6b82e9d7d81b8de6fcdbfc643c7b58/simplejson-3.8.2.tar.gz";
		890	md5 = "53b1371bbf883b129a12d594a97e9a18";
		891	};
		892	meta = {
		893	license = [ { fullName = "Academic Free License (AFL)"; } pkgs.lib.licenses.mit ];
		894	};
		895	};
		896	six = super.buildPythonPackage {
		897	name = "six-1.9.0";
		898	buildInputs = with self; [];
		899	doCheck = false;
		900	propagatedBuildInputs = with self; [];
		901	src = fetchurl {
		902	url = "https://pypi.python.org/packages/16/64/1dc5e5976b17466fd7d712e59cbe9fb1e18bec153109e5ba3ed6c9102f1a/six-1.9.0.tar.gz";
		903	md5 = "476881ef4012262dfc8adc645ee786c4";
		904	};
		905	meta = {
		906	license = [ pkgs.lib.licenses.mit ];
		907	};
		908	};
		909	tlslite = super.buildPythonPackage {
		910	name = "tlslite-0.4.9";
		911	buildInputs = with self; [];
		912	doCheck = false;
		913	propagatedBuildInputs = with self; [];
		914	src = fetchurl {
		915	url = "https://pypi.python.org/packages/92/2b/7904cf913d9bf150b3e408a92c9cb5ce0b97a9ec19f998af48bf4c607f0e/tlslite-0.4.9.tar.gz";
		916	md5 = "9f3b3797f595dd66cd36a65c83a87869";
		917	};
		918	meta = {
		919	license = [ { fullName = "public domain and BSD"; } ];
		920	};
		921	};
		922	transaction = super.buildPythonPackage {
		923	name = "transaction-1.4.3";
		924	buildInputs = with self; [];
		925	doCheck = false;
		926	propagatedBuildInputs = with self; [zope.interface];
		927	src = fetchurl {
		928	url = "https://pypi.python.org/packages/9d/9d/afb5c4904fb41edc14029744ff030ac0596846262bda6145edf23791c880/transaction-1.4.3.tar.gz";
		929	md5 = "b4ca5983c9e3a0808ff5ff7648092c76";
		930	};
		931	meta = {
		932	license = [ pkgs.lib.licenses.zpt21 ];
		933	};
		934	};
		935	translationstring = super.buildPythonPackage {
		936	name = "translationstring-1.3";
		937	buildInputs = with self; [];
		938	doCheck = false;
		939	propagatedBuildInputs = with self; [];
		940	src = fetchurl {
		941	url = "https://pypi.python.org/packages/5e/eb/bee578cc150b44c653b63f5ebe258b5d0d812ddac12497e5f80fcad5d0b4/translationstring-1.3.tar.gz";
		942	md5 = "a4b62e0f3c189c783a1685b3027f7c90";
		943	};
		944	meta = {
		945	license = [ { fullName = "BSD-like (http://repoze.org/license.html)"; } ];
		946	};
		947	};
		948	urllib3 = super.buildPythonPackage {
		949	name = "urllib3-1.17";
		950	buildInputs = with self; [];
		951	doCheck = false;
		952	propagatedBuildInputs = with self; [];
		953	src = fetchurl {
		954	url = "https://pypi.python.org/packages/c2/79/8851583070bac203561d21b9478340535893f587759608156aaca60a615a/urllib3-1.17.tar.gz";
		955	md5 = "12d5520f0fffed0e65cb66b5bdc6ddec";
		956	};
		957	meta = {
		958	license = [ pkgs.lib.licenses.mit ];
		959	};
		960	};
		961	venusian = super.buildPythonPackage {
		962	name = "venusian-1.0";
		963	buildInputs = with self; [];
		964	doCheck = false;
		965	propagatedBuildInputs = with self; [];
		966	src = fetchurl {
		967	url = "https://pypi.python.org/packages/86/20/1948e0dfc4930ddde3da8c33612f6a5717c0b4bc28f591a5c5cf014dd390/venusian-1.0.tar.gz";
		968	md5 = "dccf2eafb7113759d60c86faf5538756";
		969	};
		970	meta = {
		971	license = [ { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
		972	};
		973	};
		974	waitress = super.buildPythonPackage {
		975	name = "waitress-1.0.0";
		976	buildInputs = with self; [];
		977	doCheck = false;
		978	propagatedBuildInputs = with self; [];
		979	src = fetchurl {
		980	url = "https://pypi.python.org/packages/a5/c3/264a56b2470de29f35dda8369886663303c8a2294673b2e6b9975e59f471/waitress-1.0.0.tar.gz";
		981	md5 = "b900c4d793e218d77742f47ece58dd43";
		982	};
		983	meta = {
		984	license = [ pkgs.lib.licenses.zpt21 ];
		985	};
		986	};
		987	webassets = super.buildPythonPackage {
		988	name = "webassets-0.11.1";
		989	buildInputs = with self; [];
		990	doCheck = false;
		991	propagatedBuildInputs = with self; [];
		992	src = fetchurl {
		993	url = "https://pypi.python.org/packages/0e/97/f0cd013a3ae074672e9fdfa8629e4071b5cc420a2c82bef5622a87631d1c/webassets-0.11.1.tar.gz";
		994	md5 = "6acca51bd12fbdc0399ab1a9b67a1599";
		995	};
		996	meta = {
		997	license = [ pkgs.lib.licenses.bsdOriginal ];
		998	};
		999	};
		1000	webhelpers2 = super.buildPythonPackage {
		1001	name = "webhelpers2-2.0";
		1002	buildInputs = with self; [];
		1003	doCheck = false;
		1004	propagatedBuildInputs = with self; [MarkupSafe six];
		1005	src = fetchurl {
		1006	url = "https://pypi.python.org/packages/ff/30/56342c6ea522439e3662427c8d7b5e5b390dff4ff2dc92d8afcb8ab68b75/WebHelpers2-2.0.tar.gz";
		1007	md5 = "0f6b68d70c12ee0aed48c00b24da13d3";
		1008	};
		1009	meta = {
		1010	license = [ pkgs.lib.licenses.mit ];
		1011	};
		1012	};
		1013	wtforms = super.buildPythonPackage {
		1014	name = "wtforms-2.1";
		1015	buildInputs = with self; [];
		1016	doCheck = false;
		1017	propagatedBuildInputs = with self; [];
		1018	src = fetchurl {
		1019	url = "https://pypi.python.org/packages/bf/91/2e553b86c55e9cf2f33265de50e052441fb753af46f5f20477fe9c61280e/WTForms-2.1.zip";
		1020	md5 = "6938a541fafd1a1ae2f6b9b88588eef2";
		1021	};
		1022	meta = {
		1023	license = [ pkgs.lib.licenses.bsdOriginal ];
		1024	};
		1025	};
		1026	ziggurat-foundations = super.buildPythonPackage {
		1027	name = "ziggurat-foundations-0.6.8";
		1028	buildInputs = with self; [];
		1029	doCheck = false;
		1030	propagatedBuildInputs = with self; [SQLAlchemy passlib paginate paginate-sqlalchemy alembic six];
		1031	src = fetchurl {
		1032	url = "https://pypi.python.org/packages/b2/3c/f9a0112a30424a58fccdd357338b4559fdda9e1bb3c9611b1ad263abf49e/ziggurat_foundations-0.6.8.tar.gz";
		1033	md5 = "d2cc7201667b0e01099456a77726179c";
		1034	};
		1035	meta = {
		1036	license = [ pkgs.lib.licenses.bsdOriginal ];
		1037	};
		1038	};
		1039	zope.deprecation = super.buildPythonPackage {
		1040	name = "zope.deprecation-4.1.2";
		1041	buildInputs = with self; [];
		1042	doCheck = false;
		1043	propagatedBuildInputs = with self; [setuptools];
		1044	src = fetchurl {
		1045	url = "https://pypi.python.org/packages/c1/d3/3919492d5e57d8dd01b36f30b34fc8404a30577392b1eb817c303499ad20/zope.deprecation-4.1.2.tar.gz";
		1046	md5 = "e9a663ded58f4f9f7881beb56cae2782";
		1047	};
		1048	meta = {
		1049	license = [ pkgs.lib.licenses.zpt21 ];
		1050	};
		1051	};
		1052	zope.interface = super.buildPythonPackage {
		1053	name = "zope.interface-4.3.2";
		1054	buildInputs = with self; [];
		1055	doCheck = false;
		1056	propagatedBuildInputs = with self; [setuptools];
		1057	src = fetchurl {
		1058	url = "https://pypi.python.org/packages/38/1b/d55c39f2cf442bd9fb2c59760ed058c84b57d25c680819c25f3aff741e1f/zope.interface-4.3.2.tar.gz";
		1059	md5 = "5f7e15a5bcdfa3c6c0e93ffe45caf87c";
		1060	};
		1061	meta = {
		1062	license = [ pkgs.lib.licenses.zpt21 ];
		1063	};
		1064	};
		1065	zope.sqlalchemy = super.buildPythonPackage {
		1066	name = "zope.sqlalchemy-0.7.6";
		1067	buildInputs = with self; [];
		1068	doCheck = false;
		1069	propagatedBuildInputs = with self; [setuptools SQLAlchemy transaction zope.interface];
		1070	src = fetchurl {
		1071	url = "https://pypi.python.org/packages/d0/e0/5df0d7f9f1955e2e2edecbb1367cf1fa76bc2f84d700661ffd4161c7e2e9/zope.sqlalchemy-0.7.6.zip";
		1072	md5 = "0f5bf14679951e59007e090b6922688c";
		1073	};
		1074	meta = {
		1075	license = [ pkgs.lib.licenses.zpt21 ];
		1076	};
		1077	};
		1078
		1079	### Test requirements
		1080
		1081
		1082	}

backend/src/appenlight/VERSION

0 created 644 +1 0

	@@ -0,0 +1,1 b''
				1	1.1.0

frontend/bower-packages.nix

0 created 644 0 0

	1	NO CONTENT: new file 100644			NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff

frontend/default.nix

0 created 644 0 0

	1	NO CONTENT: new file 100644			NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff

frontend/node-env.nix

0 created 644 0 0

	1	NO CONTENT: new file 100644			NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff

frontend/node-packages.nix

0 created 644 0 0

	1	NO CONTENT: new file 100644			NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff

.gitignore

0 +2 -2

             # Created by .ignore support plugin (hsz.mobi)
             ### Node template
             # Logs
             logs
             *.log
             npm-debug.log*
             yarn-debug.log*
             yarn-error.log*
             # Runtime data
             pids
             *.pid
             *.seed
             *.pid.lock
             # Directory for instrumented libs generated by jscoverage/JSCover
             lib-cov
             # Coverage directory used by tools like istanbul
             coverage
             # nyc test coverage
             .nyc_output
             # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
             .grunt
             # Bower dependency directory (https://bower.io/)
             bower_components
             # node-waf configuration
             .lock-wscript
             # Compiled binary addons (https://nodejs.org/api/addons.html)
             build/Release
             # Dependency directories
             node_modules/
             jspm_packages/
             # Typescript v1 declaration files
             typings/
             # Optional npm cache directory
             .npm
             # Optional eslint cache
             .eslintcache
             # Optional REPL history
             .node_repl_history
             # Output of 'npm pack'
             *.tgz
             # Yarn Integrity file
             .yarn-integrity
             # dotenv environment variables file
             .env
             ### JetBrains template
             # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
             # Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
             # User-specific stuff:
             .idea/**/workspace.xml
             .idea/**/tasks.xml
             .idea/dictionaries
             # Sensitive or high-churn files:
             .idea/**/dataSources/
             .idea/**/dataSources.ids
             .idea/**/dataSources.xml
             .idea/**/dataSources.local.xml
             .idea/**/sqlDataSources.xml
             .idea/**/dynamic.xml
             .idea/**/uiDesigner.xml
             # Gradle:
             .idea/**/gradle.xml
             .idea/**/libraries
             # CMake
             cmake-build-debug/
             # Mongo Explorer plugin:
             .idea/**/mongoSettings.xml
             ## File-based project format:
             *.iws
             ## Plugin-specific files:
             # IntelliJ
             out/
             # mpeltonen/sbt-idea plugin
             .idea_modules/
             # JIRA plugin
             atlassian-ide-plugin.xml
             # Cursive Clojure plugin
             .idea/replstate.xml
             # Crashlytics plugin (for Android Studio and IntelliJ)
             com_crashlytics_export_strings.xml
             crashlytics.properties
             crashlytics-build.properties
             fabric.properties
             ### Python template
             # Byte-compiled / optimized / DLL files
             __pycache__/
             *.py[cod]
             *$py.class
             # C extensions
             *.so
             # Distribution / packaging
             .Python
             build/
             develop-eggs/
             dist/
             downloads/
             eggs/
             .eggs/
-            /lib/
+            lib/
-            /lib64/
+            lib64/
             parts/
             sdist/
             var/
             wheels/
             *.egg-info/
             .installed.cfg
             *.egg
             MANIFEST
             # PyInstaller
             #  Usually these files are written by a python script from a template
             #  before PyInstaller builds the exe, so as to inject date/other infos into it.
             *.manifest
             *.spec
             # Installer logs
             pip-log.txt
             pip-delete-this-directory.txt
             # Unit test / coverage reports
             htmlcov/
             .tox/
             .coverage
             .coverage.*
             .cache
             nosetests.xml
             coverage.xml
             *.cover
             .hypothesis/
             # Translations
             *.mo
             *.pot
             # Django stuff:
             local_settings.py
             # Flask stuff:
             instance/
             .webassets-cache
             # Scrapy stuff:
             .scrapy
             # Sphinx documentation
             docs/_build/
             # PyBuilder
             target/
             # Jupyter Notebook
             .ipynb_checkpoints
             # pyenv
             .python-version
             # celery beat schedule file
             celerybeat-schedule
             # SageMath parsed files
             *.sage.py
             # Environments
             .venv
             env/
             venv/
             ENV/
             env.bak/
             venv.bak/
             # Spyder project settings
             .spyderproject
             .spyproject
             # Rope project settings
             .ropeproject
             # mkdocs documentation
             /site
             # mypy
             .mypy_cache/
             ### Example user template template
             ### Example user template
             # IntelliJ project files
             .idea
             *.iml
             out
             gen

LICENSE.txt

0 +5 0

+            This program is free software: you can redistribute it and/or modify
+            it under the terms of the GNU Affero General Public License, version 3
+            (only), as published by the Free Software Foundation.
             This program incorporates work covered by the following copyright and
             permission notice:
                Copyright (c) 2014-2016 - packaging
                    file:
                Copyright (c) 2008-2011 - msgpack-python
                    file:licenses/msgpack_license.txt
                Copyright (c) 2007-2008 - amqp
                    file:licenses/amqp_license.txt
                Copyright (c) 2013 - bcrypt
                    file:licenses/bcrypt_license.txt
                Copyright (c) 2015 - elasticsearch
                    file:licenses/elasticsearch_license.txt
                Copyright (c) 2011-2013 - gevent-websocket
                    file:licenses/gevent_websocket_license.txt
                Copyright (c) 2015 - python-editor
                    file:licenses/python_editor_license.txt
                Copyright (c) 2015 - requests
                    file:licenses/requests_license.txt
                Copyright (c) 2014 - requests-toolbelt
                    file:licenses/requests_toolbelt_license.txt
             Both licensed under the Apache License, Version 2.0 (the "License");
             you may not use this file except in compliance with the License.
             You may obtain a copy of the License at
                 http://www.apache.org/licenses/LICENSE-2.0
             Unless required by applicable law or agreed to in writing, software
             distributed under the License is distributed on an "AS IS" BASIS,
             WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             See the License for the specific language governing permissions and
             imitations under the License.
             Below is the full text of Apache License, version 2.0
                                              Apache License
                                        Version 2.0, January 2004
                                     http://www.apache.org/licenses/
                TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 . Definitions.
                   "License" shall mean the terms and conditions for use, reproduction,
                   and distribution as defined by Sections 1 through 9 of this document.
                   "Licensor" shall mean the copyright owner or entity authorized by
                   the copyright owner that is granting the License.
                   "Legal Entity" shall mean the union of the acting entity and all
                   other entities that control, are controlled by, or are under common
                   control with that entity. For the purposes of this definition,
                   "control" means (i) the power, direct or indirect, to cause the
                   direction or management of such entity, whether by contract or
                   otherwise, or (ii) ownership of fifty percent (50%) or more of the
                   outstanding shares, or (iii) beneficial ownership of such entity.
                   "You" (or "Your") shall mean an individual or Legal Entity
                   exercising permissions granted by this License.
                   "Source" form shall mean the preferred form for making modifications,
                   including but not limited to software source code, documentation
                   source, and configuration files.
                   "Object" form shall mean any form resulting from mechanical
                   transformation or translation of a Source form, including but
                   not limited to compiled object code, generated documentation,
                   and conversions to other media types.
                   "Work" shall mean the work of authorship, whether in Source or
                   Object form, made available under the License, as indicated by a
                   copyright notice that is included in or attached to the work
                   (an example is provided in the Appendix below).
                   "Derivative Works" shall mean any work, whether in Source or Object
                   form, that is based on (or derived from) the Work and for which the
                   editorial revisions, annotations, elaborations, or other modifications
                   represent, as a whole, an original work of authorship. For the purposes
                   of this License, Derivative Works shall not include works that remain
                   separable from, or merely link (or bind by name) to the interfaces of,
                   the Work and Derivative Works thereof.
                   "Contribution" shall mean any work of authorship, including
                   the original version of the Work and any modifications or additions
                   to that Work or Derivative Works thereof, that is intentionally
                   submitted to Licensor for inclusion in the Work by the copyright owner
                   or by an individual or Legal Entity authorized to submit on behalf of
                   the copyright owner. For the purposes of this definition, "submitted"
                   means any form of electronic, verbal, or written communication sent
                   to the Licensor or its representatives, including but not limited to
                   communication on electronic mailing lists, source code control systems,
                   and issue tracking systems that are managed by, or on behalf of, the
                   Licensor for the purpose of discussing and improving the Work, but
                   excluding communication that is conspicuously marked or otherwise
                   designated in writing by the copyright owner as "Not a Contribution."
                   "Contributor" shall mean Licensor and any individual or Legal Entity
                   on behalf of whom a Contribution has been received by Licensor and
                   subsequently incorporated within the Work.
 . Grant of Copyright License. Subject to the terms and conditions of
                   this License, each Contributor hereby grants to You a perpetual,
                   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
                   copyright license to reproduce, prepare Derivative Works of,
                   publicly display, publicly perform, sublicense, and distribute the
                   Work and such Derivative Works in Source or Object form.
 . Grant of Patent License. Subject to the terms and conditions of
                   this License, each Contributor hereby grants to You a perpetual,
                   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
                   (except as stated in this section) patent license to make, have made,
                   use, offer to sell, sell, import, and otherwise transfer the Work,
                   where such license applies only to those patent claims licensable
                   by such Contributor that are necessarily infringed by their
                   Contribution(s) alone or by combination of their Contribution(s)
                   with the Work to which such Contribution(s) was submitted. If You
                   institute patent litigation against any entity (including a
                   cross-claim or counterclaim in a lawsuit) alleging that the Work
                   or a Contribution incorporated within the Work constitutes direct
                   or contributory patent infringement, then any patent licenses
                   granted to You under this License for that Work shall terminate
                   as of the date such litigation is filed.
 . Redistribution. You may reproduce and distribute copies of the
                   Work or Derivative Works thereof in any medium, with or without
                   modifications, and in Source or Object form, provided that You
                   meet the following conditions:
                   (a) You must give any other recipients of the Work or
                       Derivative Works a copy of this License; and
                   (b) You must cause any modified files to carry prominent notices
                       stating that You changed the files; and
                   (c) You must retain, in the Source form of any Derivative Works
                       that You distribute, all copyright, patent, trademark, and
                       attribution notices from the Source form of the Work,
                       excluding those notices that do not pertain to any part of
                       the Derivative Works; and
                   (d) If the Work includes a "NOTICE" text file as part of its
                       distribution, then any Derivative Works that You distribute must
                       include a readable copy of the attribution notices contained
                       within such NOTICE file, excluding those notices that do not
                       pertain to any part of the Derivative Works, in at least one
                       of the following places: within a NOTICE text file distributed
                       as part of the Derivative Works; within the Source form or
                       documentation, if provided along with the Derivative Works; or,
                       within a display generated by the Derivative Works, if and
                       wherever such third-party notices normally appear. The contents
                       of the NOTICE file are for informational purposes only and
                       do not modify the License. You may add Your own attribution
                       notices within Derivative Works that You distribute, alongside
                       or as an addendum to the NOTICE text from the Work, provided
                       that such additional attribution notices cannot be construed
                       as modifying the License.
                   You may add Your own copyright statement to Your modifications and
                   may provide additional or different license terms and conditions
                   for use, reproduction, or distribution of Your modifications, or
                   for any such Derivative Works as a whole, provided Your use,
                   reproduction, and distribution of the Work otherwise complies with
                   the conditions stated in this License.
 . Submission of Contributions. Unless You explicitly state otherwise,
                   any Contribution intentionally submitted for inclusion in the Work
                   by You to the Licensor shall be under the terms and conditions of
                   this License, without any additional terms or conditions.
                   Notwithstanding the above, nothing herein shall supersede or modify
                   the terms of any separate license agreement you may have executed
                   with Licensor regarding such Contributions.
 . Trademarks. This License does not grant permission to use the trade
                   names, trademarks, service marks, or product names of the Licensor,
                   except as required for reasonable and customary use in describing the
                   origin of the Work and reproducing the content of the NOTICE file.
 . Disclaimer of Warranty. Unless required by applicable law or
                   agreed to in writing, Licensor provides the Work (and each
                   Contributor provides its Contributions) on an "AS IS" BASIS,
                   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
                   implied, including, without limitation, any warranties or conditions
                   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
                   PARTICULAR PURPOSE. You are solely responsible for determining the
                   appropriateness of using or redistributing the Work and assume any
                   risks associated with Your exercise of permissions under this License.
 . Limitation of Liability. In no event and under no legal theory,
                   whether in tort (including negligence), contract, or otherwise,
                   unless required by applicable law (such as deliberate and grossly
                   negligent acts) or agreed to in writing, shall any Contributor be
                   liable to You for damages, including any direct, indirect, special,
                   incidental, or consequential damages of any character arising as a
                   result of this License or out of the use or inability to use the
                   Work (including but not limited to damages for loss of goodwill,
                   work stoppage, computer failure or malfunction, or any and all
                   other commercial damages or losses), even if such Contributor
                   has been advised of the possibility of such damages.
 . Accepting Warranty or Additional Liability. While redistributing
                   the Work or Derivative Works thereof, You may choose to offer,
                   and charge a fee for, acceptance of support, warranty, indemnity,
                   or other liability obligations and/or rights consistent with this
                   License. However, in accepting such obligations, You may act only
                   on Your own behalf and on Your sole responsibility, not on behalf
                   of any other Contributor, and only if You agree to indemnify,
                   defend, and hold each Contributor harmless for any liability
                   incurred by, or claims asserted against, such Contributor by reason
                   of your accepting any such warranty or additional liability.
                END OF TERMS AND CONDITIONS

README.md

0 +95 -5

@@ -1,9 +1,99 b''
1	# AppEnlight	1	AppEnlight
		2	-----------
2		3
3	Performance, exception, and uptime monitoring for the Web	4	Automatic Installation
		5	======================
4		6
5	![AppEnlight image](https://raw.githubusercontent.com/AppEnlight/appenlight/gh-pages/static/appenlight.png)	7	Use the ansible scripts in the `automation` repository to build complete instance of application
		8	You can also use `packer` files in `automation/packer` to create whole VM's for KVM and VMWare.
6		9
7	Visit:	10	Manual Installation
		11	===================
8		12
9	[Readme moved to backend directory](backend/README.md)	13	To run the app you need to have meet prerequsites:
		14
		15	- python 3.5+
		16	- running elasticsearch (2.3+/2.4 tested)
		17	- running postgresql (9.5+ required)
		18	- running redis
		19
		20	Install the app by performing
		21
		22	pip install -r requirements.txt
		23
		24	python setup.py develop
		25
		26	Install the appenlight uptime plugin (`ae_uptime_ce` package from `appenlight-uptime-ce` repository).
		27
		28	After installing the application you need to perform following steps:
		29
		30	1. (optional) generate production.ini (or use a copy of development.ini)
		31
		32
		33	appenlight-make-config production.ini
		34
		35	2. Setup database structure:
		36
		37
		38	appenlight-migratedb -c FILENAME.ini
		39
		40	3. To configure elasticsearch:
		41
		42
		43	appenlight-reindex-elasticsearch -t all -c FILENAME.ini
		44
		45	4. Create base database objects
		46
		47	(run this command with help flag to see how to create administrator user)
		48
		49
		50	appenlight-initializedb -c FILENAME.ini
		51
		52	5. Generate static assets
		53
		54
		55	appenlight-static -c FILENAME.ini
		56
		57	Running application
		58	===================
		59
		60	To run the main app:
		61
		62	pserve development.ini
		63
		64	To run celery workers:
		65
		66	celery worker -A appenlight.celery -Q "reports,logs,metrics,default" --ini FILENAME.ini
		67
		68	To run celery beat:
		69
		70	celery beat -A appenlight.celery --ini FILENAME.ini
		71
		72	To run appenlight's uptime plugin:
		73
		74	appenlight-uptime-monitor -c FILENAME.ini
		75
		76	Real-time Notifications
		77	=======================
		78
		79	You should also run the `channelstream websocket server for real-time notifications
		80
		81	channelstream -i filename.ini
		82
		83	Testing
		84	=======
		85
		86	To run test suite:
		87
		88	py.test appenlight/tests/tests.py --cov appenlight (this looks for testing.ini in repo root)
		89
		90
		91	Development
		92	===========
		93
		94	To develop appenlight frontend:
		95
		96	cd frontend
		97	npm install
		98	bower install
		99	grunt watch

backend/MANIFEST.in

0 +2 -2

@@ -1,2 +1,2 b''
1	include .txt .ini .cfg .rst *.md ~~VERSION~~	1	include .txt .ini .cfg .rst *.md
2	recursive-include ~~src~~ .ico .png .css .gif .jpg .pt .txt .mak .mako .js .html .xml .jinja2 .rst .otf .ttf .svg .woff .woff2 .eot	2	recursive-include appenlight .ico .png .css .gif .jpg .pt .txt .mak .mako .js .html .xml .jinja2 .rst .otf .ttf .svg .woff *.eot

backend/requirements.txt

0 +37 -35

@@ -1,47 +1,49 b''
1	repoze.sendmail==4.4.1	1	repoze.sendmail==4.1
2	pyramid==1.~~10.2~~	2	pyramid==1.7.3
3	pyramid_tm==~~2.2.1~~	3	pyramid_tm==0.12
4	pyramid_debugtoolbar	4	pyramid_debugtoolbar
5	pyramid_authstack==1.0.1	5	pyramid_authstack==1.0.1
6	SQLAlchemy==1.~~2.18~~	6	SQLAlchemy==1.0.12
7	alembic==1.0.8	7	alembic==0.8.6
8	webhelpers2==2.0	8	webhelpers2==2.0
9	transaction==~~2.4.0~~	9	transaction==1.4.3
10	zope.sqlalchemy==~~1.1~~	10	zope.sqlalchemy==0.7.6
11	pyramid_mailer==0.15.1	11	pyramid_mailer==0.14.1
12	redis==3.2.1	12	redis==2.10.5
13	redlock-py==1.0.8	13	redlock-py==1.0.8
14	pyramid_jinja2==2.8	14	pyramid_jinja2==2.6.2
15	psycopg2~~-binary==2.7.7~~	15	psycopg2==2.6.1
16	wtforms==2.2.1	16	wtforms==2.1
17	celery==~~4.2.1~~	17	celery==3.1.23
18	formencode==1.3.1	18	formencode==1.3.0
19	psutil==~~5.6.1~~	19	psutil==2.1.2
20	ziggurat_foundations~~==0.8.3~~	20	ziggurat_foundations>=0.6.7
21	bcrypt==~~3.1.6~~	21	bcrypt==2.0.0
22	appenlight_client	22	appenlight_client
23	markdown==~~3.0.1~~	23	markdown==2.5
24	colander==1.7	24	colander==1.2
25	defusedxml==0.~~5.0~~	25	defusedxml==0.4.1
26	dogpile.cache==0.~~7.1~~	26	dogpile.cache==0.5.7
27	pyramid_redis_sessions==1.0.1	27	pyramid_redis_sessions==1.0.1
28	simplejson==3.~~16.0~~	28	simplejson==3.8.2
29	waitress==1.~~2.1~~	29	waitress==1.0
30	gunicorn==19.~~9.0~~	30	gunicorn==19.4.5
31	uwsgi==2.0.18	31	requests==2.9.1
32	requests~~==2.21.0~~	32	requests_oauthlib==0.6.1
33	requests_oauthlib==1.2.0	33	gevent==1.1.1
34	gevent==1.4.0	34	gevent-websocket==0.9.5
35	pygments==2.3.1	35	pygments==2.1.3
36	lxml==~~4.3.2~~	36	lxml==3.6.0
37	paginate==0.5.6	37	paginate==0.5.4
38	paginate-sqlalchemy==0.3.0	38	paginate-sqlalchemy==0.2.0
39	elasticsearch~~>=6.0.0,<7.0.0~~	39	pyelasticsearch==1.4
		40	six==1.9.0
40	mock==1.0.1	41	mock==1.0.1
41	itsdangerous==~~1.1.0~~	42	itsdangerous==0.24
42	camplight==0.9.6	43	camplight==0.9.6
43	jira==1.0.7	44	jira==1.0.7
44	python-dateutil==2.5.3	45	python-dateutil==2.5.3
45	authomatic==0.1.0.post1	46	authomatic==0.1.0.post1
46	cryptography==~~2.6.1~~	47	cryptography==1.2.3
		48	webassets==0.11.1
47		49

backend/setup.py

0 +35 -57

@@ -1,99 +1,77 b''
1	import os	1	import os
		2	import sys
2	import re	3	import re
3		4
4	from setuptools import setup, find_packages	5	from setuptools import setup, find_packages
5		6
6	here = os.path.abspath(os.path.dirname(__file__))	7	here = os.path.abspath(os.path.dirname(__file__))
7	README = open(os.path.join(here, "README.md")).read()	8	README = open(os.path.join(here, '..', 'README.md')).read()
8	CHANGES = open(os.path.join(here, ~~"CHANGELOG.md"~~)).read()	9	CHANGES = open(os.path.join(here, 'CHANGELOG.rst')).read()
9		10
10	REQUIREMENTS = open(os.path.join(here, "requirements.txt")).readlines()	11	REQUIREMENTS = open(os.path.join(here, 'requirements.txt')).readlines()
11		12
12	compiled = re.compile("([^=><]).")	13	compiled = re.compile('([^=><]).')
13		14
14		15
15	def parse_req(req):	16	def parse_req(req):
16	return compiled.search(req).group(1).strip()	17	return compiled.search(req).group(1).strip()
17		18
18		19
19	if "APPENLIGHT_DEVELOP" in os.environ:
20	requires = [_f for _f in map(parse_req, REQUIREMENTS) if _f]	20	requires = [_f for _f in map(parse_req, REQUIREMENTS) if _f]
21	else:
22	requires = REQUIREMENTS
23		21
24		22
25	def _get_meta_var(name, data, callback_handler=None):	23	def _get_meta_var(name, data, callback_handler=None):
26	import re	24	import re
27		25	matches = re.compile(r'(?:%s)\s=\s(.*)' % name).search(data)
28	matches = re.compile(r"(?:%s)\s=\s(.*)" % name).search(data)
29	if matches:	26	if matches:
30	if not callable(callback_handler):	27	if not callable(callback_handler):
31	callback_handler = lambda v: v	28	callback_handler = lambda v: v
32		29
33	return callback_handler(eval(matches.groups()[0]))	30	return callback_handler(eval(matches.groups()[0]))
34		31
35		32	with open(os.path.join(here, 'src', 'appenlight', '__init__.py'), 'r') as _meta:
36	with open(os.path.join(here, "src", "appenlight", "__init__.py"), "r") as _meta:
37	_metadata = _meta.read()	33	_metadata = _meta.read()
38		34
39	__license__ = _get_meta_var("__license__", _metadata)	35	with open(os.path.join('src', 'appenlight', 'VERSION')) as _meta_version:
40	__author__ = _get_meta_var("__author__", _metadata)	36	__version__ = _meta_version.read().strip()
41	__url__ = _get_meta_var("__url__", _metadata)
42		37
43	found_packages = find_packages("src")	38	__license__ = _get_meta_var('__license__', _metadata)
44	found_packages.append("appenlight.migrations")	39	__author__ = _get_meta_var('__author__', _metadata)
45	found_packages.append("appenlight.migrations.versions")	40	__url__ = _get_meta_var('__url__', _metadata)
46	setup(	41
47	name="appenlight",	42	found_packages = find_packages('src')
48	description="appenlight",	43	found_packages.append('appenlight.migrations.versions')
49	long_description=README,	44	setup(name='appenlight',
		45	description='appenlight',
		46	long_description=README + '\n\n' + CHANGES,
50	classifiers=[	47	classifiers=[
51	"Framework :: Pyramid",
52	"License :: OSI Approved :: Apache Software License",
53	"Programming Language :: Python",	48	"Programming Language :: Python",
54	"Programming Language :: Python :: 3 :: Only",	49	"Framework :: Pylons",
55	"Programming Language :: Python :: 3.6",
56	"Topic :: System :: Monitoring",
57	"Topic :: Software Development",
58	"Topic :: Software Development :: Bug Tracking",
59	"Topic :: Internet :: Log Analysis",
60	"Topic :: Internet :: WWW/HTTP",	50	"Topic :: Internet :: WWW/HTTP",
61	"Topic :: Internet :: WWW/HTTP :: WSGI :: Application",	51	"Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
62	],	52	],
63	version="2.0.0rc1",	53	version=__version__,
64	license=__license__,	54	license=__license__,
65	author=__author__,	55	author=__author__,
66	url="https://github.com/AppEnlight/appenlight",	56	url=__url__,
67	keywords="web wsgi bfg pylons pyramid flask django monitoring apm instrumentation appenlight",	57	keywords='web wsgi bfg pylons pyramid',
68	python_requires=">=3.5",	58	package_dir={'': 'src'},
69	long_description_content_type="text/markdown",
70	package_dir={"": "src"},
71	packages=found_packages,	59	packages=found_packages,
72	include_package_data=True,	60	include_package_data=True,
73	zip_safe=False,	61	zip_safe=False,
74	test_suite="appenlight",	62	test_suite='appenlight',
75	install_requires=requires,	63	install_requires=requires,
76	extras_require={
77	"dev": [
78	"coverage",
79	"pytest",
80	"pyramid",
81	"tox",
82	"mock",
83	"pytest-mock",
84	"webtest",
85	],
86	"lint": ["black"],
87	},
88	entry_points={	64	entry_points={
89	"paste.app_factory": [~~"main = appenlight:main"~~],	65	'paste.app_factory': [
90	"console_scripts": [	66	'main = appenlight:main'
91	"appenlight-cleanup = appenlight.scripts.cleanup:main",
92	"appenlight-initializedb = appenlight.scripts.initialize_db:main",
93	"appenlight-migratedb = appenlight.scripts.migratedb:main",
94	"appenlight-reindex-elasticsearch = appenlight.scripts.reindex_elasticsearch:main",
95	"appenlight-static = appenlight.scripts.static:main",
96	"appenlight-make-config = appenlight.scripts.make_config:main",
97	],	67	],
98	},	68	'console_scripts': [
		69	'appenlight-cleanup = appenlight.scripts.cleanup:main',
		70	'appenlight-initializedb = appenlight.scripts.initialize_db:main',
		71	'appenlight-migratedb = appenlight.scripts.migratedb:main',
		72	'appenlight-reindex-elasticsearch = appenlight.scripts.reindex_elasticsearch:main',
		73	'appenlight-static = appenlight.scripts.static:main',
		74	'appenlight-make-config = appenlight.scripts.make_config:main',
		75	]
		76	}
99	)	77	)

backend/src/appenlight/__init__.py

0 +92 -124

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import datetime
             import logging
-            from elasticsearch import Elasticsearch
+            import pyelasticsearch
             import redis
             import os
-            import pkg_resources
             from pkg_resources import iter_entry_points
             import appenlight.lib.jinja2_filters as jinja2_filters
             import appenlight.lib.encryption as encryption
             from pyramid.config import PHASE3_CONFIG
             from pyramid.authentication import AuthTktAuthenticationPolicy
             from pyramid.authorization import ACLAuthorizationPolicy
-            from pyramid_mailer.interfaces import IMailer
+            from pyramid_mailer.mailer import Mailer
             from pyramid.renderers import JSON
             from pyramid_redis_sessions import session_factory_from_settings
             from pyramid.settings import asbool, aslist
             from pyramid.security import AllPermissionsList
             from pyramid_authstack import AuthenticationStackPolicy
             from redlock import Redlock
             from sqlalchemy import engine_from_config
             from appenlight.celery import configure_celery
-            from appenlight.lib.configurator import (
+            from appenlight.lib.configurator import (CythonCompatConfigurator,
-                CythonCompatConfigurator,
+                                                     register_appenlight_plugin)
-                register_appenlight_plugin,
             from appenlight.lib import cache_regions
             from appenlight.lib.ext_json import json
             from appenlight.security import groupfinder, AuthTokenAuthenticationPolicy
-            __license__ = "Apache 2.0"
+            __license__ = 'Apache 2.0'
-            __author__ = "RhodeCode GmbH"
+            __author__ = 'RhodeCode GmbH'
-            __url__ = "http://rhodecode.com"
+            __url__ = 'http://rhodecode.com'
-            __version__ = pkg_resources.get_distribution("appenlight").parsed_version
             json_renderer = JSON(serializer=json.dumps, indent=4)
             log = logging.getLogger(__name__)
             def datetime_adapter(obj, request):
                 return obj.isoformat()
             def all_permissions_adapter(obj, request):
-                return "__all_permissions__"
+                return '__all_permissions__'
             json_renderer.add_adapter(datetime.datetime, datetime_adapter)
             json_renderer.add_adapter(AllPermissionsList, all_permissions_adapter)
             def main(global_config, **settings):
                 """ This function returns a Pyramid WSGI application.
                 """
                 auth_tkt_policy = AuthTktAuthenticationPolicy(
-                    settings["authtkt.secret"],
+                    settings['authtkt.secret'],
-                    hashalg="sha512",
+                    hashalg='sha512',
                     callback=groupfinder,
                     max_age=2592000,
-                    secure=asbool(settings.get("authtkt.secure", "false")),
+                    secure=asbool(settings.get('authtkt.secure', 'false')))
+                auth_token_policy = AuthTokenAuthenticationPolicy(
+                    callback=groupfinder
                 )
-                auth_token_policy = AuthTokenAuthenticationPolicy(callback=groupfinder)
                 authorization_policy = ACLAuthorizationPolicy()
                 authentication_policy = AuthenticationStackPolicy()
-                authentication_policy.add_policy("auth_tkt", auth_tkt_policy)
+                authentication_policy.add_policy('auth_tkt', auth_tkt_policy)
-                authentication_policy.add_policy("auth_token", auth_token_policy)
+                authentication_policy.add_policy('auth_token', auth_token_policy)
                 # set crypto key
-                encryption.ENCRYPTION_SECRET = settings.get("encryption_secret")
+                encryption.ENCRYPTION_SECRET = settings.get('encryption_secret')
                 # import this later so encyption key can be monkeypatched
                 from appenlight.models import DBSession, register_datastores
                 # registration
-                settings["appenlight.disable_registration"] = asbool(
+                settings['appenlight.disable_registration'] = asbool(
-                    settings.get("appenlight.disable_registration")
+                    settings.get('appenlight.disable_registration'))
                 # update config with cometd info
-                settings["cometd_servers"] = {
+                settings['cometd_servers'] = {'server': settings['cometd.server'],
-                    "server": settings["cometd.server"],
+                                              'secret': settings['cometd.secret']}
-                    "secret": settings["cometd.secret"],
                 # Create the Pyramid Configurator.
-                settings["_mail_url"] = settings["mailing.app_url"]
+                settings['_mail_url'] = settings['mailing.app_url']
                 config = CythonCompatConfigurator(
                     settings=settings,
                     authentication_policy=authentication_policy,
                     authorization_policy=authorization_policy,
-                    root_factory="appenlight.security.RootFactory",
+                    root_factory='appenlight.security.RootFactory',
-                    default_permission="view",
+                    default_permission='view')
                 # custom registry variables
                 # resource type information
-                config.registry.resource_types = ["resource", "application"]
+                config.registry.resource_types = ['resource', 'application']
                 # plugin information
                 config.registry.appenlight_plugins = {}
-                config.set_default_csrf_options(require_csrf=True, header="X-XSRF-TOKEN")
+                config.set_default_csrf_options(require_csrf=True, header='X-XSRF-TOKEN')
-                config.add_view_deriver("appenlight.predicates.csrf_view", name="csrf_view")
+                config.add_view_deriver('appenlight.predicates.csrf_view',
+                                        name='csrf_view')
                 # later, when config is available
-                dogpile_config = {
+                dogpile_config = {'url': settings['redis.url'],
-                    "url": settings["redis.url"],
                                   "redis_expiration_time": 86400,
-                    "redis_distributed_lock": True,
+                                  "redis_distributed_lock": True}
                 cache_regions.regions = cache_regions.CacheRegions(dogpile_config)
                 config.registry.cache_regions = cache_regions.regions
-                engine = engine_from_config(settings, "sqlalchemy.", json_serializer=json.dumps)
+                engine = engine_from_config(settings, 'sqlalchemy.',
+                                            json_serializer=json.dumps)
                 DBSession.configure(bind=engine)
                 # json rederer that serializes datetime
-                config.add_renderer("json", json_renderer)
+                config.add_renderer('json', json_renderer)
-                config.add_request_method(
+                config.set_request_property('appenlight.lib.request.es_conn', 'es_conn')
-                    "appenlight.lib.request.es_conn", "es_conn", property=True
+                config.set_request_property('appenlight.lib.request.get_user', 'user',
+                                            reify=True)
-                config.add_request_method(
+                config.set_request_property('appenlight.lib.request.get_csrf_token',
-                    "appenlight.lib.request.get_user", "user", reify=True, property=True
+                                            'csrf_token', reify=True)
+                config.set_request_property('appenlight.lib.request.safe_json_body',
-                config.add_request_method(
+                                            'safe_json_body', reify=True)
-                    "appenlight.lib.request.get_csrf_token", "csrf_token", reify=True, property=True
+                config.set_request_property('appenlight.lib.request.unsafe_json_body',
+                                            'unsafe_json_body', reify=True)
-                config.add_request_method(
+                config.add_request_method('appenlight.lib.request.add_flash_to_headers',
-                    "appenlight.lib.request.safe_json_body",
+                                          'add_flash_to_headers')
-                    "safe_json_body",
+                config.add_request_method('appenlight.lib.request.get_authomatic',
-                    reify=True,
+                                          'authomatic', reify=True)
-                    property=True,
+                config.include('pyramid_redis_sessions')
-                config.add_request_method(
+                config.include('pyramid_tm')
-                    "appenlight.lib.request.unsafe_json_body",
+                config.include('pyramid_jinja2')
-                    "unsafe_json_body",
+                config.include('appenlight_client.ext.pyramid_tween')
-                    reify=True,
+                config.include('ziggurat_foundations.ext.pyramid.sign_in')
-                    property=True,
+                es_server_list = aslist(settings['elasticsearch.nodes'])
+                redis_url = settings['redis.url']
-                config.add_request_method(
+                log.warning('Elasticsearch server list: {}'.format(es_server_list))
-                    "appenlight.lib.request.add_flash_to_headers", "add_flash_to_headers"
+                log.warning('Redis server: {}'.format(redis_url))
+                config.registry.es_conn = pyelasticsearch.ElasticSearch(es_server_list)
-                config.add_request_method(
-                    "appenlight.lib.request.get_authomatic", "authomatic", reify=True
-                config.include("pyramid_redis_sessions")
-                config.include("pyramid_tm")
-                config.include("pyramid_jinja2")
-                config.include("pyramid_mailer")
-                config.include("appenlight_client.ext.pyramid_tween")
-                config.include("ziggurat_foundations.ext.pyramid.sign_in")
-                es_server_list = aslist(settings["elasticsearch.nodes"])
-                redis_url = settings["redis.url"]
-                log.warning("Elasticsearch server list: {}".format(es_server_list))
-                log.warning("Redis server: {}".format(redis_url))
-                config.registry.es_conn = Elasticsearch(es_server_list)
                 config.registry.redis_conn = redis.StrictRedis.from_url(redis_url)
-                config.registry.redis_lockmgr = Redlock(
+                config.registry.redis_lockmgr = Redlock([settings['redis.redlock.url'], ],
-                    [settings["redis.redlock.url"]], retry_count=0, retry_delay=0
+                                                        retry_count=0, retry_delay=0)
+                # mailer
-                # mailer bw compat
+                config.registry.mailer = Mailer.from_settings(settings)
-                config.registry.mailer = config.registry.getUtility(IMailer)
                 # Configure sessions
                 session_factory = session_factory_from_settings(settings)
                 config.set_session_factory(session_factory)
                 # Configure renderers and event subscribers
-                config.add_jinja2_extension("jinja2.ext.loopcontrols")
+                config.add_jinja2_extension('jinja2.ext.loopcontrols')
-                config.add_jinja2_search_path("appenlight:templates")
+                config.add_jinja2_search_path('appenlight:templates')
                 # event subscribers
-                config.add_subscriber(
+                config.add_subscriber("appenlight.subscribers.application_created",
-                    "appenlight.subscribers.application_created",
+                                      "pyramid.events.ApplicationCreated")
-                    "pyramid.events.ApplicationCreated",
+                config.add_subscriber("appenlight.subscribers.add_renderer_globals",
+                                      "pyramid.events.BeforeRender")
-                config.add_subscriber(
+                config.add_subscriber('appenlight.subscribers.new_request',
-                    "appenlight.subscribers.add_renderer_globals", "pyramid.events.BeforeRender"
+                                      'pyramid.events.NewRequest')
+                config.add_view_predicate('context_type_class',
-                config.add_subscriber(
+                                          'appenlight.predicates.contextTypeClass')
-                    "appenlight.subscribers.new_request", "pyramid.events.NewRequest"
+                register_datastores(es_conn=config.registry.es_conn,
-                config.add_view_predicate(
-                    "context_type_class", "appenlight.predicates.contextTypeClass"
-                register_datastores(
-                    es_conn=config.registry.es_conn,
                                     redis_conn=config.registry.redis_conn,
-                    redis_lockmgr=config.registry.redis_lockmgr,
+                                    redis_lockmgr=config.registry.redis_lockmgr)
                 # base stuff and scan
                 # need to ensure webassets exists otherwise config.override_asset()
                 # throws exception
-                if not os.path.exists(settings["webassets.dir"]):
+                if not os.path.exists(settings['webassets.dir']):
-                    os.mkdir(settings["webassets.dir"])
+                    os.mkdir(settings['webassets.dir'])
-                config.add_static_view(
+                config.add_static_view(path='appenlight:webassets',
-                    path="appenlight:webassets", name="static", cache_max_age=3600
+                                       name='static', cache_max_age=3600)
+                config.override_asset(to_override='appenlight:webassets/',
-                config.override_asset(
+                                      override_with=settings['webassets.dir'])
-                    to_override="appenlight:webassets/", override_with=settings["webassets.dir"]
+                config.include('appenlight.views')
+                config.include('appenlight.views.admin')
-                config.include("appenlight.views")
+                config.scan(ignore=['appenlight.migrations', 'appenlight.scripts',
-                config.include("appenlight.views.admin")
+                                    'appenlight.tests'])
-                config.scan(
-                    ignore=["appenlight.migrations", "appenlight.scripts", "appenlight.tests"]
+                config.add_directive('register_appenlight_plugin',
+                                     register_appenlight_plugin)
-                config.add_directive("register_appenlight_plugin", register_appenlight_plugin)
+                for entry_point in iter_entry_points(group='appenlight.plugins'):
-                for entry_point in iter_entry_points(group="appenlight.plugins"):
                     plugin = entry_point.load()
                     plugin.includeme(config)
                 # include other appenlight plugins explictly if needed
-                includes = aslist(settings.get("appenlight.includes", []))
+                includes = aslist(settings.get('appenlight.includes', []))
                 for inc in includes:
                     config.include(inc)
                 # run this after everything registers in configurator
                 def pre_commit():
                     jinja_env = config.get_jinja2_environment()
-                    jinja_env.filters["tojson"] = json.dumps
+                    jinja_env.filters['tojson'] = json.dumps
-                    jinja_env.filters["toJSONUnsafe"] = jinja2_filters.toJSONUnsafe
+                    jinja_env.filters['toJSONUnsafe'] = jinja2_filters.toJSONUnsafe
                 config.action(None, pre_commit, order=PHASE3_CONFIG + 999)
                 def wrap_config_celery():
                     configure_celery(config.registry)
                 config.action(None, wrap_config_celery, order=PHASE3_CONFIG + 999)
                 app = config.make_wsgi_app()
                 return app

backend/src/appenlight/celery/__init__.py

0 +68 -78

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from datetime import timedelta
             from celery import Celery
             from celery.bin import Option
             from celery.schedules import crontab
             from celery.signals import worker_init, task_revoked, user_preload_options
             from celery.signals import task_prerun, task_retry, task_failure, task_success
             from kombu.serialization import register
             from pyramid.paster import bootstrap
             from pyramid.request import Request
             from pyramid.scripting import prepare
             from pyramid.settings import asbool
             from pyramid.threadlocal import get_current_request
             from appenlight.celery.encoders import json_dumps, json_loads
             from appenlight_client.ext.celery import register_signals
             log = logging.getLogger(__name__)
-            register(
+            register('date_json', json_dumps, json_loads,
-                "date_json",
+                     content_type='application/x-date_json',
-                json_dumps,
+                     content_encoding='utf-8')
-                json_loads,
-                content_type="application/x-date_json",
-                content_encoding="utf-8",
             celery = Celery()
-            celery.user_options["preload"].add(
+            celery.user_options['preload'].add(
-                Option(
+                Option('--ini', dest='ini', default=None,
-                    "--ini",
+                       help='Specifies pyramid configuration file location.')
-                    dest="ini",
-                    default=None,
-                    help="Specifies pyramid configuration file location.",
             )
             @user_preload_options.connect
             def on_preload_parsed(options, **kwargs):
                 """
                 This actually configures celery from pyramid config file
                 """
-                celery.conf["INI_PYRAMID"] = options["ini"]
+                celery.conf['INI_PYRAMID'] = options['ini']
                 import appenlight_client.client as e_client
+                ini_location = options['ini']
-                ini_location = options["ini"]
                 if not ini_location:
-                    raise Exception(
+                    raise Exception('You need to pass pyramid ini location using '
-                        "You need to pass pyramid ini location using "
+                                    '--ini=filename.ini argument to the worker')
-                        "--ini=filename.ini argument to the worker"
+                env = bootstrap(ini_location)
+                api_key = env['request'].registry.settings['appenlight.api_key']
-                env = bootstrap(ini_location[0])
+                tr_config = env['request'].registry.settings.get(
-                api_key = env["request"].registry.settings["appenlight.api_key"]
+                    'appenlight.transport_config')
-                tr_config = env["request"].registry.settings.get("appenlight.transport_config")
+                CONFIG = e_client.get_config({'appenlight.api_key': api_key})
-                CONFIG = e_client.get_config({"appenlight.api_key": api_key})
                 if tr_config:
-                    CONFIG["appenlight.transport_config"] = tr_config
+                    CONFIG['appenlight.transport_config'] = tr_config
                 APPENLIGHT_CLIENT = e_client.Client(CONFIG)
                 # log.addHandler(APPENLIGHT_CLIENT.log_handler)
                 register_signals(APPENLIGHT_CLIENT)
                 celery.pyramid = env
             celery_config = {
-                "CELERY_IMPORTS": ["appenlight.celery.tasks"],
+                'CELERY_IMPORTS': ["appenlight.celery.tasks", ],
-                "CELERYD_TASK_TIME_LIMIT": 60,
+                'CELERYD_TASK_TIME_LIMIT': 60,
-                "CELERYD_MAX_TASKS_PER_CHILD": 1000,
+                'CELERYD_MAX_TASKS_PER_CHILD': 1000,
-                "CELERY_IGNORE_RESULT": True,
+                'CELERY_IGNORE_RESULT': True,
-                "CELERY_ACCEPT_CONTENT": ["date_json"],
+                'CELERY_ACCEPT_CONTENT': ['date_json'],
-                "CELERY_TASK_SERIALIZER": "date_json",
+                'CELERY_TASK_SERIALIZER': 'date_json',
-                "CELERY_RESULT_SERIALIZER": "date_json",
+                'CELERY_RESULT_SERIALIZER': 'date_json',
-                "BROKER_URL": None,
+                'BROKER_URL': None,
-                "CELERYD_CONCURRENCY": None,
+                'CELERYD_CONCURRENCY': None,
-                "CELERY_TIMEZONE": None,
+                'CELERY_TIMEZONE': None,
-                "CELERYBEAT_SCHEDULE": {
+                'CELERYBEAT_SCHEDULE': {
-                    "alerting_reports": {
+                    'alerting_reports': {
-                        "task": "appenlight.celery.tasks.alerting_reports",
+                        'task': 'appenlight.celery.tasks.alerting_reports',
-                        "schedule": timedelta(seconds=60),
+                        'schedule': timedelta(seconds=60)
-                    },
-                    "close_alerts": {
-                        "task": "appenlight.celery.tasks.close_alerts",
-                        "schedule": timedelta(seconds=60),
-                    },
                     },
+                    'close_alerts': {
+                        'task': 'appenlight.celery.tasks.close_alerts',
+                        'schedule': timedelta(seconds=60)
+                    }
+                }
             }
             celery.config_from_object(celery_config)
             def configure_celery(pyramid_registry):
                 settings = pyramid_registry.settings
-                celery_config["BROKER_URL"] = settings["celery.broker_url"]
+                celery_config['BROKER_URL'] = settings['celery.broker_url']
-                celery_config["CELERYD_CONCURRENCY"] = settings["celery.concurrency"]
+                celery_config['CELERYD_CONCURRENCY'] = settings['celery.concurrency']
-                celery_config["CELERY_TIMEZONE"] = settings["celery.timezone"]
+                celery_config['CELERY_TIMEZONE'] = settings['celery.timezone']
-                notifications_seconds = int(
+                notifications_seconds = int(settings.get('tasks.notifications_reports.interval', 60))
-                    settings.get("tasks.notifications_reports.interval", 60)
-                celery_config["CELERYBEAT_SCHEDULE"]["notifications"] = {
+                celery_config['CELERYBEAT_SCHEDULE']['notifications'] = {
-                    "task": "appenlight.celery.tasks.notifications_reports",
+                    'task': 'appenlight.celery.tasks.notifications_reports',
-                    "schedule": timedelta(seconds=notifications_seconds),
+                    'schedule': timedelta(seconds=notifications_seconds)
                 }
-                celery_config["CELERYBEAT_SCHEDULE"]["daily_digest"] = {
+                celery_config['CELERYBEAT_SCHEDULE']['daily_digest'] = {
-                    "task": "appenlight.celery.tasks.daily_digest",
+                    'task': 'appenlight.celery.tasks.daily_digest',
-                    "schedule": crontab(minute=1, hour="4,12,20"),
+                    'schedule': crontab(minute=1, hour='4,12,20')
                 }
-                if asbool(settings.get("celery.always_eager")):
+                if asbool(settings.get('celery.always_eager')):
-                    celery_config["CELERY_ALWAYS_EAGER"] = True
+                    celery_config['CELERY_ALWAYS_EAGER'] = True
-                    celery_config["CELERY_EAGER_PROPAGATES_EXCEPTIONS"] = True
+                    celery_config['CELERY_EAGER_PROPAGATES_EXCEPTIONS'] = True
                 for plugin in pyramid_registry.appenlight_plugins.values():
-                    if plugin.get("celery_tasks"):
+                    if plugin.get('celery_tasks'):
-                        celery_config["CELERY_IMPORTS"].extend(plugin["celery_tasks"])
+                        celery_config['CELERY_IMPORTS'].extend(plugin['celery_tasks'])
-                    if plugin.get("celery_beats"):
+                    if plugin.get('celery_beats'):
-                        for name, config in plugin["celery_beats"]:
+                        for name, config in plugin['celery_beats']:
-                            celery_config["CELERYBEAT_SCHEDULE"][name] = config
+                            celery_config['CELERYBEAT_SCHEDULE'][name] = config
                 celery.config_from_object(celery_config)
             @task_prerun.connect
             def task_prerun_signal(task_id, task, args, kwargs, **kwaargs):
-                if hasattr(celery, "pyramid"):
+                if hasattr(celery, 'pyramid'):
                     env = celery.pyramid
-                    env = prepare(registry=env["request"].registry)
+                    env = prepare(registry=env['request'].registry)
-                    proper_base_url = env["request"].registry.settings["mailing.app_url"]
+                    proper_base_url = env['request'].registry.settings['mailing.app_url']
-                    tmp_req = Request.blank("/", base_url=proper_base_url)
+                    tmp_req = Request.blank('/', base_url=proper_base_url)
                     # ensure tasks generate url for right domain from config
-                    env["request"].environ["HTTP_HOST"] = tmp_req.environ["HTTP_HOST"]
+                    env['request'].environ['HTTP_HOST'] = tmp_req.environ['HTTP_HOST']
-                    env["request"].environ["SERVER_PORT"] = tmp_req.environ["SERVER_PORT"]
+                    env['request'].environ['SERVER_PORT'] = tmp_req.environ['SERVER_PORT']
-                    env["request"].environ["SERVER_NAME"] = tmp_req.environ["SERVER_NAME"]
+                    env['request'].environ['SERVER_NAME'] = tmp_req.environ['SERVER_NAME']
-                    env["request"].environ["wsgi.url_scheme"] = tmp_req.environ["wsgi.url_scheme"]
+                    env['request'].environ['wsgi.url_scheme'] = \
+                        tmp_req.environ['wsgi.url_scheme']
                 get_current_request().tm.begin()
             @task_success.connect
             def task_success_signal(result, **kwargs):
                 get_current_request().tm.commit()
-                if hasattr(celery, "pyramid"):
+                if hasattr(celery, 'pyramid'):
                     celery.pyramid["closer"]()
             @task_retry.connect
             def task_retry_signal(request, reason, einfo, **kwargs):
                 get_current_request().tm.abort()
-                if hasattr(celery, "pyramid"):
+                if hasattr(celery, 'pyramid'):
                     celery.pyramid["closer"]()
             @task_failure.connect
-            def task_failure_signal(task_id, exception, args, kwargs, traceback, einfo, **kwaargs):
+            def task_failure_signal(task_id, exception, args, kwargs, traceback, einfo,
+                                    **kwaargs):
                 get_current_request().tm.abort()
-                if hasattr(celery, "pyramid"):
+                if hasattr(celery, 'pyramid'):
                     celery.pyramid["closer"]()
             @task_revoked.connect
             def task_revoked_signal(request, terminated, signum, expired, **kwaargs):
                 get_current_request().tm.abort()
-                if hasattr(celery, "pyramid"):
+                if hasattr(celery, 'pyramid'):
                     celery.pyramid["closer"]()

backend/src/appenlight/celery/encoders.py

0 +21 -12

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import json
             from datetime import datetime, date, timedelta
-            DATE_FORMAT = "%Y-%m-%dT%H:%M:%S.%f"
+            DATE_FORMAT = '%Y-%m-%dT%H:%M:%S.%f'
             class DateEncoder(json.JSONEncoder):
                 def default(self, obj):
                     if isinstance(obj, datetime):
-                        return {"__type__": "__datetime__", "iso": obj.strftime(DATE_FORMAT)}
+                        return {
+                            '__type__': '__datetime__',
+                            'iso': obj.strftime(DATE_FORMAT)
+                        }
                     elif isinstance(obj, date):
-                        return {"__type__": "__date__", "iso": obj.strftime(DATE_FORMAT)}
+                        return {
+                            '__type__': '__date__',
+                            'iso': obj.strftime(DATE_FORMAT)
+                        }
                     elif isinstance(obj, timedelta):
-                        return {"__type__": "__timedelta__", "seconds": obj.total_seconds()}
+                        return {
+                            '__type__': '__timedelta__',
+                            'seconds': obj.total_seconds()
+                        }
                     else:
                         return json.JSONEncoder.default(self, obj)
             def date_decoder(dct):
-                if "__type__" in dct:
+                if '__type__' in dct:
-                    if dct["__type__"] == "__datetime__":
+                    if dct['__type__'] == '__datetime__':
-                        return datetime.strptime(dct["iso"], DATE_FORMAT)
+                        return datetime.strptime(dct['iso'], DATE_FORMAT)
-                    elif dct["__type__"] == "__date__":
+                    elif dct['__type__'] == '__date__':
-                        return datetime.strptime(dct["iso"], DATE_FORMAT).date()
+                        return datetime.strptime(dct['iso'], DATE_FORMAT).date()
-                    elif dct["__type__"] == "__timedelta__":
+                    elif dct['__type__'] == '__timedelta__':
-                        return timedelta(seconds=dct["seconds"])
+                        return timedelta(seconds=dct['seconds'])
                 return dct
             def json_dumps(obj):
                 return json.dumps(obj, cls=DateEncoder)
             def json_loads(obj):
-                return json.loads(obj.decode("utf8"), object_hook=date_decoder)
+                return json.loads(obj.decode('utf8'), object_hook=date_decoder)

backend/src/appenlight/celery/tasks.py

0 +194 -239

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import bisect
             import collections
             import math
             from datetime import datetime, timedelta
             import sqlalchemy as sa
-            import elasticsearch.exceptions
+            import pyelasticsearch
-            import elasticsearch.helpers
             from celery.utils.log import get_task_logger
             from zope.sqlalchemy import mark_changed
             from pyramid.threadlocal import get_current_request, get_current_registry
-            from ziggurat_foundations.models.services.resource import ResourceService
             from appenlight.celery import celery
             from appenlight.models.report_group import ReportGroup
             from appenlight.models import DBSession, Datastores
             from appenlight.models.report import Report
             from appenlight.models.log import Log
             from appenlight.models.metric import Metric
             from appenlight.models.event import Event
             from appenlight.models.services.application import ApplicationService
             from appenlight.models.services.event import EventService
             from appenlight.models.services.log import LogService
             from appenlight.models.services.report import ReportService
             from appenlight.models.services.report_group import ReportGroupService
             from appenlight.models.services.user import UserService
             from appenlight.models.tag import Tag
             from appenlight.lib import print_traceback
             from appenlight.lib.utils import parse_proto, in_batches
             from appenlight.lib.ext_json import json
             from appenlight.lib.redis_keys import REDIS_KEYS
             from appenlight.lib.enums import ReportType
             log = get_task_logger(__name__)
-            sample_boundries = (
+            sample_boundries = list(range(100, 1000, 100)) + \
-                list(range(100, 1000, 100))
+                               list(range(1000, 10000, 1000)) + \
-                + list(range(1000, 10000, 1000))
+                               list(range(10000, 100000, 5000))
-                + list(range(10000, 100000, 5000))
             def pick_sample(total_occurences, report_type=None):
                 every = 1.0
                 position = bisect.bisect_left(sample_boundries, total_occurences)
                 if position > 0:
                     if report_type == ReportType.not_found:
                         divide = 10.0
                     else:
                         divide = 100.0
                     every = sample_boundries[position - 1] / divide
                 return total_occurences % every == 0
             @celery.task(queue="default", default_retry_delay=1, max_retries=2)
             def test_exception_task():
-                log.error("test celery log", extra={"location": "celery"})
+                log.error('test celery log', extra={'location': 'celery'})
-                log.warning("test celery log", extra={"location": "celery"})
+                log.warning('test celery log', extra={'location': 'celery'})
-                raise Exception("Celery exception test")
+                raise Exception('Celery exception test')
             @celery.task(queue="default", default_retry_delay=1, max_retries=2)
             def test_retry_exception_task():
                 try:
                     import time
                     time.sleep(1.3)
-                    log.error("test retry celery log", extra={"location": "celery"})
+                    log.error('test retry celery log', extra={'location': 'celery'})
-                    log.warning("test retry celery log", extra={"location": "celery"})
+                    log.warning('test retry celery log', extra={'location': 'celery'})
-                    raise Exception("Celery exception test")
+                    raise Exception('Celery exception test')
                 except Exception as exc:
-                    if celery.conf["CELERY_EAGER_PROPAGATES_EXCEPTIONS"]:
-                        raise
                     test_retry_exception_task.retry(exc=exc)
             @celery.task(queue="reports", default_retry_delay=600, max_retries=144)
             def add_reports(resource_id, request_params, dataset, **kwargs):
-                proto_version = parse_proto(request_params.get("protocol_version", ""))
+                proto_version = parse_proto(request_params.get('protocol_version', ''))
                 current_time = datetime.utcnow().replace(second=0, microsecond=0)
                 try:
                     # we will store solr docs here for single insert
                     es_report_docs = {}
                     es_report_group_docs = {}
                     resource = ApplicationService.by_id(resource_id)
                     tags = []
                     es_slow_calls_docs = {}
                     es_reports_stats_rows = {}
                     for report_data in dataset:
                         # build report details for later
                         added_details = 0
                         report = Report()
                         report.set_data(report_data, resource, proto_version)
                         report._skip_ft_index = True
                         # find latest group in this months partition
                         report_group = ReportGroupService.by_hash_and_resource(
                             report.resource_id,
                             report.grouping_hash,
-                            since_when=datetime.utcnow().date().replace(day=1),
+                            since_when=datetime.utcnow().date().replace(day=1)
                         )
-                        occurences = report_data.get("occurences", 1)
+                        occurences = report_data.get('occurences', 1)
                         if not report_group:
                             # total reports will be +1 moment later
-                            report_group = ReportGroup(
+                            report_group = ReportGroup(grouping_hash=report.grouping_hash,
-                                grouping_hash=report.grouping_hash,
+                                                       occurences=0, total_reports=0,
-                                occurences=0,
-                                total_reports=0,
                                                        last_report=0,
                                                        priority=report.priority,
                                                        error=report.error,
-                                first_timestamp=report.start_time,
+                                                       first_timestamp=report.start_time)
                             report_group._skip_ft_index = True
                             report_group.report_type = report.report_type
                         report.report_group_time = report_group.first_timestamp
-                        add_sample = pick_sample(
+                        add_sample = pick_sample(report_group.occurences,
-                            report_group.occurences, report_type=report_group.report_type
+                                                 report_type=report_group.report_type)
                         if add_sample:
                             resource.report_groups.append(report_group)
                             report_group.reports.append(report)
                             added_details += 1
                             DBSession.flush()
                             if report.partition_id not in es_report_docs:
                                 es_report_docs[report.partition_id] = []
                             es_report_docs[report.partition_id].append(report.es_doc())
                             tags.extend(list(report.tags.items()))
                             slow_calls = report.add_slow_calls(report_data, report_group)
                             DBSession.flush()
                             for s_call in slow_calls:
                                 if s_call.partition_id not in es_slow_calls_docs:
                                     es_slow_calls_docs[s_call.partition_id] = []
-                                es_slow_calls_docs[s_call.partition_id].append(s_call.es_doc())
+                                es_slow_calls_docs[s_call.partition_id].append(
+                                    s_call.es_doc())
                                 # try generating new stat rows if needed
                         else:
                             # required for postprocessing to not fail later
                             report.report_group = report_group
-                        stat_row = ReportService.generate_stat_rows(report, resource, report_group)
+                        stat_row = ReportService.generate_stat_rows(
+                            report, resource, report_group)
                         if stat_row.partition_id not in es_reports_stats_rows:
                             es_reports_stats_rows[stat_row.partition_id] = []
-                        es_reports_stats_rows[stat_row.partition_id].append(stat_row.es_doc())
+                        es_reports_stats_rows[stat_row.partition_id].append(
+                            stat_row.es_doc())
                         # see if we should mark 10th occurence of report
                         last_occurences_10 = int(math.floor(report_group.occurences / 10))
-                        curr_occurences_10 = int(
+                        curr_occurences_10 = int(math.floor(
-                            math.floor((report_group.occurences + report.occurences) / 10)
+                            (report_group.occurences + report.occurences) / 10))
+                        last_occurences_100 = int(
-                        last_occurences_100 = int(math.floor(report_group.occurences / 100))
+                            math.floor(report_group.occurences / 100))
-                        curr_occurences_100 = int(
+                        curr_occurences_100 = int(math.floor(
-                            math.floor((report_group.occurences + report.occurences) / 100)
+                            (report_group.occurences + report.occurences) / 100))
                         notify_occurences_10 = last_occurences_10 != curr_occurences_10
                         notify_occurences_100 = last_occurences_100 != curr_occurences_100
                         report_group.occurences = ReportGroup.occurences + occurences
                         report_group.last_timestamp = report.start_time
                         report_group.summed_duration = ReportGroup.summed_duration + report.duration
                         summed_duration = ReportGroup.summed_duration + report.duration
                         summed_occurences = ReportGroup.occurences + occurences
                         report_group.average_duration = summed_duration / summed_occurences
                         report_group.run_postprocessing(report)
                         if added_details:
                             report_group.total_reports = ReportGroup.total_reports + 1
                             report_group.last_report = report.id
-                        report_group.set_notification_info(
+                        report_group.set_notification_info(notify_10=notify_occurences_10,
-                            notify_10=notify_occurences_10, notify_100=notify_occurences_100
+                                                           notify_100=notify_occurences_100)
                         DBSession.flush()
                         report_group.get_report().notify_channel(report_group)
                         if report_group.partition_id not in es_report_group_docs:
                             es_report_group_docs[report_group.partition_id] = []
                         es_report_group_docs[report_group.partition_id].append(
-                            report_group.es_doc()
+                            report_group.es_doc())
-                        action = "REPORT"
+                        action = 'REPORT'
-                        log_msg = "%s: %s %s, client: %s, proto: %s" % (
+                        log_msg = '%s: %s %s, client: %s, proto: %s' % (
                             action,
-                            report_data.get("http_status", "unknown"),
+                            report_data.get('http_status', 'unknown'),
                             str(resource),
-                            report_data.get("client"),
+                            report_data.get('client'),
-                            proto_version,
+                            proto_version)
                         log.info(log_msg)
                     total_reports = len(dataset)
                     redis_pipeline = Datastores.redis.pipeline(transaction=False)
-                    key = REDIS_KEYS["counters"]["reports_per_minute"].format(current_time)
+                    key = REDIS_KEYS['counters']['reports_per_minute'].format(current_time)
                     redis_pipeline.incr(key, total_reports)
                     redis_pipeline.expire(key, 3600 * 24)
-                    key = REDIS_KEYS["counters"]["events_per_minute_per_user"].format(
+                    key = REDIS_KEYS['counters']['events_per_minute_per_user'].format(
-                        resource.owner_user_id, current_time
+                        resource.owner_user_id, current_time)
                     redis_pipeline.incr(key, total_reports)
                     redis_pipeline.expire(key, 3600)
-                    key = REDIS_KEYS["counters"]["reports_per_hour_per_app"].format(
+                    key = REDIS_KEYS['counters']['reports_per_hour_per_app'].format(
-                        resource_id, current_time.replace(minute=0)
+                        resource_id, current_time.replace(minute=0))
                     redis_pipeline.incr(key, total_reports)
                     redis_pipeline.expire(key, 3600 * 24 * 7)
                     redis_pipeline.sadd(
-                        REDIS_KEYS["apps_that_got_new_data_per_hour"].format(
+                        REDIS_KEYS['apps_that_got_new_data_per_hour'].format(
-                            current_time.replace(minute=0)
+                            current_time.replace(minute=0)), resource_id)
-                        ),
-                        resource_id,
                     redis_pipeline.execute()
                     add_reports_es(es_report_group_docs, es_report_docs)
                     add_reports_slow_calls_es(es_slow_calls_docs)
                     add_reports_stats_rows_es(es_reports_stats_rows)
                     return True
                 except Exception as exc:
                     print_traceback(log)
-                    if celery.conf["CELERY_EAGER_PROPAGATES_EXCEPTIONS"]:
-                        raise
                     add_reports.retry(exc=exc)
             @celery.task(queue="es", default_retry_delay=600, max_retries=144)
             def add_reports_es(report_group_docs, report_docs):
                 for k, v in report_group_docs.items():
-                    to_update = {"_index": k, "_type": "report"}
+                    Datastores.es.bulk_index(k, 'report_group', v, id_field="_id")
-                    [i.update(to_update) for i in v]
-                    elasticsearch.helpers.bulk(Datastores.es, v)
                 for k, v in report_docs.items():
-                    to_update = {"_index": k, "_type": "report"}
+                    Datastores.es.bulk_index(k, 'report', v, id_field="_id",
-                    [i.update(to_update) for i in v]
+                                             parent_field='_parent')
-                    elasticsearch.helpers.bulk(Datastores.es, v)
             @celery.task(queue="es", default_retry_delay=600, max_retries=144)
             def add_reports_slow_calls_es(es_docs):
                 for k, v in es_docs.items():
-                    to_update = {"_index": k, "_type": "log"}
+                    Datastores.es.bulk_index(k, 'log', v)
-                    [i.update(to_update) for i in v]
-                    elasticsearch.helpers.bulk(Datastores.es, v)
             @celery.task(queue="es", default_retry_delay=600, max_retries=144)
             def add_reports_stats_rows_es(es_docs):
                 for k, v in es_docs.items():
-                    to_update = {"_index": k, "_type": "report"}
+                    Datastores.es.bulk_index(k, 'log', v)
-                    [i.update(to_update) for i in v]
-                    elasticsearch.helpers.bulk(Datastores.es, v)
             @celery.task(queue="logs", default_retry_delay=600, max_retries=144)
             def add_logs(resource_id, request_params, dataset, **kwargs):
-                proto_version = request_params.get("protocol_version")
+                proto_version = request_params.get('protocol_version')
                 current_time = datetime.utcnow().replace(second=0, microsecond=0)
                 try:
                     es_docs = collections.defaultdict(list)
                     resource = ApplicationService.by_id_cached()(resource_id)
                     resource = DBSession.merge(resource, load=False)
                     ns_pairs = []
                     for entry in dataset:
                         # gather pk and ns so we can remove older versions of row later
-                        if entry["primary_key"] is not None:
+                        if entry['primary_key'] is not None:
-                            ns_pairs.append({"pk": entry["primary_key"], "ns": entry["namespace"]})
+                            ns_pairs.append({"pk": entry['primary_key'],
+                                             "ns": entry['namespace']})
                         log_entry = Log()
                         log_entry.set_data(entry, resource=resource)
                         log_entry._skip_ft_index = True
                         resource.logs.append(log_entry)
                         DBSession.flush()
                         # insert non pk rows first
-                        if entry["primary_key"] is None:
+                        if entry['primary_key'] is None:
                             es_docs[log_entry.partition_id].append(log_entry.es_doc())
-                    # 2nd pass to delete all log entries from db for same pk/ns pair
+                    # 2nd pass to delete all log entries from db foe same pk/ns pair
                     if ns_pairs:
                         ids_to_delete = []
                         es_docs = collections.defaultdict(list)
                         es_docs_to_delete = collections.defaultdict(list)
                         found_pkey_logs = LogService.query_by_primary_key_and_namespace(
-                            list_of_pairs=ns_pairs
+                            list_of_pairs=ns_pairs)
                         log_dict = {}
                         for log_entry in found_pkey_logs:
                             log_key = (log_entry.primary_key, log_entry.namespace)
                             if log_key not in log_dict:
                                 log_dict[log_key] = []
                             log_dict[log_key].append(log_entry)
                         for ns, entry_list in log_dict.items():
                             entry_list = sorted(entry_list, key=lambda x: x.timestamp)
                             # newest row needs to be indexed in es
                             log_entry = entry_list[-1]
                             # delete everything from pg and ES, leave the last row in pg
                             for e in entry_list[:-1]:
                                 ids_to_delete.append(e.log_id)
                                 es_docs_to_delete[e.partition_id].append(e.delete_hash)
-                            es_docs_to_delete[log_entry.partition_id].append(log_entry.delete_hash)
+                            es_docs_to_delete[log_entry.partition_id].append(
+                                log_entry.delete_hash)
                             es_docs[log_entry.partition_id].append(log_entry.es_doc())
                         if ids_to_delete:
-                            query = DBSession.query(Log).filter(Log.log_id.in_(ids_to_delete))
+                            query = DBSession.query(Log).filter(
+                                Log.log_id.in_(ids_to_delete))
                             query.delete(synchronize_session=False)
                         if es_docs_to_delete:
                             # batch this to avoid problems with default ES bulk limits
                             for es_index in es_docs_to_delete.keys():
                                 for batch in in_batches(es_docs_to_delete[es_index], 20):
-                                    query = {"query": {"terms": {"delete_hash": batch}}}
+                                    query = {'terms': {'delete_hash': batch}}
                                     try:
                                         Datastores.es.delete_by_query(
-                                            index=es_index,
+                                            es_index, 'log', query)
-                                            doc_type="log",
+                                    except pyelasticsearch.ElasticHttpNotFoundError as exc:
-                                            body=query,
+                                        msg = 'skipping index {}'.format(es_index)
-                                            conflicts="proceed",
-                                    except elasticsearch.exceptions.NotFoundError as exc:
-                                        msg = "skipping index {}".format(es_index)
                                         log.info(msg)
                     total_logs = len(dataset)
-                    log_msg = "LOG_NEW: %s, entries: %s, proto:%s" % (
+                    log_msg = 'LOG_NEW: %s, entries: %s, proto:%s' % (
                         str(resource),
                         total_logs,
-                        proto_version,
+                        proto_version)
                     log.info(log_msg)
                     # mark_changed(session)
                     redis_pipeline = Datastores.redis.pipeline(transaction=False)
-                    key = REDIS_KEYS["counters"]["logs_per_minute"].format(current_time)
+                    key = REDIS_KEYS['counters']['logs_per_minute'].format(current_time)
                     redis_pipeline.incr(key, total_logs)
                     redis_pipeline.expire(key, 3600 * 24)
-                    key = REDIS_KEYS["counters"]["events_per_minute_per_user"].format(
+                    key = REDIS_KEYS['counters']['events_per_minute_per_user'].format(
-                        resource.owner_user_id, current_time
+                        resource.owner_user_id, current_time)
                     redis_pipeline.incr(key, total_logs)
                     redis_pipeline.expire(key, 3600)
-                    key = REDIS_KEYS["counters"]["logs_per_hour_per_app"].format(
+                    key = REDIS_KEYS['counters']['logs_per_hour_per_app'].format(
-                        resource_id, current_time.replace(minute=0)
+                        resource_id, current_time.replace(minute=0))
                     redis_pipeline.incr(key, total_logs)
                     redis_pipeline.expire(key, 3600 * 24 * 7)
                     redis_pipeline.sadd(
-                        REDIS_KEYS["apps_that_got_new_data_per_hour"].format(
+                        REDIS_KEYS['apps_that_got_new_data_per_hour'].format(
-                            current_time.replace(minute=0)
+                            current_time.replace(minute=0)), resource_id)
-                        ),
-                        resource_id,
                     redis_pipeline.execute()
                     add_logs_es(es_docs)
                     return True
                 except Exception as exc:
                     print_traceback(log)
-                    if celery.conf["CELERY_EAGER_PROPAGATES_EXCEPTIONS"]:
-                        raise
                     add_logs.retry(exc=exc)
             @celery.task(queue="es", default_retry_delay=600, max_retries=144)
             def add_logs_es(es_docs):
                 for k, v in es_docs.items():
-                    to_update = {"_index": k, "_type": "log"}
+                    Datastores.es.bulk_index(k, 'log', v)
-                    [i.update(to_update) for i in v]
-                    elasticsearch.helpers.bulk(Datastores.es, v)
             @celery.task(queue="metrics", default_retry_delay=600, max_retries=144)
             def add_metrics(resource_id, request_params, dataset, proto_version):
                 current_time = datetime.utcnow().replace(second=0, microsecond=0)
                 try:
                     resource = ApplicationService.by_id_cached()(resource_id)
                     resource = DBSession.merge(resource, load=False)
                     es_docs = []
                     rows = []
                     for metric in dataset:
-                        tags = dict(metric["tags"])
+                        tags = dict(metric['tags'])
-                        server_n = tags.get("server_name", metric["server_name"]).lower()
+                        server_n = tags.get('server_name', metric['server_name']).lower()
-                        tags["server_name"] = server_n or "unknown"
+                        tags['server_name'] = server_n or 'unknown'
                         new_metric = Metric(
-                            timestamp=metric["timestamp"],
+                            timestamp=metric['timestamp'],
                             resource_id=resource.resource_id,
-                            namespace=metric["namespace"],
+                            namespace=metric['namespace'],
-                            tags=tags,
+                            tags=tags)
                         rows.append(new_metric)
                         es_docs.append(new_metric.es_doc())
                     session = DBSession()
                     session.bulk_save_objects(rows)
                     session.flush()
-                    action = "METRICS"
+                    action = 'METRICS'
-                    metrics_msg = "%s: %s, metrics: %s, proto:%s" % (
+                    metrics_msg = '%s: %s, metrics: %s, proto:%s' % (
                         action,
                         str(resource),
                         len(dataset),
-                        proto_version,
+                        proto_version
                     )
                     log.info(metrics_msg)
                     mark_changed(session)
                     redis_pipeline = Datastores.redis.pipeline(transaction=False)
-                    key = REDIS_KEYS["counters"]["metrics_per_minute"].format(current_time)
+                    key = REDIS_KEYS['counters']['metrics_per_minute'].format(current_time)
                     redis_pipeline.incr(key, len(rows))
                     redis_pipeline.expire(key, 3600 * 24)
-                    key = REDIS_KEYS["counters"]["events_per_minute_per_user"].format(
+                    key = REDIS_KEYS['counters']['events_per_minute_per_user'].format(
-                        resource.owner_user_id, current_time
+                        resource.owner_user_id, current_time)
                     redis_pipeline.incr(key, len(rows))
                     redis_pipeline.expire(key, 3600)
-                    key = REDIS_KEYS["counters"]["metrics_per_hour_per_app"].format(
+                    key = REDIS_KEYS['counters']['metrics_per_hour_per_app'].format(
-                        resource_id, current_time.replace(minute=0)
+                        resource_id, current_time.replace(minute=0))
                     redis_pipeline.incr(key, len(rows))
                     redis_pipeline.expire(key, 3600 * 24 * 7)
                     redis_pipeline.sadd(
-                        REDIS_KEYS["apps_that_got_new_data_per_hour"].format(
+                        REDIS_KEYS['apps_that_got_new_data_per_hour'].format(
-                            current_time.replace(minute=0)
+                            current_time.replace(minute=0)), resource_id)
-                        ),
-                        resource_id,
                     redis_pipeline.execute()
                     add_metrics_es(es_docs)
                     return True
                 except Exception as exc:
                     print_traceback(log)
-                    if celery.conf["CELERY_EAGER_PROPAGATES_EXCEPTIONS"]:
-                        raise
                     add_metrics.retry(exc=exc)
             @celery.task(queue="es", default_retry_delay=600, max_retries=144)
             def add_metrics_es(es_docs):
                 for doc in es_docs:
-                    partition = "rcae_m_%s" % doc["timestamp"].strftime("%Y_%m_%d")
+                    partition = 'rcae_m_%s' % doc['timestamp'].strftime('%Y_%m_%d')
-                    Datastores.es.index(partition, "log", doc)
+                    Datastores.es.index(partition, 'log', doc)
             @celery.task(queue="default", default_retry_delay=5, max_retries=2)
             def check_user_report_notifications(resource_id):
                 since_when = datetime.utcnow()
                 try:
                     request = get_current_request()
                     application = ApplicationService.by_id(resource_id)
                     if not application:
                         return
-                    error_key = REDIS_KEYS["reports_to_notify_per_type_per_app"].format(
+                    error_key = REDIS_KEYS['reports_to_notify_per_type_per_app'].format(
-                        ReportType.error, resource_id
+                        ReportType.error, resource_id)
+                    slow_key = REDIS_KEYS['reports_to_notify_per_type_per_app'].format(
-                    slow_key = REDIS_KEYS["reports_to_notify_per_type_per_app"].format(
+                        ReportType.slow, resource_id)
-                        ReportType.slow, resource_id
                     error_group_ids = Datastores.redis.smembers(error_key)
                     slow_group_ids = Datastores.redis.smembers(slow_key)
                     Datastores.redis.delete(error_key)
                     Datastores.redis.delete(slow_key)
                     err_gids = [int(g_id) for g_id in error_group_ids]
                     slow_gids = [int(g_id) for g_id in list(slow_group_ids)]
                     group_ids = err_gids + slow_gids
                     occurence_dict = {}
                     for g_id in group_ids:
-                        key = REDIS_KEYS["counters"]["report_group_occurences"].format(g_id)
+                        key = REDIS_KEYS['counters']['report_group_occurences'].format(
+                            g_id)
                         val = Datastores.redis.get(key)
                         Datastores.redis.delete(key)
                         if val:
                             occurence_dict[g_id] = int(val)
                         else:
                             occurence_dict[g_id] = 1
                     report_groups = ReportGroupService.by_ids(group_ids)
                     report_groups.options(sa.orm.joinedload(ReportGroup.last_report_ref))
                     ApplicationService.check_for_groups_alert(
-                        application,
+                        application, 'alert', report_groups=report_groups,
-                        "alert",
+                        occurence_dict=occurence_dict)
-                        report_groups=report_groups,
+                    users = set([p.user for p in application.users_for_perm('view')])
-                        occurence_dict=occurence_dict,
-                    users = set(
-                        [p.user for p in ResourceService.users_for_perm(application, "view")]
                     report_groups = report_groups.all()
                     for user in users:
-                        UserService.report_notify(
+                        UserService.report_notify(user, request, application,
-                            user,
-                            request,
-                            application,
                                                   report_groups=report_groups,
-                            occurence_dict=occurence_dict,
+                                                  occurence_dict=occurence_dict)
                     for group in report_groups:
                         # marks report_groups as notified
                         if not group.notified:
                             group.notified = True
                 except Exception as exc:
                     print_traceback(log)
                     raise
             @celery.task(queue="default", default_retry_delay=5, max_retries=2)
             def check_alerts(resource_id):
                 since_when = datetime.utcnow()
                 try:
                     request = get_current_request()
                     application = ApplicationService.by_id(resource_id)
                     if not application:
                         return
-                    error_key = REDIS_KEYS["reports_to_notify_per_type_per_app_alerting"].format(
+                    error_key = REDIS_KEYS[
-                        ReportType.error, resource_id
+                        'reports_to_notify_per_type_per_app_alerting'].format(
+                        ReportType.error, resource_id)
-                    slow_key = REDIS_KEYS["reports_to_notify_per_type_per_app_alerting"].format(
+                    slow_key = REDIS_KEYS[
-                        ReportType.slow, resource_id
+                        'reports_to_notify_per_type_per_app_alerting'].format(
+                        ReportType.slow, resource_id)
                     error_group_ids = Datastores.redis.smembers(error_key)
                     slow_group_ids = Datastores.redis.smembers(slow_key)
                     Datastores.redis.delete(error_key)
                     Datastores.redis.delete(slow_key)
                     err_gids = [int(g_id) for g_id in error_group_ids]
                     slow_gids = [int(g_id) for g_id in list(slow_group_ids)]
                     group_ids = err_gids + slow_gids
                     occurence_dict = {}
                     for g_id in group_ids:
-                        key = REDIS_KEYS["counters"]["report_group_occurences_alerting"].format(
+                        key = REDIS_KEYS['counters'][
-                            g_id
+                            'report_group_occurences_alerting'].format(
+                            g_id)
                         val = Datastores.redis.get(key)
                         Datastores.redis.delete(key)
                         if val:
                             occurence_dict[g_id] = int(val)
                         else:
                             occurence_dict[g_id] = 1
                     report_groups = ReportGroupService.by_ids(group_ids)
                     report_groups.options(sa.orm.joinedload(ReportGroup.last_report_ref))
                     ApplicationService.check_for_groups_alert(
-                        application,
+                        application, 'alert', report_groups=report_groups,
-                        "alert",
+                        occurence_dict=occurence_dict, since_when=since_when)
-                        report_groups=report_groups,
-                        occurence_dict=occurence_dict,
-                        since_when=since_when,
                 except Exception as exc:
                     print_traceback(log)
                     raise
             @celery.task(queue="default", default_retry_delay=1, max_retries=2)
             def close_alerts():
-                log.warning("Checking alerts")
+                log.warning('Checking alerts')
                 since_when = datetime.utcnow()
                 try:
-                    event_types = [
+                    event_types = [Event.types['error_report_alert'],
-                        Event.types["error_report_alert"],
+                                   Event.types['slow_report_alert'], ]
-                        Event.types["slow_report_alert"],
+                    statuses = [Event.statuses['active']]
-                    statuses = [Event.statuses["active"]]
                     # get events older than 5 min
                     events = EventService.by_type_and_status(
-                        event_types, statuses, older_than=(since_when - timedelta(minutes=5))
+                        event_types,
+                        statuses,
+                        older_than=(since_when - timedelta(minutes=5)))
                     for event in events:
                         # see if we can close them
-                        event.validate_or_close(since_when=(since_when - timedelta(minutes=1)))
+                        event.validate_or_close(
+                            since_when=(since_when - timedelta(minutes=1)))
                 except Exception as exc:
                     print_traceback(log)
                     raise
             @celery.task(queue="default", default_retry_delay=600, max_retries=144)
             def update_tag_counter(tag_name, tag_value, count):
                 try:
-                    query = (
+                    query = DBSession.query(Tag).filter(Tag.name == tag_name).filter(
-                        DBSession.query(Tag)
+                        sa.cast(Tag.value, sa.types.TEXT) == sa.cast(json.dumps(tag_value),
-                        .filter(Tag.name == tag_name)
+                                                                     sa.types.TEXT))
-                        .filter(
+                    query.update({'times_seen': Tag.times_seen + count,
-                            sa.cast(Tag.value, sa.types.TEXT)
+                                  'last_timestamp': datetime.utcnow()},
-                            == sa.cast(json.dumps(tag_value), sa.types.TEXT)
+                                 synchronize_session=False)
-                    query.update(
-                        {"times_seen": Tag.times_seen + count, "last_timestamp": datetime.utcnow()},
-                        synchronize_session=False,
                     session = DBSession()
                     mark_changed(session)
                     return True
                 except Exception as exc:
                     print_traceback(log)
-                    if celery.conf["CELERY_EAGER_PROPAGATES_EXCEPTIONS"]:
-                        raise
                     update_tag_counter.retry(exc=exc)
             @celery.task(queue="default")
             def update_tag_counters():
                 """
                 Sets task to update counters for application tags
                 """
-                tags = Datastores.redis.lrange(REDIS_KEYS["seen_tag_list"], 0, -1)
+                tags = Datastores.redis.lrange(REDIS_KEYS['seen_tag_list'], 0, -1)
-                Datastores.redis.delete(REDIS_KEYS["seen_tag_list"])
+                Datastores.redis.delete(REDIS_KEYS['seen_tag_list'])
                 c = collections.Counter(tags)
                 for t_json, count in c.items():
                     tag_info = json.loads(t_json)
                     update_tag_counter.delay(tag_info[0], tag_info[1], count)
             @celery.task(queue="default")
             def daily_digest():
                 """
                 Sends daily digest with top 50 error reports
                 """
                 request = get_current_request()
-                apps = Datastores.redis.smembers(REDIS_KEYS["apps_that_had_reports"])
+                apps = Datastores.redis.smembers(REDIS_KEYS['apps_that_had_reports'])
-                Datastores.redis.delete(REDIS_KEYS["apps_that_had_reports"])
+                Datastores.redis.delete(REDIS_KEYS['apps_that_had_reports'])
                 since_when = datetime.utcnow() - timedelta(hours=8)
-                log.warning("Generating daily digests")
+                log.warning('Generating daily digests')
                 for resource_id in apps:
-                    resource_id = resource_id.decode("utf8")
+                    resource_id = resource_id.decode('utf8')
                     end_date = datetime.utcnow().replace(microsecond=0, second=0)
-                    filter_settings = {
+                    filter_settings = {'resource': [resource_id],
-                        "resource": [resource_id],
+                                       'tags': [{'name': 'type',
-                        "tags": [{"name": "type", "value": ["error"], "op": None}],
+                                                 'value': ['error'], 'op': None}],
-                        "type": "error",
+                                       'type': 'error', 'start_date': since_when,
-                        "start_date": since_when,
+                                       'end_date': end_date}
-                        "end_date": end_date,
                     reports = ReportGroupService.get_trending(
-                        request, filter_settings=filter_settings, limit=50
+                        request, filter_settings=filter_settings, limit=50)
                     application = ApplicationService.by_id(resource_id)
                     if application:
-                        users = set(
+                        users = set([p.user for p in application.users_for_perm('view')])
-                            [p.user for p in ResourceService.users_for_perm(application, "view")]
                         for user in users:
-                            user.send_digest(
+                            user.send_digest(request, application, reports=reports,
-                                request, application, reports=reports, since_when=since_when
+                                             since_when=since_when)
             @celery.task(queue="default")
             def notifications_reports():
                 """
                 Loop that checks redis for info and then issues new tasks to celery to
                 issue notifications
                 """
-                apps = Datastores.redis.smembers(REDIS_KEYS["apps_that_had_reports"])
+                apps = Datastores.redis.smembers(REDIS_KEYS['apps_that_had_reports'])
-                Datastores.redis.delete(REDIS_KEYS["apps_that_had_reports"])
+                Datastores.redis.delete(REDIS_KEYS['apps_that_had_reports'])
                 for app in apps:
-                    log.warning("Notify for app: %s" % app)
+                    log.warning('Notify for app: %s' % app)
-                    check_user_report_notifications.delay(app.decode("utf8"))
+                    check_user_report_notifications.delay(app.decode('utf8'))
             @celery.task(queue="default")
             def alerting_reports():
                 """
                 Loop that checks redis for info and then issues new tasks to celery to
                 perform the following:
                 - which applications should have new alerts opened
                 """
-                apps = Datastores.redis.smembers(REDIS_KEYS["apps_that_had_reports_alerting"])
+                apps = Datastores.redis.smembers(REDIS_KEYS['apps_that_had_reports_alerting'])
-                Datastores.redis.delete(REDIS_KEYS["apps_that_had_reports_alerting"])
+                Datastores.redis.delete(REDIS_KEYS['apps_that_had_reports_alerting'])
                 for app in apps:
-                    log.warning("Notify for app: %s" % app)
+                    log.warning('Notify for app: %s' % app)
-                    check_alerts.delay(app.decode("utf8"))
+                    check_alerts.delay(app.decode('utf8'))
-            @celery.task(
+            @celery.task(queue="default", soft_time_limit=3600 * 4,
-                queue="default", soft_time_limit=3600 * 4, hard_time_limit=3600 * 4, max_retries=144
+                         hard_time_limit=3600 * 4, max_retries=144)
             def logs_cleanup(resource_id, filter_settings):
                 request = get_current_request()
                 request.tm.begin()
-                es_query = {"query": {"bool": {"filter": [{"term": {"resource_id": resource_id}}]}}}
+                es_query = {
+                    "_source": False,
+                    "size": 5000,
+                    "query": {
+                        "filtered": {
+                            "filter": {
+                                "and": [{"term": {"resource_id": resource_id}}]
+                            }
+                        }
+                    }
+                }
                 query = DBSession.query(Log).filter(Log.resource_id == resource_id)
-                if filter_settings["namespace"]:
+                if filter_settings['namespace']:
-                    query = query.filter(Log.namespace == filter_settings["namespace"][0])
+                    query = query.filter(Log.namespace == filter_settings['namespace'][0])
-                    es_query["query"]["bool"]["filter"].append(
+                    es_query['query']['filtered']['filter']['and'].append(
-                        {"term": {"namespace": filter_settings["namespace"][0]}}
+                        {"term": {"namespace": filter_settings['namespace'][0]}}
                     )
                 query.delete(synchronize_session=False)
                 request.tm.commit()
-                Datastores.es.delete_by_query(
+                result = request.es_conn.search(es_query, index='rcae_l_*',
-                    index="rcae_l_*", doc_type="log", body=es_query, conflicts="proceed"
+                                                doc_type='log', es_scroll='1m',
+                                                es_search_type='scan')
+                scroll_id = result['_scroll_id']
+                while True:
+                    log.warning('log_cleanup, app:{} ns:{} batch'.format(
+                        resource_id,
+                        filter_settings['namespace']
+                    ))
+                    es_docs_to_delete = []
+                    result = request.es_conn.send_request(
+                        'POST', ['_search', 'scroll'],
+                        body=scroll_id, query_params={"scroll": '1m'})
+                    scroll_id = result['_scroll_id']
+                    if not result['hits']['hits']:
+                        break
+                    for doc in result['hits']['hits']:
+                        es_docs_to_delete.append({"id": doc['_id'],
+                                                  "index": doc['_index']})
+                    for batch in in_batches(es_docs_to_delete, 10):
+                        Datastores.es.bulk([Datastores.es.delete_op(doc_type='log',
+                                                                    **to_del)
+                                            for to_del in batch])

backend/src/appenlight/fil.py

0 +1 -2

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             def filter_callable(structure, section=None):
-                structure["SOMEVAL"] = "***REMOVED***"
+                structure['SOMEVAL'] = '***REMOVED***'
                 return structure

backend/src/appenlight/forms.py

0 +365 -450

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import wtforms
             import formencode
             import re
             import pyramid.threadlocal
             import datetime
             import appenlight.lib.helpers as h
-            from ziggurat_foundations.models.services.user import UserService
+            from appenlight.models.user import User
-            from ziggurat_foundations.models.services.group import GroupService
+            from appenlight.models.group import Group
             from appenlight.models import DBSession
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.integrations import IntegrationException
             from appenlight.models.integrations.campfire import CampfireIntegration
             from appenlight.models.integrations.bitbucket import BitbucketIntegration
             from appenlight.models.integrations.github import GithubIntegration
             from appenlight.models.integrations.flowdock import FlowdockIntegration
             from appenlight.models.integrations.hipchat import HipchatIntegration
             from appenlight.models.integrations.jira import JiraClient
             from appenlight.models.integrations.slack import SlackIntegration
             from appenlight.lib.ext_json import json
             from wtforms.ext.csrf.form import SecureForm
             from wtforms.compat import iteritems
             from collections import defaultdict
             _ = str
             strip_filter = lambda x: x.strip() if x else None
             uppercase_filter = lambda x: x.upper() if x else None
-            FALSE_VALUES = ("false", "", False, None)
+            FALSE_VALUES = ('false', '', False, None)
             class CSRFException(Exception):
                 pass
             class ReactorForm(SecureForm):
-                def __init__(self, formdata=None, obj=None, prefix="", csrf_context=None, **kwargs):
+                def __init__(self, formdata=None, obj=None, prefix='', csrf_context=None,
-                    super(ReactorForm, self).__init__(
+                             **kwargs):
-                        formdata=formdata,
+                    super(ReactorForm, self).__init__(formdata=formdata, obj=obj,
-                        obj=obj,
                                                       prefix=prefix,
-                        csrf_context=csrf_context,
+                                                      csrf_context=csrf_context, **kwargs)
-                        **kwargs
                     self._csrf_context = csrf_context
                 def generate_csrf_token(self, csrf_context):
                     return csrf_context.session.get_csrf_token()
                 def validate_csrf_token(self, field):
                     request = self._csrf_context or pyramid.threadlocal.get_current_request()
-                    is_from_auth_token = "auth:auth_token" in request.effective_principals
+                    is_from_auth_token = 'auth:auth_token' in request.effective_principals
                     if is_from_auth_token:
                         return True
                     if field.data != field.current_token:
                         # try to save the day by using token from angular
-                        if request.headers.get("X-XSRF-TOKEN") != field.current_token:
+                        if request.headers.get('X-XSRF-TOKEN') != field.current_token:
-                            raise CSRFException("Invalid CSRF token")
+                            raise CSRFException('Invalid CSRF token')
                 @property
                 def errors_dict(self):
                     r_dict = defaultdict(list)
                     for k, errors in self.errors.items():
                         r_dict[k].extend([str(e) for e in errors])
                     return r_dict
                 @property
                 def errors_json(self):
                     return json.dumps(self.errors_dict)
                 def populate_obj(self, obj, ignore_none=False):
                     """
                     Populates the attributes of the passed `obj` with data from the form's
                     fields.
                     :note: This is a destructive operation; Any attribute with the same name
                            as a field will be overridden. Use with caution.
                     """
                     if ignore_none:
                         for name, field in iteritems(self._fields):
                             if field.data is not None:
                                 field.populate_obj(obj, name)
                     else:
                         for name, field in iteritems(self._fields):
                             field.populate_obj(obj, name)
                 css_classes = {}
                 ignore_labels = {}
             class SignInForm(ReactorForm):
                 came_from = wtforms.HiddenField()
-                sign_in_user_name = wtforms.StringField(_("User Name"))
+                sign_in_user_name = wtforms.StringField(_('User Name'))
-                sign_in_user_password = wtforms.PasswordField(_("Password"))
+                sign_in_user_password = wtforms.PasswordField(_('Password'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
-                html_attrs = {
+                html_attrs = {'sign_in_user_name': {'placeholder': 'Your login'},
-                    "sign_in_user_name": {"placeholder": "Your login"},
+                              'sign_in_user_password': {
-                    "sign_in_user_password": {"placeholder": "Your password"},
+                                  'placeholder': 'Your password'}}
             from wtforms.widgets import html_params, HTMLString
-            def select_multi_checkbox(field, ul_class="set", **kwargs):
+            def select_multi_checkbox(field, ul_class='set', **kwargs):
                 """Render a multi-checkbox widget"""
-                kwargs.setdefault("type", "checkbox")
+                kwargs.setdefault('type', 'checkbox')
-                field_id = kwargs.pop("id", field.id)
+                field_id = kwargs.pop('id', field.id)
-                html = ["<ul %s>" % html_params(id=field_id, class_=ul_class)]
+                html = ['<ul %s>' % html_params(id=field_id, class_=ul_class)]
                 for value, label, checked in field.iter_choices():
-                    choice_id = "%s-%s" % (field_id, value)
+                    choice_id = '%s-%s' % (field_id, value)
                     options = dict(kwargs, name=field.name, value=value, id=choice_id)
                     if checked:
-                        options["checked"] = "checked"
+                        options['checked'] = 'checked'
-                    html.append("<li><input %s /> " % html_params(**options))
+                    html.append('<li><input %s /> ' % html_params(**options))
                     html.append('<label for="%s">%s</label></li>' % (choice_id, label))
-                html.append("</ul>")
+                html.append('</ul>')
-                return HTMLString("".join(html))
+                return HTMLString(''.join(html))
-            def button_widget(field, button_cls="ButtonField btn btn-default", **kwargs):
+            def button_widget(field, button_cls='ButtonField btn btn-default', **kwargs):
                 """Render a button widget"""
-                kwargs.setdefault("type", "button")
+                kwargs.setdefault('type', 'button')
-                field_id = kwargs.pop("id", field.id)
+                field_id = kwargs.pop('id', field.id)
-                kwargs.setdefault("value", field.label.text)
+                kwargs.setdefault('value', field.label.text)
-                html = [
+                html = ['<button %s>%s</button>' % (html_params(id=field_id,
-                    "<button %s>%s</button>"
+                                                                 class_=button_cls),
-                    % (html_params(id=field_id, class_=button_cls), kwargs["value"])
+                                                     kwargs['value'],)]
+                return HTMLString(''.join(html))
-                return HTMLString("".join(html))
             def clean_whitespace(value):
                 if value:
                     return value.strip()
                 return value
             def found_username_validator(form, field):
-                user = UserService.by_user_name(field.data)
+                user = User.by_user_name(field.data)
                 # sets user to recover in email validator
                 form.field_user = user
                 if not user:
-                    raise wtforms.ValidationError("This username does not exist")
+                    raise wtforms.ValidationError('This username does not exist')
             def found_username_email_validator(form, field):
-                user = UserService.by_email(field.data)
+                user = User.by_email(field.data)
                 if not user:
-                    raise wtforms.ValidationError("Email is incorrect")
+                    raise wtforms.ValidationError('Email is incorrect')
             def unique_username_validator(form, field):
-                user = UserService.by_user_name(field.data)
+                user = User.by_user_name(field.data)
                 if user:
-                    raise wtforms.ValidationError("This username already exists in system")
+                    raise wtforms.ValidationError('This username already exists in system')
             def unique_groupname_validator(form, field):
-                group = GroupService.by_group_name(field.data)
+                group = Group.by_group_name(field.data)
-                mod_group = getattr(form, "_modified_group", None)
+                mod_group = getattr(form, '_modified_group', None)
                 if group and (not mod_group or mod_group.id != group.id):
-                    raise wtforms.ValidationError("This group name already exists in system")
+                    raise wtforms.ValidationError(
+                        'This group name already exists in system')
             def unique_email_validator(form, field):
-                user = UserService.by_email(field.data)
+                user = User.by_email(field.data)
                 if user:
-                    raise wtforms.ValidationError("This email already exists in system")
+                    raise wtforms.ValidationError('This email already exists in system')
             def email_validator(form, field):
                 validator = formencode.validators.Email()
                 try:
                     validator.to_python(field.data)
                 except formencode.Invalid as e:
                     raise wtforms.ValidationError(e)
             def unique_alert_email_validator(form, field):
                 q = DBSession.query(AlertChannel)
-                q = q.filter(AlertChannel.channel_name == "email")
+                q = q.filter(AlertChannel.channel_name == 'email')
                 q = q.filter(AlertChannel.channel_value == field.data)
                 email = q.first()
                 if email:
-                    raise wtforms.ValidationError("This email already exists in alert system")
+                    raise wtforms.ValidationError(
+                        'This email already exists in alert system')
             def blocked_email_validator(form, field):
                 blocked_emails = [
-                    "goood-mail.org",
+                    'goood-mail.org',
-                    "shoeonlineblog.com",
+                    'shoeonlineblog.com',
-                    "louboutinemart.com",
+                    'louboutinemart.com',
-                    "guccibagshere.com",
+                    'guccibagshere.com',
-                    "nikeshoesoutletforsale.com",
+                    'nikeshoesoutletforsale.com'
                 ]
-                data = field.data or ""
+                data = field.data or ''
-                domain = data.split("@")[-1]
+                domain = data.split('@')[-1]
                 if domain in blocked_emails:
-                    raise wtforms.ValidationError("Don't spam")
+                    raise wtforms.ValidationError('Don\'t spam')
             def old_password_validator(form, field):
-                if not UserService.check_password(field.user, field.data or ""):
+                if not field.user.check_password(field.data or ''):
-                    raise wtforms.ValidationError("You need to enter correct password")
+                    raise wtforms.ValidationError('You need to enter correct password')
             class UserRegisterForm(ReactorForm):
                 user_name = wtforms.StringField(
-                    _("User Name"),
+                    _('User Name'),
                     filters=[strip_filter],
                     validators=[
                         wtforms.validators.Length(min=2, max=30),
                         wtforms.validators.Regexp(
-                            re.compile(r"^[\.\w-]+$", re.UNICODE), message="Invalid characters used"
+                            re.compile(r'^[\.\w-]+$', re.UNICODE),
-                        ),
+                            message="Invalid characters used"),
                         unique_username_validator,
-                        wtforms.validators.DataRequired(),
+                        wtforms.validators.DataRequired()
-                    ],
+                    ])
-                user_password = wtforms.PasswordField(
+                user_password = wtforms.PasswordField(_('User Password'),
-                    _("User Password"),
                                                       filters=[strip_filter],
                                                       validators=[
                                                           wtforms.validators.Length(min=4),
-                        wtforms.validators.DataRequired(),
+                                                          wtforms.validators.DataRequired()
-                    ],
+                                                      ])
-                email = wtforms.StringField(
+                email = wtforms.StringField(_('Email Address'),
-                    _("Email Address"),
                                             filters=[strip_filter],
-                    validators=[
+                                            validators=[email_validator,
-                        email_validator,
                                                         unique_email_validator,
                                                         blocked_email_validator,
-                        wtforms.validators.DataRequired(),
+                                                        wtforms.validators.DataRequired()])
-                    ],
+                first_name = wtforms.HiddenField(_('First Name'))
+                last_name = wtforms.HiddenField(_('Last Name'))
-                first_name = wtforms.HiddenField(_("First Name"))
-                last_name = wtforms.HiddenField(_("Last Name"))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
-                html_attrs = {
+                html_attrs = {'user_name': {'placeholder': 'Your login'},
-                    "user_name": {"placeholder": "Your login"},
+                              'user_password': {'placeholder': 'Your password'},
-                    "user_password": {"placeholder": "Your password"},
+                              'email': {'placeholder': 'Your email'}}
-                    "email": {"placeholder": "Your email"},
             class UserCreateForm(UserRegisterForm):
-                status = wtforms.BooleanField("User status", false_values=FALSE_VALUES)
+                status = wtforms.BooleanField('User status',
+                                              false_values=FALSE_VALUES)
             class UserUpdateForm(UserCreateForm):
                 user_name = None
-                user_password = wtforms.PasswordField(
+                user_password = wtforms.PasswordField(_('User Password'),
-                    _("User Password"),
                                                       filters=[strip_filter],
-                    validators=[wtforms.validators.Length(min=4), wtforms.validators.Optional()],
+                                                      validators=[
+                                                          wtforms.validators.Length(min=4),
-                email = wtforms.StringField(
+                                                          wtforms.validators.Optional()
-                    _("Email Address"),
+                                                      ])
+                email = wtforms.StringField(_('Email Address'),
                                             filters=[strip_filter],
-                    validators=[email_validator, wtforms.validators.DataRequired()],
+                                            validators=[email_validator,
+                                                        wtforms.validators.DataRequired()])
             class LostPasswordForm(ReactorForm):
-                email = wtforms.StringField(
+                email = wtforms.StringField(_('Email Address'),
-                    _("Email Address"),
                                             filters=[strip_filter],
-                    validators=[
+                                            validators=[email_validator,
-                        email_validator,
                                                         found_username_email_validator,
-                        wtforms.validators.DataRequired(),
+                                                        wtforms.validators.DataRequired()])
-                    ],
-                submit = wtforms.SubmitField(_("Reset password"))
+                submit = wtforms.SubmitField(_('Reset password'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
             class ChangePasswordForm(ReactorForm):
                 old_password = wtforms.PasswordField(
-                    "Old Password",
+                    'Old Password',
                     filters=[strip_filter],
-                    validators=[old_password_validator, wtforms.validators.DataRequired()],
+                    validators=[old_password_validator,
+                                wtforms.validators.DataRequired()])
                 new_password = wtforms.PasswordField(
-                    "New Password",
+                    'New Password',
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.Length(min=4),
-                        wtforms.validators.Length(min=4),
+                                wtforms.validators.DataRequired()])
-                        wtforms.validators.DataRequired(),
-                    ],
                 new_password_confirm = wtforms.PasswordField(
-                    "Confirm Password",
+                    'Confirm Password',
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.EqualTo('new_password'),
-                        wtforms.validators.EqualTo("new_password"),
+                                wtforms.validators.DataRequired()])
-                        wtforms.validators.DataRequired(),
+                submit = wtforms.SubmitField('Change Password')
-                    ],
+                ignore_labels = ['submit']
+                css_classes = {'submit': 'btn btn-primary'}
-                submit = wtforms.SubmitField("Change Password")
-                ignore_labels = ["submit"]
-                css_classes = {"submit": "btn btn-primary"}
             class CheckPasswordForm(ReactorForm):
                 password = wtforms.PasswordField(
-                    "Password",
+                    'Password',
                     filters=[strip_filter],
-                    validators=[old_password_validator, wtforms.validators.DataRequired()],
+                    validators=[old_password_validator,
+                                wtforms.validators.DataRequired()])
             class NewPasswordForm(ReactorForm):
                 new_password = wtforms.PasswordField(
-                    "New Password",
+                    'New Password',
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.Length(min=4),
-                        wtforms.validators.Length(min=4),
+                                wtforms.validators.DataRequired()])
-                        wtforms.validators.DataRequired(),
-                    ],
                 new_password_confirm = wtforms.PasswordField(
-                    "Confirm Password",
+                    'Confirm Password',
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.EqualTo('new_password'),
-                        wtforms.validators.EqualTo("new_password"),
+                                wtforms.validators.DataRequired()])
-                        wtforms.validators.DataRequired(),
+                submit = wtforms.SubmitField('Set Password')
-                    ],
+                ignore_labels = ['submit']
+                css_classes = {'submit': 'btn btn-primary'}
-                submit = wtforms.SubmitField("Set Password")
-                ignore_labels = ["submit"]
-                css_classes = {"submit": "btn btn-primary"}
             class CORSTextAreaField(wtforms.StringField):
                 """
                 This field represents an HTML ``<textarea>`` and can be used to take
                 multi-line input.
                 """
                 widget = wtforms.widgets.TextArea()
                 def process_formdata(self, valuelist):
                     self.data = []
                     if valuelist:
-                        data = [x.strip() for x in valuelist[0].split("\n")]
+                        data = [x.strip() for x in valuelist[0].split('\n')]
                         for d in data:
                             if not d:
                                 continue
-                            if d.startswith("www."):
+                            if d.startswith('www.'):
                                 d = d[4:]
                             if data:
                                 self.data.append(d)
                     else:
                         self.data = []
-                    self.data = "\n".join(self.data)
+                    self.data = '\n'.join(self.data)
             class ApplicationCreateForm(ReactorForm):
                 resource_name = wtforms.StringField(
-                    _("Application name"),
+                    _('Application name'),
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.Length(min=1),
-                        wtforms.validators.Length(min=1),
+                                wtforms.validators.DataRequired()])
-                        wtforms.validators.DataRequired(),
-                    ],
                 domains = CORSTextAreaField(
-                    _("Domain names for CORS headers "),
+                    _('Domain names for CORS headers '),
-                    validators=[wtforms.validators.Length(min=1), wtforms.validators.Optional()],
+                    validators=[wtforms.validators.Length(min=1),
-                    description="Required for Javascript error "
+                                wtforms.validators.Optional()],
-                    "tracking (one line one domain, skip http:// part)",
+                    description='Required for Javascript error '
+                                'tracking (one line one domain, skip http:// part)')
-                submit = wtforms.SubmitField(_("Create Application"))
+                submit = wtforms.SubmitField(_('Create Application'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
-                html_attrs = {
+                html_attrs = {'resource_name': {'placeholder': 'Application Name'},
-                    "resource_name": {"placeholder": "Application Name"},
+                              'uptime_url': {'placeholder': 'http://somedomain.com'}}
-                    "uptime_url": {"placeholder": "http://somedomain.com"},
             class ApplicationUpdateForm(ApplicationCreateForm):
                 default_grouping = wtforms.SelectField(
-                    _("Default grouping for errors"),
+                    _('Default grouping for errors'),
-                    choices=[
+                    choices=[('url_type', 'Error Type + location',),
-                        ("url_type", "Error Type + location"),
+                             ('url_traceback', 'Traceback + location',),
-                        ("url_traceback", "Traceback + location"),
+                             ('traceback_server', 'Traceback + Server',)],
-                        ("traceback_server", "Traceback + Server"),
+                    default='url_traceback')
-                    ],
-                    default="url_traceback",
                 error_report_threshold = wtforms.IntegerField(
-                    _("Alert on error reports"),
+                    _('Alert on error reports'),
                     validators=[
                         wtforms.validators.NumberRange(min=1),
-                        wtforms.validators.DataRequired(),
+                        wtforms.validators.DataRequired()
                     ],
-                    description="Application requires to send at least this amount of "
+                    description='Application requires to send at least this amount of '
-                    "error reports per minute to open alert",
+                                'error reports per minute to open alert'
                 )
                 slow_report_threshold = wtforms.IntegerField(
-                    _("Alert on slow reports"),
+                    _('Alert on slow reports'),
-                    validators=[
+                    validators=[wtforms.validators.NumberRange(min=1),
-                        wtforms.validators.NumberRange(min=1),
+                                wtforms.validators.DataRequired()],
-                        wtforms.validators.DataRequired(),
+                    description='Application requires to send at least this amount of '
-                    ],
+                                'slow reports per minute to open alert')
-                    description="Application requires to send at least this amount of "
-                    "slow reports per minute to open alert",
                 allow_permanent_storage = wtforms.BooleanField(
-                    _("Permanent logs"),
+                    _('Permanent logs'),
                     false_values=FALSE_VALUES,
-                    description=_("Allow permanent storage of logs in separate DB partitions"),
+                    description=_(
+                        'Allow permanent storage of logs in separate DB partitions'))
-                submit = wtforms.SubmitField(_("Create Application"))
+                submit = wtforms.SubmitField(_('Create Application'))
             class UserSearchSchemaForm(ReactorForm):
-                user_name = wtforms.StringField("User Name", filters=[strip_filter])
+                user_name = wtforms.StringField('User Name',
+                                                filters=[strip_filter], )
-                submit = wtforms.SubmitField(_("Search User"))
+                submit = wtforms.SubmitField(_('Search User'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
                 '<li class="user_exists"><span></span></li>'
             class YesNoForm(ReactorForm):
-                no = wtforms.SubmitField("No", default="")
+                no = wtforms.SubmitField('No', default='')
-                yes = wtforms.SubmitField("Yes", default="")
+                yes = wtforms.SubmitField('Yes', default='')
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
-            status_codes = [("", "All"), ("500", "500"), ("404", "404")]
+            status_codes = [('', 'All',), ('500', '500',), ('404', '404',)]
-            priorities = [("", "All")]
+            priorities = [('', 'All',)]
             for i in range(1, 11):
-                priorities.append((str(i), str(i)))
+                priorities.append((str(i), str(i),))
-            report_status_choices = [
-                ("", "All"),
-                ("never_reviewed", "Never revieved"),
-                ("reviewed", "Revieved"),
-                ("public", "Public"),
-                ("fixed", "Fixed"),
+            report_status_choices = [('', 'All',),
+                                     ('never_reviewed', 'Never revieved',),
+                                     ('reviewed', 'Revieved',),
+                                     ('public', 'Public',),
+                                     ('fixed', 'Fixed',), ]
-            class ReportBrowserForm(ReactorForm):
-                applications = wtforms.SelectMultipleField(
-                    "Applications", widget=select_multi_checkbox
-                http_status = wtforms.SelectField("HTTP Status", choices=status_codes)
-                priority = wtforms.SelectField("Priority", choices=priorities, default="")
-                start_date = wtforms.DateField("Start Date")
-                end_date = wtforms.DateField("End Date")
-                error = wtforms.StringField("Error")
-                url_path = wtforms.StringField("URL Path")
-                url_domain = wtforms.StringField("URL Domain")
-                report_status = wtforms.SelectField(
-                    "Report status", choices=report_status_choices, default=""
-                submit = wtforms.SubmitField(
-                    '<span class="glyphicon glyphicon-search">' "</span> Filter results",
-                    widget=button_widget,
-                ignore_labels = ["submit"]
-                css_classes = {"submit": "btn btn-primary"}
+            class ReportBrowserForm(ReactorForm):
-            slow_report_status_choices = [
+                applications = wtforms.SelectMultipleField('Applications',
-                ("", "All"),
+                                                           widget=select_multi_checkbox)
-                ("never_reviewed", "Never revieved"),
+                http_status = wtforms.SelectField('HTTP Status', choices=status_codes)
-                ("reviewed", "Revieved"),
+                priority = wtforms.SelectField('Priority', choices=priorities, default='')
-                ("public", "Public"),
+                start_date = wtforms.DateField('Start Date')
+                end_date = wtforms.DateField('End Date')
+                error = wtforms.StringField('Error')
+                url_path = wtforms.StringField('URL Path')
+                url_domain = wtforms.StringField('URL Domain')
+                report_status = wtforms.SelectField('Report status',
+                                                    choices=report_status_choices,
+                                                    default='')
+                submit = wtforms.SubmitField('<span class="glyphicon glyphicon-search">'
+                                             '</span> Filter results',
+                                             widget=button_widget)
+                ignore_labels = ['submit']
+                css_classes = {'submit': 'btn btn-primary'}
+            slow_report_status_choices = [('', 'All',),
+                                          ('never_reviewed', 'Never revieved',),
+                                          ('reviewed', 'Revieved',),
+                                          ('public', 'Public',), ]
             class BulkOperationForm(ReactorForm):
-                applications = wtforms.SelectField("Applications")
+                applications = wtforms.SelectField('Applications')
                 start_date = wtforms.DateField(
-                    "Start Date",
+                    'Start Date',
-                    default=lambda: datetime.datetime.utcnow() - datetime.timedelta(days=90),
+                    default=lambda: datetime.datetime.utcnow() - datetime.timedelta(
+                        days=90))
-                end_date = wtforms.DateField("End Date")
+                end_date = wtforms.DateField('End Date')
                 confirm = wtforms.BooleanField(
-                    "Confirm operation", validators=[wtforms.validators.DataRequired()]
+                    'Confirm operation',
+                    validators=[wtforms.validators.DataRequired()])
             class LogBrowserForm(ReactorForm):
-                applications = wtforms.SelectMultipleField(
+                applications = wtforms.SelectMultipleField('Applications',
-                    "Applications", widget=select_multi_checkbox
+                                                           widget=select_multi_checkbox)
+                start_date = wtforms.DateField('Start Date')
-                start_date = wtforms.DateField("Start Date")
+                log_level = wtforms.StringField('Log level')
-                log_level = wtforms.StringField("Log level")
+                message = wtforms.StringField('Message')
-                message = wtforms.StringField("Message")
+                namespace = wtforms.StringField('Namespace')
-                namespace = wtforms.StringField("Namespace")
                 submit = wtforms.SubmitField(
                     '<span class="glyphicon glyphicon-search"></span> Filter results',
-                    widget=button_widget,
+                    widget=button_widget)
+                ignore_labels = ['submit']
-                ignore_labels = ["submit"]
+                css_classes = {'submit': 'btn btn-primary'}
-                css_classes = {"submit": "btn btn-primary"}
             class CommentForm(ReactorForm):
-                body = wtforms.TextAreaField(
+                body = wtforms.TextAreaField('Comment', validators=[
-                    "Comment",
-                    validators=[
                     wtforms.validators.Length(min=1),
-                        wtforms.validators.DataRequired(),
+                    wtforms.validators.DataRequired()
-                    ],
+                ])
+                submit = wtforms.SubmitField('Comment', )
-                submit = wtforms.SubmitField("Comment")
+                ignore_labels = ['submit']
-                ignore_labels = ["submit"]
+                css_classes = {'submit': 'btn btn-primary'}
-                css_classes = {"submit": "btn btn-primary"}
             class EmailChannelCreateForm(ReactorForm):
-                email = wtforms.StringField(
+                email = wtforms.StringField(_('Email Address'),
-                    _("Email Address"),
                                             filters=[strip_filter],
-                    validators=[
+                                            validators=[email_validator,
-                        email_validator,
                                                         unique_alert_email_validator,
-                        wtforms.validators.DataRequired(),
+                                                        wtforms.validators.DataRequired()])
-                    ],
+                submit = wtforms.SubmitField('Add email channel', )
+                ignore_labels = ['submit']
-                submit = wtforms.SubmitField("Add email channel")
+                css_classes = {'submit': 'btn btn-primary'}
-                ignore_labels = ["submit"]
-                css_classes = {"submit": "btn btn-primary"}
             def gen_user_profile_form():
                 class UserProfileForm(ReactorForm):
                     email = wtforms.StringField(
-                        _("Email Address"),
+                        _('Email Address'),
-                        validators=[email_validator, wtforms.validators.DataRequired()],
+                        validators=[email_validator, wtforms.validators.DataRequired()])
+                    first_name = wtforms.StringField(_('First Name'))
-                    first_name = wtforms.StringField(_("First Name"))
+                    last_name = wtforms.StringField(_('Last Name'))
-                    last_name = wtforms.StringField(_("Last Name"))
+                    company_name = wtforms.StringField(_('Company Name'))
-                    company_name = wtforms.StringField(_("Company Name"))
+                    company_address = wtforms.TextAreaField(_('Company Address'))
-                    company_address = wtforms.TextAreaField(_("Company Address"))
+                    zip_code = wtforms.StringField(_('ZIP code'))
-                    zip_code = wtforms.StringField(_("ZIP code"))
+                    city = wtforms.StringField(_('City'))
-                    city = wtforms.StringField(_("City"))
+                    notifications = wtforms.BooleanField('Account notifications',
-                    notifications = wtforms.BooleanField(
+                                                         false_values=FALSE_VALUES)
-                        "Account notifications", false_values=FALSE_VALUES
+                    submit = wtforms.SubmitField(_('Update Account'))
+                    ignore_labels = ['submit']
-                    submit = wtforms.SubmitField(_("Update Account"))
+                    css_classes = {'submit': 'btn btn-primary'}
-                    ignore_labels = ["submit"]
-                    css_classes = {"submit": "btn btn-primary"}
                 return UserProfileForm
             class PurgeAppForm(ReactorForm):
                 resource_id = wtforms.HiddenField(
-                    "App Id", validators=[wtforms.validators.DataRequired()]
+                    'App Id',
+                    validators=[wtforms.validators.DataRequired()])
-                days = wtforms.IntegerField("Days", validators=[wtforms.validators.DataRequired()])
+                days = wtforms.IntegerField(
+                    'Days',
+                    validators=[wtforms.validators.DataRequired()])
                 password = wtforms.PasswordField(
-                    "Admin Password",
+                    'Admin Password',
-                    validators=[old_password_validator, wtforms.validators.DataRequired()],
+                    validators=[old_password_validator, wtforms.validators.DataRequired()])
+                submit = wtforms.SubmitField(_('Purge Data'))
-                submit = wtforms.SubmitField(_("Purge Data"))
+                ignore_labels = ['submit']
-                ignore_labels = ["submit"]
+                css_classes = {'submit': 'btn btn-primary'}
-                css_classes = {"submit": "btn btn-primary"}
             class IntegrationRepoForm(ReactorForm):
-                host_name = wtforms.StringField("Service Host", default="")
+                host_name = wtforms.StringField("Service Host", default='')
                 user_name = wtforms.StringField(
                     "User Name",
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.DataRequired(),
-                        wtforms.validators.DataRequired(),
+                                wtforms.validators.Length(min=1)])
-                        wtforms.validators.Length(min=1),
-                    ],
                 repo_name = wtforms.StringField(
                     "Repo Name",
                     filters=[strip_filter],
-                    validators=[
+                    validators=[wtforms.validators.DataRequired(),
-                        wtforms.validators.DataRequired(),
+                                wtforms.validators.Length(min=1)])
-                        wtforms.validators.Length(min=1),
-                    ],
             class IntegrationBitbucketForm(IntegrationRepoForm):
-                host_name = wtforms.StringField("Service Host", default="https://bitbucket.org")
+                host_name = wtforms.StringField("Service Host",
+                                                default='https://bitbucket.org')
                 def validate_user_name(self, field):
                     try:
                         request = pyramid.threadlocal.get_current_request()
                         client = BitbucketIntegration.create_client(
-                            request, self.user_name.data, self.repo_name.data
+                            request,
+                            self.user_name.data,
+                            self.repo_name.data)
                         client.get_assignees()
                     except IntegrationException as e:
                         raise wtforms.validators.ValidationError(str(e))
             class IntegrationGithubForm(IntegrationRepoForm):
-                host_name = wtforms.StringField("Service Host", default="https://github.com")
+                host_name = wtforms.StringField("Service Host",
+                                                default='https://github.com')
                 def validate_user_name(self, field):
                     try:
                         request = pyramid.threadlocal.get_current_request()
                         client = GithubIntegration.create_client(
-                            request, self.user_name.data, self.repo_name.data
+                            request,
+                            self.user_name.data,
+                            self.repo_name.data)
                         client.get_assignees()
                     except IntegrationException as e:
                         raise wtforms.validators.ValidationError(str(e))
                         raise wtforms.validators.ValidationError(str(e))
             def filter_rooms(data):
                 if data is not None:
-                    rooms = data.split(",")
+                    rooms = data.split(',')
-                    return ",".join([r.strip() for r in rooms])
+                    return ','.join([r.strip() for r in rooms])
             class IntegrationCampfireForm(ReactorForm):
                 account = wtforms.StringField(
-                    "Account",
+                    'Account',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 api_token = wtforms.StringField(
-                    "Api Token",
+                    'Api Token',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
+                rooms = wtforms.StringField('Room ID list', filters=[filter_rooms])
-                rooms = wtforms.StringField("Room ID list", filters=[filter_rooms])
                 def validate_api_token(self, field):
                     try:
-                        client = CampfireIntegration.create_client(
+                        client = CampfireIntegration.create_client(self.api_token.data,
-                            self.api_token.data, self.account.data
+                                                                   self.account.data)
                         client.get_account()
                     except IntegrationException as e:
                         raise wtforms.validators.ValidationError(str(e))
                 def validate_rooms(self, field):
                     if not field.data:
                         return
-                    client = CampfireIntegration.create_client(
+                    client = CampfireIntegration.create_client(self.api_token.data,
-                        self.api_token.data, self.account.data
+                                                               self.account.data)
                     try:
-                        room_list = [r["id"] for r in client.get_rooms()]
+                        room_list = [r['id'] for r in client.get_rooms()]
                     except IntegrationException as e:
                         raise wtforms.validators.ValidationError(str(e))
-                    rooms = field.data.split(",")
+                    rooms = field.data.split(',')
                     if len(rooms) > 3:
-                        msg = "You can use up to 3 room ids"
+                        msg = 'You can use up to 3 room ids'
                         raise wtforms.validators.ValidationError(msg)
                     if rooms:
                         for room_id in rooms:
                             if int(room_id) not in room_list:
                                 msg = "Room %s doesn't exist"
                                 raise wtforms.validators.ValidationError(msg % room_id)
                             if not room_id.strip().isdigit():
-                                msg = "You must use only integers for room ids"
+                                msg = 'You must use only integers for room ids'
                                 raise wtforms.validators.ValidationError(msg)
-                submit = wtforms.SubmitField(_("Connect to Campfire"))
+                submit = wtforms.SubmitField(_('Connect to Campfire'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-primary"}
+                css_classes = {'submit': 'btn btn-primary'}
             def filter_rooms(data):
                 if data is not None:
-                    rooms = data.split(",")
+                    rooms = data.split(',')
-                    return ",".join([r.strip() for r in rooms])
+                    return ','.join([r.strip() for r in rooms])
             class IntegrationHipchatForm(ReactorForm):
                 api_token = wtforms.StringField(
-                    "Api Token",
+                    'Api Token',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 rooms = wtforms.StringField(
-                    "Room ID list",
+                    'Room ID list',
                     filters=[filter_rooms],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 def validate_rooms(self, field):
                     if not field.data:
                         return
                     client = HipchatIntegration.create_client(self.api_token.data)
-                    rooms = field.data.split(",")
+                    rooms = field.data.split(',')
                     if len(rooms) > 3:
-                        msg = "You can use up to 3 room ids"
+                        msg = 'You can use up to 3 room ids'
                         raise wtforms.validators.ValidationError(msg)
                     if rooms:
                         for room_id in rooms:
                             if not room_id.strip().isdigit():
-                                msg = "You must use only integers for room ids"
+                                msg = 'You must use only integers for room ids'
                                 raise wtforms.validators.ValidationError(msg)
                             try:
-                                client.send(
+                                client.send({
+                                    "message_format": 'text',
-                                        "message_format": "text",
                                     "message": "testing for room existence",
                                     "from": "AppEnlight",
                                     "room_id": room_id,
-                                        "color": "green",
+                                    "color": "green"
+                                })
                             except IntegrationException as exc:
-                                msg = "Room id: %s exception: %s"
+                                msg = 'Room id: %s exception: %s'
-                                raise wtforms.validators.ValidationError(msg % (room_id, exc))
+                                raise wtforms.validators.ValidationError(msg % (room_id,
+                                                                                exc))
             class IntegrationFlowdockForm(ReactorForm):
-                api_token = wtforms.StringField(
+                api_token = wtforms.StringField('API Token',
-                    "API Token",
                                                 filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                                                validators=[
+                                                    wtforms.validators.DataRequired()
+                                                ], )
                 def validate_api_token(self, field):
                     try:
                         client = FlowdockIntegration.create_client(self.api_token.data)
                         registry = pyramid.threadlocal.get_current_registry()
                         payload = {
-                            "source": registry.settings["mailing.from_name"],
+                            "source": registry.settings['mailing.from_name'],
-                            "from_address": registry.settings["mailing.from_email"],
+                            "from_address": registry.settings['mailing.from_email'],
                             "subject": "Integration test",
                             "content": "If you can see this it was successful",
                             "tags": ["appenlight"],
-                            "link": registry.settings["mailing.app_url"],
+                            "link": registry.settings['mailing.app_url']
                         }
                         client.send_to_inbox(payload)
                     except IntegrationException as e:
                         raise wtforms.validators.ValidationError(str(e))
             class IntegrationSlackForm(ReactorForm):
                 webhook_url = wtforms.StringField(
-                    "Reports webhook",
+                    'Reports webhook',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 def validate_webhook_url(self, field):
                     registry = pyramid.threadlocal.get_current_registry()
                     client = SlackIntegration.create_client(field.data)
-                    link = "<%s|%s>" % (
+                    link = "<%s|%s>" % (registry.settings['mailing.app_url'],
-                        registry.settings["mailing.app_url"],
+                                        registry.settings['mailing.from_name'])
-                        registry.settings["mailing.from_name"],
                     test_data = {
                         "username": "AppEnlight",
                         "icon_emoji": ":fire:",
                         "attachments": [
+                            {"fallback": "Testing integration channel: %s" % link,
-                                "fallback": "Testing integration channel: %s" % link,
                              "pretext": "Testing integration channel:  %s" % link,
                              "color": "good",
                              "fields": [
                                  {
                                      "title": "Status",
                                      "value": "Integration is working fine",
-                                        "short": False,
+                                     "short": False
                                  }
-                                ],
+                             ]}
+                        ]
-                        ],
                     }
                     try:
                         client.make_request(data=test_data)
                     except IntegrationException as exc:
                         raise wtforms.validators.ValidationError(str(exc))
             class IntegrationWebhooksForm(ReactorForm):
                 reports_webhook = wtforms.StringField(
-                    "Reports webhook",
+                    'Reports webhook',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 alerts_webhook = wtforms.StringField(
-                    "Alerts webhook",
+                    'Alerts webhook',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
+                submit = wtforms.SubmitField(_('Setup webhooks'))
-                submit = wtforms.SubmitField(_("Setup webhooks"))
+                ignore_labels = ['submit']
-                ignore_labels = ["submit"]
+                css_classes = {'submit': 'btn btn-primary'}
-                css_classes = {"submit": "btn btn-primary"}
             class IntegrationJiraForm(ReactorForm):
                 host_name = wtforms.StringField(
-                    "Server URL",
+                    'Server URL',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 user_name = wtforms.StringField(
-                    "Username",
+                    'Username',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 password = wtforms.PasswordField(
-                    "Password",
+                    'Password',
                     filters=[strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 project = wtforms.StringField(
-                    "Project key",
+                    'Project key',
                     filters=[uppercase_filter, strip_filter],
-                    validators=[wtforms.validators.DataRequired()],
+                    validators=[wtforms.validators.DataRequired()])
                 def validate_project(self, field):
                     if not field.data:
                         return
                     try:
-                        client = JiraClient(
+                        client = JiraClient(self.user_name.data,
-                            self.user_name.data,
                                             self.password.data,
                                             self.host_name.data,
-                            self.project.data,
+                                            self.project.data)
                     except Exception as exc:
                         raise wtforms.validators.ValidationError(str(exc))
                     room_list = [r.key.upper() for r in client.get_projects()]
                     if field.data.upper() not in room_list:
                         msg = "Project %s doesn\t exist in your Jira Instance"
                         raise wtforms.validators.ValidationError(msg % field.data)
             def get_deletion_form(resource):
                 class F(ReactorForm):
                     application_name = wtforms.StringField(
-                        "Application Name",
+                        'Application Name',
                         filters=[strip_filter],
-                        validators=[wtforms.validators.AnyOf([resource.resource_name])],
+                        validators=[wtforms.validators.AnyOf([resource.resource_name])])
                     resource_id = wtforms.HiddenField(default=resource.resource_id)
-                    submit = wtforms.SubmitField(_("Delete my application"))
+                    submit = wtforms.SubmitField(_('Delete my application'))
-                    ignore_labels = ["submit"]
+                    ignore_labels = ['submit']
-                    css_classes = {"submit": "btn btn-danger"}
+                    css_classes = {'submit': 'btn btn-danger'}
                 return F
             class ChangeApplicationOwnerForm(ReactorForm):
                 password = wtforms.PasswordField(
-                    "Password",
+                    'Password',
                     filters=[strip_filter],
-                    validators=[old_password_validator, wtforms.validators.DataRequired()],
+                    validators=[old_password_validator,
+                                wtforms.validators.DataRequired()])
                 user_name = wtforms.StringField(
-                    "New owners username",
+                    'New owners username',
                     filters=[strip_filter],
-                    validators=[found_username_validator, wtforms.validators.DataRequired()],
+                    validators=[found_username_validator,
+                                wtforms.validators.DataRequired()])
-                submit = wtforms.SubmitField(_("Transfer ownership of application"))
+                submit = wtforms.SubmitField(_('Transfer ownership of application'))
-                ignore_labels = ["submit"]
+                ignore_labels = ['submit']
-                css_classes = {"submit": "btn btn-danger"}
+                css_classes = {'submit': 'btn btn-danger'}
             def default_filename():
-                return "Invoice %s" % datetime.datetime.utcnow().strftime("%Y/%m")
+                return 'Invoice %s' % datetime.datetime.utcnow().strftime('%Y/%m')
             class FileUploadForm(ReactorForm):
-                title = wtforms.StringField(
+                title = wtforms.StringField('File Title',
-                    "File Title",
                                             default=default_filename,
-                    validators=[wtforms.validators.DataRequired()],
+                                            validators=[wtforms.validators.DataRequired()])
+                file = wtforms.FileField('File')
-                file = wtforms.FileField("File")
                 def validate_file(self, field):
-                    if not hasattr(field.data, "file"):
+                    if not hasattr(field.data, 'file'):
-                        raise wtforms.ValidationError("File is missing")
+                        raise wtforms.ValidationError('File is missing')
-                submit = wtforms.SubmitField(_("Upload"))
+                submit = wtforms.SubmitField(_('Upload'))
             def get_partition_deletion_form(es_indices, pg_indices):
                 class F(ReactorForm):
-                    es_index = wtforms.SelectMultipleField(
+                    es_index = wtforms.SelectMultipleField('Elasticsearch',
-                        "Elasticsearch", choices=[(ix, "") for ix in es_indices]
+                                                           choices=[(ix, '') for ix in
+                                                                    es_indices])
-                    pg_index = wtforms.SelectMultipleField(
+                    pg_index = wtforms.SelectMultipleField('pg',
-                        "pg", choices=[(ix, "") for ix in pg_indices]
+                                                           choices=[(ix, '') for ix in
+                                                                    pg_indices])
-                    confirm = wtforms.TextField(
+                    confirm = wtforms.TextField('Confirm',
-                        "Confirm",
                                                 filters=[uppercase_filter, strip_filter],
                                                 validators=[
-                            wtforms.validators.AnyOf(["CONFIRM"]),
+                                                    wtforms.validators.AnyOf(['CONFIRM']),
-                            wtforms.validators.DataRequired(),
+                                                    wtforms.validators.DataRequired()])
-                        ],
+                    ignore_labels = ['submit']
+                    css_classes = {'submit': 'btn btn-danger'}
-                    ignore_labels = ["submit"]
-                    css_classes = {"submit": "btn btn-danger"}
                 return F
             class GroupCreateForm(ReactorForm):
                 group_name = wtforms.StringField(
-                    _("Group Name"),
+                    _('Group Name'),
                     filters=[strip_filter],
                     validators=[
                         wtforms.validators.Length(min=2, max=50),
                         unique_groupname_validator,
-                        wtforms.validators.DataRequired(),
+                        wtforms.validators.DataRequired()
-                    ],
+                    ])
+                description = wtforms.StringField(_('Group description'))
-                description = wtforms.StringField(_("Group description"))
-            time_choices = [(k, v["label"]) for k, v in h.time_deltas.items()]
+            time_choices = [(k, v['label'],) for k, v in h.time_deltas.items()]
             class AuthTokenCreateForm(ReactorForm):
-                description = wtforms.StringField(_("Token description"))
+                description = wtforms.StringField(_('Token description'))
-                expires = wtforms.SelectField(
+                expires = wtforms.SelectField('Expires',
-                    "Expires",
                                               coerce=lambda x: x,
                                               choices=time_choices,
-                    validators=[wtforms.validators.Optional()],
+                                              validators=[wtforms.validators.Optional()])

backend/src/appenlight/lib/__init__.py

0 +8 -7

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             """Miscellaneous support packages for {{project}}.
             """
             import random
             import string
             import importlib
             from appenlight_client.exceptions import get_current_traceback
             def generate_random_string(chars=10):
-                return "".join(random.sample(string.ascii_letters * 2 + string.digits, chars))
+                return ''.join(random.sample(string.ascii_letters * 2 + string.digits,
+                                             chars))
             def to_integer_safe(input):
                 try:
                     return int(input)
-                except (TypeError, ValueError):
+                except (TypeError, ValueError,):
                     return None
             def print_traceback(log):
-                traceback = get_current_traceback(
+                traceback = get_current_traceback(skip=1, show_hidden_frames=True,
-                    skip=1, show_hidden_frames=True, ignore_system_exceptions=True
+                                                  ignore_system_exceptions=True)
                 exception_text = traceback.exception
                 log.error(exception_text)
                 log.error(traceback.plaintext)
                 del traceback
             def get_callable(import_string):
-                import_module, indexer_callable = import_string.split(":")
+                import_module, indexer_callable = import_string.split(':')
-                return getattr(importlib.import_module(import_module), indexer_callable)
+                return getattr(importlib.import_module(import_module),
+                               indexer_callable)

backend/src/appenlight/lib/api.py

0 +28 -27

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import datetime
             import logging
             from pyramid.httpexceptions import HTTPForbidden, HTTPTooManyRequests
+            from appenlight.models import Datastores
             from appenlight.models.services.config import ConfigService
             from appenlight.lib.redis_keys import REDIS_KEYS
             log = logging.getLogger(__name__)
             def rate_limiting(request, resource, section, to_increment=1):
                 tsample = datetime.datetime.utcnow().replace(second=0, microsecond=0)
-                key = REDIS_KEYS["rate_limits"][section].format(tsample, resource.resource_id)
+                key = REDIS_KEYS['rate_limits'][section].format(tsample,
+                                                                resource.resource_id)
                 redis_pipeline = request.registry.redis_conn.pipeline()
                 redis_pipeline.incr(key, to_increment)
                 redis_pipeline.expire(key, 3600 * 24)
                 results = redis_pipeline.execute()
                 current_count = results[0]
-                config = ConfigService.by_key_and_section(section, "global")
+                config = ConfigService.by_key_and_section(section, 'global')
                 limit = config.value if config else 1000
                 if current_count > int(limit):
-                    log.info("RATE LIMITING: {}: {}, {}".format(section, resource, current_count))
+                    log.info('RATE LIMITING: {}: {}, {}'.format(
-                    abort_msg = "Rate limits are in effect for this application"
+                        section, resource, current_count))
-                    raise HTTPTooManyRequests(abort_msg, headers={"X-AppEnlight": abort_msg})
+                    abort_msg = 'Rate limits are in effect for this application'
+                    raise HTTPTooManyRequests(abort_msg,
+                                              headers={'X-AppEnlight': abort_msg})
             def check_cors(request, application, should_return=True):
                 """
                 Performs a check and validation if request comes from authorized domain for
                 application, otherwise return 403
                 """
                 origin_found = False
-                origin = request.headers.get("Origin")
+                origin = request.headers.get('Origin')
                 if should_return:
-                    log.info("CORS for %s" % origin)
+                    log.info('CORS for %s' % origin)
                 if not origin:
                     return False
-                for domain in application.domains.split("\n"):
+                for domain in application.domains.split('\n'):
                     if domain in origin:
                         origin_found = True
                 if origin_found:
-                    request.response.headers.add("Access-Control-Allow-Origin", origin)
+                    request.response.headers.add('Access-Control-Allow-Origin', origin)
-                    request.response.headers.add("XDomainRequestAllowed", "1")
+                    request.response.headers.add('XDomainRequestAllowed', '1')
-                    request.response.headers.add(
+                    request.response.headers.add('Access-Control-Allow-Methods',
-                        "Access-Control-Allow-Methods", "GET, POST, OPTIONS"
+                                                 'GET, POST, OPTIONS')
+                    request.response.headers.add('Access-Control-Allow-Headers',
-                    request.response.headers.add(
+                                                 'Accept-Encoding, Accept-Language, '
-                        "Access-Control-Allow-Headers",
+                                                 'Content-Type, '
-                        "Accept-Encoding, Accept-Language, "
+                                                 'Depth, User-Agent, X-File-Size, '
-                        "Content-Type, "
+                                                 'X-Requested-With, If-Modified-Since, '
-                        "Depth, User-Agent, X-File-Size, "
+                                                 'X-File-Name, '
-                        "X-Requested-With, If-Modified-Since, "
+                                                 'Cache-Control, Host, Pragma, Accept, '
-                        "X-File-Name, "
+                                                 'Origin, Connection, '
-                        "Cache-Control, Host, Pragma, Accept, "
+                                                 'Referer, Cookie, '
-                        "Origin, Connection, "
+                                                 'X-appenlight-public-api-key, '
-                        "Referer, Cookie, "
+                                                 'x-appenlight-public-api-key')
-                        "X-appenlight-public-api-key, "
+                    request.response.headers.add('Access-Control-Max-Age', '86400')
-                        "x-appenlight-public-api-key",
-                    request.response.headers.add("Access-Control-Max-Age", "86400")
                     return request.response
                 else:
                     return HTTPForbidden()

backend/src/appenlight/lib/cache_regions.py

0 +82 -68

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import copy
             import hashlib
             import inspect
-            from dogpile.cache import make_region
+            from dogpile.cache import make_region, compat
-            from dogpile.cache.util import compat
             regions = None
             def key_mangler(key):
                 return "appenlight:dogpile:{}".format(key)
             def hashgen(namespace, fn, to_str=compat.string_type):
                 """Return a function that generates a string
                 key, based on a given function as well as
                 arguments to the returned function itself.
                 This is used by :meth:`.CacheRegion.cache_on_arguments`
                 to generate a cache key from a decorated function.
                 It can be replaced using the ``function_key_generator``
                 argument passed to :func:`.make_region`.
                 """
                 if namespace is None:
-                    namespace = "%s:%s" % (fn.__module__, fn.__name__)
+                    namespace = '%s:%s' % (fn.__module__, fn.__name__)
                 else:
-                    namespace = "%s:%s|%s" % (fn.__module__, fn.__name__, namespace)
+                    namespace = '%s:%s|%s' % (fn.__module__, fn.__name__, namespace)
                 args = inspect.getargspec(fn)
-                has_self = args[0] and args[0][0] in ("self", "cls")
+                has_self = args[0] and args[0][0] in ('self', 'cls')
                 def generate_key(*args, **kw):
                     if kw:
                         raise ValueError(
                             "dogpile.cache's default key creation "
-                            "function does not accept keyword arguments."
+                            "function does not accept keyword arguments.")
                     if has_self:
                         args = args[1:]
-                    return (
+                    return namespace + "|" + hashlib.sha1(
-                        namespace
+                        " ".join(map(to_str, args)).encode('utf8')).hexdigest()
-                        + "|"
-                        + hashlib.sha1(" ".join(map(to_str, args)).encode("utf8")).hexdigest()
                 return generate_key
             class CacheRegions(object):
                 def __init__(self, settings):
                     config_redis = {"arguments": settings}
                     self.redis_min_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=60, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=60,
+                        **copy.deepcopy(config_redis))
                     self.redis_min_5 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=300, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=300,
+                        **copy.deepcopy(config_redis))
                     self.redis_min_10 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=60, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=60,
+                        **copy.deepcopy(config_redis))
                     self.redis_min_60 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=3600, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=3600,
+                        **copy.deepcopy(config_redis))
                     self.redis_sec_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=1, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=1,
+                        **copy.deepcopy(config_redis))
                     self.redis_sec_5 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=5, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=5,
+                        **copy.deepcopy(config_redis))
                     self.redis_sec_30 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=30, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=30,
+                        **copy.deepcopy(config_redis))
                     self.redis_day_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.redis", expiration_time=86400, **copy.deepcopy(config_redis)
+                        "dogpile.cache.redis",
+                        expiration_time=86400,
+                        **copy.deepcopy(config_redis))
                     self.redis_day_7 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
                         "dogpile.cache.redis",
                         expiration_time=86400 * 7,
-                        **copy.deepcopy(config_redis)
+                        **copy.deepcopy(config_redis))
                     self.redis_day_30 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
                         "dogpile.cache.redis",
                         expiration_time=86400 * 30,
-                        **copy.deepcopy(config_redis)
+                        **copy.deepcopy(config_redis))
                     self.memory_day_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure(
+                        key_mangler=key_mangler).configure(
-                        "dogpile.cache.memory", expiration_time=86400, **copy.deepcopy(config_redis)
+                        "dogpile.cache.memory",
+                        expiration_time=86400,
+                        **copy.deepcopy(config_redis))
                     self.memory_sec_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=1)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=1)
                     self.memory_sec_5 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=5)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=5)
                     self.memory_min_1 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=60)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=60)
                     self.memory_min_5 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=300)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=300)
                     self.memory_min_10 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=600)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=600)
                     self.memory_min_60 = make_region(
-                        function_key_generator=hashgen, key_mangler=key_mangler
+                        function_key_generator=hashgen,
-                    ).configure("dogpile.cache.memory", expiration_time=3600)
+                        key_mangler=key_mangler).configure(
+                        "dogpile.cache.memory",
+                        expiration_time=3600)
             def get_region(region):
                 return getattr(regions, region)

backend/src/appenlight/lib/configurator.py

0 +21 -21

             import inspect
             import logging
             from pyramid.config import Configurator
             log = logging.getLogger(__name__)
             class InspectProxy(object):
                 """
                 Proxy to the `inspect` module that allows us to use the pyramid include
                 mechanism for cythonized modules without source file.
                 """
                 def _get_cyfunction_func_code(self, cyfunction):
                     """
                     Unpack the `func_code` attribute of a cython function.
                     """
                     if inspect.ismethod(cyfunction):
                         cyfunction = cyfunction.im_func
-                    return getattr(cyfunction, "func_code")
+                    return getattr(cyfunction, 'func_code')
                 def getmodule(self, *args, **kwds):
                     """
                     Simple proxy to `inspect.getmodule`.
                     """
                     return inspect.getmodule(*args, **kwds)
                 def getsourcefile(self, obj):
                     """
                     Proxy to `inspect.getsourcefile` or `inspect.getfile` depending on if
                     it's called to look up the source file that contains the magic pyramid
                     `includeme` callable.
                     For cythonized modules the source file may be deleted. Therefore we
                     return the result of `inspect.getfile` instead. In the case of the
                     `configurator.include` method this is OK, because the result is passed
                     to `os.path.dirname` which strips the file name. So it doesn't matter
                     if we return the path to the source file or another file in the same
                     directory.
                     """
                     # Check if it's called to look up the source file that contains the
                     # magic pyramid `includeme` callable.
-                    if getattr(obj, "__name__") == "includeme":
+                    if getattr(obj, '__name__') == 'includeme':
                         try:
                             return inspect.getfile(obj)
                         except TypeError as e:
                             # Cython functions are not recognized as functions by the
                             # inspect module. We have to unpack the func_code attribute
                             # ourself.
-                            if "cyfunction" in e.message:
+                            if 'cyfunction' in e.message:
                                 obj = self._get_cyfunction_func_code(obj)
                                 return inspect.getfile(obj)
                             raise
                     else:
                         return inspect.getsourcefile(obj)
             class CythonCompatConfigurator(Configurator):
                 """
                 Customized configurator to replace the inspect class attribute with
                 a custom one that is cython compatible.
                 """
                 inspect = InspectProxy()
             def register_appenlight_plugin(config, plugin_name, plugin_config):
                 def register():
-                    log.warning("Registering plugin: {}".format(plugin_name))
+                    log.warning('Registering plugin: {}'.format(plugin_name))
                     if plugin_name not in config.registry.appenlight_plugins:
                         config.registry.appenlight_plugins[plugin_name] = {
-                            "javascript": None,
+                            'javascript': None,
-                            "static": None,
+                            'static': None,
-                            "css": None,
+                            'css': None,
-                            "celery_tasks": None,
+                            'celery_tasks': None,
-                            "celery_beats": None,
+                            'celery_beats': None,
-                            "fulltext_indexer": None,
+                            'fulltext_indexer': None,
-                            "sqlalchemy_migrations": None,
+                            'sqlalchemy_migrations': None,
-                            "default_values_setter": None,
+                            'default_values_setter': None,
-                            "header_html": None,
+                            'header_html': None,
-                            "resource_types": [],
+                            'resource_types': [],
-                            "url_gen": None,
+                            'url_gen': None
                         }
-                    config.registry.appenlight_plugins[plugin_name].update(plugin_config)
+                    config.registry.appenlight_plugins[plugin_name].update(
+                        plugin_config)
                     # inform AE what kind of resource types we have available
                     # so we can avoid failing when a plugin is removed but data
                     # is still present in the db
-                    if plugin_config.get("resource_types"):
+                    if plugin_config.get('resource_types'):
-                        config.registry.resource_types.extend(plugin_config["resource_types"])
+                        config.registry.resource_types.extend(
+                            plugin_config['resource_types'])
-                config.action("appenlight_plugin={}".format(plugin_name), register)
+                config.action('appenlight_plugin={}'.format(plugin_name), register)

backend/src/appenlight/lib/encryption.py

0 +4 -4

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             # this gets set on runtime
             from cryptography.fernet import Fernet
             ENCRYPTION_SECRET = None
             def encrypt_fernet(value):
                 # avoid double encryption
                 # not sure if this is needed but it won't hurt too much to have this
-                if value.startswith("enc$fernet$"):
+                if value.startswith('enc$fernet$'):
                     return value
                 f = Fernet(ENCRYPTION_SECRET)
-                return "enc$fernet${}".format(f.encrypt(value.encode("utf8")).decode("utf8"))
+                return 'enc$fernet${}'.format(f.encrypt(value.encode('utf8')).decode('utf8'))
             def decrypt_fernet(value):
-                parts = value.split("$", 3)
+                parts = value.split('$', 3)
                 if not len(parts) == 3:
                     # not encrypted values
                     return value
                 else:
                     f = Fernet(ENCRYPTION_SECRET)
-                    decrypted_data = f.decrypt(parts[2].encode("utf8")).decode("utf8")
+                    decrypted_data = f.decrypt(parts[2].encode('utf8')).decode('utf8')
                     return decrypted_data
             def encrypt_dictionary_keys(_dict, exclude_keys=None):
                 if not exclude_keys:
                     exclude_keys = []
                 keys = [k for k in _dict.keys() if k not in exclude_keys]
                 for k in keys:
                     _dict[k] = encrypt_fernet(_dict[k])
                 return _dict
             def decrypt_dictionary_keys(_dict, exclude_keys=None):
                 if not exclude_keys:
                     exclude_keys = []
                 keys = [k for k in _dict.keys() if k not in exclude_keys]
                 for k in keys:
                     _dict[k] = decrypt_fernet(_dict[k])
                 return _dict

backend/src/appenlight/lib/enums.py

0 +3 -5

             import collections
             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             class StupidEnum(object):
                 @classmethod
                 def set_inverse(cls):
                     cls._inverse_values = dict(
-                        (y, x)
+                        (y, x) for x, y in vars(cls).items() if
-                        for x, y in vars(cls).items()
+                        not x.startswith('_') and not callable(y)
-                        if not x.startswith("_") and not callable(y)
                     )
                 @classmethod
                 def key_from_value(cls, value):
-                    if not hasattr(cls, "_inverse_values"):
+                    if not hasattr(cls, '_inverse_values'):
                         cls.set_inverse()
                     return cls._inverse_values.get(value)
             class ReportType(StupidEnum):
                 unknown = 0
                 error = 1
                 not_found = 2
                 slow = 3
             class Language(StupidEnum):
                 unknown = 0
                 python = 1
                 javascript = 2
                 java = 3
                 objectivec = 4
                 swift = 5
                 cpp = 6
                 basic = 7
                 csharp = 8
                 php = 9
                 perl = 10
                 vb = 11
                 vbnet = 12
                 ruby = 13
                 fsharp = 14
                 actionscript = 15
                 go = 16
                 scala = 17
                 haskell = 18
                 erlang = 19
                 haxe = 20
                 scheme = 21
             class LogLevel(StupidEnum):
                 UNKNOWN = 0
                 DEBUG = 2
                 TRACE = 4
                 INFO = 6
                 WARNING = 8
                 ERROR = 10
                 CRITICAL = 12
                 FATAL = 14
             class LogLevelPython(StupidEnum):
                 CRITICAL = 50
                 ERROR = 40
                 WARNING = 30
                 INFO = 20
                 DEBUG = 10
                 NOTSET = 0
             class ParsedSentryEventType(StupidEnum):
                 ERROR_REPORT = 1
                 LOG = 2

backend/src/appenlight/lib/ext_json.py

0 +20 -15

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             """
             ex-json borrowed from Marcin Kuzminski
             source: https://secure.rhodecode.org/ext-json
             """
             import datetime
             import functools
             import decimal
             import imp
-            __all__ = ["json", "simplejson", "stdlibjson"]
+            __all__ = ['json', 'simplejson', 'stdlibjson']
             def _is_aware(value):
                 """
                 Determines if a given datetime.time is aware.
                 The logic is described in Python's docs:
                 http://docs.python.org/library/datetime.html#datetime.tzinfo
                 """
-                return value.tzinfo is not None and value.tzinfo.utcoffset(value) is not None
+                return (value.tzinfo is not None
+                        and value.tzinfo.utcoffset(value) is not None)
             def _obj_dump(obj):
                 """
                 Custom function for dumping objects to JSON, if obj has __json__ attribute
                 or method defined it will be used for serialization
                 :param obj:
                 """
                 if isinstance(obj, complex):
                     return [obj.real, obj.imag]
                 # See "Date Time String Format" in the ECMA-262 specification.
                 # some code borrowed from django 1.4
                 elif isinstance(obj, datetime.datetime):
                     r = obj.isoformat()
                     # if obj.microsecond:
                     #     r = r[:23] + r[26:]
-                    if r.endswith("+00:00"):
+                    if r.endswith('+00:00'):
-                        r = r[:-6] + "Z"
+                        r = r[:-6] + 'Z'
                     return r
                 elif isinstance(obj, datetime.date):
                     return obj.isoformat()
                 elif isinstance(obj, decimal.Decimal):
                     return str(obj)
                 elif isinstance(obj, datetime.time):
                     if _is_aware(obj):
                         raise ValueError("JSON can't represent timezone-aware times.")
                     r = obj.isoformat()
                     if obj.microsecond:
                         r = r[:12]
                     return r
                 elif isinstance(obj, set):
                     return list(obj)
-                elif hasattr(obj, "__json__"):
+                elif hasattr(obj, '__json__'):
                     if callable(obj.__json__):
                         return obj.__json__()
                     else:
                         return obj.__json__
                 else:
                     raise NotImplementedError
             # Import simplejson
             try:
                 # import simplejson initially
-                _sj = imp.load_module("_sj", *imp.find_module("simplejson"))
+                _sj = imp.load_module('_sj', *imp.find_module('simplejson'))
                 def extended_encode(obj):
                     try:
                         return _obj_dump(obj)
                     except NotImplementedError:
                         pass
                     raise TypeError("%r is not JSON serializable" % (obj,))
                 # we handle decimals our own it makes unified behavior of json vs
                 # simplejson
-                sj_version = [int(x) for x in _sj.__version__.split(".")]
+                sj_version = [int(x) for x in _sj.__version__.split('.')]
                 major, minor = sj_version[0], sj_version[1]
                 if major < 2 or (major == 2 and minor < 1):
                     # simplejson < 2.1 doesnt support use_decimal
-                    _sj.dumps = functools.partial(_sj.dumps, default=extended_encode)
+                    _sj.dumps = functools.partial(
-                    _sj.dump = functools.partial(_sj.dump, default=extended_encode)
+                        _sj.dumps, default=extended_encode)
+                    _sj.dump = functools.partial(
+                        _sj.dump, default=extended_encode)
                 else:
                     _sj.dumps = functools.partial(
-                        _sj.dumps, default=extended_encode, use_decimal=False
+                        _sj.dumps, default=extended_encode, use_decimal=False)
                     _sj.dump = functools.partial(
-                        _sj.dump, default=extended_encode, use_decimal=False
+                        _sj.dump, default=extended_encode, use_decimal=False)
                 simplejson = _sj
             except ImportError:
                 # no simplejson set it to None
                 simplejson = None
             try:
                 # simplejson not found try out regular json module
-                _json = imp.load_module("_json", *imp.find_module("json"))
+                _json = imp.load_module('_json', *imp.find_module('json'))
                 # extended JSON encoder for json
                 class ExtendedEncoder(_json.JSONEncoder):
                     def default(self, obj):
                         try:
                             return _obj_dump(obj)
                         except NotImplementedError:
                             pass
                         raise TypeError("%r is not JSON serializable" % (obj,))
                 # monkey-patch JSON encoder to use extended version
                 _json.dumps = functools.partial(_json.dumps, cls=ExtendedEncoder)
                 _json.dump = functools.partial(_json.dump, cls=ExtendedEncoder)
             except ImportError:
                 json = None
             stdlibjson = _json
             # set all available json modules
             if simplejson:
                 json = _sj
             elif _json:
                 json = _json
             else:
-                raise ImportError("Could not find any json modules")
+                raise ImportError('Could not find any json modules')

backend/src/appenlight/lib/helpers.py

0 +56 -97

@@ -1,160 +1,119 b''
1	# -- coding: utf-8 --	1	# -- coding: utf-8 --
2		2
3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors	3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
4	#	4	#
5	# Licensed under the Apache License, Version 2.0 (the "License");	5	# Licensed under the Apache License, Version 2.0 (the "License");
6	# you may not use this file except in compliance with the License.	6	# you may not use this file except in compliance with the License.
7	# You may obtain a copy of the License at	7	# You may obtain a copy of the License at
8	#	8	#
9	# http://www.apache.org/licenses/LICENSE-2.0	9	# http://www.apache.org/licenses/LICENSE-2.0
10	#	10	#
11	# Unless required by applicable law or agreed to in writing, software	11	# Unless required by applicable law or agreed to in writing, software
12	# distributed under the License is distributed on an "AS IS" BASIS,	12	# distributed under the License is distributed on an "AS IS" BASIS,
13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.	13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14	# See the License for the specific language governing permissions and	14	# See the License for the specific language governing permissions and
15	# limitations under the License.	15	# limitations under the License.
16		16
17	"""	17	"""
18	Helper functions	18	Helper functions
19	"""	19	"""
20	import copy	20	import copy
21	import datetime	21	import datetime
22		22
23	from collections import namedtuple, OrderedDict	23	from collections import namedtuple, OrderedDict
24		24
25	_ = lambda x: x	25	_ = lambda x: x
26		26
27	time_deltas = OrderedDict()	27	time_deltas = OrderedDict()
28		28
29	time_deltas["1m"] = {	29	time_deltas['1m'] = {'delta': datetime.timedelta(minutes=1),
30	"delta": datetime.timedelta(minutes=1),	30	'label': '1 minute', 'minutes': 1}
31	"label": "1 minute",	31
32	"minutes": 1,	32	time_deltas['5m'] = {'delta': datetime.timedelta(minutes=5),
33	}	33	'label': '5 minutes', 'minutes': 5}
34		34	time_deltas['30m'] = {'delta': datetime.timedelta(minutes=30),
35	time_deltas["5m"] = {	35	'label': '30 minutes', 'minutes': 30}
36	"delta": datetime.timedelta(~~minute~~s=5),	36	time_deltas['1h'] = {'delta': datetime.timedelta(hours=1),
37	"label": "5 minutes",	37	'label': '60 minutes', 'minutes': 60}
38	"minutes": 5,	38	time_deltas['4h'] = {'delta': datetime.timedelta(hours=4), 'label': '4 hours',
39	}	39	'minutes': 60 * 4}
40	time_deltas["30m"] = {	40	time_deltas['12h'] = {'delta': datetime.timedelta(hours=12),
41	"delta": datetime.timedelta(minutes=30),	41	'label': '12 hours', 'minutes': 60 * 12}
42	"label": "30 minutes",	42	time_deltas['24h'] = {'delta': datetime.timedelta(hours=24),
43	"minutes": 30,	43	'label': '24 hours', 'minutes': 60 * 24}
44	}	44	time_deltas['3d'] = {'delta': datetime.timedelta(days=3), 'label': '3 days',
45	time_deltas["1h"] = {	45	'minutes': 60 * 24 * 3}
46	"delta": datetime.timedelta(hours=1),	46	time_deltas['1w'] = {'delta': datetime.timedelta(days=7), 'label': '7 days',
47	"label": "60 minutes",	47	'minutes': 60 * 24 * 7}
48	"minutes": 60,	48	time_deltas['2w'] = {'delta': datetime.timedelta(days=14), 'label': '14 days',
49	}	49	'minutes': 60 * 24 * 14}
50	time_deltas["4h"] = {	50	time_deltas['1M'] = {'delta': datetime.timedelta(days=31), 'label': '31 days',
51	"delta": datetime.timedelta(hours=4),	51	'minutes': 60 * 24 * 31}
52	"label": "4 hours",	52	time_deltas['3M'] = {'delta': datetime.timedelta(days=31 * 3),
53	"minutes": 60 * 4,	53	'label': '3 months',
54	}	54	'minutes': 60 * 24 * 31 * 3}
55	time_deltas["12h"] = {	55	time_deltas['6M'] = {'delta': datetime.timedelta(days=31 * 6),
56	"delta": datetime.timedelta(hours=12),	56	'label': '6 months',
57	"label": "12 hours",	57	'minutes': 60 * 24 * 31 * 6}
58	"minutes": 60 * 12,	58	time_deltas['12M'] = {'delta': datetime.timedelta(days=31 * 12),
59	}	59	'label': '12 months',
60	time_deltas["24h"] = {	60	'minutes': 60 * 24 * 31 * 12}
61	"delta": datetime.timedelta(hours=24),
62	"label": "24 hours",
63	"minutes": 60 * 24,
64	}
65	time_deltas["3d"] = {
66	"delta": datetime.timedelta(days=3),
67	"label": "3 days",
68	"minutes": 60 * 24 * 3,
69	}
70	time_deltas["1w"] = {
71	"delta": datetime.timedelta(days=7),
72	"label": "7 days",
73	"minutes": 60 * 24 * 7,
74	}
75	time_deltas["2w"] = {
76	"delta": datetime.timedelta(days=14),
77	"label": "14 days",
78	"minutes": 60 * 24 * 14,
79	}
80	time_deltas["1M"] = {
81	"delta": datetime.timedelta(days=31),
82	"label": "31 days",
83	"minutes": 60 * 24 * 31,
84	}
85	time_deltas["3M"] = {
86	"delta": datetime.timedelta(days=31 * 3),
87	"label": "3 months",
88	"minutes": 60 * 24 * 31 * 3,
89	}
90	time_deltas["6M"] = {
91	"delta": datetime.timedelta(days=31 * 6),
92	"label": "6 months",
93	"minutes": 60 * 24 * 31 * 6,
94	}
95	time_deltas["12M"] = {
96	"delta": datetime.timedelta(days=31 * 12),
97	"label": "12 months",
98	"minutes": 60 * 24 * 31 * 12,
99	}
100		61
101	# used in json representation	62	# used in json representation
102	time_options = dict(	63	time_options = dict([(k, {'label': v['label'], 'minutes': v['minutes']})
103	[	64	for k, v in time_deltas.items()])
104	(k, {"label": v["label"], "minutes": v["minutes"]})	65	FlashMsg = namedtuple('FlashMsg', ['msg', 'level'])
105	for k, v in time_deltas.items()
106	]
107	)
108	FlashMsg = namedtuple("FlashMsg", ["msg", "level"])
109		66
110		67
111	def get_flash(request):	68	def get_flash(request):
112	messages = []	69	messages = []
113	messages.extend(	70	messages.extend(
114	[FlashMsg(msg, "error") for msg in request.session.peek_flash("error")]	71	[FlashMsg(msg, 'error')
115	)	72	for msg in request.session.peek_flash('error')])
		73	messages.extend([FlashMsg(msg, 'warning')
		74	for msg in request.session.peek_flash('warning')])
116	messages.extend(	75	messages.extend(
117	[FlashMsg(msg, ~~"warning"~~) for msg in request.session.peek_flash(~~"warning"~~)]	76	[FlashMsg(msg, 'notice') for msg in request.session.peek_flash()])
118	)
119	messages.extend([FlashMsg(msg, "notice") for msg in request.session.peek_flash()])
120	return messages	77	return messages
121		78
122		79
123	def clear_flash(request):	80	def clear_flash(request):
124	request.session.pop_flash("error")	81	request.session.pop_flash('error')
125	request.session.pop_flash("warning")	82	request.session.pop_flash('warning')
126	request.session.pop_flash()	83	request.session.pop_flash()
127		84
128		85
129	def get_type_formatted_flash(request):	86	def get_type_formatted_flash(request):
130	return [	87	return [{'msg': message.msg, 'type': message.level}
131	{~~"msg"~~: ~~message~~.~~msg~~, ~~"type"~~: ~~message~~.~~level~~} for message in get_flash(request)	88	for message in get_flash(request)]
132	]
133		89
134		90
135	def gen_pagination_headers(request, paginator):	91	def gen_pagination_headers(request, paginator):
136	headers = {	92	headers = {
137	"x-total-count": str(paginator.item_count),	93	'x-total-count': str(paginator.item_count),
138	"x-current-page": str(paginator.page),	94	'x-current-page': str(paginator.page),
139	"x-items-per-page": str(paginator.items_per_page),	95	'x-items-per-page': str(paginator.items_per_page)
140	}	96	}
141	params_dict = request.GET.dict_of_lists()	97	params_dict = request.GET.dict_of_lists()
142	last_page_params = copy.deepcopy(params_dict)	98	last_page_params = copy.deepcopy(params_dict)
143	last_page_params["page"] = paginator.last_page or 1	99	last_page_params['page'] = paginator.last_page or 1
144	first_page_params = copy.deepcopy(params_dict)	100	first_page_params = copy.deepcopy(params_dict)
145	first_page_params.pop("page", None)	101	first_page_params.pop('page', None)
146	next_page_params = copy.deepcopy(params_dict)	102	next_page_params = copy.deepcopy(params_dict)
147	next_page_params["page"] = paginator.next_page or paginator.last_page or 1	103	next_page_params['page'] = paginator.next_page or paginator.last_page or 1
148	prev_page_params = copy.deepcopy(params_dict)	104	prev_page_params = copy.deepcopy(params_dict)
149	prev_page_params["page"] = paginator.previous_page or 1	105	prev_page_params['page'] = paginator.previous_page or 1
150	lp_url = request.current_route_url(_query=last_page_params)	106	lp_url = request.current_route_url(_query=last_page_params)
151	fp_url = request.current_route_url(_query=first_page_params)	107	fp_url = request.current_route_url(_query=first_page_params)
152	links = ['rel="last", <{}>'.format(lp_url), 'rel="first", <{}>'.format(fp_url)]	108	links = [
		109	'rel="last", <{}>'.format(lp_url),
		110	'rel="first", <{}>'.format(fp_url),
		111	]
153	if first_page_params != prev_page_params:	112	if first_page_params != prev_page_params:
154	prev_url = request.current_route_url(_query=prev_page_params)	113	prev_url = request.current_route_url(_query=prev_page_params)
155	links.append('rel="prev", <{}>'.format(prev_url))	114	links.append('rel="prev", <{}>'.format(prev_url))
156	if last_page_params != next_page_params:	115	if last_page_params != next_page_params:
157	next_url = request.current_route_url(_query=next_page_params)	116	next_url = request.current_route_url(_query=next_page_params)
158	links.append('rel="next", <{}>'.format(next_url))	117	links.append('rel="next", <{}>'.format(next_url))
159	headers["link"] = "; ".join(links)	118	headers['link'] = '; '.join(links)
160	return headers	119	return headers

backend/src/appenlight/lib/jinja2_filters.py

0 +12 -19

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import re
             from appenlight.lib.ext_json import json
             from jinja2 import Markup, escape, evalcontextfilter
-            _paragraph_re = re.compile(r"(?:\r\n|\r|\n){2,}")
+            _paragraph_re = re.compile(r'(?:\r\n|\r|\n){2,}')
             @evalcontextfilter
             def nl2br(eval_ctx, value):
                 if eval_ctx.autoescape:
-                    result = "\n\n".join(
+                    result = '\n\n'.join('<p>%s</p>' % p.replace('\n', Markup('<br>\n'))
-                        "<p>%s</p>" % p.replace("\n", Markup("<br>\n"))
+                                          for p in _paragraph_re.split(escape(value)))
-                        for p in _paragraph_re.split(escape(value))
                 else:
-                    result = "\n\n".join(
+                    result = '\n\n'.join('<p>%s</p>' % p.replace('\n', '<br>\n')
-                        "<p>%s</p>" % p.replace("\n", "<br>\n")
+                                          for p in _paragraph_re.split(escape(value)))
-                        for p in _paragraph_re.split(escape(value))
                 if eval_ctx.autoescape:
                     result = Markup(result)
                 return result
             @evalcontextfilter
             def toJSONUnsafe(eval_ctx, value):
-                encoded = (
+                encoded = json.dumps(value).replace('&', '\\u0026') \
-                    json.dumps(value)
+                    .replace('<', '\\u003c') \
-                    .replace("&", "\\u0026")
+                    .replace('>', '\\u003e') \
-                    .replace("<", "\\u003c")
+                    .replace('>', '\\u003e') \
-                    .replace(">", "\\u003e")
+                    .replace('"', '\\u0022') \
-                    .replace(">", "\\u003e")
+                    .replace("'", '\\u0027') \
-                    .replace('"', "\\u0022")
+                    .replace(r'\n', '/\\\n')
-                    .replace("'", "\\u0027")
-                    .replace(r"\n", "/\\\n")
                 return Markup("'%s'" % encoded)

backend/src/appenlight/lib/logging.py

0 +6 -25

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import json
             import logging
-            ignored_keys = [
+            ignored_keys = ['args', 'asctime', 'created', 'exc_info', 'exc_text',
-                "args",
+                            'filename', 'funcName', 'levelname', 'levelno', 'lineno',
-                "asctime",
+                            'message', 'module', 'msecs', 'msg', 'name', 'pathname',
-                "created",
+                            'process', 'processName', 'relativeCreated', 'stack_info',
-                "exc_info",
+                            'thread', 'threadName']
-                "exc_text",
-                "filename",
-                "funcName",
-                "levelname",
-                "levelno",
-                "lineno",
-                "message",
-                "module",
-                "msecs",
-                "msg",
-                "name",
-                "pathname",
-                "process",
-                "processName",
-                "relativeCreated",
-                "stack_info",
-                "thread",
-                "threadName",
             class JSONFormatter(logging.Formatter):
                 def format(self, record):
                     """
                     Format the specified record as text.
                     The record's attribute dictionary is used as the operand to a
                     string formatting operation which yields the returned string.
                     Before formatting the dictionary, a couple of preparatory steps
                     are carried out. The message attribute of the record is computed
                     using LogRecord.getMessage(). If the formatting string uses the
                     time (as determined by a call to usesTime(), formatTime() is
                     called to format the event time. If there is exception information,
                     it is formatted using formatException() and appended to the message.
                     """
                     record.message = record.getMessage()
                     log_dict = vars(record)
                     keys = [k for k in log_dict.keys() if k not in ignored_keys]
-                    payload = {"message": record.message}
+                    payload = {'message': record.message}
                     payload.update({k: log_dict[k] for k in keys})
                     record.message = json.dumps(payload, default=lambda x: str(x))
                     if self.usesTime():
                         record.asctime = self.formatTime(record, self.datefmt)
                     s = self.formatMessage(record)
                     if record.exc_info:
                         # Cache the traceback text to avoid converting it multiple times
                         # (it's constant anyway)
                         if not record.exc_text:
                             record.exc_text = self.formatException(record.exc_info)
                     if record.exc_text:
                         if s[-1:] != "\n":
                             s = s + "\n"
                         s = s + record.exc_text
                     if record.stack_info:
                         if s[-1:] != "\n":
                             s = s + "\n"
                         s = s + self.formatStack(record.stack_info)
                     return s

backend/src/appenlight/lib/redis_keys.py

0 +43 -47

@@ -1,69 +1,65 b''
1	# -- coding: utf-8 --	1	# -- coding: utf-8 --
2		2
3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors	3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
4	#	4	#
5	# Licensed under the Apache License, Version 2.0 (the "License");	5	# Licensed under the Apache License, Version 2.0 (the "License");
6	# you may not use this file except in compliance with the License.	6	# you may not use this file except in compliance with the License.
7	# You may obtain a copy of the License at	7	# You may obtain a copy of the License at
8	#	8	#
9	# http://www.apache.org/licenses/LICENSE-2.0	9	# http://www.apache.org/licenses/LICENSE-2.0
10	#	10	#
11	# Unless required by applicable law or agreed to in writing, software	11	# Unless required by applicable law or agreed to in writing, software
12	# distributed under the License is distributed on an "AS IS" BASIS,	12	# distributed under the License is distributed on an "AS IS" BASIS,
13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.	13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14	# See the License for the specific language governing permissions and	14	# See the License for the specific language governing permissions and
15	# limitations under the License.	15	# limitations under the License.
16		16
17	BASE = "appenlight:data:{}"	17	BASE = 'appenlight:data:{}'
18		18
19	REDIS_KEYS = {	19	REDIS_KEYS = {
20	"tasks": {	20	'tasks': {
21	"add_reports_lock": BASE.format("add_reports_lock:{}"),	21	'add_reports_lock': BASE.format('add_reports_lock:{}'),
22	"add_logs_lock": BASE.format("add_logs_lock:{}"),	22	'add_logs_lock': BASE.format('add_logs_lock:{}'),
23	},	23	},
24	"counters": {	24	'counters': {
25	"events_per_minute_per_user": BASE.format(~~"events_per_minute_per_user:~~{}:{}"),	25	'events_per_minute_per_user': BASE.format(
26	"reports_per_minute": BASE.format("reports_per_minute:{}"),	26	'events_per_minute_per_user:{}:{}'),
27	"reports_per_~~hour_per_app"~~: BASE.format("reports_per_~~hour_per_app~~:{}:{}"),	27	'reports_per_minute': BASE.format('reports_per_minute:{}'),
28	"reports_per_~~type"~~: BASE.format(~~"reports_per_type:~~{}"),	28	'reports_per_hour_per_app': BASE.format(
29	"logs_per_minute": BASE.format("logs_per_minute:{}"),	29	'reports_per_hour_per_app:{}:{}'),
30	~~"logs_per_hour_per_app"~~: BASE.format(~~"logs_per_hour_per_app~~:{}:{}"),	30	'reports_per_type': BASE.format('reports_per_type:{}'),
31	~~"metric~~s_per_minute": BASE.format(~~"metric~~s_per_minute:{}"),	31	'logs_per_minute': BASE.format('logs_per_minute:{}'),
32	~~"metric~~s_per_hour_per_app": BASE.format(~~"metrics_per_hour_per_app:~~{}:{}"),	32	'logs_per_hour_per_app': BASE.format(
33	"report_group_occurences": BASE.format("report_group_occurences:{}"),	33	'logs_per_hour_per_app:{}:{}'),
34	"report_group_occurences_alerting": BASE.format(	34	'metrics_per_minute': BASE.format('metrics_per_minute:{}'),
35	"report_group_occurences_alerting:{}"	35	'metrics_per_hour_per_app': BASE.format(
36	),	36	'metrics_per_hour_per_app:{}:{}'),
37	"report_group_occurences~~_10th"~~: BASE.format("report_group_occurences~~_10th~~:{}"),	37	'report_group_occurences': BASE.format('report_group_occurences:{}'),
38	"report_group_occurences_~~100th"~~: BASE.format(	38	'report_group_occurences_alerting': BASE.format(
39	"report_group_occurences_~~100th~~:{}"	39	'report_group_occurences_alerting:{}'),
40	),	40	'report_group_occurences_10th': BASE.format(
		41	'report_group_occurences_10th:{}'),
		42	'report_group_occurences_100th': BASE.format(
		43	'report_group_occurences_100th:{}'),
41	},	44	},
42	"rate_limits": {	45	'rate_limits': {
43	"per_application_reports_rate_limit": BASE.format(	46	'per_application_reports_rate_limit': BASE.format(
44	"per_application_reports_limit:{}:{}"	47	'per_application_reports_limit:{}:{}'),
45	),	48	'per_application_logs_rate_limit': BASE.format(
46	"per_application_logs_rate_limit": ~~BASE~~.~~format~~(	49	'per_application_logs_rate_limit:{}:{}'),
47	"per_application_~~log~~s_rate_limit:{}:{}"	50	'per_application_metrics_rate_limit': BASE.format(
48	),	51	'per_application_metrics_rate_limit:{}:{}'),
49	"per_application_metrics_rate_limit": BASE.format(
50	"per_application_metrics_rate_limit:{}:{}"
51	),
52	},	52	},
53	"apps_that_got_new_data_per_hour": BASE.format(	53	'apps_that_got_new_data_per_hour': BASE.format('apps_that_got_new_data_per_hour:{}'),
54	"apps_that_got_new_data_per_hour:{}"	54	'apps_that_had_reports': BASE.format('apps_that_had_reports'),
55	),	55	'apps_that_had_error_reports': BASE.format('apps_that_had_error_reports'),
56	"apps_that_had_reports": BASE.format(~~"apps_that_had_reports"~~),	56	'apps_that_had_reports_alerting': BASE.format(
57	"apps_that_had_error_reports": BASE.format("apps_that_had_error_reports"),	57	'apps_that_had_reports_alerting'),
58	"apps_that_had_reports_alerting": BASE.format(~~"apps_that_had_reports_alerting"~~),	58	'apps_that_had_error_reports_alerting': BASE.format(
59	"apps_that_had_error_reports_alerting": ~~BASE~~.~~format~~(	59	'apps_that_had_error_reports_alerting'),
60	"apps_that_had_error_reports_alerting"	60	'reports_to_notify_per_type_per_app': BASE.format(
61	),	61	'reports_to_notify_per_type_per_app:{}:{}'),
62	"reports_to_notify_per_type_per_app": BASE.format(	62	'reports_to_notify_per_type_per_app_alerting': BASE.format(
63	"reports_to_notify_per_type_per_app:{}:{}"	63	'reports_to_notify_per_type_per_app_alerting:{}:{}'),
64	),	64	'seen_tag_list': BASE.format('seen_tag_list')
65	"reports_to_notify_per_type_per_app_alerting": BASE.format(
66	"reports_to_notify_per_type_per_app_alerting:{}:{}"
67	),
68	"seen_tag_list": BASE.format("seen_tag_list"),
69	}	65	}

backend/src/appenlight/lib/request.py

0 +39 -35

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import json
             from pyramid.security import unauthenticated_userid
             import appenlight.lib.helpers as helpers
             from authomatic.providers import oauth2, oauth1
             from authomatic import Authomatic
-            from ziggurat_foundations.models.services.user import UserService
+            from appenlight.models.user import User
             class CSRFException(Exception):
                 pass
             class JSONException(Exception):
                 pass
             def get_csrf_token(request):
                 return request.session.get_csrf_token()
             def safe_json_body(request):
                 """
                 Returns None if json body is missing or erroneous
                 """
                 try:
                     return request.json_body
                 except ValueError:
                     return None
             def unsafe_json_body(request):
                 """
                 Throws JSONException if json can't deserialize
                 """
                 try:
                     return request.json_body
                 except ValueError:
-                    raise JSONException("Incorrect JSON")
+                    raise JSONException('Incorrect JSON')
             def get_user(request):
-                if not request.path_info.startswith("/static"):
+                if not request.path_info.startswith('/static'):
                     user_id = unauthenticated_userid(request)
                     try:
                         user_id = int(user_id)
                     except Exception:
                         return None
                     if user_id:
-                        user = UserService.by_id(user_id)
+                        user = User.by_id(user_id)
                         if user:
-                            request.environ["appenlight.username"] = "%d:%s" % (
+                            request.environ['appenlight.username'] = '%d:%s' % (
-                                user_id,
+                                user_id, user.user_name)
-                                user.user_name,
                         return user
                     else:
                         return None
             def es_conn(request):
                 return request.registry.es_conn
             def add_flash_to_headers(request, clear=True):
                 """
                 Adds pending flash messages to response, if clear is true clears out the
                 flash queue
                 """
                 flash_msgs = helpers.get_type_formatted_flash(request)
-                request.response.headers["x-flash-messages"] = json.dumps(flash_msgs)
+                request.response.headers['x-flash-messages'] = json.dumps(flash_msgs)
                 helpers.clear_flash(request)
             def get_authomatic(request):
                 settings = request.registry.settings
                 # authomatic social auth
                 authomatic_conf = {
                     # callback http://yourapp.com/social_auth/twitter
-                    "twitter": {
+                    'twitter': {
-                        "class_": oauth1.Twitter,
+                        'class_': oauth1.Twitter,
-                        "consumer_key": settings.get("authomatic.pr.twitter.key", ""),
+                        'consumer_key': settings.get('authomatic.pr.twitter.key', ''),
-                        "consumer_secret": settings.get("authomatic.pr.twitter.secret", ""),
+                        'consumer_secret': settings.get('authomatic.pr.twitter.secret',
+                                                        ''),
                     },
                     # callback http://yourapp.com/social_auth/facebook
-                    "facebook": {
+                    'facebook': {
-                        "class_": oauth2.Facebook,
+                        'class_': oauth2.Facebook,
-                        "consumer_key": settings.get("authomatic.pr.facebook.app_id", ""),
+                        'consumer_key': settings.get('authomatic.pr.facebook.app_id', ''),
-                        "consumer_secret": settings.get("authomatic.pr.facebook.secret", ""),
+                        'consumer_secret': settings.get('authomatic.pr.facebook.secret',
-                        "scope": ["email"],
+                                                        ''),
+                        'scope': ['email'],
                     },
                     # callback http://yourapp.com/social_auth/google
-                    "google": {
+                    'google': {
-                        "class_": oauth2.Google,
+                        'class_': oauth2.Google,
-                        "consumer_key": settings.get("authomatic.pr.google.key", ""),
+                        'consumer_key': settings.get('authomatic.pr.google.key', ''),
-                        "consumer_secret": settings.get("authomatic.pr.google.secret", ""),
+                        'consumer_secret': settings.get(
-                        "scope": ["profile", "email"],
+                            'authomatic.pr.google.secret', ''),
+                        'scope': ['profile', 'email'],
                     },
-                    "github": {
+                    'github': {
-                        "class_": oauth2.GitHub,
+                        'class_': oauth2.GitHub,
-                        "consumer_key": settings.get("authomatic.pr.github.key", ""),
+                        'consumer_key': settings.get('authomatic.pr.github.key', ''),
-                        "consumer_secret": settings.get("authomatic.pr.github.secret", ""),
+                        'consumer_secret': settings.get(
-                        "scope": ["repo", "public_repo", "user:email"],
+                            'authomatic.pr.github.secret', ''),
-                        "access_headers": {"User-Agent": "AppEnlight"},
+                        'scope': ['repo', 'public_repo', 'user:email'],
-                    },
+                        'access_headers': {'User-Agent': 'AppEnlight'},
-                    "bitbucket": {
-                        "class_": oauth1.Bitbucket,
-                        "consumer_key": settings.get("authomatic.pr.bitbucket.key", ""),
-                        "consumer_secret": settings.get("authomatic.pr.bitbucket.secret", ""),
                     },
+                    'bitbucket': {
+                        'class_': oauth1.Bitbucket,
+                        'consumer_key': settings.get('authomatic.pr.bitbucket.key', ''),
+                        'consumer_secret': settings.get(
+                            'authomatic.pr.bitbucket.secret', '')
+                    }
                 }
-                return Authomatic(config=authomatic_conf, secret=settings["authomatic.secret"])
+                return Authomatic(
+                    config=authomatic_conf, secret=settings['authomatic.secret'])

backend/src/appenlight/lib/rule.py

0 +55 -69

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             import operator
             log = logging.getLogger(__name__)
             class RuleException(Exception):
                 pass
             class KeyNotFoundException(RuleException):
                 pass
             class UnknownTypeException(RuleException):
                 pass
             class BadConfigException(RuleException):
                 pass
             class InvalidValueException(RuleException):
                 pass
             class RuleBase(object):
                 @classmethod
                 def default_dict_struct_getter(cls, struct, field_name):
                     """
                     returns a key from dictionary based on field_name, if the name contains
                     `:` then it means additional nesting levels should be checked for the
                     key so `a:b:c` means return struct['a']['b']['c']
                     :param struct:
                     :param field_name:
                     :return:
                     """
-                    parts = field_name.split(":") if field_name else []
+                    parts = field_name.split(':') if field_name else []
                     found = struct
                     while parts:
                         current_key = parts.pop(0)
                         found = found.get(current_key)
                         if not found and parts:
-                            raise KeyNotFoundException("Key not found in structure")
+                            raise KeyNotFoundException('Key not found in structure')
                     return found
                 @classmethod
                 def default_obj_struct_getter(cls, struct, field_name):
                     """
                     returns a key from instance based on field_name, if the name contains
                     `:` then it means additional nesting levels should be checked for the
                     key so `a:b:c` means return struct.a.b.c
                     :param struct:
                     :param field_name:
                     :return:
                     """
-                    parts = field_name.split(":")
+                    parts = field_name.split(':')
                     found = struct
                     while parts:
                         current_key = parts.pop(0)
                         found = getattr(found, current_key, None)
                         if not found and parts:
-                            raise KeyNotFoundException("Key not found in structure")
+                            raise KeyNotFoundException('Key not found in structure')
                     return found
                 def normalized_type(self, field, value):
                     """
                     Converts text values from self.conf_value based on type_matrix below
                     check_matrix defines what kind of checks we can perform on a field
                     value based on field name
                     """
                     f_type = self.type_matrix.get(field)
                     if f_type:
-                        cast_to = f_type["type"]
+                        cast_to = f_type['type']
                     else:
-                        raise UnknownTypeException("Unknown type")
+                        raise UnknownTypeException('Unknown type')
                     if value is None:
                         return None
                     try:
-                        if cast_to == "int":
+                        if cast_to == 'int':
                             return int(value)
-                        elif cast_to == "float":
+                        elif cast_to == 'float':
                             return float(value)
-                        elif cast_to == "unicode":
+                        elif cast_to == 'unicode':
                             return str(value)
                     except ValueError as exc:
                         raise InvalidValueException(exc)
             class Rule(RuleBase):
-                def __init__(
+                def __init__(self, config, type_matrix,
-                    self,
-                    config,
-                    type_matrix,
                              struct_getter=RuleBase.default_dict_struct_getter,
-                    config_manipulator=None,
+                             config_manipulator=None):
-                ):
                     """
                     :param config: dict - contains rule configuration
                                           example::
                                                 {
                                                     "field": "__OR__",
                                                     "rules": [
                                                         {
                                                             "field": "__AND__",
                                                             "rules": [
                                                                 {
                                                                     "op": "ge",
                                                                     "field": "occurences",
                                                                     "value": "10"
                                                                 },
                                                                 {
                                                                     "op": "ge",
                                                                     "field": "priority",
                                                                     "value": "4"
                                                                 }
                                                             ]
                                                         },
                                                         {
                                                             "op": "eq",
                                                             "field": "http_status",
                                                             "value": "500"
                                                         }
                                                     ]
                                                 }
                     :param type_matrix: dict - contains map of type casts
                                                example::
                                                     {
                                                     'http_status': 'int',
                                                     'priority': 'unicode',
                                                     }
                     :param struct_getter: callable - used to grab the value of field from
                                                      the structure passed to match() based
                                                      on key, default
                     """
                     self.type_matrix = type_matrix
                     self.config = config
                     self.struct_getter = struct_getter
                     self.config_manipulator = config_manipulator
                     if config_manipulator:
                         config_manipulator(self)
                 def subrule_check(self, rule_config, struct):
-                    rule = Rule(
+                    rule = Rule(rule_config, self.type_matrix,
-                        rule_config, self.type_matrix, config_manipulator=self.config_manipulator
+                                config_manipulator=self.config_manipulator)
                     return rule.match(struct)
                 def match(self, struct):
                     """
                     Check if rule matched for this specific report
                     First tries report value, then tests tags in not found, then finally
                     report group
                     """
-                    field_name = self.config.get("field")
+                    field_name = self.config.get('field')
-                    test_value = self.config.get("value")
+                    test_value = self.config.get('value')
                     if not field_name:
                         return False
-                    if field_name == "__AND__":
+                    if field_name == '__AND__':
-                        rule = AND(
+                        rule = AND(self.config['rules'], self.type_matrix,
-                            self.config["rules"],
+                                   config_manipulator=self.config_manipulator)
-                            self.type_matrix,
-                            config_manipulator=self.config_manipulator,
                         return rule.match(struct)
-                    elif field_name == "__OR__":
+                    elif field_name == '__OR__':
-                        rule = OR(
+                        rule = OR(self.config['rules'], self.type_matrix,
-                            self.config["rules"],
+                                  config_manipulator=self.config_manipulator)
-                            self.type_matrix,
-                            config_manipulator=self.config_manipulator,
                         return rule.match(struct)
-                    elif field_name == "__NOT__":
+                    elif field_name == '__NOT__':
-                        rule = NOT(
+                        rule = NOT(self.config['rules'], self.type_matrix,
-                            self.config["rules"],
+                                  config_manipulator=self.config_manipulator)
-                            self.type_matrix,
-                            config_manipulator=self.config_manipulator,
                         return rule.match(struct)
                     if test_value is None:
                         return False
                     try:
-                        struct_value = self.normalized_type(
+                        struct_value = self.normalized_type(field_name,
-                            field_name, self.struct_getter(struct, field_name)
+                                                            self.struct_getter(struct,
+                                                                               field_name))
                     except (UnknownTypeException, InvalidValueException) as exc:
                         log.error(str(exc))
                         return False
                     try:
                         test_value = self.normalized_type(field_name, test_value)
                     except (UnknownTypeException, InvalidValueException) as exc:
                         log.error(str(exc))
                         return False
-                    if self.config["op"] not in ("startswith", "endswith", "contains"):
+                    if self.config['op'] not in ('startswith', 'endswith', 'contains'):
                         try:
-                            return getattr(operator, self.config["op"])(struct_value, test_value)
+                            return getattr(operator,
+                                           self.config['op'])(struct_value, test_value)
                         except TypeError:
                             return False
-                    elif self.config["op"] == "startswith":
+                    elif self.config['op'] == 'startswith':
                         return struct_value.startswith(test_value)
-                    elif self.config["op"] == "endswith":
+                    elif self.config['op'] == 'endswith':
                         return struct_value.endswith(test_value)
-                    elif self.config["op"] == "contains":
+                    elif self.config['op'] == 'contains':
                         return test_value in struct_value
-                    raise BadConfigException(
+                    raise BadConfigException('Invalid configuration, '
-                        "Invalid configuration, " "unknown operator: {}".format(self.config)
+                                             'unknown operator: {}'.format(self.config))
                 def __repr__(self):
-                    return "<Rule {} {}>".format(self.config.get("field"), self.config.get("value"))
+                    return '<Rule {} {}>'.format(self.config.get('field'),
+                                                 self.config.get('value'))
             class AND(Rule):
                 def __init__(self, rules, *args, **kwargs):
                     super(AND, self).__init__({}, *args, **kwargs)
                     self.rules = rules
                 def match(self, struct):
-                    return all([self.subrule_check(r_conf, struct) for r_conf in self.rules])
+                    return all([self.subrule_check(r_conf, struct) for r_conf
+                                in self.rules])
             class NOT(Rule):
                 def __init__(self, rules, *args, **kwargs):
                     super(NOT, self).__init__({}, *args, **kwargs)
                     self.rules = rules
                 def match(self, struct):
-                    return all([not self.subrule_check(r_conf, struct) for r_conf in self.rules])
+                    return all([not self.subrule_check(r_conf, struct) for r_conf
+                                in self.rules])
             class OR(Rule):
                 def __init__(self, rules, *args, **kwargs):
                     super(OR, self).__init__({}, *args, **kwargs)
                     self.rules = rules
                 def match(self, struct):
-                    return any([self.subrule_check(r_conf, struct) for r_conf in self.rules])
+                    return any([self.subrule_check(r_conf, struct) for r_conf
+                                in self.rules])
             class RuleService(object):
                 @staticmethod
-                def rule_from_config(config, field_mappings, labels_dict, manipulator_func=None):
+                def rule_from_config(config, field_mappings, labels_dict,
+                                     manipulator_func=None):
                     """
                     Returns modified rule with manipulator function
                     By default manipulator function replaces field id from labels_dict
                     with current field id proper for the rule from fields_mappings
                     because label X_X id might be pointing different value on next request
                     when new term is returned from elasticsearch - this ensures things
                     are kept 1:1 all the time
                     """
                     rev_map = {}
                     for k, v in labels_dict.items():
-                        rev_map[(v["agg"], v["key"])] = k
+                        rev_map[(v['agg'], v['key'],)] = k
                     if manipulator_func is None:
                         def label_rewriter_func(rule):
-                            field = rule.config.get("field")
+                            field = rule.config.get('field')
-                            if not field or rule.config["field"] in [
+                            if not field or rule.config['field'] in ['__OR__',
-                                "__OR__",
+                                                                     '__AND__', '__NOT__']:
-                                "__AND__",
-                                "__NOT__",
-                            ]:
                                 return
-                            to_map = field_mappings.get(rule.config["field"])
+                            to_map = field_mappings.get(rule.config['field'])
                             # we need to replace series field with _AE_NOT_FOUND_ to not match
                             # accidently some other field which happens to have the series that
                             # was used when the alert was created
                             if to_map:
-                                to_replace = rev_map.get(
+                                to_replace = rev_map.get((to_map['agg'], to_map['key'],),
-                                    (to_map["agg"], to_map["key"]), "_AE_NOT_FOUND_"
+                                                         '_AE_NOT_FOUND_')
                             else:
-                                to_replace = "_AE_NOT_FOUND_"
+                                to_replace = '_AE_NOT_FOUND_'
-                            rule.config["field"] = to_replace
+                            rule.config['field'] = to_replace
-                            rule.type_matrix[to_replace] = {"type": "float"}
+                            rule.type_matrix[to_replace] = {"type": 'float'}
                         manipulator_func = label_rewriter_func
                     return Rule(config, {}, config_manipulator=manipulator_func)

backend/src/appenlight/lib/social.py

0 +19 -21

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
-            from ziggurat_foundations.models.services.external_identity import (
+            from ziggurat_foundations.models.services.external_identity import \
-                ExternalIdentityService,
+                ExternalIdentityService
             from appenlight.models.external_identity import ExternalIdentity
             def handle_social_data(request, user, social_data):
                 social_data = social_data
                 update_identity = False
                 extng_id = ExternalIdentityService.by_external_id_and_provider(
-                    social_data["user"]["id"], social_data["credentials"].provider_name
+                    social_data['user']['id'],
+                    social_data['credentials'].provider_name
                 )
                 # fix legacy accounts with wrong google ID
-                if not extng_id and social_data["credentials"].provider_name == "google":
+                if not extng_id and social_data['credentials'].provider_name == 'google':
                     extng_id = ExternalIdentityService.by_external_id_and_provider(
-                        social_data["user"]["email"], social_data["credentials"].provider_name
+                        social_data['user']['email'],
+                        social_data['credentials'].provider_name
                     )
                 if extng_id:
                     extng_id.delete()
                     update_identity = True
-                if not social_data["user"]["id"]:
+                if not social_data['user']['id']:
                     request.session.flash(
-                        "No external user id found? Perhaps permissions for "
+                        'No external user id found? Perhaps permissions for '
-                        "authentication are set incorrectly",
+                        'authentication are set incorrectly', 'error')
-                        "error",
                     return False
                 if not extng_id or update_identity:
                     if not update_identity:
-                        request.session.flash(
+                        request.session.flash('Your external identity is now '
-                            "Your external identity is now " "connected with your account"
+                                              'connected with your account')
                     ex_identity = ExternalIdentity()
-                    ex_identity.external_id = social_data["user"]["id"]
+                    ex_identity.external_id = social_data['user']['id']
-                    ex_identity.external_user_name = social_data["user"]["user_name"]
+                    ex_identity.external_user_name = social_data['user']['user_name']
-                    ex_identity.provider_name = social_data["credentials"].provider_name
+                    ex_identity.provider_name = social_data['credentials'].provider_name
-                    ex_identity.access_token = social_data["credentials"].token
+                    ex_identity.access_token = social_data['credentials'].token
-                    ex_identity.token_secret = social_data["credentials"].token_secret
+                    ex_identity.token_secret = social_data['credentials'].token_secret
-                    ex_identity.alt_token = social_data["credentials"].refresh_token
+                    ex_identity.alt_token = social_data['credentials'].refresh_token
                     user.external_identities.append(ex_identity)
-                    request.session.pop("zigg.social_auth", None)
+                    request.session.pop('zigg.social_auth', None)

backend/src/appenlight/lib/utils/__init__.py

0 +116 -186

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             """
             Utility functions.
             """
             import logging
             import requests
             import hashlib
             import json
             import copy
             import uuid
             import appenlight.lib.helpers as h
             from collections import namedtuple
             from datetime import timedelta, datetime, date
             from dogpile.cache.api import NO_VALUE
             from appenlight.models import Datastores
-            from appenlight.validators import LogSearchSchema, TagListSchema, accepted_search_params
+            from appenlight.validators import (LogSearchSchema,
+                                               TagListSchema,
+                                               accepted_search_params)
             from itsdangerous import TimestampSigner
             from ziggurat_foundations.permissions import ALL_PERMISSIONS
-            from ziggurat_foundations.models.services.user import UserService
             from dateutil.relativedelta import relativedelta
             from dateutil.rrule import rrule, MONTHLY, DAILY
             log = logging.getLogger(__name__)
-            Stat = namedtuple("Stat", "start_interval value")
+            Stat = namedtuple('Stat', 'start_interval value')
             def default_extractor(item):
                 """
                 :param item - item to extract date from
                 """
-                if hasattr(item, "start_interval"):
+                if hasattr(item, 'start_interval'):
                     return item.start_interval
-                return item["start_interval"]
+                return item['start_interval']
             # fast gap generator
-            def gap_gen_default(start, step, itemiterator, end_time=None, iv_extractor=None):
+            def gap_gen_default(start, step, itemiterator, end_time=None,
+                                iv_extractor=None):
                 """ generates a list of time/value items based on step and itemiterator
                     if there are entries missing from iterator time/None will be returned
                     instead
                 :param start - datetime - what time should we start generating our values
                 :param step - timedelta - stepsize
                 :param itemiterator - iterable - we will check this iterable for values
                 corresponding to generated steps
                 :param end_time - datetime - when last step is >= end_time stop iterating
                 :param iv_extractor - extracts current step from iterable items
                 """
                 if not iv_extractor:
                     iv_extractor = default_extractor
                 next_step = start
                 minutes = step.total_seconds() / 60.0
                 while next_step.minute % minutes != 0:
                     next_step = next_step.replace(minute=next_step.minute - 1)
                 for item in itemiterator:
                     item_start_interval = iv_extractor(item)
                     # do we have a match for current time step in our data?
                     # no gen a new tuple with 0 values
                     while next_step < item_start_interval:
                         yield Stat(next_step, None)
                         next_step = next_step + step
                     if next_step == item_start_interval:
                         yield Stat(item_start_interval, item)
                         next_step = next_step + step
                 if end_time:
                     while next_step < end_time:
                         yield Stat(next_step, None)
                         next_step = next_step + step
             class DateTimeEncoder(json.JSONEncoder):
                 """ Simple datetime to ISO encoder for json serialization"""
                 def default(self, obj):
                     if isinstance(obj, date):
                         return obj.isoformat()
                     if isinstance(obj, datetime):
                         return obj.isoformat()
                     return json.JSONEncoder.default(self, obj)
-            def channelstream_request(
+            def channelstream_request(secret, endpoint, payload, throw_exceptions=False,
-                secret, endpoint, payload, throw_exceptions=False, servers=None
+                               servers=None):
-            ):
                 responses = []
                 if not servers:
                     servers = []
                 signer = TimestampSigner(secret)
                 sig_for_server = signer.sign(endpoint)
-                for secret, server in [(s["secret"], s["server"]) for s in servers]:
+                for secret, server in [(s['secret'], s['server']) for s in servers]:
                     response = {}
-                    secret_headers = {
+                    secret_headers = {'x-channelstream-secret': sig_for_server,
-                        "x-channelstream-secret": sig_for_server,
+                                      'x-channelstream-endpoint': endpoint,
-                        "x-channelstream-endpoint": endpoint,
+                                      'Content-Type': 'application/json'}
-                        "Content-Type": "application/json",
+                    url = '%s%s' % (server, endpoint)
-                    url = "%s%s" % (server, endpoint)
                     try:
-                        response = requests.post(
+                        response = requests.post(url,
-                            url,
+                                                 data=json.dumps(payload,
-                            data=json.dumps(payload, cls=DateTimeEncoder),
+                                                                 cls=DateTimeEncoder),
                                                  headers=secret_headers,
                                                  verify=False,
-                            timeout=2,
+                                                 timeout=2).json()
-                        ).json()
                     except requests.exceptions.RequestException as e:
                         if throw_exceptions:
                             raise
                     responses.append(response)
                 return responses
             def add_cors_headers(response):
                 # allow CORS
-                response.headers.add("Access-Control-Allow-Origin", "*")
+                response.headers.add('Access-Control-Allow-Origin', '*')
-                response.headers.add("XDomainRequestAllowed", "1")
+                response.headers.add('XDomainRequestAllowed', '1')
-                response.headers.add("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
+                response.headers.add('Access-Control-Allow-Methods', 'GET, POST, OPTIONS')
                 # response.headers.add('Access-Control-Allow-Credentials', 'true')
-                response.headers.add(
+                response.headers.add('Access-Control-Allow-Headers',
-                    "Access-Control-Allow-Headers",
+                                     'Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control, Pragma, Origin, Connection, Referer, Cookie')
-                    "Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control, Pragma, Origin, Connection, Referer, Cookie",
+                response.headers.add('Access-Control-Max-Age', '86400')
-                response.headers.add("Access-Control-Max-Age", "86400")
             from sqlalchemy.sql import compiler
             from psycopg2.extensions import adapt as sqlescape
             # or use the appropiate escape function from your db driver
             def compile_query(query):
                 dialect = query.session.bind.dialect
                 statement = query.statement
                 comp = compiler.SQLCompiler(dialect, statement)
                 comp.compile()
                 enc = dialect.encoding
                 params = {}
                 for k, v in comp.params.items():
                     if isinstance(v, str):
                         v = v.encode(enc)
                     params[k] = sqlescape(v)
                 return (comp.string.encode(enc) % params).decode(enc)
             def convert_es_type(input_data):
                 """
                 This might need to convert some text or other types to corresponding ES types
                 """
                 return str(input_data)
-            ProtoVersion = namedtuple("ProtoVersion", ["major", "minor", "patch"])
+            ProtoVersion = namedtuple('ProtoVersion', ['major', 'minor', 'patch'])
             def parse_proto(input_data):
                 try:
-                    parts = [int(x) for x in input_data.split(".")]
+                    parts = [int(x) for x in input_data.split('.')]
                     while len(parts) < 3:
                         parts.append(0)
                     return ProtoVersion(*parts)
                 except Exception as e:
-                    log.info("Unknown protocol version: %s" % e)
+                    log.info('Unknown protocol version: %s' % e)
                 return ProtoVersion(99, 99, 99)
-            def es_index_name_limiter(
+            def es_index_name_limiter(start_date=None, end_date=None, months_in_past=6,
-                start_date=None, end_date=None, months_in_past=6, ixtypes=None
+                                      ixtypes=None):
-            ):
                 """
                 This function limits the search to 6 months by default so we don't have to
                 query 300 elasticsearch indices for 20 years of historical data for example
                 """
                 # should be cached later
                 def get_possible_names():
-                    return list(Datastores.es.indices.get_alias("*"))
+                    return list(Datastores.es.aliases().keys())
                 possible_names = get_possible_names()
                 es_index_types = []
                 if not ixtypes:
-                    ixtypes = ["reports", "metrics", "logs"]
+                    ixtypes = ['reports', 'metrics', 'logs']
                 for t in ixtypes:
-                    if t == "reports":
+                    if t == 'reports':
-                        es_index_types.append("rcae_r_%s")
+                        es_index_types.append('rcae_r_%s')
-                    elif t == "logs":
+                    elif t == 'logs':
-                        es_index_types.append("rcae_l_%s")
+                        es_index_types.append('rcae_l_%s')
-                    elif t == "metrics":
+                    elif t == 'metrics':
-                        es_index_types.append("rcae_m_%s")
+                        es_index_types.append('rcae_m_%s')
-                    elif t == "uptime":
+                    elif t == 'uptime':
-                        es_index_types.append("rcae_uptime_ce_%s")
+                        es_index_types.append('rcae_u_%s')
-                    elif t == "slow_calls":
+                    elif t == 'slow_calls':
-                        es_index_types.append("rcae_sc_%s")
+                        es_index_types.append('rcae_sc_%s')
                 if start_date:
                     start_date = copy.copy(start_date)
                 else:
                     if not end_date:
                         end_date = datetime.utcnow()
                     start_date = end_date + relativedelta(months=months_in_past * -1)
                 if not end_date:
                     end_date = start_date + relativedelta(months=months_in_past)
-                index_dates = list(
+                index_dates = list(rrule(MONTHLY,
-                    rrule(
-                        MONTHLY,
                                          dtstart=start_date.date().replace(day=1),
                                          until=end_date.date(),
-                        count=36,
+                                         count=36))
                 index_names = []
                 for ix_type in es_index_types:
-                    to_extend = [
+                    to_extend = [ix_type % d.strftime('%Y_%m') for d in index_dates
-                        ix_type % d.strftime("%Y_%m")
+                                 if ix_type % d.strftime('%Y_%m') in possible_names]
-                        for d in index_dates
-                        if ix_type % d.strftime("%Y_%m") in possible_names
                     index_names.extend(to_extend)
-                    for day in list(
+                    for day in list(rrule(DAILY, dtstart=start_date.date(),
-                        rrule(DAILY, dtstart=start_date.date(), until=end_date.date(), count=366)
+                                          until=end_date.date(), count=366)):
-                    ):
+                        ix_name = ix_type % day.strftime('%Y_%m_%d')
-                        ix_name = ix_type % day.strftime("%Y_%m_%d")
                         if ix_name in possible_names:
                             index_names.append(ix_name)
                 return index_names
             def build_filter_settings_from_query_dict(
-                request, params=None, override_app_ids=None, resource_permissions=None
+                    request, params=None, override_app_ids=None,
-            ):
+                    resource_permissions=None):
                 """
                 Builds list of normalized search terms for ES from query params
                 ensuring application list is restricted to only applications user
                 has access to
                 :param params (dictionary)
                 :param override_app_ids - list of application id's to use instead of
                 applications user normally has access to
                 """
                 params = copy.deepcopy(params)
                 applications = []
                 if not resource_permissions:
-                    resource_permissions = ["view"]
+                    resource_permissions = ['view']
                 if request.user:
-                    applications = UserService.resources_with_perms(
+                    applications = request.user.resources_with_perms(
-                        request.user, resource_permissions, resource_types=["application"]
+                        resource_permissions, resource_types=['application'])
                 # CRITICAL - this ensures our resultset is limited to only the ones
                 # user has view permissions
                 all_possible_app_ids = set([app.resource_id for app in applications])
                 # if override is preset we force permission for app to be present
                 # this allows users to see dashboards and applications they would
                 # normally not be able to
                 if override_app_ids:
                     all_possible_app_ids = set(override_app_ids)
                 schema = LogSearchSchema().bind(resources=all_possible_app_ids)
                 tag_schema = TagListSchema()
                 filter_settings = schema.deserialize(params)
                 tag_list = []
                 for k, v in list(filter_settings.items()):
                     if k in accepted_search_params:
                         continue
-                    tag_list.append({"name": k, "value": v, "op": "eq"})
+                    tag_list.append({"name": k, "value": v, "op": 'eq'})
                     # remove the key from filter_settings
                     filter_settings.pop(k, None)
                 tags = tag_schema.deserialize(tag_list)
-                filter_settings["tags"] = tags
+                filter_settings['tags'] = tags
                 return filter_settings
             def gen_uuid():
                 return str(uuid.uuid4())
             def gen_uuid4_sha_hex():
                 return hashlib.sha1(uuid.uuid4().bytes).hexdigest()
             def permission_tuple_to_dict(data):
                 out = {
                     "user_name": None,
                     "perm_name": data.perm_name,
                     "owner": data.owner,
                     "type": data.type,
                     "resource_name": None,
                     "resource_type": None,
                     "resource_id": None,
                     "group_name": None,
-                    "group_id": None,
+                    "group_id": None
                 }
                 if data.user:
                     out["user_name"] = data.user.user_name
                 if data.perm_name == ALL_PERMISSIONS:
-                    out["perm_name"] = "__all_permissions__"
+                    out['perm_name'] = '__all_permissions__'
                 if data.resource:
-                    out["resource_name"] = data.resource.resource_name
+                    out['resource_name'] = data.resource.resource_name
-                    out["resource_type"] = data.resource.resource_type
+                    out['resource_type'] = data.resource.resource_type
-                    out["resource_id"] = data.resource.resource_id
+                    out['resource_id'] = data.resource.resource_id
                 if data.group:
-                    out["group_name"] = data.group.group_name
+                    out['group_name'] = data.group.group_name
-                    out["group_id"] = data.group.id
+                    out['group_id'] = data.group.id
                 return out
-            def get_cached_buckets(
+            def get_cached_buckets(request, stats_since, end_time, fn, cache_key,
-                request,
+                                   gap_gen=None, db_session=None, step_interval=None,
-                stats_since,
-                end_time,
-                fn,
-                cache_key,
-                gap_gen=None,
-                db_session=None,
-                step_interval=None,
                                    iv_extractor=None,
-                rerange=False,
+                                   rerange=False, *args, **kwargs):
-                *args,
-                **kwargs
-            ):
                 """ Takes "fn" that should return some data and tries to load the data
                 dividing it into daily buckets - if the stats_since and end time give a
                 delta bigger than 24hours, then only "todays" data is computed on the fly
                 :param request: (request) request object
                 :param stats_since: (datetime) start date of buckets range
                 :param end_time: (datetime) end date of buckets range - utcnow() if None
                 :param fn: (callable) callable to use to populate buckets should have
                 following signature:
                     def get_data(request, since_when, until, *args, **kwargs):
                 :param cache_key: (string) cache key that will be used to build bucket
                 caches
                 :param gap_gen: (callable) gap generator - should return step intervals
                 to use with out `fn` callable
                 :param db_session: (Session) sqlalchemy session
                 :param step_interval: (timedelta) optional step interval if we want to
                 override the default determined from total start/end time delta
                 :param iv_extractor: (callable) used to get step intervals from data
                 returned by `fn` callable
                 :param rerange: (bool) handy if we want to change ranges from hours to
                 days when cached data is missing - will shorten execution time if `fn`
                 callable supports that and we are working with multiple rows - like metrics
                 :param args:
                 :param kwargs:
                 :return: iterable
                 """
                 if not end_time:
                     end_time = datetime.utcnow().replace(second=0, microsecond=0)
                 delta = end_time - stats_since
                 # if smaller than 3 days we want to group by 5min else by 1h,
                 # for 60 min group by min
                 if not gap_gen:
                     gap_gen = gap_gen_default
                 if not iv_extractor:
                     iv_extractor = default_extractor
                 # do not use custom interval if total time range with new iv would exceed
                 # end time
                 if not step_interval or stats_since + step_interval >= end_time:
-                    if delta < h.time_deltas.get("12h")["delta"]:
+                    if delta < h.time_deltas.get('12h')['delta']:
                         step_interval = timedelta(seconds=60)
-                    elif delta < h.time_deltas.get("3d")["delta"]:
+                    elif delta < h.time_deltas.get('3d')['delta']:
                         step_interval = timedelta(seconds=60 * 5)
-                    elif delta > h.time_deltas.get("2w")["delta"]:
+                    elif delta > h.time_deltas.get('2w')['delta']:
                         step_interval = timedelta(days=1)
                     else:
                         step_interval = timedelta(minutes=60)
                 if step_interval >= timedelta(minutes=60):
-                    log.info(
+                    log.info('cached_buckets:{}: adjusting start time '
-                        "cached_buckets:{}: adjusting start time "
+                             'for hourly or daily intervals'.format(cache_key))
-                        "for hourly or daily intervals".format(cache_key)
                     stats_since = stats_since.replace(hour=0, minute=0)
-                ranges = [
+                ranges = [i.start_interval for i in list(gap_gen(stats_since,
-                    i.start_interval
+                                                                 step_interval, [],
-                    for i in list(gap_gen(stats_since, step_interval, [], end_time=end_time))
+                                                                 end_time=end_time))]
                 buckets = {}
-                storage_key = "buckets:" + cache_key + "{}|{}"
+                storage_key = 'buckets:' + cache_key + '{}|{}'
                 # this means we basicly cache per hour in 3-14 day intervals but i think
                 # its fine at this point - will be faster than db access anyways
                 if len(ranges) >= 1:
                     last_ranges = [ranges[-1]]
                 else:
                     last_ranges = []
                 if step_interval >= timedelta(minutes=60):
                     for r in ranges:
                         k = storage_key.format(step_interval.total_seconds(), r)
                         value = request.registry.cache_regions.redis_day_30.get(k)
                         # last buckets are never loaded from cache
-                        is_last_result = r >= end_time - timedelta(hours=6) or r in last_ranges
+                        is_last_result = (
+                            r >= end_time - timedelta(hours=6) or r in last_ranges)
                         if value is not NO_VALUE and not is_last_result:
-                            log.info(
+                            log.info("cached_buckets:{}: "
-                                "cached_buckets:{}: "
+                                     "loading range {} from cache".format(cache_key, r))
-                                "loading range {} from cache".format(cache_key, r)
                             buckets[r] = value
                         else:
-                            log.info(
+                            log.info("cached_buckets:{}: "
-                                "cached_buckets:{}: "
+                                     "loading range {} from storage".format(cache_key, r))
-                                "loading range {} from storage".format(cache_key, r)
                             range_size = step_interval
-                            if (
+                            if (step_interval == timedelta(minutes=60) and
-                                step_interval == timedelta(minutes=60)
+                                    not is_last_result and rerange):
-                                and not is_last_result
-                                and rerange
-                            ):
                                 range_size = timedelta(days=1)
                                 r = r.replace(hour=0, minute=0)
-                                log.info(
+                                log.info("cached_buckets:{}: "
-                                    "cached_buckets:{}: "
                                          "loading collapsed "
-                                    "range {} {}".format(cache_key, r, r + range_size)
+                                         "range {} {}".format(cache_key, r,
+                                                              r + range_size))
                             bucket_data = fn(
-                                request,
+                                request, r, r + range_size, step_interval,
-                                r,
+                                gap_gen, bucket_count=len(ranges), *args, **kwargs)
-                                r + range_size,
-                                step_interval,
-                                gap_gen,
-                                bucket_count=len(ranges),
-                                *args,
-                                **kwargs
                             for b in bucket_data:
                                 b_iv = iv_extractor(b)
                                 buckets[b_iv] = b
-                                k2 = storage_key.format(step_interval.total_seconds(), b_iv)
+                                k2 = storage_key.format(
+                                    step_interval.total_seconds(), b_iv)
                                 request.registry.cache_regions.redis_day_30.set(k2, b)
                     log.info("cached_buckets:{}: saving cache".format(cache_key))
                 else:
                     # bucket count is 1 for short time ranges <= 24h from now
-                    bucket_data = fn(
+                    bucket_data = fn(request, stats_since, end_time, step_interval,
-                        request,
+                                     gap_gen, bucket_count=1, *args, **kwargs)
-                        stats_since,
-                        end_time,
-                        step_interval,
-                        gap_gen,
-                        bucket_count=1,
-                        *args,
-                        **kwargs
                     for b in bucket_data:
                         buckets[iv_extractor(b)] = b
                 return buckets
-            def get_cached_split_data(
+            def get_cached_split_data(request, stats_since, end_time, fn, cache_key,
-                request, stats_since, end_time, fn, cache_key, db_session=None, *args, **kwargs
+                                      db_session=None, *args, **kwargs):
-            ):
                 """ Takes "fn" that should return some data and tries to load the data
                 dividing it into 2 buckets - cached "since_from" bucket and "today"
                 bucket - then the data can be reduced into single value
                 Data is cached if the stats_since and end time give a delta bigger
                 than 24hours - then only 24h is computed on the fly
                 """
                 if not end_time:
                     end_time = datetime.utcnow().replace(second=0, microsecond=0)
                 delta = end_time - stats_since
                 if delta >= timedelta(minutes=60):
-                    log.info(
+                    log.info('cached_split_data:{}: adjusting start time '
-                        "cached_split_data:{}: adjusting start time "
+                             'for hourly or daily intervals'.format(cache_key))
-                        "for hourly or daily intervals".format(cache_key)
                     stats_since = stats_since.replace(hour=0, minute=0)
-                storage_key = "buckets_split_data:" + cache_key + ":{}|{}"
+                storage_key = 'buckets_split_data:' + cache_key + ':{}|{}'
                 old_end_time = end_time.replace(hour=0, minute=0)
-                final_storage_key = storage_key.format(delta.total_seconds(), old_end_time)
+                final_storage_key = storage_key.format(delta.total_seconds(),
+                                                       old_end_time)
                 older_data = None
-                cdata = request.registry.cache_regions.redis_day_7.get(final_storage_key)
+                cdata = request.registry.cache_regions.redis_day_7.get(
+                    final_storage_key)
                 if cdata:
-                    log.info("cached_split_data:{}: found old " "bucket data".format(cache_key))
+                    log.info("cached_split_data:{}: found old "
+                             "bucket data".format(cache_key))
                     older_data = cdata
-                if stats_since < end_time - h.time_deltas.get("24h")["delta"] and not cdata:
+                if (stats_since < end_time - h.time_deltas.get('24h')['delta'] and
-                    log.info(
+                        not cdata):
-                        "cached_split_data:{}: didn't find the "
+                    log.info("cached_split_data:{}: didn't find the "
-                        "start bucket in cache so load older data".format(cache_key)
+                             "start bucket in cache so load older data".format(cache_key))
                     recent_stats_since = old_end_time
-                    older_data = fn(
+                    older_data = fn(request, stats_since, recent_stats_since,
-                        request,
+                                    db_session=db_session, *args, **kwargs)
-                        stats_since,
+                    request.registry.cache_regions.redis_day_7.set(final_storage_key,
-                        recent_stats_since,
+                                                                   older_data)
-                        db_session=db_session,
+                elif stats_since < end_time - h.time_deltas.get('24h')['delta']:
-                        *args,
-                        **kwargs
-                    request.registry.cache_regions.redis_day_7.set(final_storage_key, older_data)
-                elif stats_since < end_time - h.time_deltas.get("24h")["delta"]:
                     recent_stats_since = old_end_time
                 else:
                     recent_stats_since = stats_since
-                log.info(
+                log.info("cached_split_data:{}: loading fresh "
-                    "cached_split_data:{}: loading fresh "
+                         "data bucksts from last 24h ".format(cache_key))
-                    "data bucksts from last 24h ".format(cache_key)
+                todays_data = fn(request, recent_stats_since, end_time,
+                                 db_session=db_session, *args, **kwargs)
-                todays_data = fn(
-                    request, recent_stats_since, end_time, db_session=db_session, *args, **kwargs
                 return older_data, todays_data
             def in_batches(seq, size):
                 """
                 Splits am iterable into batches of specified size
                 :param seq (iterable)
                 :param size integer
                 """
                 return (seq[pos:pos + size] for pos in range(0, len(seq), size))
-            def get_es_info(cache_regions, es_conn):
-                @cache_regions.memory_min_10.cache_on_arguments()
-                def get_es_info_cached():
-                    returned_info = {"raw_info": es_conn.info()}
-                    returned_info["version"] = returned_info["raw_info"]["version"]["number"].split(
-                        "."
-                    return returned_info
-                return get_es_info_cached()

backend/src/appenlight/lib/utils/airbrake.py

0 +66 -85

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             import uuid
             from datetime import datetime
             log = logging.getLogger(__name__)
             def parse_airbrake_xml(request):
                 root = request.context.airbrake_xml_etree
-                error = root.find("error")
+                error = root.find('error')
-                notifier = root.find("notifier")
+                notifier = root.find('notifier')
-                server_env = root.find("server-environment")
+                server_env = root.find('server-environment')
-                request_data = root.find("request")
+                request_data = root.find('request')
-                user = root.find("current-user")
+                user = root.find('current-user')
                 if request_data is not None:
-                    cgi_data = request_data.find("cgi-data")
+                    cgi_data = request_data.find('cgi-data')
                     if cgi_data is None:
                         cgi_data = []
                 error_dict = {
-                    "class_name": error.findtext("class") or "",
+                    'class_name': error.findtext('class') or '',
-                    "error": error.findtext("message") or "",
+                    'error': error.findtext('message') or '',
                     "occurences": 1,
                     "http_status": 500,
                     "priority": 5,
-                    "server": "unknown",
+                    "server": 'unknown',
-                    "url": "unknown",
+                    'url': 'unknown', 'request': {}
-                    "request": {},
                 }
                 if user is not None:
-                    error_dict["username"] = user.findtext("username") or user.findtext("id")
+                    error_dict['username'] = user.findtext('username') or \
+                                             user.findtext('id')
                 if notifier is not None:
-                    error_dict["client"] = notifier.findtext("name")
+                    error_dict['client'] = notifier.findtext('name')
                 if server_env is not None:
-                    error_dict["server"] = server_env.findtext("hostname", "unknown")
+                    error_dict["server"] = server_env.findtext('hostname', 'unknown')
-                whitelist_environ = [
+                whitelist_environ = ['REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME',
-                    "REMOTE_USER",
+                                     'CONTENT_TYPE', 'HTTP_REFERER']
-                    "REMOTE_ADDR",
-                    "SERVER_NAME",
-                    "CONTENT_TYPE",
-                    "HTTP_REFERER",
                 if request_data is not None:
-                    error_dict["url"] = request_data.findtext("url", "unknown")
+                    error_dict['url'] = request_data.findtext('url', 'unknown')
-                    component = request_data.findtext("component")
+                    component = request_data.findtext('component')
-                    action = request_data.findtext("action")
+                    action = request_data.findtext('action')
                     if component and action:
-                        error_dict["view_name"] = "%s:%s" % (component, action)
+                        error_dict['view_name'] = '%s:%s' % (component, action)
                     for node in cgi_data:
-                        key = node.get("key")
+                        key = node.get('key')
-                        if key.startswith("HTTP") or key in whitelist_environ:
+                        if key.startswith('HTTP') or key in whitelist_environ:
-                            error_dict["request"][key] = node.text
+                            error_dict['request'][key] = node.text
-                        elif "query_parameters" in key:
+                        elif 'query_parameters' in key:
-                            error_dict["request"]["GET"] = {}
+                            error_dict['request']['GET'] = {}
                             for x in node:
-                                error_dict["request"]["GET"][x.get("key")] = x.text
+                                error_dict['request']['GET'][x.get('key')] = x.text
-                        elif "request_parameters" in key:
+                        elif 'request_parameters' in key:
-                            error_dict["request"]["POST"] = {}
+                            error_dict['request']['POST'] = {}
                             for x in node:
-                                error_dict["request"]["POST"][x.get("key")] = x.text
+                                error_dict['request']['POST'][x.get('key')] = x.text
-                        elif key.endswith("cookie"):
+                        elif key.endswith('cookie'):
-                            error_dict["request"]["COOKIE"] = {}
+                            error_dict['request']['COOKIE'] = {}
                             for x in node:
-                                error_dict["request"]["COOKIE"][x.get("key")] = x.text
+                                error_dict['request']['COOKIE'][x.get('key')] = x.text
-                        elif key.endswith("request_id"):
+                        elif key.endswith('request_id'):
-                            error_dict["request_id"] = node.text
+                            error_dict['request_id'] = node.text
-                        elif key.endswith("session"):
+                        elif key.endswith('session'):
-                            error_dict["request"]["SESSION"] = {}
+                            error_dict['request']['SESSION'] = {}
                             for x in node:
-                                error_dict["request"]["SESSION"][x.get("key")] = x.text
+                                error_dict['request']['SESSION'][x.get('key')] = x.text
                         else:
-                            if key in ["rack.session.options"]:
+                            if key in ['rack.session.options']:
                                 # skip secret configs
                                 continue
                             try:
                                 if len(node):
-                                    error_dict["request"][key] = dict(
+                                    error_dict['request'][key] = dict(
-                                        [(x.get("key"), x.text) for x in node]
+                                        [(x.get('key'), x.text,) for x in node])
                                 else:
-                                    error_dict["request"][key] = node.text
+                                    error_dict['request'][key] = node.text
                             except Exception as e:
-                                log.warning("Airbrake integration exception: %s" % e)
+                                log.warning('Airbrake integration exception: %s' % e)
-                        error_dict["request"].pop("HTTP_COOKIE", "")
+                        error_dict['request'].pop('HTTP_COOKIE', '')
-                    error_dict["ip"] = error_dict.pop("REMOTE_ADDR", "")
+                    error_dict['ip'] = error_dict.pop('REMOTE_ADDR', '')
-                    error_dict["user_agent"] = error_dict.pop("HTTP_USER_AGENT", "")
+                    error_dict['user_agent'] = error_dict.pop('HTTP_USER_AGENT', '')
-                if "request_id" not in error_dict:
+                if 'request_id' not in error_dict:
-                    error_dict["request_id"] = str(uuid.uuid4())
+                    error_dict['request_id'] = str(uuid.uuid4())
                 if request.context.possibly_public:
                     # set ip for reports that come from airbrake js client
                     error_dict["timestamp"] = datetime.utcnow()
                     if request.environ.get("HTTP_X_FORWARDED_FOR"):
-                        ip = request.environ.get("HTTP_X_FORWARDED_FOR", "")
+                        ip = request.environ.get("HTTP_X_FORWARDED_FOR", '')
-                        first_ip = ip.split(",")[0]
+                        first_ip = ip.split(',')[0]
                         remote_addr = first_ip.strip()
                     else:
-                        remote_addr = request.environ.get("HTTP_X_REAL_IP") or request.environ.get(
+                        remote_addr = (request.environ.get("HTTP_X_REAL_IP") or
-                            "REMOTE_ADDR"
+                                       request.environ.get('REMOTE_ADDR'))
                     error_dict["ip"] = remote_addr
-                blacklist = [
+                blacklist = ['password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf',
-                    "password",
+                             'session', 'test']
-                    "passwd",
-                    "pwd",
-                    "auth_tkt",
-                    "secret",
-                    "csrf",
-                    "session",
-                    "test",
                 lines = []
-                for l in error.find("backtrace"):
+                for l in error.find('backtrace'):
-                    lines.append(
+                    lines.append({'file': l.get("file", ""),
+                                  'line': l.get("number", ""),
-                            "file": l.get("file", ""),
+                                  'fn': l.get("method", ""),
-                            "line": l.get("number", ""),
+                                  'module': l.get("module", ""),
-                            "fn": l.get("method", ""),
+                                  'cline': l.get("method", ""),
-                            "module": l.get("module", ""),
+                                  'vars': {}})
-                            "cline": l.get("method", ""),
+                error_dict['traceback'] = list(reversed(lines))
-                            "vars": {},
-                error_dict["traceback"] = list(reversed(lines))
                 # filtering is not provided by airbrake
                 keys_to_check = (
-                    error_dict["request"].get("COOKIE"),
+                    error_dict['request'].get('COOKIE'),
-                    error_dict["request"].get("COOKIES"),
+                    error_dict['request'].get('COOKIES'),
-                    error_dict["request"].get("POST"),
+                    error_dict['request'].get('POST'),
-                    error_dict["request"].get("SESSION"),
+                    error_dict['request'].get('SESSION'),
                 )
                 for source in [_f for _f in keys_to_check if _f]:
                     for k in source.keys():
                         for bad_key in blacklist:
                             if bad_key in k.lower():
-                                source[k] = "***"
+                                source[k] = '***'
                 return error_dict

backend/src/appenlight/lib/utils/date_utils.py

0 +11 -11

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             from datetime import tzinfo, timedelta, datetime
             from dateutil.relativedelta import relativedelta
             import logging
             log = logging.getLogger(__name__)
             def to_relativedelta(time_delta):
-                return relativedelta(
+                return relativedelta(seconds=int(time_delta.total_seconds()),
-                    seconds=int(time_delta.total_seconds()), microseconds=time_delta.microseconds
+                                     microseconds=time_delta.microseconds)
-            def convert_date(date_str, return_utcnow_if_wrong=True, normalize_future=False):
+            def convert_date(date_str, return_utcnow_if_wrong=True,
+                             normalize_future=False):
                 utcnow = datetime.utcnow()
                 if isinstance(date_str, datetime):
                     # get rid of tzinfo
                     return date_str.replace(tzinfo=None)
                 if not date_str and return_utcnow_if_wrong:
                     return utcnow
                 try:
                     try:
-                        if "Z" in date_str:
+                        if 'Z' in date_str:
-                            date_str = date_str[: date_str.index("Z")]
+                            date_str = date_str[:date_str.index('Z')]
-                        if "." in date_str:
+                        if '.' in date_str:
-                            date = datetime.strptime(date_str, "%Y-%m-%dT%H:%M:%S.%f")
+                            date = datetime.strptime(date_str, '%Y-%m-%dT%H:%M:%S.%f')
                         else:
-                            date = datetime.strptime(date_str, "%Y-%m-%dT%H:%M:%S")
+                            date = datetime.strptime(date_str, '%Y-%m-%dT%H:%M:%S')
                     except Exception:
                         # bw compat with old client
-                        date = datetime.strptime(date_str, "%Y-%m-%d %H:%M:%S,%f")
+                        date = datetime.strptime(date_str, '%Y-%m-%d %H:%M:%S,%f')
                 except Exception:
                     if return_utcnow_if_wrong:
                         date = utcnow
                     else:
                         date = None
                 if normalize_future and date and date > (utcnow + timedelta(minutes=3)):
-                    log.warning("time %s in future + 3 min, normalizing" % date)
+                    log.warning('time %s in future + 3 min, normalizing' % date)
                     return utcnow
                 return date

backend/src/appenlight/lib/utils/sentry.py

0 +149 -170

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             from datetime import timedelta
             from appenlight.lib.enums import LogLevelPython, ParsedSentryEventType
             EXCLUDED_LOG_VARS = [
-                "args",
+                'args', 'asctime', 'created', 'exc_info', 'exc_text', 'filename',
-                "asctime",
+                'funcName', 'levelname', 'levelno', 'lineno', 'message', 'module', 'msecs',
-                "created",
+                'msg', 'name', 'pathname', 'process', 'processName', 'relativeCreated',
-                "exc_info",
+                'thread', 'threadName']
-                "exc_text",
-                "filename",
-                "funcName",
-                "levelname",
-                "levelno",
-                "lineno",
-                "message",
-                "module",
-                "msecs",
-                "msg",
-                "name",
-                "pathname",
-                "process",
-                "processName",
-                "relativeCreated",
-                "thread",
-                "threadName",
             EXCLUDE_SENTRY_KEYS = [
-                "csp",
+                'csp',
-                "culprit",
+                'culprit',
-                "event_id",
+                'event_id',
-                "exception",
+                'exception',
-                "extra",
+                'extra',
-                "level",
+                'level',
-                "logentry",
+                'logentry',
-                "logger",
+                'logger',
-                "message",
+                'message',
-                "modules",
+                'modules',
-                "platform",
+                'platform',
-                "query",
+                'query',
-                "release",
+                'release',
-                "request",
+                'request',
-                "sentry.interfaces.Csp",
+                'sentry.interfaces.Csp', 'sentry.interfaces.Exception',
-                "sentry.interfaces.Exception",
+                'sentry.interfaces.Http', 'sentry.interfaces.Message',
-                "sentry.interfaces.Http",
+                'sentry.interfaces.Query',
-                "sentry.interfaces.Message",
+                'sentry.interfaces.Stacktrace',
-                "sentry.interfaces.Query",
+                'sentry.interfaces.Template', 'sentry.interfaces.User',
-                "sentry.interfaces.Stacktrace",
+                'sentry.interfaces.csp.Csp',
-                "sentry.interfaces.Template",
+                'sentry.interfaces.exception.Exception',
-                "sentry.interfaces.User",
+                'sentry.interfaces.http.Http',
-                "sentry.interfaces.csp.Csp",
+                'sentry.interfaces.message.Message',
-                "sentry.interfaces.exception.Exception",
+                'sentry.interfaces.query.Query',
-                "sentry.interfaces.http.Http",
+                'sentry.interfaces.stacktrace.Stacktrace',
-                "sentry.interfaces.message.Message",
+                'sentry.interfaces.template.Template',
-                "sentry.interfaces.query.Query",
+                'sentry.interfaces.user.User', 'server_name',
-                "sentry.interfaces.stacktrace.Stacktrace",
+                'stacktrace',
-                "sentry.interfaces.template.Template",
+                'tags',
-                "sentry.interfaces.user.User",
+                'template',
-                "server_name",
+                'time_spent',
-                "stacktrace",
+                'timestamp',
-                "tags",
+                'user']
-                "template",
-                "time_spent",
-                "timestamp",
-                "user",
             def get_keys(list_of_keys, json_body):
                 for k in list_of_keys:
                     if k in json_body:
                         return json_body[k]
             def get_logentry(json_body):
-                key_names = [
+                key_names = ['logentry',
-                    "logentry",
+                             'sentry.interfaces.message.Message',
-                    "sentry.interfaces.message.Message",
+                             'sentry.interfaces.Message'
-                    "sentry.interfaces.Message",
                              ]
                 logentry = get_keys(key_names, json_body)
                 return logentry
             def get_exception(json_body):
                 parsed_exception = {}
-                key_names = [
+                key_names = ['exception',
-                    "exception",
+                             'sentry.interfaces.exception.Exception',
-                    "sentry.interfaces.exception.Exception",
+                             'sentry.interfaces.Exception'
-                    "sentry.interfaces.Exception",
                              ]
                 exception = get_keys(key_names, json_body) or {}
                 if exception:
                     if isinstance(exception, dict):
-                        exception = exception["values"][0]
+                        exception = exception['values'][0]
                     else:
                         exception = exception[0]
-                parsed_exception["type"] = exception.get("type")
+                parsed_exception['type'] = exception.get('type')
-                parsed_exception["value"] = exception.get("value")
+                parsed_exception['value'] = exception.get('value')
-                parsed_exception["module"] = exception.get("module")
+                parsed_exception['module'] = exception.get('module')
                 parsed_stacktrace = get_stacktrace(exception) or {}
                 parsed_exception = exception or {}
                 return parsed_exception, parsed_stacktrace
             def get_stacktrace(json_body):
                 parsed_stacktrace = []
-                key_names = [
+                key_names = ['stacktrace',
-                    "stacktrace",
+                             'sentry.interfaces.stacktrace.Stacktrace',
-                    "sentry.interfaces.stacktrace.Stacktrace",
+                             'sentry.interfaces.Stacktrace'
-                    "sentry.interfaces.Stacktrace",
                              ]
                 stacktrace = get_keys(key_names, json_body)
                 if stacktrace:
-                    for frame in stacktrace["frames"]:
+                    for frame in stacktrace['frames']:
                         parsed_stacktrace.append(
+                            {"cline": frame.get('context_line', ''),
-                                "cline": frame.get("context_line", ""),
+                             "file": frame.get('filename', ''),
-                                "file": frame.get("filename", ""),
+                             "module": frame.get('module', ''),
-                                "module": frame.get("module", ""),
+                             "fn": frame.get('function', ''),
-                                "fn": frame.get("function", ""),
+                             "line": frame.get('lineno', ''),
-                                "line": frame.get("lineno", ""),
+                             "vars": list(frame.get('vars', {}).items())
-                                "vars": list(frame.get("vars", {}).items()),
                              }
                         )
                 return parsed_stacktrace
             def get_template(json_body):
                 parsed_template = {}
-                key_names = [
+                key_names = ['template',
-                    "template",
+                             'sentry.interfaces.template.Template',
-                    "sentry.interfaces.template.Template",
+                             'sentry.interfaces.Template'
-                    "sentry.interfaces.Template",
                              ]
                 template = get_keys(key_names, json_body)
                 if template:
-                    for frame in template["frames"]:
+                    for frame in template['frames']:
                         parsed_template.append(
+                            {"cline": frame.get('context_line', ''),
-                                "cline": frame.get("context_line", ""),
+                             "file": frame.get('filename', ''),
-                                "file": frame.get("filename", ""),
+                             "fn": '',
-                                "fn": "",
+                             "line": frame.get('lineno', ''),
-                                "line": frame.get("lineno", ""),
+                             "vars": []
-                                "vars": [],
                              }
                         )
                 return parsed_template
             def get_request(json_body):
                 parsed_http = {}
-                key_names = ["request", "sentry.interfaces.http.Http", "sentry.interfaces.Http"]
+                key_names = ['request',
+                             'sentry.interfaces.http.Http',
+                             'sentry.interfaces.Http'
+                             ]
                 http = get_keys(key_names, json_body) or {}
                 for k, v in http.items():
-                    if k == "headers":
+                    if k == 'headers':
-                        parsed_http["headers"] = {}
+                        parsed_http['headers'] = {}
-                        for sk, sv in http["headers"].items():
+                        for sk, sv in http['headers'].items():
-                            parsed_http["headers"][sk.title()] = sv
+                            parsed_http['headers'][sk.title()] = sv
                     else:
                         parsed_http[k.lower()] = v
                 return parsed_http
             def get_user(json_body):
                 parsed_user = {}
-                key_names = ["user", "sentry.interfaces.user.User", "sentry.interfaces.User"]
+                key_names = ['user',
+                             'sentry.interfaces.user.User',
+                             'sentry.interfaces.User'
+                             ]
                 user = get_keys(key_names, json_body)
                 if user:
-                    parsed_user["id"] = user.get("id")
+                    parsed_user['id'] = user.get('id')
-                    parsed_user["username"] = user.get("username")
+                    parsed_user['username'] = user.get('username')
-                    parsed_user["email"] = user.get("email")
+                    parsed_user['email'] = user.get('email')
-                    parsed_user["ip_address"] = user.get("ip_address")
+                    parsed_user['ip_address'] = user.get('ip_address')
                 return parsed_user
             def get_query(json_body):
                 query = None
-                key_name = ["query", "sentry.interfaces.query.Query", "sentry.interfaces.Query"]
+                key_name = ['query',
+                            'sentry.interfaces.query.Query',
+                            'sentry.interfaces.Query'
+                            ]
                 query = get_keys(key_name, json_body)
                 return query
             def parse_sentry_event(json_body):
-                request_id = json_body.get("event_id")
+                request_id = json_body.get('event_id')
                 # required
-                message = json_body.get("message")
+                message = json_body.get('message')
-                log_timestamp = json_body.get("timestamp")
+                log_timestamp = json_body.get('timestamp')
-                level = json_body.get("level")
+                level = json_body.get('level')
                 if isinstance(level, int):
                     level = LogLevelPython.key_from_value(level)
-                namespace = json_body.get("logger")
+                namespace = json_body.get('logger')
-                language = json_body.get("platform")
+                language = json_body.get('platform')
                 # optional
-                server_name = json_body.get("server_name")
+                server_name = json_body.get('server_name')
-                culprit = json_body.get("culprit")
+                culprit = json_body.get('culprit')
-                release = json_body.get("release")
+                release = json_body.get('release')
-                tags = json_body.get("tags", {})
+                tags = json_body.get('tags', {})
-                if hasattr(tags, "items"):
+                if hasattr(tags, 'items'):
                     tags = list(tags.items())
-                extra = json_body.get("extra", {})
+                extra = json_body.get('extra', {})
-                if hasattr(extra, "items"):
+                if hasattr(extra, 'items'):
                     extra = list(extra.items())
                 parsed_req = get_request(json_body)
                 user = get_user(json_body)
                 template = get_template(json_body)
                 query = get_query(json_body)
                 # other unidentified keys found
-                other_keys = [
+                other_keys = [(k, json_body[k]) for k in json_body.keys()
-                    (k, json_body[k]) for k in json_body.keys() if k not in EXCLUDE_SENTRY_KEYS
+                              if k not in EXCLUDE_SENTRY_KEYS]
                 logentry = get_logentry(json_body)
                 if logentry:
-                    message = logentry["message"]
+                    message = logentry['message']
                 exception, stacktrace = get_exception(json_body)
                 alt_stacktrace = get_stacktrace(json_body)
                 event_type = None
                 if not exception and not stacktrace and not alt_stacktrace and not template:
                     event_type = ParsedSentryEventType.LOG
                     event_dict = {
-                        "log_level": level,
+                        'log_level': level,
-                        "message": message,
+                        'message': message,
-                        "namespace": namespace,
+                        'namespace': namespace,
-                        "request_id": request_id,
+                        'request_id': request_id,
-                        "server": server_name,
+                        'server': server_name,
-                        "date": log_timestamp,
+                        'date': log_timestamp,
-                        "tags": tags,
+                        'tags': tags
                     }
-                    event_dict["tags"].extend(
+                    event_dict['tags'].extend(
-                        [(k, v) for k, v in extra if k not in EXCLUDED_LOG_VARS]
+                        [(k, v) for k, v in extra if k not in EXCLUDED_LOG_VARS])
                     # other keys can be various object types
-                    event_dict["tags"].extend([(k, v) for k, v in other_keys if isinstance(v, str)])
+                    event_dict['tags'].extend([(k, v) for k, v in other_keys
+                                               if isinstance(v, str)])
                     if culprit:
-                        event_dict["tags"].append(("sentry_culprit", culprit))
+                        event_dict['tags'].append(('sentry_culprit', culprit))
                     if language:
-                        event_dict["tags"].append(("sentry_language", language))
+                        event_dict['tags'].append(('sentry_language', language))
                     if release:
-                        event_dict["tags"].append(("sentry_release", release))
+                        event_dict['tags'].append(('sentry_release', release))
                 if exception or stacktrace or alt_stacktrace or template:
                     event_type = ParsedSentryEventType.ERROR_REPORT
                     event_dict = {
-                        "client": "sentry",
+                        'client': 'sentry',
-                        "error": message,
+                        'error': message,
-                        "namespace": namespace,
+                        'namespace': namespace,
-                        "request_id": request_id,
+                        'request_id': request_id,
-                        "server": server_name,
+                        'server': server_name,
-                        "start_time": log_timestamp,
+                        'start_time': log_timestamp,
-                        "end_time": None,
+                        'end_time': None,
-                        "tags": tags,
+                        'tags': tags,
-                        "extra": extra,
+                        'extra': extra,
-                        "language": language,
+                        'language': language,
-                        "view_name": json_body.get("culprit"),
+                        'view_name': json_body.get('culprit'),
-                        "http_status": None,
+                        'http_status': None,
-                        "username": None,
+                        'username': None,
-                        "url": parsed_req.get("url"),
+                        'url': parsed_req.get('url'),
-                        "ip": None,
+                        'ip': None,
-                        "user_agent": None,
+                        'user_agent': None,
-                        "request": None,
+                        'request': None,
-                        "slow_calls": None,
+                        'slow_calls': None,
-                        "request_stats": None,
+                        'request_stats': None,
-                        "traceback": None,
+                        'traceback': None
                     }
-                    event_dict["extra"].extend(other_keys)
+                    event_dict['extra'].extend(other_keys)
                     if release:
-                        event_dict["tags"].append(("sentry_release", release))
+                        event_dict['tags'].append(('sentry_release', release))
-                    event_dict["request"] = parsed_req
+                    event_dict['request'] = parsed_req
-                    if "headers" in parsed_req:
+                    if 'headers' in parsed_req:
-                        event_dict["user_agent"] = parsed_req["headers"].get("User-Agent")
+                        event_dict['user_agent'] = parsed_req['headers'].get('User-Agent')
-                    if "env" in parsed_req:
+                    if 'env' in parsed_req:
-                        event_dict["ip"] = parsed_req["env"].get("REMOTE_ADDR")
+                        event_dict['ip'] = parsed_req['env'].get('REMOTE_ADDR')
-                    ts_ms = int(json_body.get("time_spent") or 0)
+                    ts_ms = int(json_body.get('time_spent') or 0)
                     if ts_ms > 0:
-                        event_dict["end_time"] = event_dict["start_time"] + timedelta(
+                        event_dict['end_time'] = event_dict['start_time'] + \
-                            milliseconds=ts_ms
+                                                 timedelta(milliseconds=ts_ms)
                     if stacktrace or alt_stacktrace or template:
-                        event_dict["traceback"] = stacktrace or alt_stacktrace or template
+                        event_dict['traceback'] = stacktrace or alt_stacktrace or template
                     for k in list(event_dict.keys()):
                         if event_dict[k] is None:
                             del event_dict[k]
                     if user:
-                        event_dict["username"] = user["username"] or user["id"] or user["email"]
+                        event_dict['username'] = user['username'] or user['id'] \
+                                                 or user['email']
                 return event_dict, event_type

backend/src/appenlight/migrations/__init__.py

0 +2 0

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.

backend/src/appenlight/migrations/env.py

0 +6 -10

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             from alembic import context
             from sqlalchemy import engine_from_config, pool, MetaData
             from logging.config import fileConfig
             from appenlight.models import NAMING_CONVENTION
             # this is the Alembic Config object, which provides
             # access to the values within the .ini file in use.
             config = context.config
             # Interpret the config file for Python logging.
             # This line sets up loggers basically.
             if config.config_file_name:
                 fileConfig(config.config_file_name)
             # add your model's MetaData object here
             # for 'autogenerate' support
             # from myapp import mymodel
             # target_metadata = mymodel.Base.metadata
             target_metadata = MetaData(naming_convention=NAMING_CONVENTION)
             # other values from the config, defined by the needs of env.py,
             # can be acquired:
             # my_important_option = config.get_main_option("my_important_option")
             # ... etc.
-            VERSION_TABLE_NAME = "alembic_appenlight_version"
+            VERSION_TABLE_NAME = 'alembic_appenlight_version'
             def run_migrations_offline():
                 """Run migrations in 'offline' mode.
                 This configures the context with just a URL
                 and not an Engine, though an Engine is acceptable
                 here as well.  By skipping the Engine creation
                 we don't even need a DBAPI to be available.
                 Calls to context.execute() here emit the given string to the
                 script output.
                 """
                 url = config.get_main_option("sqlalchemy.url")
-                context.configure(
+                context.configure(url=url, target_metadata=target_metadata,
-                    url=url,
-                    target_metadata=target_metadata,
                                   transaction_per_migration=True,
-                    version_table=VERSION_TABLE_NAME,
+                                  version_table=VERSION_TABLE_NAME)
                 with context.begin_transaction():
                     context.run_migrations()
             def run_migrations_online():
                 """Run migrations in 'online' mode.
                 In this scenario we need to create an Engine
                 and associate a connection with the context.
                 """
                 engine = engine_from_config(
                     config.get_section(config.config_ini_section),
-                    prefix="sqlalchemy.",
+                    prefix='sqlalchemy.',
-                    poolclass=pool.NullPool,
+                    poolclass=pool.NullPool)
                 connection = engine.connect()
                 context.configure(
                     connection=connection,
                     target_metadata=target_metadata,
                     transaction_per_migration=True,
-                    version_table=VERSION_TABLE_NAME,
+                    version_table=VERSION_TABLE_NAME
                 )
                 try:
                     with context.begin_transaction():
                         context.run_migrations()
                 finally:
                     connection.close()
             if context.is_offline_mode():
                 run_migrations_offline()
             else:
                 run_migrations_online()

backend/src/appenlight/migrations/versions/55b6e612672f_initial_tables.py

0 +299 -488

This diff has been collapsed as it changes many lines, (787 lines changed) Show them Hide them
	@@ -1,813 +1,624 b''
	1	# -- coding: utf-8 --	1	# -- coding: utf-8 --
	2		2
	3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors	3	# Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
	4	#	4	#
	5	# Licensed under the Apache License, Version 2.0 (the "License");	5	# Licensed under the Apache License, Version 2.0 (the "License");
	6	# you may not use this file except in compliance with the License.	6	# you may not use this file except in compliance with the License.
	7	# You may obtain a copy of the License at	7	# You may obtain a copy of the License at
	8	#	8	#
	9	# http://www.apache.org/licenses/LICENSE-2.0	9	# http://www.apache.org/licenses/LICENSE-2.0
	10	#	10	#
	11	# Unless required by applicable law or agreed to in writing, software	11	# Unless required by applicable law or agreed to in writing, software
	12	# distributed under the License is distributed on an "AS IS" BASIS,	12	# distributed under the License is distributed on an "AS IS" BASIS,
	13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.	13	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	14	# See the License for the specific language governing permissions and	14	# See the License for the specific language governing permissions and
	15	# limitations under the License.	15	# limitations under the License.
	16		16
	17	"""initial tables	17	"""initial tables
	18		18
	19	Revision ID: 55b6e612672f	19	Revision ID: 55b6e612672f
	20	Revises: None	20	Revises: None
	21	Create Date: 2014-10-13 23:47:38.295159	21	Create Date: 2014-10-13 23:47:38.295159
	22		22
	23	"""	23	"""
	24		24
	25	# revision identifiers, used by Alembic.	25	# revision identifiers, used by Alembic.
	26	revision = "55b6e612672f"	26	revision = '55b6e612672f'
	27	down_revision = None	27	down_revision = None
	28		28
	29	from alembic import op	29	from alembic import op
	30	import sqlalchemy as sa	30	import sqlalchemy as sa
	31		31
	32		32
	33	def upgrade():	33	def upgrade():
	34	op.add_column("users", sa.Column("first_name", sa.Unicode(25)))	34	op.add_column('users', sa.Column('first_name', sa.Unicode(25)))
	35	op.add_column("users", sa.Column("last_name", sa.Unicode(50)))	35	op.add_column('users', sa.Column('last_name', sa.Unicode(50)))
	36	op.add_column("users", sa.Column("company_name", sa.Unicode(255)))	36	op.add_column('users', sa.Column('company_name', sa.Unicode(255)))
	37	op.add_column("users", sa.Column("company_address", sa.Unicode(255)))	37	op.add_column('users', sa.Column('company_address', sa.Unicode(255)))
	38	op.add_column("users", sa.Column("phone1", sa.Unicode(25)))	38	op.add_column('users', sa.Column('phone1', sa.Unicode(25)))
	39	op.add_column("users", sa.Column("phone2", sa.Unicode(25)))	39	op.add_column('users', sa.Column('phone2', sa.Unicode(25)))
	40	op.add_column("users", sa.Column("zip_code", sa.Unicode(25)))	40	op.add_column('users', sa.Column('zip_code', sa.Unicode(25)))
	41	op.add_column(	41	op.add_column('users', sa.Column('default_report_sort', sa.Unicode(20), nullable=False, server_default="newest"))
	42	"users",	42	op.add_column('users', sa.Column('city', sa.Unicode(128)))
	43	sa.Column(	43	op.add_column('users', sa.Column('notes', sa.UnicodeText, server_default=''))
	44	"default_report_sort",	44	op.add_column('users', sa.Column('notifications', sa.Boolean(), nullable=False, server_default='true'))
	45	sa.Unicode(20),	45	op.add_column('users', sa.Column('registration_ip', sa.Unicode(40), nullable=False, server_default=''))
	46	nullable=False,
	47	server_default="newest",
	48	),
	49	)
	50	op.add_column("users", sa.Column("city", sa.Unicode(128)))
	51	op.add_column("users", sa.Column("notes", sa.UnicodeText, server_default=""))
	52	op.add_column(
	53	"users",
	54	sa.Column("notifications", sa.Boolean(), nullable=False, server_default="true"),
	55	)
	56	op.add_column(
	57	"users",
	58	sa.Column("registration_ip", sa.Unicode(40), nullable=False, server_default=""),
	59	)
	60		46
	61	op.create_table(	47	op.create_table(
	62	"integrations",	48	'integrations',
	63	sa.Column("id", sa.Integer(), primary_key=True),	49	sa.Column('id', sa.Integer(), primary_key=True),
	64	sa.Column(	50	sa.Column('resource_id', sa.Integer(),
	65	"resource_id",	51	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	66	sa.Integer(),	52	ondelete='cascade')),
	67	sa.ForeignKey(	53	sa.Column('integration_name', sa.Unicode(64)),
	68	"resources.resource_id", onupdate="cascade", ondelete="cascade"	54	sa.Column('config', sa.dialects.postgresql.JSON, nullable=False),
	69	),	55	sa.Column('modified_date', sa.DateTime(), nullable=False, server_default=sa.func.now()),
	70	),	56	sa.Column('external_id', sa.Unicode(255)),
	71	sa.Column(~~"integration_name"~~, sa.Unicode(64)),	57	sa.Column('external_id2', sa.Unicode(255))
	72	sa.Column("config", sa.dialects.postgresql.JSON, nullable=False),
	73	sa.Column(
	74	"modified_date", sa.DateTime(), nullable=False, server_default=sa.func.now()
	75	),
	76	sa.Column("external_id", sa.Unicode(255)),
	77	sa.Column("external_id2", sa.Unicode(255)),
	78	)	58	)
	79		59
	80	op.create_table(	60	op.create_table(
	81	"alert_channels",	61	'alert_channels',
	82	sa.Column(	62	sa.Column('owner_id', sa.Integer(),
	83	"owner_id",	63	sa.ForeignKey('users.id', onupdate='cascade',
	84	sa.Integer(),	64	ondelete='cascade'), nullable=False),
	85	sa.ForeignKey("users.id", onupdate="cascade", ondelete="cascade"),	65	sa.Column('channel_name', sa.Unicode(25), nullable=False),
	86	nullable=False,	66	sa.Column('channel_value', sa.Unicode(80), nullable=False),
	87	),	67	sa.Column('channel_json_conf', sa.dialects.postgresql.JSON, nullable=False),
	88	sa.Column(~~"channel_name"~~, sa.~~Unicode~~(25), nullable=False),	68	sa.Column('channel_validated', sa.Boolean, nullable=False, server_default='False'),
	89	sa.Column(~~"channel_value"~~, sa.~~Unicode~~(80), nullable=False),	69	sa.Column('send_alerts', sa.Boolean, nullable=False, server_default='True'),
	90	sa.Column("channel_json_conf", sa.dialects.postgresql.JSON, nullable=False),	70	sa.Column('notify_only_first', sa.Boolean, nullable=False, server_default='False'),
	91	sa.Column(	71	sa.Column('daily_digest', sa.Boolean, nullable=False, server_default='True'),
	92	"channel_validated", sa.Boolean, nullable=False, server_default="False"	72	sa.Column('pkey', sa.Integer(), primary_key=True),
	93	),	73	sa.Column('integration_id', sa.Integer,
	94	sa.Column("send_alerts", sa.Boolean, nullable=False, server_default="True"),	74	sa.ForeignKey('integrations.id', onupdate='cascade',
	95	sa.Column(	75	ondelete='cascade')),
	96	"notify_only_first", sa.Boolean, nullable=False, server_default="False"	76	)
	97	),	77	op.create_unique_constraint('uq_alert_channels', 'alert_channels',
	98	sa.Column("daily_digest", sa.Boolean, nullable=False, server_default="True"),	78	["owner_id", "channel_name", "channel_value"])
	99	sa.Column("pkey", sa.Integer(), primary_key=True),
	100	sa.Column(
	101	"integration_id",
	102	sa.Integer,
	103	sa.ForeignKey("integrations.id", onupdate="cascade", ondelete="cascade"),
	104	),
	105	)
	106	op.create_unique_constraint(
	107	"uq_alert_channels",
	108	"alert_channels",
	109	["owner_id", "channel_name", "channel_value"],
	110	)
	111		79
	112	op.create_table(	80	op.create_table(
	113	"alert_channels_actions",	81	'alert_channels_actions',
	114	sa.Column("owner_id", sa.Integer(), nullable=False),	82	sa.Column('owner_id', sa.Integer(), nullable=False),
	115	sa.Column(	83	sa.Column('resource_id', sa.Integer(),
	116	"resource_id",	84	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	117	sa.Integer(),	85	ondelete='cascade')),
	118	sa.ForeignKey(	86	sa.Column('pkey', sa.Integer(), primary_key=True),
	119	"resources.resource_id", onupdate="cascade", ondelete="cascade"	87	sa.Column('action', sa.Unicode(10), nullable=False, server_default='always'),
	120	),	88	sa.Column('rule', sa.dialects.postgresql.JSON),
	121	),	89	sa.Column('type', sa.Unicode(10), index=True),
	122	sa.Column(~~"pkey"~~, sa.~~Integer~~(), ~~primary_key~~=True),	90	sa.Column('other_id', sa.Unicode(40), index=True),
	123	sa.Column("action", sa.Unicode(10), nullable=False, server_default="always"),	91	sa.Column('config', sa.dialects.postgresql.JSON),
	124	sa.Column("rule", sa.dialects.postgresql.JSON),	92	sa.Column('name', sa.Unicode(255), server_default='')
	125	sa.Column("type", sa.Unicode(10), index=True),
	126	sa.Column("other_id", sa.Unicode(40), index=True),
	127	sa.Column("config", sa.dialects.postgresql.JSON),
	128	sa.Column("name", sa.Unicode(255), server_default=""),
	129	)	93	)
	130		94
			95
	131	op.create_table(	96	op.create_table(
	132	"application_postprocess_conf",	97	'application_postprocess_conf',
	133	sa.Column("pkey", sa.Integer(), primary_key=True),	98	sa.Column('pkey', sa.Integer(), primary_key=True),
	134	sa.Column("do", sa.Unicode(25), nullable=False),	99	sa.Column('do', sa.Unicode(25), nullable=False),
	135	sa.Column("new_value", sa.UnicodeText(), nullable=False, server_default=""),	100	sa.Column('new_value', sa.UnicodeText(), nullable=False, server_default=''),
	136	sa.Column(	101	sa.Column('resource_id', sa.Integer(),
	137	"resource_id",	102	sa.ForeignKey('resources.resource_id',
	138	sa.Integer(),	103	onupdate='cascade',
	139	sa.ForeignKey(	104	ondelete='cascade'), nullable=False),
	140	"resources.resource_id", onupdate="cascade", ondelete="cascade"	105	sa.Column('rule', sa.dialects.postgresql.JSON),
	141	),
	142	nullable=False,
	143	),
	144	sa.Column("rule", sa.dialects.postgresql.JSON),
	145	)	106	)
	146		107
	147	op.create_table(	108	op.create_table(
	148	"applications",	109	'applications',
	149	sa.Column(	110	sa.Column('resource_id', sa.Integer(),
	150	"resource_id",	111	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	151	sa.Integer(),	112	ondelete='cascade'), nullable=False,
	152	sa.ForeignKey(	113	primary_key=True, autoincrement=False),
	153	"resources.resource_id", onupdate="cascade", ondelete="cascade"	114	sa.Column('domains', sa.UnicodeText, nullable=False),
	154	),	115	sa.Column('api_key', sa.Unicode(32), nullable=False, index=True),
	155	nullable=False,	116	sa.Column('default_grouping', sa.Unicode(20), nullable=False, server_default='url_type'),
	156	primary_key=True,	117	sa.Column('public_key', sa.Unicode(32), nullable=False, index=True),
	157	autoincrement=False,	118	sa.Column('error_report_threshold', sa.Integer(), server_default='10', nullable=False),
	158	),	119	sa.Column('slow_report_threshold', sa.Integer(), server_default='10', nullable=False),
	159	sa.Column(~~"domains"~~, sa.~~UnicodeText~~, nullable=False),	120	sa.Column('apdex_threshold', sa.Float(), server_default='0.7', nullable=False),
	160	sa.Column("api_key", sa.Unicode(32), nullable=False, index=True),	121	sa.Column('allow_permanent_storage', sa.Boolean(), server_default="false", nullable=False),
	161	sa.Column(	122	)
	162	"default_grouping",	123	op.create_unique_constraint(None, 'applications',
	163	sa.Unicode(20),	124	["public_key"])
	164	nullable=False,	125	op.create_unique_constraint(None, 'applications',
	165	server_default="url_type",	126	["api_key"])
	166	),
	167	sa.Column("public_key", sa.Unicode(32), nullable=False, index=True),
	168	sa.Column(
	169	"error_report_threshold", sa.Integer(), server_default="10", nullable=False
	170	),
	171	sa.Column(
	172	"slow_report_threshold", sa.Integer(), server_default="10", nullable=False
	173	),
	174	sa.Column("apdex_threshold", sa.Float(), server_default="0.7", nullable=False),
	175	sa.Column(
	176	"allow_permanent_storage",
	177	sa.Boolean(),
	178	server_default="false",
	179	nullable=False,
	180	),
	181	)
	182	op.create_unique_constraint(None, "applications", ["public_key"])
	183	op.create_unique_constraint(None, "applications", ["api_key"])
	184		127
	185	op.create_table(	128	op.create_table(
	186	"metrics",	129	'metrics',
	187	sa.Column("pkey", sa.types.BigInteger, nullable=False, primary_key=True),	130	sa.Column('pkey', sa.types.BigInteger, nullable=False, primary_key=True),
	188	sa.Column(	131	sa.Column('resource_id', sa.Integer(),
	189	"resource_id",	132	sa.ForeignKey('resources.resource_id',
	190	sa.Integer(),	133	onupdate='cascade',
	191	sa.ForeignKey(	134	ondelete='cascade')),
	192	"resources.resource_id", onupdate="cascade", ondelete="cascade"	135	sa.Column('timestamp', sa.DateTime),
	193	),	136	sa.Column('namespace', sa.Unicode(255)),
	194	),	137	sa.Column('tags', sa.dialects.postgresql.JSON, server_default="{}")
	195	sa.Column("timestamp", sa.DateTime),
	196	sa.Column("namespace", sa.Unicode(255)),
	197	sa.Column("tags", sa.dialects.postgresql.JSON, server_default="{}"),
	198	)	138	)
	199		139
	200	op.create_table(	140	op.create_table(
	201	"events",	141	'events',
	202	sa.Column("id", sa.Integer, nullable=False, primary_key=True),	142	sa.Column('id', sa.Integer, nullable=False, primary_key=True),
	203	sa.Column("start_date", sa.DateTime, nullable=False, index=True),	143	sa.Column('start_date', sa.DateTime, nullable=False, index=True),
	204	sa.Column("end_date", sa.DateTime),	144	sa.Column('end_date', sa.DateTime),
	205	sa.Column("status", sa.Integer(), nullable=False, index=True),	145	sa.Column('status', sa.Integer(), nullable=False, index=True),
	206	sa.Column("event_type", sa.Integer(), nullable=False, index=True),	146	sa.Column('event_type', sa.Integer(), nullable=False, index=True),
	207	sa.Column("origin_user_id", sa.Integer()),	147	sa.Column('origin_user_id', sa.Integer()),
	208	sa.Column("target_user_id", sa.Integer()),	148	sa.Column('target_user_id', sa.Integer()),
	209	sa.Column("resource_id", sa.Integer(), index=True),	149	sa.Column('resource_id', sa.Integer(), index=True),
	210	sa.Column("text", sa.UnicodeText, server_default=""),	150	sa.Column('text', sa.UnicodeText, server_default=''),
	211	sa.Column("values", sa.dialects.postgresql.JSON),	151	sa.Column('values', sa.dialects.postgresql.JSON),
	212	sa.Column("target_id", sa.Integer()),	152	sa.Column('target_id', sa.Integer()),
	213	sa.Column("target_uuid", sa.Unicode(40), index=True),	153	sa.Column('target_uuid', sa.Unicode(40), index=True)
	214	)	154	)
	215		155
	216	op.create_table(	156	op.create_table(
	217	"logs",	157	'logs',
	218	sa.Column("log_id", sa.types.BigInteger, nullable=False, primary_key=True),	158	sa.Column('log_id', sa.types.BigInteger, nullable=False, primary_key=True),
	219	sa.Column(	159	sa.Column('resource_id', sa.Integer(),
	220	"resource_id",	160	sa.ForeignKey('resources.resource_id',
	221	sa.Integer(),	161	onupdate='cascade',
	222	sa.ForeignKey(	162	ondelete='cascade')),
	223	"resources.resource_id", onupdate="cascade", ondelete="cascade"	163	sa.Column('log_level', sa.SmallInteger(), nullable=False),
	224	),	164	sa.Column('primary_key', sa.Unicode(128), nullable=True),
	225	),	165	sa.Column('message', sa.UnicodeText, nullable=False, server_default=''),
	226	sa.Column("log_level", sa.SmallInteger(), nullable=False),	166	sa.Column('timestamp', sa.DateTime),
	227	sa.Column(~~"primary_key"~~, sa.Unicode(~~128~~), ~~nullable~~=~~True~~),	167	sa.Column('namespace', sa.Unicode(255)),
	228	sa.Column("message", sa.UnicodeText, nullable=False, server_default=""),	168	sa.Column('request_id', sa.Unicode(40)),
	229	sa.Column("timestamp", sa.DateTime),	169	sa.Column('tags', sa.dialects.postgresql.JSON, server_default="{}"),
	230	sa.Column("namespace", sa.Unicode(255)),	170	sa.Column('permanent', sa.Boolean(), server_default="false",
	231	sa.Column("request_id", sa.Unicode(40)),	171	nullable=False)
	232	sa.Column("tags", sa.dialects.postgresql.JSON, server_default="{}"),
	233	sa.Column("permanent", sa.Boolean(), server_default="false", nullable=False),
	234	)	172	)
	235		173
	236	op.create_table(	174	op.create_table(
	237	"reports_groups",	175	'reports_groups',
	238	sa.Column("id", sa.types.BigInteger, primary_key=True),	176	sa.Column('id', sa.types.BigInteger, primary_key=True),
	239	sa.Column(	177	sa.Column('resource_id', sa.Integer,
	240	"resource_id",	178	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	241	sa.Integer,	179	ondelete='cascade'), nullable=False),
	242	sa.ForeignKey(	180	sa.Column('priority', sa.Integer, nullable=False, server_default="5"),
	243	"resources.resource_id", onupdate="cascade", ondelete="cascade"	181	sa.Column('first_timestamp', sa.DateTime(), nullable=False, server_default=sa.func.now()),
	244	),	182	sa.Column('last_timestamp', sa.DateTime()),
	245	nullable=False,	183	sa.Column('error', sa.UnicodeText, nullable=False, server_default=""),
	246	),	184	sa.Column('grouping_hash', sa.Unicode(40), nullable=False, server_default=""),
	247	sa.Column(~~"priority"~~, sa.~~Integer~~, nullable=False, server_default="5"),	185	sa.Column('triggered_postprocesses_ids', sa.dialects.postgresql.JSON, nullable=False, server_default="[]"),
	248	sa.Column(	186	sa.Column('report_type', sa.Integer, nullable=False, server_default="0"),
	249	"first_timestamp",	187	sa.Column('total_reports', sa.Integer, nullable=False, server_default="0"),
	250	sa.DateTime(),	188	sa.Column('last_report', sa.Integer, nullable=False, server_default="0"),
	251	nullable=False,	189	sa.Column('occurences', sa.Integer, nullable=False, server_default="1"),
	252	server_default=sa.func.now(),	190	sa.Column('average_duration', sa.Float(), nullable=False, server_default="0"),
	253	),	191	sa.Column('summed_duration', sa.Float(), nullable=False, server_default="0"),
	254	sa.Column("last_timestamp", sa.DateTime()),	192	sa.Column('notified', sa.Boolean, nullable=False, server_default="False"),
	255	sa.Column(~~"error"~~, sa.~~UnicodeText~~, nullable=False, server_default=""),	193	sa.Column('fixed', sa.Boolean, nullable=False, server_default="False"),
	256	sa.Column(~~"grouping_hash"~~, sa.~~Unicode~~(40), nullable=False, server_default=""),	194	sa.Column('public', sa.Boolean, nullable=False, server_default="False"),
	257	sa.Column(	195	sa.Column('read', sa.Boolean, nullable=False, server_default="False"),
	258	"triggered_postprocesses_ids",
	259	sa.dialects.postgresql.JSON,
	260	nullable=False,
	261	server_default="[]",
	262	),
	263	sa.Column("report_type", sa.Integer, nullable=False, server_default="0"),
	264	sa.Column("total_reports", sa.Integer, nullable=False, server_default="0"),
	265	sa.Column("last_report", sa.Integer, nullable=False, server_default="0"),
	266	sa.Column("occurences", sa.Integer, nullable=False, server_default="1"),
	267	sa.Column("average_duration", sa.Float(), nullable=False, server_default="0"),
	268	sa.Column("summed_duration", sa.Float(), nullable=False, server_default="0"),
	269	sa.Column("notified", sa.Boolean, nullable=False, server_default="False"),
	270	sa.Column("fixed", sa.Boolean, nullable=False, server_default="False"),
	271	sa.Column("public", sa.Boolean, nullable=False, server_default="False"),
	272	sa.Column("read", sa.Boolean, nullable=False, server_default="False"),
	273	)	196	)
	274		197
	275	op.create_table(	198	op.create_table(
	276	"reports",	199	'reports',
	277	sa.Column("id", sa.types.BigInteger, primary_key=True),	200	sa.Column('id', sa.types.BigInteger, primary_key=True),
	278	sa.Column(	201	sa.Column('group_id', sa.types.BigInteger,
	279	"group_id",	202	sa.ForeignKey('reports_groups.id', onupdate='cascade',
	280	sa.types.BigInteger,	203	ondelete='cascade'), nullable=False, index=True),
	281	sa.ForeignKey("reports_groups.id", onupdate="cascade", ondelete="cascade"),	204	sa.Column('resource_id', sa.Integer, nullable=False, index=True),
	282	nullable=False,	205	sa.Column('report_type', sa.Integer, nullable=False, server_default="0"),
	283	index=True,	206	sa.Column('error', sa.UnicodeText, nullable=False, server_default=""),
	284	),	207	sa.Column('extra', sa.dialects.postgresql.JSON, nullable=False, server_default="{}"),
	285	sa.Column(~~"resource_id"~~, sa.~~Integer~~, nullable=False, ~~index~~=~~True~~),	208	sa.Column('request', sa.dialects.postgresql.JSON, nullable=False, server_default="{}"),
	286	sa.Column(~~"report_type"~~, sa.~~Integer~~, nullable=False, server_default="0"),	209	sa.Column('tags', sa.dialects.postgresql.JSON, nullable=False, server_default="{}"),
	287	sa.Column(~~"error"~~, sa.Unicode~~Text~~, nullable=False, server_default=""),	210	sa.Column('ip', sa.Unicode(39), nullable=False, server_default=""),
	288	sa.Column(	211	sa.Column('username', sa.Unicode(255), nullable=False, server_default=""),
	289	~~"extra"~~, sa.~~dialects~~.~~postgresql~~.~~JSON~~, nullable=False, server_default="{}"	212	sa.Column('user_agent', sa.Unicode(512), nullable=False, server_default=""),
	290	),	213	sa.Column('url', sa.UnicodeText, nullable=False, server_default=""),
	291	sa.Column(	214	sa.Column('request_id', sa.Unicode(40), nullable=False, server_default=""),
	292	~~"request"~~, sa.dialects.postgresql.JSON, nullable=False, server_default="{}"	215	sa.Column('request_stats', sa.dialects.postgresql.JSON, nullable=False, server_default="{}"),
	293	),	216	sa.Column('traceback', sa.dialects.postgresql.JSON, nullable=False, server_default="{}"),
	294	sa.Column(	217	sa.Column('traceback_hash', sa.Unicode(40), nullable=False, server_default=""),
	295	"tags", sa.dialects.postgresql.JSON, nullable=False, server_default="{}"	218	sa.Column('start_time', sa.DateTime(), nullable=False, server_default=sa.func.now()),
	296	),	219	sa.Column('end_time', sa.DateTime()),
	297	sa.Column(~~"ip"~~, sa.~~Unicode~~(39), nullable=False, server_default=""),	220	sa.Column('report_group_time', sa.DateTime, index=True, nullable=False, server_default=sa.func.now()),
	298	sa.Column(~~"username"~~, sa.~~Unicode~~(~~255~~), nullable=False, server_default=""),	221	sa.Column('duration', sa.Float(), nullable=False, server_default="0"),
	299	sa.Column("user_agent", sa.Unicode(512), nullable=False, server_default=""),	222	sa.Column('http_status', sa.Integer, index=True),
	300	sa.Column("url", sa.UnicodeText, nullable=False, server_default=""),	223	sa.Column('url_domain', sa.Unicode(128)),
	301	sa.Column("request_id", sa.Unicode(40), nullable=False, server_default=""),	224	sa.Column('url_path', sa.UnicodeText),
	302	sa.Column(	225	sa.Column('language', sa.Integer, server_default="0"),
	303	"request_stats",	226	)
	304	sa.dialects.postgresql.JSON,	227	op.create_index(None, 'reports',
	305	nullable=False,	228	[sa.text("(tags ->> 'server_name')")])
	306	server_default="{}",	229	op.create_index(None, 'reports',
	307	),	230	[sa.text("(tags ->> 'view_name')")])
	308	sa.Column(
	309	"traceback",
	310	sa.dialects.postgresql.JSON,
	311	nullable=False,
	312	server_default="{}",
	313	),
	314	sa.Column("traceback_hash", sa.Unicode(40), nullable=False, server_default=""),
	315	sa.Column(
	316	"start_time", sa.DateTime(), nullable=False, server_default=sa.func.now()
	317	),
	318	sa.Column("end_time", sa.DateTime()),
	319	sa.Column(
	320	"report_group_time",
	321	sa.DateTime,
	322	index=True,
	323	nullable=False,
	324	server_default=sa.func.now(),
	325	),
	326	sa.Column("duration", sa.Float(), nullable=False, server_default="0"),
	327	sa.Column("http_status", sa.Integer, index=True),
	328	sa.Column("url_domain", sa.Unicode(128)),
	329	sa.Column("url_path", sa.UnicodeText),
	330	sa.Column("language", sa.Integer, server_default="0"),
	331	)
	332	op.create_index(None, "reports", [sa.text("(tags ->> 'server_name')")])
	333	op.create_index(None, "reports", [sa.text("(tags ->> 'view_name')")])
	334		231
	335	op.create_table(	232	op.create_table(
	336	"reports_assignments",	233	'reports_assignments',
	337	sa.Column("group_id", sa.types.BigInteger, nullable=False, primary_key=True),	234	sa.Column('group_id', sa.types.BigInteger, nullable=False, primary_key=True),
	338	sa.Column(	235	sa.Column('owner_id', sa.Integer,
	339	"owner_id",	236	sa.ForeignKey('users.id', onupdate='cascade',ondelete='cascade'),
	340	sa.Integer,	237	nullable=False, primary_key=True),
	341	sa.ForeignKey("users.id", onupdate="cascade", ondelete="cascade"),	238	sa.Column('report_time', sa.DateTime, nullable=False)
	342	nullable=False,
	343	primary_key=True,
	344	),
	345	sa.Column("report_time", sa.DateTime, nullable=False),
	346	)	239	)
	347		240
	348	op.create_table(	241	op.create_table(
	349	"reports_comments",	242	'reports_comments',
	350	sa.Column("comment_id", sa.Integer, primary_key=True),	243	sa.Column('comment_id', sa.Integer, primary_key=True),
	351	sa.Column("body", sa.UnicodeText, nullable=False, server_default=""),	244	sa.Column('body', sa.UnicodeText, nullable=False, server_default=''),
	352	sa.Column(	245	sa.Column('owner_id', sa.Integer,
	353	"owner_id",	246	sa.ForeignKey('users.id', onupdate='cascade',
	354	sa.Integer,	247	ondelete='set null'), nullable=True),
	355	sa.ForeignKey("users.id", onupdate="cascade", ondelete="set null"),	248	sa.Column('created_timestamp', sa.DateTime, nullable=False, server_default=sa.func.now()),
	356	nullable=True,	249	sa.Column('report_time', sa.DateTime, nullable=False),
	357	),	250	sa.Column('group_id', sa.types.BigInteger, nullable=False)
	358	sa.Column(
	359	"created_timestamp",
	360	sa.DateTime,
	361	nullable=False,
	362	server_default=sa.func.now(),
	363	),
	364	sa.Column("report_time", sa.DateTime, nullable=False),
	365	sa.Column("group_id", sa.types.BigInteger, nullable=False),
	366	)	251	)
	367		252
	368	op.create_table(	253	op.create_table(
	369	"reports_stats",	254	'reports_stats',
	370	sa.Column("resource_id", sa.Integer, nullable=False, index=True),	255	sa.Column('resource_id', sa.Integer, nullable=False, index=True),
	371	sa.Column("start_interval", sa.DateTime, nullable=False, index=True),	256	sa.Column('start_interval', sa.DateTime, nullable=False, index=True),
	372	sa.Column("group_id", sa.types.BigInteger, index=True),	257	sa.Column('group_id', sa.types.BigInteger, index=True),
	373	sa.Column(	258	sa.Column('occurences', sa.Integer, nullable=False, server_default='0', index=True),
	374	"occurences", sa.Integer, nullable=False, server_default="0", index=True	259	sa.Column('owner_user_id', sa.Integer),
	375	),	260	sa.Column('type', sa.Integer, index=True, nullable=False),
	376	sa.Column("owner_user_id", sa.Integer),	261	sa.Column('duration', sa.Float(), server_default='0'),
	377	sa.Column("type", sa.Integer, index=True, nullable=False),	262	sa.Column('server_name', sa.Unicode(128),
	378	sa.Column("duration", sa.Float(), server_default="0"),	263	server_default=''),
	379	sa.Column(~~"server_name"~~, sa.Unicode(128), ~~server_default~~=""),	264	sa.Column('view_name', sa.Unicode(128),
	380	sa.Column("view_name", sa.Unicode(128), server_default=""),	265	server_default=''),
	381	sa.Column("id", sa.BigInteger(), nullable=False, primary_key=True),	266	sa.Column('id', sa.BigInteger(), nullable=False, primary_key=True),
	382	)	267	)
	383	op.create_index(	268	op.create_index('ix_reports_stats_start_interval_group_id', 'reports_stats',
	384	"~~ix_reports_stats_~~start_interval~~_group_id~~",	269	["start_interval", "group_id"])
	385	"reports_stats",
	386	["start_interval", "group_id"],
	387	)
	388		270
	389	op.create_table(	271	op.create_table(
	390	"slow_calls",	272	'slow_calls',
	391	sa.Column("id", sa.types.BigInteger, primary_key=True),	273	sa.Column('id', sa.types.BigInteger, primary_key=True),
	392	sa.Column(	274	sa.Column('report_id', sa.types.BigInteger, sa.ForeignKey('reports.id', onupdate='cascade', ondelete='cascade'),
	393	"report_id",	275	nullable=False, index=True),
	394	sa.types.BigInteger,	276	sa.Column('duration', sa.Float(), nullable=False, server_default="0", index=True),
	395	sa.ForeignKey("reports.id", onupdate="cascade", ondelete="cascade"),	277	sa.Column('timestamp', sa.DateTime, nullable=False, server_default=sa.func.now(), index=True),
	396	nullable=False,	278	sa.Column('report_group_time', sa.DateTime, index=True, nullable=False, server_default=sa.func.now()),
	397	index=True,	279	sa.Column('type', sa.Unicode(16), nullable=False, index=True),
	398	),	280	sa.Column('statement', sa.UnicodeText, nullable=False, server_default=''),
	399	sa.Column(	281	sa.Column('parameters', sa.dialects.postgresql.JSON, nullable=False),
	400	"duration", sa.Float(), nullable=False, server_default="0", index=True	282	sa.Column('location', sa.UnicodeText, server_default=''),
	401	),	283	sa.Column('subtype', sa.Unicode(16), nullable=False, index=True),
	402	sa.Column(	284	sa.Column('resource_id', sa.Integer, nullable=False, index=True),
	403	"timestamp",	285	sa.Column('statement_hash', sa.Unicode(60), index=True)
	404	sa.DateTime,
	405	nullable=False,
	406	server_default=sa.func.now(),
	407	index=True,
	408	),
	409	sa.Column(
	410	"report_group_time",
	411	sa.DateTime,
	412	index=True,
	413	nullable=False,
	414	server_default=sa.func.now(),
	415	),
	416	sa.Column("type", sa.Unicode(16), nullable=False, index=True),
	417	sa.Column("statement", sa.UnicodeText, nullable=False, server_default=""),
	418	sa.Column("parameters", sa.dialects.postgresql.JSON, nullable=False),
	419	sa.Column("location", sa.UnicodeText, server_default=""),
	420	sa.Column("subtype", sa.Unicode(16), nullable=False, index=True),
	421	sa.Column("resource_id", sa.Integer, nullable=False, index=True),
	422	sa.Column("statement_hash", sa.Unicode(60), index=True),
	423	)	286	)
	424		287
	425	op.create_table(	288	op.create_table(
	426	"tags",	289	'tags',
	427	sa.Column("id", sa.types.BigInteger, primary_key=True),	290	sa.Column('id', sa.types.BigInteger, primary_key=True),
	428	sa.Column(	291	sa.Column('resource_id', sa.Integer,
	429	"resource_id",	292	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	430	sa.Integer,	293	ondelete='cascade')),
	431	sa.ForeignKey(	294	sa.Column('first_timestamp', sa.DateTime, nullable=False, server_default=sa.func.now()),
	432	"resources.resource_id", onupdate="cascade", ondelete="cascade"	295	sa.Column('last_timestamp', sa.DateTime, nullable=False, server_default=sa.func.now()),
	433	),	296	sa.Column('name', sa.Unicode(32), nullable=False),
	434	),	297	sa.Column('value', sa.dialects.postgresql.JSON, nullable=False),
	435	sa.Column(	298	sa.Column('times_seen', sa.Integer, nullable=False, server_default='1')
	436	"first_timestamp", sa.DateTime, nullable=False, server_default=sa.func.now()
	437	),
	438	sa.Column(
	439	"last_timestamp", sa.DateTime, nullable=False, server_default=sa.func.now()
	440	),
	441	sa.Column("name", sa.Unicode(32), nullable=False),
	442	sa.Column("value", sa.dialects.postgresql.JSON, nullable=False),
	443	sa.Column("times_seen", sa.Integer, nullable=False, server_default="1"),
	444	)	299	)
	445		300
	446	op.create_table(	301	op.create_table(
	447	"auth_tokens",	302	'auth_tokens',
	448	sa.Column("id", sa.Integer, nullable=False, primary_key=True),	303	sa.Column('id', sa.Integer, nullable=False, primary_key=True),
	449	sa.Column("token", sa.Unicode),	304	sa.Column('token', sa.Unicode),
	450	sa.Column(	305	sa.Column('creation_date', sa.DateTime, nullable=False, server_default=sa.func.now()),
	451	"creation_date", sa.DateTime, nullable=False, server_default=sa.func.now()	306	sa.Column('expires', sa.DateTime),
	452	),	307	sa.Column('owner_id', sa.Integer,
	453	sa.Column("expires", sa.DateTime),	308	sa.ForeignKey('users.id', onupdate='cascade',
	454	sa.Column(	309	ondelete='cascade')),
	455	"owner_id",	310	sa.Column('description', sa.Unicode),
	456	sa.Integer,
	457	sa.ForeignKey("users.id", onupdate="cascade", ondelete="cascade"),
	458	),
	459	sa.Column("description", sa.Unicode),
	460	)	311	)
	461		312
	462	op.create_table(	313	op.create_table(
	463	"channels_actions",	314	'channels_actions',
	464	sa.Column(	315	sa.Column('channel_pkey', sa.Integer,
	465	"channel_pkey",	316	sa.ForeignKey('alert_channels.pkey',
	466	sa.Integer,	317	ondelete='CASCADE', onupdate='CASCADE')),
	467	sa.ForeignKey(	318	sa.Column('action_pkey', sa.Integer,
	468	"alert_channels.pkey", ondelete="CASCADE", onupdate="CASCADE"	319	sa.ForeignKey('alert_channels_actions.pkey',
	469	),	320	ondelete='CASCADE', onupdate='CASCADE'))
	470	),
	471	sa.Column(
	472	"action_pkey",
	473	sa.Integer,
	474	sa.ForeignKey(
	475	"alert_channels_actions.pkey", ondelete="CASCADE", onupdate="CASCADE"
	476	),
	477	),
	478	)	321	)
	479		322
	480	op.create_table(	323	op.create_table(
	481	"config",	324	'config',
	482	sa.Column("key", sa.Unicode(128), primary_key=True),	325	sa.Column('key', sa.Unicode(128), primary_key=True),
	483	sa.Column("section", sa.Unicode(128), primary_key=True),	326	sa.Column('section', sa.Unicode(128), primary_key=True),
	484	sa.Column("value", sa.dialects.postgresql.JSON, ~~server_default~~="{}"),	327	sa.Column('value', sa.dialects.postgresql.JSON,
			328	server_default="{}")
	485	)	329	)
	486		330
	487	op.create_table(	331	op.create_table(
	488	"plugin_configs",	332	'plugin_configs',
	489	sa.Column("id", sa.Integer, primary_key=True),	333	sa.Column('id', sa.Integer, primary_key=True),
	490	sa.Column("plugin_name", sa.Unicode(128)),	334	sa.Column('plugin_name', sa.Unicode(128)),
	491	sa.Column("section", sa.Unicode(128)),	335	sa.Column('section', sa.Unicode(128)),
	492	sa.Column("config", sa.dialects.postgresql.JSON, ~~server_default~~="{}"),	336	sa.Column('config', sa.dialects.postgresql.JSON,
	493	sa.Column(	337	server_default="{}"),
	494	"resource_id",	338	sa.Column('resource_id', sa.Integer(),
	495	sa.Integer(),	339	sa.ForeignKey('resources.resource_id', onupdate='cascade',
	496	sa.ForeignKey(	340	ondelete='cascade')),
	497	"resources.resource_id", onupdate="cascade", ondelete="cascade"	341	sa.Column('owner_id', sa.Integer(),
	498	),	342	sa.ForeignKey('users.id', onupdate='cascade',
	499	),	343	ondelete='cascade')))
	500	sa.Column(
	501	"owner_id",
	502	sa.Integer(),
	503	sa.ForeignKey("users.id", onupdate="cascade", ondelete="cascade"),
	504	),
	505	)
	506		344
	507	op.create_table(	345	op.create_table(
	508	"rc_versions",	346	'rc_versions',
	509	sa.Column("name", sa.Unicode(40), primary_key=True),	347	sa.Column('name', sa.Unicode(40), primary_key=True),
	510	sa.Column("value", sa.Unicode(40)),	348	sa.Column('value', sa.Unicode(40)),
	511	)
	512	version_table = sa.table(
	513	"rc_versions",
	514	sa.Column("name", sa.Unicode(40)),
	515	sa.Column("value", sa.Unicode(40)),
	516	)	349	)
			350	version_table = sa.table('rc_versions',
			351	sa.Column('name', sa.Unicode(40)),
			352	sa.Column('value', sa.Unicode(40)))
	517		353
	518	insert = version_table.insert().values(name="es_reports")	354	insert = version_table.insert().values(name='es_reports')
	519	op.execute(insert)	355	op.execute(insert)
	520	insert = version_table.insert().values(name="es_reports_groups")	356	insert = version_table.insert().values(name='es_reports_groups')
	521	op.execute(insert)	357	op.execute(insert)
	522	insert = version_table.insert().values(name="es_reports_stats")	358	insert = version_table.insert().values(name='es_reports_stats')
	523	op.execute(insert)	359	op.execute(insert)
	524	insert = version_table.insert().values(name="es_logs")	360	insert = version_table.insert().values(name='es_logs')
	525	op.execute(insert)	361	op.execute(insert)
	526	insert = version_table.insert().values(name="es_metrics")	362	insert = version_table.insert().values(name='es_metrics')
	527	op.execute(insert)	363	op.execute(insert)
	528	insert = version_table.insert().values(name="es_slow_calls")	364	insert = version_table.insert().values(name='es_slow_calls')
	529	op.execute(insert)	365	op.execute(insert)
	530		366
	531	op.execute(	367
	532	"""	368	op.execute('''
	533	CREATE OR REPLACE FUNCTION floor_time_5min(timestamp without time zone)	369	CREATE OR REPLACE FUNCTION floor_time_5min(timestamp without time zone)
	534	RETURNS timestamp without time zone AS	370	RETURNS timestamp without time zone AS
	535	$BODY$SELECT date_trunc('hour', $1) + INTERVAL '5 min' * FLOOR(date_part('minute', $1) / 5.0)$BODY$	371	$BODY$SELECT date_trunc('hour', $1) + INTERVAL '5 min' * FLOOR(date_part('minute', $1) / 5.0)$BODY$
	536	LANGUAGE sql VOLATILE;	372	LANGUAGE sql VOLATILE;
	537	"""	373	''')
	538	)
	539		374
	540	op.execute(	375	op.execute('''
	541	"""
	542	CREATE OR REPLACE FUNCTION partition_logs() RETURNS trigger	376	CREATE OR REPLACE FUNCTION partition_logs() RETURNS trigger
	543	LANGUAGE plpgsql SECURITY DEFINER	377	LANGUAGE plpgsql SECURITY DEFINER
	544	AS $$	378	AS $$
	545	DECLARE	379	DECLARE
	546	main_table varchar := 'logs';	380	main_table varchar := 'logs';
	547	partitioned_table varchar := '';	381	partitioned_table varchar := '';
	548	BEGIN	382	BEGIN
	549		383
	550	IF NEW.permanent THEN	384	IF NEW.permanent THEN
	551	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp);	385	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp);
	552	ELSE	386	ELSE
	553	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp) \|\| '_' \|\| DATE_part('day', NEW.timestamp);	387	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp) \|\| '_' \|\| DATE_part('day', NEW.timestamp);
	554	END IF;	388	END IF;
	555		389
	556	BEGIN	390	BEGIN
	557	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	391	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	558	EXCEPTION	392	EXCEPTION
	559	WHEN undefined_table THEN	393	WHEN undefined_table THEN
	560	RAISE NOTICE 'A partition has been created %', partitioned_table;	394	RAISE NOTICE 'A partition has been created %', partitioned_table;
	561	IF NEW.permanent THEN	395	IF NEW.permanent THEN
	562	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',	396	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',
	563	partitioned_table,	397	partitioned_table,
	564	quote_literal(date_trunc('month', NEW.timestamp)::date) ,	398	quote_literal(date_trunc('month', NEW.timestamp)::date) ,
	565	quote_literal((date_trunc('month', NEW.timestamp)::date + interval '1 month')::text),	399	quote_literal((date_trunc('month', NEW.timestamp)::date + interval '1 month')::text),
	566	main_table);	400	main_table);
	567	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(log_id);', partitioned_table, partitioned_table);	401	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(log_id);', partitioned_table, partitioned_table);
	568	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	402	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	569	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);	403	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);
	570	EXECUTE format('CREATE INDEX ix_%s_namespace_resource_id ON %s (namespace, resource_id);', partitioned_table, partitioned_table);	404	EXECUTE format('CREATE INDEX ix_%s_namespace_resource_id ON %s (namespace, resource_id);', partitioned_table, partitioned_table);
	571	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);	405	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);
	572	EXECUTE format('CREATE INDEX ix_%s_pkey_namespace ON %s (primary_key, namespace);', partitioned_table, partitioned_table);	406	EXECUTE format('CREATE INDEX ix_%s_pkey_namespace ON %s (primary_key, namespace);', partitioned_table, partitioned_table);
	573	ELSE	407	ELSE
	574	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',	408	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',
	575	partitioned_table,	409	partitioned_table,
	576	quote_literal(date_trunc('day', NEW.timestamp)::date) ,	410	quote_literal(date_trunc('day', NEW.timestamp)::date) ,
	577	quote_literal((date_trunc('day', NEW.timestamp)::date + interval '1 day')::text),	411	quote_literal((date_trunc('day', NEW.timestamp)::date + interval '1 day')::text),
	578	main_table);	412	main_table);
	579	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s_ PRIMARY KEY(log_id);', partitioned_table, partitioned_table);	413	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s_ PRIMARY KEY(log_id);', partitioned_table, partitioned_table);
	580	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	414	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	581	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);	415	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);
	582	EXECUTE format('CREATE INDEX ix_%s_namespace_resource_id ON %s (namespace, resource_id);', partitioned_table, partitioned_table);	416	EXECUTE format('CREATE INDEX ix_%s_namespace_resource_id ON %s (namespace, resource_id);', partitioned_table, partitioned_table);
	583	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);	417	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);
	584	EXECUTE format('CREATE INDEX ix_%s_primary_key_namespace ON %s (primary_key,namespace);', partitioned_table, partitioned_table);	418	EXECUTE format('CREATE INDEX ix_%s_primary_key_namespace ON %s (primary_key,namespace);', partitioned_table, partitioned_table);
	585	END IF;	419	END IF;
	586		420
	587		421
	588	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	422	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	589	END;	423	END;
	590		424
	591		425
	592	RETURN NULL;	426	RETURN NULL;
	593	END	427	END
	594	$$;	428	$$;
	595	"""	429	''')
	596	)
	597		430
	598	op.execute(	431	op.execute('''
	599	"""
	600	CREATE TRIGGER partition_logs BEFORE INSERT ON logs FOR EACH ROW EXECUTE PROCEDURE partition_logs();	432	CREATE TRIGGER partition_logs BEFORE INSERT ON logs FOR EACH ROW EXECUTE PROCEDURE partition_logs();
	601	"""	433	''')
	602	)
	603		434
	604	op.execute(	435	op.execute('''
	605	"""
	606	CREATE OR REPLACE FUNCTION partition_metrics() RETURNS trigger	436	CREATE OR REPLACE FUNCTION partition_metrics() RETURNS trigger
	607	LANGUAGE plpgsql SECURITY DEFINER	437	LANGUAGE plpgsql SECURITY DEFINER
	608	AS $$	438	AS $$
	609	DECLARE	439	DECLARE
	610	main_table varchar := 'metrics';	440	main_table varchar := 'metrics';
	611	partitioned_table varchar := '';	441	partitioned_table varchar := '';
	612	BEGIN	442	BEGIN
	613		443
	614	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp) \|\| '_' \|\| DATE_part('day', NEW.timestamp);	444	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.timestamp) \|\| '_' \|\| DATE_part('day', NEW.timestamp);
	615		445
	616	BEGIN	446	BEGIN
	617	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	447	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	618	EXCEPTION	448	EXCEPTION
	619	WHEN undefined_table THEN	449	WHEN undefined_table THEN
	620	RAISE NOTICE 'A partition has been created %', partitioned_table;	450	RAISE NOTICE 'A partition has been created %', partitioned_table;
	621	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',	451	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( timestamp >= DATE %s AND timestamp < DATE %s)) INHERITS (%s)',
	622	partitioned_table,	452	partitioned_table,
	623	quote_literal(date_trunc('day', NEW.timestamp)::date) ,	453	quote_literal(date_trunc('day', NEW.timestamp)::date) ,
	624	quote_literal((date_trunc('day', NEW.timestamp)::date + interval '1 day')::text),	454	quote_literal((date_trunc('day', NEW.timestamp)::date + interval '1 day')::text),
	625	main_table);	455	main_table);
	626	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(pkey);', partitioned_table, partitioned_table);	456	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(pkey);', partitioned_table, partitioned_table);
	627	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	457	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	628	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);	458	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s (timestamp);', partitioned_table, partitioned_table);
	629	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);	459	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s (resource_id);', partitioned_table, partitioned_table);
	630	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	460	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	631	END;	461	END;
	632		462
	633	RETURN NULL;	463	RETURN NULL;
	634	END	464	END
	635	$$;	465	$$;
	636	"""	466	''')
	637	)
	638		467
	639	op.execute(	468	op.execute('''
	640	"""
	641	CREATE TRIGGER partition_metrics BEFORE INSERT ON metrics FOR EACH ROW EXECUTE PROCEDURE partition_metrics();	469	CREATE TRIGGER partition_metrics BEFORE INSERT ON metrics FOR EACH ROW EXECUTE PROCEDURE partition_metrics();
	642	"""	470	''')
	643	)
	644		471
	645	op.execute(	472	op.execute('''
	646	"""
	647	CREATE FUNCTION partition_reports_stats() RETURNS trigger	473	CREATE FUNCTION partition_reports_stats() RETURNS trigger
	648	LANGUAGE plpgsql SECURITY DEFINER	474	LANGUAGE plpgsql SECURITY DEFINER
	649	AS $$	475	AS $$
	650	DECLARE	476	DECLARE
	651	main_table varchar := 'reports_stats';	477	main_table varchar := 'reports_stats';
	652	partitioned_table varchar := '';	478	partitioned_table varchar := '';
	653	BEGIN	479	BEGIN
	654		480
	655	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.start_interval)::TEXT \|\| '_' \|\| DATE_part('month', NEW.start_interval);	481	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.start_interval)::TEXT \|\| '_' \|\| DATE_part('month', NEW.start_interval);
	656		482
	657	BEGIN	483	BEGIN
	658	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	484	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	659	EXCEPTION	485	EXCEPTION
	660	WHEN undefined_table THEN	486	WHEN undefined_table THEN
	661	RAISE NOTICE 'A partition has been created %', partitioned_table;	487	RAISE NOTICE 'A partition has been created %', partitioned_table;
	662	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( start_interval >= DATE %s AND start_interval < DATE %s )) INHERITS (%s)',	488	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( start_interval >= DATE %s AND start_interval < DATE %s )) INHERITS (%s)',
	663	partitioned_table,	489	partitioned_table,
	664	quote_literal(date_trunc('month', NEW.start_interval)::date) ,	490	quote_literal(date_trunc('month', NEW.start_interval)::date) ,
	665	quote_literal((date_trunc('month', NEW.start_interval)::date + interval '1 month')::text),	491	quote_literal((date_trunc('month', NEW.start_interval)::date + interval '1 month')::text),
	666	main_table);	492	main_table);
	667	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);	493	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);
	668	EXECUTE format('CREATE INDEX ix_%s_start_interval ON %s USING btree (start_interval);', partitioned_table, partitioned_table);	494	EXECUTE format('CREATE INDEX ix_%s_start_interval ON %s USING btree (start_interval);', partitioned_table, partitioned_table);
	669	EXECUTE format('CREATE INDEX ix_%s_type ON %s USING btree (type);', partitioned_table, partitioned_table);	495	EXECUTE format('CREATE INDEX ix_%s_type ON %s USING btree (type);', partitioned_table, partitioned_table);
	670	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);	496	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);
	671	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	497	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	672	END;	498	END;
	673	RETURN NULL;	499	RETURN NULL;
	674	END	500	END
	675	$$;	501	$$;
	676	"""	502	''')
	677	)
	678		503
	679	op.execute(	504	op.execute('''
	680	"""
	681	CREATE TRIGGER partition_reports_stats BEFORE INSERT ON reports_stats FOR EACH ROW EXECUTE PROCEDURE partition_reports_stats();	505	CREATE TRIGGER partition_reports_stats BEFORE INSERT ON reports_stats FOR EACH ROW EXECUTE PROCEDURE partition_reports_stats();
	682	"""	506	''')
	683	)
	684		507
	685	op.execute(	508	op.execute('''
	686	"""
	687	CREATE OR REPLACE FUNCTION partition_reports_groups() RETURNS trigger	509	CREATE OR REPLACE FUNCTION partition_reports_groups() RETURNS trigger
	688	LANGUAGE plpgsql SECURITY DEFINER	510	LANGUAGE plpgsql SECURITY DEFINER
	689	AS $$	511	AS $$
	690	DECLARE	512	DECLARE
	691	main_table varchar := 'reports_groups';	513	main_table varchar := 'reports_groups';
	692	partitioned_table varchar := '';	514	partitioned_table varchar := '';
	693	BEGIN	515	BEGIN
	694		516
	695	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.first_timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.first_timestamp);	517	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.first_timestamp)::TEXT \|\| '_' \|\| DATE_part('month', NEW.first_timestamp);
	696		518
	697	BEGIN	519	BEGIN
	698	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	520	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	699	EXCEPTION	521	EXCEPTION
	700	WHEN undefined_table THEN	522	WHEN undefined_table THEN
	701	RAISE NOTICE 'A partition has been created %', partitioned_table;	523	RAISE NOTICE 'A partition has been created %', partitioned_table;
	702	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( first_timestamp >= DATE %s AND first_timestamp < DATE %s )) INHERITS (%s)',	524	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( first_timestamp >= DATE %s AND first_timestamp < DATE %s )) INHERITS (%s)',
	703	partitioned_table,	525	partitioned_table,
	704	quote_literal(date_trunc('month', NEW.first_timestamp)::date) ,	526	quote_literal(date_trunc('month', NEW.first_timestamp)::date) ,
	705	quote_literal((date_trunc('month', NEW.first_timestamp)::date + interval '1 month')::text),	527	quote_literal((date_trunc('month', NEW.first_timestamp)::date + interval '1 month')::text),
	706	main_table);	528	main_table);
	707	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);	529	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);
	708	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	530	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	709	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	531	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	710	END;	532	END;
	711	RETURN NULL;	533	RETURN NULL;
	712	END	534	END
	713	$$;	535	$$;
	714	"""	536	''')
	715	)
	716		537
	717	op.execute(	538	op.execute('''
	718	"""
	719	CREATE TRIGGER partition_reports_groups BEFORE INSERT ON reports_groups FOR EACH ROW EXECUTE PROCEDURE partition_reports_groups();	539	CREATE TRIGGER partition_reports_groups BEFORE INSERT ON reports_groups FOR EACH ROW EXECUTE PROCEDURE partition_reports_groups();
	720	"""	540	''')
	721	)
	722		541
	723	op.execute(	542	op.execute('''
	724	"""
	725	CREATE OR REPLACE FUNCTION partition_reports() RETURNS trigger	543	CREATE OR REPLACE FUNCTION partition_reports() RETURNS trigger
	726	LANGUAGE plpgsql SECURITY DEFINER	544	LANGUAGE plpgsql SECURITY DEFINER
	727	AS $$	545	AS $$
	728	DECLARE	546	DECLARE
	729	main_table varchar := 'reports';	547	main_table varchar := 'reports';
	730	partitioned_table varchar := '';	548	partitioned_table varchar := '';
	731	partitioned_parent_table varchar := '';	549	partitioned_parent_table varchar := '';
	732	BEGIN	550	BEGIN
	733		551
	734	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);	552	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);
	735	partitioned_parent_table := 'reports_groups_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);	553	partitioned_parent_table := 'reports_groups_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);
	736		554
	737	BEGIN	555	BEGIN
	738	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	556	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	739	EXCEPTION	557	EXCEPTION
	740	WHEN undefined_table THEN	558	WHEN undefined_table THEN
	741	RAISE NOTICE 'A partition has been created %', partitioned_table;	559	RAISE NOTICE 'A partition has been created %', partitioned_table;
	742	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( report_group_time >= DATE %s AND report_group_time < DATE %s )) INHERITS (%s)',	560	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( report_group_time >= DATE %s AND report_group_time < DATE %s )) INHERITS (%s)',
	743	partitioned_table,	561	partitioned_table,
	744	quote_literal(date_trunc('month', NEW.report_group_time)::date) ,	562	quote_literal(date_trunc('month', NEW.report_group_time)::date) ,
	745	quote_literal((date_trunc('month', NEW.report_group_time)::date + interval '1 month')::text),	563	quote_literal((date_trunc('month', NEW.report_group_time)::date + interval '1 month')::text),
	746	main_table);	564	main_table);
	747	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);	565	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);
	748	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	566	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	749	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_group_id FOREIGN KEY (group_id) REFERENCES %s (id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table, partitioned_parent_table);	567	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_group_id FOREIGN KEY (group_id) REFERENCES %s (id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table, partitioned_parent_table);
	750	EXECUTE format('CREATE INDEX ix_%s_report_group_time ON %s USING btree (report_group_time);', partitioned_table, partitioned_table);	568	EXECUTE format('CREATE INDEX ix_%s_report_group_time ON %s USING btree (report_group_time);', partitioned_table, partitioned_table);
	751	EXECUTE format('CREATE INDEX ix_%s_group_id ON %s USING btree (group_id);', partitioned_table, partitioned_table);	569	EXECUTE format('CREATE INDEX ix_%s_group_id ON %s USING btree (group_id);', partitioned_table, partitioned_table);
	752	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);	570	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);
	753	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	571	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	754	END;	572	END;
	755	RETURN NULL;	573	RETURN NULL;
	756	END	574	END
	757	$$;	575	$$;
	758	"""	576	''')
	759	)
	760		577
	761	op.execute(	578	op.execute('''
	762	"""
	763	CREATE TRIGGER partition_reports BEFORE INSERT ON reports FOR EACH ROW EXECUTE PROCEDURE partition_reports();	579	CREATE TRIGGER partition_reports BEFORE INSERT ON reports FOR EACH ROW EXECUTE PROCEDURE partition_reports();
	764	"""	580	''')
	765	)
	766		581
	767	op.execute(	582
	768	"""	583	op.execute('''
	769	CREATE OR REPLACE FUNCTION partition_slow_calls() RETURNS trigger	584	CREATE OR REPLACE FUNCTION partition_slow_calls() RETURNS trigger
	770	LANGUAGE plpgsql SECURITY DEFINER	585	LANGUAGE plpgsql SECURITY DEFINER
	771	AS $$	586	AS $$
	772	DECLARE	587	DECLARE
	773	main_table varchar := 'slow_calls';	588	main_table varchar := 'slow_calls';
	774	partitioned_table varchar := '';	589	partitioned_table varchar := '';
	775	partitioned_parent_table varchar := '';	590	partitioned_parent_table varchar := '';
	776	BEGIN	591	BEGIN
	777		592
	778	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);	593	partitioned_table := main_table \|\| '_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);
	779	partitioned_parent_table := 'reports_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);	594	partitioned_parent_table := 'reports_p_' \|\| date_part('year', NEW.report_group_time)::TEXT \|\| '_' \|\| DATE_part('month', NEW.report_group_time);
	780		595
	781	BEGIN	596	BEGIN
	782	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	597	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	783	EXCEPTION	598	EXCEPTION
	784	WHEN undefined_table THEN	599	WHEN undefined_table THEN
	785	RAISE NOTICE 'A partition has been created %', partitioned_table;	600	RAISE NOTICE 'A partition has been created %', partitioned_table;
	786	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( report_group_time >= DATE %s AND report_group_time < DATE %s )) INHERITS (%s)',	601	EXECUTE format('CREATE TABLE IF NOT EXISTS %s ( CHECK( report_group_time >= DATE %s AND report_group_time < DATE %s )) INHERITS (%s)',
	787	partitioned_table,	602	partitioned_table,
	788	quote_literal(date_trunc('month', NEW.report_group_time)::date) ,	603	quote_literal(date_trunc('month', NEW.report_group_time)::date) ,
	789	quote_literal((date_trunc('month', NEW.report_group_time)::date + interval '1 month')::text),	604	quote_literal((date_trunc('month', NEW.report_group_time)::date + interval '1 month')::text),
	790	main_table);	605	main_table);
	791	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);	606	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT pk_%s PRIMARY KEY(id);', partitioned_table, partitioned_table);
	792	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);	607	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_resource_id FOREIGN KEY (resource_id) REFERENCES resources (resource_id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table);
	793	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_report_id FOREIGN KEY (report_id) REFERENCES %s (id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table, partitioned_parent_table);	608	EXECUTE format('ALTER TABLE %s ADD CONSTRAINT fk_%s_report_id FOREIGN KEY (report_id) REFERENCES %s (id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE;', partitioned_table, partitioned_table, partitioned_parent_table);
	794	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);	609	EXECUTE format('CREATE INDEX ix_%s_resource_id ON %s USING btree (resource_id);', partitioned_table, partitioned_table);
	795	EXECUTE format('CREATE INDEX ix_%s_report_id ON %s USING btree (report_id);', partitioned_table, partitioned_table);	610	EXECUTE format('CREATE INDEX ix_%s_report_id ON %s USING btree (report_id);', partitioned_table, partitioned_table);
	796	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s USING btree (timestamp);', partitioned_table, partitioned_table);	611	EXECUTE format('CREATE INDEX ix_%s_timestamp ON %s USING btree (timestamp);', partitioned_table, partitioned_table);
	797	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';	612	EXECUTE 'INSERT INTO ' \|\| partitioned_table \|\| ' SELECT(' \|\| TG_TABLE_NAME \|\| ' ' \|\| quote_literal(NEW) \|\| ').*;';
	798	END;	613	END;
	799	RETURN NULL;	614	RETURN NULL;
	800	END	615	END
	801	$$;	616	$$;
	802	"""	617	''')
	803	)
	804		618
	805	op.execute(	619	op.execute('''
	806	"""
	807	CREATE TRIGGER partition_slow_calls BEFORE INSERT ON slow_calls FOR EACH ROW EXECUTE PROCEDURE partition_slow_calls();	620	CREATE TRIGGER partition_slow_calls BEFORE INSERT ON slow_calls FOR EACH ROW EXECUTE PROCEDURE partition_slow_calls();
	808	"""	621	''')
	809	)
	810
	811		622
	812	def downgrade():	623	def downgrade():
	813	pass	624	pass

backend/src/appenlight/migrations/versions/e9fcfbdd9498_connect_resources_to_alert_channels.py

0 +12 -20

             """connect resources to alert_channels
             Revision ID: e9fcfbdd9498
             Revises: 55b6e612672f
             Create Date: 2018-02-28 13:52:50.717217
             """
             # revision identifiers, used by Alembic.
-            revision = "e9fcfbdd9498"
+            revision = 'e9fcfbdd9498'
-            down_revision = "55b6e612672f"
+            down_revision = '55b6e612672f'
             from alembic import op
             import sqlalchemy as sa
             def upgrade():
                 op.create_table(
-                    "channels_resources",
+                    'channels_resources',
-                    sa.Column(
+                    sa.Column('channel_pkey', sa.Integer,
-                        "channel_pkey",
+                              sa.ForeignKey('alert_channels.pkey',
-                        sa.Integer,
+                                            ondelete='CASCADE', onupdate='CASCADE'),
-                        sa.ForeignKey(
+                              primary_key=True),
-                            "alert_channels.pkey", ondelete="CASCADE", onupdate="CASCADE"
+                    sa.Column('resource_id', sa.Integer,
-                        ),
+                              sa.ForeignKey('resources.resource_id',
-                        primary_key=True,
+                                            ondelete='CASCADE', onupdate='CASCADE'),
-                    ),
+                              primary_key=True)
-                    sa.Column(
-                        "resource_id",
-                        sa.Integer,
-                        sa.ForeignKey(
-                            "resources.resource_id", ondelete="CASCADE", onupdate="CASCADE"
-                        ),
-                        primary_key=True,
-                    ),
                 )
             def downgrade():
-                op.drop_table("channels_resources")
+                op.drop_table('channels_resources')

backend/src/appenlight/models/__init__.py

0 +19 -28

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from sqlalchemy.ext.declarative import declarative_base
             from sqlalchemy import MetaData
             from sqlalchemy.orm import scoped_session
             from sqlalchemy.orm import sessionmaker
             from zope.sqlalchemy import ZopeTransactionExtension
             import ziggurat_foundations
             from ziggurat_foundations.models.base import get_db_session
             log = logging.getLogger(__name__)
             DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
             NAMING_CONVENTION = {
-                "ix": "ix_%(column_0_label)s",
+                "ix": 'ix_%(column_0_label)s',
                 "uq": "uq_%(table_name)s_%(column_0_name)s",
                 "ck": "ck_%(table_name)s_%(constraint_name)s",
                 "fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
-                "pk": "pk_%(table_name)s",
+                "pk": "pk_%(table_name)s"
             }
             metadata = MetaData(naming_convention=NAMING_CONVENTION)
             Base = declarative_base(metadata=metadata)
             # optional for request.db approach
             ziggurat_foundations.models.DBSession = DBSession
             class Datastores(object):
                 redis = None
                 es = None
             def register_datastores(es_conn, redis_conn, redis_lockmgr):
                 Datastores.es = es_conn
                 Datastores.redis = redis_conn
                 Datastores.lockmgr = redis_lockmgr
             class SliceableESQuery(object):
                 def __init__(self, query, sort_query=None, aggregations=False, **kwconfig):
                     self.query = query
                     self.sort_query = sort_query
                     self.aggregations = aggregations
-                    self.items_per_page = kwconfig.pop("items_per_page", 10)
+                    self.items_per_page = kwconfig.pop('items_per_page', 10)
-                    self.page = kwconfig.pop("page", 1)
+                    self.page = kwconfig.pop('page', 1)
                     self.kwconfig = kwconfig
                     self.result = None
                 def __getitem__(self, index):
                     config = self.kwconfig.copy()
-                    config["from_"] = index.start
+                    config['es_from'] = index.start
                     query = self.query.copy()
                     if self.sort_query:
                         query.update(self.sort_query)
-                    self.result = Datastores.es.search(
+                    self.result = Datastores.es.search(query, size=self.items_per_page,
-                        body=query, size=self.items_per_page, **config
+                                                       **config)
                     if self.aggregations:
-                        self.items = self.result.get("aggregations")
+                        self.items = self.result.get('aggregations')
                     else:
-                        self.items = self.result["hits"]["hits"]
+                        self.items = self.result['hits']['hits']
                     return self.items
                 def __iter__(self):
                     return self.result
                 def __len__(self):
                     config = self.kwconfig.copy()
                     query = self.query.copy()
-                    self.result = Datastores.es.search(
+                    self.result = Datastores.es.search(query, size=self.items_per_page,
-                        body=query, size=self.items_per_page, **config
+                                                       **config)
                     if self.aggregations:
-                        self.items = self.result.get("aggregations")
+                        self.items = self.result.get('aggregations')
                     else:
-                        self.items = self.result["hits"]["hits"]
+                        self.items = self.result['hits']['hits']
-                    count = int(self.result["hits"]["total"])
+                    count = int(self.result['hits']['total'])
                     return count if count < 5000 else 5000
             from appenlight.models.resource import Resource
             from appenlight.models.application import Application
             from appenlight.models.user import User
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.alert_channel_action import AlertChannelAction
             from appenlight.models.metric import Metric
-            from appenlight.models.application_postprocess_conf import ApplicationPostprocessConf
+            from appenlight.models.application_postprocess_conf import \
+                ApplicationPostprocessConf
             from appenlight.models.auth_token import AuthToken
             from appenlight.models.event import Event
             from appenlight.models.external_identity import ExternalIdentity
             from appenlight.models.group import Group
             from appenlight.models.group_permission import GroupPermission
             from appenlight.models.group_resource_permission import GroupResourcePermission
             from appenlight.models.log import Log
             from appenlight.models.plugin_config import PluginConfig
             from appenlight.models.report import Report
             from appenlight.models.report_group import ReportGroup
             from appenlight.models.report_comment import ReportComment
             from appenlight.models.report_assignment import ReportAssignment
             from appenlight.models.report_stat import ReportStat
             from appenlight.models.slow_call import SlowCall
             from appenlight.models.tag import Tag
             from appenlight.models.user_group import UserGroup
             from appenlight.models.user_permission import UserPermission
             from appenlight.models.user_resource_permission import UserResourcePermission
             from ziggurat_foundations import ziggurat_model_init
-            ziggurat_model_init(
+            ziggurat_model_init(User, Group, UserGroup, GroupPermission, UserPermission,
-                User,
+                                UserResourcePermission, GroupResourcePermission,
-                Group,
-                UserGroup,
-                GroupPermission,
-                UserPermission,
-                UserResourcePermission,
-                GroupResourcePermission,
                                 Resource,
-                ExternalIdentity,
+                                ExternalIdentity, passwordmanager=None)
-                passwordmanager=None,

backend/src/appenlight/models/alert_channel.py

0 +134 -127

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             import sqlalchemy as sa
             import urllib.request, urllib.parse, urllib.error
             from datetime import timedelta
             from appenlight.models import Base
             from appenlight.lib.utils.date_utils import convert_date
             from sqlalchemy.dialects.postgresql import JSON
             from ziggurat_foundations.models.base import BaseModel
             log = logging.getLogger(__name__)
             #
             channel_rules_m2m_table = sa.Table(
-                "channels_actions",
+                'channels_actions', Base.metadata,
-                Base.metadata,
+                sa.Column('channel_pkey', sa.Integer,
-                sa.Column("channel_pkey", sa.Integer, sa.ForeignKey("alert_channels.pkey")),
+                          sa.ForeignKey('alert_channels.pkey')),
-                sa.Column("action_pkey", sa.Integer, sa.ForeignKey("alert_channels_actions.pkey")),
+                sa.Column('action_pkey', sa.Integer,
+                          sa.ForeignKey('alert_channels_actions.pkey'))
             )
             channel_resources_m2m_table = sa.Table(
-                "channels_resources",
+                'channels_resources', Base.metadata,
-                Base.metadata,
+                sa.Column('channel_pkey', sa.Integer,
-                sa.Column("channel_pkey", sa.Integer, sa.ForeignKey("alert_channels.pkey")),
+                          sa.ForeignKey('alert_channels.pkey')),
-                sa.Column("resource_id", sa.Integer, sa.ForeignKey("resources.resource_id")),
+                sa.Column('resource_id', sa.Integer,
+                          sa.ForeignKey('resources.resource_id'))
             )
-            DATE_FRMT = "%Y-%m-%dT%H:%M"
+            DATE_FRMT = '%Y-%m-%dT%H:%M'
             class AlertChannel(Base, BaseModel):
                 """
                 Stores information about possible alerting options
                 """
+                __tablename__ = 'alert_channels'
-                __tablename__ = "alert_channels"
+                __possible_channel_names__ = ['email']
-                __possible_channel_names__ = ["email"]
                 __mapper_args__ = {
-                    "polymorphic_on": "channel_name",
+                    'polymorphic_on': 'channel_name',
-                    "polymorphic_identity": "integration",
+                    'polymorphic_identity': 'integration'
                 }
-                owner_id = sa.Column(
+                owner_id = sa.Column(sa.Unicode(30),
-                    sa.Unicode(30),
+                                     sa.ForeignKey('users.id', onupdate='CASCADE',
-                    sa.ForeignKey("users.id", onupdate="CASCADE", ondelete="CASCADE"),
+                                                   ondelete='CASCADE'))
                 channel_name = sa.Column(sa.Unicode(25), nullable=False)
-                channel_value = sa.Column(sa.Unicode(80), nullable=False, default="")
+                channel_value = sa.Column(sa.Unicode(80), nullable=False, default='')
-                channel_json_conf = sa.Column(JSON(), nullable=False, default="")
+                channel_json_conf = sa.Column(JSON(), nullable=False, default='')
-                channel_validated = sa.Column(sa.Boolean, nullable=False, default=False)
+                channel_validated = sa.Column(sa.Boolean, nullable=False,
-                send_alerts = sa.Column(sa.Boolean, nullable=False, default=True)
+                                              default=False)
-                daily_digest = sa.Column(sa.Boolean, nullable=False, default=True)
+                send_alerts = sa.Column(sa.Boolean, nullable=False,
-                integration_id = sa.Column(
+                                        default=True)
-                    sa.Integer, sa.ForeignKey("integrations.id"), nullable=True
+                daily_digest = sa.Column(sa.Boolean, nullable=False,
+                                         default=True)
+                integration_id = sa.Column(sa.Integer, sa.ForeignKey('integrations.id'),
+                                           nullable=True)
                 pkey = sa.Column(sa.Integer(), nullable=False, primary_key=True)
-                channel_actions = sa.orm.relationship(
+                channel_actions = sa.orm.relationship('AlertChannelAction',
-                    "AlertChannelAction",
                                                       cascade="all",
                                                       passive_deletes=True,
                                                       passive_updates=True,
                                                       secondary=channel_rules_m2m_table,
-                    backref="channels",
+                                                      backref='channels')
+                resources = sa.orm.relationship('Resource',
-                resources = sa.orm.relationship(
-                    "Resource",
                                                 cascade="all",
                                                 passive_deletes=True,
                                                 passive_updates=True,
                                                 secondary=channel_resources_m2m_table,
-                    backref="resources",
+                                                backref='resources')
                 @property
                 def channel_visible_value(self):
                     if self.integration:
-                        return "{}: {}".format(
+                        return '{}: {}'.format(
-                            self.channel_name, self.integration.resource.resource_name
+                            self.channel_name,
+                            self.integration.resource.resource_name
                         )
-                    return "{}: {}".format(self.channel_name, self.channel_value)
+                    return '{}: {}'.format(
+                        self.channel_name,
+                        self.channel_value
+                    )
-                def get_dict(self, exclude_keys=None, include_keys=None, extended_info=True):
+                def get_dict(self, exclude_keys=None, include_keys=None,
+                             extended_info=True):
                     """
                     Returns dictionary with required information that will be consumed by
                     angular
                     """
-                    instance_dict = super(AlertChannel, self).get_dict(exclude_keys, include_keys)
+                    instance_dict = super(AlertChannel, self).get_dict(exclude_keys,
+                                                                       include_keys)
                     exclude_keys_list = exclude_keys or []
                     include_keys_list = include_keys or []
-                    instance_dict["supports_report_alerting"] = True
+                    instance_dict['supports_report_alerting'] = True
-                    instance_dict["channel_visible_value"] = self.channel_visible_value
+                    instance_dict['channel_visible_value'] = self.channel_visible_value
                     if extended_info:
-                        instance_dict["actions"] = [
+                        instance_dict['actions'] = [
-                            rule.get_dict(extended_info=True) for rule in self.channel_actions
+                            rule.get_dict(extended_info=True) for
+                            rule in self.channel_actions]
-                    del instance_dict["channel_json_conf"]
+                    del instance_dict['channel_json_conf']
                     if self.integration:
                         instance_dict[
-                            "supports_report_alerting"
+                            'supports_report_alerting'] = \
-                        ] = self.integration.supports_report_alerting
+                            self.integration.supports_report_alerting
                     d = {}
                     for k in instance_dict.keys():
-                        if k not in exclude_keys_list and (
+                        if (k not in exclude_keys_list and
-                            k in include_keys_list or not include_keys
+                                (k in include_keys_list or not include_keys)):
-                        ):
                             d[k] = instance_dict[k]
                     return d
                 def __repr__(self):
-                    return "<AlertChannel: (%s,%s), user:%s>" % (
+                    return '<AlertChannel: (%s,%s), user:%s>' % (self.channel_name,
-                        self.channel_name,
                                                                  self.channel_value,
-                        self.user_name,
+                                                                 self.user_name,)
                 def send_digest(self, **kwargs):
                     """
                     This should implement daily top error report notifications
                     """
-                    log.warning("send_digest NOT IMPLEMENTED")
+                    log.warning('send_digest NOT IMPLEMENTED')
                 def notify_reports(self, **kwargs):
                     """
                     This should implement notification of reports that occured in 1 min
                     interval
                     """
-                    log.warning("notify_reports NOT IMPLEMENTED")
+                    log.warning('notify_reports NOT IMPLEMENTED')
                 def notify_alert(self, **kwargs):
                     """
                     Notify user of report/uptime/chart threshold events based on events alert
                     type
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
-                    alert_name = kwargs["event"].unified_alert_name()
+                    alert_name = kwargs['event'].unified_alert_name()
-                    if alert_name in ["slow_report_alert", "error_report_alert"]:
+                    if alert_name in ['slow_report_alert', 'error_report_alert']:
                         self.notify_report_alert(**kwargs)
-                    elif alert_name == "uptime_alert":
+                    elif alert_name == 'uptime_alert':
                         self.notify_uptime_alert(**kwargs)
-                    elif alert_name == "chart_alert":
+                    elif alert_name == 'chart_alert':
                         self.notify_chart_alert(**kwargs)
                 def notify_chart_alert(self, **kwargs):
                     """
                     This should implement report open/close alerts notifications
                     """
-                    log.warning("notify_chart_alert NOT IMPLEMENTED")
+                    log.warning('notify_chart_alert NOT IMPLEMENTED')
                 def notify_report_alert(self, **kwargs):
                     """
                     This should implement report open/close alerts notifications
                     """
-                    log.warning("notify_report_alert NOT IMPLEMENTED")
+                    log.warning('notify_report_alert NOT IMPLEMENTED')
                 def notify_uptime_alert(self, **kwargs):
                     """
                     This should implement uptime open/close alerts notifications
                     """
-                    log.warning("notify_uptime_alert NOT IMPLEMENTED")
+                    log.warning('notify_uptime_alert NOT IMPLEMENTED')
                 def get_notification_basic_vars(self, kwargs):
                     """
                     Sets most common variables used later for rendering notifications for
                     channel
                     """
-                    if "event" in kwargs:
+                    if 'event' in kwargs:
-                        kwargs["since_when"] = kwargs["event"].start_date
+                        kwargs['since_when'] = kwargs['event'].start_date
-                    url_start_date = kwargs.get("since_when") - timedelta(minutes=1)
+                    url_start_date = kwargs.get('since_when') - timedelta(minutes=1)
-                    url_end_date = kwargs.get("since_when") + timedelta(minutes=4)
+                    url_end_date = kwargs.get('since_when') + timedelta(minutes=4)
                     tmpl_vars = {
-                        "timestamp": kwargs["since_when"],
+                        "timestamp": kwargs['since_when'],
-                        "user": kwargs["user"],
+                        "user": kwargs['user'],
-                        "since_when": kwargs.get("since_when"),
+                        "since_when": kwargs.get('since_when'),
                         "url_start_date": url_start_date,
-                        "url_end_date": url_end_date,
+                        "url_end_date": url_end_date
                     }
-                    tmpl_vars["resource_name"] = kwargs["resource"].resource_name
+                    tmpl_vars["resource_name"] = kwargs['resource'].resource_name
-                    tmpl_vars["resource"] = kwargs["resource"]
+                    tmpl_vars["resource"] = kwargs['resource']
-                    if "event" in kwargs:
+                    if 'event' in kwargs:
-                        tmpl_vars["event_values"] = kwargs["event"].values
+                        tmpl_vars['event_values'] = kwargs['event'].values
-                        tmpl_vars["alert_type"] = kwargs["event"].unified_alert_name()
+                        tmpl_vars['alert_type'] = kwargs['event'].unified_alert_name()
-                        tmpl_vars["alert_action"] = kwargs["event"].unified_alert_action()
+                        tmpl_vars['alert_action'] = kwargs['event'].unified_alert_action()
                     return tmpl_vars
                 def report_alert_notification_vars(self, kwargs):
                     tmpl_vars = self.get_notification_basic_vars(kwargs)
-                    reports = kwargs.get("reports", [])
+                    reports = kwargs.get('reports', [])
                     tmpl_vars["reports"] = reports
                     tmpl_vars["confirmed_total"] = len(reports)
                     tmpl_vars["report_type"] = "error reports"
-                    tmpl_vars["url_report_type"] = "report/list"
+                    tmpl_vars["url_report_type"] = 'report/list'
-                    alert_type = tmpl_vars.get("alert_type", "")
+                    alert_type = tmpl_vars.get('alert_type', '')
-                    if "slow_report" in alert_type:
+                    if 'slow_report' in alert_type:
                         tmpl_vars["report_type"] = "slow reports"
-                        tmpl_vars["url_report_type"] = "report/list_slow"
+                        tmpl_vars["url_report_type"] = 'report/list_slow'
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", _app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/',
+                                                                  _app_url=app_url)
                     if alert_type:
-                        destination_url += "ui/{}?resource={}&start_date={}&end_date={}".format(
+                        destination_url += 'ui/{}?resource={}&start_date={}&end_date={}'.format(
                             tmpl_vars["url_report_type"],
-                            tmpl_vars["resource"].resource_id,
+                            tmpl_vars['resource'].resource_id,
-                            tmpl_vars["url_start_date"].strftime(DATE_FRMT),
+                            tmpl_vars['url_start_date'].strftime(DATE_FRMT),
-                            tmpl_vars["url_end_date"].strftime(DATE_FRMT),
+                            tmpl_vars['url_end_date'].strftime(DATE_FRMT)
                         )
                     else:
-                        destination_url += "ui/{}?resource={}".format(
+                        destination_url += 'ui/{}?resource={}'.format(
-                            tmpl_vars["url_report_type"], tmpl_vars["resource"].resource_id
+                            tmpl_vars["url_report_type"],
+                            tmpl_vars['resource'].resource_id
                         )
                     tmpl_vars["destination_url"] = destination_url
                     return tmpl_vars
                 def uptime_alert_notification_vars(self, kwargs):
                     tmpl_vars = self.get_notification_basic_vars(kwargs)
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", _app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/', _app_url=app_url)
-                    destination_url += "ui/{}?resource={}".format(
+                    destination_url += 'ui/{}?resource={}'.format(
-                        "uptime", tmpl_vars["resource"].resource_id
+                        'uptime',
+                        tmpl_vars['resource'].resource_id)
-                    tmpl_vars["destination_url"] = destination_url
+                    tmpl_vars['destination_url'] = destination_url
-                    reason = ""
+                    reason = ''
-                    e_values = tmpl_vars.get("event_values")
+                    e_values = tmpl_vars.get('event_values')
-                    if e_values and e_values.get("response_time") == 0:
+                    if e_values and e_values.get('response_time') == 0:
-                        reason += " Response time was slower than 20 seconds."
+                        reason += ' Response time was slower than 20 seconds.'
                     elif e_values:
-                        code = e_values.get("status_code")
+                        code = e_values.get('status_code')
-                        reason += " Response status code: %s." % code
+                        reason += ' Response status code: %s.' % code
-                    tmpl_vars["reason"] = reason
+                    tmpl_vars['reason'] = reason
                     return tmpl_vars
                 def chart_alert_notification_vars(self, kwargs):
                     tmpl_vars = self.get_notification_basic_vars(kwargs)
-                    tmpl_vars["chart_name"] = tmpl_vars["event_values"]["chart_name"]
+                    tmpl_vars['chart_name'] = tmpl_vars['event_values']['chart_name']
-                    tmpl_vars["action_name"] = tmpl_vars["event_values"].get("action_name") or ""
+                    tmpl_vars['action_name'] = tmpl_vars['event_values'].get(
-                    matched_values = tmpl_vars["event_values"]["matched_step_values"]
+                        'action_name') or ''
-                    tmpl_vars["readable_values"] = []
+                    matched_values = tmpl_vars['event_values']['matched_step_values']
-                    for key, value in list(matched_values["values"].items()):
+                    tmpl_vars['readable_values'] = []
-                        matched_label = matched_values["labels"].get(key)
+                    for key, value in list(matched_values['values'].items()):
+                        matched_label = matched_values['labels'].get(key)
                         if matched_label:
-                            tmpl_vars["readable_values"].append(
+                            tmpl_vars['readable_values'].append({
-                                {"label": matched_label["human_label"], "value": value}
+                                'label': matched_label['human_label'],
+                                'value': value
-                    tmpl_vars["readable_values"] = sorted(
+                            })
-                        tmpl_vars["readable_values"], key=lambda x: x["label"]
+                    tmpl_vars['readable_values'] = sorted(tmpl_vars['readable_values'],
+                                                          key=lambda x: x['label'])
-                    start_date = convert_date(tmpl_vars["event_values"]["start_interval"])
+                    start_date = convert_date(tmpl_vars['event_values']['start_interval'])
                     end_date = None
-                    if tmpl_vars["event_values"].get("end_interval"):
+                    if tmpl_vars['event_values'].get('end_interval'):
-                        end_date = convert_date(tmpl_vars["event_values"]["end_interval"])
+                        end_date = convert_date(tmpl_vars['event_values']['end_interval'])
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", _app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/', _app_url=app_url)
                     to_encode = {
-                        "resource": tmpl_vars["event_values"]["resource"],
+                        'resource': tmpl_vars['event_values']['resource'],
-                        "start_date": start_date.strftime(DATE_FRMT),
+                        'start_date': start_date.strftime(DATE_FRMT),
                     }
                     if end_date:
-                        to_encode["end_date"] = end_date.strftime(DATE_FRMT)
+                        to_encode['end_date'] = end_date.strftime(DATE_FRMT)
-                    destination_url += "ui/{}?{}".format("logs", urllib.parse.urlencode(to_encode))
+                    destination_url += 'ui/{}?{}'.format(
-                    tmpl_vars["destination_url"] = destination_url
+                        'logs',
+                        urllib.parse.urlencode(to_encode)
+                    )
+                    tmpl_vars['destination_url'] = destination_url
                     return tmpl_vars

backend/src/appenlight/models/alert_channel_action.py

0 +21 -26

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import sqlalchemy as sa
-            from ziggurat_foundations.models.services.resource import ResourceService
+            from appenlight.models.resource import Resource
             from appenlight.models import Base, get_db_session
             from sqlalchemy.orm import validates
             from ziggurat_foundations.models.base import BaseModel
             class AlertChannelAction(Base, BaseModel):
                 """
                 Stores notifications conditions for user's alert channels
                 This is later used for rule parsing like "alert if http_status == 500"
                 """
+                __tablename__ = 'alert_channels_actions'
-                __tablename__ = "alert_channels_actions"
+                types = ['report', 'chart']
-                types = ["report", "chart"]
+                owner_id = sa.Column(sa.Integer,
+                                     sa.ForeignKey('users.id', onupdate='CASCADE',
-                owner_id = sa.Column(
+                                                   ondelete='CASCADE'))
-                    sa.Integer, sa.ForeignKey("users.id", onupdate="CASCADE", ondelete="CASCADE")
                 resource_id = sa.Column(sa.Integer())
-                action = sa.Column(sa.Unicode(10), nullable=False, default="always")
+                action = sa.Column(sa.Unicode(10), nullable=False, default='always')
                 type = sa.Column(sa.Unicode(10), nullable=False)
                 other_id = sa.Column(sa.Unicode(40))
                 pkey = sa.Column(sa.Integer(), nullable=False, primary_key=True)
-                rule = sa.Column(
+                rule = sa.Column(sa.dialects.postgresql.JSON,
-                    sa.dialects.postgresql.JSON,
+                                 nullable=False, default={'field': 'http_status',
-                    nullable=False,
+                                                          "op": "ge", "value": "500"})
-                    default={"field": "http_status", "op": "ge", "value": "500"},
                 config = sa.Column(sa.dialects.postgresql.JSON)
                 name = sa.Column(sa.Unicode(255))
-                @validates("notify_type")
+                @validates('notify_type')
                 def validate_email(self, key, notify_type):
-                    assert notify_type in ["always", "only_first"]
+                    assert notify_type in ['always', 'only_first']
                     return notify_type
                 def resource_name(self, db_session=None):
                     db_session = get_db_session(db_session)
                     if self.resource_id:
-                        return ResourceService.by_resource_id(
+                        return Resource.by_resource_id(self.resource_id,
-                            self.resource_id, db_session=db_session
+                                                       db_session=db_session).resource_name
-                        ).resource_name
                     else:
-                        return "any resource"
+                        return 'any resource'
-                def get_dict(self, exclude_keys=None, include_keys=None, extended_info=False):
+                def get_dict(self, exclude_keys=None, include_keys=None,
+                             extended_info=False):
                     """
                     Returns dictionary with required information that will be consumed by
                     angular
                     """
                     instance_dict = super(AlertChannelAction, self).get_dict()
                     exclude_keys_list = exclude_keys or []
                     include_keys_list = include_keys or []
                     if extended_info:
-                        instance_dict["channels"] = [
+                        instance_dict['channels'] = [
-                            c.get_dict(extended_info=False) for c in self.channels
+                            c.get_dict(extended_info=False) for c in self.channels]
                     d = {}
                     for k in instance_dict.keys():
-                        if k not in exclude_keys_list and (
+                        if (k not in exclude_keys_list and
-                            k in include_keys_list or not include_keys
+                                (k in include_keys_list or not include_keys)):
-                        ):
                             d[k] = instance_dict[k]
                     return d

backend/src/appenlight/models/alert_channels/__init__.py

0 +1 0

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.

backend/src/appenlight/models/alert_channels/campfire.py

0 +63 -66

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.integrations.campfire import CampfireIntegration
             from webhelpers2.text import truncate
             log = logging.getLogger(__name__)
             class CampfireAlertChannel(AlertChannel):
-                __mapper_args__ = {"polymorphic_identity": "campfire"}
+                __mapper_args__ = {
+                    'polymorphic_identity': 'campfire'
+                }
                 @property
                 def client(self):
                     client = CampfireIntegration.create_client(
-                        self.integration.config["api_token"], self.integration.config["account"]
+                        self.integration.config['api_token'],
+                        self.integration.config['account'])
                     return client
                 def notify_reports(self, **kwargs):
                     """
                     Notify user of individual reports
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/',
-                    f_args = (
+                                                                  app_url=app_url)
-                        "report",
+                    f_args = ('report',
-                        template_vars["resource"].resource_id,
+                              template_vars['resource'].resource_id,
-                        template_vars["url_start_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_start_date'].strftime('%Y-%m-%dT%H:%M'),
-                        template_vars["url_end_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_end_date'].strftime('%Y-%m-%dT%H:%M'))
+                    destination_url += 'ui/{}?resource={}&start_date={}&end_date={}'.format(
-                    destination_url += "ui/{}?resource={}&start_date={}&end_date={}".format(*f_args)
+                        *f_args)
-                    if template_vars["confirmed_total"] > 1:
+                    if template_vars['confirmed_total'] > 1:
                         template_vars["title"] = "%s - %s reports" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["confirmed_total"],
+                            template_vars['confirmed_total'],
                         )
                     else:
-                        error_title = truncate(
+                        error_title = truncate(template_vars['reports'][0][1].error or
-                            template_vars["reports"][0][1].error or "slow report", 90
+                                               'slow report', 90)
                         template_vars["title"] = "%s - '%s' report" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            error_title,
+                            error_title)
-                    template_vars["title"] += " " + destination_url
+                    template_vars["title"] += ' ' + destination_url
-                    log_msg = "NOTIFY  : %s via %s :: %s reports" % (
+                    log_msg = 'NOTIFY  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
                         self.client.speak_to_room(room.strip(), template_vars["title"])
                 def notify_report_alert(self, **kwargs):
                     """
                     Build and send report alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if kwargs["event"].unified_alert_action() == "OPEN":
+                    if kwargs['event'].unified_alert_action() == 'OPEN':
-                        title = "ALERT %s: %s - %s %s %s" % (
+                        title = 'ALERT %s: %s - %s %s %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            kwargs["event"].values["reports"],
+                            kwargs['event'].values['reports'],
-                            template_vars["report_type"],
+                            template_vars['report_type'],
-                            template_vars["destination_url"],
+                            template_vars['destination_url']
                         )
                     else:
-                        title = "ALERT %s: %s type: %s" % (
+                        title = 'ALERT %s: %s type: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["alert_type"].replace("_", " "),
+                            template_vars['alert_type'].replace('_', ' '),
                         )
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        self.client.speak_to_room(room.strip(), title, sound="VUVUZELA")
+                        self.client.speak_to_room(room.strip(), title, sound='VUVUZELA')
                 def notify_uptime_alert(self, **kwargs):
                     """
                     Build and send uptime alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.uptime_alert_notification_vars(kwargs)
-                    message = "ALERT %s: %s has uptime issues %s\n\n" % (
+                    message = 'ALERT %s: %s has uptime issues %s\n\n' % (
-                        template_vars["alert_action"],
+                        template_vars['alert_action'],
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
-                        template_vars["destination_url"],
+                        template_vars['destination_url']
                     )
-                    message += template_vars["reason"]
+                    message += template_vars['reason']
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        self.client.speak_to_room(room.strip(), message, sound="VUVUZELA")
+                        self.client.speak_to_room(room.strip(), message, sound='VUVUZELA')
                 def send_digest(self, **kwargs):
                     """
                     Build and send daily digest notification
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    f_args = (template_vars["resource_name"], template_vars["confirmed_total"])
+                    f_args = (template_vars['resource_name'],
+                              template_vars['confirmed_total'],)
                     message = "Daily report digest: %s - %s reports" % f_args
-                    message += "{}\n".format(template_vars["destination_url"])
+                    message += '{}\n'.format(template_vars['destination_url'])
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
                         self.client.speak_to_room(room.strip(), message)
-                    log_msg = "DIGEST  : %s via %s :: %s reports" % (
+                    log_msg = 'DIGEST  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
                 def notify_chart_alert(self, **kwargs):
                     """
                     Build and send chart alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.chart_alert_notification_vars(kwargs)
-                    message = (
+                    message = 'ALERT {}: value in "{}" chart: ' \
-                        'ALERT {}: value in "{}" chart: '
                               'met alert "{}" criteria {} \n'.format(
-                            template_vars["alert_action"],
+                        template_vars['alert_action'],
-                            template_vars["chart_name"],
+                        template_vars['chart_name'],
-                            template_vars["action_name"],
+                        template_vars['action_name'],
-                            template_vars["destination_url"],
+                        template_vars['destination_url']
                     )
-                    for item in template_vars["readable_values"]:
+                    for item in template_vars['readable_values']:
-                        message += "{}: {}\n".format(item["label"], item["value"])
+                        message += '{}: {}\n'.format(item['label'], item['value'])
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        self.client.speak_to_room(room.strip(), message, sound="VUVUZELA")
+                        self.client.speak_to_room(room.strip(), message, sound='VUVUZELA')

backend/src/appenlight/models/alert_channels/email.py

0 +49 -66

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.services.user import UserService
             from webhelpers2.text import truncate
             log = logging.getLogger(__name__)
             class EmailAlertChannel(AlertChannel):
                 """
                 Default email alerting channel
                 """
-                __mapper_args__ = {"polymorphic_identity": "email"}
+                __mapper_args__ = {
+                    'polymorphic_identity': 'email'
+                }
                 def notify_reports(self, **kwargs):
                     """
                     Notify user of individual reports
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if template_vars["confirmed_total"] > 1:
+                    if template_vars['confirmed_total'] > 1:
                         template_vars["title"] = "AppEnlight :: %s - %s reports" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["confirmed_total"],
+                            template_vars['confirmed_total'],
                         )
                     else:
-                        error_title = truncate(
+                        error_title = truncate(template_vars['reports'][0][1].error or
-                            template_vars["reports"][0][1].error or "slow report", 20
+                                               'slow report', 20)
                         template_vars["title"] = "AppEnlight :: %s - '%s' report" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            error_title,
+                            error_title)
+                    UserService.send_email(kwargs['request'],
-                    UserService.send_email(
-                        kwargs["request"],
                                            [self.channel_value],
                                            template_vars,
-                        "/email_templates/notify_reports.jinja2",
+                                           '/email_templates/notify_reports.jinja2')
+                    log_msg = 'NOTIFY  : %s via %s :: %s reports' % (
-                    log_msg = "NOTIFY  : %s via %s :: %s reports" % (
+                        kwargs['user'].user_name,
-                        kwargs["user"].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
                 def send_digest(self, **kwargs):
                     """
                     Build and send daily digest notification
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
                     title = "AppEnlight :: Daily report digest: %s - %s reports"
                     template_vars["email_title"] = title % (
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'],
                     )
-                    UserService.send_email(
+                    UserService.send_email(kwargs['request'],
-                        kwargs["request"],
                                            [self.channel_value],
                                            template_vars,
-                        "/email_templates/notify_reports.jinja2",
+                                           '/email_templates/notify_reports.jinja2',
                                            immediately=True,
-                        silent=True,
+                                           silent=True)
+                    log_msg = 'DIGEST  : %s via %s :: %s reports' % (
-                    log_msg = "DIGEST  : %s via %s :: %s reports" % (
+                        kwargs['user'].user_name,
-                        kwargs["user"].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
                 def notify_report_alert(self, **kwargs):
                     """
                     Build and send report alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if kwargs["event"].unified_alert_action() == "OPEN":
+                    if kwargs['event'].unified_alert_action() == 'OPEN':
-                        title = "AppEnlight :: ALERT %s: %s - %s %s" % (
+                        title = 'AppEnlight :: ALERT %s: %s - %s %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            kwargs["event"].values["reports"],
+                            kwargs['event'].values['reports'],
-                            template_vars["report_type"],
+                            template_vars['report_type'],
                         )
                     else:
-                        title = "AppEnlight :: ALERT %s: %s type: %s" % (
+                        title = 'AppEnlight :: ALERT %s: %s type: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["alert_type"].replace("_", " "),
+                            template_vars['alert_type'].replace('_', ' '),
                         )
-                    template_vars["email_title"] = title
+                    template_vars['email_title'] = title
-                    UserService.send_email(
+                    UserService.send_email(kwargs['request'], [self.channel_value],
-                        kwargs["request"],
-                        [self.channel_value],
                                            template_vars,
-                        "/email_templates/alert_reports.jinja2",
+                                           '/email_templates/alert_reports.jinja2')
                 def notify_uptime_alert(self, **kwargs):
                     """
                     Build and send uptime alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.uptime_alert_notification_vars(kwargs)
-                    title = "AppEnlight :: ALERT %s: %s has uptime issues" % (
+                    title = 'AppEnlight :: ALERT %s: %s has uptime issues' % (
-                        template_vars["alert_action"],
+                        template_vars['alert_action'],
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
                     )
-                    template_vars["email_title"] = title
+                    template_vars['email_title'] = title
-                    UserService.send_email(
+                    UserService.send_email(kwargs['request'], [self.channel_value],
-                        kwargs["request"],
-                        [self.channel_value],
                                            template_vars,
-                        "/email_templates/alert_uptime.jinja2",
+                                           '/email_templates/alert_uptime.jinja2')
                 def notify_chart_alert(self, **kwargs):
                     """
                     Build and send chart alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.chart_alert_notification_vars(kwargs)
-                    title = (
+                    title = 'AppEnlight :: ALERT {} value in "{}" chart' \
-                        'AppEnlight :: ALERT {} value in "{}" chart'
                             ' met alert "{}" criteria'.format(
-                            template_vars["alert_action"],
+                        template_vars['alert_action'],
-                            template_vars["chart_name"],
+                        template_vars['chart_name'],
-                            template_vars["action_name"],
+                        template_vars['action_name'],
                     )
+                    template_vars['email_title'] = title
-                    template_vars["email_title"] = title
+                    UserService.send_email(kwargs['request'], [self.channel_value],
-                    UserService.send_email(
-                        kwargs["request"],
-                        [self.channel_value],
                                            template_vars,
-                        "/email_templates/alert_chart.jinja2",
+                                           '/email_templates/alert_chart.jinja2')

backend/src/appenlight/models/alert_channels/flowdock.py

0 +82 -74

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.integrations.flowdock import FlowdockIntegration
             from webhelpers2.text import truncate
             log = logging.getLogger(__name__)
             class FlowdockAlertChannel(AlertChannel):
-                __mapper_args__ = {"polymorphic_identity": "flowdock"}
+                __mapper_args__ = {
+                    'polymorphic_identity': 'flowdock'
+                }
                 def notify_reports(self, **kwargs):
                     """
                     Notify user of individual reports
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", _app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/',
-                    f_args = (
+                                                                  _app_url=app_url)
-                        "report",
+                    f_args = ('report',
-                        template_vars["resource"].resource_id,
+                              template_vars['resource'].resource_id,
-                        template_vars["url_start_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_start_date'].strftime('%Y-%m-%dT%H:%M'),
-                        template_vars["url_end_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_end_date'].strftime('%Y-%m-%dT%H:%M'))
+                    destination_url += 'ui/{}?resource={}&start_date={}&end_date={}'.format(
-                    destination_url += "ui/{}?resource={}&start_date={}&end_date={}".format(*f_args)
+                        *f_args)
-                    if template_vars["confirmed_total"] > 1:
+                    if template_vars['confirmed_total'] > 1:
                         template_vars["title"] = "%s - %s reports" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["confirmed_total"],
+                            template_vars['confirmed_total'],
                         )
                     else:
-                        error_title = truncate(
+                        error_title = truncate(template_vars['reports'][0][1].error or
-                            template_vars["reports"][0][1].error or "slow report", 90
+                                               'slow report', 90)
                         template_vars["title"] = "%s - '%s' report" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            error_title,
+                            error_title)
-                    log_msg = "NOTIFY : %s via %s :: %s reports" % (
+                    log_msg = 'NOTIFY : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
-                    client = FlowdockIntegration.create_client(self.integration.config["api_token"])
+                    client = FlowdockIntegration.create_client(
+                        self.integration.config['api_token'])
                     payload = {
                         "source": "AppEnlight",
-                        "from_address": kwargs["request"].registry.settings["mailing.from_email"],
+                        "from_address": kwargs['request'].registry.settings[
+                            'mailing.from_email'],
                         "subject": template_vars["title"],
                         "content": "New report present",
                         "tags": ["appenlight"],
-                        "link": destination_url,
+                        "link": destination_url
                     }
                     client.send_to_inbox(payload)
                 def notify_report_alert(self, **kwargs):
                     """
                     Build and send report alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if kwargs["event"].unified_alert_action() == "OPEN":
+                    if kwargs['event'].unified_alert_action() == 'OPEN':
-                        title = "ALERT %s: %s - %s %s" % (
+                        title = 'ALERT %s: %s - %s %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            kwargs["event"].values["reports"],
+                            kwargs['event'].values['reports'],
-                            template_vars["report_type"],
+                            template_vars['report_type'],
                         )
                     else:
-                        title = "ALERT %s: %s type: %s" % (
+                        title = 'ALERT %s: %s type: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["alert_type"].replace("_", " "),
+                            template_vars['alert_type'].replace('_', ' '),
                         )
-                    client = FlowdockIntegration.create_client(self.integration.config["api_token"])
+                    client = FlowdockIntegration.create_client(
+                        self.integration.config['api_token'])
                     payload = {
                         "source": "AppEnlight",
-                        "from_address": kwargs["request"].registry.settings["mailing.from_email"],
+                        "from_address": kwargs['request'].registry.settings[
+                            'mailing.from_email'],
                         "subject": title,
-                        "content": "Investigation required",
+                        "content": 'Investigation required',
-                        "tags": ["appenlight", "alert", template_vars["alert_type"]],
+                        "tags": ["appenlight", "alert", template_vars['alert_type']],
-                        "link": template_vars["destination_url"],
+                        "link": template_vars['destination_url']
                     }
                     client.send_to_inbox(payload)
                 def notify_uptime_alert(self, **kwargs):
                     """
                     Build and send uptime alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.uptime_alert_notification_vars(kwargs)
-                    message = "ALERT %s: %s has uptime issues" % (
+                    message = 'ALERT %s: %s has uptime issues' % (
-                        template_vars["alert_action"],
+                        template_vars['alert_action'],
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
                     )
-                    submessage = "Info: "
+                    submessage = 'Info: '
-                    submessage += template_vars["reason"]
+                    submessage += template_vars['reason']
-                    client = FlowdockIntegration.create_client(self.integration.config["api_token"])
+                    client = FlowdockIntegration.create_client(
+                        self.integration.config['api_token'])
                     payload = {
                         "source": "AppEnlight",
-                        "from_address": kwargs["request"].registry.settings["mailing.from_email"],
+                        "from_address": kwargs['request'].registry.settings[
+                            'mailing.from_email'],
                         "subject": message,
                         "content": submessage,
-                        "tags": ["appenlight", "alert", "uptime"],
+                        "tags": ["appenlight", "alert", 'uptime'],
-                        "link": template_vars["destination_url"],
+                        "link": template_vars['destination_url']
                     }
                     client.send_to_inbox(payload)
                 def send_digest(self, **kwargs):
                     """
                     Build and send daily digest notification
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
                     message = "Daily report digest: %s - %s reports" % (
-                        template_vars["resource_name"],
+                        template_vars['resource_name'], template_vars['confirmed_total'])
-                        template_vars["confirmed_total"],
-                    f_args = (template_vars["confirmed_total"], template_vars["timestamp"])
+                    f_args = (template_vars['confirmed_total'],
+                              template_vars['timestamp'])
                     payload = {
                         "source": "AppEnlight",
-                        "from_address": kwargs["request"].registry.settings["mailing.from_email"],
+                        "from_address": kwargs['request'].registry.settings[
+                            'mailing.from_email'],
                         "subject": message,
-                        "content": "%s reports in total since %s" % f_args,
+                        "content": '%s reports in total since %s' % f_args,
                         "tags": ["appenlight", "digest"],
-                        "link": template_vars["destination_url"],
+                        "link": template_vars['destination_url']
                     }
-                    client = FlowdockIntegration.create_client(self.integration.config["api_token"])
+                    client = FlowdockIntegration.create_client(
+                        self.integration.config['api_token'])
                     client.send_to_inbox(payload)
-                    log_msg = "DIGEST  : %s via %s :: %s reports" % (
+                    log_msg = 'DIGEST  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
                 def notify_chart_alert(self, **kwargs):
                     """
                     Build and send chart alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.chart_alert_notification_vars(kwargs)
-                    message = 'ALERT {}: value in "{}" chart ' 'met alert "{}" criteria'.format(
+                    message = 'ALERT {}: value in "{}" chart ' \
-                        template_vars["alert_action"],
+                              'met alert "{}" criteria'.format(
-                        template_vars["chart_name"],
+                        template_vars['alert_action'],
-                        template_vars["action_name"],
+                        template_vars['chart_name'],
+                        template_vars['action_name'],
                     )
-                    submessage = "Info: "
+                    submessage = 'Info: '
-                    for item in template_vars["readable_values"]:
+                    for item in template_vars['readable_values']:
-                        submessage += "{}: {}\n".format(item["label"], item["value"])
+                        submessage += '{}: {}\n'.format(item['label'], item['value'])
-                    client = FlowdockIntegration.create_client(self.integration.config["api_token"])
+                    client = FlowdockIntegration.create_client(
+                        self.integration.config['api_token'])
                     payload = {
                         "source": "AppEnlight",
-                        "from_address": kwargs["request"].registry.settings["mailing.from_email"],
+                        "from_address": kwargs['request'].registry.settings[
+                            'mailing.from_email'],
                         "subject": message,
                         "content": submessage,
-                        "tags": ["appenlight", "alert", "chart"],
+                        "tags": ["appenlight", "alert", 'chart'],
-                        "link": template_vars["destination_url"],
+                        "link": template_vars['destination_url']
                     }
                     client.send_to_inbox(payload)

backend/src/appenlight/models/alert_channels/hipchat.py

0 +85 -94

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.integrations.hipchat import HipchatIntegration
             from webhelpers2.text import truncate
             log = logging.getLogger(__name__)
             class HipchatAlertChannel(AlertChannel):
-                __mapper_args__ = {"polymorphic_identity": "hipchat"}
+                __mapper_args__ = {
+                    'polymorphic_identity': 'hipchat'
+                }
                 def notify_reports(self, **kwargs):
                     """
                     Notify user of individual reports
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    app_url = kwargs["request"].registry.settings["_mail_url"]
+                    app_url = kwargs['request'].registry.settings['_mail_url']
-                    destination_url = kwargs["request"].route_url("/", _app_url=app_url)
+                    destination_url = kwargs['request'].route_url('/',
-                    f_args = (
+                                                                  _app_url=app_url)
-                        "report",
+                    f_args = ('report',
-                        template_vars["resource"].resource_id,
+                              template_vars['resource'].resource_id,
-                        template_vars["url_start_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_start_date'].strftime('%Y-%m-%dT%H:%M'),
-                        template_vars["url_end_date"].strftime("%Y-%m-%dT%H:%M"),
+                              template_vars['url_end_date'].strftime('%Y-%m-%dT%H:%M'))
+                    destination_url += 'ui/{}?resource={}&start_date={}&end_date={}'.format(
-                    destination_url += "ui/{}?resource={}&start_date={}&end_date={}".format(*f_args)
+                        *f_args)
-                    if template_vars["confirmed_total"] > 1:
+                    if template_vars['confirmed_total'] > 1:
                         template_vars["title"] = "%s - %s reports" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["confirmed_total"],
+                            template_vars['confirmed_total'],
                         )
                     else:
-                        error_title = truncate(
+                        error_title = truncate(template_vars['reports'][0][1].error or
-                            template_vars["reports"][0][1].error or "slow report", 90
+                                               'slow report', 90)
                         template_vars["title"] = "%s - '%s' report" % (
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            error_title,
+                            error_title)
-                    template_vars["title"] += " " + destination_url
+                    template_vars["title"] += ' ' + destination_url
-                    log_msg = "NOTIFY  : %s via %s :: %s reports" % (
+                    log_msg = 'NOTIFY  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
-                    client = HipchatIntegration.create_client(self.integration.config["api_token"])
+                    client = HipchatIntegration.create_client(
-                    for room in self.integration.config["rooms"].split(","):
+                        self.integration.config['api_token'])
-                        client.send(
+                    for room in self.integration.config['rooms'].split(','):
+                        client.send({
-                                "message_format": "text",
+                            "message_format": 'text',
                             "message": template_vars["title"],
                             "from": "AppEnlight",
                             "room_id": room.strip(),
-                                "color": "yellow",
+                            "color": "yellow"
+                        })
                 def notify_report_alert(self, **kwargs):
                     """
                     Build and send report alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if kwargs["event"].unified_alert_action() == "OPEN":
+                    if kwargs['event'].unified_alert_action() == 'OPEN':
-                        title = "ALERT %s: %s - %s %s" % (
+                        title = 'ALERT %s: %s - %s %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            kwargs["event"].values["reports"],
+                            kwargs['event'].values['reports'],
-                            template_vars["report_type"],
+                            template_vars['report_type'],
                         )
                     else:
-                        title = "ALERT %s: %s type: %s" % (
+                        title = 'ALERT %s: %s type: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
-                            template_vars["alert_type"].replace("_", " "),
+                            template_vars['alert_type'].replace('_', ' '),
                         )
-                    title += "\n " + template_vars["destination_url"]
+                    title += '\n ' + template_vars['destination_url']
-                    api_token = self.integration.config["api_token"]
+                    api_token = self.integration.config['api_token']
                     client = HipchatIntegration.create_client(api_token)
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        client.send(
+                        client.send({
+                            "message_format": 'text',
-                                "message_format": "text",
                             "message": title,
                             "from": "AppEnlight",
                             "room_id": room.strip(),
                             "color": "red",
-                                "notify": "1",
+                            "notify": '1'
+                        })
                 def notify_uptime_alert(self, **kwargs):
                     """
                     Build and send uptime alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.uptime_alert_notification_vars(kwargs)
-                    message = "ALERT %s: %s has uptime issues\n" % (
+                    message = 'ALERT %s: %s has uptime issues\n' % (
-                        template_vars["alert_action"],
+                        template_vars['alert_action'],
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
                     )
-                    message += template_vars["reason"]
+                    message += template_vars['reason']
-                    message += "\n{}".format(template_vars["destination_url"])
+                    message += '\n{}'.format(template_vars['destination_url'])
-                    api_token = self.integration.config["api_token"]
+                    api_token = self.integration.config['api_token']
                     client = HipchatIntegration.create_client(api_token)
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        client.send(
+                        client.send({
+                            "message_format": 'text',
-                                "message_format": "text",
                             "message": message,
                             "from": "AppEnlight",
                             "room_id": room.strip(),
                             "color": "red",
-                                "notify": "1",
+                            "notify": '1'
+                        })
                 def notify_chart_alert(self, **kwargs):
                     """
                     Build and send chart alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.chart_alert_notification_vars(kwargs)
-                    message = 'ALERT {}: value in "{}" chart: ' 'met alert "{}" criteria\n'.format(
+                    message = 'ALERT {}: value in "{}" chart: ' \
-                        template_vars["alert_action"],
+                              'met alert "{}" criteria\n'.format(
-                        template_vars["chart_name"],
+                        template_vars['alert_action'],
-                        template_vars["action_name"],
+                        template_vars['chart_name'],
+                        template_vars['action_name'],
                     )
-                    for item in template_vars["readable_values"]:
+                    for item in template_vars['readable_values']:
-                        message += "{}: {}\n".format(item["label"], item["value"])
+                        message += '{}: {}\n'.format(item['label'], item['value'])
-                    message += template_vars["destination_url"]
+                    message += template_vars['destination_url']
-                    api_token = self.integration.config["api_token"]
+                    api_token = self.integration.config['api_token']
                     client = HipchatIntegration.create_client(api_token)
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        client.send(
+                        client.send({
+                            "message_format": 'text',
-                                "message_format": "text",
                             "message": message,
                             "from": "AppEnlight",
                             "room_id": room.strip(),
                             "color": "red",
-                                "notify": "1",
+                            "notify": '1'
+                        })
                 def send_digest(self, **kwargs):
                     """
                     Build and send daily digest notification
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    f_args = (template_vars["resource_name"], template_vars["confirmed_total"])
+                    f_args = (template_vars['resource_name'],
+                              template_vars['confirmed_total'],)
                     message = "Daily report digest: %s - %s reports" % f_args
-                    message += "\n{}".format(template_vars["destination_url"])
+                    message += '\n{}'.format(template_vars['destination_url'])
-                    api_token = self.integration.config["api_token"]
+                    api_token = self.integration.config['api_token']
                     client = HipchatIntegration.create_client(api_token)
-                    for room in self.integration.config["rooms"].split(","):
+                    for room in self.integration.config['rooms'].split(','):
-                        client.send(
+                        client.send({
+                            "message_format": 'text',
-                                "message_format": "text",
                             "message": message,
                             "from": "AppEnlight",
                             "room_id": room.strip(),
                             "color": "green",
-                                "notify": "1",
+                            "notify": '1'
+                        })
-                    log_msg = "DIGEST  : %s via %s :: %s reports" % (
+                    log_msg = 'DIGEST  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)

backend/src/appenlight/models/alert_channels/slack.py

0 +101 -86

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             from appenlight.models.alert_channel import AlertChannel
             from appenlight.models.integrations.slack import SlackIntegration
             from webhelpers2.text import truncate
             log = logging.getLogger(__name__)
             class SlackAlertChannel(AlertChannel):
-                __mapper_args__ = {"polymorphic_identity": "slack"}
+                __mapper_args__ = {
+                    'polymorphic_identity': 'slack'
+                }
                 def notify_reports(self, **kwargs):
                     """
                     Notify user of individual reports
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    template_vars["title"] = template_vars["resource_name"]
+                    template_vars["title"] = template_vars['resource_name']
-                    if template_vars["confirmed_total"] > 1:
+                    if template_vars['confirmed_total'] > 1:
-                        template_vars["subtext"] = "%s reports" % template_vars["confirmed_total"]
+                        template_vars['subtext'] = '%s reports' % template_vars[
+                            'confirmed_total']
                     else:
-                        error_title = truncate(
+                        error_title = truncate(template_vars['reports'][0][1].error or
-                            template_vars["reports"][0][1].error or "slow report", 90
+                                               'slow report', 90)
+                        template_vars['subtext'] = error_title
-                        template_vars["subtext"] = error_title
-                    log_msg = "NOTIFY  : %s via %s :: %s reports" % (
+                    log_msg = 'NOTIFY  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)
-                    client = SlackIntegration.create_client(self.integration.config["webhook_url"])
+                    client = SlackIntegration.create_client(
+                        self.integration.config['webhook_url'])
                     report_data = {
                         "username": "AppEnlight",
                         "icon_emoji": ":fire:",
                         "attachments": [
                             {
                                 "mrkdwn_in": ["text", "pretext", "title", "fallback"],
-                                "fallback": "*%s* - <%s| Browse>"
+                                "fallback": "*%s* - <%s| Browse>" % (
-                                % (template_vars["title"], template_vars["destination_url"]),
+                                    template_vars["title"],
-                                "pretext": "*%s* - <%s| Browse>"
+                                    template_vars['destination_url']),
-                                % (template_vars["title"], template_vars["destination_url"]),
+                                "pretext": "*%s* - <%s| Browse>" % (
+                                    template_vars["title"],
+                                    template_vars['destination_url']),
                                 "color": "warning",
                                 "fields": [
-                                    {"value": "Info: %s" % template_vars["subtext"], "short": False}
-                                ],
+                                        "value": 'Info: %s' % template_vars['subtext'],
+                                        "short": False
                                     }
-                        ],
+                            }
+                        ]
                     }
                     client.make_request(data=report_data)
                 def notify_report_alert(self, **kwargs):
                     """
                     Build and send report alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    if kwargs["event"].unified_alert_action() == "OPEN":
+                    if kwargs['event'].unified_alert_action() == 'OPEN':
-                        title = "*ALERT %s*: %s" % (
+                        title = '*ALERT %s*: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name']
                         )
-                        template_vars["subtext"] = "Got at least %s %s" % (
+                        template_vars['subtext'] = 'Got at least %s %s' % (
-                            kwargs["event"].values["reports"],
+                            kwargs['event'].values['reports'],
-                            template_vars["report_type"],
+                            template_vars['report_type']
                         )
                     else:
-                        title = "*ALERT %s*: %s" % (
+                        title = '*ALERT %s*: %s' % (
-                            template_vars["alert_action"],
+                            template_vars['alert_action'],
-                            template_vars["resource_name"],
+                            template_vars['resource_name'],
                         )
-                        template_vars["subtext"] = ""
+                        template_vars['subtext'] = ''
-                    alert_type = template_vars["alert_type"].replace("_", " ")
+                    alert_type = template_vars['alert_type'].replace('_', ' ')
-                    alert_type = alert_type.replace("alert", "").capitalize()
+                    alert_type = alert_type.replace('alert', '').capitalize()
-                    template_vars["type"] = "Type: %s" % alert_type
+                    template_vars['type'] = "Type: %s" % alert_type
-                    client = SlackIntegration.create_client(self.integration.config["webhook_url"])
+                    client = SlackIntegration.create_client(
+                        self.integration.config['webhook_url']
+                    )
                     report_data = {
                         "username": "AppEnlight",
                         "icon_emoji": ":rage:",
                         "attachments": [
                             {
                                 "mrkdwn_in": ["text", "pretext", "title", "fallback"],
-                                "fallback": "%s - <%s| Browse>"
+                                "fallback": "%s - <%s| Browse>" % (
-                                % (title, template_vars["destination_url"]),
+                                    title, template_vars['destination_url']),
-                                "pretext": "%s - <%s| Browse>"
+                                "pretext": "%s - <%s| Browse>" % (
-                                % (title, template_vars["destination_url"]),
+                                    title, template_vars['destination_url']),
                                 "color": "danger",
                                 "fields": [
                                     {
-                                        "title": template_vars["type"],
+                                        "title": template_vars['type'],
-                                        "value": template_vars["subtext"],
+                                        "value": template_vars['subtext'],
-                                        "short": False,
+                                        "short": False
                                     }
-                                ],
                             }
-                        ],
                     }
                     client.make_request(data=report_data)
                 def notify_uptime_alert(self, **kwargs):
                     """
                     Build and send uptime alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.uptime_alert_notification_vars(kwargs)
-                    title = "*ALERT %s*: %s" % (
+                    title = '*ALERT %s*: %s' % (
-                        template_vars["alert_action"],
+                        template_vars['alert_action'],
-                        template_vars["resource_name"],
+                        template_vars['resource_name'],
+                    )
+                    client = SlackIntegration.create_client(
+                        self.integration.config['webhook_url']
                     )
-                    client = SlackIntegration.create_client(self.integration.config["webhook_url"])
                     report_data = {
                         "username": "AppEnlight",
                         "icon_emoji": ":rage:",
                         "attachments": [
                             {
                                 "mrkdwn_in": ["text", "pretext", "title", "fallback"],
                                 "fallback": "{} - <{}| Browse>".format(
-                                    title, template_vars["destination_url"]
+                                    title, template_vars['destination_url']),
-                                ),
                                 "pretext": "{} - <{}| Browse>".format(
-                                    title, template_vars["destination_url"]
+                                    title, template_vars['destination_url']),
-                                ),
                                 "color": "danger",
                                 "fields": [
                                     {
                                         "title": "Application has uptime issues",
-                                        "value": template_vars["reason"],
+                                        "value": template_vars['reason'],
-                                        "short": False,
+                                        "short": False
                                     }
-                                ],
                             }
-                        ],
                     }
                     client.make_request(data=report_data)
                 def notify_chart_alert(self, **kwargs):
                     """
                     Build and send chart alert notification
                     Kwargs:
                         application: application that the event applies for,
                         event: event that is notified,
                         user: user that should be notified
                         request: request object
                     """
                     template_vars = self.chart_alert_notification_vars(kwargs)
-                    title = '*ALERT {}*: value in *"{}"* chart ' 'met alert *"{}"* criteria'.format(
+                    title = '*ALERT {}*: value in *"{}"* chart ' \
-                        template_vars["alert_action"],
+                            'met alert *"{}"* criteria'.format(
-                        template_vars["chart_name"],
+                        template_vars['alert_action'],
-                        template_vars["action_name"],
+                        template_vars['chart_name'],
+                        template_vars['action_name'],
                     )
-                    subtext = ""
+                    subtext = ''
-                    for item in template_vars["readable_values"]:
+                    for item in template_vars['readable_values']:
-                        subtext += "{} - {}\n".format(item["label"], item["value"])
+                        subtext += '{} - {}\n'.format(item['label'], item['value'])
-                    client = SlackIntegration.create_client(self.integration.config["webhook_url"])
+                    client = SlackIntegration.create_client(
+                        self.integration.config['webhook_url']
+                    )
                     report_data = {
                         "username": "AppEnlight",
                         "icon_emoji": ":rage:",
                         "attachments": [
+                            {"mrkdwn_in": ["text", "pretext", "title", "fallback"],
-                                "mrkdwn_in": ["text", "pretext", "title", "fallback"],
                              "fallback": "{} - <{}| Browse>".format(
-                                    title, template_vars["destination_url"]
+                                 title, template_vars['destination_url']),
-                                ),
                              "pretext": "{} - <{}| Browse>".format(
-                                    title, template_vars["destination_url"]
+                                 title, template_vars['destination_url']),
-                                ),
                              "color": "danger",
                              "fields": [
                                  {
                                      "title": "Following criteria were met:",
                                      "value": subtext,
-                                        "short": False,
+                                     "short": False
                                  }
-                                ],
                              }
-                        ],
                     }
                     client.make_request(data=report_data)
                 def send_digest(self, **kwargs):
                     """
                     Build and send daily digest notification
                     kwargs:
                         application: application that the event applies for,
                         user: user that should be notified
                         request: request object
                         since_when: reports are newer than this time value,
                         reports: list of reports to render
                     """
                     template_vars = self.report_alert_notification_vars(kwargs)
-                    title = "*Daily report digest*: %s" % template_vars["resource_name"]
+                    title = "*Daily report digest*: %s" % template_vars['resource_name']
-                    subtext = "%s reports" % template_vars["confirmed_total"]
+                    subtext = '%s reports' % template_vars['confirmed_total']
-                    client = SlackIntegration.create_client(self.integration.config["webhook_url"])
+                    client = SlackIntegration.create_client(
+                        self.integration.config['webhook_url']
+                    )
                     report_data = {
                         "username": "AppEnlight",
                         "attachments": [
                             {
                                 "mrkdwn_in": ["text", "pretext", "title", "fallback"],
-                                "fallback": "%s : <%s| Browse>"
+                                "fallback": "%s : <%s| Browse>" % (
-                                % (title, template_vars["destination_url"]),
+                                    title, template_vars['destination_url']),
-                                "pretext": "%s: <%s| Browse>"
+                                "pretext": "%s: <%s| Browse>" % (
-                                % (title, template_vars["destination_url"]),
+                                    title, template_vars['destination_url']),
                                 "color": "good",
-                                "fields": [{"title": "Got at least: %s" % subtext, "short": False}],
+                                "fields": [
+                                    {
+                                        "title": "Got at least: %s" % subtext,
+                                        "short": False
+                                    }
+                                ]
                             }
-                        ],
                     }
                     client.make_request(data=report_data)
-                    log_msg = "DIGEST  : %s via %s :: %s reports" % (
+                    log_msg = 'DIGEST  : %s via %s :: %s reports' % (
-                        kwargs["user"].user_name,
+                        kwargs['user'].user_name,
                         self.channel_visible_value,
-                        template_vars["confirmed_total"],
+                        template_vars['confirmed_total'])
                     log.warning(log_msg)

backend/src/appenlight/models/application.py

0 +38 -47

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import uuid
             import logging
             import sqlalchemy as sa
             from appenlight.models.resource import Resource
             from sqlalchemy.orm import aliased
             log = logging.getLogger(__name__)
             def generate_api_key():
-                uid = str(uuid.uuid4()).replace("-", "")
+                uid = str(uuid.uuid4()).replace('-', '')
                 return uid[0:32]
             class Application(Resource):
                 """
                 Resource of application type
                 """
-                __tablename__ = "applications"
+                __tablename__ = 'applications'
-                __mapper_args__ = {"polymorphic_identity": "application"}
+                __mapper_args__ = {'polymorphic_identity': 'application'}
                 # lists configurable possible permissions for this resource type
-                __possible_permissions__ = ("view", "update_reports")
+                __possible_permissions__ = ('view', 'update_reports')
-                resource_id = sa.Column(
+                resource_id = sa.Column(sa.Integer(),
-                    sa.Integer(),
+                                        sa.ForeignKey('resources.resource_id',
-                    sa.ForeignKey("resources.resource_id", onupdate="CASCADE", ondelete="CASCADE"),
+                                                      onupdate='CASCADE',
-                    primary_key=True,
+                                                      ondelete='CASCADE', ),
+                                        primary_key=True, )
-                domains = sa.Column(sa.UnicodeText(), nullable=False, default="")
+                domains = sa.Column(sa.UnicodeText(), nullable=False, default='')
-                api_key = sa.Column(
+                api_key = sa.Column(sa.String(32), nullable=False, unique=True, index=True,
-                    sa.String(32), nullable=False, unique=True, index=True, default=generate_api_key
+                                    default=generate_api_key)
+                public_key = sa.Column(sa.String(32), nullable=False, unique=True,
-                public_key = sa.Column(
+                                       index=True,
-                    sa.String(32), nullable=False, unique=True, index=True, default=generate_api_key
+                                       default=generate_api_key)
+                default_grouping = sa.Column(sa.Unicode(20), nullable=False,
-                default_grouping = sa.Column(
+                                             default='url_traceback')
-                    sa.Unicode(20), nullable=False, default="url_traceback"
                 error_report_threshold = sa.Column(sa.Integer(), default=10)
                 slow_report_threshold = sa.Column(sa.Integer(), default=10)
-                allow_permanent_storage = sa.Column(sa.Boolean(), default=False, nullable=False)
+                allow_permanent_storage = sa.Column(sa.Boolean(), default=False,
+                                                    nullable=False)
-                @sa.orm.validates("default_grouping")
+                @sa.orm.validates('default_grouping')
                 def validate_default_grouping(self, key, grouping):
                     """ validate if resouce can have specific permission """
-                    assert grouping in ["url_type", "url_traceback", "traceback_server"]
+                    assert grouping in ['url_type', 'url_traceback', 'traceback_server']
                     return grouping
-                report_groups = sa.orm.relationship(
+                report_groups = sa.orm.relationship('ReportGroup',
-                    "ReportGroup",
                                                     cascade="all, delete-orphan",
                                                     passive_deletes=True,
                                                     passive_updates=True,
-                    lazy="dynamic",
+                                                    lazy='dynamic',
-                    backref=sa.orm.backref("application", lazy="joined"),
+                                                    backref=sa.orm.backref('application',
+                                                                           lazy="joined"))
-                postprocess_conf = sa.orm.relationship(
+                postprocess_conf = sa.orm.relationship('ApplicationPostprocessConf',
-                    "ApplicationPostprocessConf",
                                                        cascade="all, delete-orphan",
                                                        passive_deletes=True,
                                                        passive_updates=True,
-                    backref="resource",
+                                                       backref='resource')
-                logs = sa.orm.relationship(
+                logs = sa.orm.relationship('Log',
-                    "Log",
+                                           lazy='dynamic',
-                    lazy="dynamic",
+                                           backref='application',
-                    backref="application",
                                            passive_deletes=True,
-                    passive_updates=True,
+                                           passive_updates=True, )
-                integrations = sa.orm.relationship(
+                integrations = sa.orm.relationship('IntegrationBase',
-                    "IntegrationBase",
+                                                   backref='resource',
-                    backref="resource",
                                                    cascade="all, delete-orphan",
                                                    passive_deletes=True,
-                    passive_updates=True,
+                                                   passive_updates=True, )
                 def generate_api_key(self):
                     return generate_api_key()
             def after_update(mapper, connection, target):
                 from appenlight.models.services.application import ApplicationService
+                log.info('clearing out ApplicationService cache')
-                log.info("clearing out ApplicationService cache")
                 ApplicationService.by_id_cached().invalidate(target.resource_id)
                 ApplicationService.by_api_key_cached().invalidate(target.api_key)
-            sa.event.listen(Application, "after_update", after_update)
+            sa.event.listen(Application, 'after_update', after_update)
-            sa.event.listen(Application, "after_delete", after_update)
+            sa.event.listen(Application, 'after_delete', after_update)

backend/src/appenlight/models/application_postprocess_conf.py

0 +9 -11

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             from ziggurat_foundations.models.base import BaseModel
             import sqlalchemy as sa
             from appenlight.models import Base
             from appenlight.models.report_group import ReportGroup
             class ApplicationPostprocessConf(Base, BaseModel):
                 """
                 Stores prioritizing conditions for reports
                 This is later used for rule parsing like "if 10 occurences bump priority +1"
                 """
-                __tablename__ = "application_postprocess_conf"
+                __tablename__ = 'application_postprocess_conf'
                 pkey = sa.Column(sa.Integer(), nullable=False, primary_key=True)
-                resource_id = sa.Column(
+                resource_id = sa.Column(sa.Integer(),
-                    sa.Integer(),
+                                        sa.ForeignKey('resources.resource_id',
-                    sa.ForeignKey("resources.resource_id", onupdate="CASCADE", ondelete="CASCADE"),
+                                                      onupdate='CASCADE',
+                                                      ondelete='CASCADE'))
                 do = sa.Column(sa.Unicode(25), nullable=False)
-                new_value = sa.Column(sa.UnicodeText(), nullable=False, default="")
+                new_value = sa.Column(sa.UnicodeText(), nullable=False, default='')
-                rule = sa.Column(
+                rule = sa.Column(sa.dialects.postgresql.JSON,
-                    sa.dialects.postgresql.JSON,
+                                 nullable=False, default={'field': 'http_status',
-                    nullable=False,
+                                                          "op": "ge", "value": "500"})
-                    default={"field": "http_status", "op": "ge", "value": "500"},
                 def postprocess(self, item):
                     new_value = int(self.new_value)
                     item.priority = ReportGroup.priority + new_value

backend/src/appenlight/models/auth_token.py

0 +8 -13

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import logging
             import sqlalchemy as sa
             from datetime import datetime
             from appenlight.models import Base
             from ziggurat_foundations.models.base import BaseModel
             from ziggurat_foundations.models.services.user import UserService
             log = logging.getLogger(__name__)
             class AuthToken(Base, BaseModel):
                 """
                 Stores information about possible alerting options
                 """
+                __tablename__ = 'auth_tokens'
-                __tablename__ = "auth_tokens"
                 id = sa.Column(sa.Integer, primary_key=True, nullable=False)
-                token = sa.Column(
+                token = sa.Column(sa.Unicode(40), nullable=False,
-                    sa.Unicode(40),
+                                  default=lambda x: UserService.generate_random_string(40))
-                    nullable=False,
+                owner_id = sa.Column(sa.Unicode(30),
-                    default=lambda x: UserService.generate_random_string(40),
+                                     sa.ForeignKey('users.id', onupdate='CASCADE',
+                                                   ondelete='CASCADE'))
-                owner_id = sa.Column(
-                    sa.Unicode(30),
-                    sa.ForeignKey("users.id", onupdate="CASCADE", ondelete="CASCADE"),
                 creation_date = sa.Column(sa.DateTime, default=lambda x: datetime.utcnow())
                 expires = sa.Column(sa.DateTime)
-                description = sa.Column(sa.Unicode, default="")
+                description = sa.Column(sa.Unicode, default='')
                 @property
                 def is_expired(self):
                     if self.expires:
                         return self.expires < datetime.utcnow()
                     else:
                         return False
                 def __str__(self):
-                    return "<AuthToken u:%s t:%s...>" % (self.owner_id, self.token[0:10])
+                    return '<AuthToken u:%s t:%s...>' % (self.owner_id, self.token[0:10])

backend/src/appenlight/models/config.py

0 +2 -2

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import sqlalchemy as sa
             from ziggurat_foundations.models.base import BaseModel
-            from sqlalchemy.dialects.postgresql import JSON
+            from sqlalchemy.dialects.postgres import JSON
             from . import Base
             class Config(Base, BaseModel):
-                __tablename__ = "config"
+                __tablename__ = 'config'
                 key = sa.Column(sa.Unicode, primary_key=True)
                 section = sa.Column(sa.Unicode, primary_key=True)
                 value = sa.Column(JSON, nullable=False)
                 def __json__(self, request):
                     return self.get_dict()

backend/src/appenlight/models/event.py

0 +55 -65

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import sqlalchemy as sa
             import logging
             from datetime import datetime
             from appenlight.models import Base, get_db_session
             from appenlight.models.services.report_stat import ReportStatService
+            from appenlight.models.resource import Resource
             from appenlight.models.integrations import IntegrationException
             from pyramid.threadlocal import get_current_request
             from sqlalchemy.dialects.postgresql import JSON
             from ziggurat_foundations.models.base import BaseModel
-            from ziggurat_foundations.models.services.resource import ResourceService
             log = logging.getLogger(__name__)
             class Event(Base, BaseModel):
-                __tablename__ = "events"
+                __tablename__ = 'events'
-                types = {
+                types = {'error_report_alert': 1,
-                    "error_report_alert": 1,
+                         'slow_report_alert': 3,
-                    "slow_report_alert": 3,
+                         'comment': 5,
-                    "comment": 5,
+                         'assignment': 6,
-                    "assignment": 6,
+                         'uptime_alert': 7,
-                    "uptime_alert": 7,
+                         'chart_alert': 9}
-                    "chart_alert": 9,
-                statuses = {"active": 1, "closed": 0}
+                statuses = {'active': 1,
+                            'closed': 0}
                 id = sa.Column(sa.Integer, primary_key=True)
                 start_date = sa.Column(sa.DateTime, default=datetime.utcnow)
                 end_date = sa.Column(sa.DateTime)
                 status = sa.Column(sa.Integer, default=1)
                 event_type = sa.Column(sa.Integer, default=1)
-                origin_user_id = sa.Column(sa.Integer(), sa.ForeignKey("users.id"), nullable=True)
+                origin_user_id = sa.Column(sa.Integer(), sa.ForeignKey('users.id'),
-                target_user_id = sa.Column(sa.Integer(), sa.ForeignKey("users.id"), nullable=True)
+                                           nullable=True)
-                resource_id = sa.Column(
+                target_user_id = sa.Column(sa.Integer(), sa.ForeignKey('users.id'),
-                    sa.Integer(), sa.ForeignKey("resources.resource_id"), nullable=True
+                                           nullable=True)
+                resource_id = sa.Column(sa.Integer(),
+                                        sa.ForeignKey('resources.resource_id'),
+                                        nullable=True)
                 target_id = sa.Column(sa.Integer)
                 target_uuid = sa.Column(sa.Unicode(40))
                 text = sa.Column(sa.UnicodeText())
                 values = sa.Column(JSON(), nullable=False, default=None)
                 def __repr__(self):
-                    return "<Event %s, app:%s, %s>" % (
+                    return '<Event %s, app:%s, %s>' % (self.unified_alert_name(),
-                        self.unified_alert_name(),
                                                         self.resource_id,
-                        self.unified_alert_action(),
+                                                        self.unified_alert_action())
                 @property
                 def reverse_types(self):
                     return dict([(v, k) for k, v in self.types.items()])
                 def unified_alert_name(self):
                     return self.reverse_types[self.event_type]
                 def unified_alert_action(self):
                     event_name = self.reverse_types[self.event_type]
-                    if self.status == Event.statuses["closed"]:
+                    if self.status == Event.statuses['closed']:
                         return "CLOSE"
-                    if self.status != Event.statuses["closed"]:
+                    if self.status != Event.statuses['closed']:
                         return "OPEN"
                     return event_name
                 def send_alerts(self, request=None, resource=None, db_session=None):
                     """" Sends alerts to applicable channels """
                     db_session = get_db_session(db_session)
                     db_session.flush()
                     if not resource:
-                        resource = ResourceService.by_resource_id(self.resource_id)
+                        resource = Resource.by_resource_id(self.resource_id)
                     if not request:
                         request = get_current_request()
                     if not resource:
                         return
-                    users = set([p.user for p in ResourceService.users_for_perm(resource, "view")])
+                    users = set([p.user for p in resource.users_for_perm('view')])
                     for user in users:
                         for channel in user.alert_channels:
-                            matches_resource = not channel.resources or resource in [
+                            matches_resource = not channel.resources or resource in [r.resource_id for r in channel.resources]
-                                r.resource_id for r in channel.resources
                             if (
-                                not channel.channel_validated
+                                not channel.channel_validated or
-                                or not channel.send_alerts
+                                not channel.send_alerts or
-                                or not matches_resource
+                                not matches_resource
                             ):
                                 continue
                             else:
                                 try:
-                                    channel.notify_alert(
+                                    channel.notify_alert(resource=resource,
-                                        resource=resource, event=self, user=user, request=request
+                                                         event=self,
+                                                         user=user,
+                                                         request=request)
                                 except IntegrationException as e:
-                                    log.warning("%s" % e)
+                                    log.warning('%s' % e)
                 def validate_or_close(self, since_when, db_session=None):
                     """ Checks if alerts should stay open or it's time to close them.
                         Generates close alert event if alerts get closed """
-                    event_types = [
+                    event_types = [Event.types['error_report_alert'],
-                        Event.types["error_report_alert"],
+                                   Event.types['slow_report_alert']]
-                        Event.types["slow_report_alert"],
+                    app = Resource.by_resource_id(self.resource_id)
-                    app = ResourceService.by_resource_id(self.resource_id)
-                    # if app was deleted close instantly
-                    if not app:
-                        self.close()
-                        return
                     if self.event_type in event_types:
                         total = ReportStatService.count_by_type(
-                            self.event_type, self.resource_id, since_when
+                            self.event_type, self.resource_id, since_when)
+                        if Event.types['error_report_alert'] == self.event_type:
-                        if Event.types["error_report_alert"] == self.event_type:
                             threshold = app.error_report_threshold
-                        if Event.types["slow_report_alert"] == self.event_type:
+                        if Event.types['slow_report_alert'] == self.event_type:
                             threshold = app.slow_report_threshold
                         if total < threshold:
                             self.close()
                 def close(self, db_session=None):
                     """
                     Closes an event and sends notification to affected users
                     """
                     self.end_date = datetime.utcnow()
-                    self.status = Event.statuses["closed"]
+                    self.status = Event.statuses['closed']
-                    log.warning("ALERT: CLOSE: %s" % self)
+                    log.warning('ALERT: CLOSE: %s' % self)
                     self.send_alerts()
                 def text_representation(self):
                     alert_type = self.unified_alert_name()
-                    text = ""
+                    text = ''
-                    if "slow_report" in alert_type:
+                    if 'slow_report' in alert_type:
-                        text += "Slow report alert"
+                        text += 'Slow report alert'
-                    if "error_report" in alert_type:
+                    if 'error_report' in alert_type:
-                        text += "Exception report alert"
+                        text += 'Exception report alert'
-                    if "uptime_alert" in alert_type:
+                    if 'uptime_alert' in alert_type:
-                        text += "Uptime alert"
+                        text += 'Uptime alert'
-                    if "chart_alert" in alert_type:
+                    if 'chart_alert' in alert_type:
-                        text += "Metrics value alert"
+                        text += 'Metrics value alert'
                     alert_action = self.unified_alert_action()
-                    if alert_action == "OPEN":
+                    if alert_action == 'OPEN':
-                        text += " got opened."
+                        text += ' got opened.'
-                    if alert_action == "CLOSE":
+                    if alert_action == 'CLOSE':
-                        text += " got closed."
+                        text += ' got closed.'
                     return text
                 def get_dict(self, request=None):
                     dict_data = super(Event, self).get_dict()
-                    dict_data["text"] = self.text_representation()
+                    dict_data['text'] = self.text_representation()
-                    dict_data["resource_name"] = self.resource.resource_name
+                    dict_data['resource_name'] = self.resource.resource_name
                     return dict_data

backend/src/appenlight/models/external_identity.py

0 +3 -3

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             import sqlalchemy as sa
             from sqlalchemy.ext.declarative import declared_attr
             from ziggurat_foundations.models.external_identity import ExternalIdentityMixin
             from appenlight.models import Base
             from appenlight.lib.sqlalchemy_fields import EncryptedUnicode
             class ExternalIdentity(ExternalIdentityMixin, Base):
                 @declared_attr
                 def access_token(self):
-                    return sa.Column(EncryptedUnicode(255), default="")
+                    return sa.Column(EncryptedUnicode(255), default='')
                 @declared_attr
                 def alt_token(self):
-                    return sa.Column(EncryptedUnicode(255), default="")
+                    return sa.Column(EncryptedUnicode(255), default='')
                 @declared_attr
                 def token_secret(self):
-                    return sa.Column(EncryptedUnicode(255), default="")
+                    return sa.Column(EncryptedUnicode(255), default='')

backend/src/appenlight/models/group.py

0 +14 -15

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
             from ziggurat_foundations.models.group import GroupMixin
             from appenlight.models import Base
             class Group(GroupMixin, Base):
-                __possible_permissions__ = (
+                __possible_permissions__ = ('root_administration',
-                    "root_administration",
+                                            'test_features',
-                    "test_features",
+                                            'admin_panel',
-                    "admin_panel",
+                                            'admin_users',
-                    "admin_users",
+                                            'manage_partitions',)
-                    "manage_partitions",
-                def get_dict(self, exclude_keys=None, include_keys=None, include_perms=False):
+                def get_dict(self, exclude_keys=None, include_keys=None,
+                             include_perms=False):
                     result = super(Group, self).get_dict(exclude_keys, include_keys)
                     if include_perms:
-                        result["possible_permissions"] = self.__possible_permissions__
+                        result['possible_permissions'] = self.__possible_permissions__
-                        result["current_permissions"] = [p.perm_name for p in self.permissions]
+                        result['current_permissions'] = [p.perm_name for p in
+                                                         self.permissions]
                     else:
-                        result["possible_permissions"] = []
+                        result['possible_permissions'] = []
-                        result["current_permissions"] = []
+                        result['current_permissions'] = []
                     exclude_keys_list = exclude_keys or []
                     include_keys_list = include_keys or []
                     d = {}
                     for k in result.keys():
-                        if k not in exclude_keys_list and (
+                        if (k not in exclude_keys_list and
-                            k in include_keys_list or not include_keys
+                                (k in include_keys_list or not include_keys)):
-                        ):
                             d[k] = result[k]
                     return d

backend/src/appenlight/models/group_resource_permission.py

0 +2 -3

             # -*- coding: utf-8 -*-
             # Copyright 2010 - 2017 RhodeCode GmbH and the AppEnlight project authors
             #
             # Licensed under the Apache License, Version 2.0 (the "License");
             # you may not use this file except in compliance with the License.
             # You may obtain a copy of the License at
             #
             #   http://www.apache.org/licenses/LICENSE-2.0
             #
             # Unless required by applicable law or agreed to in writing, software
             # distributed under the License is distributed on an "AS IS" BASIS,
             # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
             # See the License for the specific language governing permissions and
             # limitations under the License.
-            from ziggurat_foundations.models.group_resource_permission import (
+            from ziggurat_foundations.models.group_resource_permission import \
-                GroupResourcePermissionMixin,
+                GroupResourcePermissionMixin
             from appenlight.models import Base
             class GroupResourcePermission(GroupResourcePermissionMixin, Base):
                 pass

backend/src/appenlight/models/integrations/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/bitbucket.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/campfire.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/flowdock.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/github.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/hipchat.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/jira.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/slack.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/integrations/webhooks.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/log.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/metric.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/plugin_config.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/report.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/report_assignment.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/report_comment.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/report_group.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/report_stat.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/resource.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/application.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/application_postprocess_conf.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/config.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/event.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/group.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/group_resource_permission.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/log.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/plugin_config.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/report.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/report_group.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/report_stat.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/request_metric.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/slow_call.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/tag.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/services/user.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/slow_call.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/tag.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/user.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/models/user_resource_permission.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/predicates.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/cleanup.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/initialize_db.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/make_config.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/migratedb.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/reindex_elasticsearch.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/scripts/static.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/security.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/static/js/appenlight.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/subscribers.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/templates/ini/production.ini.jinja2

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/tests/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/tests/conftest.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/tests/payload_examples.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/tests/test_integration.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/tests/test_unit.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/validators.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/admin/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/admin/admin.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/admin/config.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/admin/partitions.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/admin/users.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/api.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/applications.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/events.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/groups.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/index.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/__init__.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/bitbucket.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/campfire.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/flowdock.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/github.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/hipchat.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/jira.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/slack.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/integrations/webhooks.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/logs.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/plugin_configs.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/reports.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/tests.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

backend/src/appenlight/views/user.py

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

development.ini

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/Gruntfile.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/bower.json

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/karma.conf.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/package.json

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/readme.rst

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/appenlight-footer.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/appenlight-header/appenlight-header.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-applications-list-view/admin-applications-list-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-configuration-view/admin-configuration-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-groups-create-view/admin-groups-create-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-groups-list-view/admin-groups-list-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-partitions-view/admin-partitions-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-system-view/admin-system-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-users-create-view/admin-users-create-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-users-list-view/admin-users-list-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/admin-view/admin-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/applications-integrations-view/applications-integrations-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/applications-list-view/applications-list-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/applications-purge-logs-view/applications-purge-logs-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/applications-update-view/applications-update-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/event-browser/event-browser.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/index-dashboard/index-dashboard.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/integrations/applications-integrations-edit-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/logs-browser/logs-browser.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/report-view/report-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/reports-browser-view/reports-browser-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/reports-slow-browser-view/reports-slow-browser-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/settings-view/settings-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-alert-channel-email-new-view/user-alert-channel-email-new-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-alert-channels-list-view/user-alert-channels-list-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-auth-tokens-view/user-auth-tokens-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-identities-view/user-identities-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-password-view/user-password-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/components/views/user-profile-view/user-profile-view.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/controllers/integrations/bitbucket.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/controllers/integrations/github.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/controllers/integrations/jira.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/controllers/reports/assign_report.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/permissions/permissions.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/plugin_config/plugin_config.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/postprocess_action/postprocess_action.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/report_alert_action/report_alert_action.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/rule/rule.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

frontend/src/directives/rule_read_only/rule_read_only.js

0 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

.travis.yml

0 removed 0 -45

NO CONTENT: file was removed

backend/CHANGELOG.md

0 removed 0 -28

NO CONTENT: file was removed

backend/README.md

0 removed 0 -112

NO CONTENT: file was removed

tox.ini

0 removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

General Comments 2

mahdish8

|

Auto status change to "Under Review"

marcink

|

Please use: https://github.com/Appenlight/appenlight to contribute :) Thanks !

No TODOs yet

		Please use: https://github.com/Appenlight/appenlight to contribute :) Thanks !
		Auto status change to \|new_status\|...

In pull request description:

No Ticket data found.

In commit messages:

#1 - in 1 commit

@@ -1,2 +1,2 b''
1	include .txt .ini .cfg .rst *.md ~~VERSION~~	1	include .txt .ini .cfg .rst *.md
2	recursive-include ~~src~~ .ico .png .css .gif .jpg .pt .txt .mak .mako .js .html .xml .jinja2 .rst .otf .ttf .svg .woff .woff2 .eot	2	recursive-include appenlight .ico .png .css .gif .jpg .pt .txt .mak .mako .js .html .xml .jinja2 .rst .otf .ttf .svg .woff *.eot

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages