##// END OF EJS Templates
configitems: register the 'bugzilla.fixresolution' config
Boris Feld -
r33400:76ca5097 default
parent child Browse files
Show More
@@ -1,1097 +1,1098
1 1 # bugzilla.py - bugzilla integration for mercurial
2 2 #
3 3 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
4 4 # Copyright 2011-4 Jim Hague <jim.hague@acm.org>
5 5 #
6 6 # This software may be used and distributed according to the terms of the
7 7 # GNU General Public License version 2 or any later version.
8 8
9 9 '''hooks for integrating with the Bugzilla bug tracker
10 10
11 11 This hook extension adds comments on bugs in Bugzilla when changesets
12 12 that refer to bugs by Bugzilla ID are seen. The comment is formatted using
13 13 the Mercurial template mechanism.
14 14
15 15 The bug references can optionally include an update for Bugzilla of the
16 16 hours spent working on the bug. Bugs can also be marked fixed.
17 17
18 18 Four basic modes of access to Bugzilla are provided:
19 19
20 20 1. Access via the Bugzilla REST-API. Requires bugzilla 5.0 or later.
21 21
22 22 2. Access via the Bugzilla XMLRPC interface. Requires Bugzilla 3.4 or later.
23 23
24 24 3. Check data via the Bugzilla XMLRPC interface and submit bug change
25 25 via email to Bugzilla email interface. Requires Bugzilla 3.4 or later.
26 26
27 27 4. Writing directly to the Bugzilla database. Only Bugzilla installations
28 28 using MySQL are supported. Requires Python MySQLdb.
29 29
30 30 Writing directly to the database is susceptible to schema changes, and
31 31 relies on a Bugzilla contrib script to send out bug change
32 32 notification emails. This script runs as the user running Mercurial,
33 33 must be run on the host with the Bugzilla install, and requires
34 34 permission to read Bugzilla configuration details and the necessary
35 35 MySQL user and password to have full access rights to the Bugzilla
36 36 database. For these reasons this access mode is now considered
37 37 deprecated, and will not be updated for new Bugzilla versions going
38 38 forward. Only adding comments is supported in this access mode.
39 39
40 40 Access via XMLRPC needs a Bugzilla username and password to be specified
41 41 in the configuration. Comments are added under that username. Since the
42 42 configuration must be readable by all Mercurial users, it is recommended
43 43 that the rights of that user are restricted in Bugzilla to the minimum
44 44 necessary to add comments. Marking bugs fixed requires Bugzilla 4.0 and later.
45 45
46 46 Access via XMLRPC/email uses XMLRPC to query Bugzilla, but sends
47 47 email to the Bugzilla email interface to submit comments to bugs.
48 48 The From: address in the email is set to the email address of the Mercurial
49 49 user, so the comment appears to come from the Mercurial user. In the event
50 50 that the Mercurial user email is not recognized by Bugzilla as a Bugzilla
51 51 user, the email associated with the Bugzilla username used to log into
52 52 Bugzilla is used instead as the source of the comment. Marking bugs fixed
53 53 works on all supported Bugzilla versions.
54 54
55 55 Access via the REST-API needs either a Bugzilla username and password
56 56 or an apikey specified in the configuration. Comments are made under
57 57 the given username or the user associated with the apikey in Bugzilla.
58 58
59 59 Configuration items common to all access modes:
60 60
61 61 bugzilla.version
62 62 The access type to use. Values recognized are:
63 63
64 64 :``restapi``: Bugzilla REST-API, Bugzilla 5.0 and later.
65 65 :``xmlrpc``: Bugzilla XMLRPC interface.
66 66 :``xmlrpc+email``: Bugzilla XMLRPC and email interfaces.
67 67 :``3.0``: MySQL access, Bugzilla 3.0 and later.
68 68 :``2.18``: MySQL access, Bugzilla 2.18 and up to but not
69 69 including 3.0.
70 70 :``2.16``: MySQL access, Bugzilla 2.16 and up to but not
71 71 including 2.18.
72 72
73 73 bugzilla.regexp
74 74 Regular expression to match bug IDs for update in changeset commit message.
75 75 It must contain one "()" named group ``<ids>`` containing the bug
76 76 IDs separated by non-digit characters. It may also contain
77 77 a named group ``<hours>`` with a floating-point number giving the
78 78 hours worked on the bug. If no named groups are present, the first
79 79 "()" group is assumed to contain the bug IDs, and work time is not
80 80 updated. The default expression matches ``Bug 1234``, ``Bug no. 1234``,
81 81 ``Bug number 1234``, ``Bugs 1234,5678``, ``Bug 1234 and 5678`` and
82 82 variations thereof, followed by an hours number prefixed by ``h`` or
83 83 ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.
84 84
85 85 bugzilla.fixregexp
86 86 Regular expression to match bug IDs for marking fixed in changeset
87 87 commit message. This must contain a "()" named group ``<ids>` containing
88 88 the bug IDs separated by non-digit characters. It may also contain
89 89 a named group ``<hours>`` with a floating-point number giving the
90 90 hours worked on the bug. If no named groups are present, the first
91 91 "()" group is assumed to contain the bug IDs, and work time is not
92 92 updated. The default expression matches ``Fixes 1234``, ``Fixes bug 1234``,
93 93 ``Fixes bugs 1234,5678``, ``Fixes 1234 and 5678`` and
94 94 variations thereof, followed by an hours number prefixed by ``h`` or
95 95 ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.
96 96
97 97 bugzilla.fixstatus
98 98 The status to set a bug to when marking fixed. Default ``RESOLVED``.
99 99
100 100 bugzilla.fixresolution
101 101 The resolution to set a bug to when marking fixed. Default ``FIXED``.
102 102
103 103 bugzilla.style
104 104 The style file to use when formatting comments.
105 105
106 106 bugzilla.template
107 107 Template to use when formatting comments. Overrides style if
108 108 specified. In addition to the usual Mercurial keywords, the
109 109 extension specifies:
110 110
111 111 :``{bug}``: The Bugzilla bug ID.
112 112 :``{root}``: The full pathname of the Mercurial repository.
113 113 :``{webroot}``: Stripped pathname of the Mercurial repository.
114 114 :``{hgweb}``: Base URL for browsing Mercurial repositories.
115 115
116 116 Default ``changeset {node|short} in repo {root} refers to bug
117 117 {bug}.\\ndetails:\\n\\t{desc|tabindent}``
118 118
119 119 bugzilla.strip
120 120 The number of path separator characters to strip from the front of
121 121 the Mercurial repository path (``{root}`` in templates) to produce
122 122 ``{webroot}``. For example, a repository with ``{root}``
123 123 ``/var/local/my-project`` with a strip of 2 gives a value for
124 124 ``{webroot}`` of ``my-project``. Default 0.
125 125
126 126 web.baseurl
127 127 Base URL for browsing Mercurial repositories. Referenced from
128 128 templates as ``{hgweb}``.
129 129
130 130 Configuration items common to XMLRPC+email and MySQL access modes:
131 131
132 132 bugzilla.usermap
133 133 Path of file containing Mercurial committer email to Bugzilla user email
134 134 mappings. If specified, the file should contain one mapping per
135 135 line::
136 136
137 137 committer = Bugzilla user
138 138
139 139 See also the ``[usermap]`` section.
140 140
141 141 The ``[usermap]`` section is used to specify mappings of Mercurial
142 142 committer email to Bugzilla user email. See also ``bugzilla.usermap``.
143 143 Contains entries of the form ``committer = Bugzilla user``.
144 144
145 145 XMLRPC and REST-API access mode configuration:
146 146
147 147 bugzilla.bzurl
148 148 The base URL for the Bugzilla installation.
149 149 Default ``http://localhost/bugzilla``.
150 150
151 151 bugzilla.user
152 152 The username to use to log into Bugzilla via XMLRPC. Default
153 153 ``bugs``.
154 154
155 155 bugzilla.password
156 156 The password for Bugzilla login.
157 157
158 158 REST-API access mode uses the options listed above as well as:
159 159
160 160 bugzilla.apikey
161 161 An apikey generated on the Bugzilla instance for api access.
162 162 Using an apikey removes the need to store the user and password
163 163 options.
164 164
165 165 XMLRPC+email access mode uses the XMLRPC access mode configuration items,
166 166 and also:
167 167
168 168 bugzilla.bzemail
169 169 The Bugzilla email address.
170 170
171 171 In addition, the Mercurial email settings must be configured. See the
172 172 documentation in hgrc(5), sections ``[email]`` and ``[smtp]``.
173 173
174 174 MySQL access mode configuration:
175 175
176 176 bugzilla.host
177 177 Hostname of the MySQL server holding the Bugzilla database.
178 178 Default ``localhost``.
179 179
180 180 bugzilla.db
181 181 Name of the Bugzilla database in MySQL. Default ``bugs``.
182 182
183 183 bugzilla.user
184 184 Username to use to access MySQL server. Default ``bugs``.
185 185
186 186 bugzilla.password
187 187 Password to use to access MySQL server.
188 188
189 189 bugzilla.timeout
190 190 Database connection timeout (seconds). Default 5.
191 191
192 192 bugzilla.bzuser
193 193 Fallback Bugzilla user name to record comments with, if changeset
194 194 committer cannot be found as a Bugzilla user.
195 195
196 196 bugzilla.bzdir
197 197 Bugzilla install directory. Used by default notify. Default
198 198 ``/var/www/html/bugzilla``.
199 199
200 200 bugzilla.notify
201 201 The command to run to get Bugzilla to send bug change notification
202 202 emails. Substitutes from a map with 3 keys, ``bzdir``, ``id`` (bug
203 203 id) and ``user`` (committer bugzilla email). Default depends on
204 204 version; from 2.18 it is "cd %(bzdir)s && perl -T
205 205 contrib/sendbugmail.pl %(id)s %(user)s".
206 206
207 207 Activating the extension::
208 208
209 209 [extensions]
210 210 bugzilla =
211 211
212 212 [hooks]
213 213 # run bugzilla hook on every change pulled or pushed in here
214 214 incoming.bugzilla = python:hgext.bugzilla.hook
215 215
216 216 Example configurations:
217 217
218 218 XMLRPC example configuration. This uses the Bugzilla at
219 219 ``http://my-project.org/bugzilla``, logging in as user
220 220 ``bugmail@my-project.org`` with password ``plugh``. It is used with a
221 221 collection of Mercurial repositories in ``/var/local/hg/repos/``,
222 222 with a web interface at ``http://my-project.org/hg``. ::
223 223
224 224 [bugzilla]
225 225 bzurl=http://my-project.org/bugzilla
226 226 user=bugmail@my-project.org
227 227 password=plugh
228 228 version=xmlrpc
229 229 template=Changeset {node|short} in {root|basename}.
230 230 {hgweb}/{webroot}/rev/{node|short}\\n
231 231 {desc}\\n
232 232 strip=5
233 233
234 234 [web]
235 235 baseurl=http://my-project.org/hg
236 236
237 237 XMLRPC+email example configuration. This uses the Bugzilla at
238 238 ``http://my-project.org/bugzilla``, logging in as user
239 239 ``bugmail@my-project.org`` with password ``plugh``. It is used with a
240 240 collection of Mercurial repositories in ``/var/local/hg/repos/``,
241 241 with a web interface at ``http://my-project.org/hg``. Bug comments
242 242 are sent to the Bugzilla email address
243 243 ``bugzilla@my-project.org``. ::
244 244
245 245 [bugzilla]
246 246 bzurl=http://my-project.org/bugzilla
247 247 user=bugmail@my-project.org
248 248 password=plugh
249 249 version=xmlrpc+email
250 250 bzemail=bugzilla@my-project.org
251 251 template=Changeset {node|short} in {root|basename}.
252 252 {hgweb}/{webroot}/rev/{node|short}\\n
253 253 {desc}\\n
254 254 strip=5
255 255
256 256 [web]
257 257 baseurl=http://my-project.org/hg
258 258
259 259 [usermap]
260 260 user@emaildomain.com=user.name@bugzilladomain.com
261 261
262 262 MySQL example configuration. This has a local Bugzilla 3.2 installation
263 263 in ``/opt/bugzilla-3.2``. The MySQL database is on ``localhost``,
264 264 the Bugzilla database name is ``bugs`` and MySQL is
265 265 accessed with MySQL username ``bugs`` password ``XYZZY``. It is used
266 266 with a collection of Mercurial repositories in ``/var/local/hg/repos/``,
267 267 with a web interface at ``http://my-project.org/hg``. ::
268 268
269 269 [bugzilla]
270 270 host=localhost
271 271 password=XYZZY
272 272 version=3.0
273 273 bzuser=unknown@domain.com
274 274 bzdir=/opt/bugzilla-3.2
275 275 template=Changeset {node|short} in {root|basename}.
276 276 {hgweb}/{webroot}/rev/{node|short}\\n
277 277 {desc}\\n
278 278 strip=5
279 279
280 280 [web]
281 281 baseurl=http://my-project.org/hg
282 282
283 283 [usermap]
284 284 user@emaildomain.com=user.name@bugzilladomain.com
285 285
286 286 All the above add a comment to the Bugzilla bug record of the form::
287 287
288 288 Changeset 3b16791d6642 in repository-name.
289 289 http://my-project.org/hg/repository-name/rev/3b16791d6642
290 290
291 291 Changeset commit comment. Bug 1234.
292 292 '''
293 293
294 294 from __future__ import absolute_import
295 295
296 296 import json
297 297 import re
298 298 import time
299 299
300 300 from mercurial.i18n import _
301 301 from mercurial.node import short
302 302 from mercurial import (
303 303 cmdutil,
304 304 error,
305 305 mail,
306 306 registrar,
307 307 url,
308 308 util,
309 309 )
310 310
311 311 xmlrpclib = util.xmlrpclib
312 312
313 313 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
314 314 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
315 315 # be specifying the version(s) of Mercurial they are tested with, or
316 316 # leave the attribute unspecified.
317 317 testedwith = 'ships-with-hg-core'
318 318
319 319 configtable = {}
320 320 configitem = registrar.configitem(configtable)
321 321
322 322 configitem('bugzilla', 'apikey',
323 323 default='',
324 324 )
325 325 configitem('bugzilla', 'bzdir',
326 326 default='/var/www/html/bugzilla',
327 327 )
328 328 configitem('bugzilla', 'bzemail',
329 329 default=None,
330 330 )
331 331 configitem('bugzilla', 'bzurl',
332 332 default='http://localhost/bugzilla/',
333 333 )
334 334 configitem('bugzilla', 'bzuser',
335 335 default=None,
336 336 )
337 337 configitem('bugzilla', 'db',
338 338 default='bugs',
339 339 )
340 340 configitem('bugzilla', 'fixregexp',
341 341 default=lambda: bugzilla._default_fix_re,
342 342 )
343 configitem('bugzilla', 'fixresolution',
344 default='FIXED',
345 )
343 346
344 347 class bzaccess(object):
345 348 '''Base class for access to Bugzilla.'''
346 349
347 350 def __init__(self, ui):
348 351 self.ui = ui
349 352 usermap = self.ui.config('bugzilla', 'usermap')
350 353 if usermap:
351 354 self.ui.readconfig(usermap, sections=['usermap'])
352 355
353 356 def map_committer(self, user):
354 357 '''map name of committer to Bugzilla user name.'''
355 358 for committer, bzuser in self.ui.configitems('usermap'):
356 359 if committer.lower() == user.lower():
357 360 return bzuser
358 361 return user
359 362
360 363 # Methods to be implemented by access classes.
361 364 #
362 365 # 'bugs' is a dict keyed on bug id, where values are a dict holding
363 366 # updates to bug state. Recognized dict keys are:
364 367 #
365 368 # 'hours': Value, float containing work hours to be updated.
366 369 # 'fix': If key present, bug is to be marked fixed. Value ignored.
367 370
368 371 def filter_real_bug_ids(self, bugs):
369 372 '''remove bug IDs that do not exist in Bugzilla from bugs.'''
370 373 pass
371 374
372 375 def filter_cset_known_bug_ids(self, node, bugs):
373 376 '''remove bug IDs where node occurs in comment text from bugs.'''
374 377 pass
375 378
376 379 def updatebug(self, bugid, newstate, text, committer):
377 380 '''update the specified bug. Add comment text and set new states.
378 381
379 382 If possible add the comment as being from the committer of
380 383 the changeset. Otherwise use the default Bugzilla user.
381 384 '''
382 385 pass
383 386
384 387 def notify(self, bugs, committer):
385 388 '''Force sending of Bugzilla notification emails.
386 389
387 390 Only required if the access method does not trigger notification
388 391 emails automatically.
389 392 '''
390 393 pass
391 394
392 395 # Bugzilla via direct access to MySQL database.
393 396 class bzmysql(bzaccess):
394 397 '''Support for direct MySQL access to Bugzilla.
395 398
396 399 The earliest Bugzilla version this is tested with is version 2.16.
397 400
398 401 If your Bugzilla is version 3.4 or above, you are strongly
399 402 recommended to use the XMLRPC access method instead.
400 403 '''
401 404
402 405 @staticmethod
403 406 def sql_buglist(ids):
404 407 '''return SQL-friendly list of bug ids'''
405 408 return '(' + ','.join(map(str, ids)) + ')'
406 409
407 410 _MySQLdb = None
408 411
409 412 def __init__(self, ui):
410 413 try:
411 414 import MySQLdb as mysql
412 415 bzmysql._MySQLdb = mysql
413 416 except ImportError as err:
414 417 raise error.Abort(_('python mysql support not available: %s') % err)
415 418
416 419 bzaccess.__init__(self, ui)
417 420
418 421 host = self.ui.config('bugzilla', 'host', 'localhost')
419 422 user = self.ui.config('bugzilla', 'user', 'bugs')
420 423 passwd = self.ui.config('bugzilla', 'password')
421 424 db = self.ui.config('bugzilla', 'db')
422 425 timeout = int(self.ui.config('bugzilla', 'timeout', 5))
423 426 self.ui.note(_('connecting to %s:%s as %s, password %s\n') %
424 427 (host, db, user, '*' * len(passwd)))
425 428 self.conn = bzmysql._MySQLdb.connect(host=host,
426 429 user=user, passwd=passwd,
427 430 db=db,
428 431 connect_timeout=timeout)
429 432 self.cursor = self.conn.cursor()
430 433 self.longdesc_id = self.get_longdesc_id()
431 434 self.user_ids = {}
432 435 self.default_notify = "cd %(bzdir)s && ./processmail %(id)s %(user)s"
433 436
434 437 def run(self, *args, **kwargs):
435 438 '''run a query.'''
436 439 self.ui.note(_('query: %s %s\n') % (args, kwargs))
437 440 try:
438 441 self.cursor.execute(*args, **kwargs)
439 442 except bzmysql._MySQLdb.MySQLError:
440 443 self.ui.note(_('failed query: %s %s\n') % (args, kwargs))
441 444 raise
442 445
443 446 def get_longdesc_id(self):
444 447 '''get identity of longdesc field'''
445 448 self.run('select fieldid from fielddefs where name = "longdesc"')
446 449 ids = self.cursor.fetchall()
447 450 if len(ids) != 1:
448 451 raise error.Abort(_('unknown database schema'))
449 452 return ids[0][0]
450 453
451 454 def filter_real_bug_ids(self, bugs):
452 455 '''filter not-existing bugs from set.'''
453 456 self.run('select bug_id from bugs where bug_id in %s' %
454 457 bzmysql.sql_buglist(bugs.keys()))
455 458 existing = [id for (id,) in self.cursor.fetchall()]
456 459 for id in bugs.keys():
457 460 if id not in existing:
458 461 self.ui.status(_('bug %d does not exist\n') % id)
459 462 del bugs[id]
460 463
461 464 def filter_cset_known_bug_ids(self, node, bugs):
462 465 '''filter bug ids that already refer to this changeset from set.'''
463 466 self.run('''select bug_id from longdescs where
464 467 bug_id in %s and thetext like "%%%s%%"''' %
465 468 (bzmysql.sql_buglist(bugs.keys()), short(node)))
466 469 for (id,) in self.cursor.fetchall():
467 470 self.ui.status(_('bug %d already knows about changeset %s\n') %
468 471 (id, short(node)))
469 472 del bugs[id]
470 473
471 474 def notify(self, bugs, committer):
472 475 '''tell bugzilla to send mail.'''
473 476 self.ui.status(_('telling bugzilla to send mail:\n'))
474 477 (user, userid) = self.get_bugzilla_user(committer)
475 478 for id in bugs.keys():
476 479 self.ui.status(_(' bug %s\n') % id)
477 480 cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)
478 481 bzdir = self.ui.config('bugzilla', 'bzdir')
479 482 try:
480 483 # Backwards-compatible with old notify string, which
481 484 # took one string. This will throw with a new format
482 485 # string.
483 486 cmd = cmdfmt % id
484 487 except TypeError:
485 488 cmd = cmdfmt % {'bzdir': bzdir, 'id': id, 'user': user}
486 489 self.ui.note(_('running notify command %s\n') % cmd)
487 490 fp = util.popen('(%s) 2>&1' % cmd)
488 491 out = fp.read()
489 492 ret = fp.close()
490 493 if ret:
491 494 self.ui.warn(out)
492 495 raise error.Abort(_('bugzilla notify command %s') %
493 496 util.explainexit(ret)[0])
494 497 self.ui.status(_('done\n'))
495 498
496 499 def get_user_id(self, user):
497 500 '''look up numeric bugzilla user id.'''
498 501 try:
499 502 return self.user_ids[user]
500 503 except KeyError:
501 504 try:
502 505 userid = int(user)
503 506 except ValueError:
504 507 self.ui.note(_('looking up user %s\n') % user)
505 508 self.run('''select userid from profiles
506 509 where login_name like %s''', user)
507 510 all = self.cursor.fetchall()
508 511 if len(all) != 1:
509 512 raise KeyError(user)
510 513 userid = int(all[0][0])
511 514 self.user_ids[user] = userid
512 515 return userid
513 516
514 517 def get_bugzilla_user(self, committer):
515 518 '''See if committer is a registered bugzilla user. Return
516 519 bugzilla username and userid if so. If not, return default
517 520 bugzilla username and userid.'''
518 521 user = self.map_committer(committer)
519 522 try:
520 523 userid = self.get_user_id(user)
521 524 except KeyError:
522 525 try:
523 526 defaultuser = self.ui.config('bugzilla', 'bzuser')
524 527 if not defaultuser:
525 528 raise error.Abort(_('cannot find bugzilla user id for %s') %
526 529 user)
527 530 userid = self.get_user_id(defaultuser)
528 531 user = defaultuser
529 532 except KeyError:
530 533 raise error.Abort(_('cannot find bugzilla user id for %s or %s')
531 534 % (user, defaultuser))
532 535 return (user, userid)
533 536
534 537 def updatebug(self, bugid, newstate, text, committer):
535 538 '''update bug state with comment text.
536 539
537 540 Try adding comment as committer of changeset, otherwise as
538 541 default bugzilla user.'''
539 542 if len(newstate) > 0:
540 543 self.ui.warn(_("Bugzilla/MySQL cannot update bug state\n"))
541 544
542 545 (user, userid) = self.get_bugzilla_user(committer)
543 546 now = time.strftime('%Y-%m-%d %H:%M:%S')
544 547 self.run('''insert into longdescs
545 548 (bug_id, who, bug_when, thetext)
546 549 values (%s, %s, %s, %s)''',
547 550 (bugid, userid, now, text))
548 551 self.run('''insert into bugs_activity (bug_id, who, bug_when, fieldid)
549 552 values (%s, %s, %s, %s)''',
550 553 (bugid, userid, now, self.longdesc_id))
551 554 self.conn.commit()
552 555
553 556 class bzmysql_2_18(bzmysql):
554 557 '''support for bugzilla 2.18 series.'''
555 558
556 559 def __init__(self, ui):
557 560 bzmysql.__init__(self, ui)
558 561 self.default_notify = \
559 562 "cd %(bzdir)s && perl -T contrib/sendbugmail.pl %(id)s %(user)s"
560 563
561 564 class bzmysql_3_0(bzmysql_2_18):
562 565 '''support for bugzilla 3.0 series.'''
563 566
564 567 def __init__(self, ui):
565 568 bzmysql_2_18.__init__(self, ui)
566 569
567 570 def get_longdesc_id(self):
568 571 '''get identity of longdesc field'''
569 572 self.run('select id from fielddefs where name = "longdesc"')
570 573 ids = self.cursor.fetchall()
571 574 if len(ids) != 1:
572 575 raise error.Abort(_('unknown database schema'))
573 576 return ids[0][0]
574 577
575 578 # Bugzilla via XMLRPC interface.
576 579
577 580 class cookietransportrequest(object):
578 581 """A Transport request method that retains cookies over its lifetime.
579 582
580 583 The regular xmlrpclib transports ignore cookies. Which causes
581 584 a bit of a problem when you need a cookie-based login, as with
582 585 the Bugzilla XMLRPC interface prior to 4.4.3.
583 586
584 587 So this is a helper for defining a Transport which looks for
585 588 cookies being set in responses and saves them to add to all future
586 589 requests.
587 590 """
588 591
589 592 # Inspiration drawn from
590 593 # http://blog.godson.in/2010/09/how-to-make-python-xmlrpclib-client.html
591 594 # http://www.itkovian.net/base/transport-class-for-pythons-xml-rpc-lib/
592 595
593 596 cookies = []
594 597 def send_cookies(self, connection):
595 598 if self.cookies:
596 599 for cookie in self.cookies:
597 600 connection.putheader("Cookie", cookie)
598 601
599 602 def request(self, host, handler, request_body, verbose=0):
600 603 self.verbose = verbose
601 604 self.accept_gzip_encoding = False
602 605
603 606 # issue XML-RPC request
604 607 h = self.make_connection(host)
605 608 if verbose:
606 609 h.set_debuglevel(1)
607 610
608 611 self.send_request(h, handler, request_body)
609 612 self.send_host(h, host)
610 613 self.send_cookies(h)
611 614 self.send_user_agent(h)
612 615 self.send_content(h, request_body)
613 616
614 617 # Deal with differences between Python 2.6 and 2.7.
615 618 # In the former h is a HTTP(S). In the latter it's a
616 619 # HTTP(S)Connection. Luckily, the 2.6 implementation of
617 620 # HTTP(S) has an underlying HTTP(S)Connection, so extract
618 621 # that and use it.
619 622 try:
620 623 response = h.getresponse()
621 624 except AttributeError:
622 625 response = h._conn.getresponse()
623 626
624 627 # Add any cookie definitions to our list.
625 628 for header in response.msg.getallmatchingheaders("Set-Cookie"):
626 629 val = header.split(": ", 1)[1]
627 630 cookie = val.split(";", 1)[0]
628 631 self.cookies.append(cookie)
629 632
630 633 if response.status != 200:
631 634 raise xmlrpclib.ProtocolError(host + handler, response.status,
632 635 response.reason, response.msg.headers)
633 636
634 637 payload = response.read()
635 638 parser, unmarshaller = self.getparser()
636 639 parser.feed(payload)
637 640 parser.close()
638 641
639 642 return unmarshaller.close()
640 643
641 644 # The explicit calls to the underlying xmlrpclib __init__() methods are
642 645 # necessary. The xmlrpclib.Transport classes are old-style classes, and
643 646 # it turns out their __init__() doesn't get called when doing multiple
644 647 # inheritance with a new-style class.
645 648 class cookietransport(cookietransportrequest, xmlrpclib.Transport):
646 649 def __init__(self, use_datetime=0):
647 650 if util.safehasattr(xmlrpclib.Transport, "__init__"):
648 651 xmlrpclib.Transport.__init__(self, use_datetime)
649 652
650 653 class cookiesafetransport(cookietransportrequest, xmlrpclib.SafeTransport):
651 654 def __init__(self, use_datetime=0):
652 655 if util.safehasattr(xmlrpclib.Transport, "__init__"):
653 656 xmlrpclib.SafeTransport.__init__(self, use_datetime)
654 657
655 658 class bzxmlrpc(bzaccess):
656 659 """Support for access to Bugzilla via the Bugzilla XMLRPC API.
657 660
658 661 Requires a minimum Bugzilla version 3.4.
659 662 """
660 663
661 664 def __init__(self, ui):
662 665 bzaccess.__init__(self, ui)
663 666
664 667 bzweb = self.ui.config('bugzilla', 'bzurl')
665 668 bzweb = bzweb.rstrip("/") + "/xmlrpc.cgi"
666 669
667 670 user = self.ui.config('bugzilla', 'user', 'bugs')
668 671 passwd = self.ui.config('bugzilla', 'password')
669 672
670 673 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
671 self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
672 'FIXED')
674 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
673 675
674 676 self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
675 677 ver = self.bzproxy.Bugzilla.version()['version'].split('.')
676 678 self.bzvermajor = int(ver[0])
677 679 self.bzverminor = int(ver[1])
678 680 login = self.bzproxy.User.login({'login': user, 'password': passwd,
679 681 'restrict_login': True})
680 682 self.bztoken = login.get('token', '')
681 683
682 684 def transport(self, uri):
683 685 if util.urlreq.urlparse(uri, "http")[0] == "https":
684 686 return cookiesafetransport()
685 687 else:
686 688 return cookietransport()
687 689
688 690 def get_bug_comments(self, id):
689 691 """Return a string with all comment text for a bug."""
690 692 c = self.bzproxy.Bug.comments({'ids': [id],
691 693 'include_fields': ['text'],
692 694 'token': self.bztoken})
693 695 return ''.join([t['text'] for t in c['bugs'][str(id)]['comments']])
694 696
695 697 def filter_real_bug_ids(self, bugs):
696 698 probe = self.bzproxy.Bug.get({'ids': sorted(bugs.keys()),
697 699 'include_fields': [],
698 700 'permissive': True,
699 701 'token': self.bztoken,
700 702 })
701 703 for badbug in probe['faults']:
702 704 id = badbug['id']
703 705 self.ui.status(_('bug %d does not exist\n') % id)
704 706 del bugs[id]
705 707
706 708 def filter_cset_known_bug_ids(self, node, bugs):
707 709 for id in sorted(bugs.keys()):
708 710 if self.get_bug_comments(id).find(short(node)) != -1:
709 711 self.ui.status(_('bug %d already knows about changeset %s\n') %
710 712 (id, short(node)))
711 713 del bugs[id]
712 714
713 715 def updatebug(self, bugid, newstate, text, committer):
714 716 args = {}
715 717 if 'hours' in newstate:
716 718 args['work_time'] = newstate['hours']
717 719
718 720 if self.bzvermajor >= 4:
719 721 args['ids'] = [bugid]
720 722 args['comment'] = {'body' : text}
721 723 if 'fix' in newstate:
722 724 args['status'] = self.fixstatus
723 725 args['resolution'] = self.fixresolution
724 726 args['token'] = self.bztoken
725 727 self.bzproxy.Bug.update(args)
726 728 else:
727 729 if 'fix' in newstate:
728 730 self.ui.warn(_("Bugzilla/XMLRPC needs Bugzilla 4.0 or later "
729 731 "to mark bugs fixed\n"))
730 732 args['id'] = bugid
731 733 args['comment'] = text
732 734 self.bzproxy.Bug.add_comment(args)
733 735
734 736 class bzxmlrpcemail(bzxmlrpc):
735 737 """Read data from Bugzilla via XMLRPC, send updates via email.
736 738
737 739 Advantages of sending updates via email:
738 740 1. Comments can be added as any user, not just logged in user.
739 741 2. Bug statuses or other fields not accessible via XMLRPC can
740 742 potentially be updated.
741 743
742 744 There is no XMLRPC function to change bug status before Bugzilla
743 745 4.0, so bugs cannot be marked fixed via XMLRPC before Bugzilla 4.0.
744 746 But bugs can be marked fixed via email from 3.4 onwards.
745 747 """
746 748
747 749 # The email interface changes subtly between 3.4 and 3.6. In 3.4,
748 750 # in-email fields are specified as '@<fieldname> = <value>'. In
749 751 # 3.6 this becomes '@<fieldname> <value>'. And fieldname @bug_id
750 752 # in 3.4 becomes @id in 3.6. 3.6 and 4.0 both maintain backwards
751 753 # compatibility, but rather than rely on this use the new format for
752 754 # 4.0 onwards.
753 755
754 756 def __init__(self, ui):
755 757 bzxmlrpc.__init__(self, ui)
756 758
757 759 self.bzemail = self.ui.config('bugzilla', 'bzemail')
758 760 if not self.bzemail:
759 761 raise error.Abort(_("configuration 'bzemail' missing"))
760 762 mail.validateconfig(self.ui)
761 763
762 764 def makecommandline(self, fieldname, value):
763 765 if self.bzvermajor >= 4:
764 766 return "@%s %s" % (fieldname, str(value))
765 767 else:
766 768 if fieldname == "id":
767 769 fieldname = "bug_id"
768 770 return "@%s = %s" % (fieldname, str(value))
769 771
770 772 def send_bug_modify_email(self, bugid, commands, comment, committer):
771 773 '''send modification message to Bugzilla bug via email.
772 774
773 775 The message format is documented in the Bugzilla email_in.pl
774 776 specification. commands is a list of command lines, comment is the
775 777 comment text.
776 778
777 779 To stop users from crafting commit comments with
778 780 Bugzilla commands, specify the bug ID via the message body, rather
779 781 than the subject line, and leave a blank line after it.
780 782 '''
781 783 user = self.map_committer(committer)
782 784 matches = self.bzproxy.User.get({'match': [user],
783 785 'token': self.bztoken})
784 786 if not matches['users']:
785 787 user = self.ui.config('bugzilla', 'user', 'bugs')
786 788 matches = self.bzproxy.User.get({'match': [user],
787 789 'token': self.bztoken})
788 790 if not matches['users']:
789 791 raise error.Abort(_("default bugzilla user %s email not found")
790 792 % user)
791 793 user = matches['users'][0]['email']
792 794 commands.append(self.makecommandline("id", bugid))
793 795
794 796 text = "\n".join(commands) + "\n\n" + comment
795 797
796 798 _charsets = mail._charsets(self.ui)
797 799 user = mail.addressencode(self.ui, user, _charsets)
798 800 bzemail = mail.addressencode(self.ui, self.bzemail, _charsets)
799 801 msg = mail.mimeencode(self.ui, text, _charsets)
800 802 msg['From'] = user
801 803 msg['To'] = bzemail
802 804 msg['Subject'] = mail.headencode(self.ui, "Bug modification", _charsets)
803 805 sendmail = mail.connect(self.ui)
804 806 sendmail(user, bzemail, msg.as_string())
805 807
806 808 def updatebug(self, bugid, newstate, text, committer):
807 809 cmds = []
808 810 if 'hours' in newstate:
809 811 cmds.append(self.makecommandline("work_time", newstate['hours']))
810 812 if 'fix' in newstate:
811 813 cmds.append(self.makecommandline("bug_status", self.fixstatus))
812 814 cmds.append(self.makecommandline("resolution", self.fixresolution))
813 815 self.send_bug_modify_email(bugid, cmds, text, committer)
814 816
815 817 class NotFound(LookupError):
816 818 pass
817 819
818 820 class bzrestapi(bzaccess):
819 821 """Read and write bugzilla data using the REST API available since
820 822 Bugzilla 5.0.
821 823 """
822 824 def __init__(self, ui):
823 825 bzaccess.__init__(self, ui)
824 826 bz = self.ui.config('bugzilla', 'bzurl')
825 827 self.bzroot = '/'.join([bz, 'rest'])
826 828 self.apikey = self.ui.config('bugzilla', 'apikey')
827 829 self.user = self.ui.config('bugzilla', 'user', 'bugs')
828 830 self.passwd = self.ui.config('bugzilla', 'password')
829 831 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
830 self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
831 'FIXED')
832 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
832 833
833 834 def apiurl(self, targets, include_fields=None):
834 835 url = '/'.join([self.bzroot] + [str(t) for t in targets])
835 836 qv = {}
836 837 if self.apikey:
837 838 qv['api_key'] = self.apikey
838 839 elif self.user and self.passwd:
839 840 qv['login'] = self.user
840 841 qv['password'] = self.passwd
841 842 if include_fields:
842 843 qv['include_fields'] = include_fields
843 844 if qv:
844 845 url = '%s?%s' % (url, util.urlreq.urlencode(qv))
845 846 return url
846 847
847 848 def _fetch(self, burl):
848 849 try:
849 850 resp = url.open(self.ui, burl)
850 851 return json.loads(resp.read())
851 852 except util.urlerr.httperror as inst:
852 853 if inst.code == 401:
853 854 raise error.Abort(_('authorization failed'))
854 855 if inst.code == 404:
855 856 raise NotFound()
856 857 else:
857 858 raise
858 859
859 860 def _submit(self, burl, data, method='POST'):
860 861 data = json.dumps(data)
861 862 if method == 'PUT':
862 863 class putrequest(util.urlreq.request):
863 864 def get_method(self):
864 865 return 'PUT'
865 866 request_type = putrequest
866 867 else:
867 868 request_type = util.urlreq.request
868 869 req = request_type(burl, data,
869 870 {'Content-Type': 'application/json'})
870 871 try:
871 872 resp = url.opener(self.ui).open(req)
872 873 return json.loads(resp.read())
873 874 except util.urlerr.httperror as inst:
874 875 if inst.code == 401:
875 876 raise error.Abort(_('authorization failed'))
876 877 if inst.code == 404:
877 878 raise NotFound()
878 879 else:
879 880 raise
880 881
881 882 def filter_real_bug_ids(self, bugs):
882 883 '''remove bug IDs that do not exist in Bugzilla from bugs.'''
883 884 badbugs = set()
884 885 for bugid in bugs:
885 886 burl = self.apiurl(('bug', bugid), include_fields='status')
886 887 try:
887 888 self._fetch(burl)
888 889 except NotFound:
889 890 badbugs.add(bugid)
890 891 for bugid in badbugs:
891 892 del bugs[bugid]
892 893
893 894 def filter_cset_known_bug_ids(self, node, bugs):
894 895 '''remove bug IDs where node occurs in comment text from bugs.'''
895 896 sn = short(node)
896 897 for bugid in bugs.keys():
897 898 burl = self.apiurl(('bug', bugid, 'comment'), include_fields='text')
898 899 result = self._fetch(burl)
899 900 comments = result['bugs'][str(bugid)]['comments']
900 901 if any(sn in c['text'] for c in comments):
901 902 self.ui.status(_('bug %d already knows about changeset %s\n') %
902 903 (bugid, sn))
903 904 del bugs[bugid]
904 905
905 906 def updatebug(self, bugid, newstate, text, committer):
906 907 '''update the specified bug. Add comment text and set new states.
907 908
908 909 If possible add the comment as being from the committer of
909 910 the changeset. Otherwise use the default Bugzilla user.
910 911 '''
911 912 bugmod = {}
912 913 if 'hours' in newstate:
913 914 bugmod['work_time'] = newstate['hours']
914 915 if 'fix' in newstate:
915 916 bugmod['status'] = self.fixstatus
916 917 bugmod['resolution'] = self.fixresolution
917 918 if bugmod:
918 919 # if we have to change the bugs state do it here
919 920 bugmod['comment'] = {
920 921 'comment': text,
921 922 'is_private': False,
922 923 'is_markdown': False,
923 924 }
924 925 burl = self.apiurl(('bug', bugid))
925 926 self._submit(burl, bugmod, method='PUT')
926 927 self.ui.debug('updated bug %s\n' % bugid)
927 928 else:
928 929 burl = self.apiurl(('bug', bugid, 'comment'))
929 930 self._submit(burl, {
930 931 'comment': text,
931 932 'is_private': False,
932 933 'is_markdown': False,
933 934 })
934 935 self.ui.debug('added comment to bug %s\n' % bugid)
935 936
936 937 def notify(self, bugs, committer):
937 938 '''Force sending of Bugzilla notification emails.
938 939
939 940 Only required if the access method does not trigger notification
940 941 emails automatically.
941 942 '''
942 943 pass
943 944
944 945 class bugzilla(object):
945 946 # supported versions of bugzilla. different versions have
946 947 # different schemas.
947 948 _versions = {
948 949 '2.16': bzmysql,
949 950 '2.18': bzmysql_2_18,
950 951 '3.0': bzmysql_3_0,
951 952 'xmlrpc': bzxmlrpc,
952 953 'xmlrpc+email': bzxmlrpcemail,
953 954 'restapi': bzrestapi,
954 955 }
955 956
956 957 _default_bug_re = (r'bugs?\s*,?\s*(?:#|nos?\.?|num(?:ber)?s?)?\s*'
957 958 r'(?P<ids>(?:\d+\s*(?:,?\s*(?:and)?)?\s*)+)'
958 959 r'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')
959 960
960 961 _default_fix_re = (r'fix(?:es)?\s*(?:bugs?\s*)?,?\s*'
961 962 r'(?:nos?\.?|num(?:ber)?s?)?\s*'
962 963 r'(?P<ids>(?:#?\d+\s*(?:,?\s*(?:and)?)?\s*)+)'
963 964 r'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')
964 965
965 966 def __init__(self, ui, repo):
966 967 self.ui = ui
967 968 self.repo = repo
968 969
969 970 bzversion = self.ui.config('bugzilla', 'version')
970 971 try:
971 972 bzclass = bugzilla._versions[bzversion]
972 973 except KeyError:
973 974 raise error.Abort(_('bugzilla version %s not supported') %
974 975 bzversion)
975 976 self.bzdriver = bzclass(self.ui)
976 977
977 978 self.bug_re = re.compile(
978 979 self.ui.config('bugzilla', 'regexp',
979 980 bugzilla._default_bug_re), re.IGNORECASE)
980 981 self.fix_re = re.compile(
981 982 self.ui.config('bugzilla', 'fixregexp'), re.IGNORECASE)
982 983 self.split_re = re.compile(r'\D+')
983 984
984 985 def find_bugs(self, ctx):
985 986 '''return bugs dictionary created from commit comment.
986 987
987 988 Extract bug info from changeset comments. Filter out any that are
988 989 not known to Bugzilla, and any that already have a reference to
989 990 the given changeset in their comments.
990 991 '''
991 992 start = 0
992 993 hours = 0.0
993 994 bugs = {}
994 995 bugmatch = self.bug_re.search(ctx.description(), start)
995 996 fixmatch = self.fix_re.search(ctx.description(), start)
996 997 while True:
997 998 bugattribs = {}
998 999 if not bugmatch and not fixmatch:
999 1000 break
1000 1001 if not bugmatch:
1001 1002 m = fixmatch
1002 1003 elif not fixmatch:
1003 1004 m = bugmatch
1004 1005 else:
1005 1006 if bugmatch.start() < fixmatch.start():
1006 1007 m = bugmatch
1007 1008 else:
1008 1009 m = fixmatch
1009 1010 start = m.end()
1010 1011 if m is bugmatch:
1011 1012 bugmatch = self.bug_re.search(ctx.description(), start)
1012 1013 if 'fix' in bugattribs:
1013 1014 del bugattribs['fix']
1014 1015 else:
1015 1016 fixmatch = self.fix_re.search(ctx.description(), start)
1016 1017 bugattribs['fix'] = None
1017 1018
1018 1019 try:
1019 1020 ids = m.group('ids')
1020 1021 except IndexError:
1021 1022 ids = m.group(1)
1022 1023 try:
1023 1024 hours = float(m.group('hours'))
1024 1025 bugattribs['hours'] = hours
1025 1026 except IndexError:
1026 1027 pass
1027 1028 except TypeError:
1028 1029 pass
1029 1030 except ValueError:
1030 1031 self.ui.status(_("%s: invalid hours\n") % m.group('hours'))
1031 1032
1032 1033 for id in self.split_re.split(ids):
1033 1034 if not id:
1034 1035 continue
1035 1036 bugs[int(id)] = bugattribs
1036 1037 if bugs:
1037 1038 self.bzdriver.filter_real_bug_ids(bugs)
1038 1039 if bugs:
1039 1040 self.bzdriver.filter_cset_known_bug_ids(ctx.node(), bugs)
1040 1041 return bugs
1041 1042
1042 1043 def update(self, bugid, newstate, ctx):
1043 1044 '''update bugzilla bug with reference to changeset.'''
1044 1045
1045 1046 def webroot(root):
1046 1047 '''strip leading prefix of repo root and turn into
1047 1048 url-safe path.'''
1048 1049 count = int(self.ui.config('bugzilla', 'strip', 0))
1049 1050 root = util.pconvert(root)
1050 1051 while count > 0:
1051 1052 c = root.find('/')
1052 1053 if c == -1:
1053 1054 break
1054 1055 root = root[c + 1:]
1055 1056 count -= 1
1056 1057 return root
1057 1058
1058 1059 mapfile = None
1059 1060 tmpl = self.ui.config('bugzilla', 'template')
1060 1061 if not tmpl:
1061 1062 mapfile = self.ui.config('bugzilla', 'style')
1062 1063 if not mapfile and not tmpl:
1063 1064 tmpl = _('changeset {node|short} in repo {root} refers '
1064 1065 'to bug {bug}.\ndetails:\n\t{desc|tabindent}')
1065 1066 spec = cmdutil.logtemplatespec(tmpl, mapfile)
1066 1067 t = cmdutil.changeset_templater(self.ui, self.repo, spec,
1067 1068 False, None, False)
1068 1069 self.ui.pushbuffer()
1069 1070 t.show(ctx, changes=ctx.changeset(),
1070 1071 bug=str(bugid),
1071 1072 hgweb=self.ui.config('web', 'baseurl'),
1072 1073 root=self.repo.root,
1073 1074 webroot=webroot(self.repo.root))
1074 1075 data = self.ui.popbuffer()
1075 1076 self.bzdriver.updatebug(bugid, newstate, data, util.email(ctx.user()))
1076 1077
1077 1078 def notify(self, bugs, committer):
1078 1079 '''ensure Bugzilla users are notified of bug change.'''
1079 1080 self.bzdriver.notify(bugs, committer)
1080 1081
1081 1082 def hook(ui, repo, hooktype, node=None, **kwargs):
1082 1083 '''add comment to bugzilla for each changeset that refers to a
1083 1084 bugzilla bug id. only add a comment once per bug, so same change
1084 1085 seen multiple times does not fill bug with duplicate data.'''
1085 1086 if node is None:
1086 1087 raise error.Abort(_('hook type %s does not pass a changeset id') %
1087 1088 hooktype)
1088 1089 try:
1089 1090 bz = bugzilla(ui, repo)
1090 1091 ctx = repo[node]
1091 1092 bugs = bz.find_bugs(ctx)
1092 1093 if bugs:
1093 1094 for bug in bugs:
1094 1095 bz.update(bug, bugs[bug], ctx)
1095 1096 bz.notify(bugs, util.email(ctx.user()))
1096 1097 except Exception as e:
1097 1098 raise error.Abort(_('Bugzilla error: %s') % e)
General Comments 0
You need to be logged in to leave comments. Login now