Bitten-0.6/ 000755 000765 000120 00000000000 11536424600 013057 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/ 000755 000765 000120 00000000000 11536424600 014344 5 ustar 00simon admin 000000 000000 Bitten-0.6/Bitten.egg-info/ 000755 000765 000120 00000000000 11536424600 015776 5 ustar 00simon admin 000000 000000 Bitten-0.6/ChangeLog 000644 000765 000120 00000020134 11536403656 014641 0 ustar 00simon admin 000000 000000 Version 0.6
(11 March 2011, from 0.6.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.6
* Compatibility fix for Python 2.6 HTTPBasicAuthHandler issues.
* When checking to delete pending build, handle case where config may not
exist.
* Set 'Content-Length' header on keep-alive requests.
* Make sure slave reads attachments as binary files.
* Fixed handling of configurations that point to deleted branches.
* Fix hg:pull command for Trac 0.12.
Version 0.6b3
(21 October 2010, from 0.6.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.6b3
* Basic support for Trac 0.12, supporting just a '(default)' repository.
* For Trac 0.12+ and DVCS repository connectors, shortened revision
numbers will be displayed.
* Attachments via `` command is completely redone to eliminate
serious performance issues.
* Removed sorting of revisions received from Trac as they are already
sorted chronologically. Use `rev_time` if any other sorting is needed.
* Moved Report Format from wiki to distributable documentation.
* Use Genshi `NewTextTemplate` for uniform notification template syntax.
* Fixed issue in use of drop index during database upgrade on MySQL.
* Added upgrade script to fix sequences on PostgreSQL tables.
* Fixed missing field in group by clause that caused lint report submission
to fail.
* Coverage context menu links correctly, and Coverage from browsing will
locate the most recent annotation between currently browsed revision and
previous change for file.
* Set 'Content-Length' header on master-slave communication.
* Fix for an issue where a renamed/deleted config would crash timeline if a
build with this config was in range.
* Shell output encoding improvements.
* Redirect after login will now redirect to same /builds url.
* Improved command-line calls on Windows, using shell for built-in commands
and scripts (like `java:ant`).
* Improvements to the `hg:pull` command.
* Support linking to individual steps on builds.
* Safer parsing of `java:junit` xml in case an optional attribute is missing.
* Delete attachments properly in some corner cases where incomplete builds are
cancelled or invalidated.
* Improved error messages for failed Python recipe commands specified via
a module or function.
* Added timeout limiting for shell and python recipe commands.
* `onerror` attribute can now be specified on `` elements as well
as `` elements.
* Charts now only show if there's applicable reports, and restrict themselves
to the versions of the active configuration.
* Times shown in the ui are now consistently times from the master, and not
inter-mixed times between the server and slaves, which caused skew issues.
* Steps are now shown while in-progress, and builds are considered aborted
based on the time since the last interaction with the serve, not since
starting. Slaves now send keepalive messages to the server to avoid timing
out during long build steps.
* Fix reference to database field in pylint report generation to solve issue
running query on PostGreSQL.
* Report charting re-implemented using Flot.
* Svn commands gained username, password and no_auth_cache options. Verbose
flag now behaves less cryptically.
* No longer create 'snapshots' directory in environments of new
projects (`initenv`).
* Added page on upgrades to distributed documentation.
* New utility script for removing duplicate builds encountered when upgrading.
* Add 'Platform' to Build web display and notifications.
* Other minor fixes.
Version 0.6b2
(12 September 2010, from 0.6.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.6b2
* Slave-only install changed to `./setup.py --without-master install` (or any
other valid distribution command).
* Tools namespace changed to `http://bitten.edgewall.org/tools/`. Both new and
old namespace will work, but Admin will issue a deprecation notice when
using the old namespace.
Version 0.6b1
(10 September 2009, from 0.6.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.6b1
* Python 2.4 is now required for slave, while master should still work using
Python 2.3 (as is also Trac 0.11 minimum).
* Switch to using HTTP for communication between the build master and build
slaves. This means the `build-master` executable is no longer needed or
installed, the build simply runs in the scope of the Trac site.
* Build recipes now need to include instructions for performing the checkout
from the version control repository. The slave no longer receives a snapshot
archive of the code, but performs the checkout itself based on the
instructions in the build recipe.
* Many fixes for compatibility with more recent versions of Trac, and Bitten
now requires Trac 0.11.
* The administration interface is now properly integrated with the Trac web
administration component.
* Unicode (non-ascii) support for recipes and command-line input and output.
* Improved authentication support in bitten-slave, including support for the
popular AccountManager plugin.
* New command-line execute() using Python subprocess module that in particular
improves the situation for slaves executing commands on Windows.
* Build logs are moved from database and into project log/bitten directory as
text files.
* Slave-only install now possible by running 'python setup-slave.py install'.
* Fixes related to creating and deleting builds correctly, including database
upgrade to avoid thread race conditions.
* Some improvements to the Admin user interface for creating/editing
configurations and platforms.
* Attachment support for configurations and builds, including a new general
command to use in recipes.
* command added.
* Many fixes and improvements for pre-existing commands.
* Improvements for coverage, lint and test summary display.
Version 0.5.3
(18 April 2006, from 0.5.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.5.3
* Fix double-escaping of report summaries.
* Fix build master error when build log contains no messages.
Version 0.5.2
(17 January 2006, from 0.5.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.5.2
* Fixes the main navigation tab that was broken in 0.5.1.
Version 0.5.1
(10 January 2006, from 0.5.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.5.1
* Fixes compatibility with Trac 0.9.3 release, as well as the current trunk.
This also means that Bitten now longer works with versions of Trac earlier
than 0.9.3.
* Improves PostgreSQL compatibility.
* Fixes encoding of non-ASCII characters in command output.
* Fix for missing log output when using on Windows.
Version 0.5
(6 October 2005, from 0.5.x branch)
http://svn.edgewall.org/repos/bitten/tags/0.5
* BDB XML is no longer being used for report storage. Instead,
collected metrics data is stored in the Trac database.
* Snapshot archives created by the master are checked for integrity
prior to their transmission to the slaves.
* Improvements to the build status presentation in Trac.
* Changes to the build recipe format. See the documentation on the web
site for details.
* New recipe commands: , , ,
, , and . Various improvements to
the existing commands.
* Recipe commands and command attributes in recipes can now reference
slave configuration values.
* The names of the master and slaves scripts have changed: `bittend`
is now `bitten-master`, `bitten` is now `bitten-slave`.
* The build master can now handle multiple Trac environments.
* The build slave now by default removes any working directories when
done.
* Build configurations can now be completely deleted.
* Build configurations can now have a minimum and maximum revision
specified. Any revisions outside that range will not be built.
* The build configuration editor now validates the supplied values.
* Fix management of target platforms when running under mod_python.
* Improved performance of the build log formatter that is responsible
for linking file references in build logs to the repository browser.
* Add paging to the build configuration view.
* Fix compatibility with PySQLite2.
Bitten-0.6/COPYING 000644 000765 000120 00000003173 11453043212 014110 0 ustar 00simon admin 000000 000000 Copyright (C) 2007-2010 Edgewall Software
Copyright (C) 2005-2007 Christopher Lenz
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
3. The name of the author may not be used to endorse or promote
products derived from this software without specific prior
written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
====================================================================
This software includes copies of external libraries:
- jquery.flot.js (MIT license)
- excanvas.js (Apache License, Version 2.0)
Bitten-0.6/doc/ 000755 000765 000120 00000000000 11536424600 013624 5 ustar 00simon admin 000000 000000 Bitten-0.6/MANIFEST-SLAVE.in 000644 000765 000120 00000000763 11310206140 015456 0 ustar 00simon admin 000000 000000 prune bitten/htdocs
prune bitten/templates
prune doc
exclude MANIFEST.in
include setup.py
include ChangeLog
include COPYING
include MANIFEST-SLAVE.in
include README.txt
exclude bitten/*.py
exclude bitten/report/*.py
exclude bitten/report/tests/*.py
exclude bitten/tests/*.py
include bitten/__init__.py
include bitten/build/*.py
include bitten/build/tests/*.py
include bitten/recipe.py
include bitten/slave.py
include bitten/tests_slave/*.py
include bitten/util/*.py
include bitten/util/tests/*.py
Bitten-0.6/MANIFEST.in 000644 000765 000120 00000000513 11375466351 014626 0 ustar 00simon admin 000000 000000 include ChangeLog
include COPYING
include MANIFEST-SLAVE.in
include README.txt
include bitten/htdocs/*.*
include bitten/templates/*.*
include doc/*.*
include doc/api/*.*
include doc/common/*.html
include doc/common/*.py
include doc/common/*.txt
include doc/common/COPYING
include doc/common/conf/*.ini
include doc/common/style/*.*
Bitten-0.6/PKG-INFO 000644 000765 000120 00000000602 11536424600 014152 0 ustar 00simon admin 000000 000000 Metadata-Version: 1.0
Name: Bitten
Version: 0.6
Summary: Continuous integration for Trac
Home-page: http://bitten.edgewall.org/
Author: Edgewall Software
Author-email: info@edgewall.org
License: BSD
Download-URL: http://bitten.edgewall.org/wiki/Download
Description: A slave for running builds and submitting them to Bitten, the continuous integration system for Trac
Platform: UNKNOWN
Bitten-0.6/README.txt 000644 000765 000120 00000011404 11252540751 014556 0 ustar 00simon admin 000000 000000 About Bitten
============
Bitten is a simple distributed continuous integration system that not only
coordinates builds across multiple machines, but also collects software
metrics generated by builds, to enable feedback and reporting about
the progress of a software project.
The Bitten software consists of two separate parts:
* The build slave, which executes builds on behalf of a build master.
* The web interface, which is implemented as an add-on to Trac
(http://trac.edgewall.com/) and provides a build management interface
as well as presentation of build results.
The build master is a plugin for Trac 0.11, while the build slave can be
installed without other dependencies than Python >= 2.4 and
setuptools >= 0.6a2 in addition to any tools required by the build process
itself. A build slave may be run on any machine that can connect to the server
running the Bitten build master.
Installation
------------
Bitten is written in Python, so make sure that you have Python installed.
You'll need Python 2.4 or later for running build slave, while the Trac plugin
(master) should still work using Python 2.3. Also, make sure that setuptools
(http://peak.telecommunity.com/DevCenter/setuptools), version 0.6a2 or later,
is installed.
If that's taken care of, you just need to download and unpack the Bitten
distribution, and execute the command:
$ python setup.py install
from the top of the directory where you unpacked (or checked out) the Bitten
code. Note that you may need administrator/root privileges for this step, as
it will by default attempt to install Bitten to the Python site-packages
directory on your system.
It's also a good idea to run the unit tests at this point, to make sure that
the code works as expected on your platform:
$ python setup.py test
It is also possible to install only the build slave, installing only the
parts of Bitten that are needed by clients to run builds:
$ python setup.py --without-master install
What's left to do now depends on whether you want to use the build master and
web interface, or just the build slave. In the latter case, you're already
done. You might need to install software that the build of your project
requires, but the Bitten build slave itself doesn't require anything extra.
For the build master and web interface, you'll need to install Trac 0.11 or
later. Please refer to the Trac documentation for information on how it is
installed.
Build Master Configuration
--------------------------
Once both Bitten and Trac are installed and working, you'll have to introduce
Bitten to your Trac project environment. If you don't have a Trac project
set up yet, you'll need to do so in order to use Bitten.
If you already have a Trac project environment, the Bitten plugin needs to be
explicitly enabled in the Trac configuration. This is done by adding it to the
[components] section in /path/to/projenv/conf/trac.ini:
[components]
bitten.* = enabled
The Trac web interface should now inform you with an error message that the
environment needs to be upgraded. To do this, run:
$ trac-admin /path/to/projenv upgrade
This will create the database tables and directories that Bitten requires.
You probably also want to grant permissions to someone (such as yourself)
to manage build configurations, and allow anonymous users to view the
status and results of builds:
$ trac-admin /path/to/projenv permission add anonymous BUILD_EXEC
$ trac-admin /path/to/projenv permission add anonymous BUILD_VIEW
$ trac-admin /path/to/projenv permission add [yourname] BUILD_ADMIN
You should now see an additional tab labeled "Build Status" in the Trac
navigation bar. This link will take you to the list of build configurations,
which at this point is of course empty. If you've set up permissions
correctly as described previously, you should see a button for adding new
build configurations. Click that button and fill out the form. Also, add
at least one target platform after saving the configuration. Last but not
least, you'll have to "activate" the build configuration.
Running the Build Slave
-----------------------
The build slave can be run on any machine that can connect to the machine
on which the build master is running. The installation of Bitten should have put
a `bitten-slave` executable on your path. If the script is not on your path,
look for it in the `bin` or `scripts` subdirectory of your Python installation.
To get a list of options for the build slave, execute it with the `--help`
option:
$ bitten-slave --help
To run the build slave against a Bitten-enabled Trac site installed at
http://myproject.example.org/trac, you'd run:
$ bitten-slave http://myproject.example.org/trac/builds
More Information
----------------
For further documentation, please see the Bitten website at:
Bitten-0.6/setup.cfg 000644 000765 000120 00000000264 11536424600 014702 0 ustar 00simon admin 000000 000000 [unittest]
xml_output = build/test-results.xml
coverage_summary = build/test-coverage.txt
coverage_dir = build/coverage
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0
Bitten-0.6/setup.py 000755 000765 000120 00000014043 11536421561 014601 0 ustar 00simon admin 000000 000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import sys
from setuptools import setup, find_packages, Feature
from setuptools.command import egg_info
sys.path.append(os.path.join('doc', 'common'))
try:
from doctools import build_doc, test_doc
except ImportError:
build_doc = test_doc = None
# Turn off multiprocessing logging
# Bug in setuptools/distutils test runner using Python 2.6.2+?
import logging
if hasattr(logging, 'logMultiprocessing'):
logging.logMultiprocessing = 0
NS_old = 'http://bitten.cmlenz.net/tools/'
NS_new = 'http://bitten.edgewall.org/tools/'
tools = [
'sh#exec = bitten.build.shtools:exec_',
'sh#pipe = bitten.build.shtools:pipe',
'c#configure = bitten.build.ctools:configure',
'c#autoreconf = bitten.build.ctools:autoreconf',
'c#cppunit = bitten.build.ctools:cppunit',
'c#cunit = bitten.build.ctools:cunit',
'c#gcov = bitten.build.ctools:gcov',
'c#make = bitten.build.ctools:make',
'mono#nunit = bitten.build.monotools:nunit',
'java#ant = bitten.build.javatools:ant',
'java#junit = bitten.build.javatools:junit',
'java#cobertura = bitten.build.javatools:cobertura',
'php#phing = bitten.build.phptools:phing',
'php#phpunit = bitten.build.phptools:phpunit',
'php#coverage = bitten.build.phptools:coverage',
'python#coverage = bitten.build.pythontools:coverage',
'python#distutils = bitten.build.pythontools:distutils',
'python#exec = bitten.build.pythontools:exec_',
'python#figleaf = bitten.build.pythontools:figleaf',
'python#pylint = bitten.build.pythontools:pylint',
'python#trace = bitten.build.pythontools:trace',
'python#unittest = bitten.build.pythontools:unittest',
'svn#checkout = bitten.build.svntools:checkout',
'svn#export = bitten.build.svntools:export',
'svn#update = bitten.build.svntools:update',
'hg#pull = bitten.build.hgtools:pull',
'xml#transform = bitten.build.xmltools:transform'
]
recipe_commands = [NS_old + tool for tool in tools] \
+ [NS_new + tool for tool in tools]
class MasterFeature(Feature):
def exclude_from(self, dist):
# Called when master is disabled (--without-master)
pass
def include_in(self, dist):
# Called when master is enabled (default, or --with-master)
dist.metadata.name = 'Bitten'
dist.metadata.description = 'Continuous integration for Trac'
dist.long_description = "A Trac plugin for collecting software " \
"metrics via continuous integration."""
# Use full manifest when master is included
egg_info.manifest_maker.template = "MANIFEST.in"
# Include tests in source distribution
if 'sdist' in dist.commands:
dist.packages = find_packages()
else:
dist.packages = find_packages(exclude=['*tests*'])
dist.test_suite = 'bitten.tests.suite'
dist.package_data = {
'bitten': ['htdocs/*.*',
'templates/*.html',
'templates/*.txt']}
dist.entry_points['trac.plugins'] = [
'bitten.admin = bitten.admin',
'bitten.main = bitten.main',
'bitten.master = bitten.master',
'bitten.web_ui = bitten.web_ui',
'bitten.testing = bitten.report.testing',
'bitten.coverage = bitten.report.coverage',
'bitten.lint = bitten.report.lint',
'bitten.notify = bitten.notify']
master = MasterFeature(
description = "Bitten Master Trac plugin",
standard = True,
py_modules = [])
egg_info.manifest_maker.template = "MANIFEST-SLAVE.in"
if os.path.exists(os.path.join(os.path.dirname(__file__), 'MANIFEST.in')):
available_features = {"master": master}
else:
# Building from a slave distribution
available_features = {}
setup(
name = 'BittenSlave',
version = '0.6',
author = 'Edgewall Software',
author_email = 'info@edgewall.org',
license = 'BSD',
url = 'http://bitten.edgewall.org/',
download_url = 'http://bitten.edgewall.org/wiki/Download',
zip_safe = False,
description = 'Continuous integration build slave for Trac',
long_description = "A slave for running builds and submitting them to " \
"Bitten, the continuous integration system for Trac",
packages = {},
py_modules = ["bitten.__init__",
"bitten.build.__init__",
"bitten.build.api",
"bitten.build.config",
"bitten.build.ctools",
"bitten.build.hgtools",
"bitten.build.javatools",
"bitten.build.monotools",
"bitten.build.phptools",
"bitten.build.pythontools",
"bitten.build.shtools",
"bitten.build.svntools",
"bitten.build.xmltools",
"bitten.recipe",
"bitten.slave",
"bitten.util.__init__",
"bitten.util.compat",
"bitten.util.loc",
"bitten.util.testrunner",
"bitten.util.xmlio",
],
test_suite = 'bitten.tests_slave.suite',
tests_require = [
'figleaf',
],
entry_points = {
'console_scripts': [
'bitten-slave = bitten.slave:main'
],
'distutils.commands': [
'unittest = bitten.util.testrunner:unittest'
],
'bitten.recipe_commands': recipe_commands
},
features = available_features,
cmdclass = {'build_doc': build_doc, 'test_doc': test_doc}
)
Bitten-0.6/doc/api/ 000755 000765 000120 00000000000 11536424600 014375 5 ustar 00simon admin 000000 000000 Bitten-0.6/doc/commands.html 000644 000765 000120 00000176715 11536424526 016343 0 ustar 00simon admin 000000 000000
Bitten: Build Recipe Commands
Build recipes are represented by XML documents. This page describes what
commands are generally available in recipes, and any runtime configuration
supported by these commands. Please note, though, that
third-party packages can add additional commands, which would then be
documented by that third party.
These are commands that are used without a namespace prefix.
1.1 <report>
Parse an XML file and send it to the master as a report with a given category.
Use this command in conjunction with the <sh:pipe> or <x:transform>
commands to send custom reports to the build master.
1.1.1 Parameters
Name
Description
category
Category of the report (for example "test" or "coverage").
file
Path to the XML file containing the report data, relative
to the project directory.
Attach a file to the build or configuration as regular attachment. An already
existing attachment on the same resource with same base filename will be
replaced.
Note: Unless consistently building latest only, overwriting files on
config level may lead to unexpected results.
1.2.1 Parameters
Name
Description
file
Path to the file to attach, relative to the project
directory.
resource
The resource to attach the file to. Either
'build' (default) or 'config'. Optional.
description
Attachment description. Optional.
2 Shell Tools
A bundle of generic tools that are not specific to any programming language or
tool-chain.
Namespace:
http://bitten.edgewall.org/tools/sh
Common prefix:
sh
2.1 <sh:exec>
Executes a program or script.
2.1.1 Parameters
Name
Description
executable
The name of the executable program.
file
Path to the script to execute, relative to the project
directory
output
Path to the output file
args
Any arguments to pass to the executable or script
dir
Directory to change to before executing the command
timeout
Limits the runtime of this command to the specified
number of seconds, after which it will be terminated.
Either executable or file must be specified.
2.1.2 Examples
TODO
2.2 <sh:pipe>
Pipes the content of a file through a program or script.
2.2.1 Parameters
Name
Description
executable
The name of the executable program.
file
Path to the script to execute, relative to the project
directory
input
Path to the input file
output
Path to the output file
args
Any arguments to pass to the executable or script
dir
Directory to change to before executing the command
Either executable or file must be specified.
2.2.2 Examples
TODO
3 C/Unix Tools
These commands provide support for tools commonly used for development of C/C++
applications on Unix platforms, such as make.
Namespace:
http://bitten.edgewall.org/tools/c
Common prefix:
c
3.1 <c:autoreconf>
Executes ths autotool autoreconf.
3.1.1 Parameters
Name
Description
force
Consider all files obsolete
install
Copy missing auxiliary files
symlink
Install symbolic links instead of copies
warnings
Report the warnings related to category
(which can actually be a comma separated list)
Runs the autoreconf tool in the base directory with the option: force, install
and 3 warning categories active: cross,syntax,error. This is equivalent to:
Same as previous but execute the command in the work directory and call
the makefile with the command line argument coverage=1.
3.4.3 Configuration
[make]path=/path/to/(c|n)make
3.5 <c:cppunit>
Report the test output generated by the CppUnit unit testing framework. The
output from CppUnit must be in XML format and in already, specified by the
file argument of this recipe.
Collects the test results from all files in the build/tests/results directory
that match the pattern TEST-*.xml. Also, maps the class names in the results
files to Java source files in the directory src/tests.
5 Mono Tools
A bundle of recipe commands that support tools commonly used by Mono/.NET projects.
Namespace:
http://bitten.edgewall.org/tools/mono
Common prefix:
mono
5.1 <mono:nunit>
Extracts information about unit test results from a files in NUnit XML format.
5.1.1 Parameters
Name
Description
file
Path to the NUnit XML test results file. This can include
wildcards, in which case all the file matching the
pattern will be included.
Instructs distutils to run the unittest command (which is provided by
Bitten), and passes the options needed to determine the output paths for test
results and code coverage reports.
Option --coverage-method is one of trace, coverage or figleaf.
7.2.3 Configuration
[python]path=/path/to/python
7.3 <python:unittest>
Extracts information from unittest results recorded in an XML file.
Note: This report must be used in conjunction with the distutils
command "unittest" that comes with Bitten.
7.3.1 Parameters
Name
Description
file
Path to the XML results file, relative to the project
source directory.
7.3.2 Examples
<python:unittestfile="build/test-results.xml"/>
Extracts the test results from the XML file located at
build/test-results.xml.
7.4 <python:trace>
Extracts coverage information recorded by the built-in Python module
trace.py.
7.4.1 Parameters
Name
Description
summary
Path to the summary file written by trace.py,
relative to the project source directory.
coverdir
Path to the directory containing the coverage files written
by trace.py, relative to the project source directory.
include
List of glob patterns (separated by space) that specify
which Python file should be included in the coverage report
exclude
List of glob patterns (separated by space) that specify
which Python file should be excluded from the coverage
report
Path to the file containing the Pylint output, relative to
the project source directory.
7.7.2 Examples
<python:pylintfile="build/pylint.out"/>
8 Subversion Tools
A collection of recipe commands for working with the Subversion version
control system. This commands are commonly used as the first step of a build
recipe to actually pull the code that should be built from the repository.
Namespace:
http://bitten.edgewall.org/tools/svn
Common prefix:
svn
8.1 <svn:checkout>
Check out a working copy from a Subversion repository.
8.1.1 Parameters
Name
Description
url
URL of the repository.
path
The path inside the repository that should be checked out.
You should normally set this to ${path} so that the
path of the build configuration is used.
revision
The revision that should be checked out. You should
normally set this to ${revision} so that the revision
of the build is used.
dir
Path specifying which directory the sources should be
checked out to (defaults to '.').
verbose
Whether to log the list of checked out files (defaults to
False).
shared_path
An optional shared directory to check the sources out in,
which will be reused for each subsequent build. This is
relative to the project directory, so for standard usage
set it to something like ../trunk
This checks out the a working copy into the current directory.
8.2 <svn:export>
Download a file or directory from a Subversion repository. This is similar to
performing a checkout, but will not include the meta-data Subversion uses to
connect the local working copy to the repository (i.e. it does not include the
.svn directories.)
8.2.1 Parameters
Name
Description
url
URL of the repository.
path
The path inside the repository that should be checked out.
You should normally set this to ${path} so that the
path of the build configuration is used.
revision
The revision that should be checked out. You should
normally set this to ${revision} so that the revision
of the build is used.
dir
Path specifying which directory the sources should be
exported to (defaults to '.')
This downloads the file or directory at ${path} from the Subversion
repository at http://svn.example.org/repos/myproject/. Variables are used
for the path and revision attributes so they are populated from the
properties of the build and build configuration.
8.3 <svn:update>
Update an existing working copy from a Subversion repository to a specific
revision.
8.3.1 Parameters
Name
Description
revision
The revision that should be checked out. You should
normally set this to ${revision} so that the revision
of the build is used.
dir
Path specifying the directory containing the sources to be
updated (defaults to '.')
8.3.2 Examples
<svn:updaterevision="${revision}"/>
This updates the working copy in the current directory. The revision is
specified as a variable so that it is populated from the properties of the
build.
9 XML Tools
A collection of recipe commands for XML processing.
Namespace:
http://bitten.edgewall.org/tools/xml
Common prefix:
x
9.1 <x:transform>
Apply an XSLT stylesheet .
Note: This command requires either libxslt (with Python bindings)
or, on Windows platforms, MSXML (version 3 or later) to be installed
on the slave machine.
9.1.1 Parameters
Name
Description
src
Path of the source XML file.
dest
Path of the destition XML file.
stylesheet
Path to the XSLT stylesheet file.
All these are interpreted relative to the project source directory.
This applies the stylesheet in util/convert.xsl to the source file
src.xml, and writes the resulting XML document to dest.xml.
10 Mercurial Tools
A collection of recipe commands for working with Mercurial (hg) repositories.
Namespace:
http://bitten.edgewall.org/tools/hg
Common prefix:
hg
10.1 <hg:pull>
Pull changesets and updates a local Mercurial repository.
As the command depends on a pre-existing repository, bitten-slave must be
started with --build-dir= option for locating and working with the
repository. Also, seeing no remote location can be specified the repository
.hg/hgrc configuration file needs a [paths]default=.... location
defined.
10.1.1 Parameters
Name
Description
revision
The revision to update to (optional, defaults to tip).
dir
Local subdirectory with repository (optional,
defaults to '.').
Paths are interpreted relative to the project source directory.
10.1.2 Examples
<hg:pullrevision="${revision}"dir="src"/>
This updates the repository in src to the revision of the current build.
Bitten-0.6/doc/commands.txt 000644 000765 000120 00000123613 11446640647 016207 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
=====================
Build Recipe Commands
=====================
`Build recipes`_ are represented by XML documents. This page describes what
commands are generally available in recipes, and any `runtime configuration`_
supported by these commands. Please note, though, that
third-party packages can add additional commands, which would then be
documented by that third party.
.. _`build recipes`: recipes.html
.. _`runtime configuration`: configure.html
.. contents:: Contents
:depth: 2
.. sectnum::
Generic Commands
================
These are commands that are used without a namespace prefix.
------------
````
------------
Parse an XML file and send it to the master as a report with a given category.
Use this command in conjunction with the ```` or ````
commands to send custom reports to the build master.
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``category`` | Category of the report (for example "test" or "coverage"). |
+--------------+-------------------------------------------------------------+
| ``file`` | Path to the XML file containing the report data, relative |
| | to the project directory. |
+--------------+-------------------------------------------------------------+
Both parameters must be specified.
See also `Report Formats `_.
------------
````
------------
Attach a file to the build or configuration as regular attachment. An already
existing attachment on the same resource with same base filename will be
replaced.
**Note:** Unless consistently building latest only, overwriting files on
config level may lead to unexpected results.
Parameters
----------
+-----------------+----------------------------------------------------------+
| Name | Description |
+=================+==========================================================+
| ``file`` | Path to the file to attach, relative to the project |
| | directory. |
+-----------------+----------------------------------------------------------+
| ``resource`` | The resource to attach the file to. Either |
| | 'build' (default) or 'config'. Optional. |
+-----------------+----------------------------------------------------------+
| ``description`` | Attachment description. Optional. |
+-----------------+----------------------------------------------------------+
Shell Tools
===========
A bundle of generic tools that are not specific to any programming language or
tool-chain.
:Namespace: ``http://bitten.edgewall.org/tools/sh``
:Common prefix: ``sh``
-------------
````
-------------
Executes a program or script.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``executable`` | The name of the executable program. |
+----------------+-----------------------------------------------------------+
| ``file`` | Path to the script to execute, relative to the project |
| | directory |
+----------------+-----------------------------------------------------------+
| ``output`` | Path to the output file |
+----------------+-----------------------------------------------------------+
| ``args`` | Any arguments to pass to the executable or script |
+----------------+-----------------------------------------------------------+
| ``dir`` | Directory to change to before executing the command |
+----------------+-----------------------------------------------------------+
| ``timeout`` | Limits the runtime of this command to the specified |
| | number of seconds, after which it will be terminated. |
+----------------+-----------------------------------------------------------+
Either ``executable`` or ``file`` must be specified.
Examples
--------
TODO
-------------
````
-------------
Pipes the content of a file through a program or script.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``executable`` | The name of the executable program. |
+----------------+-----------------------------------------------------------+
| ``file`` | Path to the script to execute, relative to the project |
| | directory |
+----------------+-----------------------------------------------------------+
| ``input`` | Path to the input file |
+----------------+-----------------------------------------------------------+
| ``output`` | Path to the output file |
+----------------+-----------------------------------------------------------+
| ``args`` | Any arguments to pass to the executable or script |
+----------------+-----------------------------------------------------------+
| ``dir`` | Directory to change to before executing the command |
+----------------+-----------------------------------------------------------+
Either ``executable`` or ``file`` must be specified.
Examples
--------
TODO
C/Unix Tools
============
These commands provide support for tools commonly used for development of C/C++
applications on Unix platforms, such as ``make``.
:Namespace: ``http://bitten.edgewall.org/tools/c``
:Common prefix: ``c``
------------------
````
------------------
Executes ths autotool autoreconf.
Parameters
----------
+----------------------+-----------------------------------------------------+
| Name | Description |
+======================+=====================================================+
| ``force`` | Consider all files obsolete |
+----------------------+-----------------------------------------------------+
| ``install`` | Copy missing auxiliary files |
+----------------------+-----------------------------------------------------+
| ``symlink`` | Install symbolic links instead of copies |
+----------------------+-----------------------------------------------------+
| ``warnings`` | Report the warnings related to category |
| | (which can actually be a comma separated list) |
+----------------------+-----------------------------------------------------+
| ``prepend_include`` | Prepend directories to search path |
+----------------------+-----------------------------------------------------+
| ``include`` | Append directories to search path |
+----------------------+-----------------------------------------------------+
Examples
--------
.. code-block:: xml
Runs the ``autoreconf`` tool in the base directory with the option: force, install
and 3 warning categories active: cross,syntax,error. This is equivalent to::
autoreconf --force --install --warnings=cross,syntax,error
-----------------
````
-----------------
Executes a configure script as generated by Autoconf.
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``file`` | Name of the configure script (defaults to "configure") |
+--------------+-------------------------------------------------------------+
| ``enable`` | List of features to enable, separated by spaces. |
+--------------+-------------------------------------------------------------+
| ``disable`` | List of features to disable, separated by spaces. |
+--------------+-------------------------------------------------------------+
| ``with`` | List of packages to include, separated by spaces. |
+--------------+-------------------------------------------------------------+
| ``without`` | List of packages to exclude, separated by spaces. |
+--------------+-------------------------------------------------------------+
| ``cflags`` | Value of the `CFLAGS` variable to pass to the script. |
+--------------+-------------------------------------------------------------+
| ``cxxflags`` | Value of the `CXXFLAGS` variable to pass to the script. |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
Runs the ``configure`` script in the base directory, enable the ``threadsafe``
feature, and passing ``-O`` as ``CFLAGS``. This is equivalent to::
./configure --enable-threadsafe CFLAGS="-O"
Configuration
-------------
Parameter ``with`` will expand any package found in slave configuration:
.. code-block:: ini
[mylib]
path = /path/to/mylib
------------
````
------------
Run gcov_ to extract coverage data where available.
.. _gcov: http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html
Parameters
----------
+--------------+------------------------------------------------------------+
| Name | Description |
+==============+============================================================+
| ``include`` | List of glob patterns (separated by space) that specify |
| | which source files should be included in the coverage |
| | report |
+--------------+------------------------------------------------------------+
| ``exclude`` | List of glob patterns (separated by space) that specify |
| | which source files should be excluded from the coverage |
| | report |
+--------------+------------------------------------------------------------+
| ``prefix`` | Optional prefix name that is added to object files by the |
| | build system |
+--------------+------------------------------------------------------------+
------------
````
------------
Executes a Makefile.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``target`` | Name of the target to execute (defaults to "all") |
+----------------+-----------------------------------------------------------+
| ``file`` | Path to the Makefile that should be used. |
+----------------+-----------------------------------------------------------+
| ``keep-going`` | Whether `make` should try to continue even after |
| | encountering errors. |
+----------------+-----------------------------------------------------------+
| ``jobs`` | Number of parallel jobs used by make. |
+----------------+-----------------------------------------------------------+
| ``directory`` | Path of the directory in which make should be called. |
+----------------+-----------------------------------------------------------+
| ``args`` | Any space separated arguments to pass to the makefile. |
| | Usually in the form: |
| | ``"parameter1=value1 parameter2=value2"``. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Runs the target "compile" of the ``Makefile`` located in the sub-directory
``build``.
.. code-block:: xml
Same as previous but execute the command in the ``work`` directory and call
the makefile with the command line argument ``coverage=1``.
Configuration
-------------
.. code-block:: ini
[make]
path = /path/to/(c|n)make
---------------
````
---------------
Report the test output generated by the CppUnit_ unit testing framework. The
output from CppUnit must be in XML format and in already, specified by the
``file`` argument of this recipe.
.. _cppunit: http://cppunit.sourceforge.net
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the cppunit XML output file. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Runs the program ``run_unit_tests`` to gather the data output by CppUnit in the
``test_results.xml`` file and then reports it.
Java Tools
==========
A bundle of recipe commands that support tools commonly used by Java projects.
:Namespace: ``http://bitten.edgewall.org/tools/java``
:Common prefix: ``java``
--------------
````
--------------
Runs an Ant_ build.
.. _ant: http://ant.apache.org/
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path of the build file, relative to the project source |
| | directory (default is ``build.xml``). |
+----------------+-----------------------------------------------------------+
| ``target`` | Name of the build target(s) to execute. |
+----------------+-----------------------------------------------------------+
| ``args`` | Additional arguments to pass to Ant, separated by |
| | whitespace. |
+----------------+-----------------------------------------------------------+
| ``keep_going`` | Tell Ant to continue even when errors are in encountered |
| | in the build. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Executes the target ``compile`` of the ``build.xml`` buildfile at the top of the
project source directory.
Configuration
-------------
.. code-block:: ini
[ant]
home = /path/to/ant/dir
[java]
home = /path/to/java/dir
--------------------
````
--------------------
Extract code coverage data from a Cobertura_ XML file.
.. _cobertura: http://cobertura.sourceforge.net/
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the XML file generated by Cobertura |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Reads the specifid XML file, extracts the coverage data, and builds a coverage
report to be sent to the build master.
----------------
````
----------------
Extracts information about unit test results from a file in JUnit_ XML format.
.. _junit: http://junit.org/index.htm
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the JUnit XML test results file. This can include |
| | wildcards, in which case all the file matching the |
| | pattern will be included. |
+----------------+-----------------------------------------------------------+
| ``srcdir`` | Path of the directory unit test sources. Used to link the |
| | test cases to files. |
+----------------+-----------------------------------------------------------+
The ``file`` attribute is required.
Examples
--------
.. code-block:: xml
Collects the test results from all files in the `build/tests/results` directory
that match the pattern `TEST-*.xml`. Also, maps the class names in the results
files to Java source files in the directory `src/tests`.
Mono Tools
==========
A bundle of recipe commands that support tools commonly used by Mono/.NET projects.
:Namespace: ``http://bitten.edgewall.org/tools/mono``
:Common prefix: ``mono``
----------------
````
----------------
Extracts information about unit test results from a files in NUnit_ XML format.
.. _nunit: http://nunit.org/
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the NUnit XML test results file. This can include |
| | wildcards, in which case all the file matching the |
| | pattern will be included. |
+----------------+-----------------------------------------------------------+
The ``file`` attribute is required.
Examples
--------
.. code-block:: xml
PHP Tools
=========
A bundle of recipe commands for PHP_ projects.
:Namespace: ``http://bitten.edgewall.org/tools/php``
:Common prefix: ``php``
.. _php: http://php.net/
---------------
````
---------------
Runs a Phing_ build.
.. _phing: http://phing.info/
Parameters
----------
+-------------------+-------------------------------------------------------+
| Name | Description |
+===================+=======================================================+
| ``file`` | Path of the build file, relative to the project |
| | source directory (default is ``build.xml``). |
+-------------------+-------------------------------------------------------+
| ``target`` | Name of the build target(s) to execute. |
+-------------------+-------------------------------------------------------+
| ``args`` | Additional arguments to pass to Phing, separated by |
| | whitespace. |
+-------------------+-------------------------------------------------------+
| ``executable`` | Phing executable program (default is ``phing``). |
+-------------------+-------------------------------------------------------+
Examples
--------
.. code-block:: xml
Executes the target ``compile`` of the ``build.xml`` buildfile at the top of the
project source directory.
-----------------
````
-----------------
Extracts information from PHPUnit_ test results recorded in an XML file.
.. _phpunit: http://www.phpunit.de/
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the XML results file, relative to the project |
| | source directory. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Extracts the test results from the XML file located at
``build/test-results.xml``.
------------------
````
------------------
Extracts coverage information from Phing_'s code coverage task XML file or
from PHPUnit_ coverage-clover XML file.
Parameters
----------
+---------------+-----------------------------------------------------------+
| Name | Description |
+===============+===========================================================+
| ``file`` | Path to the XML coverage file, relative to the project |
| | source directory. |
+---------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Python Tools
============
A bundle of recipe commands that support tools commonly used by Python_
projects.
:Namespace: ``http://bitten.edgewall.org/tools/python``
:Common prefix: ``python``
.. _python: http://www.python.org/
-----------------
````
-----------------
Executes a Python script.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path of the script to execute, relative to the project |
| | source directory. |
+----------------+-----------------------------------------------------------+
| ``module`` | Name of the Python module to execute. |
+----------------+-----------------------------------------------------------+
| ``function`` | Name of the function in the Python module to run. Only |
| | works when also specifying the `module` attribute. |
+----------------+-----------------------------------------------------------+
| ``args`` | Any arguments that should be passed to the script. |
+----------------+-----------------------------------------------------------+
| ``output`` | Path to a file where any output by the script should be |
| | recorded. |
+----------------+-----------------------------------------------------------+
| ``timeout`` | Limits the runtime of this command to the specified |
| | number of seconds, after which it will be terminated. |
+----------------+-----------------------------------------------------------+
Either `file` or `module` must be specified.
Examples
--------
.. code-block:: xml
Executes Pylint_ on the module/package ``myproj`` and stores the output into a
file named ``pylint-report.txt``.
Configuration
-------------
.. code-block:: ini
[python]
path = /path/to/python
----------------------
````
----------------------
Executes a distutils_ script.
.. _distutils: http://docs.python.org/lib/module-distutils.html
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| `command` | The name of the `distutils` command that should be run |
+----------------+-----------------------------------------------------------+
| `options` | Additional options to pass to the command, separated by |
| | spaces |
+----------------+-----------------------------------------------------------+
| `timeout` | Limits the runtime of this command to the specified |
| | number of seconds, after which it will be terminated. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Instructs `distutils` to produce a source distribution.
.. code-block:: xml
Instructs `distutils` to run the ``unittest`` command (which is provided by
Bitten), and passes the options needed to determine the output paths for test
results and code coverage reports.
Option ``--coverage-method`` is one of ``trace``, ``coverage`` or ``figleaf``.
Configuration
-------------
.. code-block:: ini
[python]
path = /path/to/python
---------------------
````
---------------------
Extracts information from unittest_ results recorded in an XML file.
.. _unittest: http://docs.python.org/lib/module-unittest.html
**Note:** This report must be used in conjunction with the ``distutils``
command "unittest" that comes with Bitten.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``file`` | Path to the XML results file, relative to the project |
| | source directory. |
+----------------+-----------------------------------------------------------+
Examples
--------
.. code-block:: xml
Extracts the test results from the XML file located at
``build/test-results.xml``.
------------------
````
------------------
Extracts coverage information recorded by the built-in Python module
``trace.py``.
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``summary`` | Path to the summary file written by ``trace.py``, |
| | relative to the project source directory. |
+--------------+-------------------------------------------------------------+
| ``coverdir`` | Path to the directory containing the coverage files written |
| | by ``trace.py``, relative to the project source directory. |
+--------------+-------------------------------------------------------------+
| ``include`` | List of glob patterns (separated by space) that specify |
| | which Python file should be included in the coverage report |
+--------------+-------------------------------------------------------------+
| ``exclude`` | List of glob patterns (separated by space) that specify |
| | which Python file should be excluded from the coverage |
| | report |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
---------------------
````
---------------------
Extract data from a coverage.py_ run.
.. _coverage.py: http://nedbatchelder.com/code/coverage/
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``summary`` | Path to the summary file with ``coverage.py`` information, |
| | relative to the project source directory. |
+--------------+-------------------------------------------------------------+
| ``coverdir`` | Path to the directory containing per-module coverage |
| | details, relative to the project source directory. |
+--------------+-------------------------------------------------------------+
| ``include`` | List of glob patterns (separated by space) that specify |
| | which Python file should be included in the coverage report |
+--------------+-------------------------------------------------------------+
| ``exclude`` | List of glob patterns (separated by space) that specify |
| | which Python file should be excluded from the coverage |
| | report |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
--------------------
````
--------------------
Extracts coverage information recorded by Figleaf_.
.. _figleaf: http://darcs.idyll.org/~t/projects/figleaf/doc/
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``summary`` | Path to the summary file written by ``figleaf``, |
| | relative to the project source directory. |
+--------------+-------------------------------------------------------------+
| ``include`` | List of glob patterns (separated by space) that specify |
| | which Python file should be included in the coverage report |
+--------------+-------------------------------------------------------------+
| ``exclude`` | List of glob patterns (separated by space) that specify |
| | which Python file should be excluded from the coverage |
| | report |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
-------------------
````
-------------------
Extracts information from Pylint_ reports.
.. _pylint: http://www.logilab.org/projects/pylint
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``file`` | Path to the file containing the Pylint output, relative to |
| | the project source directory. |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
Subversion Tools
================
A collection of recipe commands for working with the Subversion_ version
control system. This commands are commonly used as the first step of a build
recipe to actually pull the code that should be built from the repository.
.. _subversion: http://subversion.tigris.org/
:Namespace: ``http://bitten.edgewall.org/tools/svn``
:Common prefix: ``svn``
------------------
````
------------------
Check out a working copy from a Subversion repository.
Parameters
----------
+-----------------+-------------------------------------------------------------+
| Name | Description |
+=================+=============================================================+
| ``url`` | URL of the repository. |
+-----------------+-------------------------------------------------------------+
| ``path`` | The path inside the repository that should be checked out. |
| | You should normally set this to ``${path}`` so that the |
| | path of the build configuration is used. |
+-----------------+-------------------------------------------------------------+
| ``revision`` | The revision that should be checked out. You should |
| | normally set this to ``${revision}`` so that the revision |
| | of the build is used. |
+-----------------+-------------------------------------------------------------+
| ``dir`` | Path specifying which directory the sources should be |
| | checked out to (defaults to '.'). |
+-----------------+-------------------------------------------------------------+
| ``verbose`` | Whether to log the list of checked out files (defaults to |
| | False). |
+-----------------+-------------------------------------------------------------+
| ``shared_path`` | An optional shared directory to check the sources out in, |
| | which will be reused for each subsequent build. This is |
| | relative to the project directory, so for standard usage |
| | set it to something like ``../trunk`` |
+-----------------+-------------------------------------------------------------+
| ``username`` | Username to pass for authentication (optional) |
+-----------------+-------------------------------------------------------------+
| ``password`` | Password to pass for authentication (optional) |
+-----------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
This checks out the a working copy into the current directory.
----------------
````
----------------
Download a file or directory from a Subversion repository. This is similar to
performing a checkout, but will not include the meta-data Subversion uses to
connect the local working copy to the repository (i.e. it does not include the
``.svn`` directories.)
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``url`` | URL of the repository. |
+--------------+-------------------------------------------------------------+
| ``path`` | The path inside the repository that should be checked out. |
| | You should normally set this to ``${path}`` so that the |
| | path of the build configuration is used. |
+--------------+-------------------------------------------------------------+
| ``revision`` | The revision that should be checked out. You should |
| | normally set this to ``${revision}`` so that the revision |
| | of the build is used. |
+--------------+-------------------------------------------------------------+
| ``dir`` | Path specifying which directory the sources should be |
| | exported to (defaults to '.') |
+--------------+-------------------------------------------------------------+
| ``username`` | Username to pass for authentication (optional) |
+--------------+-------------------------------------------------------------+
| ``password`` | Password to pass for authentication (optional) |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
This downloads the file or directory at ``${path}`` from the Subversion
repository at ``http://svn.example.org/repos/myproject/``. Variables are used
for the ``path`` and ``revision`` attributes so they are populated from the
properties of the build and build configuration.
----------------
````
----------------
Update an existing working copy from a Subversion repository to a specific
revision.
Parameters
----------
+--------------+-------------------------------------------------------------+
| Name | Description |
+==============+=============================================================+
| ``revision`` | The revision that should be checked out. You should |
| | normally set this to ``${revision}`` so that the revision |
| | of the build is used. |
+--------------+-------------------------------------------------------------+
| ``dir`` | Path specifying the directory containing the sources to be |
| | updated (defaults to '.') |
+--------------+-------------------------------------------------------------+
Examples
--------
.. code-block:: xml
This updates the working copy in the current directory. The revision is
specified as a variable so that it is populated from the properties of the
build.
XML Tools
=========
A collection of recipe commands for XML processing.
:Namespace: ``http://bitten.edgewall.org/tools/xml``
:Common prefix: ``x``
-----------------
````
-----------------
Apply an XSLT stylesheet .
**Note:** This command requires either libxslt_ (with `Python bindings`_)
or, on Windows platforms, MSXML (version 3 or later) to be installed
on the slave machine.
.. _libxslt: http://xmlsoft.org/XSLT/
.. _`python bindings`: http://xmlsoft.org/XSLT/python.html
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``src`` | Path of the source XML file. |
+----------------+-----------------------------------------------------------+
| ``dest`` | Path of the destition XML file. |
+----------------+-----------------------------------------------------------+
| ``stylesheet`` | Path to the XSLT stylesheet file. |
+----------------+-----------------------------------------------------------+
All these are interpreted relative to the project source directory.
Examples
--------
.. code-block:: xml
This applies the stylesheet in ``util/convert.xsl`` to the source file
``src.xml``, and writes the resulting XML document to ``dest.xml``.
Mercurial Tools
===============
A collection of recipe commands for working with Mercurial_ (hg) repositories.
.. _mercurial: http://mercurial.selenic.com/
:Namespace: ``http://bitten.edgewall.org/tools/hg``
:Common prefix: ``hg``
-------------
````
-------------
Pull changesets and updates a local Mercurial repository.
As the command depends on a pre-existing repository, bitten-slave must be
started with ``--build-dir=`` option for locating and working with the
repository. Also, seeing no remote location can be specified the repository
``.hg/hgrc`` configuration file needs a ``[paths] default = ....`` location
defined.
Parameters
----------
+----------------+-----------------------------------------------------------+
| Name | Description |
+================+===========================================================+
| ``revision`` | The revision to update to (optional, defaults to tip). |
+----------------+-----------------------------------------------------------+
| ``dir`` | Local subdirectory with repository (optional, |
| | defaults to '.'). |
+----------------+-----------------------------------------------------------+
Paths are interpreted relative to the project source directory.
Examples
--------
.. code-block:: xml
This updates the repository in ``src`` to the revision of the current build.
Bitten-0.6/doc/common/ 000755 000765 000120 00000000000 11536424600 015114 5 ustar 00simon admin 000000 000000 Bitten-0.6/doc/configure.html 000644 000765 000120 00000034431 11536424527 016510 0 ustar 00simon admin 000000 000000
Bitten: Configuration
While the Recipe contains the instructions for how to build,
configurations are used to determine what gets built and any runtime
parameters used when building.
A target platform is something like 'NetBSD x86' or 'Win32 Java 1.4'.
Technically, a target platform is a named set of rules against which the
properties of build slaves are matched. Each rule is a regular expression
matching a particular slave property, such as the operating system
or the processor. When a slave connects to the build master, it sends a
registration message that includes information about the slave. A slave
will only be sent builds for a given platform if the slave's properties
satisfy all of the rules associated with that platform.
A rule's regular expression is matched against the value of the slave
property using Python's re.match function so, for example, x86 will
match a value of x86_64. Use ^x86$ to match only the value x86.
A build configuration must have at least one target platform assigned to
it before it becomes fully active.
2 Slave Properties
By default, the following properties are included:
family:
The basic type of operating system, typically “posix” for
Unix-like systems and “nt” for Win32 systems.
os:
The name of the operating system (for example “Darwin”,
“Linux” or “Windows”).
version:
The operating system version.
machine:
The hardware platform (for example “i686” or “Power Macintosh”).
processor:
The processor architecture (for example “athlon” or “powerpc”).
name:
The name of the slave.
ipnr:
The IP address of the slave.
Note that not all of these properties may be available for all platforms,
depending on OS and Python version.
2.1 Examples
To set up a target platform, create rules that are checked against the
properties of the slave. For example, a target platform that matches slave
running Linux on x86 would look like this:
Property
Expression
os
^Linux
machine
^[xi]d?86$
A target platform that matches any slaves running on Windows might look
like this:
Property
Expression
family
^nt$
The build master will request a build from at most one slave for every
target platform. So, for example, if there are three slaves connected that
are matching 'NetBSD x86', only one of them will perform the build of a
specific revision. Slaves that match a particular target platform are
treated as if they were completely interchangable.
If a slave connects that doesn't match any of the configured target platforms,
the build master will reject its registration.
3 Slave Configuration
When a build slave registers with a build master, it sends information about
the machine the slave is running on, and what software it has available.
While some of this information can be automatically discovered by the slave,
other information may need to be configured explicitly. Also, a slave instance
may want to override some of the automatically computed attributes,
for example to enable cross-compilation.
There are three categories of information that can be configured for a slave:
os:
Properties of the operating system
machine:
Properties of the underlying hardware
packages:
Various pieces of software, like a language runtime or a library
3.1 Configuration File Format
For simple manual editing, the slave configuration file will be based on
the 'INI' file format known from Windows, which is also frequently used by
Python applications.
The file is included at runtime using a slave command-line option:
bitten-slave -f config.ini
A configuration file is partitioned into named sections. There are two
predefined sections named [machine] and [os]. If you supply them in
your configuration file they should include the following sections.
[os]name=Darwinversion=8.1.0family=posix[machine]name=leviprocessor=Power Macintosh
There may be any number of additional sections, where each section corresponds
to a software package. For example:
Note: Options called name is not allowed in custom sections (will
be skipped).
The build slave sends this package information as part of the build initiation,
which when using verbose logging (bitten-slave-v) will display a debug
message 'Sending slave configuration:' followed by:
The name of the slave can only be set as command-line option:
bitten-slave --name=myhost
3.2 Commands using Properties
A number of commands support runtime settings using a slave configuration
file. The example of python.path above is one such example, where all
Python commands will use the specified executable for running commands.
The documentation for commands should include information about all
runtime settings.
3.3 Properties in Build Configurations
Defined properties can be used in a build configuration to match slaves
against target platforms. For example, the following rule would match any slave
providing 'Berkeley DB XML' version 2.x:
dbxml.version ~= /^2\.\d\.\d.*/
The properties are accessible in dotted notation, where the part before the dot
is the package name, and the part after the dot is the name of the property.
3.4 Property Interpolation in Build Recipes
Property values can be interpolated into build recipes as well, so individual
slaves can parameterize how their build is perfomed. For example, given the
following build recipe excerpt:
The authentication information will be removed as soon as it is read
by the slave, and will not be passed to the master as active configuration.
Bitten-0.6/doc/configure.txt 000644 000765 000120 00000017426 11513271761 016363 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
=============
Configuration
=============
While the `Recipe`_ contains the instructions for how to build,
configurations are used to determine what gets built and any runtime
parameters used when building.
.. _recipe: recipe.html
.. contents:: Contents
:depth: 2
.. sectnum::
Target platforms
================
A target platform is something like 'NetBSD x86' or 'Win32 Java 1.4'.
Technically, a target platform is a named set of rules against which the
properties of build slaves are matched. Each rule is a regular expression
matching a particular slave property, such as the operating system
or the processor. When a slave connects to the build master, it sends a
registration message that includes information about the slave. A slave
will only be sent builds for a given platform if the slave's properties
satisfy all of the rules associated with that platform.
A rule's regular expression is matched against the value of the slave
property using Python's `re.match`_ function so, for example, `x86` will
match a value of `x86_64`. Use `^x86$` to match only the value `x86`.
.. _re.match: http://docs.python.org/library/re.html#re.match
A build configuration must have at least one target platform assigned to
it before it becomes fully active.
Slave Properties
================
By default, the following properties are included:
:`family`: The basic type of operating system, typically “posix” for
Unix-like systems and “nt” for Win32 systems.
:`os`: The name of the operating system (for example “Darwin”,
“Linux” or “Windows”).
:`version`: The operating system version.
:`machine`: The hardware platform (for example “i686” or “Power Macintosh”).
:`processor`: The processor architecture (for example “athlon” or “powerpc”).
:`name`: The name of the slave.
:`ipnr`: The IP address of the slave.
Note that not all of these properties may be available for all platforms,
depending on OS and Python version.
Examples
--------
To set up a target platform, create rules that are checked against the
properties of the slave. For example, a target platform that matches slave
running Linux on x86 would look like this:
+------------+------------------------------------+
+ Property | Expression |
+============+====================================+
| `os` | `^Linux` |
+------------+------------------------------------+
| `machine` | `^[xi]\d?86$` |
+------------+------------------------------------+
A target platform that matches any slaves running on Windows might look
like this:
+------------+------------------------------------+
+ Property | Expression |
+============+====================================+
| `family` | `^nt$` |
+------------+------------------------------------+
The build master will request a build from at most one slave for every
target platform. So, for example, if there are three slaves connected that
are matching 'NetBSD x86', only one of them will perform the build of a
specific revision. Slaves that match a particular target platform are
treated as if they were completely interchangable.
If a slave connects that doesn't match any of the configured target platforms,
the build master will reject its registration.
Slave Configuration
===================
When a build slave registers with a build master, it sends information about
the machine the slave is running on, and what software it has available.
While some of this information can be automatically discovered by the slave,
other information may need to be configured explicitly. Also, a slave instance
may want to override some of the automatically computed attributes,
for example to enable cross-compilation.
There are three categories of information that can be configured for a slave:
:`os`: Properties of the operating system
:`machine`: Properties of the underlying hardware
:`packages`: Various pieces of software, like a language runtime or a library
Configuration File Format
-------------------------
For simple manual editing, the slave configuration file will be based on
the ``'INI'`` file format known from Windows, which is also frequently used by
Python applications.
The file is included at runtime using a slave command-line option::
bitten-slave -f config.ini
A configuration file is partitioned into named sections. There are two
predefined sections named ``[machine]`` and ``[os]``. If you supply them in
your configuration file they should include the following sections.
.. code-block:: ini
[os]
name = Darwin
version = 8.1.0
family = posix
[machine]
name = levi
processor = Power Macintosh
There may be any number of additional sections, where each section corresponds
to a software package. For example:
.. code-block:: ini
[dbxml]
version = 2.1.8
[python]
version = 2.3.5
path = /usr/bin/python2.3
*Note:* Options called ``name`` is not allowed in custom sections (will
be skipped).
The build slave sends this package information as part of the build initiation,
which when using verbose logging (``bitten-slave -v``) will display a debug
message 'Sending slave configuration:' followed by:
.. code-block:: xml
leviDarwin
The name of the slave can only be set as command-line option::
bitten-slave --name=myhost
Commands using Properties
-------------------------
A number of commands_ support runtime settings using a slave configuration
file. The example of ``python.path`` above is one such example, where all
Python commands will use the specified executable for running commands.
The documentation for commands_ should include information about all
runtime settings.
.. _commands: commands.html
Properties in Build Configurations
----------------------------------
Defined properties can be used in a build configuration to match slaves
against target platforms. For example, the following rule would match any slave
providing 'Berkeley DB XML' version 2.x::
dbxml.version ~= /^2\.\d\.\d.*/
The properties are accessible in dotted notation, where the part before the dot
is the package name, and the part after the dot is the name of the property.
Property Interpolation in Build Recipes
---------------------------------------
Property values can be interpolated into build recipes_ as well, so individual
slaves can parameterize how their build is perfomed. For example, given the
following build recipe excerpt:
.. code-block:: xml
.. _recipes: recipes.html
Slaves may control which part of the repository is checked out and tested
with a configuration file excerpt like this one:
.. code-block:: ini
[repository]
branch = /branches/0.3-testing
Default slave properties are also available for use in recipes:
.. code-block:: xml
Additionally, environment variables are also interpolated, supporting
the common notations of ``$VAR`` and ``${VAR}``.
.. code-block:: xml
Authentication
==============
Authentication information can also be included in slave configuration file:
.. code-block:: ini
[authentication]
username = myusername
password = mypassword
The authentication information will be removed as soon as it is read
by the slave, and will not be passed to the master as active configuration.
Bitten-0.6/doc/index.html 000644 000765 000120 00000003543 11536424527 015636 0 ustar 00simon admin 000000 000000
Bitten: Preface
Bitten is a Python-based framework for collecting various software metrics via
continuous integration. It builds on Trac to provide an integrated web-based
user interface.
Bitten is written in Python, so make sure that you have Python installed.
You'll need Python 2.4 or later for running build slave. The build master is
a plugin for Trac 0.11, and should still work with Python 2.3. Also, make
sure that setuptools, version 0.6a2 or later, is installed.
If that's taken care of, you just need to download and unpack the Bitten
distribution, and execute the command:
$ python setup.py install
from the top of the directory where you unpacked (or checked out) the Bitten
code. Note that you may need administrator/root privileges for this step, as
it will by default attempt to install Bitten to the Python site-packages
directory on your system.
It's also a good idea to run the unit tests at this point, to make sure that
the code works as expected on your platform (please note that Trac needs to
be installed for the tests to run; see below):
$ python setup.py test
It is also possible to install only the build slave, installing only the
parts of Bitten that are needed by clients to run builds:
$ python setup.py --without-master install
What's left to do now depends on whether you want to use the build master and
web interface, or just the build slave. In the latter case, you're already
done. You might need to install software that the build of your project
requires, but the Bitten build slave itself doesn't require anything extra.
For the build master and web interface, you'll need to install Trac 0.11 or
later. Please refer to the Trac documentation for information on how it is installed.
2 Build Master Configuration
Once both Bitten and Trac are installed and working, you'll have to introduce
Bitten to your Trac project environment. If you don't have a Trac project
set up yet, you'll need to do so in order to use Bitten.
If you already have a Trac project environment, the Bitten plugin needs to be
explicitly enabled in the Trac configuration. This is done by adding it to the
[components] section in /path/to/projenv/conf/trac.ini:
[components]bitten.*=enabled
The Trac web interface should now inform you with an error message that the
environment needs to be upgraded. To do this, run:
$ trac-admin /path/to/projenv upgrade
This will create the database tables and directories that Bitten requires.
You probably also want to grant permissions to someone (such as yourself)
to manage build configurations, and allow anonymous users to view the
status and results of builds:
Alternatively you may allow anyone to submit builds by giving anonymous
users this permission.
You should now see an additional tab labeled "Build Status" in the Trac
navigation bar. This link will take you to the list of build configurations,
which at this point is of course empty.
If Bitten is installed, and you are logged in as a user with the required
permissions, you should see additional administration pages inside the “Admin”
area, under a group named “Builds”. These pages allow you to set options of the
build master, and manage build configurations.
Add a new build configuration and fill out the form. Also, add at least one
target platform after saving the configuration. Last but not least, you'll have
to "activate" your new build configuration.
3 Running the Build Slave
The build slave can be run on any machine that can connect to the machine
on which the build master is running. The installation of Bitten should have put
a bitten-slave executable on your path. If the script is not on your path,
look for it in the bin or scripts subdirectory of your Python installation.
To get a list of options for the build slave, execute it with the --help
option:
Bitten-0.6/doc/install.txt 000644 000765 000120 00000010741 11363375354 016047 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
============
Installation
============
.. contents:: Contents
:depth: 2
.. sectnum::
Prerequisites
=============
Bitten is written in Python, so make sure that you have Python installed.
You'll need Python 2.4 or later for running build slave. The build master is
a plugin for Trac_ 0.11, and should still work with Python 2.3. Also, make
sure that setuptools_, version 0.6a2 or later, is installed.
.. _trac: http://trac.edgewall.org
.. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools
If that's taken care of, you just need to download and unpack the Bitten
distribution, and execute the command::
$ python setup.py install
from the top of the directory where you unpacked (or checked out) the Bitten
code. Note that you may need administrator/root privileges for this step, as
it will by default attempt to install Bitten to the Python site-packages
directory on your system.
It's also a good idea to run the unit tests at this point, to make sure that
the code works as expected on your platform (please note that Trac_ needs to
be installed for the tests to run; see below)::
$ python setup.py test
It is also possible to install only the build slave, installing only the
parts of Bitten that are needed by clients to run builds::
$ python setup.py --without-master install
What's left to do now depends on whether you want to use the build master and
web interface, or just the build slave. In the latter case, you're already
done. You might need to install software that the build of your project
requires, but the Bitten build slave itself doesn't require anything extra.
For the build master and web interface, you'll need to install Trac_ 0.11 or
later. Please refer to the Trac documentation for information on how it is installed.
Build Master Configuration
==========================
Once both Bitten and Trac are installed and working, you'll have to introduce
Bitten to your Trac project environment. If you don't have a Trac project
set up yet, you'll need to do so in order to use Bitten.
If you already have a Trac project environment, the Bitten plugin needs to be
explicitly enabled in the Trac configuration. This is done by adding it to the
``[components]`` section in ``/path/to/projenv/conf/trac.ini``:
.. code-block:: ini
[components]
bitten.* = enabled
The Trac web interface should now inform you with an error message that the
environment needs to be upgraded. To do this, run::
$ trac-admin /path/to/projenv upgrade
This will create the database tables and directories that Bitten requires.
You probably also want to grant permissions to someone (such as yourself)
to manage build configurations, and allow anonymous users to view the
status and results of builds::
$ trac-admin /path/to/projenv permission add [yourname] BUILD_ADMIN
$ trac-admin /path/to/projenv permission add anonymous BUILD_VIEW
Build slaves (see next section) will need permission to download build
configurations and submit results. You can grant the permission using::
$ trac-admin /path/to/projenv permission add [slavegroup] BUILD_EXEC
Alternatively you may allow anyone to submit builds by giving anonymous
users this permission.
You should now see an additional tab labeled "Build Status" in the Trac
navigation bar. This link will take you to the list of build configurations,
which at this point is of course empty.
If Bitten is installed, and you are logged in as a user with the required
permissions, you should see additional administration pages inside the “Admin”
area, under a group named “Builds”. These pages allow you to set options of the
build master, and manage build configurations.
Add a new build configuration and fill out the form. Also, add at least one
target platform after saving the configuration. Last but not least, you'll have
to "activate" your new build configuration.
Running the Build Slave
=======================
The build slave can be run on any machine that can connect to the machine
on which the build master is running. The installation of Bitten should have put
a `bitten-slave` executable on your path. If the script is not on your path,
look for it in the `bin` or `scripts` subdirectory of your Python installation.
To get a list of options for the build slave, execute it with the `--help`
option::
$ bitten-slave --help
To run the build slave against a Bitten-enabled Trac site installed at
http://myproject.example.org/trac, you'd run::
$ bitten-slave http://myproject.example.org/trac/builds
Bitten-0.6/doc/links.html 000644 000765 000120 00000002506 11536424527 015645 0 ustar 00simon admin 000000 000000
Bitten: Build Links
Bitten supports the ability to link between any wiki-enabled objects in Trac.
To link to a particular build, use the TracLink syntax build:N
(where N is the number of the build).
Linking to a particular step of the build is also supported by adding an
anchor reference, like build:N#Test. If the step ID contains spaces,
the link must be quoted: build:"N#MyStep".
Bitten-0.6/doc/notify.txt 000644 000765 000120 00000006436 11250332721 015701 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
==================
Build Notification
==================
Introduction
============
Bitten includes a mechanism to send out notification mails whenever a build
has failed (or succeeded, if required).
The notification mails contain the revision number of the build, the author
name and the build log.
Notification Example:
.. code-block:: text
Failed build of MyProject [32]
---------------------------------------------------------------------
Changeset: 32 -
Committed by: author
Build Configuration: main
Build Slave: client
Build Number: 30 -
Failed Steps:
Failure Log:
Configuration
=============
The mechanism employs the trac notification system and uses most of its
options of the ``[notification]`` section of ``trac.ini``, specifically:
+-------------------------+--------------------------------------------------+
| Option | Description |
+=========================+==================================================+
| ``smtp_enabled`` | activates mail notifications |
+-------------------------+--------------------------------------------------+
| ``smtp_default_domain`` | the domain to be appended to not fully qualified |
| | usernames |
+-------------------------+--------------------------------------------------+
| ``smtp_server`` | the smtp server to use |
+-------------------------+--------------------------------------------------+
| ``smtp_user`` | smtp server username |
+-------------------------+--------------------------------------------------+
| ``smtp_password`` | smtp server password |
+-------------------------+--------------------------------------------------+
For an indepth description of the trac notification system and its options
please refer to the `Trac documentation`_.
.. _`trac documentation`: http://trac.edgewall.org/wiki/TracNotification
To further adjust the notification behaviour you can use the following
options:
+--------------------------------+-------------------------------------------+
| Option | Description |
+================================+===========================================+
| ``notify_on_failed_build`` | notifies on failed builds (defaults to |
| | ``True``) |
+--------------------------------+-------------------------------------------+
| ``notify_on_successful_build`` | notifies on successful builds (defaults |
| | to ``False``) |
+--------------------------------+-------------------------------------------+
Configuration Example:
.. code-block:: ini
[notification]
smtp_enabled = true
smtp_default_domain = mydomain.com
smtp_server = smtp.mydomain.com
smtp_user = admin@mydomain.com
smtp_password = 12345
notify_on_failed_build = true
notify_on_successful_build = true
Bitten-0.6/doc/recipes.html 000644 000765 000120 00000023750 11536424527 016163 0 ustar 00simon admin 000000 000000
Bitten: Build Recipes
A build recipe tells a build slave how a project is to be built. It consists of
multiple build steps, each defining a command to execute, and where artifacts
can be found after that command has successfully completed.
Build recipes are intended to supplement existing project build files (such as
Makefiles), not to replace them. In general, a recipe will be much simpler than
the build file itself, because it doesn't deal with all the details of the
build. It just automates the execution of the build and lets the build slave
locate any artifacts and metrics data generated in the course of the build.
A recipe can and should split the build into multiple separate steps so that the
build slave can provide better status reporting to the build master while the
build is still in progress. This is important for builds that might take long to
execute. In addition, build steps help organize the build results for a more
structured presentation.
File Format
Build recipes are stored internally in an XML-based format. Recipe documents
have a single <build> root element with one or more <step> child
elements. The steps are executed in the order they appear in the recipe.
A <step> element will consist of any number of commands and reports. Most of
these elements are declared in XML namespaces, where the namespace URI defines
a collection of related commands.
The <build> element can optionally have an onerror attribute that
dictates how a build should proceed after the failure of a step. Allowable
values are:
fail: failure of a step causes the build to terminate. (default)
continue: builds continue after step failures. Failing steps
contribute to the overall build status.
ignore: builds continue after step failures. Builds are marked
as successful even in the presence of failed steps with
onerror='ignore'
<step> elements can override the <build>onerror attribute with
their own onerror attributes.
Commonly, the first step of any build recipe will perform the checkout from the
repository.
<buildxmlns:python="http://bitten.edgewall.org/tools/python"xmlns:svn="http://bitten.edgewall.org/tools/svn"><stepid="checkout"description="Checkout source from repository"><svn:checkouturl="http://svn.example.org/repos/foo"path="${path}"revision="${revision}"/></step><stepid="build"description="Compile to byte code"><python:distutilscommand="build"/></step><stepid="test"description="Run unit tests"><python:distutilscommand="unittest"/><python:unittestfile="build/test-results.xml"/><python:tracesummary="build/test-coverage.txt"coverdir="build/coverage"include="trac*"exclude="*.tests.*"/></step></build>
See Build Recipe Commands for a comprehensive reference of the commands
available by default.
Recipes may contain variables, for example ${path}, which are expanded
before the recipe is executed. A small set of variables is pre-defined
but custom variables may be added (see Slave Configuration for further
instructions). The pre-defined recipe variables are:
Variable name
Expanded value
${path}
Repository path from the build configuration
${config}
The build configuration name
${build}
The index of this build request
${revision}
The repository revision being tested
${platform}
The name of the target platform being built
${name}
The name of the build slave
${basedir}
The absolute path of the build location, joining
work-dir (absolute) with build-dir (relative)
As the recipe needs to be valid XML, any reserved characters in attributes must
be quoted using regular XML entities:
Character
Quoted
"
"
<
<
>
>
&
&
'
'
If needed, most commands use regular shell rules to split parts of the input -
typically like args input for sh:exec command. Double-quotes
(") can be used to mark the start and end if any sub-parts contain
whitespace, alternatively '\' can be used to escape whitespace or any other
character that carries meaning as part of input - including double-quotes and
backslash itself:
This will pass 3 arguments: o\ne + 42 + "hithere".
Note: On Windows, batch scripts and built-ins will execute through a shell.
This may affect quoting of arguments.
Bitten-0.6/doc/recipes.txt 000644 000765 000120 00000013530 11365331154 016022 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
=============
Build Recipes
=============
A build recipe tells a build slave how a project is to be built. It consists of
multiple build steps, each defining a command to execute, and where artifacts
can be found after that command has successfully completed.
Build recipes are intended to supplement existing project build files (such as
Makefiles), not to replace them. In general, a recipe will be much simpler than
the build file itself, because it doesn't deal with all the details of the
build. It just automates the execution of the build and lets the build slave
locate any artifacts and metrics data generated in the course of the build.
A recipe can and should split the build into multiple separate steps so that the
build slave can provide better status reporting to the build master while the
build is still in progress. This is important for builds that might take long to
execute. In addition, build steps help organize the build results for a more
structured presentation.
File Format
===========
Build recipes are stored internally in an XML-based format. Recipe documents
have a single ```` root element with one or more ```` child
elements. The steps are executed in the order they appear in the recipe.
A ```` element will consist of any number of commands and reports. Most of
these elements are declared in XML namespaces, where the namespace URI defines
a collection of related commands.
The ```` element can optionally have an ``onerror`` attribute that
dictates how a build should proceed after the failure of a step. Allowable
values are:
- ``fail``: failure of a step causes the build to terminate. (default)
- ``continue``: builds continue after step failures. Failing steps
contribute to the overall build status.
- ``ignore``: builds continue after step failures. Builds are marked
as successful even in the presence of failed steps with
onerror='ignore'
```` elements can override the ```` ``onerror`` attribute with
their own ``onerror`` attributes.
Commonly, the first step of any build recipe will perform the checkout from the
repository.
.. code-block:: xml
See `Build Recipe Commands`_ for a comprehensive reference of the commands
available by default.
.. _`build recipe commands`: commands.html
Recipes may contain variables, for example ``${path}``, which are expanded
before the recipe is executed. A small set of variables is pre-defined
but custom variables may be added (see `Slave Configuration`_ for further
instructions). The pre-defined recipe variables are:
.. _`slave configuration`: configure.html
+-----------------+----------------------------------------------------------+
| Variable name | Expanded value |
+=================+==========================================================+
| ``${path}`` | Repository path from the build configuration |
+-----------------+----------------------------------------------------------+
| ``${config}`` | The build configuration name |
+-----------------+----------------------------------------------------------+
| ``${build}`` | The index of this build request |
+-----------------+----------------------------------------------------------+
| ``${revision}`` | The repository revision being tested |
+-----------------+----------------------------------------------------------+
| ``${platform}`` | The name of the target platform being built |
+-----------------+----------------------------------------------------------+
| ``${name}`` | The name of the build slave |
+-----------------+----------------------------------------------------------+
| ``${basedir}`` | The absolute path of the build location, joining |
| | ``work-dir`` (absolute) with ``build-dir`` (relative) |
+-----------------+----------------------------------------------------------+
As the recipe needs to be valid XML, any reserved characters in attributes must
be quoted using regular XML entities:
+-----------+------------+
| Character | Quoted |
+===========+============+
| ``"`` | ``"`` |
+-----------+------------+
| ``<`` | ``<`` |
+-----------+------------+
| ``>`` | ``>`` |
+-----------+------------+
| ``&`` | ``&`` |
+-----------+------------+
| ``'`` | ``'`` |
+-----------+------------+
If needed, most commands use regular shell rules to split parts of the input -
typically like ``args`` input for ``sh:exec`` command. Double-quotes
(``"``) can be used to mark the start and end if any sub-parts contain
whitespace, alternatively ``'\'`` can be used to escape whitespace or any other
character that carries meaning as part of input - including double-quotes and
backslash itself:
.. code-block:: xml
This will pass 3 arguments: ``o\ne`` + ``4 2`` + ``"hi there"``.
**Note:** On Windows, batch scripts and built-ins will execute through a shell.
This may affect quoting of arguments.
Bitten-0.6/doc/reports.html 000644 000765 000120 00000020264 11536424527 016224 0 ustar 00simon admin 000000 000000
Bitten: Report Formats
The base element of the report must be "report" and have an attribute
"category" that is one of "test", "coverage" or "lint":
<reportcategory="test|coverage|lint"></report>
Inside the report there must be elements for each report type. The way the
data is captured is pretty flexible because it can either be in attributes
or in child elements.
Test Reports
Test reports must have sub-elements of report of type <test/>. These
elements can have any of these attributes (or subelements with contained
cdata):
Attribute
Description
duration
Duration of test (float)
status
"success", "failure", "error", or "ignore" (string)
name
Name of the test (string)
fixture
Name of the test fixture (string)
file
Path to test file relative to the base path for the
build configuration (string)
Bitten-0.6/doc/reports.txt 000644 000765 000120 00000013451 11444363330 016067 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
==============
Report Formats
==============
The base element of the report must be "report" and have an attribute
"category" that is one of "test", "coverage" or "lint":
.. code-block:: xml
Inside the report there must be elements for each report type. The way the
data is captured is pretty flexible because it can either be in attributes
or in child elements.
Test Reports
============
Test reports must have sub-elements of report of type ````. These
elements can have any of these attributes (or subelements with contained
cdata):
+-----------------+----------------------------------------------------------+
| Attribute | Description |
+=================+==========================================================+
| ``duration`` | Duration of test (float) |
+-----------------+----------------------------------------------------------+
| ``status`` | "success", "failure", "error", or "ignore" (string) |
+-----------------+----------------------------------------------------------+
| ``name`` | Name of the test (string) |
+-----------------+----------------------------------------------------------+
| ``fixture`` | Name of the test fixture (string) |
+-----------------+----------------------------------------------------------+
| ``file`` | Path to test file relative to the base path for the |
| | build configuration (string) |
+-----------------+----------------------------------------------------------+
| ``stdout`` | The output from the test (string) |
+-----------------+----------------------------------------------------------+
| ``traceback`` | The traceback from any error or failure (string) |
+-----------------+----------------------------------------------------------+
Example:
.. code-block:: xml
0.073successbitten.tests.model.BuildConfigTestCasetest_config_update_namebitten/tests/model.pyRenaming build configuration
Coverage Reports
================
Coverage reports must be inside ```` elements. The
allowable attributes are:
+-----------------+----------------------------------------------------------+
| Attribute | Description |
+=================+==========================================================+
| ``name`` | The name of the module being tested for coverage |
+-----------------+----------------------------------------------------------+
| ``file`` | The name of the file relative to the base path in the |
| | build configuration |
+-----------------+----------------------------------------------------------+
| ``percentage`` | The percentage of lines in that file covered |
+-----------------+----------------------------------------------------------+
| ``lines`` | The number of lines covered |
+-----------------+----------------------------------------------------------+
| ``line_hits`` | Line-by-line coverage of the file, where code lines have |
| | 0 or more times covered and non-code lines are marked |
| | as `'-'` (optional) |
+-----------------+----------------------------------------------------------+
Example:
.. code-block:: xml
Lint Reports
============
Lint issues are placed inside ```` elements, with allowed attributes of:
+-----------------+----------------------------------------------------------+
| Attribute | Description |
+=================+==========================================================+
| ``file`` | The name of the file relative to the base path in the |
| | build configuration |
+-----------------+----------------------------------------------------------+
| ``tag`` | Class, method or other useful identifiable location |
| | inside the file |
+-----------------+----------------------------------------------------------+
| ``line`` | Line number |
+-----------------+----------------------------------------------------------+
| ``category`` | Category for problem; |
| | ``convention \| warning \| refactor \| error`` |
+-----------------+----------------------------------------------------------+
Example:
.. code-block:: xml
Missing docstring
Bitten-0.6/doc/upgrade.html 000644 000765 000120 00000020110 11536424527 016143 0 ustar 00simon admin 000000 000000
Bitten: Upgrading
Bitten's database schema is versioned using the bitten_version entry in
Trac's system table. If you install a version of Bitten with a more recent
schema you will need to run trac-admin update after installing Bitten to
update your database. Until this done Trac will not activate your new
Bitten installation.
Note
You should back up your Trac database and Trac environment before running
trac-admin upgrade.
Note
If you are using a SQLite database file, Trac automatically creates a copy
before proceeding with the update. This should not be relied upon but can
be useful if you have ignored the advice above.
Bitten's upgrade scripts attempt to commit changes made to the database each
time a new schema version is reached. This ensures that if an upgrade fails
to complete the Bitten database is left in the most up-to-date state possible.
For example, if you were upgrading from Bitten schema 6 to schema version 12
and duplicate builds were found during the attempt to upgrade to schema 10, your
database would be left at schema 9.
1 Removing Duplicate Builds
Bitten schema version 10 adds a unique index on the columns
(config, platform, rev) to the bitten_build table. This constraint is
relied on by Bitten but was not enforced in earlier versions and duplicate
builds could arise. Before the upgrade to schema version 10 can complete, these
duplicate builds must be removed. The trac-admin upgrade will produce a list
of duplicates but the decision on which to keep remains with the user.
A script for removing builds is provided. This scripts supports schema
versions 9 through 12.
2 Reporting Upgrade Issues
Debugging upgrade issues can be difficult since the Bitten development team
does not have access to your database. Attaching a description of your
database schema before and after running trac-admin upgrade to your
request for assistance may help others diagnose the problem more swiftly.
You can create such a schema description using Gerald or a similar tool.
3 Upgrade history
The current version of the Bitten schema is version 12.
3.1 Version 12
last activity column added to bitten_build.
3.2 Version 11
Badly named log level files that ended with .log.level created by earlier
versions of Bitten were renamed to end in .log.levels which is the correct
extension.
Log level files that had not been correctly deleted by earlier versions of
Bitten were cleaned up.
3.3 Version 10
PostgreSQL sequences on the id column that had not been correctly updated
by previous Bitten upgrade scripts were corrected. This affected four tables:
bitten_build, bitten_log, bitten_platform and bitten_report.
A unique index on the columns (config, platform, rev) was added to the
bitten_build table and an old unique index on (config, rev, slave) was
removed.
Note
The lack of an index on (config, platform, rev) in ealier versions of
Bitten allowed duplicate builds to be created in the database under some
conditions. These duplicate builds need to be manually removed before the
upgrade to version 10 can complete.
3.4 Version 9
The id column of the bitten_rule table was converted from a text column
to an integer one.
3.5 Version 8
Log messages that had been stored in the bitten_log_messages table were
migrated to files.
A filename field was added to the bitten_log table to record the
location of the new log files.
3.6 Version 7
The bitten_error table as added to record the reasons for step failures.
3.7 Version 6
Filenames stored in the value column of bitten_report_item where
normalized to use / as the path separator.
The generator columns in bitten_log and bitten_report previously
stored just the short tag name of the associated generator. These were
updated to store the full tag name of the generator (including the XML
namespace). For example, pipe was changed to
http://bitten.edgewall.org/tools/sh#pipe.
3.8 Version 5
An orderno column was added to the bitten_log table to record the
order in which sets of log messages were generated.
The bitten_report and bitten_report_item tables were added to record
reports generated by unit tests and other build tools.
Report data was imported from the old Berkeley DB XML store into the new
database tables.
3.9 Version 4
The build configuration that generated a report was added to the report
metadata in the Berkeley DB XML store.
3.10 Version 3
The recipe column was added to bitten_config to store the recipe XML.
3.11 Version 2
The bitten_log and bitten_log_message tables were created for storing
log messages.
The log column was removed from bitten_step and the log messages where
migrated to the two new logging tables.
3.12 Version 1
The first Bitten database schema.
Bitten-0.6/doc/upgrade.txt 000644 000765 000120 00000012271 11457032362 016021 0 ustar 00simon admin 000000 000000 .. -*- mode: rst; encoding: utf-8 -*-
=========
Upgrading
=========
.. contents:: Contents
:depth: 1
.. sectnum::
Bitten's database schema is versioned using the `bitten_version` entry in
Trac's system table. If you install a version of Bitten with a more recent
schema you will need to run `trac-admin update` after installing Bitten to
update your database. Until this done Trac will not activate your new
Bitten installation.
.. note::
You should back up your Trac database and Trac environment before running
`trac-admin upgrade`.
.. note::
If you are using a SQLite database file, Trac automatically creates a copy
before proceeding with the update. This should not be relied upon but can
be useful if you have ignored the advice above.
Bitten's upgrade scripts attempt to commit changes made to the database each
time a new schema version is reached. This ensures that if an upgrade fails
to complete the Bitten database is left in the most up-to-date state possible.
For example, if you were upgrading from Bitten schema 6 to schema version 12
and duplicate builds were found during the attempt to upgrade to schema 10, your
database would be left at schema 9.
Removing Duplicate Builds
=========================
Bitten schema version 10 adds a unique index on the columns
`(config, platform, rev)` to the `bitten_build` table. This constraint is
relied on by Bitten but was not enforced in earlier versions and duplicate
builds could arise. Before the upgrade to schema version 10 can complete, these
duplicate builds must be removed. The `trac-admin upgrade` will produce a list
of duplicates but the decision on which to keep remains with the user.
A script_ for removing builds is provided. This scripts supports schema
versions 9 through 12.
.. _script: http://bitten.edgewall.org/browser/trunk/contrib/deletebuild.py
Reporting Upgrade Issues
========================
Debugging upgrade issues can be difficult since the Bitten development team
does not have access to your database. Attaching a description of your
database schema before and after running `trac-admin upgrade` to your
request for assistance may help others diagnose the problem more swiftly.
You can create such a schema description using Gerald_ or a similar tool.
.. _gerald: http://pypi.python.org/pypi/gerald
Upgrade history
===============
The current version of the Bitten schema is version 12.
Version 12
----------
* `last activity` column added to `bitten_build`.
Version 11
----------
* Badly named log level files that ended with `.log.level` created by earlier
versions of Bitten were renamed to end in `.log.levels` which is the correct
extension.
* Log level files that had not been correctly deleted by earlier versions of
Bitten were cleaned up.
Version 10
----------
* PostgreSQL sequences on the `id` column that had not been correctly updated
by previous Bitten upgrade scripts were corrected. This affected four tables:
`bitten_build`, `bitten_log`, `bitten_platform` and `bitten_report`.
* A unique index on the columns `(config, platform, rev)` was added to the
`bitten_build` table and an old unique index on `(config, rev, slave)` was
removed.
.. note::
The lack of an index on `(config, platform, rev)` in ealier versions of
Bitten allowed duplicate builds to be created in the database under some
conditions. These duplicate builds need to be manually removed before the
upgrade to version 10 can complete.
Version 9
---------
* The `id` column of the `bitten_rule` table was converted from a text column
to an integer one.
Version 8
---------
* Log messages that had been stored in the `bitten_log_messages` table were
migrated to files.
* A `filename` field was added to the `bitten_log` table to record the
location of the new log files.
Version 7
---------
* The `bitten_error` table as added to record the reasons for step failures.
Version 6
---------
* Filenames stored in the `value` column of `bitten_report_item` where
normalized to use `/` as the path separator.
* The `generator` columns in `bitten_log` and `bitten_report` previously
stored just the short tag name of the associated generator. These were
updated to store the full tag name of the generator (including the XML
namespace). For example, `pipe` was changed to
`http://bitten.edgewall.org/tools/sh#pipe`.
Version 5
---------
* An `orderno` column was added to the `bitten_log` table to record the
order in which sets of log messages were generated.
* The `bitten_report` and `bitten_report_item` tables were added to record
reports generated by unit tests and other build tools.
* Report data was imported from the old Berkeley DB XML store into the new
database tables.
Version 4
---------
* The build configuration that generated a report was added to the report
metadata in the Berkeley DB XML store.
Version 3
---------
* The `recipe` column was added to `bitten_config` to store the recipe XML.
Version 2
---------
* The `bitten_log` and `bitten_log_message` tables were created for storing
log messages.
* The `log` column was removed from `bitten_step` and the log messages where
migrated to the two new logging tables.
Version 1
---------
* The first Bitten database schema.
Bitten-0.6/doc/common/conf/ 000755 000765 000120 00000000000 11536424600 016041 5 ustar 00simon admin 000000 000000 Bitten-0.6/doc/common/COPYING 000644 000765 000120 00000002604 10656055446 016163 0 ustar 00simon admin 000000 000000 Copyright (C) 2007 Edgewall Software
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
3. The name of the author may not be used to endorse or promote
products derived from this software without specific prior
written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Bitten-0.6/doc/common/doctools.py 000644 000765 000120 00000010670 11473774533 017335 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://babel.edgewall.org/wiki/License.
#
# This software consists of voluntary contributions made by many
# individuals. For the exact contribution history, see the revision
# history and logs, available at http://babel.edgewall.org/log/.
from distutils.cmd import Command
import doctest
from glob import glob
import os
import sys
TOOLS_DIR = os.path.dirname(__file__)
class build_doc(Command):
description = 'generate the documentation'
user_options = [
('force', None,
"force regeneration even if no reStructuredText files have changed"),
('without-apidocs', None,
"whether to skip the generation of API documentaton"),
]
boolean_options = ['force', 'without-apidocs']
def initialize_options(self):
self.force = False
self.without_apidocs = False
def finalize_options(self):
pass
def run(self):
from docutils.core import publish_cmdline
from docutils.nodes import raw
from docutils.parsers import rst
from genshi.input import HTMLParser
from genshi.template import TemplateLoader
docutils_conf = os.path.join(TOOLS_DIR, 'conf', 'docutils.ini')
epydoc_conf = os.path.join(TOOLS_DIR, 'conf', 'epydoc.ini')
try:
from pygments import highlight
from pygments.lexers import get_lexer_by_name
from pygments.formatters import HtmlFormatter
def code_block(name, arguments, options, content, lineno,
content_offset, block_text, state, state_machine):
lexer = get_lexer_by_name(arguments[0])
html = highlight('\n'.join(content), lexer, HtmlFormatter())
return [raw('', html, format='html')]
code_block.arguments = (1, 0, 0)
code_block.options = {'language' : rst.directives.unchanged}
code_block.content = 1
rst.directives.register_directive('code-block', code_block)
except ImportError:
print('Pygments not installed, syntax highlighting disabled')
loader = TemplateLoader(['doc', 'doc/common'], variable_lookup='strict')
for source in glob('doc/*.txt'):
dest = os.path.splitext(source)[0] + '.html'
if self.force or not os.path.exists(dest) or \
os.path.getmtime(dest) < os.path.getmtime(source):
print('building documentation file %s' % dest)
publish_cmdline(writer_name='html',
argv=['--config=%s' % docutils_conf, source,
dest])
fileobj = open(dest)
try:
html = HTMLParser(fileobj, encoding='utf-8')
template = loader.load('template.html')
output = template.generate(
html=html,
project=self.distribution
).render('html', encoding='utf-8')
finally:
fileobj.close()
fileobj = open(dest, 'w')
try:
fileobj.write(output)
finally:
fileobj.close()
if not self.without_apidocs:
try:
from epydoc import cli
old_argv = sys.argv[1:]
sys.argv[1:] = [
'--config=%s' % epydoc_conf,
'--top=%s' % self.distribution.packages[0],
'--no-private', # epydoc bug, not read from config
'--simple-term',
'--verbose'
] + self.distribution.packages
cli.cli()
sys.argv[1:] = old_argv
except ImportError:
print('epydoc not installed, skipping API documentation.')
class test_doc(Command):
description = 'test the code examples in the documentation'
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
for filename in glob('doc/*.txt'):
print('testing documentation file %s' % filename)
doctest.testfile(filename, False, optionflags=doctest.ELLIPSIS)
Bitten-0.6/doc/common/README.txt 000644 000765 000120 00000000467 10656055446 016633 0 ustar 00simon admin 000000 000000 Edgewall Documentation Utilities
================================
This repository contains distutils commands for generating offline documentation
from reStructuredText files and Python source code. These tools are shared among
a couple of different Edgewall projects to ensure common style and
functionality.
Bitten-0.6/doc/common/style/ 000755 000765 000120 00000000000 11536424600 016254 5 ustar 00simon admin 000000 000000 Bitten-0.6/doc/common/template.html 000644 000765 000120 00000001221 10661136320 017606 0 ustar 00simon admin 000000 000000
${project.get_name()}: ${select('text()')}
This function should return a tuple of the form (template,data),
where template is the name of the template to use and data is
the data to be passed to the template.
This function should return a tuple of the form (template,data),
where template`isthenameofthetemplatetouseand``data is
the data to be passed to the template.
__init__(self,
executable,
args,
input=None,
cwd=None,
shell=False)
Initialize the CommandLine object.
execute(self,
timeout=None)
Execute the command, and return a generator for iterating over
the output written to the standard output and error streams.
Encapsulates the configuration of a build machine.
Configuration values can be provided through a configuration file (in INI
format) or through command-line parameters (properties). In addition to
explicitly defined properties, this class automatically collects platform
information and stores them as properties. These defaults can be
overridden (useful for cross-compilation).
Instance Methods
__init__(self,
filename=None,
properties=None)
Create the configuration object.
__contains__(self,
key)
Return whether the configuration contains a value for the specified
key.
__getitem__(self,
key)
Return the value for the specified configuration key.
filename - the path to the configuration file, if any
properties - a dictionary of the configuration properties
provided on the command-line
Overrides:
object.__init__
__contains__(self,
key) (In operator)
Return whether the configuration contains a value for the specified
key.
Parameters:
key - name of the configuration option using dotted notation
(for example, "python.path")
__getitem__(self,
key) (Indexing operator)
Return the value for the specified configuration key.
Parameters:
key - name of the configuration option using dotted notation
(for example, "python.path")
__str__(self) (Informal representation operator)
str(x)
Overrides:
object.__str__
(inherited documentation)
get_dirpath(self,
key)
Return the value of the specified configuration key, but verify that
the value refers to the path of an existing directory.
If the value does not exist, or is not a directory path, return None.
Parameters:
key - name of the configuration option using dotted notation
(for example, "ant.home")
get_filepath(self,
key)
Return the value of the specified configuration key, but verify that
the value refers to the path of an existing file.
If the value does not exist, or is not a file path, return None.
Parameters:
key - name of the configuration option using dotted notation
(for example, "python.path")
interpolate(self,
text,
**vars)
Interpolate configuration and environment properties into a string.
Configuration properties can be referenced in the text using the notation
${property.name}. A default value can be provided by appending it to
the property name separated by a colon, for example
${property.name:defaultvalue}. This value will be used when there's
no such property in the configuration. Otherwise, if no default is
provided, the reference is not replaced at all.
Environment properties substitute from environment variables, supporting
the common notations of $VAR and ${VAR}.
Parameters:
text - the string containing variable references
vars - extra variables to use for the interpolation
Either the file_ or the module parameter must be provided. If
specified using the file_ parameter, the file must be inside the project
directory. If specified as a module, the module must either be resolvable
to a file, or the function parameter must be provided
copytree(src,
dst,
symlinks=False)
Recursively copy a directory tree using copy2().
checkout(ctxt,
url,
path=None,
revision=None,
dir_='.',
verbose='false',
shared_path=None,
username=None,
password=None,
no_auth_cache='false')
Perform a checkout from a Subversion repository.
export(ctxt,
url,
path=None,
revision=None,
dir_='.',
username=None,
password=None,
no_auth_cache='false')
Perform an export from a Subversion repository.
update(ctxt,
revision=None,
dir_='.',
username=None,
password=None,
no_auth_cache='false')
Update the local working copy from the Subversion repository.
Variables
log = logging.getLogger('bitten.build.svntools')
__package__ = 'bitten.build'
Function Details
copytree(src,
dst,
symlinks=False)
Recursively copy a directory tree using copy2().
If exception(s) occur, an Error is raised with a list of reasons.
If the optional symlinks flag is true, symbolic links in the
source tree result in symbolic links in the destination tree; if
it is false, the contents of the files pointed to by symbolic
links are copied.
check_attachment_permission(self,
action,
username,
resource,
perm)
Respond to the various actions into the legacy attachment
permissions used by the Attachment module.
__init__(self,
compmgr,
init=None,
cls=<class 'bitten.main.BuildSystem'>)
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
__init__(self,
compmgr,
init=<function __init__ at 0x1022e8d70>,
cls=<class 'bitten.master.BuildMaster'>)
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
__init__(self,
env,
config=None,
rev=None,
platform=None,
slave=None,
started=0,
stopped=0,
last_activity=0,
rev_time=0,
status='P')
Initialize a new build with the specified attributes.
fetch(cls,
env,
id,
db=None)
Retrieve an existing build from the database by ID.
select(cls,
env,
config=None,
rev=None,
platform=None,
slave=None,
status=None,
db=None,
min_rev_time=None,
max_rev_time=None)
Retrieve existing builds from the database that match the specified
criteria.
__init__(self,
env,
name=None,
path=None,
active=False,
recipe=None,
min_rev=None,
max_rev=None,
label=None,
description=None)
Initialize a new build configuration with the specified attributes.
Represents an individual step of an executed build.
Instance Methods
__init__(self,
env,
build=None,
name=None,
description=None,
status=None,
started=None,
stopped=None)
Initialize a new build step with the specified attributes.
delete(self,
db=None)
Remove the build step from the database.
insert(self,
db=None)
Insert a new build step into the database.
fetch(cls,
env,
id,
db=None)
Retrieve an existing build from the database by ID.
select(cls,
env,
config=None,
build=None,
step=None,
category=None,
db=None)
Retrieve existing reports from the database that match the specified
criteria.
__init__(self,
compmgr,
init=<function __init__ at 0x102846aa0>,
cls=<class 'bitten.notify.BittenNotify'>)
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
notify(self,
build=None)
build_started(self,
build)
build started
build_aborted(self,
build)
build aborted
build_completed(self,
build)
build completed
get_templates_dirs(self)
Return a list of directories containing the provided template
files.
get_htdocs_dirs(self)
Return the absolute path of a directory containing additional
static resources (such as images, style sheets, etc).
Implements the scheduling of builds for a project.
This module provides the functionality for scheduling builds for a specific
Trac environment. It is used by both the build master and the web interface to
get the list of required builds (revisions not built yet).
Furthermore, the BuildQueue class is used by the build master to determine
the next pending build, and to match build slaves against configured target
platforms.
Classes
BuildQueue
Enapsulates the build queue of an environment.
Functions
collect_changes(repos,
config,
db=None)
Collect all changes for a build configuration that either have already
been built, or still need to be built.
Variables
__package__ = 'bitten'
Function Details
collect_changes(repos,
config,
db=None)
Collect all changes for a build configuration that either have already
been built, or still need to be built.
This function is a generator that yields (platform,rev,build) tuples,
where platform is a TargetPlatform object, rev is the identifier
of the changeset, and build is a Build object or None.
get_build_for_slave(self,
name,
properties)
Check whether one of the pending builds can be built by the build
slave.
match_slave(self,
name,
properties)
Match a build slave against available target platforms.
populate(self)
Add a build for the next change on each build configuration to the
queue.
reset_orphaned_builds(self)
Reset all in-progress builds to PENDING state if they've been
running so long that the configured timeout has been reached.
the allocated build, or None if no build was found
match_slave(self,
name,
properties)
Match a build slave against available target platforms.
Parameters:
name (basestring) - the name of the slave
properties (dict) - the slave configuration
Returns:
the list of platforms the slave matched
populate(self)
Add a build for the next change on each build configuration to the
queue.
The next change is the latest repository check-in for which there isn't
a corresponding build on each target platform. Repeatedly calling this
method will eventually result in the entire change history of the build
configuration being in the build queue.
reset_orphaned_builds(self)
Reset all in-progress builds to PENDING state if they've been
running so long that the configured timeout has been reached.
This is used to cleanup after slaves that have unexpectedly cancelled
a build without notifying the master, or are for some other reason not
reporting back status updates.
TestCoverageAnnotator
Version in the branch should not match:
>>> context = Context.from_request(req, 'source', '/branches/blah/foo.py', 123)
>>> ann.get_annotation_data(context)
[]
Version in the branch should not match:
>>> context = Context.from_request(req, 'source', '/branches/blah/foo.py', 123)
>>> ann.get_annotation_data(context)
[]
Version in the trunk should match:
>>> context = Context.from_request(req, 'source', '/trunk/foo.py', 123)
>>> data = ann.get_annotation_data(context)
>>> print data
[u'5', u'-', u'0']
Automated upgrades for the Bitten database tables, and other data stored
in the Trac environment.
Do not import and call directly!
Functions
parse_scheme(env)
Retrieve the environment database scheme.
update_sequence(env,
db,
tbl,
col)
Update a sequence associated with an autoincrement column.
drop_index(env,
db,
tbl,
idx)
Drop an index associated with a table.
add_log_table(env,
db)
Add a table for storing the builds logs.
add_recipe_to_config(env,
db)
Add a column for storing the build recipe to the build configuration
table.
add_last_activity_to_build(env,
db)
Add a column for storing the last activity to the build table.
add_config_to_reports(env,
db)
Add the name of the build configuration as metadata to report documents
stored in the BDB XML database.
add_order_to_log(env,
db)
Add order column to log table to make sure that build logs are displayed
in the order they were generated.
add_report_tables(env,
db)
Add database tables for report storage.
xmldb_to_db(env,
db)
Migrate report data from Berkeley DB XML to SQL database.
normalize_file_paths(env,
db)
Normalize the file separator in file names in reports.
fixup_generators(env,
db)
Upgrade the identifiers for the recipe commands that generated log
messages and report data.
add_error_table(env,
db)
Add the bitten_error table for recording step failure reasons.
add_filename_to_logs(env,
db)
Add filename column to log table to save where log files are stored.
migrate_logs_to_files(env,
db)
Migrates logs that are stored in the bitten_log_messages table into files.
fix_log_levels_misnaming(env,
db)
Renames or removes *.log.level files created by older versions of migrate_logs_to_files.
remove_stray_log_levels_files(env,
db)
Remove *.log.levels files without a matching *.log file (old Bitten
versions did not delete .log.levels files when builds were deleted)
recreate_rule_with_int_id(env,
db)
Recreates the bitten_rule table with an integer id column rather than a text one.
Migrate report data from Berkeley DB XML to SQL database.
Depending on the number of reports stored, this might take rather long.
After the upgrade is done, the bitten.dbxml file (and any BDB XML log files)
may be deleted. BDB XML is no longer used by Bitten.
add_config_platform_rev_index_to_build(env,
db)
Adds a unique index on (config, platform, rev) to the bitten_build table.
Also drops the old index on bitten_build that serves no real purpose anymore.
fix_sequences(env,
db)
Fixes any auto increment sequences that might have been left in an inconsistent state.
Upgrade scripts for schema versions > 10 should handle sequence updates correctly themselves.
Patched version of Python 2.6's HTTPBasicAuthHandler.
The fix for [1] introduced an infinite recursion bug [2] into
Python 2.6.x that is triggered by attempting to connect using
Basic authentication with a bad username and/or password. This
class fixes the problem using the simple solution outlined in [3].
Support for counting the lines of code in Python programs.
Functions
count(source)
Parse the given file-like object as Python source code.
Variables
BLANK = 0
CODE = 1
COMMENT = 2
DOC = 3
Function Details
count(source)
Parse the given file-like object as Python source code.
For every line in the code, this function yields a (lineno,type,line)
tuple, where lineno is the line number (starting at 0), type is
one of BLANK, CODE, COMMENT or DOC, and line is the actual
content of the line.
Parameters:
source - a file-like object containing Python code
description = 'run unit tests after in-place build, and option...
user_options = [('test-module=', 'm', 'Run \'test_suite\' in s...
Inherited from setuptools.Command:
__doc__,
command_consumes_arguments
Inherited from distutils.cmd.Command:
sub_commands
Method Details
initialize_options(self)
Set default values for all the options that this command
supports. Note that these defaults may be overridden by other
commands, by the setup script, by config files, or by the
command-line. Thus, this is not the place to code dependencies
between options; generally, 'initialize_options()' implementations
are just a bunch of "self.foo = None" assignments.
This method must be implemented by all command classes.
Set final values for all the options that this command supports.
This is always called as late as possible, ie. after any option
assignments from the command-line or from other commands have been
done. Thus, this is the place to code option dependencies: if
'foo' depends on 'bar', then it is safe to set 'foo' from 'bar' as
long as 'foo' still has the same value it was assigned in
'initialize_options()'.
This method must be implemented by all command classes.
Overrides:
distutils.cmd.Command.finalize_options
(inherited documentation)
run_tests(self)
Overrides:
setuptools.command.test.test.run_tests
Class Variable Details
description
Value:
'run unit tests after in-place build, and optionally record code coverage'
user_options
Value:
[('test-module=', 'm', 'Run \'test_suite\' in specified module'),('test-suite=','s','Test suite to run (e.g. \'some_module.test_suite\')'),('xml-output=',
None,'Path to the XML file where test results are written to'),('coverage-dir=', None, 'Directory where coverage files are to be sto...
Text nodes can be nested in an element by using strings instead of elements
in item access. Any special characters in the strings are escaped
automatically:
Finally, text starting with an opening angle bracket is treated specially:
under the assumption that the text actually contains XML itself, the whole
thing is wrapped in a CDATA block instead of escaping all special characters
individually:
bitten.report.coverage.TestCoverageAnnotator:
Version in the branch should not match:
>>> context = Context.from_request(req, 'source', '/branches/blah/foo.py', 123)
>>> ann.get_annotation_data(context)
[]
";
}
return false;
}
function get_anchor() {
var href = location.href;
var start = href.indexOf("#")+1;
if ((start != 0) && (start != href.length))
return href.substring(start, href.length);
}
function redirect_url(dottedName) {
// Scan through each element of the "pages" list, and check
// if "name" matches with any of them.
for (var i=0; i-m" or "-c";
// extract the portion & compare it to dottedName.
var pagename = pages[i].substring(0, pages[i].length-2);
if (pagename == dottedName.substring(0,pagename.length)) {
// We've found a page that matches `dottedName`;
// construct its URL, using leftover `dottedName`
// content to form an anchor.
var pagetype = pages[i].charAt(pages[i].length-1);
var url = pagename + ((pagetype=="m")?"-module.html":
"-class.html");
if (dottedName.length > pagename.length)
url += "#" + dottedName.substring(pagename.length+1,
dottedName.length);
return url;
}
}
}
Bitten-0.6/doc/api/help.html 000644 000765 000120 00000025151 11536424537 016230 0 ustar 00simon admin 000000 000000
Help
This document contains the API (Application Programming Interface)
documentation for Documentation Index. Documentation for the Python
objects defined by the project is divided into separate pages for each
package, module, and class. The API documentation also includes two
pages containing information about the project as a whole: a trees
page, and an index page.
Object Documentation
Each Package Documentation page contains:
A description of the package.
A list of the modules and sub-packages contained by the
package.
A summary of the classes defined by the package.
A summary of the functions defined by the package.
A summary of the variables defined by the package.
A detailed description of each function defined by the
package.
A detailed description of each variable defined by the
package.
Each Module Documentation page contains:
A description of the module.
A summary of the classes defined by the module.
A summary of the functions defined by the module.
A summary of the variables defined by the module.
A detailed description of each function defined by the
module.
A detailed description of each variable defined by the
module.
Each Class Documentation page contains:
A class inheritance diagram.
A list of known subclasses.
A description of the class.
A summary of the methods defined by the class.
A summary of the instance variables defined by the class.
A summary of the class (static) variables defined by the
class.
A detailed description of each method defined by the
class.
A detailed description of each instance variable defined by the
class.
A detailed description of each class (static) variable defined
by the class.
Project Documentation
The Trees page contains the module and class hierarchies:
The module hierarchy lists every package and module, with
modules grouped into packages. At the top level, and within each
package, modules and sub-packages are listed alphabetically.
The class hierarchy lists every class, grouped by base
class. If a class has more than one base class, then it will be
listed under each base class. At the top level, and under each base
class, classes are listed alphabetically.
The Index page contains indices of terms and
identifiers:
The term index lists every term indexed by any object's
documentation. For each term, the index provides links to each
place where the term is indexed.
The identifier index lists the (short) name of every package,
module, class, method, function, variable, and parameter. For each
identifier, the index provides a short description, and a link to
its documentation.
The Table of Contents
The table of contents occupies the two frames on the left side of
the window. The upper-left frame displays the project
contents, and the lower-left frame displays the module
contents:
Project Contents...
API Documentation Frame
Module Contents ...
The project contents frame contains a list of all packages
and modules that are defined by the project. Clicking on an entry
will display its contents in the module contents frame. Clicking on a
special entry, labeled "Everything," will display the contents of
the entire project.
The module contents frame contains a list of every
submodule, class, type, exception, function, and variable defined by a
module or package. Clicking on an entry will display its
documentation in the API documentation frame. Clicking on the name of
the module, at the top of the frame, will display the documentation
for the module itself.
The "frames" and "no frames" buttons below the top
navigation bar can be used to control whether the table of contents is
displayed or not.
The Navigation Bar
A navigation bar is located at the top and bottom of every page.
It indicates what type of page you are currently viewing, and allows
you to go to related pages. The following table describes the labels
on the navigation bar. Note that not some labels (such as
[Parent]) are not displayed on all pages.
Label
Highlighted when...
Links to...
[Parent]
(never highlighted)
the parent of the current package
[Package]
viewing a package
the package containing the current object
[Module]
viewing a module
the module containing the current object
[Class]
viewing a class
the class containing the current object
[Trees]
viewing the trees page
the trees page
[Index]
viewing the index page
the index page
[Help]
viewing the help page
the help page
The "show private" and "hide private" buttons below
the top navigation bar can be used to control whether documentation
for private objects is displayed. Private objects are usually defined
as objects whose (short) names begin with a single underscore, but do
not end with an underscore. For example, "_x",
"__pprint", and "epydoc.epytext._tokenize"
are private objects; but "re.sub",
"__init__", and "type_" are not. However,
if a module defines the "__all__" variable, then its
contents are used to decide which objects are private.
A timestamp below the bottom navigation bar indicates when each
page was last updated.
When javascript is enabled, this page will redirect URLs of
the form redirect.html#dotted.name to the
documentation for the object with the given fully-qualified
dotted name.
Bitten-0.6/Bitten.egg-info/dependency_links.txt 000644 000765 000120 00000000001 11536424600 022044 0 ustar 00simon admin 000000 000000
Bitten-0.6/Bitten.egg-info/entry_points.txt 000644 000765 000120 00000010604 11536424600 021275 0 ustar 00simon admin 000000 000000 [bitten.recipe_commands]
http://bitten.edgewall.org/tools/c#make = bitten.build.ctools:make
http://bitten.cmlenz.net/tools/c#configure = bitten.build.ctools:configure
http://bitten.edgewall.org/tools/php#phing = bitten.build.phptools:phing
http://bitten.edgewall.org/tools/c#cppunit = bitten.build.ctools:cppunit
http://bitten.cmlenz.net/tools/sh#exec = bitten.build.shtools:exec_
http://bitten.edgewall.org/tools/java#ant = bitten.build.javatools:ant
http://bitten.cmlenz.net/tools/svn#checkout = bitten.build.svntools:checkout
http://bitten.cmlenz.net/tools/python#unittest = bitten.build.pythontools:unittest
http://bitten.cmlenz.net/tools/sh#pipe = bitten.build.shtools:pipe
http://bitten.cmlenz.net/tools/python#pylint = bitten.build.pythontools:pylint
http://bitten.edgewall.org/tools/svn#export = bitten.build.svntools:export
http://bitten.edgewall.org/tools/java#cobertura = bitten.build.javatools:cobertura
http://bitten.cmlenz.net/tools/c#cppunit = bitten.build.ctools:cppunit
http://bitten.edgewall.org/tools/c#cunit = bitten.build.ctools:cunit
http://bitten.cmlenz.net/tools/hg#pull = bitten.build.hgtools:pull
http://bitten.cmlenz.net/tools/php#phing = bitten.build.phptools:phing
http://bitten.cmlenz.net/tools/php#coverage = bitten.build.phptools:coverage
http://bitten.cmlenz.net/tools/java#ant = bitten.build.javatools:ant
http://bitten.edgewall.org/tools/mono#nunit = bitten.build.monotools:nunit
http://bitten.cmlenz.net/tools/python#coverage = bitten.build.pythontools:coverage
http://bitten.edgewall.org/tools/python#figleaf = bitten.build.pythontools:figleaf
http://bitten.edgewall.org/tools/xml#transform = bitten.build.xmltools:transform
http://bitten.edgewall.org/tools/php#phpunit = bitten.build.phptools:phpunit
http://bitten.edgewall.org/tools/svn#update = bitten.build.svntools:update
http://bitten.cmlenz.net/tools/c#cunit = bitten.build.ctools:cunit
http://bitten.cmlenz.net/tools/c#make = bitten.build.ctools:make
http://bitten.cmlenz.net/tools/python#figleaf = bitten.build.pythontools:figleaf
http://bitten.edgewall.org/tools/python#coverage = bitten.build.pythontools:coverage
http://bitten.edgewall.org/tools/svn#checkout = bitten.build.svntools:checkout
http://bitten.edgewall.org/tools/c#configure = bitten.build.ctools:configure
http://bitten.edgewall.org/tools/php#coverage = bitten.build.phptools:coverage
http://bitten.cmlenz.net/tools/c#gcov = bitten.build.ctools:gcov
http://bitten.cmlenz.net/tools/c#autoreconf = bitten.build.ctools:autoreconf
http://bitten.cmlenz.net/tools/xml#transform = bitten.build.xmltools:transform
http://bitten.cmlenz.net/tools/python#trace = bitten.build.pythontools:trace
http://bitten.cmlenz.net/tools/java#cobertura = bitten.build.javatools:cobertura
http://bitten.edgewall.org/tools/java#junit = bitten.build.javatools:junit
http://bitten.cmlenz.net/tools/java#junit = bitten.build.javatools:junit
http://bitten.edgewall.org/tools/python#trace = bitten.build.pythontools:trace
http://bitten.edgewall.org/tools/python#exec = bitten.build.pythontools:exec_
http://bitten.cmlenz.net/tools/python#exec = bitten.build.pythontools:exec_
http://bitten.edgewall.org/tools/c#autoreconf = bitten.build.ctools:autoreconf
http://bitten.edgewall.org/tools/sh#pipe = bitten.build.shtools:pipe
http://bitten.cmlenz.net/tools/php#phpunit = bitten.build.phptools:phpunit
http://bitten.edgewall.org/tools/sh#exec = bitten.build.shtools:exec_
http://bitten.cmlenz.net/tools/svn#export = bitten.build.svntools:export
http://bitten.edgewall.org/tools/hg#pull = bitten.build.hgtools:pull
http://bitten.cmlenz.net/tools/python#distutils = bitten.build.pythontools:distutils
http://bitten.cmlenz.net/tools/mono#nunit = bitten.build.monotools:nunit
http://bitten.edgewall.org/tools/c#gcov = bitten.build.ctools:gcov
http://bitten.edgewall.org/tools/python#pylint = bitten.build.pythontools:pylint
http://bitten.edgewall.org/tools/python#unittest = bitten.build.pythontools:unittest
http://bitten.edgewall.org/tools/python#distutils = bitten.build.pythontools:distutils
http://bitten.cmlenz.net/tools/svn#update = bitten.build.svntools:update
[trac.plugins]
bitten.main = bitten.main
bitten.master = bitten.master
bitten.coverage = bitten.report.coverage
bitten.web_ui = bitten.web_ui
bitten.notify = bitten.notify
bitten.admin = bitten.admin
bitten.testing = bitten.report.testing
bitten.lint = bitten.report.lint
[console_scripts]
bitten-slave = bitten.slave:main
[distutils.commands]
unittest = bitten.util.testrunner:unittest
Bitten-0.6/Bitten.egg-info/not-zip-safe 000644 000765 000120 00000000001 11536424455 020234 0 ustar 00simon admin 000000 000000
Bitten-0.6/Bitten.egg-info/PKG-INFO 000644 000765 000120 00000000602 11536424600 017071 0 ustar 00simon admin 000000 000000 Metadata-Version: 1.0
Name: Bitten
Version: 0.6
Summary: Continuous integration for Trac
Home-page: http://bitten.edgewall.org/
Author: Edgewall Software
Author-email: info@edgewall.org
License: BSD
Download-URL: http://bitten.edgewall.org/wiki/Download
Description: A slave for running builds and submitting them to Bitten, the continuous integration system for Trac
Platform: UNKNOWN
Bitten-0.6/Bitten.egg-info/SOURCES.txt 000644 000765 000120 00000015657 11536424600 017700 0 ustar 00simon admin 000000 000000 COPYING
ChangeLog
MANIFEST-SLAVE.in
MANIFEST.in
README.txt
setup.cfg
setup.py
Bitten.egg-info/PKG-INFO
Bitten.egg-info/SOURCES.txt
Bitten.egg-info/dependency_links.txt
Bitten.egg-info/entry_points.txt
Bitten.egg-info/not-zip-safe
Bitten.egg-info/top_level.txt
bitten/__init__.py
bitten/admin.py
bitten/api.py
bitten/main.py
bitten/master.py
bitten/model.py
bitten/notify.py
bitten/queue.py
bitten/recipe.py
bitten/slave.py
bitten/upgrades.py
bitten/web_ui.py
bitten/build/__init__.py
bitten/build/api.py
bitten/build/config.py
bitten/build/ctools.py
bitten/build/hgtools.py
bitten/build/javatools.py
bitten/build/monotools.py
bitten/build/phptools.py
bitten/build/pythontools.py
bitten/build/shtools.py
bitten/build/svntools.py
bitten/build/xmltools.py
bitten/build/tests/__init__.py
bitten/build/tests/api.py
bitten/build/tests/config.py
bitten/build/tests/ctools.py
bitten/build/tests/dummy.py
bitten/build/tests/hgtools.py
bitten/build/tests/javatools.py
bitten/build/tests/monotools.py
bitten/build/tests/phptools.py
bitten/build/tests/pythontools.py
bitten/build/tests/xmltools.py
bitten/htdocs/admin.css
bitten/htdocs/bitten.css
bitten/htdocs/bitten_build.png
bitten/htdocs/bitten_buildf.png
bitten/htdocs/bitten_coverage.css
bitten/htdocs/excanvas.js
bitten/htdocs/failure.png
bitten/htdocs/jquery.flot.js
bitten/htdocs/tabset.js
bitten/report/__init__.py
bitten/report/coverage.py
bitten/report/lint.py
bitten/report/testing.py
bitten/report/tests/__init__.py
bitten/report/tests/coverage.py
bitten/report/tests/lint.py
bitten/report/tests/testing.py
bitten/templates/bitten_admin_configs.html
bitten/templates/bitten_admin_master.html
bitten/templates/bitten_build.html
bitten/templates/bitten_config.html
bitten/templates/bitten_notify_email.txt
bitten/templates/bitten_summary_coverage.html
bitten/templates/bitten_summary_lint.html
bitten/templates/bitten_summary_tests.html
bitten/templates/json.txt
bitten/tests/__init__.py
bitten/tests/admin.py
bitten/tests/master.py
bitten/tests/model.py
bitten/tests/notify.py
bitten/tests/queue.py
bitten/tests/upgrades.py
bitten/tests/web_ui.py
bitten/tests_slave/__init__.py
bitten/tests_slave/recipe.py
bitten/tests_slave/slave.py
bitten/util/__init__.py
bitten/util/compat.py
bitten/util/json.py
bitten/util/loc.py
bitten/util/testrunner.py
bitten/util/xmlio.py
bitten/util/tests/__init__.py
bitten/util/tests/json.py
bitten/util/tests/xmlio.py
doc/commands.html
doc/commands.txt
doc/configure.html
doc/configure.txt
doc/index.html
doc/index.txt
doc/install.html
doc/install.txt
doc/links.html
doc/links.txt
doc/logo.pdf
doc/logo.png
doc/logo_small.png
doc/notify.html
doc/notify.txt
doc/recipes.html
doc/recipes.txt
doc/reports.html
doc/reports.txt
doc/upgrade.html
doc/upgrade.txt
doc/api/api-objects.txt
doc/api/bitten-module.html
doc/api/bitten.admin-module.html
doc/api/bitten.admin.BuildConfigurationsAdminPageProvider-class.html
doc/api/bitten.admin.BuildMasterAdminPageProvider-class.html
doc/api/bitten.api-module.html
doc/api/bitten.api.IBuildListener-class.html
doc/api/bitten.api.ILogFormatter-class.html
doc/api/bitten.api.IReportChartGenerator-class.html
doc/api/bitten.api.IReportSummarizer-class.html
doc/api/bitten.build-module.html
doc/api/bitten.build.api-module.html
doc/api/bitten.build.api.BuildError-class.html
doc/api/bitten.build.api.CommandLine-class.html
doc/api/bitten.build.api.FileSet-class.html
doc/api/bitten.build.api.TimeoutError-class.html
doc/api/bitten.build.config-module.html
doc/api/bitten.build.config.ConfigFileNotFound-class.html
doc/api/bitten.build.config.Configuration-class.html
doc/api/bitten.build.ctools-module.html
doc/api/bitten.build.hgtools-module.html
doc/api/bitten.build.javatools-module.html
doc/api/bitten.build.monotools-module.html
doc/api/bitten.build.phptools-module.html
doc/api/bitten.build.pythontools-module.html
doc/api/bitten.build.shtools-module.html
doc/api/bitten.build.svntools-module.html
doc/api/bitten.build.svntools.Error-class.html
doc/api/bitten.build.xmltools-module.html
doc/api/bitten.main-module.html
doc/api/bitten.main.BuildSystem-class.html
doc/api/bitten.master-module.html
doc/api/bitten.master.BuildMaster-class.html
doc/api/bitten.model-module.html
doc/api/bitten.model.Build-class.html
doc/api/bitten.model.BuildConfig-class.html
doc/api/bitten.model.BuildLog-class.html
doc/api/bitten.model.BuildStep-class.html
doc/api/bitten.model.Report-class.html
doc/api/bitten.model.TargetPlatform-class.html
doc/api/bitten.notify-module.html
doc/api/bitten.notify.BittenNotify-class.html
doc/api/bitten.notify.BuildNotifyEmail-class.html
doc/api/bitten.queue-module.html
doc/api/bitten.queue.BuildQueue-class.html
doc/api/bitten.recipe-module.html
doc/api/bitten.recipe.Context-class.html
doc/api/bitten.recipe.InvalidRecipeError-class.html
doc/api/bitten.recipe.Recipe-class.html
doc/api/bitten.recipe.Step-class.html
doc/api/bitten.report-module.html
doc/api/bitten.report.coverage-module.html
doc/api/bitten.report.coverage.TestCoverageAnnotator-class.html
doc/api/bitten.report.coverage.TestCoverageChartGenerator-class.html
doc/api/bitten.report.coverage.TestCoverageSummarizer-class.html
doc/api/bitten.report.lint-module.html
doc/api/bitten.report.lint.PyLintChartGenerator-class.html
doc/api/bitten.report.lint.PyLintSummarizer-class.html
doc/api/bitten.report.testing-module.html
doc/api/bitten.report.testing.TestResultsChartGenerator-class.html
doc/api/bitten.report.testing.TestResultsSummarizer-class.html
doc/api/bitten.slave-module.html
doc/api/bitten.slave.BuildSlave-class.html
doc/api/bitten.slave.ExitSlave-class.html
doc/api/bitten.upgrades-module.html
doc/api/bitten.util-module.html
doc/api/bitten.util.compat-module.html
doc/api/bitten.util.compat.HTTPBasicAuthHandler-class.html
doc/api/bitten.util.json-module.html
doc/api/bitten.util.loc-module.html
doc/api/bitten.util.testrunner-module.html
doc/api/bitten.util.testrunner.XMLTestResult-class.html
doc/api/bitten.util.testrunner.XMLTestRunner-class.html
doc/api/bitten.util.testrunner.unittest-class.html
doc/api/bitten.util.xmlio-module.html
doc/api/bitten.util.xmlio.Element-class.html
doc/api/bitten.util.xmlio.Fragment-class.html
doc/api/bitten.util.xmlio.ParsedElement-class.html
doc/api/bitten.web_ui-module.html
doc/api/bitten.web_ui.BittenChrome-class.html
doc/api/bitten.web_ui.BuildConfigController-class.html
doc/api/bitten.web_ui.BuildController-class.html
doc/api/bitten.web_ui.ReportChartController-class.html
doc/api/bitten.web_ui.SourceFileLinkFormatter-class.html
doc/api/class-tree.html
doc/api/crarr.png
doc/api/epydoc.css
doc/api/epydoc.js
doc/api/help.html
doc/api/identifier-index.html
doc/api/index.html
doc/api/module-tree.html
doc/api/redirect.html
doc/api/trac.core.ComponentMeta-class.html
doc/common/COPYING
doc/common/README.txt
doc/common/doctools.py
doc/common/template.html
doc/common/conf/docutils.ini
doc/common/conf/epydoc.ini
doc/common/style/bkgnd_pattern.png
doc/common/style/docutils.css
doc/common/style/edgewall.css
doc/common/style/epydoc.css
doc/common/style/pygments.css
doc/common/style/shadow.gif
doc/common/style/vertbars.png Bitten-0.6/Bitten.egg-info/top_level.txt 000644 000765 000120 00000000007 11536424600 020525 0 ustar 00simon admin 000000 000000 bitten
Bitten-0.6/bitten/__init__.py 000644 000765 000120 00000001340 11454420025 016447 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
__docformat__ = 'restructuredtext en'
try:
__version__ = __import__('pkg_resources').get_distribution('Bitten').version
except:
try:
__version__ = __import__('pkg_resources').get_distribution(
'BittenSlave').version
except:
pass
# The master-slave protocol/configuration version
PROTOCOL_VERSION = 5
Bitten-0.6/bitten/admin.py 000644 000765 000120 00000035145 11453043212 016010 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Implementation of the web administration interface."""
from pkg_resources import require, DistributionNotFound
import re
from trac.core import *
from trac.admin import IAdminPanelProvider
from trac.web.chrome import add_stylesheet, add_script, add_warning, add_notice
from bitten.model import BuildConfig, TargetPlatform
from bitten.recipe import Recipe, InvalidRecipeError
from bitten.util import xmlio
class BuildMasterAdminPageProvider(Component):
"""Web administration panel for configuring the build master."""
implements(IAdminPanelProvider)
# IAdminPanelProvider methods
def get_admin_panels(self, req):
if req.perm.has_permission('BUILD_ADMIN'):
yield ('bitten', 'Builds', 'master', 'Master Settings')
def render_admin_panel(self, req, cat, page, path_info):
from bitten.master import BuildMaster
master = BuildMaster(self.env)
if req.method == 'POST':
self._save_config_changes(req, master)
req.redirect(req.abs_href.admin(cat, page))
data = {'master': master}
add_stylesheet(req, 'bitten/admin.css')
return 'bitten_admin_master.html', data
# Internal methods
def _save_config_changes(self, req, master):
changed = False
build_all = 'build_all' in req.args
if build_all != master.build_all:
self.config['bitten'].set('build_all',
build_all and 'yes' or 'no')
changed = True
adjust_timestamps = 'adjust_timestamps' in req.args
if adjust_timestamps != master.adjust_timestamps:
self.config['bitten'].set('adjust_timestamps',
adjust_timestamps and 'yes' or 'no')
changed = True
stabilize_wait = int(req.args.get('stabilize_wait', 0))
if stabilize_wait != master.stabilize_wait:
self.config['bitten'].set('stabilize_wait', str(stabilize_wait))
changed = True
slave_timeout = int(req.args.get('slave_timeout', 0))
if slave_timeout != master.slave_timeout:
self.config['bitten'].set('slave_timeout', str(slave_timeout))
changed = True
quick_status = 'quick_status' in req.args
if quick_status != master.quick_status:
self.config['bitten'].set('quick_status',
quick_status and 'yes' or 'no')
changed = True
logs_dir = req.args.get('logs_dir', None)
if logs_dir != master.logs_dir:
self.config['bitten'].set('logs_dir', str(logs_dir))
changed = True
if changed:
self.config.save()
return master
class BuildConfigurationsAdminPageProvider(Component):
"""Web administration panel for configuring the build master."""
implements(IAdminPanelProvider)
# IAdminPanelProvider methods
def get_admin_panels(self, req):
if req.perm.has_permission('BUILD_MODIFY'):
yield ('bitten', 'Builds', 'configs', 'Configurations')
def render_admin_panel(self, req, cat, page, path_info):
data = {}
# Analyze url
try:
config_name, platform_id = path_info.split('/', 1)
except:
config_name = path_info
platform_id = None
if config_name: # Existing build config
warnings = []
if platform_id or (
# Editing or creating one of the config's target platforms
req.method == 'POST' and 'new' in req.args):
if platform_id: # Editing target platform
platform_id = int(platform_id)
platform = TargetPlatform.fetch(self.env, platform_id)
if req.method == 'POST':
if 'cancel' in req.args or \
self._update_platform(req, platform):
req.redirect(req.abs_href.admin(cat, page,
config_name))
else: # creating target platform
platform = self._create_platform(req, config_name)
req.redirect(req.abs_href.admin(cat, page,
config_name, platform.id))
# Set up template variables
data['platform'] = {
'id': platform.id, 'name': platform.name,
'exists': platform.exists,
'rules': [
{'property': propname, 'pattern': pattern}
for propname, pattern in platform.rules
] or [('', '')]
}
else: # Editing existing build config itself
config = BuildConfig.fetch(self.env, config_name)
platforms = list(TargetPlatform.select(self.env,
config=config.name))
if req.method == 'POST':
if 'remove' in req.args: # Remove selected platforms
self._remove_platforms(req)
add_notice(req, "Target Platform(s) Removed.")
req.redirect(req.abs_href.admin(cat, page, config.name))
elif 'save' in req.args: # Save this build config
warnings = self._update_config(req, config)
if not warnings:
add_notice(req, "Configuration Saved.")
req.redirect(req.abs_href.admin(cat, page, config.name))
for warning in warnings:
add_warning(req, warning)
# FIXME: Deprecation notice for old namespace.
# Remove notice code when migration to new namespace is complete
if 'http://bitten.cmlenz.net/tools/' in config.recipe:
add_notice(req, "Recipe uses a deprecated namespace. "
"Replace 'http://bitten.cmlenz.net/tools/' with "
"'http://bitten.edgewall.org/tools/'.")
# Add a notice if configuration is not active
if not warnings and not config.active and config.recipe:
add_notice(req, "Configuration is not active. Activate "
"from main 'Configurations' listing to enable it.")
# Prepare template variables
data['config'] = {
'name': config.name, 'label': config.label or config.name,
'active': config.active, 'path': config.path,
'min_rev': config.min_rev, 'max_rev': config.max_rev,
'description': config.description,
'recipe': config.recipe,
'platforms': [{
'name': platform.name,
'id': platform.id,
'href': req.href.admin('bitten', 'configs', config.name,
platform.id),
'rules': [{'property': propname, 'pattern': pattern}
for propname, pattern in platform.rules]
} for platform in platforms]
}
else: # At the top level build config list
if req.method == 'POST':
if 'add' in req.args: # Add build config
config = self._create_config(req)
req.redirect(req.abs_href.admin(cat, page, config.name))
elif 'remove' in req.args: # Remove selected build configs
self._remove_configs(req)
elif 'apply' in req.args: # Update active state of configs
self._activate_configs(req)
req.redirect(req.abs_href.admin(cat, page))
# Prepare template variables
configs = []
for config in BuildConfig.select(self.env, include_inactive=True):
configs.append({
'name': config.name, 'label': config.label or config.name,
'active': config.active, 'path': config.path,
'min_rev': config.min_rev, 'max_rev': config.max_rev,
'href': req.href.admin('bitten', 'configs', config.name),
'recipe': config.recipe and True or False
})
data['configs'] = sorted(configs, key=lambda x:x['label'].lower())
add_stylesheet(req, 'bitten/admin.css')
add_script(req, 'common/js/suggest.js')
return 'bitten_admin_configs.html', data
# Internal methods
def _activate_configs(self, req):
req.perm.assert_permission('BUILD_MODIFY')
active = req.args.get('active') or []
active = isinstance(active, list) and active or [active]
db = self.env.get_db_cnx()
for config in list(BuildConfig.select(self.env, db=db,
include_inactive=True)):
config.active = config.name in active
config.update(db=db)
db.commit()
def _create_config(self, req):
req.perm.assert_permission('BUILD_CREATE')
config = BuildConfig(self.env)
warnings = self._update_config(req, config)
if warnings:
if len(warnings) == 1:
raise TracError(warnings[0], 'Add Configuration')
else:
raise TracError('Errors: %s' % ' '.join(warnings),
'Add Configuration')
return config
def _remove_configs(self, req):
req.perm.assert_permission('BUILD_DELETE')
sel = req.args.get('sel')
if not sel:
raise TracError('No configuration selected')
sel = isinstance(sel, list) and sel or [sel]
db = self.env.get_db_cnx()
for name in sel:
config = BuildConfig.fetch(self.env, name, db=db)
if not config:
raise TracError('Configuration %r not found' % name)
config.delete(db=db)
db.commit()
def _update_config(self, req, config):
warnings = []
req.perm.assert_permission('BUILD_MODIFY')
name = req.args.get('name')
if not name:
warnings.append('Missing required field "name".')
if name and not re.match(r'^[\w.-]+$', name):
warnings.append('The field "name" may only contain letters, '
'digits, periods, or dashes.')
repos = self.env.get_repository(authname=req.authname)
if not repos:
warnings.append('No "(default)" Repository: Add a repository or '
'alias named "(default)" to Trac.')
path = req.args.get('path', '')
min_rev = req.args.get('min_rev') or None
max_rev = req.args.get('max_rev') or None
if repos:
path = repos.normalize_path(path)
try:
node = repos.get_node(path, max_rev)
assert node.isdir, '%s is not a directory' % node.path
except (AssertionError, TracError), e:
warnings.append('Invalid Repository Path: "%s" does not exist '
'within the "(default)" repository.' % path)
if min_rev:
try:
repos.get_node(path, min_rev)
except TracError, e:
warnings.append('Invalid Oldest Revision: %s.' % unicode(e))
recipe_xml = req.args.get('recipe', '')
if recipe_xml:
try:
Recipe(xmlio.parse(recipe_xml)).validate()
except xmlio.ParseError, e:
warnings.append('Failure parsing recipe: %s.' % unicode(e))
except InvalidRecipeError, e:
warnings.append('Invalid Recipe: %s.' % unicode(e))
config.name = name
config.path = path
config.recipe = recipe_xml
config.min_rev = min_rev
config.max_rev = max_rev
config.label = req.args.get('label', config.name)
config.description = req.args.get('description', '')
if warnings: # abort
return warnings
if config.exists:
config.update()
else:
config.insert()
return []
def _create_platform(self, req, config_name):
req.perm.assert_permission('BUILD_MODIFY')
name = req.args.get('platform_name')
if not name:
raise TracError('Missing required field "name"', 'Missing field')
platform = TargetPlatform(self.env, config=config_name, name=name)
platform.insert()
return platform
def _remove_platforms(self, req):
req.perm.assert_permission('BUILD_MODIFY')
sel = req.args.get('sel')
if not sel:
raise TracError('No platform selected')
sel = isinstance(sel, list) and sel or [sel]
db = self.env.get_db_cnx()
for platform_id in sel:
platform = TargetPlatform.fetch(self.env, platform_id, db=db)
if not platform:
raise TracError('Target platform %r not found' % platform_id)
platform.delete(db=db)
db.commit()
def _update_platform(self, req, platform):
platform.name = req.args.get('name')
properties = [int(key[9:]) for key in req.args.keys()
if key.startswith('property_')]
properties.sort()
patterns = [int(key[8:]) for key in req.args.keys()
if key.startswith('pattern_')]
patterns.sort()
platform.rules = [(req.args.get('property_%d' % property).strip(),
req.args.get('pattern_%d' % pattern).strip())
for property, pattern in zip(properties, patterns)
if req.args.get('property_%d' % property)]
if platform.exists:
platform.update()
else:
platform.insert()
add_rules = [int(key[9:]) for key in req.args.keys()
if key.startswith('add_rule_')]
if add_rules:
platform.rules.insert(add_rules[0] + 1, ('', ''))
return False
rm_rules = [int(key[8:]) for key in req.args.keys()
if key.startswith('rm_rule_')]
if rm_rules:
if rm_rules[0] < len(platform.rules):
del platform.rules[rm_rules[0]]
return False
return True
Bitten-0.6/bitten/api.py 000644 000765 000120 00000007633 11453043212 015472 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Interfaces of extension points provided by the Bitten Trac plugin."""
from trac.core import *
__all__ = ['IBuildListener', 'ILogFormatter', 'IReportChartGenerator',
'IReportSummarizer']
__docformat__ = 'restructuredtext en'
class IBuildListener(Interface):
"""Extension point interface for components that need to be notified of
build events.
Note that these will be notified in the process running the build master,
not the web interface.
"""
def build_started(build):
"""Called when a build slave has accepted a build initiation.
:param build: the build that was started
:type build: `Build`
"""
def build_aborted(build):
"""Called when a build slave cancels a build or disconnects.
:param build: the build that was aborted
:type build: `Build`
"""
def build_completed(build):
"""Called when a build slave has completed a build, regardless of the
outcome.
:param build: the build that was completed
:type build: `Build`
"""
class ILogFormatter(Interface):
"""Extension point interface for components that format build log
messages."""
def get_formatter(req, build):
"""Return a function that gets called for every log message.
The function must take four positional arguments, ``step``,
``generator``, ``level`` and ``message``, and return the formatted
message as a string.
:param req: the request object
:param build: the build to which the logs belong that should be
formatted
:type build: `Build`
:return: the formatted log message
:rtype: `basestring`
"""
class IReportSummarizer(Interface):
"""Extension point interface for components that render a summary of reports
of some kind."""
def get_supported_categories():
"""Return a list of strings identifying the types of reports this
component supports.
"""
def render_summary(req, config, build, step, category):
"""Render a summary for the given report.
This function should return a tuple of the form ``(template, data)``,
where ``template` is the name of the template to use and ``data`` is
the data to be passed to the template.
:param req: the request object
:param config: the build configuration
:type config: `BuildConfig`
:param build: the build
:type build: `Build`
:param step: the build step
:type step: `BuildStep`
:param category: the category of the report that should be summarized
:type category: `basestring`
"""
class IReportChartGenerator(Interface):
"""Extension point interface for components that generate a chart for a
set of reports."""
def get_supported_categories():
"""Return a list of strings identifying the types of reports this
component supports.
"""
def generate_chart_data(req, config, category):
"""Generate the data for a report chart.
This function should return a tuple of the form ``(template, data)``,
where ``template`` is the name of the template to use and ``data`` is
the data to be passed to the template.
:param req: the request object
:param config: the build configuration
:type config: `BuildConfig`
:param category: the category of reports to include in the chart
:type category: `basestring`
"""
Bitten-0.6/bitten/build/ 000755 000765 000120 00000000000 11536424600 015443 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/htdocs/ 000755 000765 000120 00000000000 11536424600 015630 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/main.py 000644 000765 000120 00000015507 11453043212 015644 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import inspect
import os
import textwrap
from trac.attachment import ILegacyAttachmentPolicyDelegate
from trac.core import *
from trac.db import DatabaseManager
from trac.env import IEnvironmentSetupParticipant
from trac.perm import IPermissionRequestor
from trac.resource import IResourceManager
from trac.util.html import escape
from trac.wiki import IWikiSyntaxProvider
from bitten.api import IBuildListener
from bitten.model import schema, schema_version, Build, BuildConfig
__all__ = ['BuildSystem']
__docformat__ = 'restructuredtext en'
class BuildSetup(Component):
implements(IEnvironmentSetupParticipant)
# IEnvironmentSetupParticipant methods
def environment_created(self):
# Create the required tables
db = self.env.get_db_cnx()
connector, _ = DatabaseManager(self.env)._get_connector()
cursor = db.cursor()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
# Insert a global version flag
cursor.execute("INSERT INTO system (name,value) "
"VALUES ('bitten_version',%s)", (schema_version,))
db.commit()
def environment_needs_upgrade(self, db):
cursor = db.cursor()
cursor.execute("SELECT value FROM system WHERE name='bitten_version'")
row = cursor.fetchone()
if not row or int(row[0]) < schema_version:
return True
def upgrade_environment(self, db):
cursor = db.cursor()
cursor.execute("SELECT value FROM system WHERE name='bitten_version'")
row = cursor.fetchone()
if not row:
self.environment_created()
else:
current_version = int(row[0])
from bitten import upgrades
for version in range(current_version + 1, schema_version + 1):
for function in upgrades.map.get(version):
print textwrap.fill(inspect.getdoc(function))
function(self.env, db)
cursor.execute("UPDATE system SET value=%s WHERE "
"name='bitten_version'", (version,))
db.commit()
print "Bitten upgrade to version %d done." % version
self.log.info('Upgraded Bitten tables to version %d',
version)
class BuildSystem(Component):
implements(IPermissionRequestor,
IWikiSyntaxProvider, IResourceManager,
ILegacyAttachmentPolicyDelegate)
listeners = ExtensionPoint(IBuildListener)
# IPermissionRequestor methods
def get_permission_actions(self):
actions = ['BUILD_VIEW', 'BUILD_CREATE', 'BUILD_MODIFY', 'BUILD_DELETE',
'BUILD_EXEC']
return actions + [('BUILD_ADMIN', actions)]
# IWikiSyntaxProvider methods
def get_wiki_syntax(self):
return []
def get_link_resolvers(self):
def _format_link(formatter, ns, name, label):
segments = name.split('#')
name = segments[0]
step = len(segments) == 2 and segments[1] or ''
try:
name = int(name)
except ValueError:
return label
build = Build.fetch(self.env, name)
if build:
config = BuildConfig.fetch(self.env, build.config)
title = 'Build %d ([%s] of %s) by %s' % (build.id, build.rev,
config.label, build.slave)
if step:
if not step.startswith('step_'):
step = 'step_' + step
step = '#' + escape(step)
return '%s' \
% (formatter.href.build(build.config, build.id) + step,
title, label)
return label
yield 'build', _format_link
# IResourceManager methods
def get_resource_realms(self):
yield 'build'
def get_resource_url(self, resource, href, **kwargs):
config_name, build_id = self._parse_resource(resource.id)
return href.build(config_name, build_id)
def get_resource_description(self, resource, format=None, context=None,
**kwargs):
config_name, build_id = self._parse_resource(resource.id)
config = BuildConfig.fetch(self.env, config_name)
config_label = config and config.label and config.label or config_name
if context:
if build_id:
return tag.a('Build %d ("%s")' % (build_id, config_label),
href=href.build(config_name, build_id))
elif config_name:
return tag.a('Build Configuration "%s"' % config_label,
href=href.build(config_name, build_id))
else:
if build_id:
return 'Build %d ("%s")' % (build_id, config_label)
elif config_name:
return 'Build Configuration "%s"' % config_label
self.log.error("Unknown build/config resource.id: %s" % resource.id)
return 'Unknown Build or Config'
def resource_exists(self, resource):
config_name, build_id = self._parse_resource(resource.id)
if build_id:
build = Build.fetch(self.env, build_id)
return build and build.exists or False
elif config_name:
config = BuildConfig.fetch(self.env, config_name)
return config and config.exists or False
return False
def _parse_resource(self, resource_id):
""" Returns a (config_name, build_id) tuple. """
r = resource_id.split('/', 1)
if len(r) == 1:
return r[0], None
elif len(r) == 2:
try:
return r[0], int(r[1])
except:
return r[0], None
return None, None
# ILegacyAttachmentPolicyDelegate methods
def check_attachment_permission(self, action, username, resource, perm):
""" Respond to the various actions into the legacy attachment
permissions used by the Attachment module. """
if resource.parent.realm == 'build':
if action == 'ATTACHMENT_VIEW':
return 'BUILD_VIEW' in perm(resource.parent)
elif action == 'ATTACHMENT_CREATE':
return 'BUILD_MODIFY' in perm(resource.parent) \
or 'BUILD_CREATE' in perm(resource.parent)
elif action == 'ATTACHMENT_DELETE':
return 'BUILD_DELETE' in perm(resource.parent)
Bitten-0.6/bitten/master.py 000644 000765 000120 00000044373 11457403025 016224 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Build master implementation."""
import calendar
import re
import time
from StringIO import StringIO
from trac.attachment import Attachment
from trac.config import BoolOption, IntOption, Option
from trac.core import *
from trac.resource import ResourceNotFound
from trac.web import IRequestHandler, RequestDone
from bitten import PROTOCOL_VERSION
from bitten.model import BuildConfig, Build, BuildStep, BuildLog, Report, \
TargetPlatform
from bitten.main import BuildSystem
from bitten.queue import BuildQueue
from bitten.recipe import Recipe
from bitten.util import xmlio
__all__ = ['BuildMaster']
__docformat__ = 'restructuredtext en'
HTTP_BAD_REQUEST = 400
HTTP_FORBIDDEN = 403
HTTP_NOT_FOUND = 404
HTTP_METHOD_NOT_ALLOWED = 405
HTTP_CONFLICT = 409
class BuildMaster(Component):
"""Trac request handler implementation for the build master."""
implements(IRequestHandler)
# Configuration options
adjust_timestamps = BoolOption('bitten', 'adjust_timestamps', False, doc=
"""Whether the timestamps of builds should be adjusted to be close
to the timestamps of the corresponding changesets.""")
build_all = BoolOption('bitten', 'build_all', False, doc=
"""Whether to request builds of older revisions even if a younger
revision has already been built.""")
stabilize_wait = IntOption('bitten', 'stabilize_wait', 0, doc=
"""The time in seconds to wait for the repository to stabilize before
queuing up a new build. This allows time for developers to check in
a group of related changes back to back without spawning multiple
builds.""")
slave_timeout = IntOption('bitten', 'slave_timeout', 3600, doc=
"""The time in seconds after which a build is cancelled if the slave
does not report progress.""")
logs_dir = Option('bitten', 'logs_dir', "log/bitten", doc=
"""The directory on the server in which client log files will be stored.""")
quick_status = BoolOption('bitten', 'quick_status', False, doc=
"""Whether to show the current build status within the Trac main
navigation bar. '''Note:''' The feature requires expensive database and
repository checks for every page request, and should not be enabled
if the project has a large repository or uses a non-Subversion
repository such as Mercurial or Git.""")
def __init__(self):
self.env.systeminfo.append(('Bitten',
__import__('bitten', ['__version__']).__version__))
# IRequestHandler methods
def match_request(self, req):
match = re.match(r'/builds(?:/(\d+)(?:/(\w+)/([^/]+)?)?)?$',
req.path_info)
if match:
if match.group(1):
req.args['id'] = match.group(1)
req.args['collection'] = match.group(2)
req.args['member'] = match.group(3)
return True
def process_request(self, req):
req.perm.assert_permission('BUILD_EXEC')
if 'trac_auth' in req.incookie:
slave_token = req.incookie['trac_auth'].value
else:
slave_token = req.session.sid
if 'id' not in req.args:
if req.method != 'POST':
self._send_response(req,
body='Only POST allowed for build creation.')
return self._process_build_creation(req, slave_token)
build = Build.fetch(self.env, req.args['id'])
if not build:
self._send_error(req, HTTP_NOT_FOUND,
'No such build (%s)' % req.args['id'])
build_token = build.slave_info.get('token', '')
if build_token != slave_token:
self._send_error(req, HTTP_CONFLICT,
'Token mismatch (wrong slave): slave=%s, build=%s' \
% (slave_token, build_token))
config = BuildConfig.fetch(self.env, build.config)
if not req.args['collection']:
if req.method == 'DELETE':
return self._process_build_cancellation(req, config, build)
else:
return self._process_build_initiation(req, config, build)
if req.method != 'POST':
self._send_error(req, HTTP_METHOD_NOT_ALLOWED,
'Method %s not allowed' % req.method)
if req.args['collection'] == 'steps':
return self._process_build_step(req, config, build)
elif req.args['collection'] == 'attach':
return self._process_attachment(req, config, build)
elif req.args['collection'] == 'keepalive':
return self._process_keepalive(req, config, build)
else:
self._send_error(req, HTTP_NOT_FOUND,
"No such collection '%s'" % req.args['collection'])
# Internal methods
def _send_response(self, req, code=200, body='', headers=None):
""" Formats and sends the response, raising ``RequestDone``. """
if isinstance(body, unicode):
body = body.encode('utf-8')
req.send_response(code)
headers = headers or {}
headers.setdefault('Content-Length', len(body))
for header in headers:
req.send_header(header, headers[header])
req.write(body)
raise RequestDone
def _send_error(self, req, code=500, message=''):
""" Formats and sends the error, raising ``RequestDone``. """
headers = {'Content-Type': 'text/plain',
'Content-Length': str(len(message))}
self._send_response(req, code, body=message, headers=headers)
def _process_build_creation(self, req, slave_token):
queue = BuildQueue(self.env, build_all=self.build_all,
stabilize_wait=self.stabilize_wait,
timeout=self.slave_timeout)
try:
queue.populate()
except AssertionError, e:
self.log.error(e.message, exc_info=True)
self._send_error(req, HTTP_BAD_REQUEST, e.message)
try:
elem = xmlio.parse(req.read())
except xmlio.ParseError, e:
self.log.error('Error parsing build initialization request: %s', e,
exc_info=True)
self._send_error(req, HTTP_BAD_REQUEST, 'XML parser error')
slave_version = int(elem.attr.get('version', 1))
# FIXME: Remove version compatibility code.
# The initial difference between protocol version 3 and 4 is that
# the master allows keepalive requests-- the master must be
# at least 4 before slaves supporting version 4 are allowed. When
# the first force master/slave upgrade requirement comes in
# (or we bump the) version number again, remove this code.
if slave_version == 3 and PROTOCOL_VERSION == 4:
self.log.info('Allowing slave version %d to process build for '
'compatibility. Upgrade slave to support build '
'keepalives.', slave_version)
elif slave_version != PROTOCOL_VERSION:
self._send_error(req, HTTP_BAD_REQUEST,
"Master-Slave version mismatch: master=%d, slave=%d" % \
(PROTOCOL_VERSION, slave_version))
slavename = elem.attr['name']
properties = {'name': slavename, Build.IP_ADDRESS: req.remote_addr,
Build.TOKEN: slave_token}
self.log.info('Build slave %r connected from %s with token %s',
slavename, req.remote_addr, slave_token)
for child in elem.children():
if child.name == 'platform':
properties[Build.MACHINE] = child.gettext()
properties[Build.PROCESSOR] = child.attr.get('processor')
elif child.name == 'os':
properties[Build.OS_NAME] = child.gettext()
properties[Build.OS_FAMILY] = child.attr.get('family')
properties[Build.OS_VERSION] = child.attr.get('version')
elif child.name == 'package':
for name, value in child.attr.items():
if name == 'name':
continue
properties[child.attr['name'] + '.' + name] = value
self.log.debug('Build slave configuration: %r', properties)
build = queue.get_build_for_slave(slavename, properties)
if not build:
self._send_response(req, 204, '', {})
self._send_response(req, 201, 'Build pending', headers={
'Content-Type': 'text/plain',
'Location': req.abs_href.builds(build.id)})
def _process_build_cancellation(self, req, config, build):
self.log.info('Build slave %r cancelled build %d', build.slave,
build.id)
build.status = Build.PENDING
build.slave = None
build.slave_info = {}
build.started = 0
db = self.env.get_db_cnx()
for step in list(BuildStep.select(self.env, build=build.id, db=db)):
step.delete(db=db)
build.update(db=db)
Attachment.delete_all(self.env, 'build', build.resource.id, db)
db.commit()
for listener in BuildSystem(self.env).listeners:
listener.build_aborted(build)
self._send_response(req, 204, '', {})
def _process_build_initiation(self, req, config, build):
self.log.info('Build slave %r initiated build %d', build.slave,
build.id)
build.started = int(time.time())
build.last_activity = build.started
build.update()
for listener in BuildSystem(self.env).listeners:
listener.build_started(build)
xml = xmlio.parse(config.recipe)
xml.attr['path'] = config.path
xml.attr['revision'] = build.rev
xml.attr['config'] = config.name
xml.attr['build'] = str(build.id)
target_platform = TargetPlatform.fetch(self.env, build.platform)
xml.attr['platform'] = target_platform.name
xml.attr['name'] = build.slave
xml.attr['form_token'] = req.form_token # For posting attachments
body = str(xml)
self.log.info('Build slave %r initiated build %d', build.slave,
build.id)
# create the first step, mark it as in-progress.
recipe = Recipe(xmlio.parse(config.recipe))
stepname = recipe.__iter__().next().id
step = self._start_new_step(build, stepname)
step.insert()
self._send_response(req, 200, body, headers={
'Content-Type': 'application/x-bitten+xml',
'Content-Length': str(len(body)),
'Content-Disposition':
'attachment; filename=recipe_%s_r%s.xml' %
(config.name, build.rev)})
def _process_build_step(self, req, config, build):
try:
elem = xmlio.parse(req.read())
except xmlio.ParseError, e:
self.log.error('Error parsing build step result: %s', e,
exc_info=True)
self._send_error(req, HTTP_BAD_REQUEST, 'XML parser error')
stepname = elem.attr['step']
# we should have created this step previously; if it hasn't,
# the master and slave are processing steps out of order.
step = BuildStep.fetch(self.env, build=build.id, name=stepname)
if not step:
self._send_error(req, HTTP_CONFLICT, 'Build step has not been created.')
recipe = Recipe(xmlio.parse(config.recipe))
index = None
current_step = None
for num, recipe_step in enumerate(recipe):
if recipe_step.id == stepname:
index = num
current_step = recipe_step
if index is None:
self._send_error(req, HTTP_FORBIDDEN,
'No such build step' % stepname)
last_step = index == num
self.log.debug('Slave %s (build %d) completed step %d (%s) with '
'status %s', build.slave, build.id, index, stepname,
elem.attr['status'])
db = self.env.get_db_cnx()
step.stopped = int(time.time())
if elem.attr['status'] == 'failure':
self.log.warning('Build %s step %s failed', build.id, stepname)
step.status = BuildStep.FAILURE
if current_step.onerror == 'fail':
last_step = True
else:
step.status = BuildStep.SUCCESS
step.errors += [error.gettext() for error in elem.children('error')]
# TODO: step.update(db=db)
step.delete(db=db)
step.insert(db=db)
# Collect log messages from the request body
for idx, log_elem in enumerate(elem.children('log')):
build_log = BuildLog(self.env, build=build.id, step=stepname,
generator=log_elem.attr.get('generator'),
orderno=idx)
for message_elem in log_elem.children('message'):
build_log.messages.append((message_elem.attr['level'],
message_elem.gettext()))
build_log.insert(db=db)
# Collect report data from the request body
for report_elem in elem.children('report'):
report = Report(self.env, build=build.id, step=stepname,
category=report_elem.attr.get('category'),
generator=report_elem.attr.get('generator'))
for item_elem in report_elem.children():
item = {'type': item_elem.name}
item.update(item_elem.attr)
for child_elem in item_elem.children():
item[child_elem.name] = child_elem.gettext()
report.items.append(item)
report.insert(db=db)
# If this was the last step in the recipe we mark the build as
# completed otherwise just update last_activity
if last_step:
self.log.info('Slave %s completed build %d ("%s" as of [%s])',
build.slave, build.id, build.config, build.rev)
build.stopped = step.stopped
build.last_activity = build.stopped
# Determine overall outcome of the build by checking the outcome
# of the individual steps against the "onerror" specification of
# each step in the recipe
for num, recipe_step in enumerate(recipe):
step = BuildStep.fetch(self.env, build.id, recipe_step.id)
if step.status == BuildStep.FAILURE:
if recipe_step.onerror == 'fail' or \
recipe_step.onerror == 'continue':
build.status = Build.FAILURE
break
else:
build.status = Build.SUCCESS
build.update(db=db)
else:
build.last_activity = step.stopped
build.update(db=db)
# start the next step.
for num, recipe_step in enumerate(recipe):
if num == index + 1:
next_step = recipe_step
if next_step is None:
self._send_error(req, HTTP_FORBIDDEN,
'Unable to find step after ' % stepname)
step = self._start_new_step(build, next_step.id)
step.insert(db=db)
db.commit()
if last_step:
for listener in BuildSystem(self.env).listeners:
listener.build_completed(build)
body = 'Build step processed'
self._send_response(req, 201, body, {
'Content-Type': 'text/plain',
'Content-Length': str(len(body)),
'Location': req.abs_href.builds(
build.id, 'steps', stepname)})
def _process_attachment(self, req, config, build):
resource_id = req.args['member'] == 'config' \
and build.config or build.resource.id
upload = req.args['file']
if not upload.file:
send_error(req, message="Attachment not received.")
self.log.debug('Received attachment %s for attaching to build:%s',
upload.filename, resource_id)
# Determine size of file
upload.file.seek(0, 2) # to the end
size = upload.file.tell()
upload.file.seek(0) # beginning again
# Delete attachment if it already exists
try:
old_attach = Attachment(self.env, 'build',
parent_id=resource_id, filename=upload.filename)
old_attach.delete()
except ResourceNotFound:
pass
# Save new attachment
attachment = Attachment(self.env, 'build', parent_id=resource_id)
attachment.description = req.args.get('description', '')
attachment.author = req.authname
attachment.insert(upload.filename, upload.file, size)
self._send_response(req, 201, 'Attachment created', headers={
'Content-Type': 'text/plain',
'Content-Length': str(len('Attachment created'))})
def _process_keepalive(self, req, config, build):
build.last_activity = int(time.time())
build.update()
self.log.info('Slave %s build %d keepalive ("%s" as of [%s])',
build.slave, build.id, build.config, build.rev)
body = 'Keepalive processed'
self._send_response(req, 200, body, {
'Content-Type': 'text/plain',
'Content-Length': str(len(body))})
def _start_new_step(self, build, stepname):
"""Creates the in-memory representation for a newly started
step, ready to be persisted to the database.
"""
step = BuildStep(self.env, build=build.id, name=stepname)
step.status = BuildStep.IN_PROGRESS
step.started = int(time.time())
step.stopped = 0
return step
Bitten-0.6/bitten/model.py 000644 000765 000120 00000114171 11453043212 016015 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Model classes for objects persisted in the database."""
from trac.attachment import Attachment
from trac.db import Table, Column, Index
from trac.resource import Resource
from trac.util.text import to_unicode
from trac.util.datefmt import to_timestamp, utcmin, utcmax
from datetime import datetime
import codecs
import os
__docformat__ = 'restructuredtext en'
class BuildConfig(object):
"""Representation of a build configuration."""
_schema = [
Table('bitten_config', key='name')[
Column('name'), Column('path'), Column('active', type='int'),
Column('recipe'), Column('min_rev'), Column('max_rev'),
Column('label'), Column('description')
]
]
def __init__(self, env, name=None, path=None, active=False, recipe=None,
min_rev=None, max_rev=None, label=None, description=None):
"""Initialize a new build configuration with the specified attributes.
To actually create this configuration in the database, the `insert`
method needs to be called.
"""
self.env = env
self._old_name = None
self.name = name
self.path = path or ''
self.active = bool(active)
self.recipe = recipe or ''
self.min_rev = min_rev or None
self.max_rev = max_rev or None
self.label = label or ''
self.description = description or ''
def __repr__(self):
return '<%s %r>' % (type(self).__name__, self.name)
exists = property(fget=lambda self: self._old_name is not None,
doc='Whether this configuration exists in the database')
resource = property(fget=lambda self: Resource('build', '%s' % self.name),
doc='Build Config resource identification')
def delete(self, db=None):
"""Remove a build configuration and all dependent objects from the
database."""
assert self.exists, 'Cannot delete non-existing configuration'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
for platform in list(TargetPlatform.select(self.env, self.name, db=db)):
platform.delete(db=db)
for build in list(Build.select(self.env, config=self.name, db=db)):
build.delete(db=db)
# Delete attachments
Attachment.delete_all(self.env, 'build', self.resource.id, db)
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_config WHERE name=%s", (self.name,))
if handle_ta:
db.commit()
self._old_name = None
def insert(self, db=None):
"""Insert a new configuration into the database."""
assert not self.exists, 'Cannot insert existing configuration'
assert self.name, 'Configuration requires a name'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,active,"
"recipe,min_rev,max_rev,label,description) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s)",
(self.name, self.path, int(self.active or 0),
self.recipe or '', self.min_rev, self.max_rev,
self.label or '', self.description or ''))
if handle_ta:
db.commit()
self._old_name = self.name
def update(self, db=None):
"""Save changes to an existing build configuration."""
assert self.exists, 'Cannot update a non-existing configuration'
assert self.name, 'Configuration requires a name'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
cursor = db.cursor()
cursor.execute("UPDATE bitten_config SET name=%s,path=%s,active=%s,"
"recipe=%s,min_rev=%s,max_rev=%s,label=%s,"
"description=%s WHERE name=%s",
(self.name, self.path, int(self.active or 0),
self.recipe, self.min_rev, self.max_rev,
self.label, self.description, self._old_name))
if self.name != self._old_name:
cursor.execute("UPDATE bitten_platform SET config=%s "
"WHERE config=%s", (self.name, self._old_name))
cursor.execute("UPDATE bitten_build SET config=%s "
"WHERE config=%s", (self.name, self._old_name))
if handle_ta:
db.commit()
self._old_name = self.name
def fetch(cls, env, name, db=None):
"""Retrieve an existing build configuration from the database by
name.
"""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT path,active,recipe,min_rev,max_rev,label,"
"description FROM bitten_config WHERE name=%s", (name,))
row = cursor.fetchone()
if not row:
return None
config = BuildConfig(env)
config.name = config._old_name = name
config.path = row[0] or ''
config.active = bool(row[1])
config.recipe = row[2] or ''
config.min_rev = row[3] or None
config.max_rev = row[4] or None
config.label = row[5] or ''
config.description = row[6] or ''
return config
fetch = classmethod(fetch)
def select(cls, env, include_inactive=False, db=None):
"""Retrieve existing build configurations from the database that match
the specified criteria.
"""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
if include_inactive:
cursor.execute("SELECT name,path,active,recipe,min_rev,max_rev,"
"label,description FROM bitten_config ORDER BY name")
else:
cursor.execute("SELECT name,path,active,recipe,min_rev,max_rev,"
"label,description FROM bitten_config "
"WHERE active=1 ORDER BY name")
for name, path, active, recipe, min_rev, max_rev, label, description \
in cursor:
config = BuildConfig(env, name=name, path=path or '',
active=bool(active), recipe=recipe or '',
min_rev=min_rev or None,
max_rev=max_rev or None, label=label or '',
description=description or '')
config._old_name = name
yield config
select = classmethod(select)
def min_rev_time(self, env):
"""Returns the time of the minimum revision being built for this
configuration. Returns utcmin if not specified.
"""
repos = env.get_repository()
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
min_time = utcmin
if self.min_rev:
min_time = repos.get_changeset(self.min_rev).date
if isinstance(min_time, datetime): # Trac>=0.11
min_time = to_timestamp(min_time)
return min_time
def max_rev_time(self, env):
"""Returns the time of the maximum revision being built for this
configuration. Returns utcmax if not specified.
"""
repos = env.get_repository()
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
max_time = utcmax
if self.max_rev:
max_time = repos.get_changeset(self.max_rev).date
if isinstance(max_time, datetime): # Trac>=0.11
max_time = to_timestamp(max_time)
return max_time
class TargetPlatform(object):
"""Target platform for a build configuration."""
_schema = [
Table('bitten_platform', key='id')[
Column('id', auto_increment=True), Column('config'), Column('name')
],
Table('bitten_rule', key=('id', 'propname'))[
Column('id', type='int'), Column('propname'), Column('pattern'),
Column('orderno', type='int')
]
]
def __init__(self, env, config=None, name=None):
"""Initialize a new target platform with the specified attributes.
To actually create this platform in the database, the `insert` method
needs to be called.
"""
self.env = env
self.id = None
self.config = config
self.name = name
self.rules = []
def __repr__(self):
return '<%s %r>' % (type(self).__name__, self.id)
exists = property(fget=lambda self: self.id is not None,
doc='Whether this target platform exists in the database')
def delete(self, db=None):
"""Remove the target platform from the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
for build in Build.select(self.env, platform=self.id, status=Build.PENDING, db=db):
build.delete()
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_rule WHERE id=%s", (self.id,))
cursor.execute("DELETE FROM bitten_platform WHERE id=%s", (self.id,))
if handle_ta:
db.commit()
def insert(self, db=None):
"""Insert a new target platform into the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert not self.exists, 'Cannot insert existing target platform'
assert self.config, 'Target platform needs to be associated with a ' \
'configuration'
assert self.name, 'Target platform requires a name'
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_platform (config,name) "
"VALUES (%s,%s)", (self.config, self.name))
self.id = db.get_last_id(cursor, 'bitten_platform')
if self.rules:
cursor.executemany("INSERT INTO bitten_rule VALUES (%s,%s,%s,%s)",
[(self.id, propname, pattern, idx)
for idx, (propname, pattern)
in enumerate(self.rules)])
if handle_ta:
db.commit()
def update(self, db=None):
"""Save changes to an existing target platform."""
assert self.exists, 'Cannot update a non-existing platform'
assert self.config, 'Target platform needs to be associated with a ' \
'configuration'
assert self.name, 'Target platform requires a name'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
cursor = db.cursor()
cursor.execute("UPDATE bitten_platform SET name=%s WHERE id=%s",
(self.name, self.id))
cursor.execute("DELETE FROM bitten_rule WHERE id=%s", (self.id,))
if self.rules:
cursor.executemany("INSERT INTO bitten_rule VALUES (%s,%s,%s,%s)",
[(self.id, propname, pattern, idx)
for idx, (propname, pattern)
in enumerate(self.rules)])
if handle_ta:
db.commit()
def fetch(cls, env, id, db=None):
"""Retrieve an existing target platform from the database by ID."""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT config,name FROM bitten_platform "
"WHERE id=%s", (id,))
row = cursor.fetchone()
if not row:
return None
platform = TargetPlatform(env, config=row[0], name=row[1])
platform.id = id
cursor.execute("SELECT propname,pattern FROM bitten_rule "
"WHERE id=%s ORDER BY orderno", (id,))
for propname, pattern in cursor:
platform.rules.append((propname, pattern))
return platform
fetch = classmethod(fetch)
def select(cls, env, config=None, db=None):
"""Retrieve existing target platforms from the database that match the
specified criteria.
"""
if not db:
db = env.get_db_cnx()
where_clauses = []
if config is not None:
where_clauses.append(("config=%s", config))
if where_clauses:
where = "WHERE " + " AND ".join([wc[0] for wc in where_clauses])
else:
where = ""
cursor = db.cursor()
cursor.execute("SELECT id FROM bitten_platform %s ORDER BY name"
% where, [wc[1] for wc in where_clauses])
for (id,) in cursor:
yield TargetPlatform.fetch(env, id)
select = classmethod(select)
class Build(object):
"""Representation of a build."""
_schema = [
Table('bitten_build', key='id')[
Column('id', auto_increment=True), Column('config'), Column('rev'),
Column('rev_time', type='int'), Column('platform', type='int'),
Column('slave'), Column('started', type='int'),
Column('stopped', type='int'), Column('status', size=1),
Column('last_activity', type='int'),
Index(['config', 'rev', 'platform'], unique=True)
],
Table('bitten_slave', key=('build', 'propname'))[
Column('build', type='int'), Column('propname'), Column('propvalue')
]
]
# Build status codes
PENDING = 'P'
IN_PROGRESS = 'I'
SUCCESS = 'S'
FAILURE = 'F'
# Standard slave properties
IP_ADDRESS = 'ipnr'
MAINTAINER = 'owner'
OS_NAME = 'os'
OS_FAMILY = 'family'
OS_VERSION = 'version'
MACHINE = 'machine'
PROCESSOR = 'processor'
TOKEN = 'token'
def __init__(self, env, config=None, rev=None, platform=None, slave=None,
started=0, stopped=0, last_activity=0,
rev_time=0, status=PENDING):
"""Initialize a new build with the specified attributes.
To actually create this build in the database, the `insert` method needs
to be called.
"""
self.env = env
self.id = None
self.config = config
self.rev = rev and str(rev) or None
self.platform = platform
self.slave = slave
self.started = started or 0
self.stopped = stopped or 0
self.last_activity = last_activity or 0
self.rev_time = rev_time
self.status = status
self.slave_info = {}
def __repr__(self):
return '<%s %r>' % (type(self).__name__, self.id)
exists = property(fget=lambda self: self.id is not None,
doc='Whether this build exists in the database')
completed = property(fget=lambda self: self.status != Build.IN_PROGRESS,
doc='Whether the build has been completed')
successful = property(fget=lambda self: self.status == Build.SUCCESS,
doc='Whether the build was successful')
resource = property(fget=lambda self: Resource('build', '%s/%s' % (self.config, self.id)),
doc='Build resource identification')
def delete(self, db=None):
"""Remove the build from the database."""
assert self.exists, 'Cannot delete a non-existing build'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
for step in list(BuildStep.select(self.env, build=self.id)):
step.delete(db=db)
# Delete attachments
Attachment.delete_all(self.env, 'build', self.resource.id, db)
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_slave WHERE build=%s", (self.id,))
cursor.execute("DELETE FROM bitten_build WHERE id=%s", (self.id,))
if handle_ta:
db.commit()
def insert(self, db=None):
"""Insert a new build into the database."""
assert not self.exists, 'Cannot insert an existing build'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert self.config and self.rev and self.rev_time and self.platform
assert self.status in (self.PENDING, self.IN_PROGRESS, self.SUCCESS,
self.FAILURE)
if not self.slave:
assert self.status == self.PENDING
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_build (config,rev,rev_time,platform,"
"slave,started,stopped,last_activity,status) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)",
(self.config, self.rev, int(self.rev_time),
self.platform, self.slave or '', self.started or 0,
self.stopped or 0, self.last_activity or 0,
self.status))
self.id = db.get_last_id(cursor, 'bitten_build')
if self.slave_info:
cursor.executemany("INSERT INTO bitten_slave VALUES (%s,%s,%s)",
[(self.id, name, value) for name, value
in self.slave_info.items()])
if handle_ta:
db.commit()
def update(self, db=None):
"""Save changes to an existing build."""
assert self.exists, 'Cannot update a non-existing build'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert self.config and self.rev
assert self.status in (self.PENDING, self.IN_PROGRESS, self.SUCCESS,
self.FAILURE)
if not self.slave:
assert self.status == self.PENDING
cursor = db.cursor()
cursor.execute("UPDATE bitten_build SET slave=%s,started=%s,"
"stopped=%s,last_activity=%s,status=%s WHERE id=%s",
(self.slave or '', self.started or 0,
self.stopped or 0, self.last_activity or 0,
self.status, self.id))
cursor.execute("DELETE FROM bitten_slave WHERE build=%s", (self.id,))
if self.slave_info:
cursor.executemany("INSERT INTO bitten_slave VALUES (%s,%s,%s)",
[(self.id, name, value) for name, value
in self.slave_info.items()])
if handle_ta:
db.commit()
def fetch(cls, env, id, db=None):
"""Retrieve an existing build from the database by ID."""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT config,rev,rev_time,platform,slave,started,"
"stopped,last_activity,status FROM bitten_build WHERE "
"id=%s", (id,))
row = cursor.fetchone()
if not row:
return None
build = Build(env, config=row[0], rev=row[1], rev_time=int(row[2]),
platform=int(row[3]), slave=row[4],
started=row[5] and int(row[5]) or 0,
stopped=row[6] and int(row[6]) or 0,
last_activity=row[7] and int(row[7]) or 0,
status=row[8])
build.id = int(id)
cursor.execute("SELECT propname,propvalue FROM bitten_slave "
"WHERE build=%s", (id,))
for propname, propvalue in cursor:
build.slave_info[propname] = propvalue
return build
fetch = classmethod(fetch)
def select(cls, env, config=None, rev=None, platform=None, slave=None,
status=None, db=None, min_rev_time=None, max_rev_time=None):
"""Retrieve existing builds from the database that match the specified
criteria.
"""
if not db:
db = env.get_db_cnx()
where_clauses = []
if config is not None:
where_clauses.append(("config=%s", config))
if rev is not None:
where_clauses.append(("rev=%s", str(rev)))
if platform is not None:
where_clauses.append(("platform=%s", platform))
if slave is not None:
where_clauses.append(("slave=%s", slave))
if status is not None:
where_clauses.append(("status=%s", status))
if min_rev_time is not None:
where_clauses.append(("rev_time>=%s", min_rev_time))
if max_rev_time is not None:
where_clauses.append(("rev_time<=%s", max_rev_time))
if where_clauses:
where = "WHERE " + " AND ".join([wc[0] for wc in where_clauses])
else:
where = ""
cursor = db.cursor()
cursor.execute("SELECT id FROM bitten_build %s "
"ORDER BY rev_time DESC,config,slave"
% where, [wc[1] for wc in where_clauses])
for (id,) in cursor:
yield Build.fetch(env, id)
select = classmethod(select)
class BuildStep(object):
"""Represents an individual step of an executed build."""
_schema = [
Table('bitten_step', key=('build', 'name'))[
Column('build', type='int'), Column('name'), Column('description'),
Column('status', size=1), Column('started', type='int'),
Column('stopped', type='int')
],
Table('bitten_error', key=('build', 'step', 'orderno'))[
Column('build', type='int'), Column('step'), Column('message'),
Column('orderno', type='int')
]
]
# Step status codes
SUCCESS = 'S'
IN_PROGRESS = 'I'
FAILURE = 'F'
def __init__(self, env, build=None, name=None, description=None,
status=None, started=None, stopped=None):
"""Initialize a new build step with the specified attributes.
To actually create this build step in the database, the `insert` method
needs to be called.
"""
self.env = env
self.build = build
self.name = name
self.description = description
self.status = status
self.started = started
self.stopped = stopped
self.errors = []
self._exists = False
exists = property(fget=lambda self: self._exists,
doc='Whether this build step exists in the database')
successful = property(fget=lambda self: self.status == BuildStep.SUCCESS,
doc='Whether the build step was successful')
completed = property(fget=lambda self: self.status == BuildStep.SUCCESS or self.status == BuildStep.FAILURE,
doc='Whether this build step has completed processing')
def delete(self, db=None):
"""Remove the build step from the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
for log in list(BuildLog.select(self.env, build=self.build,
step=self.name, db=db)):
log.delete(db=db)
for report in list(Report.select(self.env, build=self.build,
step=self.name, db=db)):
report.delete(db=db)
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_step WHERE build=%s AND name=%s",
(self.build, self.name))
cursor.execute("DELETE FROM bitten_error WHERE build=%s AND step=%s",
(self.build, self.name))
if handle_ta:
db.commit()
self._exists = False
def insert(self, db=None):
"""Insert a new build step into the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert self.build and self.name
assert self.status in (self.SUCCESS, self.IN_PROGRESS, self.FAILURE)
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_step (build,name,description,status,"
"started,stopped) VALUES (%s,%s,%s,%s,%s,%s)",
(self.build, self.name, self.description or '',
self.status, self.started or 0, self.stopped or 0))
if self.errors:
cursor.executemany("INSERT INTO bitten_error (build,step,message,"
"orderno) VALUES (%s,%s,%s,%s)",
[(self.build, self.name, message, idx)
for idx, message in enumerate(self.errors)])
if handle_ta:
db.commit()
self._exists = True
def fetch(cls, env, build, name, db=None):
"""Retrieve an existing build from the database by build ID and step
name."""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT description,status,started,stopped "
"FROM bitten_step WHERE build=%s AND name=%s",
(build, name))
row = cursor.fetchone()
if not row:
return None
step = BuildStep(env, build, name, row[0] or '', row[1],
row[2] and int(row[2]), row[3] and int(row[3]))
step._exists = True
cursor.execute("SELECT message FROM bitten_error WHERE build=%s "
"AND step=%s ORDER BY orderno", (build, name))
for row in cursor:
step.errors.append(row[0] or '')
return step
fetch = classmethod(fetch)
def select(cls, env, build=None, name=None, status=None, db=None):
"""Retrieve existing build steps from the database that match the
specified criteria.
"""
if not db:
db = env.get_db_cnx()
assert status in (None, BuildStep.SUCCESS, BuildStep.IN_PROGRESS, BuildStep.FAILURE)
where_clauses = []
if build is not None:
where_clauses.append(("build=%s", build))
if name is not None:
where_clauses.append(("name=%s", name))
if status is not None:
where_clauses.append(("status=%s", status))
if where_clauses:
where = "WHERE " + " AND ".join([wc[0] for wc in where_clauses])
else:
where = ""
cursor = db.cursor()
cursor.execute("SELECT build,name FROM bitten_step %s ORDER BY started"
% where, [wc[1] for wc in where_clauses])
for build, name in cursor:
yield BuildStep.fetch(env, build, name, db=db)
select = classmethod(select)
class BuildLog(object):
"""Represents a build log."""
_schema = [
Table('bitten_log', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('generator'), Column('orderno', type='int'),
Column('filename'),
Index(['build', 'step'])
],
]
# Message levels
DEBUG = 'D'
INFO = 'I'
WARNING = 'W'
ERROR = 'E'
UNKNOWN = ''
LEVELS_SUFFIX = '.levels'
def __init__(self, env, build=None, step=None, generator=None,
orderno=None, filename=None):
"""Initialize a new build log with the specified attributes.
To actually create this build log in the database, the `insert` method
needs to be called.
"""
self.env = env
self.id = None
self.build = build
self.step = step
self.generator = generator or ''
self.orderno = orderno and int(orderno) or 0
self.filename = filename or None
self.messages = []
self.logs_dir = env.config.get('bitten', 'logs_dir', 'log/bitten')
if not os.path.isabs(self.logs_dir):
self.logs_dir = os.path.join(env.path, self.logs_dir)
if not os.path.exists(self.logs_dir):
os.makedirs(self.logs_dir)
exists = property(fget=lambda self: self.id is not None,
doc='Whether this build log exists in the database')
def get_log_file(self, filename):
"""Returns the full path to the log file"""
if filename != os.path.basename(filename):
raise ValueError("Filename may not contain path: %s" % (filename,))
return os.path.join(self.logs_dir, filename)
def delete(self, db=None):
"""Remove the build log from the database."""
assert self.exists, 'Cannot delete a non-existing build log'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
if self.filename:
log_file = self.get_log_file(self.filename)
if os.path.exists(log_file):
try:
self.env.log.debug("Deleting log file: %s" % log_file)
os.remove(log_file)
except Exception, e:
self.env.log.warning("Error removing log file %s: %s" % (log_file, e))
level_file = log_file + self.LEVELS_SUFFIX
if os.path.exists(level_file):
try:
self.env.log.debug("Deleting level file: %s" % level_file)
os.remove(level_file)
except Exception, e:
self.env.log.warning("Error removing level file %s: %s" \
% (level_file, e))
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_log WHERE id=%s", (self.id,))
if handle_ta:
db.commit()
self.id = None
def insert(self, db=None):
"""Insert a new build log into the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert self.build and self.step
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_log (build,step,generator,orderno) "
"VALUES (%s,%s,%s,%s)", (self.build, self.step,
self.generator, self.orderno))
id = db.get_last_id(cursor, 'bitten_log')
log_file = "%s.log" % (id,)
cursor.execute("UPDATE bitten_log SET filename=%s WHERE id=%s", (log_file, id))
if self.messages:
log_file_name = self.get_log_file(log_file)
level_file_name = log_file_name + self.LEVELS_SUFFIX
codecs.open(log_file_name, "wb", "UTF-8").writelines([to_unicode(msg[1]+"\n") for msg in self.messages])
codecs.open(level_file_name, "wb", "UTF-8").writelines([to_unicode(msg[0]+"\n") for msg in self.messages])
if handle_ta:
db.commit()
self.id = id
def fetch(cls, env, id, db=None):
"""Retrieve an existing build from the database by ID."""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,generator,orderno,filename FROM bitten_log "
"WHERE id=%s", (id,))
row = cursor.fetchone()
if not row:
return None
log = BuildLog(env, int(row[0]), row[1], row[2], row[3], row[4])
log.id = id
if log.filename:
log_filename = log.get_log_file(log.filename)
if os.path.exists(log_filename):
log_lines = codecs.open(log_filename, "rb", "UTF-8").readlines()
else:
log_lines = []
level_filename = log.get_log_file(log.filename + cls.LEVELS_SUFFIX)
if os.path.exists(level_filename):
log_levels = dict(enumerate(codecs.open(level_filename, "rb", "UTF-8").readlines()))
else:
log_levels = {}
log.messages = [(log_levels.get(line_num, BuildLog.UNKNOWN).rstrip("\n"), line.rstrip("\n")) for line_num, line in enumerate(log_lines)]
else:
log.messages = []
return log
fetch = classmethod(fetch)
def select(cls, env, build=None, step=None, generator=None, db=None):
"""Retrieve existing build logs from the database that match the
specified criteria.
"""
if not db:
db = env.get_db_cnx()
where_clauses = []
if build is not None:
where_clauses.append(("build=%s", build))
if step is not None:
where_clauses.append(("step=%s", step))
if generator is not None:
where_clauses.append(("generator=%s", generator))
if where_clauses:
where = "WHERE " + " AND ".join([wc[0] for wc in where_clauses])
else:
where = ""
cursor = db.cursor()
cursor.execute("SELECT id FROM bitten_log %s ORDER BY orderno"
% where, [wc[1] for wc in where_clauses])
for (id, ) in cursor:
yield BuildLog.fetch(env, id, db=db)
select = classmethod(select)
class Report(object):
"""Represents a generated report."""
_schema = [
Table('bitten_report', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('category'), Column('generator'),
Index(['build', 'step', 'category'])
],
Table('bitten_report_item', key=('report', 'item', 'name'))[
Column('report', type='int'), Column('item', type='int'),
Column('name'), Column('value')
]
]
def __init__(self, env, build=None, step=None, category=None,
generator=None):
"""Initialize a new report with the specified attributes.
To actually create this build log in the database, the `insert` method
needs to be called.
"""
self.env = env
self.id = None
self.build = build
self.step = step
self.category = category
self.generator = generator or ''
self.items = []
exists = property(fget=lambda self: self.id is not None,
doc='Whether this report exists in the database')
def delete(self, db=None):
"""Remove the report from the database."""
assert self.exists, 'Cannot delete a non-existing report'
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
cursor = db.cursor()
cursor.execute("DELETE FROM bitten_report_item WHERE report=%s",
(self.id,))
cursor.execute("DELETE FROM bitten_report WHERE id=%s", (self.id,))
if handle_ta:
db.commit()
self.id = None
def insert(self, db=None):
"""Insert a new build log into the database."""
if not db:
db = self.env.get_db_cnx()
handle_ta = True
else:
handle_ta = False
assert self.build and self.step and self.category
# Enforce uniqueness of build-step-category.
# This should be done by the database, but the DB schema helpers in Trac
# currently don't support UNIQUE() constraints
assert not list(Report.select(self.env, build=self.build,
step=self.step, category=self.category,
db=db)), 'Report already exists'
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(self.build, self.step, self.category, self.generator))
id = db.get_last_id(cursor, 'bitten_report')
for idx, item in enumerate([item for item in self.items if item]):
cursor.executemany("INSERT INTO bitten_report_item "
"(report,item,name,value) VALUES (%s,%s,%s,%s)",
[(id, idx, key, value) for key, value
in item.items()])
if handle_ta:
db.commit()
self.id = id
def fetch(cls, env, id, db=None):
"""Retrieve an existing build from the database by ID."""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,category,generator "
"FROM bitten_report WHERE id=%s", (id,))
row = cursor.fetchone()
if not row:
return None
report = Report(env, int(row[0]), row[1], row[2] or '', row[3] or '')
report.id = id
cursor.execute("SELECT item,name,value FROM bitten_report_item "
"WHERE report=%s ORDER BY item", (id,))
items = {}
for item, name, value in cursor:
items.setdefault(item, {})[name] = value
report.items = items.values()
return report
fetch = classmethod(fetch)
def select(cls, env, config=None, build=None, step=None, category=None,
db=None):
"""Retrieve existing reports from the database that match the specified
criteria.
"""
where_clauses = []
joins = []
if config is not None:
where_clauses.append(("config=%s", config))
joins.append("INNER JOIN bitten_build ON (bitten_build.id=build)")
if build is not None:
where_clauses.append(("build=%s", build))
if step is not None:
where_clauses.append(("step=%s", step))
if category is not None:
where_clauses.append(("category=%s", category))
if where_clauses:
where = "WHERE " + " AND ".join([wc[0] for wc in where_clauses])
else:
where = ""
if not db:
db = env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT bitten_report.id FROM bitten_report %s %s "
"ORDER BY category" % (' '.join(joins), where),
[wc[1] for wc in where_clauses])
for (id, ) in cursor:
yield Report.fetch(env, id, db=db)
select = classmethod(select)
schema = BuildConfig._schema + TargetPlatform._schema + Build._schema + \
BuildStep._schema + BuildLog._schema + Report._schema
schema_version = 12
Bitten-0.6/bitten/notify.py 000644 000765 000120 00000013256 11460004634 016232 0 ustar 00simon admin 000000 000000 #-*- coding: utf-8 -*-
#
# Copyright (C) 2007 Ole Trenner,
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution.
from genshi.template.text import NewTextTemplate
from trac.core import Component, implements
from trac.web.chrome import ITemplateProvider, Chrome
from trac.config import BoolOption
from trac.notification import NotifyEmail
from bitten.api import IBuildListener
from bitten.model import Build, BuildStep, BuildLog, TargetPlatform
class BittenNotify(Component):
"""Sends notifications on build status by mail."""
implements(IBuildListener, ITemplateProvider)
notify_on_failure = BoolOption('notification',
'notify_on_failed_build', 'true',
"""Notify if bitten build fails.""")
notify_on_success = BoolOption('notification',
'notify_on_successful_build', 'false',
"""Notify if bitten build succeeds.""")
def __init__(self):
self.log.debug('Initializing BittenNotify plugin')
def notify(self, build=None):
self.log.info('BittenNotify invoked for build %r', build)
self.log.debug('build status: %s', build.status)
if not self._should_notify(build):
return
self.log.info('Sending notification for build %r', build)
try:
email = BuildNotifyEmail(self.env)
email.notify(build)
except Exception, e:
self.log.exception("Failure sending notification for build "
"%s: %s", build.id, e)
def _should_notify(self, build):
if build.status == Build.FAILURE:
return self.notify_on_failure
elif build.status == Build.SUCCESS:
return self.notify_on_success
else:
return False
# IBuildListener methods
def build_started(self, build):
"""build started"""
self.notify(build)
def build_aborted(self, build):
"""build aborted"""
self.notify(build)
def build_completed(self, build):
"""build completed"""
self.notify(build)
# ITemplateProvider methods
def get_templates_dirs(self):
"""Return a list of directories containing the provided template
files."""
from pkg_resources import resource_filename
return [resource_filename(__name__, 'templates')]
def get_htdocs_dirs(self):
"""Return the absolute path of a directory containing additional
static resources (such as images, style sheets, etc)."""
return []
class BuildNotifyEmail(NotifyEmail):
"""Notification of failed builds."""
readable_states = {
Build.SUCCESS: 'Successful',
Build.FAILURE: 'Failed',
}
template_name = 'bitten_notify_email.txt'
from_email = 'bitten@localhost'
def __init__(self, env):
NotifyEmail.__init__(self, env)
# Override the template type to always use NewTextTemplate
if not isinstance(self.template, NewTextTemplate):
self.template = Chrome(env).templates.load(
self.template.filepath, cls=NewTextTemplate)
def notify(self, build):
self.build = build
self.data.update(self.template_data())
subject = '[%s Build] %s [%s] %s' % (self.readable_states[build.status],
self.env.project_name,
self.build.rev,
self.build.config)
NotifyEmail.notify(self, self.build.id, subject)
def get_recipients(self, resid):
to = [self.get_author()]
cc = []
return (to, cc)
def send(self, torcpts, ccrcpts):
mime_headers = {
'X-Trac-Build-ID': str(self.build.id),
'X-Trac-Build-URL': self.build_link(),
}
NotifyEmail.send(self, torcpts, ccrcpts, mime_headers)
def build_link(self):
return self.env.abs_href.build(self.build.config, self.build.id)
def template_data(self):
failed_steps = BuildStep.select(self.env, build=self.build.id,
status=BuildStep.FAILURE)
platform = TargetPlatform.fetch(self.env, id=self.build.platform)
change = self.get_changeset()
return {
'build': {
'id': self.build.id,
'status': self.readable_states[self.build.status],
'link': self.build_link(),
'config': self.build.config,
'platform': getattr(platform, 'name', 'unknown'),
'slave': self.build.slave,
'failed_steps': [{
'name': step.name,
'description': step.description,
'errors': step.errors,
'log_messages': self.get_all_log_messages_for_step(step),
} for step in failed_steps],
},
'change': {
'rev': change.rev,
'link': self.env.abs_href.changeset(change.rev),
'author': change.author,
},
}
def get_all_log_messages_for_step(self, step):
messages = []
for log in BuildLog.select(self.env, build=self.build.id,
step=step.name):
messages.extend(log.messages)
return messages
def get_changeset(self):
repos = self.env.get_repository()
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
return repos.get_changeset(self.build.rev)
def get_author(self):
return self.get_changeset().author
Bitten-0.6/bitten/queue.py 000644 000765 000120 00000033417 11535320455 016054 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Implements the scheduling of builds for a project.
This module provides the functionality for scheduling builds for a specific
Trac environment. It is used by both the build master and the web interface to
get the list of required builds (revisions not built yet).
Furthermore, the `BuildQueue` class is used by the build master to determine
the next pending build, and to match build slaves against configured target
platforms.
"""
from itertools import ifilter
import re
import time
from trac.util.datefmt import to_timestamp
from trac.util import pretty_timedelta, format_datetime
from trac.attachment import Attachment
from bitten.model import BuildConfig, TargetPlatform, Build, BuildStep
__docformat__ = 'restructuredtext en'
def collect_changes(repos, config, db=None):
"""Collect all changes for a build configuration that either have already
been built, or still need to be built.
This function is a generator that yields ``(platform, rev, build)`` tuples,
where ``platform`` is a `TargetPlatform` object, ``rev`` is the identifier
of the changeset, and ``build`` is a `Build` object or `None`.
:param repos: the version control repository
:param config: the build configuration
:param db: a database connection (optional)
"""
env = config.env
if not db:
db = env.get_db_cnx()
try:
node = repos.get_node(config.path, config.max_rev)
except Exception, e:
env.log.warn('Error accessing path %r for configuration %r',
config.path, config.name, exc_info=True)
return
for path, rev, chg in node.get_history():
# Don't follow moves/copies
if path != repos.normalize_path(config.path):
break
# Stay within the limits of the build config
if config.min_rev and repos.rev_older_than(rev, config.min_rev):
break
if config.max_rev and repos.rev_older_than(config.max_rev, rev):
continue
# Make sure the repository directory isn't empty at this
# revision
old_node = repos.get_node(path, rev)
is_empty = True
for entry in old_node.get_entries():
is_empty = False
break
if is_empty:
continue
# For every target platform, check whether there's a build
# of this revision
for platform in TargetPlatform.select(env, config.name, db=db):
builds = list(Build.select(env, config.name, rev, platform.id,
db=db))
if builds:
build = builds[0]
else:
build = None
yield platform, rev, build
class BuildQueue(object):
"""Enapsulates the build queue of an environment.
A build queue manages the the registration of build slaves and detection of
repository revisions that need to be built.
"""
def __init__(self, env, build_all=False, stabilize_wait=0, timeout=0):
"""Create the build queue.
:param env: the Trac environment
:param build_all: whether older revisions should be built
:param stabilize_wait: The time in seconds to wait before considering
the repository stable to create a build in the queue.
:param timeout: the time in seconds after which an in-progress build
should be considered orphaned, and reset to pending
state
"""
self.env = env
self.log = env.log
self.build_all = build_all
self.stabilize_wait = stabilize_wait
self.timeout = timeout
# Build scheduling
def get_build_for_slave(self, name, properties):
"""Check whether one of the pending builds can be built by the build
slave.
:param name: the name of the slave
:type name: `basestring`
:param properties: the slave configuration
:type properties: `dict`
:return: the allocated build, or `None` if no build was found
:rtype: `Build`
"""
self.log.debug('Checking for pending builds...')
db = self.env.get_db_cnx()
repos = self.env.get_repository()
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
self.reset_orphaned_builds()
# Iterate through pending builds by descending revision timestamp, to
# avoid the first configuration/platform getting all the builds
platforms = [p.id for p in self.match_slave(name, properties)]
builds_to_delete = []
build_found = False
for build in Build.select(self.env, status=Build.PENDING, db=db):
if self.should_delete_build(build, repos):
self.log.info('Scheduling build %d for deletion', build.id)
builds_to_delete.append(build)
elif build.platform in platforms:
build_found = True
break
if not build_found:
self.log.debug('No pending builds.')
build = None
# delete any obsolete builds
for build_to_delete in builds_to_delete:
build_to_delete.delete(db=db)
if build:
build.slave = name
build.slave_info.update(properties)
build.status = Build.IN_PROGRESS
build.update(db=db)
if build or builds_to_delete:
db.commit()
return build
def match_slave(self, name, properties):
"""Match a build slave against available target platforms.
:param name: the name of the slave
:type name: `basestring`
:param properties: the slave configuration
:type properties: `dict`
:return: the list of platforms the slave matched
"""
platforms = []
for config in BuildConfig.select(self.env):
for platform in TargetPlatform.select(self.env, config=config.name):
match = True
for propname, pattern in ifilter(None, platform.rules):
try:
propvalue = properties.get(propname)
if not propvalue or not re.match(pattern, propvalue):
match = False
break
except re.error:
self.log.error('Invalid platform matching pattern "%s"',
pattern, exc_info=True)
match = False
break
if match:
self.log.debug('Slave %r matched target platform %r of '
'build configuration %r', name,
platform.name, config.name)
platforms.append(platform)
if not platforms:
self.log.warning('Slave %r matched none of the target platforms',
name)
return platforms
def populate(self):
"""Add a build for the next change on each build configuration to the
queue.
The next change is the latest repository check-in for which there isn't
a corresponding build on each target platform. Repeatedly calling this
method will eventually result in the entire change history of the build
configuration being in the build queue.
"""
repos = self.env.get_repository()
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
db = self.env.get_db_cnx()
builds = []
for config in BuildConfig.select(self.env, db=db):
platforms = []
for platform, rev, build in collect_changes(repos, config, db):
if not self.build_all and platform.id in platforms:
# We've seen this platform already, so these are older
# builds that should only be built if built_all=True
self.log.debug('Ignoring older revisions for configuration '
'%r on %r', config.name, platform.name)
break
platforms.append(platform.id)
if build is None:
self.log.info('Enqueuing build of configuration "%s" at '
'revision [%s] on %s', config.name, rev,
platform.name)
rev_time = to_timestamp(repos.get_changeset(rev).date)
age = int(time.time()) - rev_time
if self.stabilize_wait and age < self.stabilize_wait:
self.log.info('Delaying build of revision %s until %s '
'seconds pass. Current age is: %s '
'seconds' % (rev, self.stabilize_wait,
age))
continue
build = Build(self.env, config=config.name,
platform=platform.id, rev=str(rev),
rev_time=rev_time)
builds.append(build)
for build in builds:
try:
build.insert(db=db)
db.commit()
except Exception, e:
# really only want to catch IntegrityErrors raised when
# a second slave attempts to add builds with the same
# (config, platform, rev) as an existing build.
self.log.info('Failed to insert build of configuration "%s" '
'at revision [%s] on platform [%s]: %s',
build.config, build.rev, build.platform, e)
db.rollback()
def reset_orphaned_builds(self):
"""Reset all in-progress builds to ``PENDING`` state if they've been
running so long that the configured timeout has been reached.
This is used to cleanup after slaves that have unexpectedly cancelled
a build without notifying the master, or are for some other reason not
reporting back status updates.
"""
if not self.timeout:
# If no timeout is set, none of the in-progress builds can be
# considered orphaned
return
db = self.env.get_db_cnx()
now = int(time.time())
for build in Build.select(self.env, status=Build.IN_PROGRESS, db=db):
if now - build.last_activity < self.timeout:
# This build has not reached the timeout yet, assume it's still
# being executed
continue
self.log.info('Orphaning build %d. Last activity was %s (%s)' % \
(build.id, format_datetime(build.last_activity),
pretty_timedelta(build.last_activity)))
build.status = Build.PENDING
build.slave = None
build.slave_info = {}
build.started = 0
build.stopped = 0
build.last_activity = 0
for step in list(BuildStep.select(self.env, build=build.id, db=db)):
step.delete(db=db)
build.update(db=db)
Attachment.delete_all(self.env, 'build', build.resource.id, db)
db.commit()
def should_delete_build(self, build, repos):
config = BuildConfig.fetch(self.env, build.config)
config_name = config and config.name \
or 'unknown config "%s"' % build.config
platform = TargetPlatform.fetch(self.env, build.platform)
# Platform may or may not exist anymore - get safe name for logging
platform_name = platform and platform.name \
or 'unknown platform "%s"' % build.platform
# Drop build if platform no longer exists
if not platform:
self.log.info('Dropping build of configuration "%s" at '
'revision [%s] on %s because the platform no longer '
'exists', config.name, build.rev, platform_name)
return True
# Ignore pending builds for deactived build configs
if not (config and config.active):
self.log.info('Dropping build of configuration "%s" at '
'revision [%s] on %s because the configuration is '
'deactivated', config_name, build.rev, platform_name)
return True
# Stay within the revision limits of the build config
if (config.min_rev and repos.rev_older_than(build.rev,
config.min_rev)) \
or (config.max_rev and repos.rev_older_than(config.max_rev,
build.rev)):
self.log.info('Dropping build of configuration "%s" at revision [%s] on '
'"%s" because it is outside of the revision range of the '
'configuration', config.name, build.rev, platform_name)
return True
# If not 'build_all', drop if a more recent revision is available
if not self.build_all and \
len(list(Build.select(self.env, config=build.config,
min_rev_time=build.rev_time, platform=build.platform))) > 1:
self.log.info('Dropping build of configuration "%s" at revision [%s] '
'on "%s" because a more recent build exists',
config.name, build.rev, platform_name)
return True
return False
Bitten-0.6/bitten/recipe.py 000644 000765 000120 00000027361 11454420025 016172 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Execution of build recipes.
This module provides various classes that can be used to process build recipes,
most importantly the `Recipe` class.
"""
import inspect
import keyword
import logging
import os
import time
try:
set
except NameError:
from sets import Set as set
from pkg_resources import WorkingSet
from bitten.build import BuildError, TimeoutError
from bitten.build.config import Configuration
from bitten.util import xmlio
__all__ = ['Context', 'Recipe', 'Step', 'InvalidRecipeError']
__docformat__ = 'restructuredtext en'
log = logging.getLogger('bitten.recipe')
class InvalidRecipeError(Exception):
"""Exception raised when a recipe is not valid."""
class Context(object):
"""The context in which a build is executed."""
step = None # The current step
generator = None # The current generator (namespace#name)
def __init__(self, basedir, config=None, vars=None):
"""Initialize the context.
:param basedir: a string containing the working directory for the build.
(may be a pattern for replacement ex: 'build_${build}'
:param config: the build slave configuration
:type config: `Configuration`
"""
self.config = config or Configuration()
self.vars = vars or {}
self.output = []
self.basedir = os.path.realpath(self.config.interpolate(basedir,
**self.vars))
self.vars['basedir'] = self.basedir.replace('\\', '\\\\')
def run(self, step, namespace, name, attr):
"""Run the specified recipe command.
:param step: the build step that the command belongs to
:param namespace: the namespace URI of the command
:param name: the local tag name of the command
:param attr: a dictionary containing the attributes defined on the
command element
"""
self.step = step
try:
function = None
qname = '#'.join(filter(None, [namespace, name]))
if namespace:
group = 'bitten.recipe_commands'
for entry_point in WorkingSet().iter_entry_points(group, qname):
function = entry_point.load()
break
elif name == 'report':
function = Context.report_file
elif name == 'attach':
function = Context.attach
if not function:
raise InvalidRecipeError('Unknown recipe command %s' % qname)
def escape(name):
name = name.replace('-', '_')
if keyword.iskeyword(name) or name in __builtins__:
name = name + '_'
return name
args = dict([(escape(name),
self.config.interpolate(attr[name], **self.vars))
for name in attr])
function_args, has_kwargs = inspect.getargspec(function)[0:3:2]
for arg in args:
if not (arg in function_args or has_kwargs):
raise InvalidRecipeError(
"Unsupported argument '%s' for command %s" % \
(arg, qname))
self.generator = qname
log.debug('Executing %s with arguments: %s', function, args)
function(self, **args)
finally:
self.generator = None
self.step = None
def error(self, message):
"""Record an error message.
:param message: a string containing the error message.
"""
self.output.append((Recipe.ERROR, None, self.generator, message))
def log(self, xml):
"""Record log output.
:param xml: an XML fragment containing the log messages
"""
self.output.append((Recipe.LOG, None, self.generator, xml))
def report(self, category, xml):
"""Record report data.
:param category: the name of category of the report
:param xml: an XML fragment containing the report data
"""
self.output.append((Recipe.REPORT, category, self.generator, xml))
def report_file(self, category=None, file_=None):
"""Read report data from a file and record it.
:param category: the name of the category of the report
:param file\_: the path to the file containing the report data, relative
to the base directory
"""
filename = self.resolve(file_)
try:
fileobj = file(filename, 'r')
try:
xml_elem = xmlio.Fragment()
for child in xmlio.parse(fileobj).children():
child_elem = xmlio.Element(child.name, **dict([
(name, value) for name, value in child.attr.items()
if value is not None
]))
xml_elem.append(child_elem[
[xmlio.Element(grandchild.name)[grandchild.gettext()]
for grandchild in child.children()]
])
self.output.append((Recipe.REPORT, category, None, xml_elem))
finally:
fileobj.close()
except xmlio.ParseError, e:
self.error('Failed to parse %s report at %s: %s'
% (category, filename, e))
except IOError, e:
self.error('Failed to read %s report at %s: %s'
% (category, filename, e))
def attach(self, file_=None, description=None, resource=None):
"""Attach a file to the build or build configuration.
:param file\_: the path to the file to attach, relative to
base directory.
:param description: description saved with attachment
:param resource: which resource to attach the file to,
either 'build' (default) or 'config'
"""
# Attachments are not added as inline xml, so only adding
# the details for later processing.
if not file_:
self.error('No attachment file specified.')
return
xml_elem = xmlio.Element('file', filename=file_,
description=description or '',
resource=resource or 'build')
self.output.append((Recipe.ATTACH, None, None, xml_elem))
def resolve(self, *path):
"""Return the path of a file relative to the base directory.
Accepts any number of positional arguments, which are joined using the
system path separator to form the path.
"""
return os.path.normpath(os.path.join(self.basedir, *path))
class Step(object):
"""Represents a single step of a build recipe.
Iterate over an object of this class to get the commands to execute, and
their keyword arguments.
"""
def __init__(self, elem, onerror_default):
"""Create the step.
:param elem: the XML element representing the step
:type elem: `ParsedElement`
"""
self._elem = elem
self.id = elem.attr['id']
self.description = elem.attr.get('description')
self.onerror = elem.attr.get('onerror', onerror_default)
assert self.onerror in ('fail', 'ignore', 'continue')
def __repr__(self):
return '<%s %r>' % (type(self).__name__, self.id)
def execute(self, ctxt):
"""Execute this step in the given context.
:param ctxt: the build context
:type ctxt: `Context`
"""
last_finish = time.time()
for child in self._elem:
try:
ctxt.run(self, child.namespace, child.name, child.attr)
except (BuildError, InvalidRecipeError, TimeoutError), e:
ctxt.error(e)
if time.time() < last_finish + 1:
# Add a delay to make sure steps appear in correct order
time.sleep(1)
errors = []
while ctxt.output:
type, category, generator, output = ctxt.output.pop(0)
yield type, category, generator, output
if type == Recipe.ERROR:
errors.append((generator, output))
if errors:
for _t, error in errors:
log.error(error)
if self.onerror != 'ignore':
raise BuildError("Build step '%s' failed" % self.id)
log.warning("Continuing despite errors in step '%s'", self.id)
class Recipe(object):
"""A build recipe.
Iterate over this object to get the individual build steps in the order
they have been defined in the recipe file.
"""
ERROR = 'error'
LOG = 'log'
REPORT = 'report'
ATTACH = 'attach'
def __init__(self, xml, basedir=os.getcwd(), config=None):
"""Create the recipe.
:param xml: the XML document representing the recipe
:type xml: `ParsedElement`
:param basedir: the base directory for the build
:param config: the slave configuration (optional)
:type config: `Configuration`
"""
assert isinstance(xml, xmlio.ParsedElement)
vars = dict([(name, value) for name, value in xml.attr.items()
if not name.startswith('xmlns')])
self.ctxt = Context(basedir, config, vars)
self._root = xml
self.onerror_default = vars.get('onerror', 'fail')
assert self.onerror_default in ('fail', 'ignore', 'continue')
def __iter__(self):
"""Iterate over the individual steps of the recipe."""
for child in self._root.children('step'):
yield Step(child, self.onerror_default)
def validate(self):
"""Validate the recipe.
This method checks a number of constraints:
- the name of the root element must be "build"
- the only permitted child elements or the root element with the name
"step"
- the recipe must contain at least one step
- step elements must have a unique "id" attribute
- a step must contain at least one nested command
- commands must not have nested content
:raise InvalidRecipeError: in case any of the above contraints is
violated
"""
if self._root.name != 'build':
raise InvalidRecipeError('Root element must be ')
steps = list(self._root.children())
if not steps:
raise InvalidRecipeError('Recipe defines no build steps')
step_ids = set()
for step in steps:
if step.name != 'step':
raise InvalidRecipeError('Only elements allowed at '
'top level of recipe')
if not step.attr.get('id'):
raise InvalidRecipeError('Steps must have an "id" attribute')
if step.attr['id'] in step_ids:
raise InvalidRecipeError('Duplicate step ID "%s"' %
step.attr['id'])
step_ids.add(step.attr['id'])
cmds = list(step.children())
if not cmds:
raise InvalidRecipeError('Step "%s" has no recipe commands' %
step.attr['id'])
for cmd in cmds:
if len(list(cmd.children())):
raise InvalidRecipeError('Recipe command <%s> has nested '
'content' % cmd.name)
Bitten-0.6/bitten/report/ 000755 000765 000120 00000000000 11536424600 015657 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/slave.py 000755 000765 000120 00000067120 11535713255 016047 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Implementation of the build slave."""
from datetime import datetime
import errno
import urllib
import urllib2
import logging
import os
import platform
import shutil
import socket
import sys
import tempfile
import time
import re
import cookielib
import threading
import mimetools
from ConfigParser import MissingSectionHeaderError
from bitten import PROTOCOL_VERSION
from bitten.build import BuildError
from bitten.build.config import Configuration, ConfigFileNotFound
from bitten.recipe import Recipe
from bitten.util import xmlio
from bitten.util.compat import HTTPBasicAuthHandler
EX_OK = getattr(os, "EX_OK", 0)
EX_UNAVAILABLE = getattr(os, "EX_UNAVAILABLE", 69)
EX_IOERR = getattr(os, "EX_IOERR", 74)
EX_PROTOCOL = getattr(os, "EX_PROTOCOL", 76)
EX_NOPERM = getattr(os, "EX_NOPERM", 77)
FORM_TOKEN_RE = re.compile('__FORM_TOKEN\" value=\"(.+)\"')
__all__ = ['BuildSlave', 'ExitSlave']
__docformat__ = 'restructuredtext en'
log = logging.getLogger('bitten.slave')
# List of network errors which are usually temporary and non critical.
temp_net_errors = [errno.ENETUNREACH, errno.ENETDOWN, errno.ETIMEDOUT,
errno.ECONNREFUSED]
def _rmtree(root):
"""Catch shutil.rmtree failures on Windows when files are read-only, and only remove if root exists."""
def _handle_error(fn, path, excinfo):
os.chmod(path, 0666)
fn(path)
if os.path.exists(root):
return shutil.rmtree(root, onerror=_handle_error)
else:
return False
class SaneHTTPRequest(urllib2.Request):
def __init__(self, method, url, data=None, headers={}):
urllib2.Request.__init__(self, url, data, headers)
self.method = method
def get_method(self):
if self.method is None:
self.method = self.has_data() and 'POST' or 'GET'
return self.method
def encode_multipart_formdata(fields):
"""
Given a dictionary field parameters, returns the HTTP request body and the
content_type (which includes the boundary string), to be used with an
httplib-like call.
Normal key/value items are treated as regular parameters, but key/tuple
items are treated as files, where a value tuple is a (filename, data) tuple.
For example::
fields = {
'foo': 'bar',
'foofile': ('foofile.txt', 'contents of foofile'),
}
body, content_type = encode_multipart_formdata(fields)
Note: Adapted from http://code.google.com/p/urllib3/ (MIT license)
"""
BOUNDARY = mimetools.choose_boundary()
ENCODE_TEMPLATE= "--%(boundary)s\r\n" \
"Content-Disposition: form-data; name=\"%(name)s\"\r\n" \
"\r\n%(value)s\r\n"
ENCODE_TEMPLATE_FILE = "--%(boundary)s\r\n" \
"Content-Disposition: form-data; name=\"%(name)s\"; " \
"filename=\"%(filename)s\"\r\n" \
"Content-Type: %(contenttype)s\r\n" \
"\r\n%(value)s\r\n"
body = ""
for key, value in fields.iteritems():
if isinstance(value, tuple):
filename, value = value
body += ENCODE_TEMPLATE_FILE % {
'boundary': BOUNDARY,
'name': str(key),
'value': str(value),
'filename': str(filename),
'contenttype': 'application/octet-stream'
}
else:
body += ENCODE_TEMPLATE % {
'boundary': BOUNDARY,
'name': str(key),
'value': str(value)
}
body += '--%s--\r\n' % BOUNDARY
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return body, content_type
class KeepAliveThread(threading.Thread):
"A thread to periodically send keep-alive messages to the master"
def __init__(self, opener, build_url, single_build, keepalive_interval):
threading.Thread.__init__(self, None, None, "KeepaliveThread")
self.build_url = build_url
self.keepalive_interval = keepalive_interval
self.single_build = single_build
self.last_keepalive = int(time.time())
self.kill = False
self.opener = opener
def keepalive(self):
log.debug('Sending keepalive')
method = 'POST'
url = self.build_url + '/keepalive/'
body = None
shutdown = False
headers = {
'Content-Type': 'application/x-bitten+xml',
'Content-Length': '0'
}
log.debug('Sending %s request to %r', method, url)
req = SaneHTTPRequest(method, url, body, headers or {})
try:
return self.opener.open(req)
except urllib2.HTTPError, e:
# a conflict error lets us know that we've been
# invalidated. Ideally, we'd engineer something to stop any
# running steps in progress, but killing threads is tricky
# stuff. For now, we'll wait for whatever's going
# on to stop, and the main thread'll figure out that we've
# been invalidated.
log.warning('Server returned keepalive error %d: %s', e.code, e.msg)
except:
log.warning('Server returned unknown keepalive error')
def run(self):
log.debug('Keepalive thread starting.')
while (not self.kill):
now = int(time.time())
if (self.last_keepalive + self.keepalive_interval) < now:
self.keepalive()
self.last_keepalive = now
time.sleep(1)
log.debug('Keepalive thread exiting.')
def stop(self):
log.debug('Stopping keepalive thread')
self.kill = True
self.join(30)
log.debug('Keepalive thread stopped')
class BuildSlave(object):
"""HTTP client implementation for the build slave."""
def __init__(self, urls, name=None, config=None, dry_run=False,
work_dir=None, build_dir="build_${build}",
keep_files=False, single_build=False,
poll_interval=300, keepalive_interval = 60,
username=None, password=None,
dump_reports=False, no_loop=False, form_auth=False):
"""Create the build slave instance.
:param urls: a list of URLs of the build masters to connect to, or a
single-element list containing the path to a build recipe
file
:param name: the name with which this slave should identify itself
:param config: the path to the slave configuration file
:param dry_run: wether the build outcome should not be reported back
to the master
:param work_dir: the working directory to use for build execution
:param build_dir: the pattern to use for naming the build subdir
:param keep_files: whether files and directories created for build
execution should be kept when done
:param single_build: whether this slave should exit after completing a
single build, or continue processing builds forever
:param poll_interval: the time in seconds to wait between requesting
builds from the build master (default is five
minutes)
:param keepalive_interval: the time in seconds to wait between sending
keepalive heartbeats (default is 30 seconds)
:param username: the username to use when authentication against the
build master is requested
:param password: the password to use when authentication is needed
:param dump_reports: whether report data should be written to the
standard output, in addition to being transmitted
to the build master
:param no_loop: for this slave to just perform a single check, regardless
of whether a build is done or not
:param form_auth: login using AccountManager HTML form instead of
HTTP authentication for all urls
"""
self.urls = urls
self.local = len(urls) == 1 and not urls[0].startswith('http://') \
and not urls[0].startswith('https://')
if name is None:
name = platform.node().split('.', 1)[0].lower()
self.name = name
self.config = Configuration(config)
self.dry_run = dry_run
if not work_dir:
work_dir = tempfile.mkdtemp(prefix='bitten')
elif not os.path.exists(work_dir):
os.makedirs(work_dir)
self.work_dir = work_dir
self.build_dir = build_dir
self.keep_files = keep_files
self.single_build = single_build
self.no_loop = no_loop
self.poll_interval = poll_interval
self.keepalive_interval = keepalive_interval
self.dump_reports = dump_reports
self.cookiejar = cookielib.CookieJar()
self.username = username \
or self.config['authentication.username'] or ''
if not self.local:
self.password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
if self.username:
log.debug('Enabling authentication with username %r',
self.username)
self.form_auth = form_auth
password = password \
or self.config['authentication.password'] or ''
self.config.packages.pop('authentication', None)
urls = [url[:-7] for url in urls]
self.password_mgr.add_password(
None, urls, self.username, password)
self.auth_map = dict(map(lambda x: (x, False), urls))
def _get_opener(self):
opener = urllib2.build_opener(urllib2.HTTPErrorProcessor())
opener.add_handler(HTTPBasicAuthHandler(self.password_mgr))
opener.add_handler(urllib2.HTTPDigestAuthHandler(self.password_mgr))
opener.add_handler(urllib2.HTTPCookieProcessor(self.cookiejar))
return opener
opener = property(_get_opener)
def request(self, method, url, body=None, headers=None):
log.debug('Sending %s request to %r', method, url)
req = SaneHTTPRequest(method, url, body, headers or {})
try:
resp = self.opener.open(req)
if not hasattr(resp, 'code'):
resp.code = 200
return resp
except urllib2.HTTPError, e:
if e.code >= 300:
if hasattr(e, 'headers') and \
e.headers.getheader('Content-Type', ''
).startswith('text/plain'):
content = e.read()
else:
content = 'no message available'
log.debug('Server returned error %d: %s (%s)',
e.code, e.msg, content)
raise
return e
def run(self):
if self.local:
fileobj = open(self.urls[0])
try:
self._execute_build(None, fileobj)
finally:
fileobj.close()
return EX_OK
urls = []
while True:
if not urls:
urls[:] = self.urls
url = urls.pop(0)
try:
try:
if self.username and not self.auth_map.get(url):
login_url = '%s/login?referer=%s' % (url[:-7],
urllib.quote_plus(url))
# First request to url, authentication needed
if self.form_auth:
log.debug('Performing http form authentication')
resp = self.request('POST', login_url)
match = FORM_TOKEN_RE.search(resp.read())
if not match:
log.error("Project %s does not support form "
"authentication" % url[:-7])
raise ExitSlave(EX_NOPERM)
values = {'user': self.username,
'password':
self.password_mgr.find_user_password(
None, url)[1],
'referer': '',
'__FORM_TOKEN': match.group(1)}
self.request('POST', login_url,
body=urllib.urlencode(values))
else:
log.debug('Performing basic/digest authentication')
self.request('HEAD', login_url)
self.auth_map[url] = True
elif self.username:
log.debug('Reusing authentication information.')
else:
log.debug('Authentication not provided. Attempting to '
'execute build anonymously.')
job_done = self._create_build(url)
if job_done:
continue
except urllib2.HTTPError, e:
# HTTPError doesn't have the "reason" attribute of URLError
log.error(e)
raise ExitSlave(EX_UNAVAILABLE)
except urllib2.URLError, e:
# Is this a temporary network glitch or something a bit
# more severe?
if isinstance(e.reason, socket.error) and \
e.reason.args[0] in temp_net_errors:
log.warning(e)
else:
log.error(e)
raise ExitSlave(EX_UNAVAILABLE)
except ExitSlave, e:
return e.exit_code
if self.no_loop:
break
time.sleep(self.poll_interval)
def quit(self):
log.info('Shutting down')
raise ExitSlave(EX_OK)
def _create_build(self, url):
xml = xmlio.Element('slave', name=self.name, version=PROTOCOL_VERSION)[
xmlio.Element('platform', processor=self.config['processor'])[
self.config['machine']
],
xmlio.Element('os', family=self.config['family'],
version=self.config['version'])[
self.config['os']
],
]
log.debug('Configured packages: %s', self.config.packages)
for package, properties in self.config.packages.items():
xml.append(xmlio.Element('package', name=package, **properties))
body = str(xml)
log.debug('Sending slave configuration: %s', body)
resp = self.request('POST', url, body, {
'Content-Length': str(len(body)),
'Content-Type': 'application/x-bitten+xml'
})
if resp.code == 201:
self._initiate_build(resp.info().get('location'))
return True
elif resp.code == 204:
log.info('No pending builds')
return False
else:
log.error('Unexpected response (%d %s)', resp.code, resp.msg)
raise ExitSlave(EX_PROTOCOL)
def _initiate_build(self, build_url):
log.info('Build pending at %s', build_url)
try:
resp = self.request('GET', build_url)
if resp.code == 200:
self._execute_build(build_url, resp)
else:
log.error('Unexpected response (%d): %s', resp.code, resp.msg)
self._cancel_build(build_url, exit_code=EX_PROTOCOL)
except KeyboardInterrupt:
log.warning('Build interrupted')
self._cancel_build(build_url)
def _execute_build(self, build_url, fileobj):
build_id = build_url and int(build_url.split('/')[-1]) or 0
xml = xmlio.parse(fileobj)
basedir = ''
try:
if not self.local:
keepalive_thread = KeepAliveThread(self.opener, build_url,
self.single_build, self.keepalive_interval)
keepalive_thread.start()
recipe = Recipe(xml, os.path.join(self.work_dir, self.build_dir),
self.config)
basedir = recipe.ctxt.basedir
log.debug('Running build in directory %s' % basedir)
if not os.path.exists(basedir):
os.mkdir(basedir)
for step in recipe:
try:
log.info('Executing build step %r, onerror = %s', step.id, step.onerror)
if not self._execute_step(build_url, recipe, step):
log.warning('Stopping build due to failure')
break
except Exception, e:
log.error('Exception raised processing step %s. Reraising %s', step.id, e)
raise
else:
log.info('Build completed')
if self.dry_run:
self._cancel_build(build_url)
finally:
if not self.local:
keepalive_thread.stop()
if not self.keep_files and os.path.isdir(basedir):
log.debug('Removing build directory %s' % basedir)
_rmtree(basedir)
if self.single_build:
log.info('Exiting after single build completed.')
raise ExitSlave(EX_OK)
def _execute_step(self, build_url, recipe, step):
failed = False
started = int(time.time())
xml = xmlio.Element('result', step=step.id)
try:
for type, category, generator, output in \
step.execute(recipe.ctxt):
if type == Recipe.ERROR:
failed = True
if type == Recipe.REPORT and self.dump_reports:
print output
if type == Recipe.ATTACH:
# Attachments are added out-of-band due to major
# performance issues with inlined base64 xml content
self._attach_file(build_url, recipe, output)
xml.append(xmlio.Element(type, category=category,
generator=generator)[
output
])
except KeyboardInterrupt:
log.warning('Build interrupted')
self._cancel_build(build_url)
except BuildError, e:
log.error('Build step %r failed', step.id)
failed = True
except Exception, e:
log.error('Internal error in build step %r', step.id, exc_info=True)
failed = True
xml.attr['duration'] = (time.time() - started)
if failed:
xml.attr['status'] = 'failure'
else:
xml.attr['status'] = 'success'
log.info('Build step %s completed successfully', step.id)
if not self.local and not self.dry_run:
try:
resp = self.request('POST', build_url + '/steps/', str(xml), {
'Content-Type': 'application/x-bitten+xml'
})
if resp.code != 201:
log.error('Unexpected response (%d): %s', resp.code,
resp.msg)
except KeyboardInterrupt:
log.warning('Build interrupted')
self._cancel_build(build_url)
return not failed or step.onerror != 'fail'
def _cancel_build(self, build_url, exit_code=EX_OK):
log.info('Cancelling build at %s', build_url)
if not self.local:
resp = self.request('DELETE', build_url)
if resp.code not in (200, 204):
log.error('Unexpected response (%d): %s', resp.code, resp.msg)
raise ExitSlave(exit_code)
def _attach_file(self, build_url, recipe, attachment):
form_token = recipe._root.attr.get('form_token', '')
if self.local or self.dry_run or not form_token:
log.info('Attachment %s not sent due to current slave options',
attachment.attr['file'])
return
resource_type = attachment.attr['resource']
url = str(build_url + '/attach/' + resource_type)
path = recipe.ctxt.resolve(attachment.attr['filename'])
filename = os.path.basename(path)
log.debug('Attaching file %s to %s...', attachment.attr['filename'],
resource_type)
f = open(path, 'rb')
try:
data, content_type = encode_multipart_formdata({
'file': (filename, f.read()),
'description': attachment.attr['description'],
'__FORM_TOKEN': form_token})
finally:
f.close()
resp = self.request('POST', url , data, {
'Content-Type': content_type})
if not resp.code == 201:
msg = 'Error attaching %s to %s'
log.error(msg, filename, resource_type)
raise BuildError(msg, filename, resource_type)
class ExitSlave(Exception):
"""Exception used internally by the slave to signal that the slave process
should be stopped.
"""
def __init__(self, exit_code):
self.exit_code = exit_code
Exception.__init__(self)
def main():
"""Main entry point for running the build slave."""
from bitten import __version__ as VERSION
from optparse import OptionParser
parser = OptionParser(usage='usage: %prog [options] url1 [url2] ...',
version='%%prog %s' % VERSION)
parser.add_option('--name', action='store', dest='name',
help='name of this slave (defaults to host name)')
parser.add_option('-f', '--config', action='store', dest='config',
metavar='FILE', help='path to configuration file')
parser.add_option('-u', '--user', dest='username',
help='the username to use for authentication')
parser.add_option('-p', '--password', dest='password',
help='the password to use when authenticating')
def _ask_password(option, opt_str, value, parser):
from getpass import getpass
parser.values.password = getpass('Passsword: ')
parser.add_option('-P', '--ask-password', action='callback',
callback=_ask_password, help='Prompt for password')
parser.add_option('--form-auth', action='store_true',
dest='form_auth',
help='login using AccountManager HTML form instead of '
'HTTP authentication for all urls')
group = parser.add_option_group('building')
group.add_option('-d', '--work-dir', action='store', dest='work_dir',
metavar='DIR', help='working directory for builds')
group.add_option('--build-dir', action='store', dest='build_dir',
default = 'build_${config}_${build}',
help='name pattern for the build dir to use inside the '
'working dir ["%default"]')
group.add_option('-k', '--keep-files', action='store_true',
dest='keep_files',
help='don\'t delete files after builds')
group.add_option('-s', '--single', action='store_true',
dest='single_build',
help='exit after completing a single build')
group.add_option('', '--no-loop', action='store_true',
dest='no_loop',
help='exit after completing a single check and running '
'the required builds')
group.add_option('-n', '--dry-run', action='store_true', dest='dry_run',
help='don\'t report results back to master')
group.add_option('-i', '--interval', dest='interval', metavar='SECONDS',
type='int', help='time to wait between requesting builds')
group.add_option('-b', '--keepalive_interval', dest='keepalive_interval', metavar='SECONDS', type='int', help='time to wait between keepalive heartbeats')
group = parser.add_option_group('logging')
group.add_option('-l', '--log', dest='logfile', metavar='FILENAME',
help='write log messages to FILENAME')
group.add_option('-v', '--verbose', action='store_const', dest='loglevel',
const=logging.DEBUG, help='print as much as possible')
group.add_option('-q', '--quiet', action='store_const', dest='loglevel',
const=logging.WARN, help='print as little as possible')
group.add_option('--dump-reports', action='store_true', dest='dump_reports',
help='whether report data should be printed')
parser.set_defaults(dry_run=False, keep_files=False,
loglevel=logging.INFO, single_build=False, no_loop=False,
dump_reports=False, interval=300, keepalive_interval=60,
form_auth=False)
options, args = parser.parse_args()
if len(args) < 1:
parser.error('incorrect number of arguments')
urls = args
logger = logging.getLogger('bitten')
logger.setLevel(options.loglevel)
handler = logging.StreamHandler()
handler.setLevel(options.loglevel)
formatter = logging.Formatter('[%(levelname)-8s] %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
if options.logfile:
handler = logging.FileHandler(options.logfile)
handler.setLevel(options.loglevel)
formatter = logging.Formatter('%(asctime)s [%(name)s] %(levelname)s: '
'%(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
log.info("Slave launched at %s" % \
datetime.now().strftime('%Y-%m-%d %H:%M:%S'))
slave = None
try:
slave = BuildSlave(urls, name=options.name, config=options.config,
dry_run=options.dry_run, work_dir=options.work_dir,
build_dir=options.build_dir,
keep_files=options.keep_files,
single_build=options.single_build,
no_loop=options.no_loop,
poll_interval=options.interval,
keepalive_interval=options.keepalive_interval,
username=options.username, password=options.password,
dump_reports=options.dump_reports,
form_auth=options.form_auth)
try:
exit_code = slave.run()
except KeyboardInterrupt:
slave.quit()
except ConfigFileNotFound, e:
log.error(e)
exit_code = EX_IOERR
except MissingSectionHeaderError:
log.error("Error parsing configuration file %r. Wrong format?" \
% options.config)
exit_code = EX_IOERR
except ExitSlave, e:
exit_code = e.exit_code
if slave and not (options.work_dir or options.keep_files):
log.debug('Removing working directory %s' % slave.work_dir)
_rmtree(slave.work_dir)
log.info("Slave exited at %s" % \
datetime.now().strftime('%Y-%m-%d %H:%M:%S'))
return exit_code
if __name__ == '__main__':
sys.exit(main())
Bitten-0.6/bitten/templates/ 000755 000765 000120 00000000000 11536424600 016342 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/tests/ 000755 000765 000120 00000000000 11536424600 015506 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/tests_slave/ 000755 000765 000120 00000000000 11536424600 016700 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/upgrades.py 000644 000765 000120 00000063046 11457032700 016537 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# Copyright (C) 2005-2007 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Automated upgrades for the Bitten database tables, and other data stored
in the Trac environment.
**Do not import and call directly!**"""
import os
import sys
from trac.core import TracError
from trac.db import DatabaseManager, Table, Column, Index
from trac.util.text import to_unicode
import codecs
__docformat__ = 'restructuredtext en'
# database abstraction functions
def parse_scheme(env):
"""Retrieve the environment database scheme."""
connection_uri = DatabaseManager(env).connection_uri
parts = connection_uri.split(':', 1)
scheme = parts[0].lower()
return scheme
def update_sequence(env, db, tbl, col):
"""Update a sequence associated with an autoincrement column."""
# Hopefully Trac will eventually implement its own version
# of this function.
scheme = parse_scheme(env)
if scheme == "postgres":
seq = '%s_%s_seq' % (tbl, col)
cursor = db.cursor()
cursor.execute("SELECT setval('%s', (SELECT MAX(%s) FROM %s))"
% (seq, col, tbl))
def drop_index(env, db, tbl, idx):
"""Drop an index associated with a table."""
# Hopefully Trac will eventually implement its own version
# of this function.
scheme = parse_scheme(env)
cursor = db.cursor()
if scheme == "mysql":
cursor.execute("DROP INDEX %s ON %s" % (idx, tbl))
else:
cursor.execute("DROP INDEX %s" % (idx,))
# upgrade scripts
def add_log_table(env, db):
"""Add a table for storing the builds logs."""
INFO_LEVEL = 'I'
cursor = db.cursor()
build_log_schema_v3 = [
Table('bitten_log', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('type')
],
Table('bitten_log_message', key=('log', 'line'))[
Column('log', type='int'), Column('line', type='int'),
Column('level', size=1), Column('message')
]
]
build_step_schema_v3 = [
Table('bitten_step', key=('build', 'name'))[
Column('build', type='int'), Column('name'), Column('description'),
Column('status', size=1), Column('started', type='int'),
Column('stopped', type='int')
]
]
connector, _ = DatabaseManager(env)._get_connector()
for table in build_log_schema_v3:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
update_cursor = db.cursor()
cursor.execute("SELECT build,name,log FROM bitten_step "
"WHERE log IS NOT NULL")
for build, step, log in cursor:
update_cursor.execute("INSERT INTO bitten_log (build, step) "
"VALUES (%s,%s)", (build, step))
log_id = db.get_last_id(update_cursor, 'bitten_log')
messages = [(log_id, line, INFO_LEVEL, msg)
for line, msg in enumerate(log.splitlines())]
update_cursor.executemany("INSERT INTO bitten_log_message (log, line, level, message) "
"VALUES (%s, %s, %s, %s)", messages)
cursor.execute("CREATE TEMPORARY TABLE old_step AS SELECT * FROM bitten_step")
cursor.execute("DROP TABLE bitten_step")
for table in build_step_schema_v3:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
cursor.execute("INSERT INTO bitten_step (build,name,description,status,"
"started,stopped) SELECT build,name,description,status,"
"started,stopped FROM old_step")
def add_recipe_to_config(env, db):
"""Add a column for storing the build recipe to the build configuration
table."""
cursor = db.cursor()
build_config_schema_v3 = Table('bitten_config', key='name')[
Column('name'), Column('path'), Column('active', type='int'),
Column('recipe'), Column('min_rev'), Column('max_rev'),
Column('label'), Column('description')
]
cursor.execute("CREATE TEMPORARY TABLE old_config_v2 AS "
"SELECT * FROM bitten_config")
cursor.execute("DROP TABLE bitten_config")
connector, _ = DatabaseManager(env)._get_connector()
for stmt in connector.to_sql(build_config_schema_v3):
cursor.execute(stmt)
cursor.execute("INSERT INTO bitten_config (name,path,active,recipe,min_rev,"
"max_rev,label,description) SELECT name,path,0,'',NULL,"
"NULL,label,description FROM old_config_v2")
def add_last_activity_to_build(env, db):
"""Add a column for storing the last activity to the build table."""
cursor = db.cursor()
build_table_schema_v12 = Table('bitten_build', key='id')[
Column('id', auto_increment=True), Column('config'), Column('rev'),
Column('rev_time', type='int'), Column('platform', type='int'),
Column('slave'), Column('started', type='int'),
Column('stopped', type='int'), Column('status', size=1),
Column('last_activity', type='int'),
Index(['config', 'rev', 'platform'], unique=True)
]
cursor.execute("CREATE TEMPORARY TABLE old_build_v11 AS "
"SELECT * FROM bitten_build")
cursor.execute("DROP TABLE bitten_build")
connector, _ = DatabaseManager(env)._get_connector()
for stmt in connector.to_sql(build_table_schema_v12):
cursor.execute(stmt)
# it's safe to make the last activity the stop time of the build
cursor.execute("INSERT INTO bitten_build (id,config,rev,rev_time,platform,"
"slave,started,stopped,last_activity,status) "
"SELECT id,config,rev,rev_time,platform,"
"slave,started,stopped,stopped,status FROM old_build_v11")
update_sequence(env, db, 'bitten_build', 'id')
def add_config_to_reports(env, db):
"""Add the name of the build configuration as metadata to report documents
stored in the BDB XML database."""
try:
from bsddb3 import db as bdb
import dbxml
except ImportError:
return
dbfile = os.path.join(env.path, 'db', 'bitten.dbxml')
if not os.path.isfile(dbfile):
return
dbenv = bdb.DBEnv()
dbenv.open(os.path.dirname(dbfile),
bdb.DB_CREATE | bdb.DB_INIT_LOCK | bdb.DB_INIT_LOG |
bdb.DB_INIT_MPOOL | bdb.DB_INIT_TXN, 0)
mgr = dbxml.XmlManager(dbenv, 0)
xtn = mgr.createTransaction()
container = mgr.openContainer(dbfile, dbxml.DBXML_TRANSACTIONAL)
uc = mgr.createUpdateContext()
container.addIndex(xtn, '', 'config', 'node-metadata-equality-string', uc)
qc = mgr.createQueryContext()
for value in mgr.query(xtn, 'collection("%s")/report' % dbfile, qc):
doc = value.asDocument()
metaval = dbxml.XmlValue()
if doc.getMetaData('', 'build', metaval):
build_id = int(metaval.asNumber())
cursor = db.cursor()
cursor.execute("SELECT config FROM bitten_build WHERE id=%s", (build_id,))
row = cursor.fetchone()
if row:
doc.setMetaData('', 'config', dbxml.XmlValue(row[0]))
container.updateDocument(xtn, doc, uc)
else:
# an orphaned report, for whatever reason... just remove it
container.deleteDocument(xtn, doc, uc)
xtn.commit()
container.close()
dbenv.close(0)
def add_order_to_log(env, db):
"""Add order column to log table to make sure that build logs are displayed
in the order they were generated."""
cursor = db.cursor()
log_table_schema_v6 = Table('bitten_log', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('generator'), Column('orderno', type='int'),
Index(['build', 'step'])
]
cursor.execute("CREATE TEMPORARY TABLE old_log_v5 AS "
"SELECT * FROM bitten_log")
cursor.execute("DROP TABLE bitten_log")
connector, _ = DatabaseManager(env)._get_connector()
for stmt in connector.to_sql(log_table_schema_v6):
cursor.execute(stmt)
cursor.execute("INSERT INTO bitten_log (id,build,step,generator,orderno) "
"SELECT id,build,step,type,0 FROM old_log_v5")
def add_report_tables(env, db):
"""Add database tables for report storage."""
cursor = db.cursor()
report_schema_v6 = Table('bitten_report', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('category'), Column('generator'),
Index(['build', 'step', 'category'])
]
report_item_schema_v6 = Table('bitten_report_item', key=('report', 'item', 'name'))[
Column('report', type='int'), Column('item', type='int'),
Column('name'), Column('value')
]
connector, _ = DatabaseManager(env)._get_connector()
for table in [report_schema_v6, report_item_schema_v6]:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
def xmldb_to_db(env, db):
"""Migrate report data from Berkeley DB XML to SQL database.
Depending on the number of reports stored, this might take rather long.
After the upgrade is done, the bitten.dbxml file (and any BDB XML log files)
may be deleted. BDB XML is no longer used by Bitten.
"""
from bitten.util import xmlio
try:
from bsddb3 import db as bdb
import dbxml
except ImportError:
return
dbfile = os.path.join(env.path, 'db', 'bitten.dbxml')
if not os.path.isfile(dbfile):
return
dbenv = bdb.DBEnv()
dbenv.open(os.path.dirname(dbfile),
bdb.DB_CREATE | bdb.DB_INIT_LOCK | bdb.DB_INIT_LOG |
bdb.DB_INIT_MPOOL | bdb.DB_INIT_TXN, 0)
mgr = dbxml.XmlManager(dbenv, 0)
xtn = mgr.createTransaction()
container = mgr.openContainer(dbfile, dbxml.DBXML_TRANSACTIONAL)
def get_pylint_items(xml):
for problems_elem in xml.children('problems'):
for problem_elem in problems_elem.children('problem'):
item = {'type': 'problem'}
item.update(problem_elem.attr)
yield item
def get_trace_items(xml):
for cov_elem in xml.children('coverage'):
item = {'type': 'coverage', 'name': cov_elem.attr['module'],
'file': cov_elem.attr['file'],
'percentage': cov_elem.attr['percentage']}
lines = 0
line_hits = []
for line_elem in cov_elem.children('line'):
lines += 1
line_hits.append(line_elem.attr['hits'])
item['lines'] = lines
item['line_hits'] = ' '.join(line_hits)
yield item
def get_unittest_items(xml):
for test_elem in xml.children('test'):
item = {'type': 'test'}
item.update(test_elem.attr)
for child_elem in test_elem.children():
item[child_elem.name] = child_elem.gettext()
yield item
qc = mgr.createQueryContext()
for value in mgr.query(xtn, 'collection("%s")/report' % dbfile, qc, 0):
doc = value.asDocument()
metaval = dbxml.XmlValue()
build, step = None, None
if doc.getMetaData('', 'build', metaval):
build = metaval.asNumber()
if doc.getMetaData('', 'step', metaval):
step = metaval.asString()
report_types = {'pylint': ('lint', get_pylint_items),
'trace': ('coverage', get_trace_items),
'unittest': ('test', get_unittest_items)}
xml = xmlio.parse(value.asString())
report_type = xml.attr['type']
category, get_items = report_types[report_type]
sys.stderr.write('.')
sys.stderr.flush()
items = list(get_items(xml))
cursor = db.cursor()
cursor.execute("SELECT bitten_report.id FROM bitten_report "
"WHERE build=%s AND step=%s AND category=%s",
(build, step, category))
rows = cursor.fetchall()
if rows:
# Duplicate report, skip
continue
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(build, step, category, report_type))
id = db.get_last_id(cursor, 'bitten_report')
for idx, item in enumerate(items):
cursor.executemany("INSERT INTO bitten_report_item "
"(report,item,name,value) VALUES (%s,%s,%s,%s)",
[(id, idx, key, value) for key, value
in item.items()])
sys.stderr.write('\n')
sys.stderr.flush()
xtn.abort()
container.close()
dbenv.close(0)
def normalize_file_paths(env, db):
"""Normalize the file separator in file names in reports."""
cursor = db.cursor()
cursor.execute("SELECT report,item,value FROM bitten_report_item "
"WHERE name='file'")
rows = cursor.fetchall() or []
for report, item, value in rows:
if '\\' in value:
cursor.execute("UPDATE bitten_report_item SET value=%s "
"WHERE report=%s AND item=%s AND name='file'",
(value.replace('\\', '/'), report, item))
def fixup_generators(env, db):
"""Upgrade the identifiers for the recipe commands that generated log
messages and report data."""
mapping = {
'pipe': 'http://bitten.edgewall.org/tools/sh#pipe',
'make': 'http://bitten.edgewall.org/tools/c#make',
'distutils': 'http://bitten.edgewall.org/tools/python#distutils',
'exec_': 'http://bitten.edgewall.org/tools/python#exec' # Ambigious
}
cursor = db.cursor()
cursor.execute("SELECT id,generator FROM bitten_log "
"WHERE generator IN (%s)"
% ','.join([repr(key) for key in mapping.keys()]))
for log_id, generator in cursor:
cursor.execute("UPDATE bitten_log SET generator=%s "
"WHERE id=%s", (mapping[generator], log_id))
mapping = {
'unittest': 'http://bitten.edgewall.org/tools/python#unittest',
'trace': 'http://bitten.edgewall.org/tools/python#trace',
'pylint': 'http://bitten.edgewall.org/tools/python#pylint'
}
cursor.execute("SELECT id,generator FROM bitten_report "
"WHERE generator IN (%s)"
% ','.join([repr(key) for key in mapping.keys()]))
for report_id, generator in cursor:
cursor.execute("UPDATE bitten_report SET generator=%s "
"WHERE id=%s", (mapping[generator], report_id))
def add_error_table(env, db):
"""Add the bitten_error table for recording step failure reasons."""
table = Table('bitten_error', key=('build', 'step', 'orderno'))[
Column('build', type='int'), Column('step'), Column('message'),
Column('orderno', type='int')
]
cursor = db.cursor()
connector, _ = DatabaseManager(env)._get_connector()
for stmt in connector.to_sql(table):
cursor.execute(stmt)
def add_filename_to_logs(env, db):
"""Add filename column to log table to save where log files are stored."""
cursor = db.cursor()
build_log_schema_v9 = Table('bitten_log', key='id')[
Column('id', auto_increment=True), Column('build', type='int'),
Column('step'), Column('generator'), Column('orderno', type='int'),
Column('filename'),
Index(['build', 'step'])
]
cursor.execute("CREATE TEMPORARY TABLE old_log_v8 AS "
"SELECT * FROM bitten_log")
cursor.execute("DROP TABLE bitten_log")
connector, _ = DatabaseManager(env)._get_connector()
for stmt in connector.to_sql(build_log_schema_v9):
cursor.execute(stmt)
cursor.execute("INSERT INTO bitten_log (id,build,step,generator,orderno,filename) "
"SELECT id,build,step,generator,orderno,'' FROM old_log_v8")
def migrate_logs_to_files(env, db):
"""Migrates logs that are stored in the bitten_log_messages table into files."""
logs_dir = env.config.get("bitten", "logs_dir", "log/bitten")
if not os.path.isabs(logs_dir):
logs_dir = os.path.join(env.path, logs_dir)
if os.path.exists(logs_dir):
print "Bitten log folder %r already exists" % (logs_dir,)
print "Upgrade cannot be performed until the existing folder is moved."
print "The upgrade script will now exit with an error:\n"
raise TracError("")
os.makedirs(logs_dir)
cursor = db.cursor()
message_cursor = db.cursor()
update_cursor = db.cursor()
cursor.execute("SELECT id FROM bitten_log")
for log_id, in cursor.fetchall():
filename = "%s.log" % (log_id,)
message_cursor.execute("SELECT message, level FROM bitten_log_message WHERE log=%s ORDER BY line", (log_id,))
full_filename = os.path.join(logs_dir, filename)
message_file = codecs.open(full_filename, "wb", "UTF-8")
# Note: the original version of this code erroneously wrote to filename + ".level" instead of ".levels", producing unused level files
level_file = codecs.open(full_filename + '.levels', "wb", "UTF-8")
for message, level in message_cursor.fetchall() or []:
message_file.write(to_unicode(message) + "\n")
level_file.write(to_unicode(level) + "\n")
message_file.close()
level_file.close()
update_cursor.execute("UPDATE bitten_log SET filename=%s WHERE id=%s", (filename, log_id))
env.log.info("Migrated log %s", log_id)
env.log.warning("Logs have been migrated from the database to files in %s. "
"Ensure permissions are set correctly on this file. "
"Since we presume that the migration worked correctly, "
"we are now dropping the bitten_log_message table in the database (aren't you glad you backed up)", logs_dir)
cursor.close()
cursor = db.cursor()
cursor.execute("DROP TABLE bitten_log_message")
cursor.close()
env.log.warning("We have dropped the bitten_log_message table - you may want to vaccuum/compress your database to save space")
def fix_log_levels_misnaming(env, db):
"""Renames or removes \*.log.level files created by older versions of migrate_logs_to_files."""
logs_dir = env.config.get("bitten", "logs_dir", "log/bitten")
if not os.path.isabs(logs_dir):
logs_dir = os.path.join(env.path, logs_dir)
if not os.path.isdir(logs_dir):
return
rename_count = 0
rename_error_count = 0
delete_count = 0
delete_error_count = 0
for wrong_filename in os.listdir(logs_dir):
if not wrong_filename.endswith('.log.level'):
continue
log_filename = os.path.splitext(wrong_filename)[0]
right_filename = log_filename + '.levels'
full_log_filename = os.path.join(logs_dir, log_filename)
full_wrong_filename = os.path.join(logs_dir, wrong_filename)
full_right_filename = os.path.join(logs_dir, right_filename)
if not os.path.exists(full_log_filename):
try:
os.remove(full_wrong_filename)
delete_count += 1
env.log.info("Deleted stray log level file %s", wrong_filename)
except Exception, e:
delete_error_count += 1
env.log.warning("Error removing stray log level file %s: %s", wrong_filename, e)
else:
if os.path.exists(full_right_filename):
env.log.warning("Error renaming %s to %s in fix_log_levels_misnaming: new filename already exists",
full_wrong_filename, full_right_filename)
rename_error_count += 1
continue
try:
os.rename(full_wrong_filename, full_right_filename)
rename_count += 1
env.log.info("Renamed incorrectly named log level file %s to %s", wrong_filename, right_filename)
except Exception, e:
env.log.warning("Error renaming %s to %s in fix_log_levels_misnaming: %s", full_wrong_filename, full_right_filename, e)
rename_error_count += 1
env.log.info("Renamed %d incorrectly named log level files from previous migrate (%d errors)", rename_count, rename_error_count)
env.log.info("Deleted %d stray log level (%d errors)", delete_count, delete_error_count)
def remove_stray_log_levels_files(env, db):
"""Remove \*.log.levels files without a matching \*.log file (old Bitten
versions did not delete .log.levels files when builds were deleted)"""
logs_dir = env.config.get("bitten", "logs_dir", "log/bitten")
if not os.path.isabs(logs_dir):
logs_dir = os.path.join(env.path, logs_dir)
if not os.path.isdir(logs_dir):
return
delete_count = 0
delete_error_count = 0
for filename in os.listdir(logs_dir):
if not filename.endswith('.log.levels'):
continue
log_filename = os.path.splitext(filename)[0]
full_log_filename = os.path.join(logs_dir, log_filename)
full_filename = os.path.join(logs_dir, filename)
if not os.path.exists(full_log_filename):
try:
os.remove(full_filename)
delete_count += 1
env.log.info("Deleted stray log levels file %s", filename)
except Exception, e:
delete_error_count += 1
env.log.warning("Error removing stray log levels file %s: %s", filename, e)
env.log.info("Deleted %d stray log levels (%d errors)", delete_count, delete_error_count)
def recreate_rule_with_int_id(env, db):
"""Recreates the bitten_rule table with an integer id column rather than a text one."""
cursor = db.cursor()
rule_schema_v9 = Table('bitten_rule', key=('id', 'propname'))[
Column('id', type='int'), Column('propname'), Column('pattern'),
Column('orderno', type='int')
]
env.log.info("Migrating bitten_rule table to integer ids")
connector, _ = DatabaseManager(env)._get_connector()
cursor.execute("CREATE TEMPORARY TABLE old_rule_v9 AS SELECT * FROM bitten_rule")
cursor.execute("DROP TABLE bitten_rule")
for stmt in connector.to_sql(rule_schema_v9):
cursor.execute(stmt)
cursor.execute("INSERT INTO bitten_rule (id,propname,pattern,orderno) "
"SELECT %s,propname,pattern,orderno FROM old_rule_v9" % db.cast('id', 'int'))
def add_config_platform_rev_index_to_build(env, db):
"""Adds a unique index on (config, platform, rev) to the bitten_build table.
Also drops the old index on bitten_build that serves no real purpose anymore."""
# check for existing duplicates
duplicates_cursor = db.cursor()
build_cursor = db.cursor()
duplicates_cursor.execute("SELECT config, rev, platform FROM bitten_build GROUP BY config, rev, platform HAVING COUNT(config) > 1")
duplicates_exist = False
for config, rev, platform in duplicates_cursor.fetchall():
if not duplicates_exist:
duplicates_exist = True
print "\nConfig Name, Revision, Platform :: []"
print "--------------------------------------------------------"
build_cursor.execute("SELECT id FROM bitten_build WHERE config='%s' AND rev='%s' AND platform='%s'" % (config, rev, platform))
build_ids = [row[0] for row in build_cursor.fetchall()]
print "%s, %s, %s :: %s" % (config, rev, platform, build_ids)
if duplicates_exist:
print "--------------------------------------------------------\n"
print "Duplicate builds found. You can obtain help on removing the"
print "builds you don't want by reading the Bitten upgrade"
print "documentation at:"
print "http://bitten.edgewall.org/wiki/Documentation/upgrade.html"
print "Upgrades cannot be performed until conflicts are resolved."
print "The upgrade script will now exit with an error:\n"
duplicates_cursor.close()
build_cursor.close()
if not duplicates_exist:
cursor = db.cursor()
scheme = parse_scheme(env)
if scheme == "mysql":
# 111 = 333 / len(columns in index) -- this is the Trac default
cursor.execute("CREATE UNIQUE INDEX bitten_build_config_rev_platform_idx ON bitten_build (config(111), rev(111), platform)")
else:
cursor.execute("CREATE UNIQUE INDEX bitten_build_config_rev_platform_idx ON bitten_build (config,rev,platform)")
drop_index(env, db, 'bitten_build', 'bitten_build_config_rev_slave_idx')
else:
raise TracError('')
def fix_sequences(env, db):
"""Fixes any auto increment sequences that might have been left in an inconsistent state.
Upgrade scripts for schema versions > 10 should handle sequence updates correctly themselves.
"""
update_sequence(env, db, 'bitten_build', 'id')
update_sequence(env, db, 'bitten_log', 'id')
update_sequence(env, db, 'bitten_platform', 'id')
update_sequence(env, db, 'bitten_report', 'id')
map = {
2: [add_log_table],
3: [add_recipe_to_config],
4: [add_config_to_reports],
5: [add_order_to_log, add_report_tables, xmldb_to_db],
6: [normalize_file_paths, fixup_generators],
7: [add_error_table],
8: [add_filename_to_logs,migrate_logs_to_files],
9: [recreate_rule_with_int_id],
10: [add_config_platform_rev_index_to_build, fix_sequences],
11: [fix_log_levels_misnaming, remove_stray_log_levels_files],
12: [add_last_activity_to_build],
}
Bitten-0.6/bitten/util/ 000755 000765 000120 00000000000 11536424600 015321 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/web_ui.py 000644 000765 000120 00000106556 11500370724 016203 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Implementation of the Bitten web interface."""
import posixpath
import re
import time
from StringIO import StringIO
from datetime import datetime
import pkg_resources
from genshi.builder import tag
from trac.attachment import AttachmentModule, Attachment
from trac.core import *
from trac.config import Option
from trac.mimeview.api import Context
from trac.perm import PermissionError
from trac.resource import Resource
from trac.timeline import ITimelineEventProvider
from trac.util import escape, pretty_timedelta, format_datetime, shorten_line, \
Markup, arity
from trac.util.datefmt import to_timestamp, to_datetime, utc
from trac.util.html import html
from trac.web import IRequestHandler, IRequestFilter, HTTPNotFound
from trac.web.chrome import INavigationContributor, ITemplateProvider, \
add_link, add_stylesheet, add_ctxtnav, \
prevnext_nav, add_script, add_warning
from trac.versioncontrol import NoSuchChangeset, NoSuchNode
from trac.wiki import wiki_to_html, wiki_to_oneliner
from bitten.api import ILogFormatter, IReportChartGenerator, IReportSummarizer
from bitten.master import BuildMaster
from bitten.model import BuildConfig, TargetPlatform, Build, BuildStep, \
BuildLog, Report
from bitten.queue import collect_changes
from bitten.util import json
_status_label = {Build.PENDING: 'pending',
Build.IN_PROGRESS: 'in progress',
Build.SUCCESS: 'completed',
Build.FAILURE: 'failed'}
_status_title = {Build.PENDING: 'Pending',
Build.IN_PROGRESS: 'In Progress',
Build.SUCCESS: 'Success',
Build.FAILURE: 'Failure'}
_step_status_label = {BuildStep.SUCCESS: 'success',
BuildStep.FAILURE: 'failed',
BuildStep.IN_PROGRESS: 'in progress'}
def _get_build_data(env, req, build):
platform = TargetPlatform.fetch(env, build.platform)
data = {'id': build.id, 'name': build.slave, 'rev': build.rev,
'status': _status_label[build.status],
'platform': getattr(platform, 'name', 'unknown'),
'cls': _status_label[build.status].replace(' ', '-'),
'href': req.href.build(build.config, build.id),
'chgset_href': req.href.changeset(build.rev)}
if build.started:
data['started'] = format_datetime(build.started)
data['started_delta'] = pretty_timedelta(build.started)
data['duration'] = pretty_timedelta(build.started)
if build.stopped:
data['stopped'] = format_datetime(build.stopped)
data['stopped_delta'] = pretty_timedelta(build.stopped)
data['duration'] = pretty_timedelta(build.stopped, build.started)
data['slave'] = {
'name': build.slave,
'ipnr': build.slave_info.get(Build.IP_ADDRESS),
'os_name': build.slave_info.get(Build.OS_NAME),
'os_family': build.slave_info.get(Build.OS_FAMILY),
'os_version': build.slave_info.get(Build.OS_VERSION),
'machine': build.slave_info.get(Build.MACHINE),
'processor': build.slave_info.get(Build.PROCESSOR)
}
return data
def _has_permission(perm, repos, path, rev=None, raise_error=False):
if hasattr(repos, 'authz'):
if not repos.authz.has_permission(path):
if not raise_error:
return False
repos.authz.assert_permission(path)
else:
node = repos.get_node(path, rev)
if not node.can_view(perm):
if not raise_error:
return False
raise PermissionError('BROWSER_VIEW', node.resource)
return True
class BittenChrome(Component):
"""Provides the Bitten templates and static resources."""
implements(INavigationContributor, ITemplateProvider)
# INavigationContributor methods
def get_active_navigation_item(self, req):
pass
def get_navigation_items(self, req):
"""Return the navigation item for access the build status overview from
the Trac navigation bar."""
if 'BUILD_VIEW' in req.perm:
status = ''
if BuildMaster(self.env).quick_status:
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository ' \
'or alias named "(default)" to Trac.'
for config in BuildConfig.select(self.env,
include_inactive=False):
prev_rev = None
for platform, rev, build in collect_changes(repos, config):
if rev != prev_rev:
if prev_rev is not None:
break
prev_rev = rev
if build:
build_data = _get_build_data(self.env, req, build)
if build_data['status'] == 'failed':
status='bittenfailed'
break
if build_data['status'] == 'in progress':
status='bitteninprogress'
elif not status:
if (build_data['status'] == 'completed'):
status='bittencompleted'
if not status:
status='bittenpending'
yield ('mainnav', 'build',
tag.a('Build Status', href=req.href.build(), accesskey=5,
class_=status))
# ITemplatesProvider methods
def get_htdocs_dirs(self):
"""Return the directories containing static resources."""
return [('bitten', pkg_resources.resource_filename(__name__, 'htdocs'))]
def get_templates_dirs(self):
"""Return the directories containing templates."""
return [pkg_resources.resource_filename(__name__, 'templates')]
class BuildConfigController(Component):
"""Implements the web interface for build configurations."""
implements(IRequestHandler, IRequestFilter, INavigationContributor)
# Configuration options
chart_style = Option('bitten', 'chart_style', 'height: 220px; width: 220px;', doc=
"""Style attribute for charts. Mostly useful for setting the height and width.""")
# INavigationContributor methods
def get_active_navigation_item(self, req):
return 'build'
def get_navigation_items(self, req):
return []
# IRequestHandler methods
def match_request(self, req):
match = re.match(r'/build(?:/([\w.-]+))?/?$', req.path_info)
if match:
if match.group(1):
req.args['config'] = match.group(1)
return True
def process_request(self, req):
req.perm.require('BUILD_VIEW')
action = req.args.get('action')
view = req.args.get('view')
config = req.args.get('config')
if config:
data = self._render_config(req, config)
elif view == 'inprogress':
data = self._render_inprogress(req)
else:
data = self._render_overview(req)
add_stylesheet(req, 'bitten/bitten.css')
return 'bitten_config.html', data, None
# IRequestHandler methods
def pre_process_request(self, req, handler):
return handler
def post_process_request(self, req, template, data, content_type):
if template:
add_stylesheet(req, 'bitten/bitten.css')
return template, data, content_type
# Internal methods
def _render_overview(self, req):
data = {'title': 'Build Status'}
show_all = False
if req.args.get('show') == 'all':
show_all = True
data['show_all'] = show_all
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
configs = []
for config in BuildConfig.select(self.env, include_inactive=show_all):
rev = config.max_rev or repos.youngest_rev
try:
if not _has_permission(req.perm, repos, config.path, rev=rev):
continue
except NoSuchNode:
add_warning(req, "Configuration '%s' points to non-existing "
"path '/%s' at revision '%s'. Configuration skipped." \
% (config.name, config.path, rev))
continue
description = config.description
if description:
description = wiki_to_html(description, self.env, req)
platforms_data = []
for platform in TargetPlatform.select(self.env, config=config.name):
pd = { 'name': platform.name,
'id': platform.id,
'builds_pending': len(list(Build.select(self.env,
config=config.name, status=Build.PENDING,
platform=platform.id))),
'builds_inprogress': len(list(Build.select(self.env,
config=config.name, status=Build.IN_PROGRESS,
platform=platform.id)))
}
platforms_data.append(pd)
config_data = {
'name': config.name, 'label': config.label or config.name,
'active': config.active, 'path': config.path,
'description': description,
'builds_pending' : len(list(Build.select(self.env,
config=config.name,
status=Build.PENDING))),
'builds_inprogress' : len(list(Build.select(self.env,
config=config.name,
status=Build.IN_PROGRESS))),
'href': req.href.build(config.name),
'builds': [],
'platforms': platforms_data
}
configs.append(config_data)
if not config.active:
continue
prev_rev = None
for platform, rev, build in collect_changes(repos, config):
if rev != prev_rev:
if prev_rev is None:
chgset = repos.get_changeset(rev)
config_data['youngest_rev'] = {
'id': rev, 'href': req.href.changeset(rev),
'display_rev': repos.normalize_rev(rev),
'author': chgset.author or 'anonymous',
'date': format_datetime(chgset.date),
'message': wiki_to_oneliner(
shorten_line(chgset.message), self.env, req=req)
}
else:
break
prev_rev = rev
if build:
build_data = _get_build_data(self.env, req, build)
build_data['platform'] = platform.name
config_data['builds'].append(build_data)
else:
config_data['builds'].append({
'platform': platform.name, 'status': 'pending'
})
data['configs'] = sorted(configs, key=lambda x:x['label'].lower())
data['page_mode'] = 'overview'
in_progress_builds = Build.select(self.env, status=Build.IN_PROGRESS)
pending_builds = Build.select(self.env, status=Build.PENDING)
data['builds_pending'] = len(list(pending_builds))
data['builds_inprogress'] = len(list(in_progress_builds))
add_link(req, 'views', req.href.build(view='inprogress'),
'In Progress Builds')
add_ctxtnav(req, 'In Progress Builds',
req.href.build(view='inprogress'))
return data
def _render_inprogress(self, req):
data = {'title': 'In Progress Builds',
'page_mode': 'view-inprogress'}
db = self.env.get_db_cnx()
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
configs = []
for config in BuildConfig.select(self.env, include_inactive=False):
rev = config.max_rev or repos.youngest_rev
try:
if not _has_permission(req.perm, repos, config.path, rev=rev):
continue
except NoSuchNode:
add_warning(req, "Configuration '%s' points to non-existing "
"path '/%s' at revision '%s'. Configuration skipped." \
% (config.name, config.path, rev))
continue
self.log.debug(config.name)
if not config.active:
continue
in_progress_builds = Build.select(self.env, config=config.name,
status=Build.IN_PROGRESS, db=db)
current_builds = 0
builds = []
# sort correctly by revision.
for build in sorted(in_progress_builds,
cmp=lambda x, y: int(y.rev_time) - int(x.rev_time)):
rev = build.rev
build_data = _get_build_data(self.env, req, build)
build_data['rev'] = rev
build_data['rev_href'] = req.href.changeset(rev)
platform = TargetPlatform.fetch(self.env, build.platform)
build_data['platform'] = platform.name
build_data['steps'] = []
for step in BuildStep.select(self.env, build=build.id, db=db):
build_data['steps'].append({
'name': step.name,
'description': step.description,
'duration': to_datetime(step.stopped or int(time.time()), utc) - \
to_datetime(step.started, utc),
'status': _step_status_label[step.status],
'cls': _step_status_label[step.status].replace(' ', '-'),
'errors': step.errors,
'href': build_data['href'] + '#step_' + step.name
})
builds.append(build_data)
current_builds += 1
if current_builds == 0:
continue
description = config.description
if description:
description = wiki_to_html(description, self.env, req)
configs.append({
'name': config.name, 'label': config.label or config.name,
'active': config.active, 'path': config.path,
'description': description,
'href': req.href.build(config.name),
'builds': builds
})
data['configs'] = sorted(configs, key=lambda x:x['label'].lower())
return data
def _render_config(self, req, config_name):
db = self.env.get_db_cnx()
config = BuildConfig.fetch(self.env, config_name, db=db)
if not config:
raise HTTPNotFound("Build configuration '%s' does not exist." \
% config_name)
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
rev = config.max_rev or repos.youngest_rev
try:
_has_permission(req.perm, repos, config.path, rev=rev,
raise_error=True)
except NoSuchNode:
raise TracError("Permission checking against repository path %s "
"at revision %s failed." % (config.path, rev))
data = {'title': 'Build Configuration "%s"' \
% config.label or config.name,
'page_mode': 'view_config'}
add_link(req, 'up', req.href.build(), 'Build Status')
description = config.description
if description:
description = wiki_to_html(description, self.env, req)
pending_builds = list(Build.select(self.env,
config=config.name, status=Build.PENDING))
inprogress_builds = list(Build.select(self.env,
config=config.name, status=Build.IN_PROGRESS))
data['config'] = {
'name': config.name, 'label': config.label, 'path': config.path,
'min_rev': config.min_rev,
'min_rev_href': req.href.changeset(config.min_rev),
'max_rev': config.max_rev,
'max_rev_href': req.href.changeset(config.max_rev),
'active': config.active, 'description': description,
'browser_href': req.href.browser(config.path),
'builds_pending' : len(pending_builds),
'builds_inprogress' : len(inprogress_builds)
}
context = Context.from_request(req, config.resource)
data['context'] = context
data['config']['attachments'] = AttachmentModule(self.env).attachment_data(context)
platforms = list(TargetPlatform.select(self.env, config=config_name,
db=db))
data['config']['platforms'] = [
{ 'name': platform.name,
'id': platform.id,
'builds_pending': len(list(Build.select(self.env,
config=config.name,
status=Build.PENDING,
platform=platform.id))),
'builds_inprogress': len(list(Build.select(self.env,
config=config.name,
status=Build.IN_PROGRESS,
platform=platform.id)))
}
for platform in platforms
]
has_reports = False
for report in Report.select(self.env, config=config.name, db=db):
has_reports = True
break
if has_reports:
chart_generators = []
report_categories = list(self._report_categories_for_config(config))
for generator in ReportChartController(self.env).generators:
for category in generator.get_supported_categories():
if category in report_categories:
chart_generators.append({
'href': req.href.build(config.name, 'chart/' + category),
'category': category,
'style': self.config.get('bitten', 'chart_style'),
})
data['config']['charts'] = chart_generators
page = max(1, int(req.args.get('page', 1)))
more = False
data['page_number'] = page
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
builds_per_page = 12 * len(platforms)
idx = 0
builds = {}
revisions = []
for platform, rev, build in collect_changes(repos, config):
if idx >= page * builds_per_page:
more = True
break
elif idx >= (page - 1) * builds_per_page:
if rev not in builds:
revisions.append(rev)
builds.setdefault(rev, {})
builds[rev].setdefault('href', req.href.changeset(rev))
builds[rev].setdefault('display_rev', repos.normalize_rev(rev))
if build and build.status != Build.PENDING:
build_data = _get_build_data(self.env, req, build)
build_data['steps'] = []
for step in BuildStep.select(self.env, build=build.id,
db=db):
build_data['steps'].append({
'name': step.name,
'description': step.description,
'duration': to_datetime(step.stopped or int(time.time()), utc) - \
to_datetime(step.started, utc),
'status': _step_status_label[step.status],
'cls': _step_status_label[step.status].replace(' ', '-'),
'errors': step.errors,
'href': build_data['href'] + '#step_' + step.name
})
builds[rev][platform.id] = build_data
idx += 1
data['config']['builds'] = builds
data['config']['revisions'] = revisions
if page > 1:
if page == 2:
prev_href = req.href.build(config.name)
else:
prev_href = req.href.build(config.name, page=page - 1)
add_link(req, 'prev', prev_href, 'Previous Page')
if more:
next_href = req.href.build(config.name, page=page + 1)
add_link(req, 'next', next_href, 'Next Page')
if arity(prevnext_nav) == 4: # Trac 0.12 compat, see #450
prevnext_nav(req, 'Previous Page', 'Next Page')
else:
prevnext_nav (req, 'Page')
return data
def _report_categories_for_config(self, config):
"""Yields the categories of reports that exist for active builds
of this configuration.
"""
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""SELECT DISTINCT report.category as category
FROM bitten_build AS build
JOIN bitten_report AS report ON (report.build=build.id)
WHERE build.config=%s AND build.rev_time >= %s AND build.rev_time <= %s""",
(config.name,
config.min_rev_time(self.env),
config.max_rev_time(self.env)))
for (category,) in cursor:
yield category
class BuildController(Component):
"""Renders the build page."""
implements(INavigationContributor, IRequestHandler, ITimelineEventProvider)
log_formatters = ExtensionPoint(ILogFormatter)
report_summarizers = ExtensionPoint(IReportSummarizer)
# INavigationContributor methods
def get_active_navigation_item(self, req):
return 'build'
def get_navigation_items(self, req):
return []
# IRequestHandler methods
def match_request(self, req):
match = re.match(r'/build/([\w.-]+)/(\d+)', req.path_info)
if match:
if match.group(1):
req.args['config'] = match.group(1)
if match.group(2):
req.args['id'] = match.group(2)
return True
def process_request(self, req):
req.perm.require('BUILD_VIEW')
db = self.env.get_db_cnx()
build_id = int(req.args.get('id'))
build = Build.fetch(self.env, build_id, db=db)
if not build:
raise HTTPNotFound("Build '%s' does not exist." \
% build_id)
if req.method == 'POST':
if req.args.get('action') == 'invalidate':
self._do_invalidate(req, build, db)
req.redirect(req.href.build(build.config, build.id))
add_link(req, 'up', req.href.build(build.config),
'Build Configuration')
data = {'title': 'Build %s - %s' % (build_id,
_status_title[build.status]),
'page_mode': 'view_build',
'build': {}}
config = BuildConfig.fetch(self.env, build.config, db=db)
data['build']['config'] = {
'name': config.label or config.name,
'href': req.href.build(config.name)
}
context = Context.from_request(req, build.resource)
data['context'] = context
data['build']['attachments'] = AttachmentModule(self.env).attachment_data(context)
formatters = []
for formatter in self.log_formatters:
formatters.append(formatter.get_formatter(req, build))
summarizers = {} # keyed by report type
for summarizer in self.report_summarizers:
categories = summarizer.get_supported_categories()
summarizers.update(dict([(cat, summarizer) for cat in categories]))
data['build'].update(_get_build_data(self.env, req, build))
steps = []
for step in BuildStep.select(self.env, build=build.id, db=db):
steps.append({
'name': step.name, 'description': step.description,
'duration': pretty_timedelta(step.started, step.stopped or int(time.time())),
'status': _step_status_label[step.status],
'cls': _step_status_label[step.status].replace(' ', '-'),
'errors': step.errors,
'log': self._render_log(req, build, formatters, step),
'reports': self._render_reports(req, config, build, summarizers,
step)
})
data['build']['steps'] = steps
data['build']['can_delete'] = ('BUILD_DELETE' in req.perm \
and build.status != build.PENDING)
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
_has_permission(req.perm, repos, config.path, rev=build.rev, raise_error=True)
chgset = repos.get_changeset(build.rev)
data['build']['chgset_author'] = chgset.author
data['build']['display_rev'] = repos.normalize_rev(build.rev)
add_script(req, 'common/js/folding.js')
add_script(req, 'bitten/tabset.js')
add_script(req, 'bitten/jquery.flot.js')
add_stylesheet(req, 'bitten/bitten.css')
return 'bitten_build.html', data, None
# ITimelineEventProvider methods
def get_timeline_filters(self, req):
if 'BUILD_VIEW' in req.perm:
yield ('build', 'Builds')
def get_timeline_events(self, req, start, stop, filters):
if 'build' not in filters:
return
# Attachments (will be rendered by attachment module)
for event in AttachmentModule(self.env).get_timeline_events(
req, Resource('build'), start, stop):
yield event
start = to_timestamp(start)
stop = to_timestamp(stop)
add_stylesheet(req, 'bitten/bitten.css')
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT b.id,b.config,c.label,c.path, b.rev,p.name,"
"b.stopped,b.status FROM bitten_build AS b"
" INNER JOIN bitten_config AS c ON (c.name=b.config) "
" INNER JOIN bitten_platform AS p ON (p.id=b.platform) "
"WHERE b.stopped>=%s AND b.stopped<=%s "
"AND b.status IN (%s, %s) ORDER BY b.stopped",
(start, stop, Build.SUCCESS, Build.FAILURE))
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
event_kinds = {Build.SUCCESS: 'successbuild',
Build.FAILURE: 'failedbuild'}
for id_, config, label, path, rev, platform, stopped, status in cursor:
if not _has_permission(req.perm, repos, path, rev=rev):
continue
errors = []
if status == Build.FAILURE:
for step in BuildStep.select(self.env, build=id_,
status=BuildStep.FAILURE,
db=db):
errors += [(step.name, error) for error
in step.errors]
display_rev = repos.normalize_rev(rev)
yield (event_kinds[status], to_datetime(stopped, utc), None,
(id_, config, label, display_rev, platform, status,
errors))
def render_timeline_event(self, context, field, event):
id_, config, label, rev, platform, status, errors = event[3]
if field == 'url':
return context.href.build(config, id_)
elif field == 'title':
return tag('Build of ', tag.em('%s [%s]' % (label, rev)),
' on %s %s' % (platform, _status_label[status]))
elif field == 'description':
message = ''
if context.req.args.get('format') == 'rss':
if errors:
buf = StringIO()
prev_step = None
for step, error in errors:
if step != prev_step:
if prev_step is not None:
buf.write('')
buf.write('
Step %s failed:
' \
% escape(step))
prev_step = step
buf.write('
%s
' % escape(error))
buf.write('
')
message = Markup(buf.getvalue())
else:
if errors:
steps = []
for step, error in errors:
if step not in steps:
steps.append(step)
steps = [Markup('%s') % step for step in steps]
if len(steps) < 2:
message = steps[0]
elif len(steps) == 2:
message = Markup(' and ').join(steps)
elif len(steps) > 2:
message = Markup(', ').join(steps[:-1]) + ', and ' + \
steps[-1]
message = Markup('Step%s %s failed') % (
len(steps) != 1 and 's' or '', message)
return message
# Internal methods
def _do_invalidate(self, req, build, db):
self.log.info('Invalidating build %d', build.id)
for step in BuildStep.select(self.env, build=build.id, db=db):
step.delete(db=db)
build.slave = None
build.started = 0
build.stopped = 0
build.last_activity = 0
build.status = Build.PENDING
build.slave_info = {}
build.update()
Attachment.delete_all(self.env, 'build', build.resource.id, db)
db.commit()
req.redirect(req.href.build(build.config))
def _render_log(self, req, build, formatters, step):
items = []
for log in BuildLog.select(self.env, build=build.id, step=step.name):
for level, message in log.messages:
for format in formatters:
message = format(step, log.generator, level, message)
items.append({'level': level, 'message': message})
return items
def _render_reports(self, req, config, build, summarizers, step):
reports = []
for report in Report.select(self.env, build=build.id, step=step.name):
summarizer = summarizers.get(report.category)
if summarizer:
tmpl, data = summarizer.render_summary(req, config, build,
step, report.category)
reports.append({'category': report.category,
'template': tmpl, 'data': data})
else:
tmpl = data = None
return reports
class ReportChartController(Component):
implements(IRequestHandler)
generators = ExtensionPoint(IReportChartGenerator)
# IRequestHandler methods
def match_request(self, req):
match = re.match(r'/build/([\w.-]+)/chart/(\w+)', req.path_info)
if match:
req.args['config'] = match.group(1)
req.args['category'] = match.group(2)
return True
def process_request(self, req):
category = req.args.get('category')
config = BuildConfig.fetch(self.env, name=req.args.get('config'))
for generator in self.generators:
if category in generator.get_supported_categories():
tmpl, data = generator.generate_chart_data(req, config,
category)
break
else:
raise TracError('Unknown report category "%s"' % category)
data['dumps'] = json.to_json
return tmpl, data, 'text/plain'
class SourceFileLinkFormatter(Component):
"""Detects references to files in the build log and renders them as links
to the repository browser.
"""
implements(ILogFormatter)
_fileref_re = re.compile(r'(?P-[A-Za-z])?(?P[\w.-]+(?:[\\/][\w.-]+)+)(?P:\d+)?')
def get_formatter(self, req, build):
"""Return the log message formatter function."""
config = BuildConfig.fetch(self.env, name=build.config)
repos = self.env.get_repository(authname=req.authname)
assert repos, 'No "(default)" Repository: Add a repository or alias ' \
'named "(default)" to Trac.'
href = req.href.browser
cache = {}
def _replace(m):
filepath = posixpath.normpath(m.group('path').replace('\\', '/'))
if not cache.get(filepath) is True:
parts = filepath.split('/')
path = ''
for part in parts:
path = posixpath.join(path, part)
if path not in cache:
try:
full_path = posixpath.join(config.path, path)
full_path = posixpath.normpath(full_path)
if full_path.startswith(config.path + "/") \
or full_path == config.path:
repos.get_node(full_path,
build.rev)
cache[path] = True
else:
cache[path] = False
except TracError:
cache[path] = False
if cache[path] is False:
return m.group(0)
link = href(config.path, filepath)
if m.group('line'):
link += '#L' + m.group('line')[1:]
return Markup(tag.a(m.group(0), href=link))
def _formatter(step, type, level, message):
buf = []
offset = 0
for mo in self._fileref_re.finditer(message):
start, end = mo.span()
if start > offset:
buf.append(message[offset:start])
buf.append(_replace(mo))
offset = end
if offset < len(message):
buf.append(message[offset:])
return Markup("").join(buf)
return _formatter
Bitten-0.6/bitten/util/__init__.py 000644 000765 000120 00000001115 11453043212 017422 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Generic utility functions and classes.
Functionality in these modules have no dependencies on modules outside of this
package, so that they could theoretically be used in other projects.
"""
__docformat__ = 'restructuredtext en'
Bitten-0.6/bitten/util/compat.py 000644 000765 000120 00000003244 11535711762 017170 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2011 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Compatibility fixes for external libraries and Python."""
import sys
# Fix for issue http://bugs.python.org/issue8797 in Python 2.6
if sys.version_info[:2] == (2, 6):
import urllib2
import base64
class HTTPBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
"""Patched version of Python 2.6's HTTPBasicAuthHandler.
The fix for [1]_ introduced an infinite recursion bug [2]_ into
Python 2.6.x that is triggered by attempting to connect using
Basic authentication with a bad username and/or password. This
class fixes the problem using the simple solution outlined in [3]_.
.. [1] http://bugs.python.org/issue3819
.. [2] http://bugs.python.org/issue8797
.. [3] http://bugs.python.org/issue8797#msg126657
"""
def retry_http_basic_auth(self, host, req, realm):
user, pw = self.passwd.find_user_password(realm, host)
if pw is not None:
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
if req.get_header(self.auth_header, None) == auth:
return None
req.add_unredirected_header(self.auth_header, auth)
return self.parent.open(req, timeout=req.timeout)
else:
return None
else:
from urllib2 import HTTPBasicAuthHandler
Bitten-0.6/bitten/util/json.py 000644 000765 000120 00000006473 11366024332 016655 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C)2006-2009 Edgewall Software
# Copyright (C) 2006 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://trac.edgewall.org/wiki/TracLicense.
#
# This software consists of voluntary contributions made by many
# individuals. For the exact contribution history, see the revision
# history and logs, available at http://trac.edgewall.org/log/.
"""Utility functions for converting to web formats"""
# to_json is really a wrapper for trac.util.presentation.to_json, but we have lots of fallbacks for compatibility
# trac.util.presentation.to_json is present from Trac 0.12
# If that's not present, we fall back to the default Python json module, present from Python 2.6 onwards
# If that's not present, we have a copy of the to_json method, which we will remove once Trac 0.11 support is removed
# And finally, the to_json method requires trac.util.text.javascript_quote, which is only present from Trac 0.11.3, so we have a copy of that too
import re
try:
# to_json is present from Trac 0.12 onwards - should remove once Trac 0.11 support is removed
from trac.util.presentation import to_json
except ImportError:
try:
# If we have Python 2.6 onwards, use the json method directly
from json import dumps
def to_json(value):
"""Encode `value` to JSON."""
return dumps(value, sort_keys=True, separators=(',', ':'))
except ImportError:
# javascript_quote is present from Trac 0.11.3 onwards - should remove once Trac 0.11.2 support is removed
try:
from trac.util.text import javascript_quote
except ImportError:
_js_quote = {'\\': '\\\\', '"': '\\"', '\b': '\\b', '\f': '\\f',
'\n': '\\n', '\r': '\\r', '\t': '\\t', "'": "\\'"}
for i in range(0x20):
_js_quote.setdefault(chr(i), '\\u%04x' % i)
_js_quote_re = re.compile(r'[\x00-\x1f\\"\b\f\n\r\t\']')
def javascript_quote(text):
"""Quote strings for inclusion in javascript"""
if not text:
return ''
def replace(match):
return _js_quote[match.group(0)]
return _js_quote_re.sub(replace, text)
def to_json(value):
"""Encode `value` to JSON."""
if isinstance(value, basestring):
return '"%s"' % javascript_quote(value)
elif value is None:
return 'null'
elif value is False:
return 'false'
elif value is True:
return 'true'
elif isinstance(value, (int, long)):
return str(value)
elif isinstance(value, float):
return repr(value)
elif isinstance(value, (list, tuple)):
return '[%s]' % ','.join(to_json(each) for each in value)
elif isinstance(value, dict):
return '{%s}' % ','.join('%s:%s' % (to_json(k), to_json(v))
for k, v in sorted(value.iteritems()))
else:
raise TypeError('Cannot encode type %s' % value.__class__.__name__)
Bitten-0.6/bitten/util/loc.py 000644 000765 000120 00000012252 11453043212 016444 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 1998 Dinu C. Gherman
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
#
# This module is based on the pycount.py script written by Dinu C.
# Gherman, and is used here under the following license:
#
# Permission to use, copy, modify, and distribute this software
# and its documentation without fee and for any purpose, except
# direct commerial advantage, is hereby granted, provided that
# the above copyright notice appear in all copies and that both
# that copyright notice and this permission notice appear in
# supporting documentation.
"""Support for counting the lines of code in Python programs."""
import re
__all__ = ['BLANK', 'CODE', 'COMMENT', 'DOC', 'count']
__docformat__ = 'restructuredtext en'
# Reg. exps. to find the end of a triple quote, given that
# we know we're in one; use the "match" method; .span()[1]
# will be the index of the character following the final
# quote.
_squote3_finder = re.compile(
r"([^\']|"
r"\.|"
r"'[^\']|"
r"'\.|"
r"''[^\']|"
r"''\.)*'''")
_dquote3_finder = re.compile(
r'([^\"]|'
r'\.|'
r'"[^\"]|'
r'"\.|'
r'""[^\"]|'
r'""\.)*"""')
# Reg. exps. to find the leftmost one-quoted string; use the
# "search" method; .span()[0] bounds the string found.
_dquote1_finder = re.compile(r'"([^"]|\.)*"')
_squote1_finder = re.compile(r"'([^']|\.)*'")
# _is_comment matches pure comment line.
_is_comment = re.compile(r"^[ \t]*#").match
# _is_blank matches empty line.
_is_blank = re.compile(r"^[ \t]*$").match
# find leftmost splat or quote.
_has_nightmare = re.compile(r"""[\"'#]""").search
# _is_doc_candidate matches lines that start with a triple quote.
_is_doc_candidate = re.compile(r"^[ \t]*('''|\"\"\")")
BLANK, CODE, COMMENT, DOC = 0, 1, 2, 3
def count(source):
"""Parse the given file-like object as Python source code.
For every line in the code, this function yields a ``(lineno, type, line)``
tuple, where ``lineno`` is the line number (starting at 0), ``type`` is
one of `BLANK`, `CODE`, `COMMENT` or `DOC`, and ``line`` is the actual
content of the line.
:param source: a file-like object containing Python code
"""
quote3_finder = {'"': _dquote3_finder, "'": _squote3_finder}
quote1_finder = {'"': _dquote1_finder, "'": _squote1_finder }
in_doc = False
in_triple_quote = None
for lineno, line in enumerate(source):
classified = False
if in_triple_quote:
if in_doc:
yield lineno, DOC, line
else:
yield lineno, CODE, line
classified = True
m = in_triple_quote.match(line)
if m == None:
continue
# Get rid of everything through the end of the triple.
end = m.span()[1]
line = line[end:]
in_doc = in_triple_quote = False
if _is_blank(line):
if not classified:
yield lineno, BLANK, line
continue
if _is_comment(line):
if not classified:
yield lineno, COMMENT, line
continue
# Now we have a code line, a doc start line, or crap left
# over following the close of a multi-line triple quote; in
# (& only in) the last case, classified==1.
if not classified:
if _is_doc_candidate.match(line):
yield lineno, DOC, line
in_doc = True
else:
yield lineno, CODE, line
# The only reason to continue parsing is to make sure the
# start of a multi-line triple quote isn't missed.
while True:
m = _has_nightmare(line)
if not m:
break
else:
i = m.span()[0]
ch = line[i] # splat or quote
if ch == '#':
# Chop off comment; and there are no quotes
# remaining because splat was leftmost.
break
# A quote is leftmost.
elif ch * 3 == line[i:i + 3]:
# at the start of a triple quote
in_triple_quote = quote3_finder[ch]
m = in_triple_quote.match(line, i + 3)
if m:
# Remove the string & continue.
end = m.span()[1]
line = line[:i] + line[end:]
in_doc = in_triple_quote = False
else:
# Triple quote doesn't end on this line.
break
else:
# At a single quote; remove the string & continue.
prev_line = line[:]
line = re.sub(quote1_finder[ch], ' ', line, 1)
# No more change detected, so be quiet or give up.
if prev_line == line:
# Let's be quiet and hope only one line is affected.
line = ''
Bitten-0.6/bitten/util/testrunner.py 000644 000765 000120 00000023572 11453043212 020107 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2008 Matt Good
# Copyright (C) 2008-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
from distutils import log
from distutils.errors import DistutilsOptionError
import os
import re
from StringIO import StringIO
import sys
import time
from pkg_resources import Distribution, EntryPoint, PathMetadata, \
normalize_path, require, working_set
from setuptools.command.test import test
from unittest import _TextTestResult, TextTestRunner
from bitten import __version__ as VERSION
from bitten.util import xmlio
__docformat__ = 'restructuredtext en'
class XMLTestResult(_TextTestResult):
def __init__(self, stream, descriptions, verbosity):
_TextTestResult.__init__(self, stream, descriptions, verbosity)
self.tests = []
def startTest(self, test):
_TextTestResult.startTest(self, test)
filename = sys.modules[test.__module__].__file__
if filename.endswith('.pyc') or filename.endswith('.pyo'):
filename = filename[:-1]
self.tests.append([test, filename, time.time(), None, None])
def stopTest(self, test):
self.tests[-1][2] = time.time() - self.tests[-1][2]
_TextTestResult.stopTest(self, test)
class XMLTestRunner(TextTestRunner):
def __init__(self, stream=sys.stdout, xml_stream=None):
TextTestRunner.__init__(self, stream, descriptions=0, verbosity=2)
self.xml_stream = xml_stream
def _makeResult(self):
return XMLTestResult(self.stream, self.descriptions, self.verbosity)
def run(self, test):
result = TextTestRunner.run(self, test)
if not self.xml_stream:
return result
root = xmlio.Element('unittest-results')
for testcase, filename, timetaken, stdout, stderr in result.tests:
status = 'success'
tb = None
if testcase in [e[0] for e in result.errors]:
status = 'error'
tb = [e[1] for e in result.errors if e[0] is testcase][0]
elif testcase in [f[0] for f in result.failures]:
status = 'failure'
tb = [f[1] for f in result.failures if f[0] is testcase][0]
name = str(testcase)
fixture = None
description = testcase.shortDescription() or ''
if description.startswith('doctest of '):
name = 'doctest'
fixture = description[11:]
description = None
else:
match = re.match('(\w+)\s+\(([\w.]+)\)', name)
if match:
name = match.group(1)
fixture = match.group(2)
test_elem = xmlio.Element('test', file=filename, name=name,
fixture=fixture, status=status,
duration=timetaken)
if description:
test_elem.append(xmlio.Element('description')[description])
if stdout:
test_elem.append(xmlio.Element('stdout')[stdout])
if stderr:
test_elem.append(xmlio.Element('stdout')[stderr])
if tb:
test_elem.append(xmlio.Element('traceback')[tb])
root.append(test_elem)
root.write(self.xml_stream, newlines=True)
return result
class unittest(test):
description = test.description + ', and optionally record code coverage'
user_options = test.user_options + [
('xml-output=', None,
"Path to the XML file where test results are written to"),
('coverage-dir=', None,
"Directory where coverage files are to be stored"),
('coverage-summary=', None,
"Path to the file where the coverage summary should be stored"),
('coverage-method=', None,
"Whether to use trace.py or coverage.py to collect code coverage. "
"Valid options are 'trace' (the default) or 'coverage'.")
]
def initialize_options(self):
test.initialize_options(self)
self.xml_output = None
self.xml_output_file = None
self.coverage_summary = None
self.coverage_dir = None
self.coverage_method = 'trace'
def finalize_options(self):
test.finalize_options(self)
if self.xml_output is not None:
output_dir = os.path.dirname(self.xml_output) or '.'
if not os.path.exists(output_dir):
os.makedirs(output_dir)
self.xml_output_file = open(self.xml_output, 'w')
if self.coverage_method not in ('trace', 'coverage', 'figleaf'):
raise DistutilsOptionError('Unknown coverage method %r' %
self.coverage_method)
def run_tests(self):
if self.coverage_summary:
if self.coverage_method == 'coverage':
self._run_with_coverage()
elif self.coverage_method == 'figleaf':
self._run_with_figleaf()
else:
self._run_with_trace()
else:
self._run_tests()
def _run_with_figleaf(self):
import figleaf
figleaf.start()
try:
self._run_tests()
finally:
figleaf.stop()
figleaf.write_coverage(self.coverage_summary)
def _run_with_coverage(self):
import coverage
coverage.use_cache(False)
coverage.start()
try:
self._run_tests()
finally:
coverage.stop()
modules = [m for _, m in sys.modules.items()
if m is not None and hasattr(m, '__file__')
and os.path.splitext(m.__file__)[-1] in ('.py', '.pyc')]
# Generate summary file
buf = StringIO()
coverage.report(modules, file=buf)
buf.seek(0)
fileobj = open(self.coverage_summary, 'w')
try:
filter_coverage(buf, fileobj)
finally:
fileobj.close()
if self.coverage_dir:
if not os.path.exists(self.coverage_dir):
os.makedirs(self.coverage_dir)
coverage.annotate(modules, directory=self.coverage_dir,
ignore_errors=True)
def _run_with_trace(self):
from trace import Trace
trace = Trace(ignoredirs=[sys.prefix, sys.exec_prefix], trace=False,
count=True)
try:
trace.runfunc(self._run_tests)
finally:
results = trace.results()
real_stdout = sys.stdout
sys.stdout = open(self.coverage_summary, 'w')
try:
results.write_results(show_missing=True, summary=True,
coverdir=self.coverage_dir)
finally:
sys.stdout.close()
sys.stdout = real_stdout
def _run_tests(self):
old_path = sys.path[:]
ei_cmd = self.get_finalized_command("egg_info")
path_item = normalize_path(ei_cmd.egg_base)
metadata = PathMetadata(
path_item, normalize_path(ei_cmd.egg_info)
)
dist = Distribution(path_item, metadata, project_name=ei_cmd.egg_name)
working_set.add(dist)
require(str(dist.as_requirement()))
loader_ep = EntryPoint.parse("x=" + self.test_loader)
loader_class = loader_ep.load(require=False)
try:
import unittest
unittest.main(
None, None, [unittest.__file__] + self.test_args,
testRunner=XMLTestRunner(stream=sys.stdout,
xml_stream=self.xml_output_file),
testLoader=loader_class()
)
except SystemExit, e:
return e.code
def filter_coverage(infile, outfile):
for idx, line in enumerate(infile):
if idx < 2 or line.startswith('--'):
outfile.write(line)
continue
parts = line.split()
name = parts[0]
if name == 'TOTAL':
continue
if name not in sys.modules:
outfile.write(line)
continue
filename = os.path.normpath(sys.modules[name].__file__)
if filename.endswith('.pyc') or filename.endswith('.pyo'):
filename = filename[:-1]
outfile.write(line.rstrip() + ' ' + filename + '\n')
def main():
from distutils.dist import Distribution
from optparse import OptionParser
parser = OptionParser(usage='usage: %prog [options] test_suite ...',
version='%%prog %s' % VERSION)
parser.add_option('-o', '--xml-output', action='store', dest='xml_output',
metavar='FILE', help='write XML test results to FILE')
parser.add_option('-d', '--coverage-dir', action='store',
dest='coverage_dir', metavar='DIR',
help='store coverage results in DIR')
parser.add_option('-s', '--coverage-summary', action='store',
dest='coverage_summary', metavar='FILE',
help='write coverage summary to FILE')
options, args = parser.parse_args()
if len(args) < 1:
parser.error('incorrect number of arguments')
cmd = unittest(Distribution())
cmd.initialize_options()
cmd.test_suite = args[0]
if hasattr(options, 'xml_output'):
cmd.xml_output = options.xml_output
if hasattr(options, 'coverage_summary'):
cmd.coverage_summary = options.coverage_summary
if hasattr(options, 'coverage_dir'):
cmd.coverage_dir = options.coverage_dir
cmd.finalize_options()
cmd.run()
if __name__ == '__main__':
main()
Bitten-0.6/bitten/util/tests/ 000755 000765 000120 00000000000 11536424600 016463 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/util/xmlio.py 000644 000765 000120 00000026200 11453043212 017015 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
"""Utility code for easy input and output of XML.
The current implementation uses ``xml.dom.minidom`` under the hood for parsing.
"""
import os
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
from UserDict import DictMixin
import cgi
import string
__all__ = ['Fragment', 'Element', 'ParsedElement', 'parse']
__docformat__ = 'restructuredtext en'
def _from_utf8(text):
"""Convert utf-8 string to unicode. All other input returned as-is."""
if isinstance(text, str):
return text.decode('utf-8')
else:
return text
def _to_utf8(text):
"""Convert any input to utf-8 byte string."""
if isinstance(text, str):
return text # presumes utf-8
elif not isinstance(text, unicode):
text = unicode(text)
return text.encode('utf-8')
__trans = string.maketrans('', '')
# http://www.w3.org/TR/xml11/#charsets (partial)
__todel = ('\x01\x02\x03\x04\x05\x06\x07\x08\x0b\x0c\x0e\x0f\x10\x12\x13\x14'
'\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x7f\x80\x81\x82\x83'
'\x84\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94'
'\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f')
__uni_trans = dict([(ord(c), None) for c in __todel])
def _escape_text(text):
"""Escape special characters in the provided text so that it can be safely
included in XML text nodes.
"""
if isinstance(text, str):
text = cgi.escape(text.translate(__trans, __todel))
elif isinstance(text, unicode):
text = cgi.escape(text.translate(__uni_trans))
return text
def _escape_attr(attr):
"""Escape special characters in the provided text so that it can be safely
included in XML attribute values.
"""
if isinstance(attr, basestring):
return _escape_text(attr).replace('"', '"')
else:
return attr
class Fragment(object):
"""A collection of XML elements."""
__slots__ = ['children']
def __init__(self):
"""Create an XML fragment."""
self.children = []
def __getitem__(self, nodes):
"""Add nodes to the fragment."""
if not isinstance(nodes, (list, tuple)):
nodes = [nodes]
for node in nodes:
self.append(node)
return self
def __str__(self):
"""Return a string representation of the XML fragment."""
buf = StringIO()
self.write(buf)
return buf.getvalue()
def append(self, node):
"""Append an element or fragment as child."""
if isinstance(node, Element):
self.children.append(node)
elif isinstance(node, Fragment):
self.children += node.children
elif node is not None and node != '':
if isinstance(node, basestring):
self.children.append(_from_utf8(node))
else:
self.children.append(unicode(node))
def write(self, out, newlines=False):
"""Serializes the element and writes the XML to the given output
stream.
"""
for child in self.children:
if isinstance(child, (Element, ParsedElement)):
child.write(out, newlines=newlines)
else:
if child.startswith('<'):
out.write('')
else:
out.write(_to_utf8(_escape_text(child)))
class Element(Fragment):
"""Simple XML output generator based on the builder pattern.
Construct XML elements by passing the tag name to the constructor:
>>> print Element('foo')
Attributes can be specified using keyword arguments. The values of the
arguments will be converted to strings and any special XML characters
escaped:
>>> print Element('foo', bar=42)
>>> print Element('foo', bar='1 < 2')
>>> print Element('foo', bar='"baz"')
The order in which attributes are rendered is undefined.
Elements can be using item access notation:
>>> print Element('foo')[Element('bar'), Element('baz')]
Text nodes can be nested in an element by using strings instead of elements
in item access. Any special characters in the strings are escaped
automatically:
>>> print Element('foo')['Hello world']
Hello world
>>> print Element('foo')[42]
42
>>> print Element('foo')['1 < 2']
1 < 2
This technique also allows mixed content:
>>> print Element('foo')['Hello ', Element('b')['world']]
Hello world
Finally, text starting with an opening angle bracket is treated specially:
under the assumption that the text actually contains XML itself, the whole
thing is wrapped in a CDATA block instead of escaping all special characters
individually:
>>> print Element('foo')['']
]]>
Valid input are utf-8 or unicode strings, or any type easily converted
to unicode such as integers. Output is always utf-8.
"""
__slots__ = ['name', 'attr']
def __init__(self, name_, **attr):
"""Create an XML element using the specified tag name.
The tag name must be supplied as the first positional argument. All
keyword arguments following it are handled as attributes of the element.
"""
Fragment.__init__(self)
self.name = _from_utf8(name_)
self.attr = dict([(_from_utf8(name), _from_utf8(value)) \
for name, value in attr.items() \
if value is not None])
def write(self, out, newlines=False):
"""Serializes the element and writes the XML to the given output
stream.
"""
out.write('<')
out.write(_to_utf8(self.name))
for name, value in self.attr.items():
out.write(_to_utf8(' %s="%s"' % (name, _escape_attr(value))))
if self.children:
out.write('>')
Fragment.write(self, out, newlines)
out.write('')
else:
out.write('/>')
if newlines:
out.write(os.linesep)
class ParseError(Exception):
"""Exception thrown when there's an error parsing an XML document."""
def parse(text_or_file):
"""Parse an XML document provided as string or file-like object.
Returns an instance of `ParsedElement` that can be used to traverse the
parsed document.
"""
from xml.dom import minidom
from xml.parsers import expat
try:
if isinstance(text_or_file, basestring):
dom = minidom.parseString(_to_utf8(text_or_file))
else:
dom = minidom.parse(text_or_file)
return ParsedElement(dom.documentElement)
except expat.error, e:
raise ParseError(e)
class ParsedElement(object):
"""Representation of an XML element that was parsed from a string or
file.
This class should not be used directly. Rather, XML text parsed using
`xmlio.parse()` will return an instance of this class.
>>> xml = parse('')
>>> print xml.name
root
Parsed elements can be serialized to a string using the `write()` method:
>>> import sys
>>> parse('').write(sys.stdout)
For convenience, this is also done when coercing the object to a string
using the builtin ``str()`` function, which is used when printing an
object:
>>> print parse('')
(Note that serializing the element will produce a normalized representation
that may not excatly match the input string.)
Attributes are accessed via the `attr` member:
>>> print parse('').attr['foo']
bar
Attributes can also be updated, added or removed:
>>> xml = parse('')
>>> xml.attr['foo'] = 'baz'
>>> print xml
>>> del xml.attr['foo']
>>> print xml
>>> xml.attr['foo'] = 'bar'
>>> print xml
CDATA sections are included in the text content of the element returned by
`gettext()`:
>>> xml = parse('foo ]]>baz')
>>> xml.gettext()
'foo baz'
Valid input are utf-8 or unicode strings, or any type easily converted
to unicode such as integers. Output is always utf-8.
"""
__slots__ = ['_node', 'attr']
class _Attrs(DictMixin):
"""Simple wrapper around the element attributes to provide a dictionary
interface."""
def __init__(self, node):
self._node = node
def __getitem__(self, name):
attr = self._node.getAttributeNode(name)
if not attr:
raise KeyError(name)
return _to_utf8(attr.value)
def __setitem__(self, name, value):
self._node.setAttribute(name, value)
def __delitem__(self, name):
self._node.removeAttribute(name)
def keys(self):
return [_to_utf8(key) for key in self._node.attributes.keys()]
def __init__(self, node):
self._node = node
self.attr = ParsedElement._Attrs(node)
name = property(fget=lambda self: self._node.localName,
doc='Local name of the element')
namespace = property(fget=lambda self: self._node.namespaceURI,
doc='Namespace URI of the element')
def children(self, name=None):
"""Iterate over the child elements of this element.
If the parameter `name` is provided, only include elements with a
matching local name. Otherwise, include all elements.
"""
for child in [c for c in self._node.childNodes if c.nodeType == 1]:
if name in (None, child.tagName):
yield ParsedElement(child)
def __iter__(self):
return self.children()
def gettext(self):
"""Return the text content of this element.
This concatenates the values of all text and CDATA nodes that are
immediate children of this element.
"""
return ''.join([_to_utf8(c.nodeValue)
for c in self._node.childNodes
if c.nodeType in (3, 4)])
def write(self, out, newlines=False):
"""Serializes the element and writes the XML to the given output
stream.
"""
out.write(self._node.toprettyxml(newl=newlines and '\n' or '',
indent=newlines and '\t' or '', encoding='utf-8'))
def __str__(self):
"""Return a string representation of the XML element."""
buf = StringIO()
self.write(buf)
return buf.getvalue()
if __name__ == '__main__':
import doctest
doctest.testmod()
Bitten-0.6/bitten/util/tests/__init__.py 000644 000765 000120 00000001433 11453043212 020567 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import doctest
import unittest
from bitten.util import xmlio as xmlio_module
from bitten.util.tests import xmlio as xmlio_tests
from bitten.util.tests import json as json_tests
def suite():
suite = unittest.TestSuite()
suite.addTest(doctest.DocTestSuite(xmlio_module))
suite.addTest(json_tests.suite())
suite.addTest(xmlio_tests.suite())
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/util/tests/json.py 000644 000765 000120 00000003112 11365777015 020016 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C)2006-2009 Edgewall Software
# Copyright (C) 2006 Christopher Lenz
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://trac.edgewall.org/wiki/TracLicense.
#
# This software consists of voluntary contributions made by many
# individuals. For the exact contribution history, see the revision
# history and logs, available at http://trac.edgewall.org/log/.
import doctest
import unittest
from bitten.util import json
class ToJsonTestCase(unittest.TestCase):
def test_simple_types(self):
self.assertEqual('42', json.to_json(42))
self.assertEqual('123.456', json.to_json(123.456))
self.assertEqual('true', json.to_json(True))
self.assertEqual('false', json.to_json(False))
self.assertEqual('null', json.to_json(None))
self.assertEqual('"String"', json.to_json('String'))
self.assertEqual(r'"a \" quote"', json.to_json('a " quote'))
def test_compound_types(self):
self.assertEqual('[1,2,[true,false]]',
json.to_json([1, 2, [True, False]]))
self.assertEqual('{"one":1,"other":[null,0],"two":2}',
json.to_json({"one": 1, "two": 2,
"other": [None, 0]}))
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(ToJsonTestCase))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/util/tests/xmlio.py 000644 000765 000120 00000005005 11453043212 020157 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import shutil
import tempfile
import unittest
from bitten.util import xmlio
class XMLIOTestCase(unittest.TestCase):
def test_parse(self):
"""Tests that simple test data is parsed correctly"""
s = """\
"""
x = xmlio.parse(s)
assert x.name == "build"
def test_Element_encoding(self):
self.assertEquals('<\xc3\xb8\xc3\xbc arg="\xc3\xa9\xe2\x82\xac"/>',
str(xmlio.Element(u'\xf8\xfc', arg=u'\xe9\u20ac'.encode('utf-8'))))
def test_ParsedElement_encoding(self):
u = u''
s = ''
self.assertEquals(u, s.decode('utf-8'))
# unicode input
x = xmlio.parse(u)
out_u = str(x)
self.assertEquals(out_u, s)
self.assertEquals(out_u.decode('utf-8'), u)
# utf-8 input
x = xmlio.parse(s)
out_s = str(x)
self.assertEquals(out_s, s)
self.assertEquals(out_s.decode('utf-8'), u)
# identical results
self.assertEquals(out_u, out_s)
def test_escape_text(self):
# unicode
self.assertEquals(u"Me & you!",
xmlio._escape_text(u"Me\x01 & you\x86!"))
# str
self.assertEquals("Me & you!",
xmlio._escape_text("Me\x01 & you\x86!"))
# not basestring
self.assertEquals(42, xmlio._escape_text(42))
def test_escape_attr(self):
# unicode
self.assertEquals(u'"Me & you!"',
xmlio._escape_attr(u'"Me\x01 & you\x86!"'))
# str
self.assertEquals('"Me & you!"',
xmlio._escape_attr('"Me\x01 & you\x86!"'))
# not basestring
self.assertEquals(42, xmlio._escape_text(42))
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(XMLIOTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests_slave/__init__.py 000644 000765 000120 00000001365 11453043212 021010 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
from bitten.tests_slave import recipe, slave
from bitten.build import tests as build
from bitten.util import tests as util
def suite():
suite = unittest.TestSuite()
suite.addTest(recipe.suite())
suite.addTest(slave.suite())
suite.addTest(build.suite())
suite.addTest(util.suite())
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests_slave/recipe.py 000644 000765 000120 00000016247 11454420025 020527 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import shutil
import tempfile
import unittest
from bitten.build.config import Configuration
from bitten.recipe import Context, Recipe, InvalidRecipeError
from bitten.util import xmlio
class ContextTestCase(unittest.TestCase):
def setUp(self):
self.basedir = os.path.realpath(tempfile.mkdtemp())
def tearDown(self):
shutil.rmtree(self.basedir)
def test_vars_basedir(self):
config = Configuration(properties={'foo.bar': 'baz'})
ctxt = Context('%s/${path}/${foo.bar}' % os.path.realpath('/foo'),
config, {'path': 'bar'})
self.assertEquals(os.path.realpath('/foo/bar/baz'),
os.path.realpath(ctxt.vars['basedir']))
if os.name == 'nt':
# Make sure paths are double-escaped
self.failUnless('\\\\' in ctxt.vars['basedir'])
def test_run_wrong_arg(self):
ctxt = Context(self.basedir)
try:
ctxt.run(1, 'http://bitten.edgewall.org/tools/sh', 'exec', {'foo':'bar'})
self.fail("InvalidRecipeError expected")
except InvalidRecipeError, e:
self.failUnless("Unsupported argument 'foo'" in str(e))
def test_attach_file_config(self):
# Verify output from attaching a file to a config
ctxt = Context(self.basedir, Configuration())
ctxt.attach(file_='config.txt', description='config config',
resource='config')
self.assertEquals(1, len(ctxt.output))
self.assertEquals(Recipe.ATTACH, ctxt.output[0][0])
attach_xml = ctxt.output[0][3]
self.assertEquals('', str(attach_xml))
def test_attach_file_build(self):
# Verify output from attaching a file to a build
ctxt = Context(self.basedir, Configuration())
ctxt.attach(file_='build.txt', description='build build')
self.assertEquals(1, len(ctxt.output))
self.assertEquals(Recipe.ATTACH, ctxt.output[0][0])
attach_xml = ctxt.output[0][3]
self.assertEquals('', str(attach_xml))
class RecipeTestCase(unittest.TestCase):
def setUp(self):
self.basedir = os.path.realpath(tempfile.mkdtemp())
def tearDown(self):
shutil.rmtree(self.basedir)
def test_empty_recipe(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertEqual(self.basedir, recipe.ctxt.basedir)
steps = list(recipe)
self.assertEqual(0, len(steps))
def test_empty_step(self):
xml = xmlio.parse(''
' '
'')
recipe = Recipe(xml, basedir=self.basedir)
steps = list(recipe)
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].id)
self.assertEqual('Bar', steps[0].description)
self.assertEqual('fail', steps[0].onerror)
def test_validate_bad_root(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_no_steps(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_child_not_step(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_child_not_step(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_step_without_id(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_step_with_empty_id(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_step_without_commands(self):
xml = xmlio.parse('')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_step_with_command_children(self):
xml = xmlio.parse(''
''
'')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_step_with_duplicate_id(self):
xml = xmlio.parse(''
''
''
'')
recipe = Recipe(xml, basedir=self.basedir)
self.assertRaises(InvalidRecipeError, recipe.validate)
def test_validate_successful(self):
xml = xmlio.parse(''
''
''
'')
recipe = Recipe(xml, basedir=self.basedir)
recipe.validate()
def test_onerror_defaults(self):
xml = xmlio.parse(''
' '
'')
recipe = Recipe(xml, basedir=self.basedir)
steps = list(recipe)
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].id)
self.assertEqual('Bar', steps[0].description)
self.assertEqual('continue', steps[0].onerror)
def test_onerror_override(self):
xml = xmlio.parse(''
' '
'')
recipe = Recipe(xml, basedir=self.basedir)
steps = list(recipe)
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].id)
self.assertEqual('Bar', steps[0].description)
self.assertEqual('continue', steps[0].onerror)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(ContextTestCase, 'test'))
suite.addTest(unittest.makeSuite(RecipeTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests_slave/slave.py 000644 000765 000120 00000011177 11454431226 020374 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import sys
import shutil
import tempfile
import unittest
from bitten.slave import BuildSlave, ExitSlave
from bitten.util import xmlio
from bitten.slave import encode_multipart_formdata
class DummyResponse(object):
def __init__(self, code):
self.code = code
class TestSlave(BuildSlave):
def __init__(self, filename, work_dir):
BuildSlave.__init__(self, [filename], work_dir=work_dir)
self.results = []
def _gather(self, method, url, body=None, headers=None):
self.results.append(xmlio.parse(body))
return DummyResponse(201)
def _execute_step(self, _build_url, recipe, step):
old_local, old_request = self.local, self.request
try:
self.local, self.request = False, self._gather
return BuildSlave._execute_step(self, 'dummy_build', recipe, step)
finally:
self.local, self.request = old_local, old_request
class BuildSlaveTestCase(unittest.TestCase):
def setUp(self):
self.work_dir = tempfile.mkdtemp(prefix='bitten_test')
self.python_path = xmlio._escape_attr(sys.executable)
def tearDown(self):
shutil.rmtree(self.work_dir)
def _create_file(self, *path):
filename = os.path.join(self.work_dir, *path)
fd = file(filename, 'w')
fd.close()
return filename
def _run_slave(self, recipe):
results = []
filename = self._create_file("recipe.xml")
recipe_file = file(filename, "wb")
recipe_file.write(recipe)
recipe_file.close()
slave = TestSlave(filename, self.work_dir)
slave.run()
return slave.results
def test_quit_raises(self):
self.slave = BuildSlave([], work_dir=self.work_dir)
self.assertRaises(ExitSlave, self.slave.quit)
def test_simple_recipe(self):
results = self._run_slave("""
""" % self.python_path)
result = results[0]
self.assertEqual(result.attr["step"], "print")
self.assertEqual(result.attr["status"], "success")
log = list(result)[0]
msg = list(log)[0]
self.assertEqual(str(msg), 'Hello')
def test_non_utf8(self):
results = self._run_slave("""
""" % self.python_path)
result = results[0]
self.assertEqual(result.attr["step"], "print")
self.assertEqual(result.attr["status"], "success")
log = list(result)[0]
msg = list(log)[0]
# check replacement character (\uFFFD) was generated correctly
self.assertEqual(str(msg).decode("utf-8"),
u'\uFFFD')
class MultiPartEncodeTestCase(unittest.TestCase):
def setUp(self):
self.work_dir = tempfile.mkdtemp(prefix='bitten_test')
def tearDown(self):
shutil.rmtree(self.work_dir)
def test_mutlipart_encode_one(self):
fields = {
'foo': 'bar',
'foofile': ('test.txt', 'contents of foofile'),
}
body, content_type = encode_multipart_formdata(fields)
boundary = content_type.split(';')[1].strip().split('=')[1]
self.assertEquals('multipart/form-data; boundary=%s' % boundary,
content_type)
self.assertEquals('--%s\r\nContent-Disposition: form-data; ' \
'name="foo"\r\n\r\nbar\r\n--%s\r\nContent-Disposition: ' \
'form-data; name="foofile"; filename="test.txt"\r\n' \
'Content-Type: application/octet-stream\r\n\r\n' \
'contents of foofile\r\n--%s--\r\n' % (
boundary,boundary,boundary), body)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(BuildSlaveTestCase, 'test'))
suite.addTest(unittest.makeSuite(MultiPartEncodeTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/__init__.py 000644 000765 000120 00000002277 11453043212 017621 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
def master_suite():
from bitten.tests import admin, master, model, queue, web_ui, notify, \
upgrades
from bitten.report import tests as report
suite = unittest.TestSuite()
suite.addTest(admin.suite())
suite.addTest(master.suite())
suite.addTest(model.suite())
suite.addTest(queue.suite())
suite.addTest(web_ui.suite())
suite.addTest(report.suite())
suite.addTest(notify.suite())
suite.addTest(upgrades.suite())
return suite
def suite():
suite = unittest.TestSuite()
try:
import trac
suite.addTest(master_suite())
except ImportError:
print "\nTrac not installed -- Skipping master tests\n"
import bitten.tests_slave
suite.addTest(bitten.tests_slave.suite())
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/admin.py 000644 000765 000120 00000100570 11453043212 017145 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import shutil
import tempfile
import unittest
from trac.core import TracError
from trac.db import DatabaseManager
from trac.perm import PermissionCache, PermissionError, PermissionSystem
from trac.test import EnvironmentStub, Mock
from trac.web.href import Href
from trac.web.main import RequestDone
from bitten.main import BuildSystem
from bitten.model import BuildConfig, TargetPlatform, schema
from bitten.admin import BuildMasterAdminPageProvider, \
BuildConfigurationsAdminPageProvider
try:
from trac.perm import DefaultPermissionPolicy
except ImportError:
DefaultPermissionPolicy = None
class BuildMasterAdminPageProviderTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub(enable=['trac.*', 'bitten.*'])
self.env.path = tempfile.mkdtemp()
# Create tables
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
# Set up permissions
self.env.config.set('trac', 'permission_store',
'DefaultPermissionStore')
PermissionSystem(self.env).grant_permission('joe', 'BUILD_ADMIN')
if DefaultPermissionPolicy is not None and hasattr(DefaultPermissionPolicy, "CACHE_EXPIRY"):
self.old_perm_cache_expiry = DefaultPermissionPolicy.CACHE_EXPIRY
DefaultPermissionPolicy.CACHE_EXPIRY = -1
# Hook up a dummy repository
self.repos = Mock(
get_node=lambda path, rev=None: Mock(get_history=lambda: [],
isdir=True),
normalize_path=lambda path: path,
sync=lambda: None
)
self.env.get_repository = lambda authname=None: self.repos
def tearDown(self):
if DefaultPermissionPolicy is not None and hasattr(DefaultPermissionPolicy, "CACHE_EXPIRY"):
DefaultPermissionPolicy.CACHE_EXPIRY = self.old_perm_cache_expiry
shutil.rmtree(self.env.path)
def test_get_admin_panels(self):
provider = BuildMasterAdminPageProvider(self.env)
req = Mock(perm=PermissionCache(self.env, 'joe'))
self.assertEqual([('bitten', 'Builds', 'master', 'Master Settings')],
list(provider.get_admin_panels(req)))
PermissionSystem(self.env).revoke_permission('joe', 'BUILD_ADMIN')
req = Mock(perm=PermissionCache(self.env, 'joe'))
self.assertEqual([], list(provider.get_admin_panels(req)))
def test_process_get_request(self):
req = Mock(method='GET', chrome={}, href=Href('/'),
perm=PermissionCache(self.env, 'joe'))
provider = BuildMasterAdminPageProvider(self.env)
template_name, data = provider.render_admin_panel(
req, 'bitten', 'master', ''
)
self.assertEqual('bitten_admin_master.html', template_name)
assert 'master' in data
master = data['master']
self.assertEqual(3600, master.slave_timeout)
self.assertEqual(0, master.stabilize_wait)
assert not master.adjust_timestamps
assert not master.build_all
self.assertEqual('log/bitten', master.logs_dir)
def test_process_config_changes(self):
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
args={'slave_timeout': '60', 'adjust_timestamps': ''})
provider = BuildMasterAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'master', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/master',
redirected_to[0])
section = self.env.config['bitten']
self.assertEqual(60, section.getint('slave_timeout'))
self.assertEqual(True, section.getbool('adjust_timestamps'))
self.assertEqual(False, section.getbool('build_all'))
class BuildConfigurationsAdminPageProviderTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub(enable=['trac.*', 'bitten.*'])
self.env.path = tempfile.mkdtemp()
# Create tables
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
# Set up permissions
self.env.config.set('trac', 'permission_store',
'DefaultPermissionStore')
PermissionSystem(self.env).grant_permission('joe', 'BUILD_CREATE')
PermissionSystem(self.env).grant_permission('joe', 'BUILD_DELETE')
PermissionSystem(self.env).grant_permission('joe', 'BUILD_MODIFY')
if DefaultPermissionPolicy is not None and hasattr(DefaultPermissionPolicy, "CACHE_EXPIRY"):
self.old_perm_cache_expiry = DefaultPermissionPolicy.CACHE_EXPIRY
DefaultPermissionPolicy.CACHE_EXPIRY = -1
# Hook up a dummy repository
self.repos = Mock(
get_node=lambda path, rev=None: Mock(get_history=lambda: [],
isdir=True),
normalize_path=lambda path: path,
sync=lambda: None
)
self.env.get_repository = lambda authname=None: self.repos
def tearDown(self):
if DefaultPermissionPolicy is not None and hasattr(DefaultPermissionPolicy, "CACHE_EXPIRY"):
DefaultPermissionPolicy.CACHE_EXPIRY = self.old_perm_cache_expiry
shutil.rmtree(self.env.path)
def test_get_admin_panels(self):
provider = BuildConfigurationsAdminPageProvider(self.env)
req = Mock(perm=PermissionCache(self.env, 'joe'))
self.assertEqual([('bitten', 'Builds', 'configs', 'Configurations')],
list(provider.get_admin_panels(req)))
PermissionSystem(self.env).revoke_permission('joe', 'BUILD_MODIFY')
req = Mock(perm=PermissionCache(self.env, 'joe'))
self.assertEqual([], list(provider.get_admin_panels(req)))
def test_process_view_configs_empty(self):
req = Mock(method='GET', chrome={}, href=Href('/'),
perm=PermissionCache(self.env, 'joe'))
provider = BuildConfigurationsAdminPageProvider(self.env)
template_name, data = provider.render_admin_panel(
req, 'bitten', 'configs', ''
)
self.assertEqual('bitten_admin_configs.html', template_name)
self.assertEqual([], data['configs'])
def test_process_view_configs(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
BuildConfig(self.env, name='bar', label='Bar', path='branches/bar',
min_rev='123', max_rev='456').insert()
req = Mock(method='GET', chrome={}, href=Href('/'),
perm=PermissionCache(self.env, 'joe'))
provider = BuildConfigurationsAdminPageProvider(self.env)
template_name, data = provider.render_admin_panel(
req, 'bitten', 'configs', ''
)
self.assertEqual('bitten_admin_configs.html', template_name)
assert 'configs' in data
configs = data['configs']
self.assertEqual(2, len(configs))
self.assertEqual({
'name': 'bar', 'href': '/admin/bitten/configs/bar',
'label': 'Bar', 'min_rev': '123', 'max_rev': '456',
'path': 'branches/bar', 'active': False, 'recipe': False
}, configs[0])
self.assertEqual({
'name': 'foo', 'href': '/admin/bitten/configs/foo',
'label': 'Foo', 'min_rev': None, 'max_rev': None,
'path': 'branches/foo', 'active': True, 'recipe': False
}, configs[1])
def test_process_view_config(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
TargetPlatform(self.env, config='foo', name='any').insert()
req = Mock(method='GET', chrome={}, href=Href('/'),
perm=PermissionCache(self.env, 'joe'))
provider = BuildConfigurationsAdminPageProvider(self.env)
template_name, data = provider.render_admin_panel(
req, 'bitten', 'configs', 'foo'
)
self.assertEqual('bitten_admin_configs.html', template_name)
assert 'config' in data
config = data['config']
self.assertEqual({
'name': 'foo', 'label': 'Foo', 'description': '', 'recipe': '',
'path': 'branches/foo', 'min_rev': None, 'max_rev': None,
'active': True, 'platforms': [{
'href': '/admin/bitten/configs/foo/1',
'name': 'any', 'id': 1, 'rules': []
}]
}, config)
def test_process_activate_config(self):
BuildConfig(self.env, name='foo', path='branches/foo').insert()
BuildConfig(self.env, name='bar', path='branches/bar').insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'apply': '', 'active': ['foo']})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs',
redirected_to[0])
config = BuildConfig.fetch(self.env, name='foo')
self.assertEqual(True, config.active)
def test_process_deactivate_config(self):
BuildConfig(self.env, name='foo', path='branches/foo',
active=True).insert()
BuildConfig(self.env, name='bar', path='branches/bar',
active=True).insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'apply': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs',
redirected_to[0])
config = BuildConfig.fetch(self.env, name='foo')
self.assertEqual(False, config.active)
config = BuildConfig.fetch(self.env, name='bar')
self.assertEqual(False, config.active)
def test_process_add_config(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'add': '', 'name': 'bar', 'label': 'Bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs/bar',
redirected_to[0])
config = BuildConfig.fetch(self.env, name='bar')
self.assertEqual('Bar', config.label)
def test_process_add_config_cancel(self):
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
args={'cancel': '', 'name': 'bar', 'label': 'Bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs',
redirected_to[0])
configs = list(BuildConfig.select(self.env, include_inactive=True))
self.assertEqual(0, len(configs))
def test_process_add_config_no_name(self):
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
chrome={'warnings': []}, href=Href('/'),
authname='joe', args={'add': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected TracError')
except TracError, e:
self.assertEqual('Missing required field "name".', e.message)
self.assertEqual('Add Configuration', e.title)
def test_process_add_config_invalid_name(self):
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
chrome={'warnings': []}, href=Href('/'), authname='joe',
args={'add': '', 'name': 'no spaces allowed'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected TracError')
except TracError, e:
self.assertEqual('The field "name" may only contain letters, '
'digits, periods, or dashes.', e.message)
self.assertEqual('Add Configuration', e.title)
def test_new_config_submit_with_invalid_path(self):
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
authname='joe',
args={'add': '', 'name': 'foo', 'path': 'invalid/path'})
def get_node(path, rev=None):
raise TracError('No such node')
self.repos.get_node = get_node
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected TracError')
except TracError, e:
self.failUnless('Invalid Repository Path' in e.message)
def test_process_add_config_no_perms(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
PermissionSystem(self.env).revoke_permission('joe', 'BUILD_CREATE')
req = Mock(method='POST',
perm=PermissionCache(self.env, 'joe'),
args={'add': '', 'name': 'bar', 'label': 'Bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
self.assertRaises(PermissionError, provider.render_admin_panel, req,
'bitten', 'configs', '')
def test_process_remove_config(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
BuildConfig(self.env, name='bar', label='Bar', path='branches/bar',
min_rev='123', max_rev='456').insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
args={'remove': '', 'sel': 'bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs',
redirected_to[0])
assert not BuildConfig.fetch(self.env, name='bar')
def test_process_remove_config_cancel(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
BuildConfig(self.env, name='bar', label='Bar', path='branches/bar',
min_rev='123', max_rev='456').insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
args={'cancel': '', 'sel': 'bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs',
redirected_to[0])
configs = list(BuildConfig.select(self.env, include_inactive=True))
self.assertEqual(2, len(configs))
def test_process_remove_config_no_selection(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
args={'remove': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected TracError')
except TracError, e:
self.assertEqual('No configuration selected', e.message)
def test_process_remove_config_bad_selection(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
args={'remove': '', 'sel': 'baz'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', '')
self.fail('Expected TracError')
except TracError, e:
self.assertEqual("Configuration 'baz' not found", e.message)
def test_process_remove_config_no_perms(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
PermissionSystem(self.env).revoke_permission('joe', 'BUILD_DELETE')
req = Mock(method='POST',
perm=PermissionCache(self.env, 'joe'),
args={'remove': '', 'sel': 'bar'})
provider = BuildConfigurationsAdminPageProvider(self.env)
self.assertRaises(PermissionError, provider.render_admin_panel, req,
'bitten', 'configs', '')
def test_process_update_config(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe', chrome={'warnings': [], 'notices': []},
href=Href('/'),
args={'save': '', 'name': 'foo', 'label': 'Foobar',
'description': 'Thanks for all the fish!'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual(['Configuration Saved.'], req.chrome['notices'])
self.assertEqual('http://example.org/admin/bitten/configs/foo',
redirected_to[0])
config = BuildConfig.fetch(self.env, name='foo')
self.assertEqual('Foobar', config.label)
self.assertEqual('Thanks for all the fish!', config.description)
def test_process_update_config_no_name(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
args={'save': '', 'name': ''}, authname='joe',
chrome={'warnings':[]}, href=Href('/'))
provider = BuildConfigurationsAdminPageProvider(self.env)
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.failUnless(req.chrome['warnings'], "No warnings?")
self.assertEquals(['Missing required field "name".'],
req.chrome['warnings'])
def test_process_update_config_invalid_name(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
args={'save': '', 'name': 'no spaces allowed'},
authname='joe', chrome={'warnings': []},
href=Href('/'))
provider = BuildConfigurationsAdminPageProvider(self.env)
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.failUnless(req.chrome['warnings'], "No warnings?")
self.assertEquals(req.chrome['warnings'], ['The field "name" may ' \
'only contain letters, digits, periods, or dashes.'])
def test_process_update_config_invalid_path(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
authname='joe', chrome={'warnings': []}, href=Href('/'),
args={'save': '', 'name': 'foo', 'path': 'invalid/path'})
def get_node(path, rev=None):
raise TracError('No such node')
self.repos.get_node = get_node
provider = BuildConfigurationsAdminPageProvider(self.env)
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.failUnless(req.chrome['warnings'], "No warnings?")
self.assertEquals(req.chrome['warnings'],
['Invalid Repository Path: "invalid/path" does not exist '
'within the "(default)" repository.'])
def test_process_update_config_non_wellformed_recipe(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
authname='joe', chrome={'warnings': []}, href=Href('/'),
args={'save': '', 'name': 'foo', 'recipe': 'not_xml'})
provider = BuildConfigurationsAdminPageProvider(self.env)
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.failUnless(req.chrome['warnings'], "No warnings?")
self.assertEquals(['Failure parsing recipe: syntax error: line 1, ' \
'column 0.'], req.chrome['warnings'])
def test_process_update_config_invalid_recipe(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
authname='joe', chrome={'warnings': []}, href=Href('/'),
args={'save': '', 'name': 'foo',
'recipe': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.failUnless(req.chrome['warnings'], "No warnings?")
self.assertEquals(req.chrome['warnings'],
['Invalid Recipe: Steps must have an "id" attribute.'])
def test_process_new_platform_no_name(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
data = {}
req = Mock(method='POST', chrome={}, hdf=data, href=Href('/'),
perm=PermissionCache(self.env, 'joe'),
args={'new': '', 'platform_name': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.fail("No TracError?")
except Exception, e:
self.assertEquals(e.message, 'Missing required field "name"')
self.assertEquals(e.title, 'Missing field')
def test_process_new_platform(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'add': '', 'new': '', 'platform_name': 'Test'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs/foo/1',
redirected_to[0])
platforms = list(TargetPlatform.select(self.env, config='foo'))
self.assertEqual(1, len(platforms))
self.assertEqual('Test', platforms[0].name)
self.assertEqual([], platforms[0].rules)
def test_process_remove_platforms(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
platform = TargetPlatform(self.env, config='foo', name='any')
platform.insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe', chrome={'warnings': [], 'notices': []},
href=Href('/'),
args={'remove': '', 'sel': str(platform.id)})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.fail('Expected RequestDone')
except RequestDone:
self.assertEquals(['Target Platform(s) Removed.'],
req.chrome['notices'])
self.assertEqual('http://example.org/admin/bitten/configs/foo',
redirected_to[0])
platforms = list(TargetPlatform.select(self.env, config='foo'))
self.assertEqual(0, len(platforms))
def test_process_remove_platforms_no_selection(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
platform = TargetPlatform(self.env, config='foo', name='any')
platform.insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'remove': ''})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs', 'foo')
self.fail('Expected TracError')
except TracError, e:
self.assertEqual('No platform selected', e.message)
def test_process_edit_platform(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
platform = TargetPlatform(self.env, config='foo', name='any')
platform.insert()
req = Mock(method='GET', chrome={}, href=Href('/'),
perm=PermissionCache(self.env, 'joe'), args={})
provider = BuildConfigurationsAdminPageProvider(self.env)
template_name, data = provider.render_admin_panel(
req, 'bitten', 'configs', 'foo/%d' % platform.id
)
self.assertEqual('bitten_admin_configs.html', template_name)
assert 'platform' in data
platform = data['platform']
self.assertEqual({
'id': 1, 'exists': True, 'name': 'any', 'rules': [('', '')],
}, platform)
def test_process_update_platform(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
platform = TargetPlatform(self.env, config='foo', name='any')
platform.insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'save': '', 'edit': '', 'name': 'Test',
'property_0': 'family', 'pattern_0': 'posix'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs',
'foo/%d' % platform.id)
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs/foo',
redirected_to[0])
platforms = list(TargetPlatform.select(self.env, config='foo'))
self.assertEqual(1, len(platforms))
self.assertEqual('Test', platforms[0].name)
self.assertEqual([('family', 'posix')], platforms[0].rules)
def test_process_update_platform_cancel(self):
BuildConfig(self.env, name='foo', label='Foo', path='branches/foo',
active=True).insert()
platform = TargetPlatform(self.env, config='foo', name='any')
platform.insert()
redirected_to = []
def redirect(url):
redirected_to.append(url)
raise RequestDone
req = Mock(method='POST', perm=PermissionCache(self.env, 'joe'),
abs_href=Href('http://example.org/'), redirect=redirect,
authname='joe',
args={'cancel': '', 'edit': '', 'name': 'Changed',
'property_0': 'family', 'pattern_0': 'posix'})
provider = BuildConfigurationsAdminPageProvider(self.env)
try:
provider.render_admin_panel(req, 'bitten', 'configs',
'foo/%d' % platform.id)
self.fail('Expected RequestDone')
except RequestDone:
self.assertEqual('http://example.org/admin/bitten/configs/foo',
redirected_to[0])
platforms = list(TargetPlatform.select(self.env, config='foo'))
self.assertEqual(1, len(platforms))
self.assertEqual('any', platforms[0].name)
self.assertEqual([], platforms[0].rules)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(
BuildMasterAdminPageProviderTestCase, 'test'
))
suite.addTest(unittest.makeSuite(
BuildConfigurationsAdminPageProviderTestCase, 'test'
))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/master.py 000644 000765 000120 00000123772 11454420025 017363 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import re
import shutil
from StringIO import StringIO
import tempfile
import unittest
import cgi
from Cookie import SimpleCookie as Cookie
from trac.attachment import Attachment
from trac.db import DatabaseManager
from trac.perm import PermissionCache, PermissionSystem
from trac.test import EnvironmentStub, Mock
from trac.util.datefmt import to_datetime, utc
from trac.web.api import RequestDone
from trac.web.href import Href
from bitten.master import BuildMaster
from bitten.slave import encode_multipart_formdata
from bitten.model import BuildConfig, TargetPlatform, Build, BuildStep, \
BuildLog, Report, schema
from bitten import PROTOCOL_VERSION
class BuildMasterTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub(enable=['trac.*', 'bitten.*'])
self.env.path = tempfile.mkdtemp()
logs_dir = self.env.config.get("bitten", "logs_dir")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
os.makedirs(logs_dir)
PermissionSystem(self.env).grant_permission('hal', 'BUILD_EXEC')
# Create tables
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
self.repos = Mock(get_changeset=lambda rev: Mock(author = 'author'))
self.env.get_repository = lambda authname=None: self.repos
def tearDown(self):
shutil.rmtree(self.env.path)
def test_create_build(self):
BuildConfig(self.env, 'test', path='somepath', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('family', 'posix'))
platform.insert()
self.repos = Mock(
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
),
get_changeset=lambda rev: Mock(date=to_datetime(42, utc)),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
inheaders = {'Content-Type': 'application/x-bitten+xml'}
inbody = StringIO("""Power MacintoshDarwin""" % PROTOCOL_VERSION)
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='', path_info='/builds',
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
get_header=lambda x: inheaders.get(x), read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('text/plain', outheaders['Content-Type'])
location = outheaders['Location']
mo = re.match('http://example.org/trac/builds/(\d+)', location)
assert mo, 'Location was %r' % location
self.assertEqual('Build pending', outbody.getvalue())
build = Build.fetch(self.env, int(mo.group(1)))
self.assertEqual(Build.IN_PROGRESS, build.status)
self.assertEqual('hal', build.slave)
def test_create_build_invalid_xml(self):
inheaders = {'Content-Type': 'application/x-bitten+xml'}
inbody = StringIO('')
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='', path_info='/builds',
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
get_header=lambda x: inheaders.get(x), read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEquals(400, outheaders['Status'])
self.assertEqual('XML parser error', outbody.getvalue())
def test_create_build_no_post(self):
outheaders = {}
outbody = StringIO()
req = Mock(method='GET', base_path='', path_info='/builds',
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEquals(200, outheaders['Status'])
self.assertEquals('Only POST allowed for build creation.',
outbody.getvalue())
def test_create_build_no_match(self):
inheaders = {'Content-Type': 'application/x-bitten+xml'}
inbody = StringIO("""Power MacintoshDarwin""" % PROTOCOL_VERSION)
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='', path_info='/builds',
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
get_header=lambda x: inheaders.get(x), read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(204, outheaders['Status'])
self.assertEqual('', outbody.getvalue())
def test_create_build_protocol_wrong_version(self):
inheaders = {'Content-Type': 'application/x-bitten+xml'}
inbody = StringIO("""Power MacintoshDarwin""" % (PROTOCOL_VERSION-2,))
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='', path_info='/builds',
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
get_header=lambda x: inheaders.get(x), read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(400, outheaders['Status'])
self.assertEqual('Master-Slave version mismatch: master=%d, slave=%d' \
% (PROTOCOL_VERSION, PROTOCOL_VERSION-2),
outbody.getvalue())
def test_create_build_protocol_no_version(self):
inheaders = {'Content-Type': 'application/x-bitten+xml'}
inbody = StringIO("""Power MacintoshDarwin""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='', path_info='/builds',
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
get_header=lambda x: inheaders.get(x), read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(400, outheaders['Status'])
self.assertEqual('Master-Slave version mismatch: master=%d, slave=1' \
% (PROTOCOL_VERSION,),
outbody.getvalue())
def test_cancel_build(self):
config = BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='')
config.insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
status=Build.IN_PROGRESS, started=42)
build.insert()
outheaders = {}
outbody = StringIO()
req = Mock(method='DELETE', base_path='',
path_info='/builds/%d' % build.id,
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(204, outheaders['Status'])
self.assertEqual('', outbody.getvalue())
# Make sure the started timestamp has been set
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.PENDING, build.status)
assert not build.started
def test_initiate_build(self):
config = BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='')
config.insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('family', 'posix'))
platform.insert()
build = Build(self.env, 'test', '123', platform.id, slave='hal',
rev_time=42)
build.insert()
outheaders = {}
outbody = StringIO()
req = Mock(method='GET', base_path='',
path_info='/builds/%d' % build.id,
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
form_token="12345",
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(200, outheaders['Status'])
self.assertEqual('131', outheaders['Content-Length'])
self.assertEqual('application/x-bitten+xml',
outheaders['Content-Type'])
self.assertEqual('attachment; filename=recipe_test_r123.xml',
outheaders['Content-Disposition'])
self.assertEqual('',
outbody.getvalue())
# Make sure the started timestamp has been set
build = Build.fetch(self.env, build.id)
assert build.started
def test_initiate_build_no_such_build(self):
outheaders = {}
outbody = StringIO()
req = Mock(method='GET', base_path='',
path_info='/builds/123', href=Href('/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEquals(404, outheaders['Status'])
self.assertEquals('No such build (123)', outbody.getvalue())
def test_process_unknown_collection(self):
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='').insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42)
build.insert()
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/files/' % build.id,
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(404, outheaders['Status'])
self.assertEqual("No such collection 'files'", outbody.getvalue())
def test_process_build_step_success(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.SUCCESS, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.SUCCESS, steps[0].status)
def test_process_build_step_success_with_log(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""Doing stuffOuch that hurt""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.SUCCESS, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.SUCCESS, steps[0].status)
logs = list(BuildLog.select(self.env, build=build.id, step='foo'))
self.assertEqual(1, len(logs))
self.assertEqual('http://bitten.edgewall.org/tools/python#unittest',
logs[0].generator)
self.assertEqual(2, len(logs[0].messages))
self.assertEqual((u'info', u'Doing stuff'), logs[0].messages[0])
self.assertEqual((u'error', u'Ouch that hurt'), logs[0].messages[1])
def test_process_build_step_success_with_report(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""Doing my thing""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.SUCCESS, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.SUCCESS, steps[0].status)
reports = list(Report.select(self.env, build=build.id, step='foo'))
self.assertEqual(1, len(reports))
self.assertEqual('test', reports[0].category)
self.assertEqual('http://bitten.edgewall.org/tools/python#unittest',
reports[0].generator)
self.assertEqual(1, len(reports[0].items))
self.assertEqual({
'fixture': 'my.Fixture',
'file': 'my/test/file.py',
'stdout': 'Doing my thing',
'type': 'test',
}, reports[0].items[0])
def test_process_build_step_wrong_slave(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""Doing stuffOuch that hurt""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(409, outheaders['Status'])
self.assertEqual('Token mismatch (wrong slave): slave=, build=123',
outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.IN_PROGRESS, build.status)
assert not build.stopped
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
def test_process_build_step_invalidated_build(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""Doing stuffOuch that hurt""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.IN_PROGRESS, build.status)
assert not build.stopped
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(2, len(steps))
# invalidate the build.
build = Build.fetch(self.env, build.id)
build.slave = None
build.status = Build.PENDING
build.slave_info = {}
for step in list(BuildStep.select(self.env, build=build.id)):
step.delete()
build.update()
# have this slave submit more data.
inbody = StringIO("""This is a step after invalidation""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEquals(409, outheaders['Status'])
self.assertEquals('Token mismatch (wrong slave): slave=123, build=',
outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.PENDING, build.status)
def test_process_build_step_failure(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.FAILURE, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.FAILURE, steps[0].status)
def test_process_build_step_failure_ignored(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.SUCCESS, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.FAILURE, steps[0].status)
def test_process_build_step_failure_continue(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.slave_info[Build.TOKEN] = '123';
build.insert()
inbody = StringIO("""""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), abs_href=Href('http://example.org/trac'),
remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth=123'))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('20', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Build step processed', outbody.getvalue())
build = Build.fetch(self.env, build.id)
self.assertEqual(Build.FAILURE, build.status)
assert build.stopped
assert build.stopped > build.started
steps = list(BuildStep.select(self.env, build.id))
self.assertEqual(1, len(steps))
self.assertEqual('foo', steps[0].name)
self.assertEqual(BuildStep.FAILURE, steps[0].status)
def test_process_build_step_invalid_xml(self):
recipe = """"""
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe=recipe).insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42)
build.insert()
inbody = StringIO("""""")
outheaders = {}
outbody = StringIO()
req = Mock(method='POST', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
read=inbody.read,
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEquals(400, outheaders['Status'])
self.assertEquals('XML parser error', outbody.getvalue())
def test_process_build_step_no_post(self):
BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='').insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42)
build.insert()
outheaders = {}
outbody = StringIO()
req = Mock(method='GET', base_path='',
path_info='/builds/%d/steps/' % build.id,
href=Href('/trac'), remote_addr='127.0.0.1', args={},
perm=PermissionCache(self.env, 'hal'),
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write,
incookie=Cookie('trac_auth='))
module = BuildMaster(self.env)
module._start_new_step(build, 'foo').insert()
assert module.match_request(req)
self.assertRaises(RequestDone, module.process_request, req)
self.assertEqual(405, outheaders['Status'])
self.assertEqual('Method GET not allowed', outbody.getvalue())
def test_process_attach_collection_default_member(self):
req = Mock(args={}, path_info='/builds/12/attach/config')
module = BuildMaster(self.env)
self.assertEquals(True, module.match_request(req))
self.assertTrue(req.args['collection'], 'attach')
self.assertTrue(req.args['member'], '')
def test_process_attach_collection_config(self):
req = Mock(args={}, path_info='/builds/12/attach/config')
module = BuildMaster(self.env)
self.assertEquals(True, module.match_request(req))
self.assertTrue(req.args['collection'], 'attach')
self.assertTrue(req.args['member'], 'config')
def test_process_attach_collection_config(self):
req = Mock(args={}, path_info='/builds/12/attach/build')
module = BuildMaster(self.env)
self.assertEquals(True, module.match_request(req))
self.assertTrue(req.args['collection'], 'attach')
self.assertTrue(req.args['member'], 'build')
def test_process_attach_config(self):
body, content_type = encode_multipart_formdata({
'description': 'baz baz',
'file': ('baz.txt', 'hello baz'),
'__FORM_TOKEN': '123456'})
args = {}
for k, v in dict(cgi.FieldStorage(fp=StringIO(body), environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': content_type})
).items():
if v.filename:
args[k] = v
else:
args[k] = v.value
args.update({'collection': 'attach', 'member': 'config'})
self.assertTrue('file' in args)
outheaders = {}
outbody = StringIO()
req = Mock(args=args, form_token='123456', authname='hal',
remote_addr='127.0.0.1',
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write)
config = BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='')
config.insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.insert()
module = BuildMaster(self.env)
self.assertRaises(RequestDone, module._process_attachment,
req, config, build)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('18', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Attachment created', outbody.getvalue())
config_atts = list(Attachment.select(self.env, 'build', 'test'))
self.assertEquals(1, len(config_atts))
self.assertEquals('hal', config_atts[0].author)
self.assertEquals('baz baz', config_atts[0].description)
self.assertEquals('baz.txt', config_atts[0].filename)
self.assertEquals('hello baz',
config_atts[0].open().read())
def test_process_attach_build(self):
body, content_type = encode_multipart_formdata({
'description': 'baz baz',
'file': ('baz.txt', 'hello baz'),
'__FORM_TOKEN': '123456'})
args = {}
for k, v in dict(cgi.FieldStorage(fp=StringIO(body), environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': content_type})
).items():
if v.filename:
args[k] = v
else:
args[k] = v.value
args.update({'collection': 'attach', 'member': 'build'})
self.assertTrue('file' in args)
outheaders = {}
outbody = StringIO()
req = Mock(args=args, form_token='123456', authname='hal',
remote_addr='127.0.0.1',
send_response=lambda x: outheaders.setdefault('Status', x),
send_header=lambda x, y: outheaders.setdefault(x, y),
write=outbody.write)
config = BuildConfig(self.env, 'test', path='somepath', active=True,
recipe='')
config.insert()
build = Build(self.env, 'test', '123', 1, slave='hal', rev_time=42,
started=42, status=Build.IN_PROGRESS)
build.insert()
module = BuildMaster(self.env)
self.assertRaises(RequestDone, module._process_attachment,
req, config, build)
self.assertEqual(201, outheaders['Status'])
self.assertEqual('18', outheaders['Content-Length'])
self.assertEqual('text/plain', outheaders['Content-Type'])
self.assertEqual('Attachment created', outbody.getvalue())
build_atts = list(Attachment.select(self.env, 'build', 'test/1'))
self.assertEquals(1, len(build_atts))
self.assertEquals('hal', build_atts[0].author)
self.assertEquals('baz baz', build_atts[0].description)
self.assertEquals('baz.txt', build_atts[0].filename)
self.assertEquals('hello baz',
build_atts[0].open().read())
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(BuildMasterTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/model.py 000644 000765 000120 00000077273 11453043212 017172 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub
from bitten.model import BuildConfig, TargetPlatform, Build, BuildStep, \
BuildLog, Report, schema
import os
import shutil
import tempfile
class BaseModelTestCase(unittest.TestCase):
""" Inheritable base for model test case classes. """
# Override with iterable containing required schemas
schemas = None
def setUp(self):
self.env = EnvironmentStub()
self.env.path = tempfile.mkdtemp()
logs_dir = self.env.config.get("bitten", "logs_dir")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
os.makedirs(logs_dir)
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for schema in self.schemas:
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
db.commit()
def tearDown(self):
shutil.rmtree(self.env.path)
class BuildConfigTestCase(BaseModelTestCase):
schemas = [schema]
def test_new(self):
config = BuildConfig(self.env, name='test')
assert not config.exists
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
config = BuildConfig.fetch(self.env, name='test')
assert config.exists
self.assertEqual('test', config.name)
self.assertEqual('trunk', config.path)
self.assertEqual('Test', config.label)
self.assertEqual(False, config.active)
def test_fetch_none(self):
config = BuildConfig.fetch(self.env, name='test')
self.assertEqual(None, config)
def test_select_none(self):
configs = BuildConfig.select(self.env)
self.assertRaises(StopIteration, configs.next)
def test_insert(self):
config = BuildConfig(self.env, name='test', path='trunk', label='Test')
config.insert()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT name,path,label,active,description "
"FROM bitten_config")
self.assertEqual(('test', 'trunk', 'Test', 0, ''), cursor.fetchone())
def test_insert_no_name(self):
config = BuildConfig(self.env)
self.assertRaises(AssertionError, config.insert)
def test_update(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
config = BuildConfig.fetch(self.env, 'test')
config.path = 'some_branch'
config.label = 'Updated'
config.active = True
config.description = 'Bla bla bla'
config.update()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT name,path,label,active,description "
"FROM bitten_config")
self.assertEqual(('test', 'some_branch', 'Updated', 1, 'Bla bla bla'),
cursor.fetchone())
self.assertEqual(None, cursor.fetchone())
def test_update_name(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
config = BuildConfig.fetch(self.env, 'test')
config.name = 'foobar'
config.update()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT name,path,label,active,description "
"FROM bitten_config")
self.assertEqual(('foobar', 'trunk', 'Test', 0, ''), cursor.fetchone())
self.assertEqual(None, cursor.fetchone())
def test_update_no_name(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
config = BuildConfig.fetch(self.env, 'test')
config.name = None
self.assertRaises(AssertionError, config.update)
def test_update_name_with_platform(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
cursor.execute("INSERT INTO bitten_platform (config,name) "
"VALUES (%s,%s)", ('test', 'NetBSD'))
config = BuildConfig.fetch(self.env, 'test')
config.name = 'foobar'
config.update()
cursor.execute("SELECT config,name FROM bitten_platform")
self.assertEqual(('foobar', 'NetBSD'), cursor.fetchone())
self.assertEqual(None, cursor.fetchone())
def test_delete(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_config (name,path,label,active) "
"VALUES (%s,%s,%s,%s)", ('test', 'trunk', 'Test', 0))
config = BuildConfig.fetch(self.env, 'test')
config.delete()
self.assertEqual(False, config.exists)
cursor.execute("SELECT * FROM bitten_config WHERE name=%s", ('test',))
self.assertEqual(None, cursor.fetchone())
def test_delete_non_existing(self):
config = BuildConfig(self.env, 'test')
self.assertRaises(AssertionError, config.delete)
class TargetPlatformTestCase(BaseModelTestCase):
schemas = [TargetPlatform._schema]
def test_new(self):
platform = TargetPlatform(self.env)
self.assertEqual(False, platform.exists)
self.assertEqual([], platform.rules)
def test_insert(self):
platform = TargetPlatform(self.env, config='test', name='Windows XP')
platform.rules += [(Build.OS_NAME, 'Windows'), (Build.OS_VERSION, 'XP')]
platform.insert()
assert platform.exists
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT config,name FROM bitten_platform "
"WHERE id=%s", (platform.id,))
self.assertEqual(('test', 'Windows XP'), cursor.fetchone())
cursor.execute("SELECT propname,pattern,orderno FROM bitten_rule "
"WHERE id=%s", (platform.id,))
self.assertEqual((Build.OS_NAME, 'Windows', 0), cursor.fetchone())
self.assertEqual((Build.OS_VERSION, 'XP', 1), cursor.fetchone())
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_platform (config,name) "
"VALUES (%s,%s)", ('test', 'Windows'))
id = db.get_last_id(cursor, 'bitten_platform')
platform = TargetPlatform.fetch(self.env, id)
assert platform.exists
self.assertEqual('test', platform.config)
self.assertEqual('Windows', platform.name)
def test_select(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.executemany("INSERT INTO bitten_platform (config,name) "
"VALUES (%s,%s)", [('test', 'Windows'),
('test', 'Mac OS X')])
platforms = list(TargetPlatform.select(self.env, config='test'))
self.assertEqual(2, len(platforms))
class BuildTestCase(BaseModelTestCase):
schemas = [Build._schema]
def test_new(self):
build = Build(self.env)
self.assertEqual(None, build.id)
self.assertEqual(Build.PENDING, build.status)
self.assertEqual(0, build.stopped)
self.assertEqual(0, build.started)
def test_insert(self):
build = Build(self.env, config='test', rev='42', rev_time=12039,
platform=1)
build.slave_info.update({Build.IP_ADDRESS: '127.0.0.1',
Build.MAINTAINER: 'joe@example.org'})
build.insert()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT config,rev,platform,slave,started,stopped,status"
" FROM bitten_build WHERE id=%s" % build.id)
self.assertEqual(('test', '42', 1, '', 0, 0, 'P'), cursor.fetchone())
cursor.execute("SELECT propname,propvalue FROM bitten_slave")
expected = {Build.IP_ADDRESS: '127.0.0.1',
Build.MAINTAINER: 'joe@example.org'}
for propname, propvalue in cursor:
self.assertEqual(expected[propname], propvalue)
def test_insert_no_config_or_rev_or_rev_time_or_platform(self):
build = Build(self.env)
self.assertRaises(AssertionError, build.insert)
build = Build(self.env, rev='42', rev_time=12039, platform=1)
self.assertRaises(AssertionError, build.insert) # No config
build = Build(self.env, config='test', rev_time=12039, platform=1)
self.assertRaises(AssertionError, build.insert) # No rev
build = Build(self.env, config='test', rev='42', platform=1)
self.assertRaises(AssertionError, build.insert) # No rev time
build = Build(self.env, config='test', rev='42', rev_time=12039)
self.assertRaises(AssertionError, build.insert) # No platform
def test_insert_no_slave(self):
build = Build(self.env, config='test', rev='42', rev_time=12039,
platform=1)
build.status = Build.SUCCESS
self.assertRaises(AssertionError, build.insert)
build.status = Build.FAILURE
self.assertRaises(AssertionError, build.insert)
build.status = Build.IN_PROGRESS
self.assertRaises(AssertionError, build.insert)
build.status = Build.PENDING
build.insert()
def test_insert_invalid_status(self):
build = Build(self.env, config='test', rev='42', rev_time=12039,
status='DUNNO')
self.assertRaises(AssertionError, build.insert)
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_build (config,rev,rev_time,platform,"
"slave,started,stopped,status) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s)",
('test', '42', 12039, 1, 'tehbox', 15006, 16007,
Build.SUCCESS))
build_id = db.get_last_id(cursor, 'bitten_build')
cursor.executemany("INSERT INTO bitten_slave VALUES (%s,%s,%s)",
[(build_id, Build.IP_ADDRESS, '127.0.0.1'),
(build_id, Build.MAINTAINER, 'joe@example.org')])
build = Build.fetch(self.env, build_id)
self.assertEquals(build_id, build.id)
self.assertEquals('127.0.0.1', build.slave_info[Build.IP_ADDRESS])
self.assertEquals('joe@example.org', build.slave_info[Build.MAINTAINER])
def test_update(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_build (config,rev,rev_time,platform,"
"slave,started,stopped,status) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s)",
('test', '42', 12039, 1, 'tehbox', 15006, 16007,
Build.SUCCESS))
build_id = db.get_last_id(cursor, 'bitten_build')
cursor.executemany("INSERT INTO bitten_slave VALUES (%s,%s,%s)",
[(build_id, Build.IP_ADDRESS, '127.0.0.1'),
(build_id, Build.MAINTAINER, 'joe@example.org')])
build = Build.fetch(self.env, build_id)
build.status = Build.FAILURE
build.update()
class BuildStepTestCase(BaseModelTestCase):
schemas = [BuildStep._schema]
def test_new(self):
step = BuildStep(self.env)
self.assertEqual(False, step.exists)
self.assertEqual(None, step.build)
self.assertEqual(None, step.name)
def test_insert(self):
step = BuildStep(self.env, build=1, name='test', description='Foo bar',
status=BuildStep.SUCCESS)
step.insert()
self.assertEqual(True, step.exists)
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,name,description,status,started,stopped "
"FROM bitten_step")
self.assertEqual((1, 'test', 'Foo bar', BuildStep.SUCCESS, 0, 0),
cursor.fetchone())
def test_insert_with_errors(self):
step = BuildStep(self.env, build=1, name='test', description='Foo bar',
status=BuildStep.SUCCESS)
step.errors += ['Foo', 'Bar']
step.insert()
self.assertEqual(True, step.exists)
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,name,description,status,started,stopped "
"FROM bitten_step")
self.assertEqual((1, 'test', 'Foo bar', BuildStep.SUCCESS, 0, 0),
cursor.fetchone())
cursor.execute("SELECT message FROM bitten_error ORDER BY orderno")
self.assertEqual(('Foo',), cursor.fetchone())
self.assertEqual(('Bar',), cursor.fetchone())
def test_insert_no_build_or_name(self):
step = BuildStep(self.env, name='test')
self.assertRaises(AssertionError, step.insert) # No build
step = BuildStep(self.env, build=1)
self.assertRaises(AssertionError, step.insert) # No name
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_step VALUES (%s,%s,%s,%s,%s,%s)",
(1, 'test', 'Foo bar', BuildStep.SUCCESS, 0, 0))
step = BuildStep.fetch(self.env, build=1, name='test')
self.assertEqual(1, step.build)
self.assertEqual('test', step.name)
self.assertEqual('Foo bar', step.description)
self.assertEqual(BuildStep.SUCCESS, step.status)
def test_fetch_with_errors(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_step VALUES (%s,%s,%s,%s,%s,%s)",
(1, 'test', 'Foo bar', BuildStep.SUCCESS, 0, 0))
cursor.executemany("INSERT INTO bitten_error VALUES (%s,%s,%s,%s)",
[(1, 'test', 'Foo', 0), (1, 'test', 'Bar', 1)])
step = BuildStep.fetch(self.env, build=1, name='test')
self.assertEqual(1, step.build)
self.assertEqual('test', step.name)
self.assertEqual('Foo bar', step.description)
self.assertEqual(BuildStep.SUCCESS, step.status)
self.assertEqual(['Foo', 'Bar'], step.errors)
def test_select(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.executemany("INSERT INTO bitten_step VALUES (%s,%s,%s,%s,%s,%s)",
[(1, 'test', 'Foo bar', BuildStep.SUCCESS, 1, 2),
(1, 'dist', 'Foo baz', BuildStep.FAILURE, 2, 3)])
steps = list(BuildStep.select(self.env, build=1))
self.assertEqual(1, steps[0].build)
self.assertEqual('test', steps[0].name)
self.assertEqual('Foo bar', steps[0].description)
self.assertEqual(BuildStep.SUCCESS, steps[0].status)
self.assertEqual(1, steps[1].build)
self.assertEqual('dist', steps[1].name)
self.assertEqual('Foo baz', steps[1].description)
self.assertEqual(BuildStep.FAILURE, steps[1].status)
class BuildLogTestCase(BaseModelTestCase):
schemas = [BuildLog._schema]
def test_new(self):
log = BuildLog(self.env)
self.assertEqual(False, log.exists)
self.assertEqual(None, log.id)
self.assertEqual(None, log.build)
self.assertEqual(None, log.step)
self.assertEqual('', log.generator)
self.assertEqual([], log.messages)
def test_insert(self):
log = BuildLog(self.env, build=1, step='test', generator='distutils', filename='1.log')
full_file = log.get_log_file('1.log')
if os.path.exists(full_file):
os.remove(full_file)
log.messages = [
(BuildLog.INFO, 'running tests'),
(BuildLog.ERROR, 'tests failed')
]
log.insert()
self.assertNotEqual(None, log.id)
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,generator,filename FROM bitten_log "
"WHERE id=%s", (log.id,))
self.assertEqual((1, 'test', 'distutils', '1.log'), cursor.fetchone())
lines = open(full_file, "rb").readlines()
self.assertEqual('running tests\n', lines[0])
self.assertEqual('tests failed\n', lines[1])
if os.path.exists(full_file):
os.remove(full_file)
def test_insert_empty(self):
log = BuildLog(self.env, build=1, step='test', generator='distutils', filename="1.log")
full_file = log.get_log_file('1.log')
if os.path.exists(full_file):
os.remove(full_file)
log.messages = []
log.insert()
self.assertNotEqual(None, log.id)
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,generator,filename FROM bitten_log "
"WHERE id=%s", (log.id,))
self.assertEqual((1, 'test', 'distutils', '1.log'), cursor.fetchone())
file_exists = os.path.exists(full_file)
if file_exists:
os.remove(full_file)
assert not file_exists
def test_insert_no_build_or_step(self):
log = BuildLog(self.env, step='test')
self.assertRaises(AssertionError, log.insert) # No build
step = BuildStep(self.env, build=1)
self.assertRaises(AssertionError, log.insert) # No step
def test_delete(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_log (build,step,generator,filename) "
"VALUES (%s,%s,%s,%s)", (1, 'test', 'distutils', '1.log'))
id = db.get_last_id(cursor, 'bitten_log')
logs_dir = self.env.config.get("bitten", "logsdir", "log/bitten")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
full_file = os.path.join(logs_dir, "1.log")
open(full_file, "wb").writelines(["running tests\n", "tests failed\n"])
log = BuildLog.fetch(self.env, id=id, db=db)
self.assertEqual(True, log.exists)
log.delete()
self.assertEqual(False, log.exists)
cursor.execute("SELECT * FROM bitten_log WHERE id=%s", (id,))
self.assertEqual(True, not cursor.fetchall())
file_exists = os.path.exists(full_file)
if os.path.exists(full_file):
os.remove(full_file)
assert not file_exists
def test_delete_new(self):
log = BuildLog(self.env, build=1, step='test', generator='foo')
self.assertRaises(AssertionError, log.delete)
def test_insert_and_delete_files(self):
# create - files should be created automatically
build_log = BuildLog(self.env, build=1, step='test', generator='make')
build_log.messages = [(BuildLog.INFO, 'running')]
build_log.insert()
# fetch it fresh - check object and files
build_log = BuildLog.fetch(self.env, id=build_log.id)
self.assertEquals(build_log.filename, "%s.log" % build_log.id)
log_file = build_log.get_log_file(build_log.filename)
levels_file = log_file + BuildLog.LEVELS_SUFFIX
self.failUnless(os.path.exists(log_file), 'log_file does not exist')
self.failUnless(os.path.exists(levels_file),
'levels_file does not exist')
self.assertEquals(build_log.messages, [(BuildLog.INFO, 'running')])
# delete - object and file should be gone
build_log.delete()
self.assertEquals(None, BuildLog.fetch(self.env, id=build_log.id))
self.failIf(os.path.exists(log_file), 'log_file exists after delete()')
self.failIf(os.path.exists(levels_file),
'levels_file exists after delete()')
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_log (build,step,generator,filename) "
"VALUES (%s,%s,%s,%s)", (1, 'test', 'distutils', '1.log'))
id = db.get_last_id(cursor, 'bitten_log')
logs_dir = self.env.config.get("bitten", "logsdir", "log/bitten")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
full_file = os.path.join(logs_dir, "1.log")
open(full_file, "wb").writelines(["running tests\n", "tests failed\n", u"test unicode\xbb\n".encode("UTF-8")])
log = BuildLog.fetch(self.env, id=id, db=db)
self.assertEqual(True, log.exists)
self.assertEqual(id, log.id)
self.assertEqual(1, log.build)
self.assertEqual('test', log.step)
self.assertEqual('distutils', log.generator)
self.assertEqual((BuildLog.UNKNOWN, 'running tests'), log.messages[0])
self.assertEqual((BuildLog.UNKNOWN, 'tests failed'), log.messages[1])
self.assertEqual((BuildLog.UNKNOWN, u'test unicode\xbb'), log.messages[2])
if os.path.exists(full_file):
os.remove(full_file)
def test_select(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_log (build,step,generator,filename) "
"VALUES (%s,%s,%s,%s)", (1, 'test', 'distutils', '1.log'))
id = db.get_last_id(cursor, 'bitten_log')
logs_dir = self.env.config.get("bitten", "logsdir", "log/bitten")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
full_file = os.path.join(logs_dir, "1.log")
open(full_file, "wb").writelines(["running tests\n", "tests failed\n", u"test unicode\xbb\n".encode("UTF-8")])
logs = BuildLog.select(self.env, build=1, step='test', db=db)
log = logs.next()
self.assertEqual(True, log.exists)
self.assertEqual(id, log.id)
self.assertEqual(1, log.build)
self.assertEqual('test', log.step)
self.assertEqual('distutils', log.generator)
self.assertEqual((BuildLog.UNKNOWN, 'running tests'), log.messages[0])
self.assertEqual((BuildLog.UNKNOWN, 'tests failed'), log.messages[1])
self.assertEqual((BuildLog.UNKNOWN, u'test unicode\xbb'), log.messages[2])
self.assertRaises(StopIteration, logs.next)
if os.path.exists(full_file):
os.remove(full_file)
class ReportTestCase(BaseModelTestCase):
schemas = [Report._schema]
def test_delete(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(1, 'test', 'test', 'unittest'))
report_id = db.get_last_id(cursor, 'bitten_report')
cursor.executemany("INSERT INTO bitten_report_item "
"(report,item,name,value) VALUES (%s,%s,%s,%s)",
[(report_id, 0, 'file', 'tests/foo.c'),
(report_id, 0, 'result', 'failure'),
(report_id, 1, 'file', 'tests/bar.c'),
(report_id, 1, 'result', 'success')])
report = Report.fetch(self.env, report_id, db=db)
report.delete(db=db)
self.assertEqual(False, report.exists)
report = Report.fetch(self.env, report_id, db=db)
self.assertEqual(None, report)
def test_insert(self):
report = Report(self.env, build=1, step='test', category='test',
generator='unittest')
report.items = [
{'file': 'tests/foo.c', 'status': 'failure'},
{'file': 'tests/bar.c', 'status': 'success'}
]
report.insert()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,category,generator "
"FROM bitten_report WHERE id=%s", (report.id,))
self.assertEqual((1, 'test', 'test', 'unittest'),
cursor.fetchone())
cursor.execute("SELECT item,name,value FROM bitten_report_item "
"WHERE report=%s ORDER BY item", (report.id,))
items = []
prev_item = None
for item, name, value in cursor:
if item != prev_item:
items.append({name: value})
prev_item = item
else:
items[-1][name] = value
self.assertEquals(2, len(items))
seen_foo, seen_bar = False, False
for item in items:
if item['file'] == 'tests/foo.c':
self.assertEqual('failure', item['status'])
seen_foo = True
if item['file'] == 'tests/bar.c':
self.assertEqual('success', item['status'])
seen_bar = True
self.assertEquals((True, True), (seen_foo, seen_bar))
def test_insert_dupe(self):
report = Report(self.env, build=1, step='test', category='test',
generator='unittest')
report.insert()
report = Report(self.env, build=1, step='test', category='test',
generator='unittest')
self.assertRaises(AssertionError, report.insert)
def test_insert_empty_items(self):
report = Report(self.env, build=1, step='test', category='test',
generator='unittest')
report.items = [{}, {}]
report.insert()
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("SELECT build,step,category,generator "
"FROM bitten_report WHERE id=%s", (report.id,))
self.assertEqual((1, 'test', 'test', 'unittest'),
cursor.fetchone())
cursor.execute("SELECT COUNT(*) FROM bitten_report_item "
"WHERE report=%s", (report.id,))
self.assertEqual(0, cursor.fetchone()[0])
def test_fetch(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(1, 'test', 'test', 'unittest'))
report_id = db.get_last_id(cursor, 'bitten_report')
cursor.executemany("INSERT INTO bitten_report_item "
"(report,item,name,value) VALUES (%s,%s,%s,%s)",
[(report_id, 0, 'file', 'tests/foo.c'),
(report_id, 0, 'result', 'failure'),
(report_id, 1, 'file', 'tests/bar.c'),
(report_id, 1, 'result', 'success')])
report = Report.fetch(self.env, report_id)
self.assertEquals(report_id, report.id)
self.assertEquals('test', report.step)
self.assertEquals('test', report.category)
self.assertEquals('unittest', report.generator)
self.assertEquals(2, len(report.items))
assert {'file': 'tests/foo.c', 'result': 'failure'} in report.items
assert {'file': 'tests/bar.c', 'result': 'success'} in report.items
def test_select(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(1, 'test', 'test', 'unittest'))
report1_id = db.get_last_id(cursor, 'bitten_report')
cursor.execute("INSERT INTO bitten_report "
"(build,step,category,generator) VALUES (%s,%s,%s,%s)",
(1, 'test', 'coverage', 'trace'))
report2_id = db.get_last_id(cursor, 'bitten_report')
cursor.executemany("INSERT INTO bitten_report_item "
"(report,item,name,value) VALUES (%s,%s,%s,%s)",
[(report1_id, 0, 'file', 'tests/foo.c'),
(report1_id, 0, 'result', 'failure'),
(report1_id, 1, 'file', 'tests/bar.c'),
(report1_id, 1, 'result', 'success'),
(report2_id, 0, 'file', 'tests/foo.c'),
(report2_id, 0, 'loc', '12'),
(report2_id, 0, 'cov', '50'),
(report2_id, 1, 'file', 'tests/bar.c'),
(report2_id, 1, 'loc', '20'),
(report2_id, 1, 'cov', '25')])
reports = Report.select(self.env, build=1, step='test')
for idx, report in enumerate(reports):
if report.id == report1_id:
self.assertEquals('test', report.step)
self.assertEquals('test', report.category)
self.assertEquals('unittest', report.generator)
self.assertEquals(2, len(report.items))
assert {'file': 'tests/foo.c', 'result': 'failure'} \
in report.items
assert {'file': 'tests/bar.c', 'result': 'success'} \
in report.items
elif report.id == report1_id:
self.assertEquals('test', report.step)
self.assertEquals('coverage', report.category)
self.assertEquals('trace', report.generator)
self.assertEquals(2, len(report.items))
assert {'file': 'tests/foo.c', 'loc': '12', 'cov': '50'} \
in report.items
assert {'file': 'tests/bar.c', 'loc': '20', 'cov': '25'} \
in report.items
self.assertEqual(1, idx)
class PlatformBuildTestCase(BaseModelTestCase):
"""Tests that involve Builds, TargetPlatforms and BuildSteps"""
schemas = [Build._schema, TargetPlatform._schema, BuildStep._schema]
def test_delete_platform_with_pending_builds(self):
"""Check that deleting a platform with pending builds removes those pending builds"""
db = self.env.get_db_cnx()
platform = TargetPlatform(self.env, config='test', name='Linux')
platform.insert()
build = Build(self.env, config='test', platform=platform.id, rev='42', rev_time=12039)
build.insert()
platform.delete()
pending = list(build.select(self.env, config='test', status=Build.PENDING))
self.assertEqual(0, len(pending))
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(BuildConfigTestCase, 'test'))
suite.addTest(unittest.makeSuite(TargetPlatformTestCase, 'test'))
suite.addTest(unittest.makeSuite(BuildTestCase, 'test'))
suite.addTest(unittest.makeSuite(BuildStepTestCase, 'test'))
suite.addTest(unittest.makeSuite(BuildLogTestCase, 'test'))
suite.addTest(unittest.makeSuite(ReportTestCase, 'test'))
suite.addTest(unittest.makeSuite(PlatformBuildTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/notify.py 000644 000765 000120 00000007705 11444412540 017377 0 ustar 00simon admin 000000 000000 #-*- coding: utf-8 -*-
#
# Copyright (C) 2007 Ole Trenner,
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution.
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub, Mock
from trac.web.session import DetachedSession
from bitten.model import schema, Build, BuildStep, BuildLog
from bitten.notify import BittenNotify, BuildNotifyEmail
class BittenNotifyBaseTest(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub(enable=['trac.*', 'bitten.notify.*'])
repos = Mock(get_changeset=lambda rev: Mock(author='author', rev=rev))
self.env.get_repository = lambda authname=None: repos
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
db.commit()
class BittenNotifyTest(BittenNotifyBaseTest):
"""unit tests for BittenNotify dispatcher class"""
def setUp(self):
BittenNotifyBaseTest.setUp(self)
self.dispatcher = BittenNotify(self.env)
self.failed_build = Build(self.env, status=Build.FAILURE)
self.successful_build = Build(self.env, status=Build.SUCCESS)
def test_do_notify_on_failed_build(self):
self.set_option(BittenNotify.notify_on_failure, 'true')
self.assertTrue(self.dispatcher._should_notify(self.failed_build),
'notifier should be called for failed builds.')
def test_do_not_notify_on_failed_build(self):
self.set_option(BittenNotify.notify_on_failure, 'false')
self.assertFalse(self.dispatcher._should_notify(self.failed_build),
'notifier should not be called for failed build.')
def test_do_notify_on_successful_build(self):
self.set_option(BittenNotify.notify_on_success, 'true')
self.assertTrue(self.dispatcher._should_notify(self.successful_build),
'notifier should be called for successful builds when configured.')
def test_do_not_notify_on_successful_build(self):
self.set_option(BittenNotify.notify_on_success, 'false')
self.assertFalse(self.dispatcher._should_notify(self.successful_build),
'notifier should not be called for successful build.')
def set_option(self, option, value):
self.env.config.set(option.section, option.name, value)
class BuildNotifyEmailTest(BittenNotifyBaseTest):
"""unit tests for BittenNotifyEmail class"""
def setUp(self):
BittenNotifyBaseTest.setUp(self)
self.env.config.set('notification','smtp_enabled','true')
self.notifications_sent_to = []
def send(to, cc, hdrs={}):
self.notifications_sent_to = to
def noop(*args, **kw):
pass
self.email = Mock(BuildNotifyEmail, self.env,
begin_send=noop,
finish_send=noop,
send=send)
self.build = Build(self.env, status=Build.SUCCESS, rev=123)
def test_notification_is_sent_to_author(self):
self.email.notify(self.build)
self.assertTrue('author' in self.notifications_sent_to,
'Recipient list should contain the author')
def test_notification_body_render(self):
self.email.notify(self.build)
output = self.email.template.generate(**self.email.data).render('text')
self.assertTrue('Successful build of My Project [123]' in output)
self.assertTrue('' in output)
# TODO functional tests of generated mails
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(BittenNotifyTest, 'test'))
suite.addTest(unittest.makeSuite(BuildNotifyEmailTest, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/queue.py 000644 000765 000120 00000050554 11535320455 017217 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import os
import shutil
import tempfile
import threading
import time
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub, Mock
from trac.util.datefmt import to_datetime, utc
from bitten.model import BuildConfig, TargetPlatform, Build, schema
from bitten.queue import BuildQueue, collect_changes
class CollectChangesTestCase(unittest.TestCase):
"""
Unit tests for the `bitten.queue.collect_changes` function.
"""
def setUp(self):
self.env = EnvironmentStub()
self.env.path = tempfile.mkdtemp()
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
self.config = BuildConfig(self.env, name='test', path='somepath')
self.config.insert(db=db)
self.platform = TargetPlatform(self.env, config='test', name='Foo')
self.platform.insert(db=db)
db.commit()
def tearDown(self):
shutil.rmtree(self.env.path)
def test_stop_on_copy(self):
self.env.get_repository = lambda authname=None: Mock(
get_node=lambda path, rev=None: Mock(
get_history=lambda: [('otherpath', 123, 'copy')]
),
normalize_path=lambda path: path
)
retval = list(collect_changes(self.env.get_repository(), self.config))
self.assertEqual(0, len(retval))
def test_stop_on_minrev(self):
self.env.get_repository = lambda authname=None: Mock(
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
self.config.min_rev = 123
self.config.update()
retval = list(collect_changes(self.env.get_repository(), self.config))
self.assertEqual(1, len(retval))
self.assertEqual(123, retval[0][1])
def test_skip_until_maxrev(self):
self.env.get_repository = lambda authname=None: Mock(
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
self.config.max_rev=121
self.config.update()
retval = list(collect_changes(self.env.get_repository(), self.config))
self.assertEqual(2, len(retval))
self.assertEqual(121, retval[0][1])
self.assertEqual(120, retval[1][1])
def test_skip_empty_dir(self):
def _mock_get_node(path, rev=None):
if rev and rev == 121:
return Mock(
get_entries=lambda: []
)
else:
return Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
)
self.env.get_repository = lambda authname=None: Mock(
get_node=_mock_get_node,
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
retval = list(collect_changes(self.env.get_repository(), self.config))
self.assertEqual(2, len(retval))
self.assertEqual(123, retval[0][1])
self.assertEqual(120, retval[1][1])
class BuildQueueTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub()
self.env.path = tempfile.mkdtemp()
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
db.commit()
# Hook up a dummy repository
self.repos = Mock()
self.env.get_repository = lambda authname=None: self.repos
def tearDown(self):
shutil.rmtree(self.env.path)
def test_get_build_for_slave(self):
"""
Make sure that a pending build of an activated configuration is
scheduled for a slave that matches the target platform.
"""
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name='Foo')
platform.insert()
build = Build(self.env, config='test', platform=platform.id, rev=123,
rev_time=42, status=Build.PENDING)
build.insert()
build_id = build.id
queue = BuildQueue(self.env)
build = queue.get_build_for_slave('foobar', {})
self.assertEqual(build_id, build.id)
def test_next_pending_build_no_matching_slave(self):
"""
Make sure that builds for which there is no slave matching the target
platform are not scheduled.
"""
BuildConfig(self.env, 'test', active=True).insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.PENDING)
build.insert()
build_id = build.id
queue = BuildQueue(self.env)
build = queue.get_build_for_slave('foobar', {})
self.assertEqual(None, build)
def test_next_pending_build_inactive_config(self):
"""
Make sure that builds for a deactived build config are not scheduled.
"""
BuildConfig(self.env, 'test').insert()
platform = TargetPlatform(self.env, config='test', name='Foo')
platform.insert()
build = Build(self.env, config='test', platform=platform.id, rev=123,
rev_time=42, status=Build.PENDING)
build.insert()
queue = BuildQueue(self.env)
build = queue.get_build_for_slave('foobar', {})
self.assertEqual(None, build)
def test_populate_no_repos(self):
"""
Cannot work when there are no repositories defined.
"""
self.env.get_repository = lambda: None
queue = BuildQueue(self.env)
self.assertRaises(AssertionError, queue.populate)
def test_populate_not_build_all(self):
self.env.get_repository = lambda authname=None: Mock(
get_changeset=lambda rev: Mock(date=to_datetime(rev * 1000, utc)),
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
BuildConfig(self.env, 'test', path='somepath', active=True).insert()
platform1 = TargetPlatform(self.env, config='test', name='P1')
platform1.insert()
platform2 = TargetPlatform(self.env, config='test', name='P2')
platform2.insert()
queue = BuildQueue(self.env)
queue.populate()
queue.populate()
queue.populate()
builds = list(Build.select(self.env, config='test'))
builds.sort(lambda a, b: cmp(a.platform, b.platform))
self.assertEqual(2, len(builds))
self.assertEqual(platform1.id, builds[0].platform)
self.assertEqual('123', builds[0].rev)
self.assertEqual(platform2.id, builds[1].platform)
self.assertEqual('123', builds[1].rev)
def test_populate_build_all(self):
self.env.get_repository = lambda authname=None: Mock(
get_changeset=lambda rev: Mock(date=to_datetime(rev * 1000, utc)),
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=lambda: [('somepath', 123, 'edit'),
('somepath', 121, 'edit'),
('somepath', 120, 'edit')]
),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
BuildConfig(self.env, 'test', path='somepath', active=True).insert()
platform1 = TargetPlatform(self.env, config='test', name='P1')
platform1.insert()
platform2 = TargetPlatform(self.env, config='test', name='P2')
platform2.insert()
queue = BuildQueue(self.env, build_all=True)
queue.populate()
queue.populate()
queue.populate()
builds = list(Build.select(self.env, config='test'))
builds.sort(lambda a, b: cmp(a.platform, b.platform))
self.assertEqual(6, len(builds))
self.assertEqual(platform1.id, builds[0].platform)
self.assertEqual('123', builds[0].rev)
self.assertEqual(platform1.id, builds[1].platform)
self.assertEqual('121', builds[1].rev)
self.assertEqual(platform1.id, builds[2].platform)
self.assertEqual('120', builds[2].rev)
self.assertEqual(platform2.id, builds[3].platform)
self.assertEqual('123', builds[3].rev)
self.assertEqual(platform2.id, builds[4].platform)
self.assertEqual('121', builds[4].rev)
self.assertEqual(platform2.id, builds[5].platform)
self.assertEqual('120', builds[5].rev)
def test_populate_thread_race_condition(self):
messages = []
self.env.log = Mock(info=lambda msg, *args: messages.append(msg))
def get_history():
yield ('somepath', 123, 'edit')
yield ('somepath', 121, 'edit')
yield ('somepath', 120, 'edit')
time.sleep(1) # sleep to make sure both threads collect
self.env.get_repository = lambda authname=None: Mock(
get_changeset=lambda rev: Mock(date=to_datetime(rev * 1000, utc)),
get_node=lambda path, rev=None: Mock(
get_entries=lambda: [Mock(), Mock()],
get_history=get_history
),
normalize_path=lambda path: path,
rev_older_than=lambda rev1, rev2: rev1 < rev2
)
BuildConfig(self.env, 'test', path='somepath', active=True).insert()
platform1 = TargetPlatform(self.env, config='test', name='P1')
platform1.insert()
platform2 = TargetPlatform(self.env, config='test', name='P2')
platform2.insert()
def build_populator():
queue = BuildQueue(self.env, build_all=True)
queue.populate()
thread1 = threading.Thread(target=build_populator)
thread2 = threading.Thread(target=build_populator)
# tiny sleep is to avoid odd segementation faults
# (on Linux) and bus errors (on Mac OS X)
thread1.start(); time.sleep(0.01); thread2.start()
thread1.join(); thread2.join()
# check builds got added
builds = list(Build.select(self.env, config='test'))
builds.sort(lambda a, b: cmp(a.platform, b.platform))
self.assertEqual(6, len(builds))
self.assertEqual(platform1.id, builds[0].platform)
self.assertEqual('123', builds[0].rev)
self.assertEqual(platform1.id, builds[1].platform)
self.assertEqual('121', builds[1].rev)
self.assertEqual(platform1.id, builds[2].platform)
self.assertEqual('120', builds[2].rev)
self.assertEqual(platform2.id, builds[3].platform)
self.assertEqual('123', builds[3].rev)
self.assertEqual(platform2.id, builds[4].platform)
self.assertEqual('121', builds[4].rev)
self.assertEqual(platform2.id, builds[5].platform)
self.assertEqual('120', builds[5].rev)
# check attempts at duplicate inserts were logged.
failure_messages = [x for x in messages if x.startswith('Failed to insert build')]
self.assertEqual(6, len(failure_messages))
def test_should_delete_build_platform_dont_exist(self):
messages = []
self.env.log = Mock(info=lambda msg, *args: messages.append(msg))
config = BuildConfig(self.env, 'test', active=True)
config.insert()
build = Build(self.env, config=config.name, rev=42,
platform="no-stuff", rev_time=123456)
build.insert()
queue = BuildQueue(self.env, build_all=True)
self.assertEqual(True, queue.should_delete_build(build, self.repos))
self.assert_("platform no longer exists" in messages[0])
def test_should_delete_build_config_deactivated(self):
messages = []
self.env.log = Mock(info=lambda msg, *args: messages.append(msg))
config = BuildConfig(self.env, 'test', active=False)
config.insert()
platform = TargetPlatform(self.env, config='test', name='stuff')
platform.insert()
build = Build(self.env, config=config.name, rev=42,
platform=platform.id, rev_time=123456)
build.insert()
queue = BuildQueue(self.env, build_all=True)
self.assertEqual(True, queue.should_delete_build(build, self.repos))
self.assert_("configuration is deactivated" in messages[0])
def test_should_delete_build_config_none(self):
out = []
self.env.log = Mock(
info=lambda msg, *args: out.extend([msg] + list(args)))
platform = TargetPlatform(self.env, config='test', name='stuff')
platform.insert()
build = Build(self.env, config='does_not_exist', rev=42,
platform=platform.id, rev_time=123456)
build.insert()
queue = BuildQueue(self.env, build_all=True)
self.assertEqual(True, queue.should_delete_build(build, self.repos))
self.assertTrue("configuration is deactivated" in out[0])
self.assertEquals('unknown config "does_not_exist"', out[1])
def test_should_delete_build_outside_revision_range(self):
messages = []
self.env.log = Mock(info=lambda msg, *args: messages.append(msg))
self.repos.rev_older_than = lambda rev1, rev2: rev1 < rev2
config = BuildConfig(self.env, 'test', active=True, min_rev=120,
max_rev=123)
config.insert()
platform = TargetPlatform(self.env, config='test', name='stuff')
platform.insert()
build1 = Build(self.env, config=config.name, rev=42,
platform=platform.id, rev_time=123456)
build1.insert()
build2 = Build(self.env, config=config.name, rev=10042,
platform=platform.id, rev_time=123456)
build2.insert()
queue = BuildQueue(self.env, build_all=True)
self.assertEqual(True, queue.should_delete_build(build1, self.repos))
self.assertEqual(True, queue.should_delete_build(build2, self.repos))
self.assert_("outside of the revision range" in messages[0])
self.assert_("outside of the revision range" in messages[1])
def test_should_delete_build_old_with_not_buildall(self):
messages = []
self.env.log = Mock(info=lambda msg, *args: messages.append(msg))
config = BuildConfig(self.env, 'test', active=True)
config.insert()
platform = TargetPlatform(self.env, config='test', name='stuff')
platform.insert()
build1 = Build(self.env, config=config.name, rev=42,
platform=platform.id, rev_time=123456)
build1.insert()
build2 = Build(self.env, config=config.name, rev=43,
platform=platform.id, rev_time=123457,
slave='slave')
build2.insert()
queue = BuildQueue(self.env, build_all=False)
self.assertEqual(True, queue.should_delete_build(build1, self.repos))
self.assert_("more recent build exists" in messages[0])
def test_reset_orphaned_builds(self):
BuildConfig(self.env, 'test').insert()
platform = TargetPlatform(self.env, config='test', name='Foo')
platform.insert()
build1 = Build(self.env, config='test', platform=platform.id, rev=123,
rev_time=42, status=Build.IN_PROGRESS, slave='heinz',
last_activity=time.time() - 600) # active ten minutes ago
build1.insert()
build2 = Build(self.env, config='test', platform=platform.id, rev=124,
rev_time=42, status=Build.IN_PROGRESS, slave='heinz',
last_activity=time.time() - 60) # active a minute ago
build2.insert()
queue = BuildQueue(self.env, timeout=300) # 5 minutes timeout
build = queue.reset_orphaned_builds()
self.assertEqual(Build.PENDING, Build.fetch(self.env, build1.id).status)
self.assertEqual(Build.IN_PROGRESS,
Build.fetch(self.env, build2.id).status)
def test_match_slave_match(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('family', 'posix'))
platform.insert()
platform_id = platform.id
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'family': 'posix'})
self.assertEqual(1, len(platforms))
self.assertEqual(platform_id, platforms[0].id)
def test_register_slave_match_simple_fail(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('family', 'posix'))
platform.insert()
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'family': 'nt'})
self.assertEqual([], platforms)
def test_register_slave_match_regexp(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('version', '8\.\d\.\d'))
platform.insert()
platform_id = platform.id
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'version': '8.2.0'})
self.assertEqual(1, len(platforms))
self.assertEqual(platform_id, platforms[0].id)
def test_register_slave_match_regexp_multi(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('os', '^Linux'))
platform.rules.append(('processor', '^[xi]\d?86$'))
platform.insert()
platform_id = platform.id
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'os': 'Linux', 'processor': 'i686'})
self.assertEqual(1, len(platforms))
self.assertEqual(platform_id, platforms[0].id)
def test_register_slave_match_regexp_fail(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('version', '8\.\d\.\d'))
platform.insert()
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'version': '7.8.1'})
self.assertEqual([], platforms)
def test_register_slave_match_regexp_invalid(self):
BuildConfig(self.env, 'test', active=True).insert()
platform = TargetPlatform(self.env, config='test', name="Unix")
platform.rules.append(('version', '8(\.\d'))
platform.insert()
queue = BuildQueue(self.env)
platforms = queue.match_slave('foo', {'version': '7.8.1'})
self.assertEqual([], platforms)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(CollectChangesTestCase, 'test'))
suite.addTest(unittest.makeSuite(BuildQueueTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/upgrades.py 000644 000765 000120 00000033670 11456702627 017714 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2009-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
import logging
import warnings
warnings.filterwarnings('ignore', '^Unknown table')
warnings.filterwarnings('ignore', '^the sets module is deprecated')
from trac.core import TracError
from trac.test import EnvironmentStub
from trac.db import Table, Column, Index, DatabaseManager
from bitten.upgrades import update_sequence, drop_index
from bitten import upgrades, main, model
import os
import shutil
import tempfile
class BaseUpgradeTestCase(unittest.TestCase):
schema = None
other_tables = []
def setUp(self):
self.env = EnvironmentStub()
if hasattr(self.env, 'dburi'):
# Trac gained support for testing against different databases in 0.11.5
# If this support is available, we copy the test db uri configuration
# into the main test config so it can be picked up by
# upgrades.parse_scheme()
self.env.config.set('trac', 'database', self.env.dburi)
self.env.path = tempfile.mkdtemp()
logs_dir = self.env.config.get("bitten", "logs_dir", "log/bitten")
if os.path.isabs(logs_dir):
raise ValueError("Should not have absolute logs directory for temporary test")
logs_dir = os.path.join(self.env.path, logs_dir)
self.logs_dir = logs_dir
db = self.env.get_db_cnx()
cursor = db.cursor()
for table_name in self.other_tables:
cursor.execute("DROP TABLE IF EXISTS %s" % (table_name,))
connector, _ = DatabaseManager(self.env)._get_connector()
for table in self.schema:
cursor.execute("DROP TABLE IF EXISTS %s" % (table.name,))
for stmt in connector.to_sql(table):
cursor.execute(stmt)
db.commit()
def tearDown(self):
shutil.rmtree(self.env.path)
del self.logs_dir
del self.env
class LogWatcher(logging.Handler):
def __init__(self, level=0):
logging.Handler.__init__(self, level=0)
self.records = []
def emit(self, record):
self.records.append(record)
class UpgradeHelperTestCase(BaseUpgradeTestCase):
schema = [
Table('test_update_sequence', key='id')[
Column('id', auto_increment=True), Column('name'),
],
Table('test_drop_index', key='id')[
Column('id', type='int'), Column('name', size=20),
Index(['name'])
],
]
def test_update_sequence(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
for rowid, name in [(1, 'a'), (2, 'b'), (3, 'c')]:
cursor.execute("INSERT INTO test_update_sequence (id, name)"
" VALUES (%s, %s)", (rowid, name))
update_sequence(self.env, db, 'test_update_sequence', 'id')
cursor.execute("INSERT INTO test_update_sequence (name)"
" VALUES (%s)", ('d',))
cursor.execute("SELECT id FROM test_update_sequence WHERE name = %s",
('d',))
row = cursor.fetchone()
self.assertEqual(row[0], 4)
def test_drop_index(self):
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("INSERT INTO test_drop_index (id, name)"
" VALUES (%s, %s)", (1, 'a'))
def do_drop():
drop_index(self.env, db, 'test_drop_index', 'test_drop_index_name_idx')
# dropping the index should succeed the first time and fail the next
do_drop()
self.assertRaises(Exception, do_drop)
class UpgradeScriptsTestCase(BaseUpgradeTestCase):
schema = [
# Sytem
Table('system', key='name')[
Column('name'), Column('value')
],
# Config
Table('bitten_config', key='name')[
Column('name'), Column('path'), Column('label'),
Column('active', type='int'), Column('description')
],
# Platform
Table('bitten_platform', key='id')[
Column('id', auto_increment=True), Column('config'), Column('name')
],
Table('bitten_rule', key=('id', 'propname'))[
Column('id'), Column('propname'), Column('pattern'),
Column('orderno', type='int')
],
# Build
Table('bitten_build', key='id')[
Column('id', auto_increment=True), Column('config'), Column('rev'),
Column('rev_time', type='int'), Column('platform', type='int'),
Column('slave'), Column('started', type='int'),
Column('stopped', type='int'), Column('status', size=1),
Index(['config', 'rev', 'slave'])
],
Table('bitten_slave', key=('build', 'propname'))[
Column('build', type='int'), Column('propname'), Column('propvalue')
],
# Build Step
Table('bitten_step', key=('build', 'name'))[
Column('build', type='int'), Column('name'), Column('description'),
Column('status', size=1), Column('log'),
Column('started', type='int'), Column('stopped', type='int')
],
]
other_tables = [
'bitten_log',
'bitten_log_message',
'bitten_report',
'bitten_report_item',
'bitten_error',
'old_step',
'old_config_v2',
'old_log_v5',
'old_log_v8',
'old_rule_v9',
'old_build_v11',
]
basic_data = [
['system',
('name', 'value'), [
('bitten_version', '1'),
]
],
['bitten_config',
('name',), [
('test_config',),
]
],
['bitten_platform',
('config', 'name'), [
('test_config', 'test_plat'),
]
],
['bitten_build',
('id', 'config', 'rev', 'platform', 'rev_time'), [
(12, 'test_config', '123', 1, 456),
]
],
['bitten_step',
('build', 'name', 'log'), [
(12, 'step1', None),
(12, 'step2', "line1\nline2"),
]
],
]
def _do_upgrade(self):
"""Do an full upgrade."""
import inspect
db = self.env.get_db_cnx()
versions = sorted(upgrades.map.keys())
for version in versions:
for function in upgrades.map.get(version):
self.assertTrue(inspect.getdoc(function))
function(self.env, db)
db.commit()
def _insert_data(self, data):
"""Insert data for upgrading."""
db = self.env.get_db_cnx()
cursor = db.cursor()
for table, cols, vals in data:
cursor.executemany("INSERT INTO %s (%s) VALUES (%s)"
% (table, ','.join(cols),
','.join(['%s' for c in cols])),
vals)
db.commit()
def _check_basic_upgrade(self):
"""Check the results of an upgrade of basic data."""
configs = list(model.BuildConfig.select(self.env,
include_inactive=True))
platforms = list(model.TargetPlatform.select(self.env))
builds = list(model.Build.select(self.env))
steps = list(model.BuildStep.select(self.env))
logs = list(model.BuildLog.select(self.env))
self.assertEqual(len(configs), 1)
self.assertEqual(configs[0].name, 'test_config')
self.assertEqual(len(platforms), 1)
self.assertEqual(platforms[0].config, 'test_config')
self.assertEqual(platforms[0].name, 'test_plat')
self.assertEqual(len(builds), 1)
self.assertEqual(builds[0].id, 12)
self.assertEqual(builds[0].config, 'test_config')
self.assertEqual(builds[0].rev, '123')
self.assertEqual(builds[0].platform, 1)
self.assertEqual(builds[0].rev_time, 456)
self.assertEqual(len(steps), 2)
self.assertEqual(steps[0].build, 12)
self.assertEqual(steps[0].name, 'step1')
self.assertEqual(steps[1].build, 12)
self.assertEqual(steps[1].name, 'step2')
self.assertEqual(len(logs), 1)
self.assertEqual(logs[0].build, 12)
self.assertEqual(logs[0].step, 'step2')
log_file = logs[0].get_log_file(logs[0].filename)
self.assertEqual(file(log_file, "rU").read(), "line1\nline2\n")
# check final sequences
for tbl, col in [
('bitten_build', 'id'),
('bitten_log', 'id'),
('bitten_platform', 'id'),
('bitten_report', 'id'),
]:
self._check_sequence(tbl, col)
def _check_sequence(self, tbl, col):
scheme = upgrades.parse_scheme(self.env)
if scheme == "postgres":
self._check_postgres_sequence(tbl, col)
def _check_postgres_sequence(self, tbl, col):
"""Check a PostgreSQL sequence for the given table and column."""
seq = '%s_%s_seq' % (tbl, col)
cursor = self.env.get_db_cnx().cursor()
cursor.execute("SELECT MAX(%s) FROM %s" % (col, tbl))
current_max = cursor.fetchone()[0] or 0 # if currently None
cursor.execute("SELECT nextval('%s')" % (seq,))
current_seq = cursor.fetchone()[0] - 1
self.assertEqual(current_max, current_seq,
"On %s (col: %s) expected column max (%d) "
"and sequence value (%d) to match"
% (tbl, col, current_max, current_seq))
def test_null_upgrade(self):
self._do_upgrade()
def test_basic_upgrade(self):
self._insert_data(self.basic_data)
self._do_upgrade()
self._check_basic_upgrade()
def test_upgrade_via_buildsetup(self):
self._insert_data(self.basic_data)
db = self.env.get_db_cnx()
build_setup = main.BuildSetup(self.env)
self.assertTrue(build_setup.environment_needs_upgrade(db))
build_setup.upgrade_environment(db)
self._check_basic_upgrade()
# check bitten table version
cursor = db.cursor()
cursor.execute("SELECT value FROM system WHERE name='bitten_version'")
rows = cursor.fetchall()
self.assertEqual(rows, [(str(model.schema_version),)])
def test_fix_log_levels_misnaming(self):
logfiles = {
"1.log": "",
"2.log": "",
"3.log": "",
"1.log.level": "info\n",
"2.log.levels": "info\ninfo\n",
"3.log.level": "warn\n",
"3.log.levels": "warn\nwarn\n",
"4.log.level": "error\n",
}
expected_deletions = [
"4.log.level",
]
os.makedirs(self.logs_dir)
for filename, data in logfiles.items():
path = os.path.join(self.logs_dir, filename)
logfile = open(path, "w")
logfile.write(data)
logfile.close()
logwatch = LogWatcher(logging.INFO)
self.env.log.setLevel(logging.INFO)
self.env.log.addHandler(logwatch)
upgrades.fix_log_levels_misnaming(self.env, None)
filenames = sorted(os.listdir(self.logs_dir))
for filename in filenames:
path = os.path.join(self.logs_dir, filename)
origfile = filename in logfiles and filename or filename.replace("levels", "level")
self.assertEqual(logfiles[origfile], open(path).read())
self.assertTrue(filename not in expected_deletions)
self.assertEqual(len(filenames), len(logfiles) - len(expected_deletions))
logs = sorted(logwatch.records, key=lambda rec: rec.getMessage())
self.assertEqual(len(logs), 5)
self.assertTrue(logs[0].getMessage().startswith(
"Deleted 1 stray log level (0 errors)"))
self.assertTrue(logs[1].getMessage().startswith(
"Deleted stray log level file 4.log.level"))
self.assertTrue(logs[2].getMessage().startswith(
"Error renaming"))
self.assertTrue(logs[3].getMessage().startswith(
"Renamed 1 incorrectly named log level files from previous migrate (1 errors)"))
self.assertTrue(logs[4].getMessage().startswith(
"Renamed incorrectly named log level file"))
def test_remove_stray_log_levels_files(self):
logfiles = {
"1.log": "",
"1.log.levels": "info\n",
"2.log.levels": "info\ninfo\n",
}
expected_deletions = [
"2.log.levels",
]
os.makedirs(self.logs_dir)
for filename, data in logfiles.items():
path = os.path.join(self.logs_dir, filename)
logfile = open(path, "w")
logfile.write(data)
logfile.close()
logwatch = LogWatcher(logging.INFO)
self.env.log.setLevel(logging.INFO)
self.env.log.addHandler(logwatch)
upgrades.remove_stray_log_levels_files(self.env, None)
filenames = sorted(os.listdir(self.logs_dir))
for filename in filenames:
path = os.path.join(self.logs_dir, filename)
self.assertEqual(logfiles[filename], open(path).read())
self.assertTrue(filename not in expected_deletions)
self.assertEqual(len(filenames), len(logfiles) - len(expected_deletions))
logs = sorted(logwatch.records, key=lambda rec: rec.getMessage())
self.assertEqual(len(logs), 2)
self.assertTrue(logs[0].getMessage().startswith(
"Deleted 1 stray log levels (0 errors)"))
self.assertTrue(logs[1].getMessage().startswith(
"Deleted stray log levels file 2.log.levels"))
def test_migrate_logs_to_files_with_logs_dir(self):
os.makedirs(self.logs_dir)
self.assertRaises(TracError, upgrades.migrate_logs_to_files,
self.env, None)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(UpgradeHelperTestCase, 'test'))
suite.addTest(unittest.makeSuite(UpgradeScriptsTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/tests/web_ui.py 000644 000765 000120 00000037713 11500370724 017343 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import shutil
import tempfile
import unittest
from trac.core import TracError
from trac.db import DatabaseManager
from trac.perm import PermissionCache, PermissionSystem
from trac.test import EnvironmentStub, Mock
from trac.util.html import Markup
from trac.web.api import HTTPNotFound
from trac.web.href import Href
from bitten.main import BuildSystem
from bitten.model import Build, BuildConfig, BuildStep, TargetPlatform, schema
from bitten.web_ui import BuildConfigController, BuildController, \
SourceFileLinkFormatter
class AbstractWebUITestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub(enable=['trac.*', 'bitten.*'])
self.env.path = tempfile.mkdtemp()
# Create tables
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
# Set up permissions
self.env.config.set('trac', 'permission_store',
'DefaultPermissionStore')
# Hook up a dummy repository
self.repos = Mock(
get_node=lambda path, rev=None: Mock(get_history=lambda: [],
isdir=True),
normalize_path=lambda path: path,
normalize_rev=lambda rev: rev,
sync=lambda: None,
)
self.repos.authz = Mock(has_permission=lambda path: True, assert_permission=lambda path: None)
self.env.get_repository = lambda authname=None: self.repos
def tearDown(self):
shutil.rmtree(self.env.path)
class BuildConfigControllerTestCase(AbstractWebUITestCase):
def test_overview(self):
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build', href=Href('/trac'), args={}, chrome={},
perm=PermissionCache(self.env, 'joe'), authname='joe')
module = BuildConfigController(self.env)
assert module.match_request(req)
_, data, _ = module.process_request(req)
self.assertEqual('overview', data['page_mode'])
def test_view_config(self):
config = BuildConfig(self.env, name='test', path='trunk')
config.insert()
platform = TargetPlatform(self.env, config='test', name='any')
platform.insert()
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/test', href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
root = Mock(get_entries=lambda: ['foo'],
get_history=lambda: [('trunk', rev, 'edit') for rev in
range(123, 111, -1)])
self.repos = Mock(get_node=lambda path, rev=None: root,
sync=lambda: None, normalize_path=lambda path: path,
normalize_rev=lambda rev: rev, youngest_rev=123)
self.repos.authz = Mock(has_permission=lambda path: True, assert_permission=lambda path: None)
module = BuildConfigController(self.env)
assert module.match_request(req)
_, data, _ = module.process_request(req)
self.assertEqual('view_config', data['page_mode'])
assert not 'next' in req.chrome['links']
from trac.resource import Resource
self.assertEquals(Resource('build', 'test'), data['context'].resource)
self.assertEquals([], data['config']['attachments']['attachments'])
self.assertEquals('/trac/attachment/build/test/',
data['config']['attachments']['attach_href'])
def test_bitten_keeps_order_of_revisions_from_versioncontrol(self):
# Trac's API specifies that they are sorted chronological (backwards)
# We must not assume that these revision numbers can be sorted later on,
# for example the mercurial plugin will return the revisions as strings
# (e.g. '880:4c19fa95fb9e')
config = BuildConfig(self.env, name='test', path='trunk')
config.insert()
platform = TargetPlatform(self.env, config='test', name='any')
platform.insert()
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/'+config.name, href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
# revisions are intentionally not sorted in any way - bitten should just keep them!
revision_ids = [5, 8, 2]
revision_list = [('trunk', revision, 'edit') for revision in revision_ids]
root = Mock(get_entries=lambda: ['foo'], get_history=lambda: revision_list)
self.repos = Mock(get_node=lambda path, rev=None: root,
sync=lambda: None, normalize_path=lambda path: path,
normalize_rev=lambda rev: rev, youngest_rev=5)
self.repos.authz = Mock(has_permission=lambda path: True, assert_permission=lambda path: None)
module = BuildConfigController(self.env)
assert module.match_request(req)
_, data, _ = module.process_request(req)
actual_revision_ids = data['config']['revisions']
self.assertEquals(revision_ids, actual_revision_ids)
def test_view_config_paging(self):
config = BuildConfig(self.env, name='test', path='trunk')
config.insert()
platform = TargetPlatform(self.env, config='test', name='any')
platform.insert()
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/test', href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
root = Mock(get_entries=lambda: ['foo'],
get_history=lambda: [('trunk', rev, 'edit') for rev in
range(123, 110, -1)])
self.repos = Mock(get_node=lambda path, rev=None: root,
sync=lambda: None, normalize_path=lambda path: path,
normalize_rev=lambda rev: rev, youngest_rev=123)
self.repos.authz = Mock(has_permission=lambda path: True, assert_permission=lambda path: None)
module = BuildConfigController(self.env)
assert module.match_request(req)
_, data, _ = module.process_request(req)
if req.chrome:
self.assertEqual('/trac/build/test?page=2',
req.chrome['links']['next'][0]['href'])
def test_raise_404(self):
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
module = BuildConfigController(self.env)
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/nonexisting', href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
self.failUnless(module.match_request(req))
try:
module.process_request(req)
except Exception, e:
self.failUnless(isinstance(e, HTTPNotFound))
self.assertEquals(str(e), "404 Not Found (Build configuration "
"'nonexisting' does not exist.)")
return
self.fail("This should have raised HTTPNotFound")
class BuildControllerTestCase(AbstractWebUITestCase):
def test_view_build(self):
config = BuildConfig(self.env, name='test', path='trunk')
config.insert()
platform = TargetPlatform(self.env, config='test', name='any')
platform.insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/test/1', href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
root = Mock(get_entries=lambda: ['foo'],
get_history=lambda: [('trunk', rev, 'edit') for rev in
range(123, 111, -1)])
self.repos = Mock(get_node=lambda path, rev=None: root,
sync=lambda: None,
normalize_path=lambda path: path,
normalize_rev=lambda rev: rev,
get_changeset=lambda rev: Mock(author='joe'))
self.repos.authz = Mock(has_permission=lambda path: True, assert_permission=lambda path: None)
module = BuildController(self.env)
assert module.match_request(req)
_, data, _ = module.process_request(req)
self.assertEqual('view_build', data['page_mode'])
from trac.resource import Resource
self.assertEquals(Resource('build', 'test/1'), data['context'].resource)
self.assertEquals([], data['build']['attachments']['attachments'])
self.assertEquals('/trac/attachment/build/test/1/',
data['build']['attachments']['attach_href'])
def test_raise_404(self):
PermissionSystem(self.env).grant_permission('joe', 'BUILD_VIEW')
module = BuildController(self.env)
config = BuildConfig(self.env, name='existing', path='trunk')
config.insert()
req = Mock(method='GET', base_path='', cgi_location='',
path_info='/build/existing/42', href=Href('/trac'), args={},
chrome={}, authname='joe',
perm=PermissionCache(self.env, 'joe'))
self.failUnless(module.match_request(req))
try:
module.process_request(req)
except Exception, e:
self.failUnless(isinstance(e, HTTPNotFound))
self.assertEquals(str(e),
"404 Not Found (Build '42' does not exist.)")
return
self.fail("This should have raised HTTPNotFound")
class SourceFileLinkFormatterTestCase(AbstractWebUITestCase):
def test_format_simple_link_in_repos(self):
BuildConfig(self.env, name='test', path='trunk').insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
step = BuildStep(self.env, build=build.id, name='foo',
status=BuildStep.SUCCESS)
step.insert()
self.repos.get_node = lambda path, rev: (path, rev)
req = Mock(method='GET', href=Href('/trac'), authname='hal')
comp = SourceFileLinkFormatter(self.env)
formatter = comp.get_formatter(req, build)
# posix
output = formatter(step, None, None, u'error in foo/bar.c: bad')
self.assertEqual(Markup, type(output))
self.assertEqual('error in '
'foo/bar.c: bad', output)
# windows
output = formatter(step, None, None, u'error in foo\\win.c: bad')
self.assertEqual(Markup, type(output))
self.assertEqual(r'error in '
'foo\win.c: bad', output)
def test_format_bad_links(self):
BuildConfig(self.env, name='test', path='trunk').insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
step = BuildStep(self.env, build=build.id, name='foo',
status=BuildStep.SUCCESS)
step.insert()
self.repos.get_node = lambda path, rev: (path, rev)
req = Mock(method='GET', href=Href('/trac'), authname='hal')
comp = SourceFileLinkFormatter(self.env)
formatter = comp.get_formatter(req, build)
output = formatter(step, None, None, u'Linking -I../.. with ../libtool')
self.assertEqual(Markup, type(output))
self.assertEqual('Linking -I../.. with ../libtool', output)
def test_format_simple_link_not_in_repos(self):
BuildConfig(self.env, name='test', path='trunk').insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
step = BuildStep(self.env, build=build.id, name='foo',
status=BuildStep.SUCCESS)
step.insert()
def _raise():
raise TracError('No such node')
self.repos.get_node = lambda path, rev: _raise()
req = Mock(method='GET', href=Href('/trac'), authname='hal')
comp = SourceFileLinkFormatter(self.env)
formatter = comp.get_formatter(req, build)
output = formatter(step, None, None, u'error in foo/bar.c: bad')
self.assertEqual(Markup, type(output))
self.assertEqual('error in foo/bar.c: bad', output)
def test_format_link_in_repos_with_line(self):
BuildConfig(self.env, name='test', path='trunk').insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
step = BuildStep(self.env, build=build.id, name='foo',
status=BuildStep.SUCCESS)
step.insert()
self.repos.get_node = lambda path, rev: (path, rev)
req = Mock(method='GET', href=Href('/trac'), authname='hal')
comp = SourceFileLinkFormatter(self.env)
formatter = comp.get_formatter(req, build)
# posix
output = formatter(step, None, None, u'error in foo/bar.c:123: bad')
self.assertEqual(Markup, type(output))
self.assertEqual('error in '
'foo/bar.c:123: bad', output)
# windows
output = formatter(step, None, None, u'error in foo\\win.c:123: bad')
self.assertEqual(Markup, type(output))
self.assertEqual(r'error in '
'foo\win.c:123: bad', output)
def test_format_link_not_in_repos_with_line(self):
BuildConfig(self.env, name='test', path='trunk').insert()
build = Build(self.env, config='test', platform=1, rev=123, rev_time=42,
status=Build.SUCCESS, slave='hal')
build.insert()
step = BuildStep(self.env, build=build.id, name='foo',
status=BuildStep.SUCCESS)
step.insert()
def _raise():
raise TracError('No such node')
self.repos.get_node = lambda path, rev: _raise()
req = Mock(method='GET', href=Href('/trac'), authname='hal')
comp = SourceFileLinkFormatter(self.env)
formatter = comp.get_formatter(req, build)
output = formatter(step, None, None, u'error in foo/bar.c:123: bad')
self.assertEqual(Markup, type(output))
self.assertEqual('error in foo/bar.c:123: bad', output)
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(BuildConfigControllerTestCase, 'test'))
suite.addTest(unittest.makeSuite(BuildControllerTestCase, 'test'))
suite.addTest(unittest.makeSuite(SourceFileLinkFormatterTestCase, 'test'))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/templates/bitten_admin_configs.html 000755 000765 000120 00000022162 11234443127 023403 0 ustar 00simon admin 000000 000000
Manage Build Configurations
Bitten-0.6/bitten/templates/json.txt 000644 000765 000120 00000000017 11364623604 020056 0 ustar 00simon admin 000000 000000 ${dumps(json)}
Bitten-0.6/bitten/report/__init__.py 000644 000765 000120 00000000516 11453043212 017764 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
__docformat__ = 'restructuredtext en'
Bitten-0.6/bitten/report/coverage.py 000644 000765 000120 00000025412 11456171632 020035 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
from genshi.builder import tag
from trac.core import *
from trac.mimeview.api import IHTMLPreviewAnnotator
from trac.resource import Resource
from trac.util.datefmt import to_timestamp
from trac.web.api import IRequestFilter
from trac.web.chrome import add_stylesheet, add_ctxtnav, add_warning
from bitten.api import IReportChartGenerator, IReportSummarizer
from bitten.model import BuildConfig, Build, Report
__docformat__ = 'restructuredtext en'
class TestCoverageChartGenerator(Component):
implements(IReportChartGenerator)
# IReportChartGenerator methods
def get_supported_categories(self):
return ['coverage']
def generate_chart_data(self, req, config, category):
assert category == 'coverage'
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT build.rev, SUM(%s) AS loc, SUM(%s * %s / 100) AS cov
FROM bitten_build AS build
LEFT OUTER JOIN bitten_report AS report ON (report.build=build.id)
LEFT OUTER JOIN bitten_report_item AS item_lines
ON (item_lines.report=report.id AND item_lines.name='lines')
LEFT OUTER JOIN bitten_report_item AS item_percentage
ON (item_percentage.report=report.id AND item_percentage.name='percentage' AND
item_percentage.item=item_lines.item)
WHERE build.config=%%s AND report.category='coverage'
AND build.rev_time >= %%s AND build.rev_time <= %%s
GROUP BY build.rev_time, build.rev, build.platform
ORDER BY build.rev_time""" % (db.cast('item_lines.value', 'int'),
db.cast('item_lines.value', 'int'),
db.cast('item_percentage.value', 'int')),
(config.name,
config.min_rev_time(self.env),
config.max_rev_time(self.env)))
prev_rev = None
coverage = []
for rev, loc, cov in cursor:
if rev != prev_rev:
coverage.append([rev, 0, 0])
if loc > coverage[-1][1]:
coverage[-1][1] = int(loc)
if cov > coverage[-1][2]:
coverage[-1][2] = int(cov)
prev_rev = rev
data = {'title': 'Test Coverage',
'data': [
{'label': 'Lines of code', 'data': [[item[0], item[1]] for item in coverage], 'lines': {'fill': True}},
{'label': 'Coverage', 'data': [[item[0], item[2]] for item in coverage]},
],
'options': {
'legend': {'position': 'sw', 'backgroundOpacity': 0.7},
'xaxis': {'tickDecimals': 0},
'yaxis': {'tickDecimals': 0},
},
}
return 'json.txt', {"json": data}
class TestCoverageSummarizer(Component):
implements(IReportSummarizer)
# IReportSummarizer methods
def get_supported_categories(self):
return ['coverage']
def render_summary(self, req, config, build, step, category):
assert category == 'coverage'
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT item_name.value AS unit, item_file.value AS file,
max(item_lines.value) AS loc, max(item_percentage.value) AS cov
FROM bitten_report AS report
LEFT OUTER JOIN bitten_report_item AS item_name
ON (item_name.report=report.id AND item_name.name='name')
LEFT OUTER JOIN bitten_report_item AS item_file
ON (item_file.report=report.id AND item_file.item=item_name.item AND
item_file.name='file')
LEFT OUTER JOIN bitten_report_item AS item_lines
ON (item_lines.report=report.id AND item_lines.item=item_name.item AND
item_lines.name='lines')
LEFT OUTER JOIN bitten_report_item AS item_percentage
ON (item_percentage.report=report.id AND
item_percentage.item=item_name.item AND
item_percentage.name='percentage')
WHERE category='coverage' AND build=%s AND step=%s
GROUP BY file, item_name.value
ORDER BY item_name.value""", (build.id, step.name))
units = []
total_loc, total_cov = 0, 0
for unit, file, loc, cov in cursor:
try:
loc, cov = int(loc), float(cov)
except TypeError:
continue # no rows
if loc:
d = {'name': unit, 'loc': loc, 'cov': int(cov)}
if file:
d['href'] = req.href.browser(config.path, file, rev=build.rev, annotate='coverage')
units.append(d)
total_loc += loc
total_cov += loc * cov
coverage = 0
if total_loc != 0:
coverage = total_cov // total_loc
return 'bitten_summary_coverage.html', {
'units': units,
'totals': {'loc': total_loc, 'cov': int(coverage)}
}
class TestCoverageAnnotator(Component):
"""
>>> from genshi.builder import tag
>>> from trac.test import Mock, MockPerm
>>> from trac.mimeview import Context
>>> from trac.util.datefmt import to_datetime, utc
>>> from trac.web.href import Href
>>> from bitten.model import BuildConfig, Build, Report
>>> from bitten.report.tests.coverage import env_stub_with_tables
>>> env = env_stub_with_tables()
>>> repos = Mock(get_changeset=lambda x: Mock(date=to_datetime(12345, utc)))
>>> env.get_repository = lambda: repos
>>> BuildConfig(env, name='trunk', path='trunk').insert()
>>> Build(env, rev=123, config='trunk', rev_time=12345, platform=1).insert()
>>> rpt = Report(env, build=1, step='test', category='coverage')
>>> rpt.items.append({'file': 'foo.py', 'line_hits': '5 - 0'})
>>> rpt.insert()
>>> ann = TestCoverageAnnotator(env)
>>> req = Mock(href=Href('/'), perm=MockPerm(),
... chrome={'warnings': []}, args={})
Version in the branch should not match:
>>> context = Context.from_request(req, 'source', '/branches/blah/foo.py', 123)
>>> ann.get_annotation_data(context)
[]
Version in the trunk should match:
>>> context = Context.from_request(req, 'source', '/trunk/foo.py', 123)
>>> data = ann.get_annotation_data(context)
>>> print data
[u'5', u'-', u'0']
>>> def annotate_row(lineno, line):
... row = tag.tr()
... ann.annotate_row(context, row, lineno, line, data)
... return row.generate().render('html')
>>> annotate_row(1, 'x = 1')
'
5
'
>>> annotate_row(2, '')
'
'
>>> annotate_row(3, 'y = x')
'
0
'
"""
implements(IRequestFilter, IHTMLPreviewAnnotator)
# IRequestFilter methods
def pre_process_request(self, req, handler):
return handler
def post_process_request(self, req, template, data, content_type):
""" Adds a 'Coverage' context navigation menu item. """
resource = data and data.get('context') \
and data.get('context').resource or None
if resource and isinstance(resource, Resource) \
and resource.realm=='source' and data.get('file') \
and not req.args.get('annotate', '') == 'coverage':
add_ctxtnav(req, tag.a('Coverage',
title='Annotate file with test coverage '
'data (if available)',
href=req.href.browser(resource.id,
annotate='coverage', rev=req.args.get('rev'),
created=data.get('rev')),
rel='nofollow'))
return template, data, content_type
# IHTMLPreviewAnnotator methods
def get_annotation_type(self):
return 'coverage', 'Cov', 'Code coverage'
def get_annotation_data(self, context):
add_stylesheet(context.req, 'bitten/bitten_coverage.css')
resource = context.resource
# attempt to use the version passed in with the request,
# otherwise fall back to the latest version of this file.
version = context.req.args.get('rev', resource.version)
# get the last change revision for the file so that we can
# pick coverage data as latest(version >= file_revision)
created = context.req.args.get('created', resource.version)
repos = self.env.get_repository()
version_time = to_timestamp(repos.get_changeset(version).date)
if version != created:
created_time = to_timestamp(repos.get_changeset(created).date)
else:
created_time = version_time
self.log.debug("Looking for coverage report for %s@%s [%s:%s]..." % (
resource.id, str(resource.version), created, version))
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT b.id, b.rev, i2.value
FROM bitten_config AS c
INNER JOIN bitten_build AS b ON c.name=b.config
INNER JOIN bitten_report AS r ON b.id=r.build
INNER JOIN bitten_report_item AS i1 ON r.id=i1.report
INNER JOIN bitten_report_item AS i2 ON (i1.item=i2.item
AND i1.report=i2.report)
WHERE i2.name='line_hits'
AND b.rev_time>=%s
AND b.rev_time<=%s
AND i1.name='file'
AND """ + db.concat('c.path', "'/'", 'i1.value') + """=%s
ORDER BY b.rev_time DESC LIMIT 1""" ,
(created_time, version_time, resource.id.lstrip('/')))
row = cursor.fetchone()
if row:
build_id, build_rev, line_hits = row
coverage = line_hits.split()
self.log.debug("Coverage annotate for %s@%s using build %d: %s",
resource.id, build_rev, build_id, coverage)
return coverage
add_warning(context.req, "No coverage annotation found for "
"/%s for revision range [%s:%s]." % (
resource.id.lstrip('/'), version, created))
return []
def annotate_row(self, context, row, lineno, line, data):
from genshi.builder import tag
lineno -= 1 # 0-based index for data
if lineno >= len(data):
row.append(tag.th())
return
row_data = data[lineno]
if row_data == '-':
row.append(tag.th())
elif row_data == '0':
row.append(tag.th(row_data, class_='uncovered'))
else:
row.append(tag.th(row_data, class_='covered'))
Bitten-0.6/bitten/report/lint.py 000644 000765 000120 00000014360 11453043212 017175 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2007 Jeffrey Kyllo
#
# Based on code from the Bitten project:
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://echospiral.com/trac/eatlint/wiki/License.
__docformat__ = 'restructuredtext en'
from trac.core import *
from bitten.api import IReportChartGenerator, IReportSummarizer
class PyLintChartGenerator(Component):
implements(IReportChartGenerator)
# IReportChartGenerator methods
def get_supported_categories(self):
return ['lint']
def generate_chart_data(self, req, config, category):
assert category == 'lint'
db = self.env.get_db_cnx()
cursor = db.cursor()
#self.log.debug('config.name=\'%s\'' % (config.name,))
query = """
select build.rev,
(select count(*) from bitten_report_item as item
where item.report = report.id and item.name='category' and item.value='convention'),
(select count(*) from bitten_report_item as item
where item.report = report.id and item.name='category' and item.value='error'),
(select count(*) from bitten_report_item as item
where item.report = report.id and item.name='category' and item.value='refactor'),
(select count(*) from bitten_report_item as item
where item.report = report.id and item.name='category' and item.value='warning')
from bitten_report as report
left outer join bitten_build as build ON (report.build=build.id)
where build.config='%s' and report.category='lint'
and build.rev_time >= %s and build.rev_time <= %s
group by build.rev_time, build.rev, build.platform, report.id
order by build.rev_time;""" % (config.name,
config.min_rev_time(self.env),
config.max_rev_time(self.env))
#self.log.debug('sql=\'%s\'' % (query,))
cursor.execute(query)
lint = []
prev_rev = None
prev_counts = None
for rev, conv, err, ref, warn in cursor:
total = conv + err + ref + warn
curr_counts = [rev, total, conv, err, ref, warn]
if rev != prev_rev:
lint.append(curr_counts)
else:
# cunningly / dubiously set rev = max(rev, rev) along with the counts
lint[-1] = [max(prev, curr) for prev, curr in zip(curr_counts, lint[-1])]
# recalculate total
lint[-1][1] = sum(lint[-1][2:])
prev_rev = rev
data = {'title': 'Lint Problems by Type',
'data': [
{'label': 'Total Problems', 'data': [[item[0], item[1]] for item in lint], 'lines': {'fill': True}},
{'label': 'Convention', 'data': [[item[0], item[2]] for item in lint]},
{'label': 'Error', 'data': [[item[0], item[3]] for item in lint]},
{'label': 'Refactor', 'data': [[item[0], item[4]] for item in lint]},
{'label': 'Warning', 'data': [[item[0], item[5]] for item in lint]},
],
'options': {
'legend': {'position': 'sw', 'backgroundOpacity': 0.7},
'xaxis': {'tickDecimals': 0},
'yaxis': {'tickDecimals': 0},
},
}
return 'json.txt', {"json": data}
class PyLintSummarizer(Component):
implements(IReportSummarizer)
# IReportSummarizer methods
def get_supported_categories(self):
return ['lint']
def render_summary(self, req, config, build, step, category):
assert category == 'lint'
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT item_type.value AS type, item_file.value AS file,
item_line.value as line, item_category.value as category,
report.category as report_category
FROM bitten_report AS report
LEFT OUTER JOIN bitten_report_item AS item_type
ON (item_type.report=report.id AND item_type.name='type')
LEFT OUTER JOIN bitten_report_item AS item_file
ON (item_file.report=report.id AND
item_file.item=item_type.item AND
item_file.name='file')
LEFT OUTER JOIN bitten_report_item AS item_line
ON (item_line.report=report.id AND
item_line.item=item_type.item AND
item_line.name='lines')
LEFT OUTER JOIN bitten_report_item AS item_category
ON (item_category.report=report.id AND
item_category.item=item_type.item AND
item_category.name='category')
WHERE report.category='lint' AND build=%s AND step=%s
ORDER BY item_type.value""", (build.id, step.name))
file_data = {}
type_total = {}
category_total = {}
line_total = 0
file_total = 0
seen_files = {}
for type, file, line, category, report_category in cursor:
if not file_data.has_key(file):
file_data[file] = {'file': file, 'type': {}, 'lines': 0, 'category': {}}
d = file_data[file]
#d = {'type': type, 'line': line, 'category': category}
if not d['type'].has_key(type):
d['type'][type] = 0
d['type'][type] += 1
d['lines'] += 1
line_total += 1
if not d['category'].has_key(category):
d['category'][category] = 0
d['category'][category] += 1
if file:
d['href'] = req.href.browser(config.path, file)
if not type_total.has_key(type):
type_total[type] = 0
type_total[type] += 1
if not category_total.has_key(category):
category_total[category] = 0
category_total[category] += 1
if not seen_files.has_key(file):
seen_files[file] = 0
file_total += 1
data = []
for d in file_data.values():
d['catnames'] = d['category'].keys()
data.append(d)
template_data = {}
template_data['data'] = data
template_data['totals'] = {'type': type_total, 'category': category_total, 'files': file_total, 'lines': line_total}
return 'bitten_summary_lint.html', template_data
Bitten-0.6/bitten/report/testing.py 000644 000765 000120 00000016253 11453043212 017707 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
from trac.core import *
from trac.web.chrome import add_script
from bitten.api import IReportChartGenerator, IReportSummarizer
__docformat__ = 'restructuredtext en'
class TestResultsChartGenerator(Component):
implements(IReportChartGenerator)
# IReportChartGenerator methods
def get_supported_categories(self):
return ['test']
def generate_chart_data(self, req, config, category):
assert category == 'test'
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT build.rev, build.platform, item_status.value AS status, COUNT(*) AS num
FROM bitten_build AS build
LEFT OUTER JOIN bitten_report AS report ON (report.build=build.id)
LEFT OUTER JOIN bitten_report_item AS item_status
ON (item_status.report=report.id AND item_status.name='status')
WHERE build.config=%s AND report.category='test'
AND build.rev_time >= %s AND build.rev_time <= %s
GROUP BY build.rev_time, build.rev, build.platform, item_status.value
ORDER BY build.rev_time, build.platform""", (config.name,
config.min_rev_time(self.env),
config.max_rev_time(self.env)))
prev_rev = None
prev_platform, platform_total = None, 0
tests = []
for rev, platform, status, num in cursor:
if rev != prev_rev:
tests.append([rev, 0, 0, 0])
prev_rev = rev
platform_total = 0
if platform != prev_platform:
prev_platform = platform
platform_total = 0
platform_total += num
tests[-1][1] = max(platform_total, tests[-1][1])
if status == 'success':
pass
elif status == 'ignore':
tests[-1][3] = max(num, tests[-1][3])
else:
tests[-1][2] = max(num, tests[-1][2])
data = {'title': 'Unit Tests',
'data': [
{'label': 'Total', 'data': [[item[0], item[1]] for item in tests], 'lines': {'fill': True}},
{'label': 'Failures', 'data': [[item[0], item[2]] for item in tests]},
{'label': 'Ignored', 'data': [[item[0], item[3]] for item in tests]},
],
'options': {
'legend': {'position': 'sw', 'backgroundOpacity': 0.7},
'xaxis': {'tickDecimals': 0},
'yaxis': {'tickDecimals': 0},
},
}
return 'json.txt', {"json": data}
class TestResultsSummarizer(Component):
implements(IReportSummarizer)
# IReportSummarizer methods
def get_supported_categories(self):
return ['test']
def render_summary(self, req, config, build, step, category):
assert category == 'test'
db = self.env.get_db_cnx()
cursor = db.cursor()
cursor.execute("""
SELECT item_fixture.value AS fixture, item_file.value AS file,
COUNT(item_success.value) AS num_success,
COUNT(item_ignore.value) AS num_ignore,
COUNT(item_failure.value) AS num_failure,
COUNT(item_error.value) AS num_error
FROM bitten_report AS report
LEFT OUTER JOIN bitten_report_item AS item_fixture
ON (item_fixture.report=report.id AND item_fixture.name='fixture')
LEFT OUTER JOIN bitten_report_item AS item_file
ON (item_file.report=report.id AND item_file.item=item_fixture.item AND
item_file.name='file')
LEFT OUTER JOIN bitten_report_item AS item_success
ON (item_success.report=report.id AND item_success.item=item_fixture.item AND
item_success.name='status' AND item_success.value='success')
LEFT OUTER JOIN bitten_report_item AS item_ignore
ON (item_ignore.report=report.id AND item_ignore.item=item_fixture.item AND
item_ignore.name='status' AND item_ignore.value='ignore')
LEFT OUTER JOIN bitten_report_item AS item_failure
ON (item_failure.report=report.id AND item_failure.item=item_fixture.item AND
item_failure.name='status' AND item_failure.value='failure')
LEFT OUTER JOIN bitten_report_item AS item_error
ON (item_error.report=report.id AND item_error.item=item_fixture.item AND
item_error.name='status' AND item_error.value='error')
WHERE category='test' AND build=%s AND step=%s
GROUP BY file, fixture
ORDER BY fixture""", (build.id, step.name))
fixtures = []
total_success, total_ignore, total_failure, total_error = 0, 0, 0, 0
for fixture, file, num_success, num_ignore, num_failure, num_error in cursor:
fixtures.append({'name': fixture,
'num_success': num_success,
'num_ignore': num_ignore,
'num_error': num_error,
'num_failure': num_failure})
total_success += num_success
total_ignore += num_ignore
total_failure += num_failure
total_error += num_error
if file:
fixtures[-1]['href'] = req.href.browser(config.path, file)
# For each fixture, get a list of tests that don't succeed
for fixture in fixtures:
cursor.execute("""
SELECT item_status.value AS status, item_name.value AS name,
item_traceback.value AS traceback
FROM bitten_report
LEFT OUTER JOIN bitten_report_item AS item_fixture
ON (item_fixture.report=bitten_report.id AND
item_fixture.name='fixture')
LEFT OUTER JOIN bitten_report_item AS item_status
ON (item_status.report=bitten_report.id AND
item_status.item=item_fixture.item AND
item_status.name='status')
LEFT OUTER JOIN bitten_report_item AS item_name
ON (item_name.report=bitten_report.id AND
item_name.item=item_fixture.item AND
item_name.name='name')
LEFT OUTER JOIN bitten_report_item AS item_traceback
ON (item_traceback.report=bitten_report.id AND
item_traceback.item=item_fixture.item AND
item_traceback.name='traceback')
WHERE category='test' AND build=%s AND step=%s AND item_status.value<>'success' AND
item_fixture.value=%s""", (build.id, step.name, fixture['name']))
failures = []
for status, name, traceback in cursor:
# use the fixture name if a name isn't supplied for the
# individual test
if not name:
name = fixture['name']
failures.append({'status': status,
'name': name,
'traceback': traceback})
if failures:
fixture['failures'] = failures
data = {'fixtures': fixtures,
'totals': {'success': total_success,
'ignore': total_ignore,
'failure': total_failure,
'error': total_error}
}
return 'bitten_summary_tests.html', data
Bitten-0.6/bitten/report/tests/ 000755 000765 000120 00000000000 11536424600 017021 5 ustar 00simon admin 000000 000000 Bitten-0.6/bitten/report/tests/__init__.py 000644 000765 000120 00000001226 11453043212 021125 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
from bitten.report.tests import coverage, lint, testing
def suite():
suite = unittest.TestSuite()
suite.addTest(coverage.suite())
suite.addTest(lint.suite())
suite.addTest(testing.suite())
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/report/tests/coverage.py 000644 000765 000120 00000010770 11453043212 021165 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import doctest
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub, Mock
from bitten.model import *
from bitten.report import coverage
from bitten.report.coverage import TestCoverageChartGenerator
def env_stub_with_tables():
env = EnvironmentStub()
db = env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
return env
class TestCoverageChartGeneratorTestCase(unittest.TestCase):
def setUp(self):
self.env = env_stub_with_tables()
self.env.path = ''
def test_supported_categories(self):
generator = TestCoverageChartGenerator(self.env)
self.assertEqual(['coverage'], generator.get_supported_categories())
def test_no_reports(self):
req = Mock()
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
generator = TestCoverageChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'coverage')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Test Coverage', data['title'])
actual_data = data['data']
self.assertEqual('Lines of code', actual_data[0]['label'])
self.assertEqual('Coverage', actual_data[1]['label'])
def test_single_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo',
category='coverage')
report.items += [{'lines': '12', 'percentage': '25'}]
report.insert()
req = Mock()
generator = TestCoverageChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'coverage')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Test Coverage', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Lines of code', actual_data[0]['label'])
self.assertEqual(12, actual_data[0]['data'][0][1])
self.assertEqual('Coverage', actual_data[1]['label'])
self.assertEqual(3, actual_data[1]['data'][0][1])
def test_multi_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo',
category='coverage')
report.items += [{'lines': '12', 'percentage': '25'}]
report.insert()
build = Build(self.env, config='trunk', platform=2, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo',
category='coverage')
report.items += [{'lines': '12', 'percentage': '50'}]
report.insert()
req = Mock()
generator = TestCoverageChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'coverage')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Test Coverage', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Lines of code', actual_data[0]['label'])
self.assertEqual(12, actual_data[0]['data'][0][1])
self.assertEqual('Coverage', actual_data[1]['label'])
self.assertEqual(6, actual_data[1]['data'][0][1])
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestCoverageChartGeneratorTestCase))
suite.addTest(doctest.DocTestSuite(coverage))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/report/tests/lint.py 000644 000765 000120 00000012721 11453043212 020336 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub, Mock
from bitten.model import *
from bitten.report.lint import PyLintChartGenerator
class PyLintChartGeneratorTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub()
self.env.path = ''
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
def test_supported_categories(self):
generator = PyLintChartGenerator(self.env)
self.assertEqual(['lint'], generator.get_supported_categories())
def test_no_reports(self):
req = Mock()
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
generator = PyLintChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'lint')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Lint Problems by Type', data['title'])
actual_data = data['data']
self.assertEqual('Total Problems', actual_data[0]['label'])
self.assertEqual('Convention', actual_data[1]['label'])
self.assertEqual('Error', actual_data[2]['label'])
self.assertEqual('Refactor', actual_data[3]['label'])
self.assertEqual('Warning', actual_data[4]['label'])
def test_single_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='lint')
report.items += [{'category': 'convention'}, {'category': 'warning'},
{'category': 'error'}, {'category': 'refactor'},
{'category': 'warning'}, {'category': 'error'},
{'category': 'refactor'}, {'category': 'error'},
{'category': 'refactor'}, {'category': 'refactor'}]
report.insert()
req = Mock()
generator = PyLintChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'lint')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Lint Problems by Type', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Total Problems', actual_data[0]['label'])
self.assertEqual(10, actual_data[0]['data'][0][1])
self.assertEqual('Convention', actual_data[1]['label'])
self.assertEqual(1, actual_data[1]['data'][0][1])
self.assertEqual('Error', actual_data[2]['label'])
self.assertEqual(3, actual_data[2]['data'][0][1])
self.assertEqual('Refactor', actual_data[3]['label'])
self.assertEqual(4, actual_data[3]['data'][0][1])
self.assertEqual('Warning', actual_data[4]['label'])
self.assertEqual(2, actual_data[4]['data'][0][1])
def test_multi_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='lint')
report.items += [{'category': 'error'}, {'category': 'refactor'}]
report.insert()
build = Build(self.env, config='trunk', platform=2, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='lint')
report.items += [{'category': 'convention'}, {'category': 'warning'}]
report.insert()
req = Mock()
generator = PyLintChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'lint')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Lint Problems by Type', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Total Problems', actual_data[0]['label'])
self.assertEqual(4, actual_data[0]['data'][0][1])
self.assertEqual('Convention', actual_data[1]['label'])
self.assertEqual(1, actual_data[1]['data'][0][1])
self.assertEqual('Error', actual_data[2]['label'])
self.assertEqual(1, actual_data[2]['data'][0][1])
self.assertEqual('Refactor', actual_data[3]['label'])
self.assertEqual(1, actual_data[3]['data'][0][1])
self.assertEqual('Warning', actual_data[4]['label'])
self.assertEqual(1, actual_data[4]['data'][0][1])
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(PyLintChartGeneratorTestCase))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/report/tests/testing.py 000644 000765 000120 00000016444 11453043212 021053 0 ustar 00simon admin 000000 000000 # -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007 Christopher Lenz
# Copyright (C) 2007-2010 Edgewall Software
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at http://bitten.edgewall.org/wiki/License.
import unittest
from trac.db import DatabaseManager
from trac.test import EnvironmentStub, Mock
from trac.web.href import Href
from bitten.model import *
from bitten.report.testing import TestResultsChartGenerator, \
TestResultsSummarizer
class TestResultsChartGeneratorTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub()
self.env.path = ''
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
def test_supported_categories(self):
generator = TestResultsChartGenerator(self.env)
self.assertEqual(['test'], generator.get_supported_categories())
def test_no_reports(self):
req = Mock()
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
generator = TestResultsChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'test')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Unit Tests', data['title'])
actual_data = data['data']
self.assertEqual([], actual_data[0]['data'])
self.assertEqual('Total', actual_data[0]['label'])
self.assertEqual('Failures', actual_data[1]['label'])
def test_single_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='test')
report.items += [{'status': 'success'}, {'status': 'failure'},
{'status': 'success'}]
report.insert()
req = Mock()
generator = TestResultsChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'test')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Unit Tests', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Total', actual_data[0]['label'])
self.assertEqual(3, actual_data[0]['data'][0][1])
self.assertEqual('Failures', actual_data[1]['label'])
self.assertEqual(1, actual_data[1]['data'][0][1])
def test_multi_platform(self):
config = Mock(name='trunk', min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
build = Build(self.env, config='trunk', platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='test')
report.items += [{'status': 'success'}, {'status': 'failure'},
{'status': 'success'}]
report.insert()
build = Build(self.env, config='trunk', platform=2, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step='foo', category='test')
report.items += [{'status': 'success'}, {'status': 'failure'},
{'status': 'failure'}]
report.insert()
req = Mock()
generator = TestResultsChartGenerator(self.env)
template, data = generator.generate_chart_data(req, config, 'test')
self.assertEqual('json.txt', template)
data = data['json']
self.assertEqual('Unit Tests', data['title'])
actual_data = data['data']
self.assertEqual('123', actual_data[0]['data'][0][0])
self.assertEqual('Total', actual_data[0]['label'])
self.assertEqual(3, actual_data[0]['data'][0][1])
self.assertEqual('123', actual_data[1]['data'][0][0])
self.assertEqual('Failures', actual_data[1]['label'])
self.assertEqual(2, actual_data[1]['data'][0][1])
self.assertEqual('123', actual_data[2]['data'][0][0])
self.assertEqual('Ignored', actual_data[2]['label'])
self.assertEqual(0, actual_data[2]['data'][0][1])
class TestResultsSummarizerTestCase(unittest.TestCase):
def setUp(self):
self.env = EnvironmentStub()
self.env.path = ''
db = self.env.get_db_cnx()
cursor = db.cursor()
connector, _ = DatabaseManager(self.env)._get_connector()
for table in schema:
for stmt in connector.to_sql(table):
cursor.execute(stmt)
def test_testcase_errors_and_failures(self):
config = Mock(name='trunk', path='/somewhere',
min_rev_time=lambda env: 0,
max_rev_time=lambda env: 1000)
step = Mock(name='foo')
build = Build(self.env, config=config.name, platform=1, rev=123,
rev_time=42)
build.insert()
report = Report(self.env, build=build.id, step=step.name,
category='test')
report.items += [{'fixture': 'test_foo',
'name': 'foo', 'file': 'foo.c',
'type': 'test', 'status': 'success'},
{'fixture': 'test_bar',
'name': 'bar', 'file': 'bar.c',
'type': 'test', 'status': 'error',
'traceback': 'Error traceback'},
{'fixture': 'test_baz',
'name': 'baz', 'file': 'baz.c',
'type': 'test', 'status': 'failure',
'traceback': 'Failure reason'}]
report.insert()
req = Mock(href=Href('trac'))
generator = TestResultsSummarizer(self.env)
template, data = generator.render_summary(req,
config, build, step, 'test')
self.assertEquals('bitten_summary_tests.html', template)
self.assertEquals(data['totals'],
{'ignore': 0, 'failure': 1, 'success': 1, 'error': 1})
for fixture in data['fixtures']:
if fixture.has_key('failures'):
if fixture['failures'][0]['status'] == 'error':
self.assertEquals('test_bar', fixture['name'])
self.assertEquals('Error traceback',
fixture['failures'][0]['traceback'])
if fixture['failures'][0]['status'] == 'failure':
self.assertEquals('test_baz', fixture['name'])
self.assertEquals('Failure reason',
fixture['failures'][0]['traceback'])
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestResultsChartGeneratorTestCase))
suite.addTest(unittest.makeSuite(TestResultsSummarizerTestCase))
return suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')
Bitten-0.6/bitten/htdocs/admin.css 000644 000765 000120 00000000571 11234443127 017435 0 ustar 00simon admin 000000 000000 table.form th { text-align: right; }
div.platforms h3 { margin-top: 3em; }
table#platformlist { width: 67%; }
table#platformlist td ul { list-style: none; margin: 0; padding: 0; }
fieldset#newplatform { float:right; width: 30%; margin-top: -0.75em;
padding-top: 0; }
dl.help { color: #666; font-size: 90%; margin: 1em .5em; }
dl.help dt { font-weight: bold; }
Bitten-0.6/bitten/htdocs/bitten.css 000644 000765 000120 00000020640 11364623604 017635 0 ustar 00simon admin 000000 000000 /* Timeline styles */
#content.timeline dt.successbuild, #content.timeline dt.successbuild a {
background-image: url(bitten_build.png) !important;
}
#content.timeline dt.failedbuild, #content.timeline dt.failedbuild a {
background-image: url(bitten_buildf.png) !important;
}
#content.build h2.config, #content.build h2.step { background: #f7f7f7;
border-bottom: 1px solid #d7d7d7; margin: 2em 0 0; clear: both;
}
#content.build h2.config :link, #content.build h2.config :visited {
color: #b00; display: block; border-bottom: none;
}
#content.build h2.deactivated { text-decoration: line-through; }
#content.build #prefs { line-height: 1.4em; }
#content.build h3.builds { font-weight: bold; text-align: left;
margin: 2em 0 0 2em;
}
#content.build table.builds { border-collapse: separate;
border-top: 1px solid #666; margin-left: 2em; table-layout: fixed;
}
#content.build table.builds th { padding: 0 1em 0 .25em; text-align: left;
vertical-align: top;
}
#content.build table.builds th p { color: #666; font-size: smaller;
margin-top: 0;
}
#content.build table.builds th p.message { font-style: italic; }
#content.build table.builds td { color: #999; border: 1px solid;
padding: .25em .5em; vertical-align: top;
}
#content.build table.builds td :link, #content.build table.builds td :visited {
font-weight: bold;
}
#content.build table.builds td.completed { background: #9d9; border-color: #696;
color: #393;
}
#content.build table.builds td.failed { background: #d99; border-color: #966;
color: #933;
}
#content.build table.builds td.in-progress { background: #dd9;
border-color: #996; color: #993;
}
#content.build table.builds td p { font-size: smaller; margin-top: 0; }
#content.build table.builds .status { color: #000; }
#content.build table.builds .system { font-size: smaller; line-height: 1.2em;
margin: .5em 0;
}
#content.build form.config { margin-top: 1em; }
#content.build form.config th { text-align: left; }
#content.build form.config fieldset { margin-bottom: 1em; }
#content.build div.platforms { margin-top: 2em; }
#content.build form.platforms ul { list-style-type: none; padding-left: 1em; }
#content.build p.path { color: #999; font-size: smaller; margin-top: 0; }
#content.build #charts { clear: right; float: right; margin-left: 0.5em; }
#content.build #builds { clear: none; margin-top: 2em; table-layout: fixed;
width: auto;
}
#content.build #builds tbody th, #content.build #builds tbody td {
background: #fff; width: 40em;
}
#content.build #builds th.chgset { width: 5em; }
#content.build #builds td :link, #content.build #builds td :visited {
font-weight: bold;
}
#content.build #builds tbody td { background-position: 2px .5em;
background-repeat: no-repeat;
}
#content.build #builds td.completed {
background-color: #e8f6e8; background-image: url(bitten_build.png);
}
#content.build #builds td.failed {
background-color: #fbe8e7; background-image: url(bitten_buildf.png);
}
#content.build #builds td.in-progress {
background-color: #f6fae0; background-image: url(bitten_build.png);
}
#content.build #builds .info { margin-left: 16px; }
#content.build #builds :link, #content.build #builds :visited {
text-decoration: none;
}
#content.build #builds .info .status { color: #000; }
#content.build #builds .info .system { color: #999; font-size: smaller;
line-height: 1.2em; margin-top: .5em;
}
#content.build #builds ul.steps {
list-style-type: none; margin: .5em 0 0; padding: 0;
}
#content.build #builds ul.steps li.success,
#content.build #builds ul.steps li.in-progress,
#content.build #builds ul.steps li.failed {
border: 1px solid; margin: 1px 0; padding: 0 2px 0 12px;
}
#content.build #builds ul.steps li.in-progress {
background: #dd9; border-color: #966; color: #993;
}
#content.build #builds ul.steps li.success {
background: #9d9; border-color: #696; color: #393;
}
#content.build #builds ul.steps li.failed {
background: #d99 url(failure.png) 2px .3em no-repeat; border-color: #966;
color: #933;
}
#content.build #builds ul.steps li :link,
#content.build #builds ul.steps li :visited { border: none; color: inherit;
font-weight: bold; text-decoration: none;
}
#content.build #builds ul.steps li .duration { float: right;
font-size: smaller;
}
#content.build #builds ul.steps li.success .duration { color: #696; }
#content.build #builds ul.steps li.failed .duration { color: #966; }
#content.build #builds ul.steps li.failed ul { font-size: smaller;
line-height: 1.2em; list-style-type: square; margin: 0;
padding: 0 0 .5em 1.5em;
}
#content.build #overview { line-height: 130%; margin-top: 1em; padding: .5em; }
#content.build #overview dt { font-weight: bold; padding-right: .25em;
position: absolute; left: 0; text-align: right; width: 11.5em;
}
#content.build #overview dd { margin-left: 12em; }
#content.build #overview .slave { margin-top: 1em; }
#content.build #overview .time { margin-top: 1em; }
#content.build div.errors { background: #d99; border: 1px solid #966;
color: #933; float: right; margin: 1em;
}
#content.build div.errors h3 { background: #966; color: #fff; margin: 0;
padding: 0 .3em;
}
#content.build div.errors ul { list-style-image: url(failure.png); margin: 0;
padding: .5em 1.75em;
}
#content.build .tabs { clear: right; list-style: none; float: left; width: 100%;
margin: 0 1em; padding: 0;
}
#content.build .tabs li { cursor: pointer; float: left; }
#content.build .tabs li a { background: #b9b9b9; color: #666; display: block;
margin: 2px 2px 0; padding: 3px 2em 0;
}
#content.build .tabs li a:hover { color: #333; text-decoration: none; }
#content.build .tabs li.active a { background: #d7d7d7; border: 1px outset;
border-bottom: none; color: #333; font-weight: bold; margin-top: 0;
padding-bottom: 1px;
}
#content.build .tab-content { background: #f4f4f4; border: 1px outset;
clear: both; margin: 0 2em 0 1em; padding: 5px;
}
#content.build .tab-content table { margin: 0; }
#content.build tbody.totals td, #content.build tbody.totals th {
font-weight: bold;
}
#content.build table.tests tr.failed th,
#content.build table.tests tr.failed td { font-weight: bold; }
#content.build table.tests tr.failed :link,
#content.build table.tests tr.failed :visited { color: #b00 }
/* collapsible failure details */
#content.build table.tests tr th p { margin: 0; padding: 0; text-align: left; }
#content.build table.tests tr th p.details {
margin: 0; padding-left: 4px; padding-top: 5px; text-align: left; font-weight: normal;
}
#content.build table.tests tr th p.details span {
white-space: pre; font-family: monospace; font-weight: normal; font-size: smaller;
color: #666;
}
#content.build table.tests .fixture { display: inline-block; }
#content.build table.tests tr.failed th .fixture a {
background: url(../common/expanded.png) 50% 50% no-repeat; padding-left: 16px;
}
#content.build table.tests tr.failed th.collapsed .fixture a {
background-image: url(../common/collapsed.png);
}
#content.build table.tests tr.failed th.collapsed p.details { display: none; }
#content.build .log { background: #fff; border: 1px inset; font-size: 90%;
overflow: auto; max-height: 20em; width: 100%; white-space: pre;
}
#content.build .log code { padding: 0 5px; }
#content.build .log .warning { color: #660; font-weight: bold; }
#content.build .log .error { color: #900; font-weight: bold; }
#content.build table.listing th, #content.build table.listing td {
font-size: 95%;
}
#content.build table.listing tbody th, #content.build table.listing tbody td {
background: #fff; padding: .1em .3em;
}
#content.build table.listing :link, #content.build table.listing :visited {
border: none;
}
#mainnav .bitteninprogress, #mainnav .bitteninprogress :hover {
border-right: 4px solid #edd400 !important;
}
#mainnav .bittencompleted, #mainnav .bittencompleted :hover {
border-right: 4px solid #0b0 !important;
}
#mainnav .bittenfailed, #mainnav .bittenfailed :hover {
border-right: 4px hidden !important;
background-color: #d99;
}
#mainnav .bittenpending, #mainnav .bittenpending :hover {
border-right: 4px hidden !important;
}
.step-toc {
padding: .5em 1em;
margin: 1em 3em 2em 1em;
float: right;
border: 1px outset #ddc;
background: #ffd;
font-size: 85%;
position: relative;
}
.step-toc h4 { font-size: 12px; margin: 0 }
.step-toc ul, .step-toc ol { list-style: none; padding: 0 0 0 1.2em; margin: 0 }
.step-toc li { margin: 0; padding: 0 }
.step-toc .active { background: #ff9; position: relative; }
.step-toc li.failed a { color: #a00; font-weight: bold; }
div.tickLabel { font-size: 10px; }
td.legendLabel { font-size: 10px; }
Bitten-0.6/bitten/htdocs/bitten_build.png 000755 000765 000120 00000000454 10656040024 021004 0 ustar 00simon admin 000000 000000 PNG
IHDR [A sBITO $PLTEϤX|BfffsZ)ZZZ:::9*
Z tRNS P& pHYs ~ !tEXtSoftware Macromedia Fireworks 4.0&'u tEXtCreation Time 07/01/05J 8IDATxc``TRd QPƐ(Eg9HlVTdX 2p ع `a
| IENDB` Bitten-0.6/bitten/htdocs/bitten_buildf.png 000644 000765 000120 00000000441 10656040024 021143 0 ustar 00simon admin 000000 000000 PNG
IHDR a gAMA OX2 tEXtSoftware Adobe ImageReadyqe< 3PLTEZZZsZ)9*
|BϤX:::ii fff_ tRNS %b WIDATx4[DQgW1= `"xQ63NĴ ]TG\FnԻk5X4 "x- IENDB` Bitten-0.6/bitten/htdocs/bitten_coverage.css 000644 000765 000120 00000000262 11234725105 021501 0 ustar 00simon admin 000000 000000 /* Code coverage file annotations */
table.code th.coverage { width: 4em; }
table.code th.covered { background-color: #6F6; }
table.code th.uncovered { background-color: #f66; }
Bitten-0.6/bitten/htdocs/excanvas.js 000644 000765 000120 00000121470 11364623604 020007 0 ustar 00simon admin 000000 000000 // Copyright 2006 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Known Issues:
//
// * Patterns only support repeat.
// * Radial gradient are not implemented. The VML version of these look very
// different from the canvas one.
// * Clipping paths are not implemented.
// * Coordsize. The width and height attribute have higher priority than the
// width and height style values which isn't correct.
// * Painting mode isn't implemented.
// * Canvas width/height should is using content-box by default. IE in
// Quirks mode will draw the canvas using border-box. Either change your
// doctype to HTML5
// (http://www.whatwg.org/specs/web-apps/current-work/#the-doctype)
// or use Box Sizing Behavior from WebFX
// (http://webfx.eae.net/dhtml/boxsizing/boxsizing.html)
// * Non uniform scaling does not correctly scale strokes.
// * Filling very large shapes (above 5000 points) is buggy.
// * Optimize. There is always room for speed improvements.
// Only add this code if we do not already have a canvas implementation
if (!document.createElement('canvas').getContext) {
(function() {
// alias some functions to make (compiled) code shorter
var m = Math;
var mr = m.round;
var ms = m.sin;
var mc = m.cos;
var abs = m.abs;
var sqrt = m.sqrt;
// this is used for sub pixel precision
var Z = 10;
var Z2 = Z / 2;
/**
* This funtion is assigned to the
