Buildbot Part II – Running automated build and tests

In my previous post we setup buildbot. Now, let’s configure it to run automated builds and tests. To do this, I’ll be using my xUnit.Net fork hosted on git/codeplex. Make the following changes to master.cfg:

Connect to the buildslave

c['slaves'] = [BuildSlave("BuildSlave01", "mysecretpwd")]

Track repository – GitPoller

The gitpoller periodically fetches from a remote git repository and process any changes.

from buildbot.changes.gitpoller import GitPoller
c['change_source'] = []
c['change_source'].append(GitPoller(
        'https://git01.codeplex.com/forks/mariangemarcano/xunit',
        gitbin='C:\\Program Files (x86)\\Git\\bin\\git.exe',
        workdir='gitpoller_work',
        pollinterval=300))

Add scheduler

There are different types of schedulers supported by buildbot and you can customize it. Let’s add a single branch scheduler to run the build whenever a change is committed to the repository and knightly schedulers to run the build once a day.

from buildbot.schedulers.basic import SingleBranchScheduler
from buildbot.schedulers.timed import Nightly
from buildbot.changes import filter

c['schedulers'] = []
c['schedulers'].append(SingleBranchScheduler(
                            name="Continuous",
                            change_filter=filter.ChangeFilter(branch='master'),
                            treeStableTimer=3*60,
                            builderNames=["Continuous"]))
c['schedulers'].append(Nightly(
                            name="Ninghtly",
                            branch='master',
                            hour=3,
                            minute=30,
                            onlyIfChanged=True,
                            builderNames=["Ninghtly"]))

Define a Build Factory

It contains the steps that a build will follow. In our case, we need to get source from repository, compile xunit.net projects and run the tests.

Get source from repository: Let’s do an incremental update for continuous build and compile debug; the full build will do a full repository checkout and compile using release configuration. The git factory needs to access git binaries, set the environment path to where you install it for example: C:\Program Files (x86)\Git\bin\.

Compile xunit.net and run tests: Let’s compile using MSbuild.exe and run the tests with xunit.console.exe. By setting flunkOnFailure=False/warnOnFailure=True, test failures will mark the overall build as having warnings.

from buildbot.process.factory import BuildFactory
from buildbot.steps.source.git import Git
from buildbot.steps.shell import ShellCommand

def MSBuildFramework4(factory, configuration, projectdir):
    factory.addStep(ShellCommand(command=['%windir%\\Microsoft.NET\\Framework\\v4.0.30319\\MSbuild.exe',
                                          'xunit.msbuild', '/t:Build', '/p:Configuration=%s' % configuration],
                                 workdir=projectdir, name="Compile xunit.net projects",
                                 description="Compiling xunit.net projects",
                                 descriptionDone="Copile xunit.net projects done"))

def RunTests(factory, testproject, configuration, testdirectory):
        factory.addStep(ShellCommand(command=['src\\xunit.console\\bin\\%s\\xunit.console.exe' % configuration,
                                '%s\\%s' % (testdirectory, testproject), "/html", 
                                "%s\\testresults.html" % testdirectory],
                                workdir='build\\', name="Run %s" % testproject,
                                description="Running %s" % testproject ,
                                descriptionDone="Run %s done" % testproject,
                                flunkOnFailure=False, warnOnFailure=True))

factory_continuous = BuildFactory()
factory_continuous.addStep(Git(repourl='https://git01.codeplex.com/forks/mariangemarcano/xunit',  mode='incremental'))
MSBuildFramework4(factory_continuous, 'Debug', 'build\\')
RunTests(factory_continuous, 'test.xunit.dll', 'Debug', 'test\\test.xunit\\bin\\Debug')
RunTests(factory_continuous, 'test.xunit.gui.dll', 'Debug', 'test\\test.xunit.gui\\bin\\Debug')

factory_ninghtly = BuildFactory()
factory_ninghtly.addStep(Git(repourl='https://git01.codeplex.com/forks/mariangemarcano/xunit', mode='full', method='clobber'))
MSBuildFramework4(factory_ninghtly, 'Release', 'build\\')
RunTests(factory_ninghtly, 'test.xunit.dll', 'Release', 'test\\test.xunit\\bin\\Release')
RunTests(factory_ninghtly, 'test.xunit.gui.dll', 'Release', 'test\\test.xunit.gui\\bin\\Release')

Builder Configuration

from buildbot.config import BuilderConfig

c['builders'] = []
c['builders'].append(
    BuilderConfig(name="Continuous",
      slavenames=["BuildSlave01"],
      factory=factory_continuous))
c['builders'].append(
    BuilderConfig(name="Ninghtly",
      slavenames=["BuildSlave01"],
      factory=factory_ninghtly))

Status notifications

You could also add MailNotifier to configure mail notifications.

c['status'] = []

from buildbot.status import html
from buildbot.status.web import authz, auth

authz_cfg=authz.Authz(
    # change any of these to True to enable; see the manual for more
    # options
    gracefulShutdown = False,
    forceBuild = True, # use this to test your slave once it is set up
    forceAllBuilds = False,
    pingBuilder = True,
    stopBuild = True,
    stopAllBuilds = False,
    cancelPendingBuild = True,
)
c['status'].append(html.WebStatus(http_port=8010, authz=authz_cfg))

Every commit done will trigger a continuous build that automatically compiles and runs tests. In the waterfall page you can see the people who has committed changes to the branch (view image bellow).

Click on the person’s name to view commit’s detail information like repository, branch and revision. At the bottom there is a list of changed files. This is very helpful to track down any failures (view image bellow).

On my next blog post I’ll show you how to add customizations.