Whenever I start a new PHP project for a client, one of the first things I set up is usually phpUnderControl - a CI (continuous integration) server specifically for PHP built on top of CruiseControl.
It provides a bunch of stuff out-of-the box, from the essential (broken-build email alerts) to the nice-to-have (code mess-detection, copy/paste-detection) to the quite frankly bewildering (yes I’m looking at you, mystifying code-dependency graph.)
The PHP CodeSniffer integration is worth the price of admission by itself, and if nothing else you can impress your non-tech boss with some nice shiny graphs :
… and a nice row of green ticks to show that your tests are all passing :
I’ve installed this on Ubuntu twice in the last month, and finally got sick enough of figuring-it-out-from-scratch to actually write down some reproducible install steps.
Since CruiseControl is Java-based, we need to install Java first :
> sudo aptitude install sun-java6-jre sun-java6-jdk
You’ll need to agree to some licenses etc while this is installing.
In more recent versions of Ubuntu (10.4 onwards), these packages have been moved to a partner repository, and are no longer part of the default Lucid repositories. If this is the case, you will get some errors similar to the following when you try to install them:
No candidate version found for sun-java6-jre
No candidate version found for sun-java6-jdk
No candidate version found for sun-java6-jre
No candidate version found for sun-java6-jdk
No packages will be installed, upgraded, or removed.
If this is the case, you need to add the following line to the end of your /etc/apt/sources.list:
deb http://archive.canonical.com/ lucid partner
… and then update the source list:
> sudo apt-get update
Next, install some of the required PHP packages
> sudo aptitude install php5-dev php-pear php5-xdebug
phpUnderControl needs the Xdebug PHP extension to work its magic, so make sure that xdebug.ini is present and contains the appropriate config (it should be somewhere like /etc/php5/conf.d/xdebug.ini) :
> more /etc/php5/conf.d/xdebug.ini
zend_extension=/usr/lib/php5/20090626/xdebug.so
xdebug.profiler_enable_trigger = onc
Check the PHP info output to make sure it’s installed correctly
> php -i | grep xdebug
/etc/php5/cli/conf.d/xdebug.ini,
xdebug
xdebug support => enabled
xdebug.auto_trace => Off => Off
...
Next, we need to install CruiseControl itself. We’re going to do this in /opt/.
> cd /opt/
> sudo wget http://heanet.dl.sourceforge.net/sourceforge/cruisecontrol/cruisecontrol-bin-2.8.4.zip
> sudo unzip cruisecontrol-bin-2.8.4.zip
> sudo ln -s cruisecontrol-bin-2.8.4 cruisecontrol
> sudo rm cruisecontrol-bin-2.8.4.zip
We also need to create a start-up script for CruiseControl, since it doesn’t come with one by default. Save the following as /etc/init.d/cruisecontrol :
#!/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:
. /lib/lsb/init-functions
JAVA_HOME=/usr/lib/jvm/java-6-sun
NAME=cruisecontrol
DAEMON=/opt/cruisecontrol/cruisecontrol.sh
PIDFILE=/opt/cruisecontrol/cc.pid
test -x $DAEMON || exit 5
RUNASUSER=www-data
UGID=$(getent passwd $RUNASUSER | cut -f 3,4 -d:) || true
case $1 in
start)
log_daemon_msg "Starting Cruisecontrol server" "cc"
if [ -z "$UGID" ]; then
log_failure_msg "user \"$RUNASUSER\" does not exist"
exit 1
fi
cd /opt/cruisecontrol/
./cruisecontrol.sh > /dev/null 2>&1
log_end_msg $?
;;
stop)
log_daemon_msg "Stopping Cruisecontrol server" "cc"
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
log_end_msg $?
rm -f $PIDFILE
;;
restart|force-reload)
$0 stop && sleep 2 && $0 start
;;
status)
pidofproc -p $PIDFILE $DAEMON >/dev/null
status=$?
if [ $status -eq 0 ]; then
log_success_msg "Cruisecontrol server is running."
else
log_failure_msg "Cruisecontrol server is not running."
fi
exit $status
;;
*)
echo "Usage: $0 {start|stop|restart|force-reload|status}"
exit 2
;;
esac
We need to make this script executable :
> sudo chmod a+x /etc/init.d/cruisecontrol
We also want CruiseControl to start up automatically when the server boots up - use update-rc.d to do this :
> sudo update-rc.d cruisecontrol defaults
I had problems with CruiseControl not knowing where JAVA_HOME was for some reason. This probably points to some kind of Java mis-configuration on my end, but since I don’t use Java for anything else on the build server it’s easier just to fix it by defining JAVA_HOME in the CruiseControl startup script by editing the script itself :
> sudo vi /opt/cruisecontrol/cruisecontrol.sh
.. and adding a line after the initial bash line :
#!/usr/bin/env bash
JAVA_HOME="/usr/lib/jvm/java-6-sun"
At this point, you should be able to start CruiseControl (keeping in mind we haven’t installed phpUnderControl itself yet) :
> /etc/init.d/cruisecontrol start
After a minute or so (it takes a while to start up), you should be able to see the basic CruiseControl page at http://localhost:8080/cruisecontrol/.
The next step is to install phpUnderControl itself. One of the main dependencies here is PHPUnit, and it is here that I usually hit the first hurdle.
By default, Ubuntu installs PEAR version 1.9.0 - this version doesn’t support the current version of PHPUnit, which is required by phpUnderControl. Depending on your system, you may not need to upgrade anything at this point, but on a fresh Ubuntu system I usually get the following error performing the next steps :
phpunit/PHPUnit requires PEAR Installer (version >= 1.9.2), installed version is 1.9.0
To avoid this, we want to make sure we have the latest version of PEAR before we try and install more dependencies :
> sudo pear upgrade pear
Depending on your setup, you may also need to install a newer version of Xdebug. You can check the version you have installed with :
> php -i | grep Xdebug
with Xdebug v2.2.0-dev, Copyright (c) 2002-2011, by Derick Rethans
If you have a version less that 2.0.5, you’re probably going to hit the wall later on in the installation with the error :
phpunit/PHP_CodeCoverage requires PHP extension "xdebug" (version >= 2.0.5), installed version is 2.0.4
If you have a version less than 2.0.5, save yourself some hurt in the next couple of minutes and just go ahead and upgrade it now. Your PHP module directory may differ depending on the Ubuntu version, but should be somewhere under /usr/lib/php5/ :
> cd /tmp
> svn co svn://svn.xdebug.org/svn/xdebug/xdebug/trunk xdebug
> cd xdebug
> phpize
> ./configure --enable-xdebug
> make
> sudo mv /usr/lib/php5/20060613+lfs/xdebug.so /usr/lib/php5/20060613+lfs/xdebug.so.orig
> sudo cp modules/xdebug.so /usr/lib/php5/20060613+lfs/
Now that everything is up-to-date, we want to install some of the PHP dependencies, as well as phpUnderControl itself.
> sudo pear channel-discover components.ez.no
> sudo pear install -a ezc/Graph
> sudo pear config-set preferred_state beta
> sudo pear channel-discover pear.phpunit.de
> sudo pear channel-discover pear.symfony-project.com
> sudo pear channel-discover pear.phpundercontrol.org
> sudo pear install --alldeps --force phpuc/phpUnderControl-beta
> sudo pear channel-discover pear.pdepend.org
> sudo pear install channel://pear.pdepend.org/PHP_Depend-0.9.11
> sudo pear channel-discover pear.phpmd.org
> sudo pear install channel://pear.phpmd.org/PHP_PMD-0.2.4
> sudo pear install --alldeps phpunit/phpcpd
> sudo pear channel-discover pear.phing.info
> sudo pear config-set preferred_state alpha
> sudo pear install --alldeps --force phing/phing
If everything installed correctly, we should now be able to apply the phpUnderControl patches to CruiseControl, which will give us our nice shiny PHP-tailored interface :
> sudo phpuc install /opt/cruisecontrol
> sudo /etc/init.d/cruisecontrol restart
Next we want to remove the example project that ships with CruiseControl, and add our actual PHP project. We’re going to call our new project php-example , and assume we’re checking it out from subversion (although git is do-able too) - you’ll need to adjust the following instructions to fit your own project and checkout procedure.
> cd /opt/cruisecontrol/projects
> sudo rm -Rf connectfour/
> sudo mkdir -p php-example/build/api
> sudo mkdir php-example/build/charts
> sudo mkdir php-example/build/coverage
> sudo mkdir php-example/build/graph
> sudo mkdir php-example/build/logs
> sudo svn co svn://path/to/php-example/trunk php-example/source
At this point you also want to set up your newly checked-out project so that database connections work, unit tests pass etc. Creating a PHPUnit test suite is beyond the scope of this article, but if you don’t have any tests yet, grab the documentation from the PHPUnit page, write and commit a couple of simple tests, then come back here afterwards - we’ll wait for you!
After you are confident that your unit tests are running correctly, your new project will need a build.xml file to tell CruiseControl how to build it. Make an /opt/cruisecontrol/projects/php-example/build.xml file similar to the one below (and make sure you add it to version control too). phpUnderControl will use this file to perform the build process for your project.
Obviously you’ll need to customize this to fit the needs of your own project - make note of the entries with path-to-ignore and path-to-check and adjust them according to your own needs.
<?xml version="1.0" encoding="UTF-8"?>
<project name="php-example" default="build" basedir=".">
<!--
Helper target that initializes the CruiseControl project and creates the base
directory structure.
-->
<target name="init">
<!-- Create the source directory -->
<mkdir dir="source" />
<!-- Create the different build directories -->
<mkdir dir="build" />
<mkdir dir="build/api" />
<mkdir dir="build/charts" />
<mkdir dir="build/coverage" />
<mkdir dir="build/graph" />
<mkdir dir="build/logs" />
<!-- Checkout phpUnderControl trunk into source -->
<exec executable="svn">
<arg line="co /path/to/php-example/trunk source" />
</exec>
</target>
<!--
The clean target is used to remove build artefacts of previous builds. Otherwise
CruiseControl will present old, maybe successful results, even if your build
process fails.
-->
<target name="clean">
<!-- Remove old log files -->
<delete>
<fileset dir="${basedir}/build/logs" includes="**.*" />
</delete>
<!-- Remove old api documentation -->
<delete>
<fileset dir="${basedir}/build/api" includes="**.*" />
</delete>
<!-- Remove old coverage report -->
<delete>
<fileset dir="${basedir}/build/coverage" includes="**.*" />
</delete>
<!-- Remove old graphs -->
<delete>
<fileset dir="${basedir}/build/graph" includes="**.*" />
</delete>
</target>
<!--
The default build target for this project. It simply depends on all sub tasks
that perform the project build. The sub targets are executed in the listed
order.
1. 'clean' Clean old project build artefacts
2. 'checkout' Update project working copy
3. 'php-documentor' Generate api documentation
4. 'php-codesniffer' Check for coding violations.
4. 'pdepend' Code dependency information.
4. 'phpcpd' Copy-paste detection.
4. 'phpcpd' Mess detection.
5. 'phpunit' Execute unit tests, generate metrics, coverage etc.
-->
<target name="build"
depends="clean,checkout,php-documentor,php-codesniffer,pdepend,phpcpd,phpmd,phpunit" />
<!--
Performs an 'svn up' for the working copy.
-->
<target name="checkout">
<exec executable="svn" dir="${basedir}/source">
<arg line="up" />
</exec>
</target>
<!--
Generates the project documentation into the <project>/build/api directory.
phpUnderControl uses the command line output of PhpDocumentor that is logged
by CruiseControl.
-->
<target name="php-documentor" depends="init">
<exec executable="phpdoc" dir="${basedir}/source">
<arg line="-ct type -ue on -t ${basedir}/build/api
-tb /usr/share/php/data/phpUnderControl/data/phpdoc/ -o HTML:Phpuc:phpuc
-d ${basedir}/source
-q
-i paths-to-ignore1,paths-to-ignore2"/>
</exec>
</target>
<!--
Execute code sniffer. You can use a different coding standard if
you want (--standard=Squiz) - check the php-codesniffer docs
for more information.
-->
<target name="php-codesniffer" depends="init">
<exec executable="phpcs"
dir="${basedir}/source"
error="/dev/null"
output="${basedir}/build/logs/checkstyle.xml">
<arg line="--report=checkstyle
--standard=Squiz
--extensions=php
--tab-width=4
-n
--ignore=path-to-ignore1,path-to-ignore2
./path-to-check1 ./path-to-check2"/>
</exec>
</target>
<!--
Calculates dependencies and adds new graphs to the metrics page.
-->
<target name="pdepend" depends="init">
<exec executable="pdepend" dir="${basedir}/source" logerror="on">
<arg line="--summary-xml=${basedir}/build/logs/pdepend.xml
--jdepend-chart=${basedir}/build/charts/jdepend.svg
--overview-pyramid=${basedir}/build/charts/overview-pyramid.svg
--coderank-mode=inheritance,property,method
./path-to-check1,/path-to-check2" />
</exec>
</target>
<!--
Copy-paste detection.
-->
<target name="phpcpd" depends="init">
<exec executable="phpcpd" failonerror="false">
<arg line="--log-pmd ${basedir}/build/logs/pmd-cpd.xml
--exclude ${basedir}/path-to-ignore1
--exclude ${basedir}/path-to-ignore2
${basedir}/path-to-check1 ${basedir}/path-to-check2" />
</exec>
</target>
<!--
Checks that the project source does not contain any 'mess'.
-->
<target name="phpmd" depends="init">
<exec executable="phpmd" failonerror="false">
<arg line="${basedir}/source
xml
codesize,unusedcode,naming
--reportfile ${basedir}/build/logs/pmd.xml
--exclude path-to-ignore1,path-to-ignore2" />
</exec>
</target>
<!--
Executes the project unit tests and stores the different logs in the
<project>/build/logs directory. Furthermore it generates the coverage report
under <project>/build/coverage.
-->
<target name="phpunit" depends="init">
<exec executable="phpunit" dir="${basedir}/source" failonerror="on">
<arg line="--log-junit ${basedir}/build/logs/phpunit.xml
--coverage-clover ${basedir}/build/logs/phpunit.coverage.xml
--coverage-html ${basedir}/build/coverage
--configuration ${basedir}/tests/phpunit.xml
${basedir}/path-to-check"/>
</exec>
</target>
</project>
Now that our project has a build.xml file, we need to tell CruiseControl where it is. Edit the main config file ( /opt/cruisecontrol/config.xml ) and enter something similar to the following :
<?xml version="1.0"?>
<cruisecontrol>
<project name="php-example" buildafterfailed="false">
<plugin name="svnbootstrapper"
classname="net.sourceforge.cruisecontrol.bootstrappers.SVNBootstrapper" />
<plugin name="svn"
classname="net.sourceforge.cruisecontrol.sourcecontrols.SVN" />
<listeners>
<currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
</listeners>
<modificationset>
<svn localWorkingCopy="projects/${project.name}/source/"/>
</modificationset>
<bootstrappers>
<svnbootstrapper localWorkingCopy="projects/${project.name}/source/" />
</bootstrappers>
<schedule interval="60">
<ant anthome="apache-ant-1.7.0"
buildfile="projects/${project.name}/build.xml"/>
</schedule>
<log dir="logs/${project.name}">
<merge dir="projects/${project.name}/build/logs/"/>
</log>
<publishers>
<!--
Copies the generated api documentation into project artifacts directory.
-->
<artifactspublisher dir="projects/${project.name}/build/api"
dest="artifacts/${project.name}" subdirectory="api"/>
<!--
Copies the generated code coverage report into project artifacts directory.
-->
<artifactspublisher dir="projects/${project.name}/build/coverage"
dest="artifacts/${project.name}"
subdirectory="coverage"/>
<!--
Generates the different metric charts with the phpUnderControl ezcGraph
extension.
-->
<execute command="phpuc graph logs/${project.name} artifacts/${project.name}"/>
<!--
Sends simple text emails after a project build. For nicer html emails,
checkout the original CruiseControl documentation.
* http://cruisecontrol.sourceforge.net/main/configxml.html#email
* http://cruisecontrol.sourceforge.net/main/configxml.html#htmlemail
-->
<email mailhost="smtp.localhost"
username="username"
password="password"
returnaddress="build@php-example.com"
buildresultsurl="http://build.php-example.com/buildresults/${project.name}"
skipusers="true"
spamwhilebroken="true">
<always address="build@php-example.com"/>
<failure address="build@php-example.com"/>
</email>
</publishers>
</project>
</cruisecontrol>
Note that the project name you enter (php-example) must match the name of the folder you checked your source files out to earlier.
Finally, restart CruiseControl and your build server should be ready to go!
> sudo /etc/init.d/cruisecontrol restart
Using phpUnderControl
You should now be able to view phpUnderControl at http://localhost:8080/cruisecontrol/ :
If you add other projects in /opt/cruisecontrol/config.xml , they will show up on this screen as well. Click the green tick button to force a build, or click the project name to view the build results for that project.
CruiseControl will check subversion for updates every minute or so and perform a build automatically if it finds any, so you shouldn’t need to force a build too often. If you entered your email details in config.xml , you should also get an email if the build fails for any reason.
Null pointer exceptions viewing the metrics page
With more recent versions of phpUnderControl , I’ve been getting a java.lang.NullPointerException whenever i try and access the metrics page :
java.lang.NullPointerException
at net.sourceforge.cruisecontrol.chart.PieChartData.produceDataset(PieChartData.java:52)
at de.laures.cewolf.taglib.DataContainer.getDataset(DataContainer.java:53)
at de.laures.cewolf.taglib.SimpleChartDefinition.getDataset(SimpleChartDefinition.java:34)
at de.laures.cewolf.taglib.SimpleChartDefinition.produceChart(SimpleChartDefinition.java:30)
at de.laures.cewolf.taglib.AbstractChartDefinition.getChart(AbstractChartDefinition.java:81)
at de.laures.cewolf.taglib.ChartImageDefinition.ensureRendered(ChartImageDefinition.java:131)
at de.laures.cewolf.taglib.ChartImageDefinition.getBytes(ChartImageDefinition.java:125)
at de.laures.cewolf.storage.SerializableChartImage.(SerializableChartImage.java:51)
at de.laures.cewolf.storage.SessionStorage.storeChartImage(SessionStorage.java:57)
at de.laures.cewolf.storage.SessionStorage.storeChartImage(SessionStorage.java:35)
at de.laures.cewolf.taglib.tags.ChartImgTag.doStartTag(ChartImgTag.java:74)
at org.apache.jsp.main_jsp._jspx_meth_cewolf_img_0(org.apache.jsp.main_jsp:1781)
at org.apache.jsp.main_jsp._jspService(org.apache.jsp.main_jsp:695)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:93)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:373)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:470)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:364)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:829)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)
It took me hours to get to the bottom of this because while it looks like a java problem, the actual problem is in PHP. The ClassComplexityInput.php class (used for generating the complexity graph) has an undefined constant or something in it (I forget the exact cause), which causes the graph not to be rendered, which in turn causes CruiseControl to fall over when it tries to render the metrics page.
The quick-and-dirty solution to this is simply to move the file somewhere else :)
> cd /usr/share/php/phpUnderControl/Graph/Input/
> sudo mv ClassComplexityInput.php ClassComplexityInput.php.bak
You lose the code dependency graph by doing this, but until I have some time to dig a bit deeper and put a patch together it’s good enough for me.