Continuous integration typically relies on an automated build and test system, with a central build server that verifies the code repository on a regular basis by running a predefined set of tests. However, it's often a good idea to supplement these tests, with pre-commit testing. At the very least, we'd like to see every developer run general regression tests before he or she commits.
The practice of pre-commit testing is now common enough that there are a number of test frameworks, e.g. Selenium, that accommodate this practice well.
In this blog post, I'll show you a simple WebKit-based application which executes and runs the Jasmine-based SpecRunner tool headlessly. Since it runs completely from the command line, I'll show you how to combine this little tool with a git pre-commit hook to prevent broken code from being checked in.
Note: I'm going to use QtWebKit so that all popular platforms (Windows, Mac, Linux) are covered. If you want to follow along, you'll need Qt libraries version 4.7 or later before you can continue. (If you're on Linux, there's no need to download the libraries, almost every distribution offers a ready-to-use binary package, usually under the name libqt4-devel or libqt4-dev.)
Let’s start by checking out the code from the Sencha Labs github, which is as easy as:
git clone git://github.com/senchalabs/examples.git
To compile the program, just run:
qmake && make
(On Mac, you might need to run qmake -spec macx-g++ instead).
Alternatively, you could open specrunner.pro file from an IDE, e.g. Qt Creator and compile it from there.
If everthing goes well, you should have a specrunner executable (on Mac OS X, it should appear under specrunner.app/Contents/MacOS subdirectory).
(Tip: copy the specrunner executable to your PATH so it is easily accessible.)
And now do a trial run by entering:
If you've installed it correctly, you'll get a simple response, e.g.
1661 specs, 0 failures in 2.71s
If something has gone wrong then you'll get something like:
FAIL: 1661 specs, 1 failure in 2.45s
Combining specrunner with Git pre-commit hook
Git, a popular version control system, has powerful support for hooks, i.e. scripts which get executed before or after certain actions. One of them is pre-commit hook, executed right before a git commit. The pre-commit hook is useful for integrated testing because git will not continue with the commit if the pre-commit hook returns non-zero. Let's go through an example to see how we can combine this with the above specrunner tool.
First create a dummy repository:
Now, create a simple Jasmine-based SpecRunner.html with the following content:
Next -- if you don't have it already -- download Jasmine and place it under the lib subdirectory.
The test above is very simple: it's a function which increments a value. It should pass, in fact you can verify it by running our headless specrunner tool:
1 spec, 0 failures in 0.011s
Since this passes, commit the code by entering:
git add *
git commit -a -m import
In order to create a pre-commit hook, create a file .git/hooks/pre-commit with the following content:
This runs Specrunner and reuses its return value, which will be passed to git when it executes this commit hook. Now, make sure that the pre-commit is executable:
chmod +x .git/hooks/pre-commit
And here comes the fun part. Let's try to break the tests, for example by changing foo++ in SpecRunner.html with foo += 2. If you do that and try to commit, you'll get the following error:
git commit -a -m "Test"
FAIL: 1 spec, 1 failure in 0.013s
should increment a variable
Expected 2 to equal 1.
The failure message comes from Specrunner tool. Because the return value is non-zero, the commit will fail. You can check this by running git diff: your repository is still dirty.
The technique outlined above is not a substitute for a comprehensive, multiple-browser test system. However, this example shows one good purpose: a very fast sanity check to prevent a potential regression quietly sneaking into the code repository. Feel free to extend and customize this example to suit your needs and pass it along!