Name: 
Why pay for a Continuous Integration server?
With plenty of good, free alternatives, such as Continuum and CruiseControl, you’d need a compelling reason to spend money on per-user licensing in order to use JetBrains’ new TeamCity product.
JetBrains’ legendary reputation for productivity-boosting, easy-to-use tools has taken a bit of dent recently (certainly within our office) thanks to the rather buggy 6.0 releases of their flagship product, IntelliJ IDEA. Regardless, they have managed to attract a legion of devoted followers eager to pay $500 each to avoid having to use Eclipse.
But can they repeat the trick for Continuous Integration? Continuum does the job with minimal fuss and is admirably simple to configure, so why pay more?
TeamCity has several nice-to-have plus points that put it ahead of the free competition, but not to the extent of $399 per user (or even the current offer price of $199 per user). Briefly, these plus points are:
- Support for .Net projects, as well as Java, in the same product (nice if you need it).
- Server-side code coverage analysis (you could get the same results by running EMMA from the Ant build.xml)
- Server-side static code analysis using IDEA inspections (nice but relies on using IDEA for development - Checkstyle and FindBugs could do something similar from Ant).
- Flashy AJAX web console (perhaps too flashy)
- Pre-tested commits. Sends your changes to the CI server for building before committing to version control. Your changes are only checked-in if the build succeeds and all tests pass.
So if like me you got a free TeamCity licence with your IDEA 6.0 purchase, there is enough in it to make it worthwhile. However, unless your entire team uses IDEA and have free TeamCity licences, you may find it a little restrictive compared to the free alternatives. You can enable guest access to allow anybody to view your builds and download artifacts, but only licensed users can control builds and receive notifications when builds fail.
This review could end here with a lukewarm thumbs-up and a recommendation to use the product if you got it for free but not to spend any money on it. But to do so would be to ignore the crucial architectural difference between TeamCity and its free competitors. The TeamCity website alludes to this key advantage without fully explaining the implications.
Continuum, CruiseControl and the like share a common approach. The web application running inside the servlet container checks out the code from version control and builds it locally (on the web server machine). In TeamCity, the web application is merely the coordinator, building happens elsewhere. TeamCity has a grid of Build Agents to which the work is delegated. You may choose to install the Build Agent on the Team City server, and/or you may decide to have one or more other machines that do the building.
The advantages of this approach were not initially obvious to me, beyond the ability to share large workloads across several nodes. The important point is that the build grid does not have to be homogenous. Build agents can be installed on different platforms with different resources available to them.
To understand the possibilities that this provides, allow me to introduce you to my current project. The server is a J2EE application but has a mission-critical native component written in C and invoked via JNI. The target platform is Solaris 10 on Sparc hardware, but developer desktops are x86 Windows XP machines. When running on Windows, the native component is replaced by a Java stub (the Ant script detects the platform and skips building the native component). In addition, we have two native client applications, one for Win32 and one for Mac OS X.
In TeamCity you can have multiple configurations per project. Each configuration can express preferences about the build agent used to execute it. In the above example we can have a server configuration that will only be built by a Solaris build agent and two client configurations, one for Windows and one for the Mac. Assuming we have at least one build agent available to satisfy each configuration, this is where the magic happens. A change is checked into Subversion. TeamCity detects that the repository has been updated and springs into life. A Solaris machine is instructed to check out the project and run the server build. Simultaneously, a Mac is prompted to build the Mac client and a Windows box gets to work on the Windows client. After x minutes, assuming there are no errors, the built and tested server and clients are available to download from the TeamCity web console. Without any human intervention we’ve built and tested across three different platforms.
But it doesn’t end there. How do we develop the Solaris-specific parts of the server on Windows desktops? There are two options. The first is to make the changes on Windows, check them into Subversion, logon to the Solaris machine, check out and test the updates and then go back to Windows to fix the bugs. This has the potential to break other people’s builds if they do an update before we have finished.
The alternative is to shorten the write-compile-debug cycle by doing the actual development on the Solaris machine using a remote session. The drawback of this approach is that using a heavyweight IDE in an X-session on the remote server may not be feasible, particularly if there are other users vying for resources.
Fortunately, TeamCity’s build grid gives us another option. Using the TeamCity IDEA plugin (plugins for Eclipse, NetBeans and Visual Studio are apparently coming soon), we can develop on Windows and submit a remote build to the grid. Without anything being committed to Subversion, a Solaris agent will do the build and report the results. When we have the final version ready to check-in, we can do a pre-tested commit as described earlier. This will make sure that the build is successful on the target platform before committing to version control.
TeamCity has its rough edges (see Hani’s review of version 1.0), but it has raised the bar for Continuous Integration functionality. If you’re going to pay money for it though, wait for version 1.2 as there are still a few bugs that might cause frustration.
on May 14th, 2007 at 1:39 pm
You pay if you see value. There certainly are tools out there that can provide the value.
TeamCity, though, has some _very_ rough edges. I draw the line at a tool that can neither tell me exactly what the compile error was, or what unit test failed, or give a meaningful view of the running build log OR even use a released version of Maven to do the building! Furthermore, the agent structure, while nice on paper, has some problems - imagine a “staggered” build process where you do one phase of the build, which triggers the next, and so forth; the agent structure makes it difficult for you to have all the builds on one box BUT have the first build kick off on any available agent. These staggered builds are useful for getting good feedback during longer builds (not to mention taking better advantage of a grunty build server), but doing it under TeamCity negates the point of the build grid in the first place.
TeamCity is also desperately in need of some sort of version control on the configuration, as well. One of the best aspects of CruiseControl is that you can place the build configuration in version control - making it easy to keep track of what is happening to the build environment. It’s very annoying (speaking from experience) to spend an hour or two diagnosing a broken build only to realise in the end that someone had changed the config options.
There’s also a hidden catch in the TeamCity licensing to watch out for - if you decide (for whatever reason) you want to have multiple TeamCity central servers, you’ll find that the licensing is per user per server; you can’t have the same user/license on multiple servers. Depending on how many builds you want to manage (and how you want to manage them), this may be an issue.
Finally - that “remote build” option can also introduce the sort of problems about flow control that it’s meant to prevent. You kick of a remote build, you get involved in some other work, and then you notice a bit later the remote build failed. Well, now you’ve got to fix it - which might just mean you need to get your environment back to the state it was when you kicked off the remote build (IntelliJ gives you no easy way to do this, let alone to get you back to your work state again AFTER you fix the bug). If the build took a long time to get to a breaking point (and if you’ve got fast builds, you don’t need this feature!) you can easily lose an hour or more of work because of a silly broken build. By contrast, if it was committed, the worst case would be you check out the code in a new work area and fix it there.
(BTW, with Subversion at least, there’s an easy way to avoid the problem of checking out broken code - do ’svn up -r )
Oh, and this was all with version 1.2.
It’s not that TeamCity doesn’t have some nice features, it’s just that it’s very rough. Compare it with Bamboo - similar price for a 10-person team (ignoring the IntelliJ discount), but Bamboo is a lot smoother. Not as many features, but the features they do have are a lot better thought out, and much better implemented.
(BTW - I’d happily pay $500 to not have to use IntelliJ - give me Eclipse any day. And yes, I’ve got access to both, so it’s not just a case of sour grapes. IntelliJ is just clunky to use)
on May 14th, 2007 at 6:55 pm
It’s certainly not perfect, but I was impressed by its potential. This posting is from 5 months ago (it’s just resurfaced because I moved sites). I think Bamboo had not been released at the time. I haven’t had a chance to try it, and I doubt I will any time soon, simply because I’d have to pay for a licence. The advantage of TeamCity is that it was free. JetBrains also provided a free upgrade to 2.0, which is a definite step forward on the 1.x series.
One of the original responses to this posting (unfortunately lost in the migration), suggested Parabuilder as an alternative.
on May 9th, 2008 at 8:55 pm
[...] IDEA 6.0 and opened my eyes to a world beyond the fairly basic functionality of Continuum. I was impressed (pre-tested commits are a neat feature), but because you needed licences for every user, I was [...]