In my earlier post, I talked about why you need to start using continuous integration and need a continuous integration server. In this post I will look more at continuous integration best practices when you have a continuous integration server set up.
One of the first things your continuous integration server needs to do is to run and report on your unit tests as part of the build. It is not really a good enough quality checkpoint to make sure that the code simply compiles without errors. Your unit tests should be executed as part of every build. Every time someone checks in code you want to know that unit tests have not broken. Along with this I will include code coverage reports. It is very useful to have your code coverage reported by your continuous integration server for the code coverage metrics on your unit tests.
Static Analysis Tools
One of the best ways to enforce coding standards and quality standards is through the use of static analysis tools. Depending on the language you are using there are several options available. For Java there are tools like PMD and Checkstyle, for .NET there are tools like FxCop and StyleCop. Here is a good list of some of the ones available for each language. There are two basic branches of static analysis tools, ones that check for formatting and convention, and ones that check for bad practices. I would strongly suggest employing both. Your CI server should run the static analysis tools at the end of each build and preferably track the progress of violations introduced in the build. Some CI servers allow you to actually fail the build if an amount or percentage of the code has violations. I would highly recommend this for a new code base. If you end up with a high number of violations the violations just become noise. CI server is a good way to make this visible to everyone on the project.
If deploying your code to any environment requires more than the push of a button, you're doing it wrong. I know this may sound like a bold statement, but why would you need to do anything more than this? One thing I always emphasize is that the only way for a build to get to an environment is from the CI server pushing it there directly. Basically, you should only be deploying builds that are built by the CI server. When you follow this practice you make sure that the code in an environment is the exact bits that you expect. Taking a build from somewhere else is error prone and risky. Doing a build manually is a waste of time and also risky. Your deployment should be a simple button push which takes bits that have already been built on your CI server and deploys them to the desired environment.
Perhaps the most important part of a CI server is notification when a build fails, or when it is fixed. If you get this one wrong, you will be constantly chasing down developers breaking the build and your efforts will be in vain. It is very important that when a build fails it is a big deal, a really BIG DEAL. Some teams use flashing lights, others use large flat-screen TVs, and others just emails, but however you do it, it must be effective and not ignored. I would recommend at a minimum setting up an email notification when the build fails and when it is fixed. When there is a broken build it is very important that fixing the build becomes top priority. One of the things which can help reduce build breaks is to make sure that developers have a way to do the exact same build which will be done by the CI server. If a developer can run the same build locally before checking in their code, there really aren't many good excuses for breaking the build. (This will also require that the build be very fast, in one of my earlier posts I talked about having a dedicated developer tools team to do things like optimizing the build.)
I won't go into extreme detail here about database integration, since the focus of this post is on CI best practices, but I want to make this important point. If your database is not in some way version controlled and built with your source code, there is no point in having the ability to replicate any code build. Basically if you cannot tie a version of the database that is also reproducible to the source code, you cannot actually roll back to a specific point in time. For some projects this is important, for others it is not, but for all it must be considered. Either way you should minimally consider how you will handle database integration with your CI server. One of the solutions I have helped to employ on the project I am working on currently is to have a database build which works very similarly to the source code build and applies database changes as SQL change scripts which are applied in a certain order to build the database.