Deploying all day - Sean O'Connor

I recently watched a technical presentation from Sean O'Connor of bitly where he discussed how they achieved up to 20 deployments a day. Whilst much of their techniques are publicized and well known - one of the interesting points he raised was how they only have a single source repository. I don't think this is new; I've heard rumours that Google do the same, and have done for years.

My eyes always light up when I hear these types of stories. They're bold and clever, but I only wish I could do the same......Unfortunately, for me my business is insurance, and not technology - so getting a project manager to accept additional costs that don't immediately result in cash is difficult.

So, why do I think this is a good move?

Its much easier to track and baseline your systems when all of the source is in a single location. Modern version control systems offer a lot of flexibility in this area, and provided you have a bunch of well disciplined developers - you wont have to worry about things like commit messages.

I do still see problems.

My biggest concern would be that unless you start doing some clever tracking on sections of your source tree instead of the tree as a whole, you could be constantly hammering your CI resulting in queues. This could potentially cause long feedback cycles which is an anti-pattern of continuous delivery.

Quote of the Day: Good plan vs Perfect plan

"A good plan today is better than a perfect plan tomorrow" - General Paton

This is an interesting quote, and one I feel can also be dangerous.

We all know that in technology time is money. The longer we spend trying to make the perfect plan, the more money we stand to loose. That being said, a plan that has not been thought out well or had the best people involved can equally be as costly. All too often I've seen management take a path of least resistance only for it to turn around and bite them in the ass.

Quote of the Day: Being Careful

"Quite frankly, I'd rather weed out the people who don't start being careful early rather than late. That sounds callous,and by God, it is callous. But it's not the kind of "if you can't stand the heat, get out of the kitchen" kind of remark that some people take it for. No, it's something much more deeper: I'd rather not work with people who aren't careful. It's Darwinism in software development."
-Linus Torvalds on kernel debuggers, Linux Kernel Mailing list

I'm a huge fan of Linus, but this solidifies his position as one of the greats in my mind. There's no excuse for not being careful and I believe its every software engineers responsibility to ensure the work they have done is to a high standard. Unfortunately, I've witnessed a workplace where the strong carry the weak, and its not good. The weak are not challenged and so become unhappy, and the strong get overloaded and become unhappy. I believe its every managers responsibility to kill this kind of behaviour early and without fail.

Build Process Metrics - What Should I Capture?

The short answer is everything!

When I talk about build process metrics I'm not talking about code coverage, or lines of code - there's a plethora of tools that will extract that information for you such as cobertura, findbugs etc. What I mean is information about the build process itself.

Here's just an example of a few metrics I find useful:

Current time is Dec 28, 2013 10:41:38 AM
System.getProperty('os.name') == 'Windows 7'
System.getProperty('os.version') == '6.1'
System.getProperty('os.arch') == 'amd64'
System.getProperty('java.version') == '1.7.0_45'
System.getProperty('java.vendor') == 'Oracle Corporation'
System.getProperty('sun.arch.data.model') == '64'

You might be asking why this is important, or why bother when you're using a build automation tool such as Jenkins. Unfortunately for me not every project is fully automated. It should be, but we're not quite there yet. I like to use Gradle as it provides the tools to extract a lot of information about your build process.  As this is performed at a lower level than CI this means for those odd tasks that are run manually we don't miss out on that juicy data. Data is valuable, just ask Google.....

So, what do we hope to find? 

The interesting thing about data is you don't really know what's there until you start digging. One such example presented itself recently ... at my company we split the task of releasing software into 2 stages. First, we have the volatile build phase, followed by the stable release phase which is performed by a different team. During one projects build phase I was repeatedly running deployments of our web application in approximately 3 minutes to our QA environments. However, when the release manager was deploying into production it was taking 45 minutes..... what the deuce??

Before even looking at the data I knew we both used the same work issued laptop, with the same version of Java, and the Gradle wrapper ensured we were using the same version of the build tool. So, what was causing this massive increase in deployment time? Well, I later found out from the data that the release manager had failed to mention he was running the deployment from his bedroom across a VPN ..... Bingo! the increased deployment time was due to the network, and was easy to fix. (we quickly put this project in Jenkins with a manual start).

The moral of the story is to collect all the information you can. Gradle is an extremely powerful DSL, and its groovy support means your options for pushing and processing the data are endless.

A Groovy Continuous Integration (Jenkins) Dashboard

I took inspiration from this post, but I wanted to use Groovy. I know there are plugins that do the same, but this was intended to be a learning exercise.

Whilst the example in the link is quite simple - I also need to address authentication of the user. Once available this would be used for displaying build status via a wall mounted monitor.

Requires "commons-codec-1.8.jar" and "commons-httpclient-3.1.jar" on the classpath

Windows 7 sometimes breaks FTP connections on Java 7 if firewall is enabled

Command netsh advfirewall set global StatefulFTP disable usually fixes the problem.

Gradle, Jenkins and ${project.name}

Problem

With Gradle the project name is derived from the project's directory name. However, in Jenkins the project's directory name is derived from the Jenkins job name.

There are a few steps in the build lifecycle that use the project.name, such as the jar task. If our Jenkins job was "myProject_build" this would result in a jar named myProject_build-1.0.jar.

Solution

With a single project the solution is to add a settings.gradle and set the (root) project name there (rootProject.name = "...").

With a multi-project build you may wish to override the sub-project name: