Build Process Metrics - What Should I Capture?

The short answer is everything!

When I talk about build process metrics I'm not talking about code coverage, or lines of code - there's a plethora of tools that will extract that information for you such as cobertura, findbugs etc. What I mean is information about the build process itself.

Here's just an example of a few metrics I find useful:

Current time is Dec 28, 2013 10:41:38 AM
System.getProperty('os.name') == 'Windows 7'
System.getProperty('os.version') == '6.1'
System.getProperty('os.arch') == 'amd64'
System.getProperty('java.version') == '1.7.0_45'
System.getProperty('java.vendor') == 'Oracle Corporation'
System.getProperty('sun.arch.data.model') == '64'

You might be asking why this is important, or why bother when you're using a build automation tool such as Jenkins. Unfortunately for me not every project is fully automated. It should be, but we're not quite there yet. I like to use Gradle as it provides the tools to extract a lot of information about your build process.  As this is performed at a lower level than CI this means for those odd tasks that are run manually we don't miss out on that juicy data. Data is valuable, just ask Google.....

So, what do we hope to find? 

The interesting thing about data is you don't really know what's there until you start digging. One such example presented itself recently ... at my company we split the task of releasing software into 2 stages. First, we have the volatile build phase, followed by the stable release phase which is performed by a different team. During one projects build phase I was repeatedly running deployments of our web application in approximately 3 minutes to our QA environments. However, when the release manager was deploying into production it was taking 45 minutes..... what the deuce??

Before even looking at the data I knew we both used the same work issued laptop, with the same version of Java, and the Gradle wrapper ensured we were using the same version of the build tool. So, what was causing this massive increase in deployment time? Well, I later found out from the data that the release manager had failed to mention he was running the deployment from his bedroom across a VPN ..... Bingo! the increased deployment time was due to the network, and was easy to fix. (we quickly put this project in Jenkins with a manual start).

The moral of the story is to collect all the information you can. Gradle is an extremely powerful DSL, and its groovy support means your options for pushing and processing the data are endless.

Sonar - UI for best user experience

Sonar has one of the most boring user interfaces I think I've ever seen -- it's butt ugly! but, as ugly as it is there's no denying how useful it is.

 

The image above shows an extract from a publicly shared project. I really liked the layout that they used so I took a few of their ideas and added some of my own.

A dashboard should be simple, provide enough information about the project ,but not overwhelm the user. If your customers or stakeholders have access to this they may not be technically minded and so this page should show info that anyone can understand.

Dashboard

To change the dashboard, log in as the admin user and on the right of the page select "Edit Filter":

You should be presented with some new options:

The column headings that I have selected are:

• Lines of code
• Rules compliance
• Unit test success (%)
• Coverage
• Complexity /method
• Complexity /class
• Public documented API (%)
• Duplicated lines (%)
• Total Useless Code (plugin)
• Build date

To add these, select the center dropdown in the Add Column row of the Display window.

Project Layout

When you select a project from the name column on the dashboard you will be presented with a page like the following:

Now this is where you can geek out and go crazy with metrics. Hopefully, by this point the non-techies have seen what they wanted and left. This page is where you can get creative.

I hope i'm not generalising too much but I would like to think most technically minded people are working with widescreen monitors for maximum productivity. If this is true I would strongly advise going for a three column layout. This can be done by selecting "Edit Layout" on the right and choosing the tri-colum option.

Finally, to select the widgets you want to display press "Configure widgets" and then you can drag and drop them at your desired locations.