Continuous Integration - the devs checklist

As I work through the book Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation (Addison-Wesley Signature) I find myself highlighting sections that I feel are extremely useful and important. I dont claim to know everything, I've only been in this field for 2 years and only worked for 1 company. My experience is limited but this book has proven to be a valuable resource - there are bits i disagree with and I will go through those as I progress through the book.

Here's a section the author discusses. When a developer is ready to submit a piece of work:

1. Before submitting any changes a developer should check to see if a build is currently in the "Successful" status. If not, a developer should assist in fixing a build before submitting new code.
2. If the status is currently "Successful", a developer should rebase his/her personal workspace to this configuration.
3. Build and test locally to ensure update doesn't break developers functionality.
4. If Successful, check in new code.
5. Allow CI to complete with new changes.
6. If build fails, stop and fix on developers machine. Return to step 3.
7. If build passes, continue to next work item.

Build & Release - Rules for Managing development tools

I've just started reading Software Build Systems: Principles and Experience by Peter Smith PhD and one of the chapters discusses development tools and how to best manage them. This is only a small part of the book, but it's one of my favourites because it's easy to implement and the benefits are huge.


#1 Take Notes
I think this is one of the most important things to do when testing tools or running a proof of concept. It's so easy to get carried away with simply getting something to work that you actually forget the steps you took to get there. I would recommend getting yourself a little notepad and jotting down some bullet points - this can be fleshed out later. I personally like to open a text editor like notepad and put everything in there. Mainly because its easier to copy and paste file paths than jott them down on paper.

#2 Use Version Control for Source Control
This rule relates to tools you use that need to be compiled for different targets. You shouldn't always trust that your files will be available online (if thats your source). By keeping your tools in version control you can always rest assured that if anything catastrophic ever happens to your source, or even if they just remove them, you will always have them.

#3 Periodically Upgrage Tools
I'm going to aproach this rule quite differently to the book. I think there has to be a happy medium here; you need to upgrade your tools regularly to stay inside support, but also do it in a controlled manner and not as soon as a new release is made available.

A short time ago I was caught out upgrading one of the plugins on our continuous integration tool. To cut a long story short the latest version of one plugin was faulty and braught down our whole CI system. At the time I done a big bang upgrade of the plugins the CI used and it was a nightmare to track down the root cause and rectify. The moral of this is time the upgrades well, give new releases time for error's to manifest, but upgrade so you get new features and bugfixes.

#4 Version Binaries
I think this should be an extension of rule #2 because the reasons are pretty much the same. Just to reiterate though, keep the tools you use versioned and in your repository so that they are always available!

Weblogic/WLST - Packing and Unpacking a Domain

When you start working with distributed domains there will come a time when you need to pack the domain and unpack it in its distributed areas.

Whether you create your domain via the GUI or by scripting, all you're actually doing is creating a series of configuration files. At this point you're not actually starting any servers - that comes later.

Lets consider the following architecture:

AdminServer = Machine A
Managed01   = Machine B
Managed02   = Machine C
Cluster01      = Managed01, Managed02


So, you run through the wizard and configured the domain above. You should now notice your domain has been created on Machine A, but if you log into Machine B or C nothing exists. This is where the need to Pack and UnPack comes in.

To pack the domain run the following WLST script:



This script opens the domain and extracts (as a jar) the configurations required for the servers that will reside on Machines B and C. It's a skeleton configuration because the Admin server information will be excluded - a domain only ever has 1 Admin server.

Now that we have a templateName.jar we can send it to the machines that the rest of the domain will reside on and run the unpack script on each machine:




Replace the dummy properties with the ones you want for your domain and the script will do the rest. Do this for each machine then you're done. Ensure you have a Nodemanager configured on each machine then start the admin server and administer the domain as per usual.

Weblogic/WLST - Remote Control of a Domain

One of the most frustrating parts of my job is the administration of a Weblogic domain when it resides on a Linux machine. A numpty has decided that my team isn't allowed to have any permissions on Linux boxes and so we cant gain access to our Weblogic files - this means we cant stop and start our domains using startWeblogic.sh.

One solution I found was to Administer the domain via the Nodemanager.

To start, any server that you wish to administer via the Nodemanager has to be associated with a machine and a nodemanager needs to be running. Once done, start a WLST session.

 nmConnect('username','password','wl.host','nm.port','domain.name','domain.dir','socket.type')  

The command above will allow you to connect to the nodemanager.

 nmStart('server.name')  

Start the server.

 nmServerStatus('server.name')  

Get server status.

 nmKill('server.name')  

Force shutdown.

 nmDisconnect()  

Disconnect from the nodemanager.

Development - Set environment script

To run many of the tools you find in Java development you first have to ensure the environment you're working in is configured correctly - this is often the command line for Windows or shell for Linux. For example when you want to run the 'javac' program from the command line, your computer needs to knows the path to that executable file.

**NOTE** it is possible to make environment variables permanent on a machine so that they only ever need to be set once. This can be dangerous for several reasons:

a) If you set a faulty path you could crash your machine and the change may be irreversible.
b) If you switch machine you will have to try and remember all the changes you made.

Here enters the setenv script!!


This is a very simple script to set up the Windows command line interface to use Java. Copy this into your favourite text editor and save it as setenv.bat. Now, edit the script to include your Java install then open a command line prompt. Once open type "set" and run. A series of value/variables pairs should be displayed but JAVA_HOME should not be set.

Now, run "setenv.bat" and "set". You should now notice both the PATH and JAVA_HOME variables have been set.

And here is the Linux version:

These scripts I have provided are very simple and only currently set up your Java environment. You can use the same principles to set up your build tools or any other third party tools you may wish to run from the command line.

Sonar - UI for best user experience

Sonar has one of the most boring user interfaces I think I've ever seen -- it's butt ugly! but, as ugly as it is there's no denying how useful it is.

 

The image above shows an extract from a publicly shared project. I really liked the layout that they used so I took a few of their ideas and added some of my own.

A dashboard should be simple, provide enough information about the project ,but not overwhelm the user. If your customers or stakeholders have access to this they may not be technically minded and so this page should show info that anyone can understand.

Dashboard

To change the dashboard, log in as the admin user and on the right of the page select "Edit Filter":

You should be presented with some new options:

The column headings that I have selected are:

• Lines of code
• Rules compliance
• Unit test success (%)
• Coverage
• Complexity /method
• Complexity /class
• Public documented API (%)
• Duplicated lines (%)
• Total Useless Code (plugin)
• Build date

To add these, select the center dropdown in the Add Column row of the Display window.

Project Layout

When you select a project from the name column on the dashboard you will be presented with a page like the following:

Now this is where you can geek out and go crazy with metrics. Hopefully, by this point the non-techies have seen what they wanted and left. This page is where you can get creative.

I hope i'm not generalising too much but I would like to think most technically minded people are working with widescreen monitors for maximum productivity. If this is true I would strongly advise going for a three column layout. This can be done by selecting "Edit Layout" on the right and choosing the tri-colum option.

Finally, to select the widgets you want to display press "Configure widgets" and then you can drag and drop them at your desired locations.

Oracle - Query DB Sessions SQL

I had an error today when trying to create a datasource in Weblogic. The error was:

ORA-27101: shared memory realm does not exist



Cause: Unable to locate shared memory realm


Action: Verify that the realm is accessible

The cause of this error was due to the number of database sessions being outnumbered by the number of Weblogic instances. We set up 7 environments, each with 50 maximum connections but only 100 sessions on the DB side.... as you can imagine we smashed the limit.

You can check the number of sessions using:

 SELECT name, value  
  
 FROM v$parameter 
  
 WHERE name = 'sessions'  

And you can check how many are currently in use using:

 SELECT COUNT(*)  
  
 FROM v$session