Jenkins Slave Environment Variables

Spent quite some time hitting my head against the wall with this issue. Turns out environment variables need to be set in Jenkins because no end of changing variables on the slave will make a blind bit of difference.

So, I suggest if you have environment variables that need setting up, such as your VCS, or build tool you go via Jenkins initially, then if you get no joy from that use the Shell/Batch build step.

Git for all ages

At my company there's a wide range of technologies being used; some as old as the Mainframe, and newer technologies such as web services. This means that we also have a wide range of age groups - some from the days of COBOL, and other newbies such as myself from the C++ and Java days.

Something I've come to learn is that the old timers are not open to change, and if they have to change they want it to be easy. I'm sure you can imagine the looks on their faces when I started showing them how to use Git bash - anyone would think I just punched a small child.

TortoiseGit is a very useful tool that nicely hides the implementation of Git from the user. I found the old timers were far more receptive to Tortoise than they were of Git bash. I still finished the Git training in bash, but I also offered a small section on Tortoise at the end. The reason for this is that I wanted them to truly understand what Git was doing before hiding that behind a nice UI.

Virtual Appliances - a level playing field

I've only played with virtual machines for the past 6 months. I always looked at VM's as a server side  technology, and never really fully appreciated their use. Quite ignorant!

One of the challenges my organisation faces is ensuring all the developers are working with the same versions of software as production. Sometimes, the production machines get a patch, but the developers continue to work on older versions. Sometimes this happens because they don't know how to upgrade, are too busy or are just plain lazy. Either way, when their code goes in for release and breaks we always get the "works on my machine" story.

So, what is a virtual appliance?

"Virtual appliances are ready-to-run virtual machines packaged with an operating system and software application. These self-contained appliances make it simpler to acquire, deploy and manage applications by eliminating underlying hardware and operating system dependencies." VMware

At my office we work with predominantly Oracle software - and so we have created a Database, Weblogic, Forms & Reports on one appliance, then Database, Weblogic and BPM on another. Now, when we make a patch we update the VM and share it amongst the developers. Job done!

Cloning in the IT world is BAD!

Something that really irks me is my companies decision to clone production environments, and distribute these clones to create new environments. So, lets say a new project starts up and they want a replica of one of our databases - a request is made to the DBA, he takes a snapshot of production and puts this snapshot on the new target host. He/she then does a series of search and replaces for environment specific variables and then hands over the new environment.

You may be asking whats my problem? Well, here they are:

• When you clone, you take everything; configuration, structure and worst of all data.
• The process involves around 3/4 people and takes about a day.
• Garbage in, garbage out - whilst production is the only "truth" when you molest it with search and replaces who knows what you're getting?

So, what solution did I offer? 

As we're talking about the database I shall start with that. Firstly, a database is no less important than application code and so should be version controlled. Once the source code is version controlled you can use tools such as dbdeploy to build your database in a repeatable manner. I'm a firm believer that no live data should exist outside of production or the disaster recovery. If you cant generate test data for the scenarios you will encounter then your test strategy is not sufficient. 

What happens when I want to retain data?

In a data retention environment such as User Acceptance, we only ever want to apply deltas to the DB not refresh it every time - well, dbdeploy handles this very well.

Cloning isnt all bad..

I think the cloning strategy has some value - version controlling your database will not suffice in a disaster recovery scenario because the most important part - the data - will be missing. I wholeheartedly agree with the cloning strategy in this scenario.

Build Management and Monolithic Systems

The Problem

I've recently started working for a new company that operates in the insurance sector. They are an Oracle outfit with a few other technologies thrown in for good measure - there's even a mainframe. JOY!

Anyway, it quickly became apparent that they didn't truly understand their estate and they were looking at everything as a single system. As a build manager this set all kinds of alarm bells ringing - I hope they're not expecting me to build this huge system everytime there's a change to one small section...

Release day is worse!! Their current approach is to collate all of the changes for each of the subsystems and deploy them in one go and call it release X. Really?? Think about that in terms of Microsoft Office - I've made changes to Word and Excel, but not PowerPoint and then said we have version 2 of MS Office. We're then losing valuable information about the structure of Office; Word V2, Excel V2 and PowerPoint V1. I know its a crude example, but I hope you see my problem.

The Solution

Well, the first thing is going to be define solid boundaries in the system. Whilst this sounds quite straight forward, there's heavy dependancies on one of the systems in particular. Think of it as a viewer on a central database that is fed information from the other systems. Most often when a change is made, this 1 system will need an update too.

Implement a build framework. Currently, individual files are copied to a staging area then copied into the environments manually - this is not acceptable.

Tools - Command Prompt Here

"Command Prompt Here" is a handy tool for Windows Explorer to open a command a prompt directly from a desired folder in filesystem. It avoids opening a new command prompt window and then traverse to your desired location.

The below are the steps to configure. Please note its not an installable, just configurable.

Hope this helps you in your day to day tasks.

In explorer, open Tools, Folder Options.

• Select the File Types tab.
• For Windows XP: Go to NONE / Folder.
• Select the entry labeled Folder
• For Windows 2000/XP: Press Advanced button.
• Select New
• In the action block type "Command Prompt" without the quotes.
• In the app block type "cmd.exe" without the quotes.
• Save and exit Folder Options.

How-to - Checklists

In one of my earlier blogs I mentioned the importance of writing down all the steps you perform as you progress through testing a new piece of software. I just found an old BEA Weblogic document, and at the end of it there was a checklist. I was so taken back by this simple type of document I had to write about it.

I was having a conversation with one of the contractors this morning and we were discussing the amount of bumff official documents have - you can write off the first 6 pages to indexes and official crap about target audience etc.

If the document is an instructional type then we may want to read all the additional info the first time, but we wont need it when we want to use it just for reference. In fact, this could hinder productivity. Instead, make it standard practice to include a checklist in the appendices outlining the instructions given in the document.

To make a checklist:

• Only have 3 columns - Requirement, Notes and a Completed checkbox.
• Requirements are triggers. Requirements should provide enough information to get the synapses in your brain firing and retrieving the memory of the additional information in the document.
• Notes provide additional info. This can be links to external resources or other parts of the document. Include gotchas here.
• Completed. As each task or step is actioned you can tick it off the list.

I'm sure you're reading this thinking "are you serious?", but as simple as this is, it's often the simple things that get forgotten about. The great thing about a checklist is, if you have followed the information I gave about making notes; this will take little extra effort to complete.

Here's a great example of how they can be used effectively: Weblogic- Deployment Checklists