Debugging Grails tests with Intellij

I was having some real headaches with debugging my unit tests today. With the introduction of forked execution came the breakage of Intellij debugging.

There are some rather long winded ways of applying remote debuggers, and Ted does a great job of breaking the problem down and offering a solution. However, in my rather tiny Grails application I was happy to sacrifice the benefits of forked execution - at least for my tests.

So, in BuildConfig.groovy I changed the following block:

	/**** FROM ****/
	grails.project.fork = [
	test: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, daemon:true],
	run: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
	war: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
	console: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256]
	]
	/**** TO ****/
	grails.project.fork = [
	test: false,
	run: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
	war: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
	console: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256]
	]

view raw enable-debugging-grails-in-intellij.groovy hosted with ❤ by GitHub

This change tells Grails that when we run the test configuration it shouldn't run in forked mode - this is anything we would run using the 'test-app' Grails command.

Method on class [] was used outside of a Grails application. If running in the context of a test using the mocking API or bootstrap Grails correctly.

 java.lang.IllegalStateException: Method on class [com.willis.heimdall.Booking] was used outside of a Grails application. If running in the context of a test using the mocking API or bootstrap Grails correctly.  
      at com.willis.heimdall.BookingIntegSpec.test saving a booking to the db(BookingIntegSpec.groovy:24)  

 I recently had this error on one of my simple examples. A real facepalm moment when I look back in retrospect but ho hum, the fix is nice and easy.

Broken Code:

	package com.willis.heimdall
	import org.joda.time.DateTime
	import spock.lang.Shared
	import spock.lang.Specification
	/**
	* Integration tests for the Booking model
	* @author Sion Williams
	*/
	class BookingIntegSpec extends Specification {
	@Shared def today = new DateTime()
	@Shared def todayPlusWeek = today.plusWeeks(1)
	def 'test saving a booking to the db'() {
	given: 'a new booking booking'
	def booking = new Booking(name: 'my booking',
	startTime: today.toDate(),
	endTime: todayPlusWeek.toDate())
	when: 'the booking is saved'
	booking.save()
	then: 'it can be successfully found in the database'
	booking.errors.errorCount == 0
	booking.id != null
	Booking.get(booking.id).name == 'my booking'
	Booking.get(booking.id).startTime == today.toDate()
	Booking.get(booking.id).endTime == todayPlusWeek.toDate()
	}
	}

view raw FailingBookingIntegSpec.groovy hosted with ❤ by GitHub

Fixed Code:

	package com.willis.heimdall
	import grails.test.mixin.TestFor
	import org.joda.time.DateTime
	import spock.lang.Shared
	import spock.lang.Specification
	/**
	* Integration tests for the Booking model
	* @author Sion Williams
	*/
	@TestFor(Booking)
	class BookingIntegSpec extends Specification {
	@Shared def today = new DateTime()
	@Shared def todayPlusWeek = today.plusWeeks(1)
	def 'test saving a booking to the db'() {
	given: 'a new booking booking'
	def booking = new Booking(name: 'my booking',
	startTime: today.toDate(),
	endTime: todayPlusWeek.toDate())
	when: 'the booking is saved'
	booking.save()
	then: 'it can be successfully found in the database'
	booking.errors.errorCount == 0
	booking.id != null
	Booking.get(booking.id).name == 'my booking'
	Booking.get(booking.id).startTime == today.toDate()
	Booking.get(booking.id).endTime == todayPlusWeek.toDate()
	}
	}

view raw BookingIntegSpec.groovy hosted with ❤ by GitHub

Note here that we have told Grails what we are testing with the @TestFor() annotation so that it can set up the relevant mocks and stubs in the background.

Securing Git repositories

Edit .git/config in the [receive] section:

	# no rewriting history
	denyNonFastForwards = true
	# no deleting history
	denyDeletes = true
	# check object consistency
	fsckObjects = true

view raw local-update-paranoid hosted with ❤ by GitHub

Gradle Weblogic Plugin

I'm pleased to announce the arrival of the Gradle Weblogic Plugin. Its been a while in the making, but I've finally got a stable version out with the 6 primary tasks. They are :

• wlDeploy - Deploys or redeploys an application or module.
• wlUndeploy - Stops the deployment unit and removes staged files from target servers.'
• wlCancel - Attempt to cancel a running deployment task.
• wlRedeploy - Redeploys a running application or part of a running application.
• wlStart - Makes a stopped (inactive) application available to clients on target servers.
• wlStop - Makes an application inactive and unavailable administration and client requests.

The latest version is currently:

You can find the source code and README at the following address:
Github - Gradle Weblogic Plugin

and the binaries are hosted at:
Bintray - Gradle Weblogic Plugin

Installing Google Chrome on Linux

In this example I'm using Xubuntu 14.04 LTS:

	$ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - && \
	$ sudo sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' && \
	$ sudo apt-get update && \
	$ sudo apt-get install google-chrome-stable

view raw gistfile1.txt hosted with ❤ by GitHub

Gradle - Filter an archive zip/war/ear

Problem

I have an archive compiled with properties pre-defined within, but they need to change as I promote my build through the different stages of the SDLC. One example could be with a connection.xml that has URLs pointing to different databases.

Gradle already offers some great solutions such as keyword expansion and ant token replace, but both methods require the token to be in a predefined format. The expand() solution looks very similar to groovy string interpolation and looks for ${..}, whereas ant replace tokens looks for @..@.

Solution

The solution here was to pass a closure to filter which is called for every line of the filtered file. The closure performs some simple processing using the MapFilter() class and its MapReplace() method. An example is shown below:

	buildscript {
	apply plugin: 'groovy'
	dependencies {
	gradleApi()
	}
	}
	ext{
	earDir = "${projectDir}/src/dist"
	explodedDir = "${buildDir}/exploded"
	}
	/**
	* SINGLETON VIA METAPROGRAMMING
	* http://groovy.codehaus.org/Singleton+Pattern
	*/
	class MapFilter {
	String MapReplace( def input, Map map ){
	try {
	map.each { key, value ->
	input = input.replace( key, value )
	}
	input
	} catch ( e ) {
	logger.error e.message
	}
	}
	}
	class MapFilterMetaClass extends MetaClassImpl {
	private final static INSTANCE = new MapFilter()
	MapFilterMetaClass() { super( MapFilter ) }
	def invokeConstructor( Object[] arguments ) { return INSTANCE }
	}
	def registry = GroovySystem.metaClassRegistry
	registry.setMetaClass( MapFilter, new MapFilterMetaClass() )
	def myMap = ["devURL" : "productionURL",
	"devProp1" : "productionProp1",
	"devProp2" : "productionProp2",
	"devProp3" : "productionProp3",
	"devProp4" : "productionProp4"]
	task unzipFilterWithMap(type: Copy){
	def zipFile = file( "${earDir}/myZip.zip")
	from zipTree(zipFile)
	into explodedDir
	filter { line ->
	new MapFilter().MapReplace(line, myMap)
	}
	}

view raw Gradle-Map-Filter hosted with ❤ by GitHub

There's more ...

You may have noticed that this example also includes a lot of meta-programming. This was just my first dip into the world of meta-programming. In this example the metaclass simply intercepts every new call to map filter and returns the original map filter instance.

myFile.toURL() deprecated

myFile.toURL() deprecated The fix to this is quite simple:

	def env = project.hasProperty('env') ? project.getProperty('env') : 'NULL'
	logger.quiet "Loading configuration for environment '$env'"
	def configFile = file("$rootDir/gradle/config/buildConfig.groovy")
	def parsedConfig = new ConfigSlurper(env).parse(configFile.toURI().toURL())
	def key = project.hasProperty( 'key') ? project.getProperty('key') : 'NULL'
	allprojects {
	ext.config = parsedConfig
	}

view raw gistfile1.txt hosted with ❤ by GitHub

Gradle dependency management

I have recently been working on writing a Gradle plugin for the popular SoapUI functional testing tool. To make things easier I decided to port the maven plugin rather than reinvent the wheel. I've been working predominantly in IntelliJ, and for some reason it didn't flag any dependency failures.

I felt the time had come to add the project to CI and was surprised to see the build failed. I have been employing test driven development so had expected test failures but not dependency failures.

To resolve the issue I first had to establish what was causing the problem; running the 'gradlew dependency' task helped nicely! The output below is quite clear:

Simply changing the dependency to a later version fixed my issue.

Multi-project distribution archive

This snippet will produce a zip archive that includes the artifacts of the subprojects.

	task distributionZip(type: Zip) {
	archiveName = "${project.name}.zip"
	subprojects.each { subproject ->
	inputs.dir subproject.buildDir
	into(subproject.name) {
	from subproject.buildDir
	include 'docs/**'
	include 'libs/**'
	include 'reports/**'
	include 'test-results/**'
	}
	}
	}

view raw distZip.gradle hosted with ❤ by GitHub

Daniel Kahneman - Thinking fast and slow

Here's my interpretation of the following presentation from Daniel Kahneman.

Thinking fast, and slow - Daniel Kahneman

What do you see as the key points?

System 1 is our fast thinking self, the one that makes decisions based on intuition, is influenced by emotions, uses tacit knowledge, and operates out of habit.

System 2 is the part of us that analyses situations, considers alternatives, plans for the future, and does the math. Whenever we find ourselves pausing to consider something carefully, its like switching from autopilot to manual mode; our analytical mind takes over from our intuition and works out rational choices.

What are the pros and cons of System 1 and 2?

System 1 is intuitive and moves easily, adjusting rapidly to changes. It has a lot of experience in its speciality areas and is comfortable trusting its expertise and intuition to guide actions.

System 2 analyses situations before acting. It knows that the best decisions are those based on evidence. System 2 would run experiments, gather data, and weigh the impact of various choices before making a decision.

System 1 proved its weakness with the baseball riddle. There was a moment where system 1 assumed it knew the answer but was incorrect.

System 2 is slow and less effective in agile situations.

How do you get the best out of both?

Modifying a mindset will require deliberate effort and considerable time as with breaking any habitual characteristic.

What are the challenges for you - what should you watch out for in tackling PoD challenges?

I am heavily driven by System 1 and don't often weigh the impact of various choices before making a decision. I'm hugely passionate about driving technical change, but often get blinded by my experience and sometimes provide a solution before fully understanding the problem.

Sir Ken Robinson - Schools Kill Creativity

What are your views on the clip?

The most interesting part of this clip was how captivating Ken was during this presentation. He used humour to get his point across instead of ranting. He commanded attention within the first few moments of being on stage and held that attention until the end.

What do you see as the key points?

School had a pretty lousy system for recognizing talented students.

Does it resonate - if so, how ?

Research has shown that talent is not something people are born with; it is something that is developed over time with hard work and diligent practice. Once students are “tracked” as low-potential, they will not be
assigned challenging classes, as Gradillas discovered. Tracking can start very early and be based on little more than a student’s command of English. (Jaime Escalante and Jack Dirmann, “The Jaime Escalante Math Program,” Journal of Negro Education59, no. 3 (Summer 1990): 407–23)

Will you do anything differently as a result of watching it?

I think its important to nurture creativity in the same way as academia. However, its important not to force one on the other unwillingly.

Quote of the Day: Shoehorn

This is a fun one - how many times have you been close to a deadline and the customer asks you to squeeze another feature in, the delivery manager commits to it and we're all working till midnight to make sure the feature is delivered? Well, for this one we like to use the term shoehorn - is it ironic the delivery manager got one in his xmas cracker this year?

Quote of the Day: Risk Accept

This is not a smart quote, but rather something that really pisses me off!

How many times have you had a deadline nearing and you find a test failure at the last minute? Well, unfortunately for me this happens most releases. What's worse is that rather than push the date and ensure we have a good delivery - the PM decides to pull out the "Risk Acceptance" card. If you're fortunate enough not to have heard this term it basically involves asking the customer to risk accept the defect - or as I like to say, accept that we are delivering a steaming pile of turd.... what's worse is 90% of the time the customer drives the delivery, not the quality.

The real shame of all this is you can guarantee a user will raise a defect, place it in the queue, then complain that it takes 6 months to deliver.

Quote of the Day: Have a Target

"If we dont have a target, we cant take aim. If we dont have a destination, we cant leave home or well be driving around in circles for hours. If we dont have a definition, how can we monitor or judge success"
- Richard Templar, The Rules of Money and How to make it

Deploying all day - Sean O'Connor

I recently watched a technical presentation from Sean O'Connor of bitly where he discussed how they achieved up to 20 deployments a day. Whilst much of their techniques are publicized and well known - one of the interesting points he raised was how they only have a single source repository. I don't think this is new; I've heard rumours that Google do the same, and have done for years.

My eyes always light up when I hear these types of stories. They're bold and clever, but I only wish I could do the same......Unfortunately, for me my business is insurance, and not technology - so getting a project manager to accept additional costs that don't immediately result in cash is difficult.

So, why do I think this is a good move?

Its much easier to track and baseline your systems when all of the source is in a single location. Modern version control systems offer a lot of flexibility in this area, and provided you have a bunch of well disciplined developers - you wont have to worry about things like commit messages.

I do still see problems.

My biggest concern would be that unless you start doing some clever tracking on sections of your source tree instead of the tree as a whole, you could be constantly hammering your CI resulting in queues. This could potentially cause long feedback cycles which is an anti-pattern of continuous delivery.

Quote of the Day: Good plan vs Perfect plan

"A good plan today is better than a perfect plan tomorrow" - General Paton

This is an interesting quote, and one I feel can also be dangerous.

We all know that in technology time is money. The longer we spend trying to make the perfect plan, the more money we stand to loose. That being said, a plan that has not been thought out well or had the best people involved can equally be as costly. All too often I've seen management take a path of least resistance only for it to turn around and bite them in the ass.

Quote of the Day: Being Careful

"Quite frankly, I'd rather weed out the people who don't start being careful early rather than late. That sounds callous,and by God, it is callous. But it's not the kind of "if you can't stand the heat, get out of the kitchen" kind of remark that some people take it for. No, it's something much more deeper: I'd rather not work with people who aren't careful. It's Darwinism in software development."
-Linus Torvalds on kernel debuggers, Linux Kernel Mailing list

I'm a huge fan of Linus, but this solidifies his position as one of the greats in my mind. There's no excuse for not being careful and I believe its every software engineers responsibility to ensure the work they have done is to a high standard. Unfortunately, I've witnessed a workplace where the strong carry the weak, and its not good. The weak are not challenged and so become unhappy, and the strong get overloaded and become unhappy. I believe its every managers responsibility to kill this kind of behaviour early and without fail.