28 December 2006

Continuous Integration Workflow

Warning: Article in construction!!!


Before reading my prose, I am going to give you some homework, you are going to print the Martin Fowler 's paper Continuous Integration and read it thoroughly.

A bit of history!

Skip this part if you want real meat, it's just rhetoric!

My obsession / passion / addiction for x-unit frameworks comes from back in 2001, I was feeling a bit depressed by the low quality of code and the amount of bugs generated by my team
Then I came across the books written by Fred, Kent and Martin and that was like a revelation ... Lots of bla bla missing!

At the end of his article, Martin affirms that "... Continuous Integration has become a mainstream technique ....". To be realistic, I'd rather say will become a mainstream technique. I have worked in France, UK and Spain and I have the feeling we are not there yet! Give us a few years, Martin!

How was it before Continuous Integration?

Ah ah! Too many bad memories here ;-) Just a lot of time wasted and a source of conflicts within the team creating a bad moooooood ...
I am willing to draw a comic, Dilbert's style to illustrate my thoughts ...

Continuous Integration Workflow

This diagram is an attempt of making a picture from Martin Fowler and James Shore's words ...

Note : A Repository is a Source Control Management Repository. In the Subversion jargon, a Repository holds the source code of a project in a folder called trunk.

Development Steps:

  1. Update repository. To make sure sure you are always working the ''latest'' source code, start with getting the latest version of the project by updating your local repository.

  2. Lock existing files. (Optional step) Optimistic Locking VS Pessimist Locking, this debate can go on for ages ;-) I tend to consider the merges as a waste of time and try to avoid it as much as I can. More than a physical lock, which seems a bit extreme, I'd recommend that developers need to be aware about the W4 (Who's Working on What and When!). It is called Communication by normal people ;-)

  3. Create new files you need. This step is quite obvious, I put it as a reminder for step for step 6 (add the new files to svn) because this is a common mistake to forget to add the new files to the SCM.

  4. Development. using a Test Driven Development, Behavior Driven Development or a more traditional approach with a good test coverage. Make sure you run all the tests of the project, you never know - your first class code might have broken something - this is called regression.

  5. Update repository. How many times have I forgotten that step! The other folks in the team have been busy since you started working and you need to make sure you have the latest version. After the update, run all the tests another time, if there is a problem, it's probably in your code! This should encourage you to integrate more frequently.

  6. Add the new files to SCM. Jay Fields wrote a rake task which does the work with subversion.
  7. Check the build status. Before committing, you should check the build status and wait for 10 minutes if there is an integration running. Cool tools such as Cruise Control help a lot regarding that matter but we don't have that in the Rails world yet (remember the title of the article), hence we'll do it "manually".

Integration Steps

  • A. Commit files. Run to the integration box and start the integration process.
  • B. Tests run on Integration machine.
  • C. Rollback your changes. If the code can't be compiled or test the fail, you must ensure that the integration repository contains safe code, hence you need to rollback your changes and go back to your box to solve the problem. Sometimes, you can't reproduce the problem on your box and you'll need to play with the integration box: check the environments variables, dependencies, etc ...
  • The End. That's it, the integration is a success: play a noise, change the color of your nabaztag! Unlock the files you were working with. The Continuous Integration application should have updated the Integration logs (journal) and send an e-mail to team so they can think about updating their local box (the sooner, the better!).

Note: You will have guessed that I am not a native English speaker. Even if I am doing my best to write in a good English, I guess the result is far from being perfect ... Hence, I won't mind you people tell me about the typos ;-) Thanks in advance!

As a matter of fact, I am not pretending to be a Continuous Integration guru and I'd be happy to learn and debate, please send me a lot of comments!

Implementing Continuous Integration in a Rails project

Just an awful bunch of links for now. I have to say I don't know yet the best method!!! But I will write another post as soon as I have more experience ...

Cruise Control is a wonderful opensource tool for java projects, developed by Thoughtworks.
Damage Control, an cruise control for ruby but this a project currently not under active development, http://damagecontrol.codehaus.org/Continuous+Integration+Server+Feature+Matrix

In the Rails world, DHH has developed us a plug-in which integrates with subversion and campfire: the Continuous Builder plug-in.
Jay Fields, a thoughtworker describes his experience in his blog: http://jayfields.blogspot.com/2006/07/ruby-continuous-integration.html

Cerberus, http://cerberus.rubyforge.org/index.html

1 comment:

Nick Nevsky said...

Only tool that I found useful for my Ruby projects is Cerberus. You have mention about it.

I am using it as a Continuous Integration tool several months already and it works like a charm. It sends all notifications to project Campfire and to GTalk of all developers.

Other tools i've tried do not match my expectations (DC, Continuous builder plugin, hand-maded python script)