Something Smells

Automate, you must!
Recently, at work, we rolled out a new version of one of our major web applications. Until recently rolling out changes was a major undertaking. Over the past few years it's evolved into a much more reliable system that takes about 11 minutes!



Here is a quick chronology of the evolution of our deployment system:

1. Pre-deployment system
   Believe it or not our earliest versions of this product (many years ago in a galaxy far far away...etc) had no deployment system, let alone automation. Unless you considered me or some other guy making a change and Ftping it to the live site for testing, a deployment system. These were carefree times with few users to be annoyed by the occasional bit of downtime. Also, we worked in mostly interpreted languages (ah the good old days!)
2. Let's get a testing server!
   As we grew and more users were around to pay attention to our untested new "features". We decided a test server was in order. The first iterations of the test server were basic configurations of the original application. Nothing was automated but at least things got tested...mostly.
3. Let's automate
   This was a though time. I made a lot of enemies during the year or so it took to automate deployments. No one wanted to spend the time required to automate the system. Even those who saw the value in it were afraid of those who did not. Luckily, my boss, a man with surprising good vision for these things, was a willing advocate for change. At this point I should mention that this was about the time I was placed in charge of several other software teams and I was proposing a sweeping change for them as well. Let's just say it was ugly.
4. Let's REALLY automate
   About the same time we were automating our builds and deployments using NAnt, I was really tied up in the school of thought that was trying to address where databases fit into the Agile methodology. Databases present many problems that aren't handled well by most agile practices. Historically you try to design databases that aren't going to be very volatile. Agile methods suggest that everything should be capable of some volatility! Refactor this, throw this other thing out...etc. So, my problem was this, how do we allow the database to be volatile and still keep data safe? Well, once again I shifted some our resources to non-moneymaking endeavors. We developed an in-house database installer system that automates the deployment or upgrade of our databases. It works quite well is likely one of the greatest timesavers we have!
5. Refactor the Automation
   Over the past few years now we've learned a few things. We've switched a few things around and reversed a few assumptions we've made in the past. Through perseverance we've become so confident in our tools that we have fully embraced automation. We intend to automate everything. We've written a few custom tasks for NAnt and discontinued the use of WIX for many things. Trying to use Windows Installer was painful and it wasn't always clear exactly what it was going to do. We decided it just wasn't worth the effort and stuck to a custom system. Another major shift for us was the realization that our build and deployment scripts should be able to run on any machine, not just the build server.

Along with automating the build and deployment we have automated the Unit Tests and Code Coverage. We are capable of automating the code documentation but frankly it takes way too long and the resulting documentation is somewhat daunting. I'm considering some ways of producing cookbook style documentation where you look up the task you're wanting to perform rather than looking up objects/methods.

We still need to automate our acceptance tests. This is something of a holy grail for me. We've played with FIT.Net but still haven't been able to fully commit to it. Either way we are on the right track and I think we'll only get better!