Friday, April 10, 2009

Deployment automation and adventures in stupid

At work we currently do not have a ton of order to software releases and requests. I've worked hard to try to bring more there but it's just not part of the way things have been done in the past and that type of change in particular takes a lot of work and persuasion.
Summary we're better than we were but we're not perfect yet.

I have a confession to make...I was being a hypocrite yesterday. I've been preaching since shortly after I got there "you can automate deployment", "manual deployment is error prone and slow", "integrate regularly, release often". With my background as a sysadmin it didn't take me long to get every at least newer software release auto deployed in some fashion or another. The other push was towards not implementing software in chunks but slowly in front to back layers.

However, about a month and a half ago, me and my co-worker were making changes that would require us to change two products at once. I had stupidly gone off on an unecessarily bold refactoring to modularity of one of those products...so I just let things fall behind and stopped pushing frequent releases.

Month and a half goes by and now the trunk has :
  1. a TON of changes to a variety of processes. Including fixes to things that were major current maintenance time sinks
  2. I had not yet successfully moved all my unit tests around to the different modules so I had managed to eliminate my hard won safety net.
  3. Several database changes that I haven't scripted.
  4. Worst of all code that was behind schedule that had been done for 4 weeks.
So yesterday I finally have had enough of excuses why I wasn't pushing this build out. I manually do everything I had not yet had time to automate, and without all my tests in the proper location!

Finding several stupid kinks, multiple bonehead "I forgot" things, and 15 hours later (with only two hours of sleep) with a process that was supposed to take 30 seconds I had both products released.

I'm quite happy to say now I have an in sync code base, and a big list of fixes to implement which I will release one at a time!

Release Frequently, do not push off things till later, and do things in small small layers.

No comments: