Writing User Stories

Writing user stories should be easy. The reality is that most users don't know how to properly write a user story. Suppose a client asks you to write a wire transfer subroutine for systems at two different banks. (below I will highlight proper Stories in green and improper stories in red)

The client says...Write a wire transfer subroutine for systems at two different banks.

While a need is expressed, I would submit that this is not a User Story. What's been stated here could be the beginning of a vision statement, perhaps for the next release or the next Iteration. Let's try to distill this a bit.

So the client reconsiders and says.....System A should use a web-service exposed by System B to electronically wire funds.

A common pitfall in writing User Stories......getting too technical. The statement above assumes that web-services will be used and it may be perfectly logical to assume that they will be used, but let's examine what a User Story is really intended for. First and foremost is the "USER";specifically, what actions a user will perform and what she will experience in return. The examples above don't expose any User Experience information. Second, is the "STORY"; how the user gets to this point, what the user expects to happen.

A user, wanting to transfer money to account XXXXX, logs into bank A's system She selects the wire transfer option. A message asks her to enter the account number and bank to transfer to (bank B). The system then asks the user for the amount to transfer. After submitting the amount the user may log into bank B and see the new wire transfer as a deposit.

This example gives us more insight into how our client intends to use the system. This provides a better foundation for acceptance tests. It also doesn't assume any particular technology. XP projects have an evolutionary design. So, whether or not the system uses web-services is unimportant.

With our story in hand we can begin to develop acceptance tests. Once acceptance test are written we can begin the process of costing and eventually developing the story.

Further Discussion
While users should avoid getting too technical, users should also feel free to express common UI elements. In most cases they have a good idea of how they want to do their work. If you have user experience people or just smart developers you can help suggest alternatives. The point is to get insight into how a user "uses" your software.

What you DO want to avoid are stories that specifically suggest infrastructure or service elements. Things that throw red flags for me are terms like "Web Service", "Array", "SQL Query", "Stored on the Hard Drive".

Checkout Part 2: Writing User Stories using the 5Ws.

Here are some other good references on writing User Stories:

• Writing Good User Stories
• Six Features of a Good User Story - INVEST Model
• C2's User Story Page
• User Stories Articles
• Gardening the Backlog (for stories)
  

New Name....Same Crap

I've changed the name of my blog!!!! Not that this makes much difference to my audience. This blog will, from this day forward, be known as "Something Smells". While my critics may consider the cause for this change to be self evident (Garreth, Rob?), I made this change for wholly different, and much shallower, reasons.

First of all I no longer fancied the old name (yes, I just used the word fancied). Second, after long deliberation in the shower at the American Airlines Admiral's Club, I decided to really re-enforce the code smell concept. Oh and I changed the masthead....I think the new title goes better with the masthead.

Why more on code smell? Well, I won't be writing more about code smell, but I do think that my general technique for coming up with interesting topics (don't say a word) is somewhat similar to how we find code smell. Also, I think the art of hunting for code smell is a bit misunderstood. For better or worse, we all do it (hunt for code smell) and the sooner we accept it the better we can start to become at it.

Enjoy, comment and try to be kind!

Happy New Years!!!!!!

ORM Hunting (in the .Net world)

I have been hunting around for some code generation tools to play with. I"m really looking for something very simple like the Ruby on Rails generator. I've come to the conclusion that no one want anything easy in the .Net world.

I've also been looking at some DALs, ORMs, etc. Really, most of the DALs and ORMs offered in the mainstream are just WAY too much. I really have to think that if you need some of the more advanced features of these systems you may want to relook your design. Heck you may want to read up on doing "The Simplest Thing that Could Possibly Work."

Anyway, I use Castle's great framework called ActiveRecord. It's so simple that you really don't need code generation. As long as you keep things simple when you begin using it, you can slowly move towards the advanced stuff (if you really need it). So what was I REALLY hunting for. I'm really looking for a database schema upgrade (or migration) that's as simple as activeRecord.

ActiveRecord has a nifty CreateSchema method. This method scans all the ActiveRecord classes in your model and creates a database schema for persisting your objects. Of course since ActiveRecord is also an ORM, it provides you nifty methods for instantiating and persisting objects (to the newly created schema). The real sticking point is this....CreateSchema deletes the database everytime! THis means all of your data is lost!

For a while we've used a very kewl in-house migration library. Whenever you modify or extend your database schema you just add another migration. It versions each migration and the database, allowing for a very painless upgrade everytime the software is run. Before our migrations tool, we kept large repositories of bloated DDL/SQL scripts...yuck.

Now, back to my hunt. It turns out that I was really looking for a tool that would look at my model and generate (or upgrade) the database schema based on the model. ActiveRecord already does the generating....so what I need is an upgrade option.

I've heard that Hibernate has such a feature...so far nHibernate (ActiveRecord wraps nHibernate) does not have this support. Maybe someday. Maybe I'll write something. What boggles me are the number of generators that focus on generating the model based on an existing database schema. Now, I do understand that many databases with lots of info are out in the field...and generating a model based on that schema might make sense. The ones that seem silly to me are the ones that generate a new set of model classes everytime the schema changes...instamagically!

It's a challenging thing to create these sorts of frameworks. I guess. I'm not sure I'm comfortable with a framework that generates and regenerates the model in such a willy-nilly way. Wouldn't this be a refactoring nightmare? Sure, you don't want to constantly refactor your database schema either. Other applications may depend on that schema. But, by generating the schema from the model your schema is less likely to change often or drastically.

Here are some of the ORMs and DALs I found:

• Castle ActiveRecord (.Net)
  Castle's port of the ActiveRecord pattern wraps the nHibernate library. ActiveRecord makes it very simple to create a model and persist it to a database. You don't need any special tools. You create a simple XML configuration (connection strings etc.). To make things really simple your classes inherit from an ActiveRecordBase. This base class provide all the methods you need for creating updating and searching for business objects.
  
• Hibernate (Java)
  I haven't used hibernate yet.
  
• nHibernate (.Net)
  I've used nHibernate briefly. Unlike ActiveRecord you have to create complex XML mappings between your model and the database. I'd recommend going with ActiveRecord.
  
• ActiveRecord (Ruby)
• doodads (.Net)
  

Kroon had a nice article on several of these. Also check the Server Side's article on ORMs. It has a pretty complete list.

Planning Poker on Rails

I'm starting a new Open Source project. It will be a Planning Poker game for distributed teams. Here's the blurb on the project site:

The planning poker on rails website will allow distributed teams to play planning poker. It will track user stories and the points assigned to each story. It should also allow for quick access to archived stories as references for future planning games.

Each player should be allowed to place a card (estimated effort). Once the cards are all placed the application will reveal the estimates to everyone. If no concensus is made then each player will have the floor to make comments. As each player makes his or her comments the application will move to the next player. Once all players have no more comments OR a player presses the VOTE NOW button. The computer will take another vote this continues until concensus is made OR all player agree to table the vote.

Before you google it...YES another project already does this. So, you may ask, "why re-invent something?" Well, here are a few reasons:

1. Of course I think I can do it better!
2. I don't think the other is open source....just free to use.
   
3. I really wanted to play with google's code projects.
4. I wanted a project for which I would be my own user.
5. Did I say I think I can do it better?
6. I wanted something that begged for Ajax and Prototype stuff.
7. I wanted a project that had users interacting in realtime over the web.
8. I really like using Planning Poker on my teams and I wanted something that was designed around "MY" rules for the game.
9. Just want to play with a few technologies.

Wana check it out? Go Here!

Wana help? Email me!

Wana check out the competition? Go Here...

Bashing Agile / XP

I just read an article that takes a very cynical view on XP and Agile methodologies. I wasn't allowed to comment without signing up for something so I will comment here.

Check out the post "On the trail with the Cowboy Coders."

I think the author misses the point of Test Driven Development (TDD) and the XP practices, as a whole. His suggestion that XP is "Cowboy Coding" couldn't be further from the truth. In fact (this is always said) XP takes a great deal of discipline. All of the practices and values must work together or you will never reap the benefits of XP. In particular, TDD is not merely a way of making sure that your code is tested. This is a common misconception amongst those who have been newly introduced to TDD.

The reality is that TDD serves as a design tool. Now, XP does "shun" up-front formal design. But XP does not ask us to do "no design". The design is expressed differently....the design evolves. Let's take a look at how TDD helps our designs evolve. First, users write Acceptance Tests. Properly written acceptance tests are the XP team's first expression of a system's design. These tests are used as a road map for what the user should experience when interacting with the new system. Second, we have unit tests. Unit tests stress those portions of the system that are invisible to the end user. The scope of Unit Tests should be determined by the team...for instance, we tend to write a test fixture for each discreet component of a system. Finally, we write tests BEFORE we write code. Once we have a failing test we begin writing code until the test passes. It sounds strange, until you realize that the tests are more than tests...they are the design document for our new system.

Ask any XP team, it takes a lot of discipline. Most traditional teams save the tests until the end. They see the tests as something that would be "nice to have" but not really necessary. In general they just never get written. This coupled with another reality, that most upfront designs go out the door after a week or two, makes a pretty strong argument for using TDD as a design tool.

Now, before I conclude my little rant, I must remind everyone that XP/TDD does not happen in a vacuum. All the practices need to be applied. Remember "evolutionary design" happens as you go! Tests should be written when you need them not just because you want a bunch of tests. So please don't run out and write a ton of tests for crap you'll be working on next year. I guarantee that your requirements will change and you will have wasted a lot of time!

What is Important (Warning: Disconjointed Thoughts abound)

Well, It's been a while since my last post...trust me I've been busy!

I've been reading quite a bit about Agile Developmet trends. Always ready to learn some new nugget of goodness. I always find myself asking, "What is really, really important to an agile project?" Is it simplicity? Test Driven Development? Pair Programming? Evolutionary Design? One of so many other values and practices?

Of course my first response is, "it's all important, XP requires a tempered application of ALL the values and practices."

We have so many around us that don't like this answer. They wonder, "where are the controls? What is the process?" My answer is, inevitably, "the team is the control and the process. They are guided by a coach and a good faith effort to provide regular software releases."

I must admit that, in business, we need to"know". Know when the software is going to be done. Know when the software is not going to be done. Here, I argue that most methodologies fail to effectively predict these things. So, why not select the methodology that wastes less time on fruitless planning and more time on fruitful development?


I am a huge fan of pair programming. Actually I am a huge fan of many of the more painful practices;TDD, Integrate Often, Simplest Thing That Could Possibly Work, etc. I think they force us to do the right thing. I've also found that most of the practices hinge on pairing. Navigators always seem to be sticklers for doing the right thing.

As usual my thoughts always return to simplicity. I like to think of simplicity as the "golden value". It's importance can never be overstated. Everything that goes wrong in software development can somehow be traced back to some complexity. Simplicity makes the code easier to write. Simple requirements require less time to design. Simple designs require less time to implement. Simple code is easier to integrate and maintain. Easier means faster and faster means a quicker and greater return on investment.

So what makes something simple? Is it simpler to just drop a datagrid on a webform or to use MonoRail views and Brail? What about IoC containers? Is dependency injection complex or simple? I bring up these points for a reason. I read a blog with a pretty half-hearted flame about these same things. I must tell you that I am easily side-tracked by these sorts of discussions. I want things to be simple! Shouldn't WebForms controls be simpler? Or why go through the through the trouble of implementing an IoC container? I'll try to answer....

I could just say, "these are the patterns we choose to use and it makes the code simpler to maintain and refactor (remember refactor mercilessly)." This works academically, but the real truth is that it "feels" better. The WebForms designer still has a cludgie feel to it. The code it generates is nasty. Also, the MVC framework takes away 90% of the design decisions and, as a framework, is simpler to wrap your mind around. The IoC containers simplify the bulk of the coding...the IoC container is complex...but it's done already! Finally, Dependency Injection really supports "once and only once" and allows you to design objects that do or express ONLY what they are meant to...that's simplicity.

My next post I hope to expand on some of the "big words" and ideas above. Please comment on what you believe is most important!

Tactical Scope Creep
In my experience, scope creep is one of the greatest threats to the completion of an iteration. I would even suggest that the most dangerous "scope creep" is the kind inflicted by engineers.

Most discussions of "scope creep" tend to focus on the customer inflicted variety. For agile teams, working iteratively, this sort of strategic creep has been managed (see embracing change). The creep we don't always catch is the creep at the tactical level.

Have you ever been faced with a simple story and somehow ended up making a Rube Goldberg machine? Damn the schedule and designed a complex framework in hopes of using it again later? Regretted wasting time on a complex API that did no more than paint your project into a corner?

In the end, all of these cases are examples of one thing...tactical scope creep. I myself have never fallen prey to these vices (yeah right). Seriously, I can't count the number of times I was going to write the next great framework that would solve ALL future problems......I wish someone had been there to say YAGNI!

So, what do we do? We apply the XP practices and principles.....duh. Remember, XP is a somewhat wholistic methodology. You MUST plan iteratively, have users write stories and play the planning game. Ok, disclaimer out of the way, here are a few XP principles/practices that will help you conquer tactical scope creep:

• Pair Programming
  No sh@t...it works. One navigator, one driver. Two minds are working on the code and the design. This creates a natural 2-way sounding board for new ideas (as opposed to having NO sounding board in normal programming). Bad ideas and transgressions of the other practices are identified and squashed quickly.
  
• YAGNI
  You Aren't Gonna Need It! This one is a no-brainer. If the story states "Users can write coments to blog entries", then write a solution that accomplishes this! Notice that this story doesn't suggest that users can view comments. It doesn't suggest that authors can manage comments either. SO...don't worry about it! Suggest that a user write stories for these other features. Writing very small stories is a good practice anyway. Small stories are easier to estimate and easier to focus on. Of course, differentiating between YAGNI and flexibility is no science.
  
• TSTTCPW
  The Simplest Thing That Could Possibly Work. This practice dives deeper into the tactical design than YAGNI does. TSTTCPW asks us to simply make a duck a duck. If writing a file to the disk is simpler than writing it to a database....I guess you should write it to the disk. Later if requirements change, and you have properly encapsulated responsibilities, you can refactor to a database. Of course, like most of the XP practices it's application is subject to team arbitration. If a pair can't agree then let the WHOLE TEAM (see below) decide if something really is TSTTCPW.
  
• Simple Design
  Over engineering a problem is the most frequent cause of tactical scope creep. If you can get away without a factory/service tier then do so. If you can use an O/R mapper (such as ActiveRecord) then do so. Stay away from glitzy user interfaces....wait until someone writes a story that requires a "glitzy" user interface.
• Whole Team
  Especially the idea of putting all engineers in one room. Let the team police itself. Inform the other pairs when you are about to make a complex design decision. If your argument sways ALL of them then go for it!
• Fail Fast
  Ok, this is actually a Test Driven Development (TDD) practice but it will help. Get something up and running quickly. Don't let yourself ponder the intricateness of the problem at hand. Over-engineering almost always starts in the "minds" of smart people.
  
• Small Frequent Releases
  Your users will love you AND you will love being loved! That killer new framework would have to be pretty damned good to give you the same validation as a satisfied user. More likely the time you waste designing the killer API will overshadow any benefit derived from it.

Above all, work with your users and testers constantly to improve your understanding of their needs. Also, avoid becoming a simplicity/YAGNI drone. XP is about putting good software before the methodology.

Now, some things can slip through the cracks. Most will be unimportant. Some will be VERY important. Here are some things to keep in mind when you are getting ready to drop the YAGNI bomb:

• Security
  This is the most serious potential problem. Security, especially on the low level, almost never makes it's way into a user story. I would suggest putting a security expert on your QA team. Have them write stories describing the security holes that require attention.
• Accesibilty / Multi-cultural / Usability
  Once again these can be handled by placing an expert on your QA team.

Another option may be to track a set of "perpetual stories".

Well, I hope I haven't wasted too much of your time. Please feel free to comment, complain or flame!

ASP.Net Frustration

It's very hard to articulate "why" I am frustrated with ASP.Net. Recently, I found a post that, while not articulating the "why", does articulate the feeling:

I think what annoys me the most is that I really like C#. It’s a great language, and the .Net framework is awfully well put-together. I really enjoy spending time with C#, but then along comes ASP.Net like the loud, overbearing, half-drunk uncle at a family reunion, tips over our picnic basket, and ruins everything. - Deane

I've tried exploring this subject on many occasions and come to the conclusion that it is a subject best discussed over a pitcher of beer! Before I go on I must give thanks to the guys (and gals?) at the castle project and their incredible Rails Framework Monorail.

So, ASP.Net is frustrating. To be fair to many who have bought into ASP.Net, ASP.Net sells itself very well. As a technology it is very heavily driven by marketing needs. The MS sales force wanted a technology that "sells"...and they got it. Viewstate, seemless event handling, Components, a response and request pipeline that can be "easily" hooked, and many more features make it a must have technology!

Wow, why the frustration? I think until you've tried something better (see rails above) you just accept the frustration. Heck, it's still better than the procedural approach of ASP or PHP...right?

The easy answer is "developers should know more about the http process and ASP.Net hide too much." That's always sounded a little like a cop-out to me. Don't get me wrong...I agree with the basic premise but I don't think it's the framework's problem. I see a real usability problem with ASP.Net.

ASP.Net wants to hide the ugly details of maintaining the viewstate between the web browser and the server code. The idea is great but in practice it seems to be heavily dependent on the IDE. Webforms seems to give us great options for seperating the UI from the logic. In practice though it gets a little hairy.

Where I work, we often put together "Scafolding" views that are meant to be thrown away or refactored once we've got everything working on the back-end. The Webforms designer in VS.Net allows you to create "Scafolding" very quickly....but when it's time to refactor, the code is difficult to un-hitch from the IDE.

Another problem with the Webforms approach is how verbose the ASPX code becomes! Unless you do everything in the Designer (which isn't much) you often need to drop into the code. Looking behind an ASP.Net page or into the hidden regions is like opening a pandora's box of problems!

Finally, why does the page event pipeline need to be SOOOO complicated! When we were using WebForms I would often override the whole thing just to get a little control back! The Framework should make it easier to munge with the request/response pipelines but in many ways Webforms makes it harder.

OK...you've wasted another perfectly good 10 minutes reading my rant. As usual I welcome the wisdom of others in all matters about which I write.

Castle Project and Monorail

I just wanted to give a quick nod to the guys at the Castle Project! We've started converting one of our major applications to their Monorail framework.

For those of you unfamiliar with Castle or Monorail, Monorail is an MVC framework for .Net web applications. It completely replaces the webforms framework with a much simpler (and more powerful) rails style implementation. The project includes an OR Mapper (ActiveRecord) that uses NHibernate as it's back-end. Their ActiveRecord implementation greatly reduces the time it takes to map objects to the database. They support several view engines based on Nvelocity, Boo (brail), string template, and some others. They also have a basic generator for scaffolding and a basic migrations framework is being developed.

If you are in a .Net shop and looking to take advantage of Rails/MVC or if you are just fed up with the overengineered webforms model then check out Monorail!

Here is a link to my Castle bookmarks: http://del.icio.us/rss/dpupek/castleproject

Agile Team Evaluations

If you didn't know then I'll tell you now...... I am a huge proponent of XP. I don't much care for Scrum.....it just doesn't work for me, deal with it. I recently read an article "Should a ScrumMaster Give Performance Appraisals?" For those of you who are unfamiliar, a scrummaster is the same as the XP coach.

My first thought was, "who else is really qualified?" My second thought was, "yeah, who else is qualified?!" Now, I'm not a big "Performance Appraisals" kinda guy. Just seeing the words makes me feel kinda sick. I've rarely written a performance appraisal that I didn't wish I could change later. Even when I was in the Army (yes I was in the Army for 12 years) we prepared many "Performance Appraisals" called Evaluation Reports. If it was a good soldier we scowered the last years calendar for good stuff he had done. If it was a bad soldier we scowered their folder for negative things to say. All said, it wasn't very constructive.

Now, to be fair, we did prepare many accurate appraisals. In the Army it's easy. You have a set number training events each year with a very specific set of tasks to be evaluated. You perform these same tasks year after year, event after event. They change sometimes but not often enough to really be noticed. You do well on the tasks.....you get a good appraisal.

Software, generally, is different. Every "new" feature is....well...NEW. In other words an unknown. In most cases, new features involve designing something that has never been designed. So, assuming that the unknown is hard to evaluate, appraising the performance of those working on the unknown must be hard as well...right?

Now that I have argued against "Performance Appraisals" I guess this post is finished...right? Wrong. The author, of said article, IMHO, missed the point. First, a "Performance Appraisal" is very difficult to administer, especially for someone not working closely with the appraisee (is that a word?). Second, software development, especially on agile teams, calls for a completely different approach to appraisals.

We need to accept that an agile team really works as a self organizing organism. No one person can be blamed for the success or failure of a team. That suggests that we really can't evaluate a single person based on the team's performance. So, do we just dispense with "Performance Appraisals"?

Before we answer that we must understand why we have "Performance Appraisals." For most the answer would be, "'cause HR told us to." The ideal answer is "to make the team better", the reality is "to determine who should get a raise" and in the real world both are probably correct. I guess we need "Performance Appraisals"? We'll assume so....

So, how do we design an appraisal that meets both needs (for betterment and promotions)? First, frame it around those competencies that are most valuable to the team. This is where I love XP....values and practices! I'm not going to list them here (it's late) but most intelligent humans could write a set of simple competencies based on the XP practices or the practices of their flavor of agility.

Second, with your handy set of competencies, have each member of your team evaluate themselves and each other member of the team (yes, I know the many reasons this is unpopular). Given the belief that the appraisor should work as close to the appraisee as possible then what bettermethod to use.

Third, sit down with each of them and discuss (yes face to face like human beings) how they did.
Be creative here, each team, depending on it's age and experience will require varying levels of "discussion". For example you may want to talk to them sitting side-by-side rather than "review board" style. Also, use good judgment. Don't just add up the numbers and say "tom" is 5 points better than "john". The whole point is to help the team be more productive. Basically, try to weed out the pettiness and let the person being appraised drive the discussion.

Finally, write an appraisal on each person, as the coach. Each organization will have some different way of recording them. In most cases you'll have little control over this. Either way, the point is to make the appraisal "meaningful" and give the person real insight. Most of all be fair and if you must write anything negative always recommend ways of improving.

Well, that's my rant today. Remember that this (and most of my diatribe) is only an opinion and likely one that could be changed. So, please, no flaming. But, intelligent arguments are not only accepted but encouraged!

5 Levels of Planning

I just read an interesting article that breaks down the Agile planning process into 5 levels:
(This is slightly modified)

1. The Vision
2. The Roadmap
3. The Next Release
4. The Next Iteration
5. The Next Day

Essentially, each agile planning session occurs in one of these levels. With each level dependent on it's predecessor. Ideally, you need a vision before you have a road map, road map before you plan the next release...you get the idea.

This really made sense to me. For years now I've used a somewhat complicated model to describe what it is we do. This model, oddly enough, can be easily distilled into this 5 level process.

In the table below I have identified each level of planning with WHAT being the expected artifact and the WHO generally describing the participants:

Level	What?	Who?
The Vision	A Vision Statement	Chickens
The Roadmap	Roadmap (duh)	Chickens
The Next Release	Release Plan/Stories/	Chickens/Pigs
The Next Iteration	Iteration Plan	Pigs
The Next Day	Engineering Tasks	Pigs

In the first level, The Vision, your goal is to provide a vision statement. Each shop will decide it's own requirement but I general recommend a condensed vision statement that can be easily hung on the wall as an "Information Radiator" (Remember...big visible charts). The Vision should be agreed upon by all stakeholders (chickens) and written without concern for technology. It should also convey some sense of purpose, since it is intended to drive the development process.

Once you have a Vision your team of chickens is ready to start working on The Roadmap. The Roadmap should also be brief enough to work well as a Big Visible Chart. Ideally your developers will keep the roadmap hung up in the shop. As items on the Roadmap are completed they can be checked off. The Roadmap will list, in order of importance, the vision for future releases. I recommend a bulleted list with each release defined by no more than 2 or 3 sentences (1 is better). As with the vision the roadmap should have a sense of purpose and as each item on the roadmap is completed you'll likely want to revisit the vision (it may have changed).

With your Roadmap written you'll want to start planning your Next Release. Planning a release is no simple task. You'll want the input of all your chickens and pigs. I'd recommend, first, relooking/revamping the vision for the next release; something may have changed. During release planning the chickens should be presenting stories to the pigs. These stories fall into 3 categories; enhancements, new features, bugs. Chickens should only consider enhancements and new features that fit into the vision for this release. Any stories considered a bug may be considered, regardless of the vision. Beware what you call a bug....just because you "don't like the way something works", does not make it a bug.

During Release Planning you should play the planning game. Chickens write a story, pigs estimate, chickens rewrite.... The final result should be a set of estimated stories with user values and generally well described acceptance tests. This may take several days. Pigs should feel free to throw back any story that can't be estimated. Chickens should make sure to stay within the vision for this release (or revise it).

Once you've hammered out your Release Plan, you're ready to begin working on the Next Iteration. Your original estimates of a story should not change unless the story is changed. Tracking progress should be done via your velocity (that is outside the scope of this article). At the beginning of each Iteration you'll select a set of stories to complete. This should be a sort of "do or die trying" commitment on the part of the Pigs. If you've tracked your velocity properly then making a commitment should be easy. Put story cards somewhere visible or use a large chart to track progress. No story should be started without an acceptance test and no story is complete until the test passes.

As each iteration progresses you'll constantly be planning The Next Day's work. Each day engineers/developers should pair up and select a story to work on. Pigs will sometimes hold design sessions with CRCs or write engineering tasks to remind themselves what needs to be done next. A coach should be available to cheer them on as the stories slowly disappear and the new builds start to appear. Always make big visible charts and information radiators to track progress of tests, builds and stories.

As you get into a rhythm your team should start to move through these stages naturally. For many companies distractions are a way of life. Try your hardest to insulate developers from these distractions and you'll see your vision realized much faster. If distractions are unavoidable them just realize that your velocity will suffer. As long as distractions are common your velocity should adequately adjust for them.

Problems with modules2 and CVSNT....
CVSNT has an interesting feature not found in the generic cvs server...."modules2". I've been using this feature for quite some time. It offers some interesting options when compared to the standard modules feature in cvs (and cvsnt). I am not going to cover all of it's features since those can be found on the cvsnt web site....suffices to say that modules2 allows you to define alias modules in cvs using a sort of regex language of expressions.

For a long time I had my guys using modules2 strictly for checkouts done by our build server. We could wrap up all the dependencies in cvs under a single "alias" module. So far one downfall has been that modules2 modules don't support the history command :P Oh well, it still made managing dependencies easy. I've always shied away from letting engineers use modules2 locally because of potential issues with check-ins and creating new modules. Recently some of the engineers (myself included) have started working directly from modules2 sandboxes. It's actually scary. Where do the new modules you create get built? What contracted agreement ensures the behavior will never change? In general most other cvsnt functionality is well documented and has a great deal of precedence to fall back on....modules2 has neither and it may be a mistake to depend on it when it really offers little more than convenience.

We have run into some trouble. One tool we use, pushok, to integrate cvs into Visual Studio tried to create a new module at the root of the modules2 module. Nothing was actually created. Problem is, what should the behavior be? How should these tools behave?

Has anyone else had trouble with modules2? Am I over-reacting? Right now it's my intent to switch our build servers over to using flat module checkouts. This will bring back the history command and speed things up. I am going to discuss the use of modules2 sandboxes the team and try to achieve some sort of consensus.

Update on 3 Jan 2007
Well, I've gone threw and replaced our modules2 definitions with a set of batch files for checking out our various "sandboxes". The batch files look something like this:
SET CVSROOT=:sspi:cvs.foobar.com:/FOO
SET NEWMODULEPATH=FOO.BAR.ALIAS
mkdir %NEWMODULEPATH%
cd %NEWMODULEPATH%
cvs co -P project1
cvs co -P project2
cvs co -P foobar34
cvs co -P common/lib
cvs co -P common/src

So far it's working out pretty well! To check out a set of modules we just run our batch file! More to follow...

Automate, you must!
Recently, at work, we rolled out a new version of one of our major web applications. Until recently rolling out changes was a major undertaking. Over the past few years it's evolved into a much more reliable system that takes about 11 minutes!



Here is a quick chronology of the evolution of our deployment system:

1. Pre-deployment system
   Believe it or not our earliest versions of this product (many years ago in a galaxy far far away...etc) had no deployment system, let alone automation. Unless you considered me or some other guy making a change and Ftping it to the live site for testing, a deployment system. These were carefree times with few users to be annoyed by the occasional bit of downtime. Also, we worked in mostly interpreted languages (ah the good old days!)
2. Let's get a testing server!
   As we grew and more users were around to pay attention to our untested new "features". We decided a test server was in order. The first iterations of the test server were basic configurations of the original application. Nothing was automated but at least things got tested...mostly.
3. Let's automate
   This was a though time. I made a lot of enemies during the year or so it took to automate deployments. No one wanted to spend the time required to automate the system. Even those who saw the value in it were afraid of those who did not. Luckily, my boss, a man with surprising good vision for these things, was a willing advocate for change. At this point I should mention that this was about the time I was placed in charge of several other software teams and I was proposing a sweeping change for them as well. Let's just say it was ugly.
4. Let's REALLY automate
   About the same time we were automating our builds and deployments using NAnt, I was really tied up in the school of thought that was trying to address where databases fit into the Agile methodology. Databases present many problems that aren't handled well by most agile practices. Historically you try to design databases that aren't going to be very volatile. Agile methods suggest that everything should be capable of some volatility! Refactor this, throw this other thing out...etc. So, my problem was this, how do we allow the database to be volatile and still keep data safe? Well, once again I shifted some our resources to non-moneymaking endeavors. We developed an in-house database installer system that automates the deployment or upgrade of our databases. It works quite well is likely one of the greatest timesavers we have!
5. Refactor the Automation
   Over the past few years now we've learned a few things. We've switched a few things around and reversed a few assumptions we've made in the past. Through perseverance we've become so confident in our tools that we have fully embraced automation. We intend to automate everything. We've written a few custom tasks for NAnt and discontinued the use of WIX for many things. Trying to use Windows Installer was painful and it wasn't always clear exactly what it was going to do. We decided it just wasn't worth the effort and stuck to a custom system. Another major shift for us was the realization that our build and deployment scripts should be able to run on any machine, not just the build server.

Along with automating the build and deployment we have automated the Unit Tests and Code Coverage. We are capable of automating the code documentation but frankly it takes way too long and the resulting documentation is somewhat daunting. I'm considering some ways of producing cookbook style documentation where you look up the task you're wanting to perform rather than looking up objects/methods.

We still need to automate our acceptance tests. This is something of a holy grail for me. We've played with FIT.Net but still haven't been able to fully commit to it. Either way we are on the right track and I think we'll only get better!

mmmm...code smell this has

"mmmm...code smell this has". I wish it was original but a co-worker of mine came up with it while we were making up a fictional Agile Jedi (yoda kinda). Some others were, "YAGNI this is" and "Refactor you must".
Having pulled our team, some kicking and screaming, into the Agile world. I have to wonder what has it gotten us? Also, what do we still need? Hopefully, over the next few posts, I can explore this. I know we're not perfect but I believe we're on the right track.


I'd like to start my blog by examining some of the lessons I have learned.

Lesson 1: You need a Coach!
This I can't stress enough! The agile approach depends so much on having a coach. Just like a viking war ship needs a guy beating a drum, an agile team needs a coach. The coach keeps the team on track. He Reinforces and evangelizes the agile (XP) practices. The coach is shield from scope creep (YAGNI). The coach is often the tie breaker. The coach even takes care of some of the more mundane aspects such as paperwork.

Lesson 2: Practice the Practices
After a year of forcing the issue on Test Driven Development (TDD), it hit us like a brick this really works! For us TDD became our design tool. User Stories and CRCs are what we work from. I'll cover the other practices later.

Lesson 3: Automate the crap out of everything!
We can't imagine not having an automated build and deployment system! In a later post I hope to discuss exactly how our build works but suffices to say it's cool. We can build a very complex web application, test it, update the database and deploy it to several load-balanced servers in about 11 minutes. Our CVS repository is tagged with every build and an archived package is kept. To accomplish this here are some of the tools we use: Luntbuild, NAnt, CVSNT, Psexec.

Lesson 4: Value the Values
Communication, Simplicity, Feedback, Courage. That's the subset we use. Above all we value the people over the process! With every new hire I take about 30 minutes to really explore what each value means. In later posts I intend to explore them here as well.

Lesson 5: Refactor
Get rid of redundancy! I really recommend Martin Fowler's book on refactoring! I also recommend Kerievsky's book "Refactoring to Patterns". The principle behind refactoring is to focus on getting something running THEN make it simpler cleaner, more readable. The point is that you must make time for the refactoring and it must happen often. We refactor our work everyday. Write something in the morning...refactor it in the afternoon.

Lesson 6: Keep User Stories simple
A user story should be something a User would write. Sometimes we HAVE to write the stories...it's a fact we haven't been able to get away from. IF an engineer HAS to write a story (man this really shouldn't happen) show it to someone who is not an engineer for a sanity check. Your user stories should not include words like web-service or phrases like "a class that can..." unless class is a classroom. Every story should have a user that wrote it. Every story should have a Acceptance Test as well.

Lesson 7: Use metaphors for your systems
This is an area we have been really bad...as the coach I blame myself. Your new software project shouldn't force users (and the team) to learn/invent a whole new set of jargon. The greatest (and most prolific) impediment I have seen to communication is jargon ambiguity. The project should have a common vocabulary that's easy to pick up. TRY VERY HARD TO MODEL IT AFTER SOMETHING COMMON IN THE REAL WORLD! You will save yourself a lot of confusion and lost hours.

Lesson 8: Don't over-engineer
Allow your project to evolve. Don't waste time on elaborate plans that will change anyway. Evolutionary design is the name of the game on agile teams. Have quick planning meetings every few weeks. Put the user stories away somewhere and work on/think about only one at a time. If you follow the practices (like Once and Only Once) you generally won't paint yourself into a corner and the time saved not future-proofing will make up for the few corners (very few) you find yourself in.

Lesson 9: Don't under-engineer
Yikes, am I bi-polar? Nope, just as you shouldn't over-engineer you should beware of under-engineering. Take time to spike solutions. Write tests up front and discuss all the stories with the team. Know responsibilities each story carries and how you will take care of them.

Lesson 10: Fail FAST
Fail Fast means write a failing test as soon as possible. Don't worry about the naming convention (that can be fixed when you refactor). Don't worry about inheritance or any other over-engineering concern. Just write a failing test as soon as possible. Self organizing teams are easily disorganized...fail fast will help keep a good pace. The team wants to be productive but sometimes the path is hidden. Chopping around in the bushes with a few failing tests will often reveal the hidden path.

Lesson 11: Embrace and Understand Pair Programming
On teams people will naturally pair. Let them find their own pairs. Don't continue to pair when it's being non-productive and always have a pair working on a single story. More than anything pairing (and a little coaching) refinforces courage.

Lesson 12: Be Courageous
Courage is very important. You can't be afraid to write tests first, to refactor or to ignore YAGNI. You can't be afraid to let your pair partner know when they are doing something the wrong way.

Lesson 13 (Added 01/29/08): Learn how to Write User Stories
I Actually discuss this here: Writing User Stories

I'm sure we've learned other lessons. On our team we make it rule that any story we can't fully cost is put on the back burner. It takes courage to tell a customer that you can't include a feature because the team can't agree on a solution. I say though, that in a world where so many software projects fail or go way over budget, it is better to know your risks earlier than later.