Ridding Your Garden of Bugs: Gardening the Backlog

I'm not sure if I agree with this "article" but, it is somewhat related to a discussion my boss (Dr. Rob Miller) and I were having earlier today. Ideally, we would like to focus every third (I just picked a number) release on "bug hunting" or finding and fixing bugs. Unfortunately, bugs are often a debt we live with for a long time, in the face of progress and new features. I have known few products that didn't have some glitch and the promise of new features often takes some of the edge off of these glitches.

Conversely, some bugs can leave users with such a bad taste that they never quite trust your software again. In the article above the author describes a bug that was reported fixed but still remains broken....after 2 years. Furthermore, as more bugs slip through the cracks a system can slowly become more and more unstable.

This suggests that, when planning product releases, stakeholders should balance the need for new features with the need for a stable system. I believe that, by better "gardening" a product's backlog, most products can avoid these pitfalls. So, here are a few points to consider when gardening your backlogs:

• Are stakeholders keeping and gardening personal backlogs? When I say, "personal backlog" I am referring to the features, enhancement and bugs that each stakeholder or program manager would like to see implemented. At every release planning meeting these "personal backlogs" should be gardened and brought to the table for consideration in the "product backlog".
  
• What have users been promised (explicitly or implicitly) and what is the impact of leaving those promises unfulfilled?
• Do you know how "real" users feel about your software? How do you collect this information? Sometimes, we get ONE very vocal user (see above). Sometimes we have a quiet majority. When should a user's/client's opinions impact the requirements for a release?
• When is a "bug hunting" expedition called for? Is there some threshold or event horizon? Do we have a good litmus test?
• Finally, is there a single decision maker (the one ringable neck) for the product backlog? Someone to take all of the personal backlogs and prioritize them. In XP this would be the On Site Customer and in SCRUM this would be the Product Owner. I suggest that it NEVER be your development team. This person maintains the backlog, the vision and the roadmap.
  

 It's not likely that you will squash all the bugs in any non-trivial system without writing some questionable code. So, you should take the time to test drive your software. Also, watch others "test drive" your software. This advice is not just for developers. In fact it's doubly important for non-developer stake holders and Decision Makers to "test-drive" the software. Kick the tires, check all the instruments. How's the suspension? Is it idling rough? Take it to a mechanic (engineer) and see if any maintenance is due (technology debt). Next time you prioritize your backlog you may have a different perspective.

Taking the time to garden your backlog will leave you smelling like roses. While just letting things grow out of control could have you digging in the weeds!

Profiling Nhibernate

I recently purchased 4 licenses of Ayenda's excellent Nhibernate Profiler for my office. The tool truly is amazing. The real time metrics provided by the profiler have already helped us improve the performance of our applications and identify some not very efficient database calls.

Here are a few features that we really like:

• Realtime view of queries being sent to the database. This information includes a stack trace and properly formated SQL! You also get to see if the entities were pulled from the query cache, level 2 cache or directly from the database. The SQL view can really expose some nasty databse calls.
  
• Number of sessions opened. The queries are broken down by session. You can see how many queries each session makes and a mirad of statistics.
• Unique Query statistics. NH Profiler offers an aggregate view of how often each query was called and the average time it took. This is invauable information for improving performance.

This tool offers a ton of other features that we haven't even touched. If you use Nhibernate or ActiveRecord (as we do) I highly recommend you pay the fee and get a copy. You'll also be helping out a brilliant programmer (Oren Eini).

AAAA Unit Testing

A while back Oren wrote about the Arrange, Act, Assert syntax for unit tests. I've seen several other discussions on this and I am not sure if he is the originator but I'll attribute him either way.

The syntax I most commonly use ressembles an Arrange, Assert, Act and Assert (AAAA) pattern.

• Arrange the system in it's initial configuration. Create any collaborators and mocks you may need. Initialize services.
• Assert that the initial state is correct. For me this is important since so many layer of dependencies exist in modern applications. This is analogous to a control group in the scientific method. In this step I would also test the inverse of later assertion such as the non-existence of objects that should only exist after the Act step.
• Act on the system by creating/deleting/updating new objects or invoking services.
• Assert again that the system is in it's expected state.

I've caught a lot of bugs using this method. Asserting the initial state may also be a good way to expose weakness' in the design.

Memcached for Monorail Update

Memcached for Monorail is a session replacement for the Castle::Monorail framework. It uses memcached on the back end for distributing the session across a web farm. I have added some additional notes about Memcached for Monorail. I have also uploaded a binary release. Although I recommend getting the source.

No time to discuss much more but feel free to ask questions and I'll answer them. The Memcached session provider is currently being used in a production environment with great success but it definitely has room for improvement. It's aimed squarely at extending the Castle::Monorail session so if you are looking for something more generic than please look else where...or consider switching to MVC with Castle::Monorail ;-)

Monorail installed on a 64bit OS

Installing Castle::Monorail on 64bit IIS7 proves to be much easier than I thought. You must use the 64bit handlers. To make this easy you can just add the handlers to your Web.Config:

  136   <system.webServer>

  137     <handlers>

  138       <add name="IconHandler" path="*.ico" verb="*" modules="StaticFileModule" resourceType="File" />      

  139       <add name="Monorail-64" 

  140            path="*" verb="*" 

  141            modules="IsapiModule" 

  142            scriptProcessor="%windir%\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll" 

  143            resourceType="Unspecified" />

  144       <add name="Monorail" 

  145            path="*" verb="*" 

  146            modules="IsapiModule" 

  147            scriptProcessor="%windir%\Microsoft.NET\Framework\v2.0.50727\aspnet_isapi.dll" 

  148            resourceType="Unspecified" />

  149     handlers>
150   system.webServer>

As you can see above, I have added handler mappings for both the 64bit and the 32bit. Notice that the 64bit is listed first, this is important. The 64bit script processor is:

%windir%\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll

A more detailed overview can be found at BitterCoder.

Javascript: Equality versus Identity

Javascript has 2 basic forms of comparison operators: === and == .== is the standard equality operator while === is the identity operator. The equality operator will perform casting and usually test both sides as strings. The identity operator compares the value each side the type, but it does not work like instanceof.

Recently we did some experiments with the different return values.

The Expressions

Expression	Result
"test" == "test"	true
"test" === "test"	true
String("test") === "test"	true
new String("test") == "test"	true
new String("test") === "test"	false
new String("test") === new String("test")	Before Firefox 3.0.1: true After: false
new String("test") === String("test")	false

Here are some type tests we performed

Expression	Type
typeof("test")	"string"
typeof(String("test"))	"string"
typeof(new String("test"))	"object"
typeof(function(){})	"function"

What Did we Learn?
First we see that a string is not always a string. Sometimes it's an object. It's easy to forget that Javascript still has the concept of primitives and objects. Integers, Strings and a few others are all primitives. When using the identity operator (===) you have to assume that it is comparing the value of the variable and it's type (as returned by typeof). Further compounding this is the fact that new String() does not type to a "STRING", while calling String as a function does.

What this all suggests is that when dealing with primatives you might as well use the equality operator (==). If you feel it's safer to use the identity operator (===) then you should use String() as a function and never use new String().

Hire the Right People

I believe my team of engineers has been very successful. This is no accident. I look for, and like to, surround myself with smart people. I'm not being egotistical. I don't consider myself the sharpest tool in the shed, but I do know a sharp tool (smart engineer) when I talk to one.

Hiring the right person is harder than it sounds. The top two impediments, in my experience, to hiring the right person have been "Hiring People that are Non-Threatening" and "knowing nothing about the job you are hiring for."

Non-threatening can mean a lot of things. A non-threatening person can be a yes-man. Yes-men will agree with you on everything and never make you uncomfortable by pointing out flaws in your plan. Yes-men are often hired on the basis of philosophy rather than actual skill. Non-threatening people can also be that dull-tool that won't outshine the person who hired them. In both cases the company is stuck paying wages to an under performer.

More often, though, I have seen poor hiring decisions made by managers that have no idea how to judge candidates for a job. These managers tend to rely heavily on applicants' resumes and sets of nebulous questions about work ethic or previous projects. Lacking a basic understanding of engineering practices and what makes a good engineer, these managers usually hire based on a feeling.

So, how do I hire....smart people? First, I give a basic skills test. I actually test each applicant on basic problem solving skills using pseudo code. If they do well on the test, then I send them a practical exercise. I look for three things in the practical exercise: how well they followed the directions, coding style, and does it work.

The test and exercise help me weed out those applicants whose time would be wasted on an interview. If I am happy with the test and the practical exercise, I schedule an interview. I'm also a true believer in team buy-in on all new hires so I get as many engineers involved in the interview as we can afford.

The interview includes some basic questions about relocating and what technologies the applicant is interested in, but the real meat of the interview is a set of problems the applicant must solve, interactively, on a white board. The problem solving progresses, "lecture style," with the applicant lecturing us on how the problems could best be solved.

Engineers are free to ask questions and the applicant is encouraged to show all his work. After the interview, I discuss the applicant with the other engineers and prepare for the next interview.

Once we have an adequate pool, we all sit down together to review, compare and contrast all the applicants.

Watin Installed

I've spent the last few days getting our testers up to speed on Watin. Watin is an Automated Web Acceptance Test Tool We are using Watin CTP v2.0. So far I have learned a few things about Watin that I didn't know when I wrote my original Watin Review. If you are completely unfamiliar with Watin I recommend you read my review first.

Our descision to go with Watin was based on 4 factors. (1) Watin handles AJAX applications very well, (2) Watin doesn't inject Javascript into your code, (3) Watin supports Firefox (Not 3.0 as of this writing) and (4) it's open source (we can mod the code).

Watin Smarts

Watin has a good set of algorithms for handling the WaitForLoad stuff. WaitForLoadTimeout is the amount of time the underlying framework should wait for a page to load even though you have asked the framework to find an element.

Example:

browser.goto("http://www.google.com");
browser.Button(Find.ByValue("submit")).Click();

In the example above the thread will reach the 2nd line before the page finishes loading. Watin is smart enough to wait for the page to load before clicking the button. This behavior is controled by the WaitForLoadTimeout property.

Nice Syntax
The Syntax for finding elements on a page is pretty nice. Here is an example:

 browser.Button(Find.ByValue("submit").And(Find.ByClass("pretty-btn")).Click();

The "Find" constraint has both an "AND" and an "OR" method. Both take a single Find as an argument. Testers will prefer this syntax to the more arcane XPath style syntax but I would have preferred something like this:

browser.Button.Find.ByValue("submit").And.ByClass("pretty-button").Click();

Either way the Find constraint offers many ByXXX methods for do all sorts of neat tricks and if need be you can drop to regular expressions (well coded sites shouldn't require that). I also appreciate the HTML mappings syntax.

Firefox Support
You'll notice that the firefox support depends on a firefox plugin called jssh. JSSH opens up a telnet server that allows Watin (or any other framework) to issue automation commands that it then forwards to Firefox. Watin 2.0 comes with a version of JSSH in it's Mozilla folder. This version only works on Firefox 2.x. We have tried several "fixed" version and we have tried turning off compatibility and security checks in Firefox 3.x. All attempts to run JSSH failed. In Firefox 2.x we experienced great success. So, I have no doubt that support will come soon.

Either way if you plan to support multiple browsers then you should use the BrowserFactory class. It has a create method that returns an IBrowser instance based on a BrowserType enum. In this way you can use the same tests for both Firefox and IE.

Final Thoughts
One of the most attractive things to me, about Watin, is the fact that the tests live in source code along with all our other tests. They run using Nunit and the same tool set we use for Unit Testing.

I also like that Watin does not want to inject javascript into our code. So far our testers seem excited and I hope we can keep up the momentum.