Disprove that the site doesn’t work

I was working at a client site this week and overheard a conversation that made my ears perk up.

Manager: Did you test the site?
Developer: Yeah. I clicked around through it and looked at the pages.
Manager: Did you try hitting the back button on all the pages?
Developer: (sheepish) No..
Manager: That’s not testing. You need to think about it backwards. Assume that the site doesn’t work and then try to disprove your hypothesis.
Developer: (confused) Huh..
Manager: I know it sounds weird. But try it. Assume it doesn’t work and then try to disprove yourself.
Developer: (cowed) Ok.. I don’t know if I can get to that today. Probably not tomorrow.
Manager: What else is on your plate?
Developer: Well, I have to do xyz for Bob, and abc for Tina. (etc etc)

To help put this in context, the developer apparently had made a site-wide change to a web application with hundreds of JSPs. He had touched almost every page. Now he is asked to “disprove that the site doesn’t work”, which if my logic works is equivalent to “prove the site works”.

Of course this isn’t possible.

Now in fairness to the manager, it appears he’s really asking the developer to do two things:

• Spend more time testing
• Be more creative in thinking of ways to test the system

The developer is probably thinking a few things:

• It appears you will hold me responsible if a regression slips in. Oh God I wish I hadn’t been tasked with this code change.
• Manual testing is really boring and I’m not totally sure what to look for.
• I have other stuff I have to do that is both more tangible and more interesting.

How can the manager be more supportive?

• Acknowledge the difficulty of the verification problem.
• Diffuse the responsibility. Regression testing is a huge job. It shouldn’t be the sole responsibility of this developer.
• Most importantly: Make test automation a core engineering practice in the group and encourage the developer to write tests to automate the cases he was thinking of. For example WebDriver now supports Firefox navigation automation (clicking back/next buttons). This work would be more interesting for the developer, and re-usable.

Asking the developer to perform a logical impossibility is not management.

Collaboration vs Interruption

There was a comment on the extremeprogramming Yahoo! group last week that I found insightful.

When two people share a common task to completion, communication between them is considered collaboration.
When two people work on a task in series, communication between them is often considered interruption.

I see this frequently on web application teams with a division between design and development. The two disciplines operate on separate tracks. Designers do visual comps of the screens and hand those off to the developers to implement. Invariably the developer needs assistance or clarification from the designer, who is now working on designing some other screen. The designer has to context switch, and may feel that time is being taken away from their current task.

This supports a few patterns of agile/lean development:

• Interdisciplinary teams. Build small teams with all the skills needed to implement a feature from start to finish.
• Work vertically in small batches. Develop one small feature from end to end. Then repeat.
• Optimize at the team level. Don’t try to sub-optimize the utilization of each team member. Let the team figure out how to best get the work done and measure their team velocity.

Don’t make a mockery of testing

If you’ve worked on a Java team that valued test automation, you’ve probably been exposed to EasyMock or JMock.

While well intended, these tools imply that teams should focus their testing efforts on granular unit tests that provide maximum code coverage. Often these teams do this at the expense of integration tests. Since time is always finite, we suggest you spend the majority of your testing time on integration tests. Only after you’ve exhausted all integration testing possibilities should you be tempted to use mocks.

(more…)