Disprove that the site doesn't work

Posted: February 28, 2008

previous post - next post

I was working at a client site this week and overheard a conversation that made my ears perk up.

Manager: Did you test the site? Developer: Yeah. I clicked around through it and looked at the pages. Manager: Did you try hitting the back button on all the pages?
Developer: (sheepish) No.. Manager: That's not testing. You need to think about it backwards. Assume that the site doesn't work and then try to disprove your hypothesis.
Developer: (confused) Huh.. Manager: I know it sounds weird. But try it. Assume it doesn't work and then try to disprove yourself. Developer: (cowed) Ok.. I don't know if I can get to that today. Probably not tomorrow. Manager: What else is on your plate? Developer: Well, I have to do xyz for Bob, and abc for Tina. (etc etc)

To help put this in context, the developer apparently had made a site-wide change to a web application with hundreds of JSPs. He had touched almost every page. Now he is asked to "disprove that the site doesn't work", which if my logic works is equivalent to "prove the site works".

Of course this isn't possible.

Now in fairness to the manager, it appears he's really asking the developer to do two things:

Spend more time testing
Be more creative in thinking of ways to test the system

The developer is probably thinking a few things:

It appears you will hold me responsible if a regression slips in. Oh God I wish I hadn't been tasked with this code change.
Manual testing is really boring and I'm not totally sure what to look for.
I have other stuff I have to do that is both more tangible and more interesting.

How can the manager be more supportive?

Acknowledge the difficulty of the verification problem.
Diffuse the responsibility. Regression testing is a huge job. It shouldn't be the sole responsibility of this developer.
Most importantly: Make test automation a core engineering practice in the group and encourage the developer to write tests to automate the cases he was thinking of. For example WebDriver now supports Firefox navigation automation (clicking back/next buttons). This work would be more interesting for the developer, and re-usable.

Asking the developer to perform a logical impossibility is not management.