Irrefragable

The summer Engineering Forum at Microsoft was held back at the start of June. It’s an annual week-long event where technical staff from across the company present lectures and hold roundtable discussions on a variety of engineering topics. All presentations were organized into topical groups called tracks. For example, there was a designing and testing for reliability track and an effective measurement track.

I attended several days’ worth of presentations and roundtables across multiple tracks. While I learned a lot in the presentations, the thing I’ve spent the most time thinking about was actually a free handout I got on the morning of the first day.

The forum event staff printed up three different styles of notepad to distribute to attendees—one each for developers, testers, and program managers. Each notepad page was bordered by a series of short sayings related to the target discipline. As a tester I picked up the test notepad and then spent the first several minute of the morning session distractedly reading around the edges of the page. I don’t know where the sayings came from originally, or who wrote them, but the sayings on the notepad have proven to be a lot of fun to ponder.

Each saying is short and highly domain-specific. I think of them as test aphorisms—pithy sayings to be unpacked and thought about from many directions. They each make some sort of claim about the right way to test, but sometimes it can be hard to see what justifies the claim, or even what the claim is to begin with.

The first four test aphorisms are as follows. I have interspersed some initial questions to help direct my thinking for each in the future.

1. Trust, but verify.
   1. What are we trusting?
   2. What constitutes verification?
2. Test functionality and the scenario.
   1. What constitutes testing functionality or testing a scenario in this context?
   2. What distinguishes one from the other?
   3. Why is that distinction important?
3. Test the design.
   1. How does one test a design?
   2. Is it the same thing as assessing the testability of a design?
4. It’s a model all the way down.
   1. Is the object of the test the model?
   2. Is the test itself the model?
   3. Is it both—a model of a model?

Some more general questions:

1. What model of testing do these aphorisms suggest?
2. What is the justification for the claims implicit in each of them?
3. What are the weaknesses in these claims?
4. What actions do these aphorisms entail for the practice of testing?
5. Can we test the practical value of each of these aphorisms? How?