What to Record

Written in 2004... still current? This is one of my swiftly knocked-out ET Notes series.

A particular phrase has rung through my life as an experimenter. I can remember the day I first heard it, and it's followed me round ever since. Let me set the scene. It's Double Physics. I'm sitting in a classroom - not our usual one, with the high lab desks and the ticker-tape timers, but a smaller one. One where a precious video recorder can be connected to a jittery television without interference from crocodile clips and galvanometers. It's small, and we're all a bit perched and huddled. Not something thirteen-year-old boys enjoy, particularly ones with a touch of Physics in their souls.

On screen, a succession of experiments. Bits of metallic stuff are being dropped into dishes of water. Before each experiment, we're shown chemical symbols, the periodic table. The voice from the screen says 'Write it down'. We do. We're shown the weight of the stuff, its colour, the ambient temperature, the volume of the dish, the air pressure. Each time, the voice from the screen tells us to 'Write it down'. We do, navy-blue hardback lab notebooks balanced on knees and the tops of seatbacks. We write it all down. The stuff drops into the water, nothing happens. We write it down. The camera zooms in; nothing. We write it down. Another experiment, more stuff. Still nothing happens. Still we write it down.

And so we get used to the nothing. When the metal looks odd after a moment or two's submergence, we write it down. When a sheen of tiny bubbles gradually creeps over it, we write that down too. When the first, single bubble escapes its hold and rushes to the surface of the water, we write it down.

The next surrounds itself promptly with a silver sheath of gas, and as it bobbles on the bottom of the bowl, we realise we really should have used a stopwatch. The geekier and richer are using their digital watches. Another experiment fizzes like an aspirin, the next positively leaps about. We're shown more metal - it's yellow grey and skinned with oil - the voice tells us it is Cadmium. We write it down. There's barely a moment after it hits the surface of the water, and the bowl explodes. Steam rises, water pours from the shards, soaking the black cloth that covers the studio table. We write it down.

Oddly enough, I've no real idea if that last bit of stuff was cadmium. The choice of subject seems odd, now I think of it, for a Physics lesson. Perhaps it was Double Chemistry - but I've always hated Chemistry. I'm twenty-five years older, and writing it down hasn't helped me retain many of those facts at this distance. The lesson itself, however, has followed me round, whispering 'write it down' in labs and libraries, concerts and car journeys, wrapping my fingers around a pen or pushing them over a keyboard even as my eyelids drop and I'm called in to bed. I've used notepads, jotters, exercise books, Moleskines, dictaphones tape and digital, Palm Pilots, laptops. Everywhere I've gone, everything I've done, I've written it down.

My life, then, is filled with scraps of rubbish paper, tapes, files in obscure formats. It'll come as no surprise that I've been writing notes throughout my time as a tester. For me, recording what I do is fundamental to doing it. I believe that I do a better job, just because I'm making notes. My mind is clearer, my concentration better, my decisions more justified - and sometimes, more surprising. It's a pain to find that I'm halfway through something, and I've lost my notes. It's worse to find I've not been making any - because although common, writing stuff down is hardly my default behaviour. Last year, I made a loaf of sourdough bread every week for six months; not a note to be seen. I started to make notes - had to find paper, made less bread, got wet flour in the laptop - but the bread got better. Way better. Why did I kid myself by not bothering?

Sometimes, I teach people to test - and sometimes, I teach the systems benath the peculiar magic that is exploratory testing. Indeed, I teach session-based exploratory testing, and recording sessions seems to be a particular problem for my student explorers. I've coached good testers, who show me three lines of post-test scribble to describe ninety minutes of exploration. I've worked with interested and well-informed teams, with only a buglog to show for their efforts. I've had a class full of people look at their pretty session templates, and write not one thing - not a bug, not a plan, not a target for testing, not their name or a date in the labelled boxes at the top of the paper. It strikes me, at these points, that perhaps I'm getting something wrong in my teaching.

I guess the most immediate thing I want to explain at these moments is why one might want to keep good notes. Better still, how keeping good notes can help. Lets do that, just to tick them off:

• You think more precisely
• You're more likely to make decisions based on the information, not on habit or expectation
• You don't have to remember everything - just get it down and move on with a clear mind
• You'll remember more when you come back
• You can show someone else, any time you like
• You can show someone that you've actually been working
• Notes don't decay over time

...and more. I go through all this, and my students nod and seem to understand. Indeed, they often suggest the reasons themselves. All compelling, all important, none making but a dribble of difference.

Perhaps there are a bunch of 'why not's. If you loaded someone with truth serum and asked away (not something I've yet tried - but that serum's tricky stuff to administer in a classroom), what might a noteless tester say?

• I don't have time
• Note-taking disrupts my testing karma
• I don't want to get caught doing a bad job
• Doing the work is more interesting than keeping notes
• I don't think it helps, so I'm not going to try, even in this classroom, even after you've pleaded with me to give it a go, even after you've told me my every effort is worthless without the backup that notes provide

Plausible? Perhaps, though disappointing. But I have another idea. I think that some testers might not have had that voice in their head for most of their adult life, telling them to Write It Down. Some testers just haven't been Writing It Down. Indeed, it is possible, I believe rather patronisingly, that some testers really aren't altogether sure What to Write Down.

Me? I did the Physics until I finished growing up. Then I segued neatly into Testing, which I've done for years, too. I never really knew what to write down, but I wrote it down anyway. A quarter of century of notes. If you're a compulsive writer-down of unconsidered trifles, I suggest you need read no further. On the other hand, if you'd like a shortcut to my personal take on the Secret Stuff that should be Written Down, I crave your attention for a few paragraphs more.

If you make a plan, write it down. If you're just tootling along all planless, you need a strategy, an approach. A sticky note will do. There are no excuses - accept no substitutes.

You'll want to remember the actions you take, the data you use, your expectations, your observations - including the time. Don't necessarily limit yourself to exactly what you're testing - you're working in some kind of context. You'll get better at this over time; there's an instinct that comes with practice that lets you separate the wheat from the chaff. There's always going to be a bit of chaff.

Keep track of things that repeat. Even if nothing happens. Dullness is a virtue in most working systems. And without track of dullness, how will you notice . . .

Surprises. Is that a goat among the sheep? If you didn't expect it, it's worth writing down. If someone else wouldn't expect it, it's a bug. Perhaps you've seen an exploitation. Have you a hypothesis? Are you making a model? And when you've supported your hypothesis, found a potential bug, had a surprise, or the dullness is just too much to bear, you need to . . . .

Make a Decision - many people get so used to testing by instinct, or by the book, that they don't notice they're making decisions. Worse, they've no idea what the decisions might have been. Scripted testing can be decisionless, but decisions are key to exploration. When you decide to take a different approach, to try different data, or just to consciously do exactly the same thing again, but watching more closely this time, you're taking a decision. Make a quick note.

A lot to keep track of? Sure - but that's why you write it down. You can't keep track of all this stuff without a bit of paper by your side, Superman. Just as continuous test design has strange and positive effects on the quality of your code, continuous note-taking works wonders on your testing - and on your thinking. A bare minimum? I always have a spare moment for a bare minimum. For me; strategy, data, surprises, decisions. For you, something else. Keep notes, and you'll be there in no time. It's not hard, it's not dull, but it needs a little persistence, a little focus, a little discipline. So; if this article has triggered one new idea, a decision, an observation, a single spark of intent or insight, I urge you to stop reading, right now, take pen and paper and . . .

Write it down.