An unscripted unit test tool


Yesterday I ran in to a bit of a debugger bashing with a couple of people disowning the debugger with statements like “I noticed as soon as my students got access to a debugger [rather than just ‘print’ statements], their design very quickly deteriorated” and “I often feel that it’s much faster to reason around potential causes for bugs than to search for them in a debugger”. Now, these statements are undoubtedly true, but I am a huge fan of the debugger and will use it extensively in just about any programming task that I do. Admittedly, I do no longer do programing as my main profession, but I still code small tool in various languages, so this dialog got me thinking on why I find it useful.

I tried to think about how I use the debugger when I’m coding and then in struck me; I use it as a tool for unscripted unit testing. I have the feeling that many people, when they think of unit testing, they think of small pieces of code that check some other code. I also do this and find if very useful, especially as a way to constantly refactor and improve my design. However, these small snippets of code can be seen as test script and although there is value in these test scripts, there is much more to testing. Also, the code snippets do not really test the code, just check it – I want to test it.

What I do when I code is after every so few lines of new code, I have a micro test-session where I explore the new code where I can really test it beyond the (simple) checks of my scripted unit tests. For me, this adds invaluable information about the code that I have just implemented, that I feel I could not get otherwise.

As with most tools, their value depend on how you use them and with the debugger, it might not be a good tool for software design or for investigating bugs, but as a tool for unscripted unit testing, I find it to be great.


A great testing experience

I have been working with test now for some years, and for each year I realize how much more there is to learn, but also that inspiration is important to maintain.

One of my best testing experiences which gave me a lot of new knowledge and inspiration was the SWET conference, Swedish Workshop on Exploratory Testing. So far three SWETs have been held, and I have had the pleasure of attending SWET2 (1) and SWET3 (2).

First of all, it was a feeling of honor to attend SWET2 because it is a conference to which you get invited. At SWET3, Henrik Andersson and I were organizing the conference and acting as facilitators.
Each SWET has a theme on which the presentations and discussions should focus, e.g. for SWET2 the theme was “Test Planning and Status Reporting for Exploratory Testing”, at SWET3 the theme was “Teaching Testing”. All conference participants need to send in an abstract on the subject. Besides relating to the conference theme, the abstracts should be based on personal experiences. The organizers then create a prioritized list of abstracts in which order the presentations will be held.

At SWET, the presentations are not very long, about 20 minutes, but the questions and comments may continue as long as it is interesting. This form is based on LAWST, http://lawst.com/. Each participant gets a set of four cards to be used for getting attention after the presentations; “New question”, “Comment on a previous question”, “Urgent attention” and “Rat hole”. “Rat hole means that a discussion is stuck and is not going forward. One participant, normally one of the organizers, is acting as facilitator and keeping track of by whom and in which order cards are raised.
Because each presentation with questions and comments may take quite a while, it is usually only the 3-4 first presentations on the list that actually are held.

The first thing that struck me when I arrived to SWET2 was the relaxed and friendly atmosphere. I was a bit nervous, but that feeling quickly went away. Some of us arrived the night before the conference and we had a nice dinner and great time together. We talked about a lot of things, including test of course. At SWET3, it was great to once again meet some of the testers from SWET2 and exciting to make some new acquaintances.

When the actual conference starts, each person in the room gets to introduce themselves shortly and to say something about their expectations for the conference. Already at this time, I noticed that focus and concentration sharpened among us. We all wanted to get a lot out of the meeting. It was a group of testers who are very serious and enthusiastic about their profession.

All of the presentations were of high quality and I found it interesting and inspiring to listen to these skilled testers telling us about their experiences. Besides that, I also got a lot of good tips and I could relate to own experiences and how I can change and improve my way of working. During the presentations, the listeners take notes to remember questions and comments they come up with.
You may think that the idea of focusing on a theme would give small variations in the different presentations, but on the contrary; there were big variations. The fact is that each presentation gave input to so many questions, and each question resulted in comments and new questions and so on. The queue of questions and comments became very long so it took some time before one got the chance to say something. Sometimes, the comments and questions actually drifted too far away from the presentation, in which case the facilitator could interrupt and remind the participants to stay focus.
One thing I noticed was that discussions started off at quite a high level, in that sense that some things were already somehow understood, because all of us already has studied Exploratory Testing and we also share many valuations about testing. The questions and comments took a lot of time and this is a situation which I rarely have experienced before, the possibility to discuss something without time pressure. I would love to do this more in life.
But it was not just getting answers for you questions that gave a value; it was also a great forum for practicing the ability to listen, to questioning, to argument and to receive feedback on my own ideas.

SWET also includes a session called “Lightning Talks”. During this session, any one of the attendances is allowed to do a very short presentation about any test related subject that they want to share with the others. There is not very much time for questions during this session, but I managed to take some interesting notes which I brought with me from the conference.

Well, after dinner, the first “working day” was over, but the rest of the evening was just as rewarding, maybe a bit less formal. Lots of interesting discussions, nice music and a few good beers. When the conference ended after lunch the second day, I was a bit tired but really glad and inspired. I had learned a lot and made new friends within the Exploratory Testing community.

1. At SWET2, we were 15 persons; Henrik Andersson, Azin Bergman, Sigurdur Birgisson, Rikard Edgren, Henrik Emilsson, Ola Hyltén, Martin Jansson, Johan Jonasson, Saam Koroorian, Simon Morley, Torbjörn Ryber, Fredrik Scheja, Christin Wiedemann, Steve Öberg and myself Robert Bergqvist.
2. At SWET3, we were 11 persons; Anders Claesson, Henrik Andersson, Johan Jonasson, Maria Kedemo, Ola Hyltén, Oscar Cosmo, Petter Mattsson, Rikard Edgren, Sigurdur Birgisson, Simon Morley and myself Robert Bergqvist.