A Range of Errors

Ah… data values within a range. So many possibilities for fun. Ran into one of those today while reviewing a potential work item being proposed. As presented:

Add "Length of Incarceration" range

Select from a list
* Less than 1 year
* 1-3 years
* 3-5 years
* 5-10 years
* 10-20 years
* More than 20 years

Spot the problem? What happens if the length of incarceration was 1, 3, 5, or 10 years? Which of the two ranges that include these values should be chosen?

Presumably we’re grouping this data into ranges for reporting purposes.

At what point does it become statistically significant that given a situation where an offender spent 5 years incarcerated, Elizabeth always chooses 3-5 years, Ross always chooses 5-10 years, and Eduardo sometimes chooses one range and sometimes chooses another?

In any system with user-entered data, GIGO can apply, but let’s help the users by designing a system which makes it harder to input garbage.

The Tester Must Understand the Stack

As testers, we test a piece of software against a variety of both written and unwritten functional and nonfunctional requirements.

One of the key tools in testing that software is having an understanding of the environment under which that software will run. We need to understand the pieces of the stack. We don’t need to be experts on every bit of it, but we need to understand what components are involved, and how they work together. I’ve found that often may folks will forget about parts of the stack, which can lead to red herrings or bad information when describing a situation or troubleshooting a problem.

Full Stack Testing, by Flickr user prettyinprint

For example, in my environment I’m usually testing web applications. For a given application, the following things are in place and need to be working correctly:

  • a web server (virtual) running on a host machine (physical)
  • a database server (virtual) running on a host machine (physical)
  • external (network) storage for the database
  • a working network between all of these servers and the client machine
  • DNS servers translating names into IP addresses to access the servers
  • a load balancer managing traffic between multiple web servers
  • IIS working correctly on the application server
  • zero or more APIs that are available and working
  • a web browser on the client machine that supports JavaScript

Even with this list, there’s a chance I’ve omitted something. Depending on the testing being performed or the issue being investigated, there’s a chance that any one of these components could be the culprit. Don’t forget about the entire stack.

Image by prettyinprint, used under Creative Commons licensing

Testers Don’t Just Test the Code

Kate Falanga chimed in recently with some thoughts around titles for testers, QA folks, and the like in Exploring Testing Titles in Agile. She lays out a few good reasons why the term Quality Assurance is a bad one, mainly that the role can’t really assure quality. I believe this. Heck, in the tagline on this site I refer to “(The myth of) Quality Assurance.”

She then outlines why she doesn’t like the title of Tester, feeling that it’s not broad enough to reflect all of the work that we do, and that it’s reactive:

It gives a very reactive rather than proactive connotation. If someone is a Tester then it is assumed that something needs to be tested. If you don’t have anything to be tested then why do you need a Tester? Why include them in planning or the project at all until you have something for them to do?

Quality Assurance / Testers / Job Title Adventures

The bad assumption here is that code is the only thing being tested, and that testing is the only thing done by a tester. Sure, once there’s code being written, a tester will probably spend a majority of her time exercising that code, but the tester participates in testing activities prior to the code. Time spent in requirements discussions helps the team write better requirements or user stories. Time spent learning about the business environment or the end user’s use cases will help the tester get into the right mindset for testing activities.

These activities aren’t testing in the sense of testing new code that’s been writing, but they’re testing activities. If testing allows us to learn information about the software being tested, and we use that information to improve product quality, all methods of learning could be considered test activities, could they not?

Do we continue the search for a better title than Tester, or do we work to help the broader software industry understand that Tester doesn’t just mean exercising code changes?

Image by Ruth Hartnop, used under Creative Commons licensing

Curiosity Killed the Cat, But It’s the Lifeblood of a Tester

My employer partners with a local university as part of an internship program; computer science students have an opportunity to participate in a series of six month paid internships with local software development groups. As a result, we’re now about three weeks into working with our latest intern. We’ve had two previous testing interns.

It’s interesting to see how they begin testing. With each of them I’ve set things up with an introduction to context driven testing and the ideas of software exploration and working with various heuristics to exercise the program.

It’s interesting to note if the new intern has an innate curiosity to explore.

Our current intern started at the beginning of the month. On her first day, as she began to explore one of our applications, she caught a bug that appeared when you altered a URL query string.

Curiosity: the lifeblood of a tester.

Management Shouldn’t Make Bug Count Jokes

A couple of years ago, a new senior manager began working at our organization (he was my boss’ boss). Shortly after his arrival, he came around to our group to introduce himself and meet the various members of the team.

He came into the room that houses my small dev team (5-6 people) on one side with another similar team on the far side of the room. He’s meets the other team, including their QA person. Then he meets our team and I’m introduced as our QA guy. He then quips:

So… do you guys keep score and see who has the least bugs?


Was it a joke? I’m not sure. He wasn’t laughing. And neither was I.

Yes, there is value in tracking some statistics, but what sort of impression does a tester get when the first interaction with a senior manager is that manager asking about bug stat competitions?

What are the odds that this person knows much about software testing? And if this person is going to evaluate software based on likely-bogus bug statistics, what other bad metrics is he going to use to make decisions?

Incidentally, said manager chose to leave the organization just a few weeks after being hired. Hopefully he found somewhere that’s a better fit.

Tip for management: testers probably are going to find your bug count jokes more scary than funny.

On Explaining Yourself

We’ve been doing a fair amount of interviewing at my current employer, primarily looking for software developers to work on the same team where I break test software.

I find the interview process fascinating. I maintain that the most important skill for a member of a software development team is the ability to communicate effectively (both orally and in writing) and our interview process exercises those abilities.

I Care About the Why Over the What

Ideas, Problems, SolutionsSeveral of our interview questions are fairly open ended. There’s not a right answer (although occasionally there is a clearly wrong one). We aren’t as concerned with which of the nearly-unlimited potentially-right answers you give us, but more concerned with how you explain said answer. In fact, I’d rather hear a questionable answer with a solid explanation than a textbook-perfect answer but no ability to explain how one reached that conclusion.

That Explanation Translates to Software Design

We don’t want folks who can explain things just for the sake of explaining things… as we design software, those explanations are key. It seems that weekly (if not even more often) I’m engaged in a discussion with a developer and we talk through how a software component will behave. Perhaps they came up with the design on their own or perhaps it came about as a result of discussions with others.

Out of each one of these discussions, things evolve. Perhaps I offer an insight that changes the course of their design. Maybe they explain the rationale and my vision becomes clearer. Sometimes I play the role of the duck and they have an epiphany on their own.

Rarely does excellent software come from one developer working entirely on his or her own. Good software generally comes from a team… a team that can communicate and explain ideas.

photo by Flickr user poportis, used under Creative Commons licensing