The longer I work as a tester, the more that I realize that to provide the most value for my team I need to not only be able to report on what’s happening, but also to be able to report in an intelligent fashion, synthesizing what we’ve seen in the software along with supporting information to provide context.
Last week a fellow tester reported an interesting bug in a web application’s login form: after he entered his username and then his password, when he hit Enter to submit the form, the application opened in an entirely new browser window.
This wasn’t behavior built into the system design… when operating normally, the software should have presented him with the application’s landing page after login. I wasn’t able to immediately reproduce the behavior, and none of the system’s users had reported the issue. Yet this tester insisted that it happened nearly every time he accessed the application.
You Know This One
Even without knowing our application… you probably have information to solve this puzzle.
Once I figured out what was going on, I decided to see if I could lead folks to the same conclusion.
I asked the tester: “Is the last letter of your password a capital letter?” He said no… but it was a symbol. “A symbol accessed via the Shift key on your keyboard?” Yep.
What happens when you click a link in Chrome while you hold down the shift key?
You get a new window.
So when you’re still holding down Shift from the last character of your password, then hit Enter which activates the form submission… boom. New window.
Testing is Information with Context
As testers, we provide information. In this case, we can provide more information beyond “sometimes this thing happens.” We can provide information of “This thing will happen every time, but in this set of circumstances.” That’s useful.
As a software tester, I have great expectations.
- I expect that as I test a feature that the functionality will match my understanding of the user story and related discussions
- I expect that the software will have an intuitive user interface
- I expect that the software will be consistent with itself, with other similar applications we’ve developed, and with industry standards
Sometimes my expectations are met. Sometimes I find that the software behaves differently than (I) expected.
When behavior differs from expectations, have I found a bug? Perhaps. Or perhaps my expectations were wrong.
When software behavior differs from my expectation as a tester, more often than not it can be a conversation starter for further discussion. It often means it’s time for a conversation either with the product owner to see if my expectations are in line with his expectations as to the functionality of the system. Maybe it’s time for a conversation with the developer to figure out if her expectations differed when she wrote the code that’s not behaving as I expected.
- I expected the client list screen to be sorted by last name, because hey, that makes sense, right? But perhaps the product owner told the developer that they wanted it sorted by last activity date instead.
- Perhaps the data field on the screen is allowing for a different sort of input than was noted in the user story. Rather than assuming the developer is incompetent, I can ask if the desired behavior changed beyond a (non-updated) user story.
- Often, especially with the non-standard use case, I run into an error situation that’s handled in what seems like a strange way. Developers on my team have learned what’s usually coming when I start a conversation with “What would you expect to happen when…” and lay out the scenario. Often I’ve discovered a workflow or use case that hadn’t been foreseen, so my expectation was based on something the developer hadn’t even considered.
Sometimes my expectations are “correct.” Sometimes the desired behavior is different than my expectations.
Expectations lead to revelations which lead to conversations, which may or may not lead to work to change software behavior. News flash: I’m not always right.
Related reading: the Huh? Really? So? section of my notes from James Bach’s workshop at STARWEST last year.