Ah… data values within a range. So many possibilities for fun. Ran into one of those today while reviewing a potential work item being proposed. As presented:
Add "Length of Incarceration" range
Select from a list
* Less than 1 year
* 1-3 years
* 3-5 years
* 5-10 years
* 10-20 years
* More than 20 years
Spot the problem? What happens if the length of incarceration was 1, 3, 5, or 10 years? Which of the two ranges that include these values should be chosen?
Presumably we’re grouping this data into ranges for reporting purposes.
At what point does it become statistically significant that given a situation where an offender spent 5 years incarcerated, Elizabeth always chooses 3-5 years, Ross always chooses 5-10 years, and Eduardo sometimes chooses one range and sometimes chooses another?
In any system with user-entered data, GIGO can apply, but let’s help the users by designing a system which makes it harder to input garbage.
The longer I work as a tester, the more that I realize that to provide the most value for my team I need to not only be able to report on what’s happening, but also to be able to report in an intelligent fashion, synthesizing what we’ve seen in the software along with supporting information to provide context.
Last week a fellow tester reported an interesting bug in a web application’s login form: after he entered his username and then his password, when he hit Enter to submit the form, the application opened in an entirely new browser window.
This wasn’t behavior built into the system design… when operating normally, the software should have presented him with the application’s landing page after login. I wasn’t able to immediately reproduce the behavior, and none of the system’s users had reported the issue. Yet this tester insisted that it happened nearly every time he accessed the application.
You Know This One
Even without knowing our application… you probably have information to solve this puzzle.
Once I figured out what was going on, I decided to see if I could lead folks to the same conclusion.
I asked the tester: “Is the last letter of your password a capital letter?” He said no… but it was a symbol. “A symbol accessed via the Shift key on your keyboard?” Yep.
What happens when you click a link in Chrome while you hold down the shift key?
You get a new window.
So when you’re still holding down Shift from the last character of your password, then hit Enter which activates the form submission… boom. New window.
Testing is Information with Context
As testers, we provide information. In this case, we can provide more information beyond “sometimes this thing happens.” We can provide information of “This thing will happen every time, but in this set of circumstances.” That’s useful.
As a software tester, I have great expectations.
- I expect that as I test a feature that the functionality will match my understanding of the user story and related discussions
- I expect that the software will have an intuitive user interface
- I expect that the software will be consistent with itself, with other similar applications we’ve developed, and with industry standards
Sometimes my expectations are met. Sometimes I find that the software behaves differently than (I) expected.
When behavior differs from expectations, have I found a bug? Perhaps. Or perhaps my expectations were wrong.
When software behavior differs from my expectation as a tester, more often than not it can be a conversation starter for further discussion. It often means it’s time for a conversation either with the product owner to see if my expectations are in line with his expectations as to the functionality of the system. Maybe it’s time for a conversation with the developer to figure out if her expectations differed when she wrote the code that’s not behaving as I expected.
- I expected the client list screen to be sorted by last name, because hey, that makes sense, right? But perhaps the product owner told the developer that they wanted it sorted by last activity date instead.
- Perhaps the data field on the screen is allowing for a different sort of input than was noted in the user story. Rather than assuming the developer is incompetent, I can ask if the desired behavior changed beyond a (non-updated) user story.
- Often, especially with the non-standard use case, I run into an error situation that’s handled in what seems like a strange way. Developers on my team have learned what’s usually coming when I start a conversation with “What would you expect to happen when…” and lay out the scenario. Often I’ve discovered a workflow or use case that hadn’t been foreseen, so my expectation was based on something the developer hadn’t even considered.
Sometimes my expectations are “correct.” Sometimes the desired behavior is different than my expectations.
Expectations lead to revelations which lead to conversations, which may or may not lead to work to change software behavior. News flash: I’m not always right.
Related reading: the Huh? Really? So? section of my notes from James Bach’s workshop at STARWEST last year.
File this one in the “who the hell would’ve ever thought this was the correct behavior?” category…
Our dev team is moving into the bootstrap world, which means that we’re again learning how to manage date fields and date pickers.
Being the good tester that I am, I tried entering February 29, 2014 as test data. And the date field automatically changed to March 1st, 2014. Hm.
February 30th led to the date field changing to March 2nd. Well this is peculiar. It seems less-than-ideal that it would change the date without informing the user.
Let’s give it a really wacky date. What happens when I input 06/72/2014?
It changes it to 8/11/2014. Because, you know, August 11th is the proper way of representing June 72nd. We can count any number of arbitrary days from the beginning of a month.