CAST 2017 Sketchnotes: Mary Thorn – Tales of Agile Testers

Sketchnotes from CAST 2017 keynote by Mary Thorn – Tales of Agile Testers. You’re welcome to share, please credit Aaron Hockley with a link back to – thanks!

CAST 2017: Mary Thorn

View / Download as PDF


Do You Ship a Steaming Pile of Turd if the Customer Doesn’t Argue?

Back in ye olden days of waterfalls, our requirements-gathering efforts would lead to reams of specifications. We’d account for each pixel, specifying the screen position of various labels. Our data entry forms would have a tab order laid out explicitly. The text of every error message would be wordsmithed to (alleged) perfection.

We all thought we knew what we wanted up front. We’d write a big pile of documentation to show just what we needed built, and how. Developers couldn’t be trusted to figure these things out on their own…

And then we all got a dose of reality, as we realized that quantity of documentation and specifications didn’t really correlate to system quality. Despite our best effort to make it difficult to change (hi there mister change control board), change still occurred. And it was expensive, since the process was built around what we thought was stability.

Enter Agile

But hey, here comes the agile world, where we work iteratively with our product owners and end users to create something that meets their needs, even as those needs change throughout the duration of development.

We don’t write as many detailed specifications, because we value working software over comprehensive documentation[1], and some of that time spent documenting is probably better spent writing code. But as we cut out the documentation, we leave more decisions for developers to evaluate and decide what to build.

This is generally a good thing. Study after study shows that agile methods usually produce better software, and they often do it faster.

But… and there’s always a but… what about the little quality details?

Things a User Won’t Ask For

Product owners, business customers, and end users are pretty good at figuring out the big obvious functionality for a piece of software. Their domain knowledge drives the big features, and when we get it right as a software team, the result makes their lives better.

I’ve found that the product owner or customer often won’t ask for little things that can increase the quality of a piece of software. When is the last time you heard a customer explicitly ask for:

  • the web application to use consistent <title> tags such that the application and particular screen are identified in a consistent manner
  • buttons for actions (Submit, Cancel) are always placed in the same place… for example, cancel on the left and submit on the right
  • consistent behavior with regards to the maximum length for data entered into text boxes, and handling cases when a user might circumvent this (copy and paste, perhaps?)
  • the Enter key performing a default action such as a search

You get the idea… I could list fifty other similar attributes. These are behaviors or patterns that are hallmarks of quality software. Some make it more intuitive to use. Some help prevent the user from losing some data. Each of these things makes the software just a bit better and makes it so that a user is less likely to use the name of your program as an expletive.

Customer Acceptance is not the Be-All, End-All

We recently had a lively discussion within our development team about setting a quality standard for our products. A couple of our developers argued for the position that if the customer accepts the work, then it’s good enough, and anything we did above and beyond the minimum needed for customer acceptance was “gold plating” and excessive.

I disagree. Just because a customer is willing to call something “good enough” doesn’t necessarily mean it’s good enough. Much like we trust the customer or product owner to come to us and contribute their domain knowledge for the problem we’re solving, as software professionals we also bring our domain knowledge to the table… in the domain of software quality. Our experience, both as individuals and as an industry, can bring great things to the table in terms of usability, reliability, stability, and other similar factors. Customers are often only able to articulate that they want the software to be “easy to use” or “user friendly.” Our expertise translates that into work tasks that can be verified by testing.

We ought to strive for the highest quality possible, even when the customer doesn’t explicitly ask for it. Establishing a set of quality standards for your software is a worthy effort and can help everyone on the product team have a clearer understanding of what is meant by quality software.

Agile Consistency Without Reams of Specs

My first actual article on this site was explaining my take on three tiers of consistency within software and assuring that a software program provides an experience where a user can expect things to be consistently intuitive and behave based on the user’s expectations.

Consistency in the Waterfall

Putting on the old man costume, one might muse a bit about the good old days of waterfall software development when everything worked out perfectly because we had reams of specifications for each project. The specifications must be right because they had sign-off by someone important and we had a big stack of change control paperwork for each modification!

Stepping off the sarcastic soapbox for a moment, I will note that implementing consistent behavior wasn’t likely easier in the waterfall days, but it was probably less ambiguous. One could always look up requirement where it specified how something should work and compare the software to the letter of the requirement.

The Agile Consistency Dilemma

When we transition to a more agile way of building software, where we value

Working software over comprehensive documentation

how do we ensure consistency in the absence of pile of specifications?

Consistency by Communication and Familiarization

Looking at the three aspects of consistency I wrote about previously, we have:

  • Consistency within a given application
  • Consistency across applications within the line of business
  • Consistency with industry norms and trends

Within an application or across a line of business, we can help facilitate consistency by enabling and facilitating communication within the development team. Developers that chat and communicate about how they’re solving problems are likely to solve those problems in similar fashion.

Familiarization with products can help prevent reinvention of the wheel. If your company has six products, a developer ought to have end-user knowledge of all six such that when implementing a new feature they would know how that feature, interface, or workflow might already be in place in a different program. This seems like it would be obvious, but I can recall several cases where I’ve raised an issue of inconsistency across products (by the same organization) and the developer indicated that he or she had never seen that other product.

Consistency failures are often failures of innocent ignorance… the developers aren’t being inconsistent with intent. They’re being inconsistent because they aren’t aware. Increasing awareness increases the consistency.

Take that day or two that you might have spent reviewing specifications and go work with the software.

As an added bonus, that time spent communicating about consistency? Your developers will also communicate about other things and build a more cohesive team.