Software tests are the final checkpoint for projects before clients and end-users of the system get a chance to take the application for a spin. As seasoned professionals in the field, we try to exploit every possible opportunity to improve test efficiency. Many different models, approaches and strategies have been tried – some with relatively better results than others.
Tests based on requirements still fail user expectations because requirements describe system specifications whereas user expectations are best represented through user centric design artifacts. Exploratory testing is a technique that shifts focus from system-centric checking to user-centric testing. To be effective, exploratory tests have to rely on the user-centric artifacts that capture the user behavior.
Exploratory testing as opposed to ad-hoc testing is a focused, well-defined and controlled testing approach that time-boxes test iterations and cycles using scenarios for reference. Exploratory testers rely on hunches, biases, conjectures, intuition, personal experience and heuristics while continuously learning from the system behavior. User experience (UX) design process tries to uncover similar aspects of user behavior that motivates users of the system that are the basis for user expectations.
Jesse James Garrett’s elements of user experience (UX)1 describes this process through the evolutionary layers that begin with abstract strategies. The abstract strategies are then transformed over the layers into concrete user experiences. Using these layers for guidance, the tester reviews decisions made through the prism of the elements represented in these layers. Though the more abstract layers are not visible to the tester, the consequences of the design decisions made in those lower layers manifests in the more concrete surface elements.
With the advent of agile methodology & principles, testers got the opportunities to engage early in the evolution of the design as teams realized the value of testing early and often. The dynamic nature of the system behavior is a dimension higher than the one-dimensional static artifacts that is the outcome of the UX design process. Transitioning from the designer’s cognition of the static model to this dynamic behavior model requires experience, skill and context. While experience takes time and skill can be learned, context is established early in the UX process when user research happens. Supplementary UX design processes like market study and competitive analysis helps to relate to the strategy that is at the root of the elements of UX. These insights provide significant background for the exploration the tester engages in.
Figure 1. Jesse James Garrett’s Elements of UX
Insights gained from the user research plays a vital role in giving context to the elicited requirements. Requirements are typically captured in the form of user stories, epics (or themes) and tasks. The central figure in all of these is the user at whom the value proposition focuses on. Given the situation, it is imperative and natural for testers to rely on these UX design process to test the system for greater return on user acceptance.
The style of presentation in scenarios sets the expectations of user behavior. Scenario-based test designs and strategies bring-together all of these functions into a user-centric focus. Exploration based on scenarios thus provide the holistic framework for user acceptance. Scenarios when committed to textual form reduces the insights gained from the UX design process due to limitation of language. Closed-form formal language specification to describe user behavior is still elusive for software engineering practice. Exploratory testing approach fills this gap and UX design supports this approach very well, naturally and intuitively.
In your next client engagement, I would encourage you to plan for exploratory testing by engaging the quality team early with the UX design team. If you are interested in knowing more about this approach and some tangible benefits realized in real-world projects, I will be speaking about this very topic at the Pacific Northwest Software Quality Conference (PNSQC) 2012 in Portland, Oregon from Oct 8-10, 2012.
1. Jesse James Garrett, “The Elements of User Experience: User-Centered Design for the Web“, Peachpit Press (October 21, 2002)