Topic: Usability Testing

Usability Testing ·         User Interface Testing ·         What Makes a Good UI? ·         Testing for the Disabled: Accessibility Testing Software is written to be used. That sounds pretty obvious, but it's sometimes forgotten in the rush to design, develop, and test a complex product. So much time and effort is spent on the technology aspects of writing the code that the development team ignores the most important aspect of software that someone will eventually use the stuff. It really doesn't matter whether the software is embedded in a microwave oven, a telephone switching station, or an Internet stock trading website. Eventually the bits and bytes bubble up to where a live person will interact with it. Usability is how appropriate, functional, and effective that interaction is. You may have heard the term ergonomics, the science of designing everyday things so that they're easy and functional to use. An ergonomist's main concern is in achieving usability. Now, you're not going to get the knowledge of a four-year ergonomics degree in the 15 or so pages of this chapter, nor do you need to. The software is difficult to understand, hard to use, slow, or in the software tester's eyes will be viewed by the end user as just plain not right. That's your blank check for usability testing. You're likely the first person, other than the programmers, to use the software. You've become familiar with the specification and investigated who the customers will be. If you have problems using the software while you're testing it, odds are the customers will, too. Because there are so many different types of software, it's impossible to go into detail about usability issues for all of them. Usability of a nuclear reactor shutdown sequence is pretty different from usability of a voicemail menu system. What you'll learn in this chapter are the basics of what to look for with a bias toward software that you use on your PC every day. You can then take those ideas and apply them to whatever software you have to test. Highlights of this chapter include ·         What usability testing involves ·         What to look for when testing a user interface ·         What special usability features are needed by the disabled

 

User Interface Testing The means that you use to interact with a software program is called its user interface or UI. All software has some sort of UI. Purists might argue that this isn't true, that software such as what's in your car to control the fuel/air ratio in the engine doesn't have a user interface. In truth, it doesn't have a conventional UI, but the extra pressure you need to apply to the gas pedal and the audible sputtering you hear from the tailpipe is indeed a user interface. The computer UI we're all familiar with has changed over time. The original computers had toggle switches and light bulbs. Paper tape, punch cards, and teletypes were popular user interfaces in the '60s and '70s. Video monitors and simple line editors such as MS-DOS came next. Now we're using personal computers with sophisticated graphical user interfaces (GUIs). Soon we'll be speaking and listening to our PCs, carrying on verbal conversations as we do with people! Although these UIs were very different, technically they all provided the same interaction with the computerthe means to give it input and receive output.

 

What Makes a Good UI? Many software companies spend large amounts of time and money researching the best way to design the user interfaces for their software. They use special usability labs run by ergonomic specialists. The labs are equipped with one-way mirrors and video cameras to record exactly how people use their software. Everything the users (subjects) do from what keys they press, how they use the mouse, what mistakes they make, and what confuses them is analyzed to make improvements to the UI. You may be wondering what a software tester could possibly contribute with such a detailed and scientific process. By the time the software is specified and written, it should have the perfect UI. But, if that's the case, why are there so many VCRs blinking 12:00? First, not every software development team designs their interface so scientifically. Many UIs are just thrown together by the programmerswho may be good at writing code, but aren't necessarily ergonomics experts. Other reasons might be that technological limitations or time constraints caused the UI to be sacrificed. The reason might be that the software wasn't properly localized. In the end, the software tester needs to assume the responsibility of testing the product's usability, and that includes its user interface. You might not feel that you're properly trained to test a UI, but you are. Remember, you don't have to design it. You just have to pretend you're the user and find problems with it. Here's a list of seven important traits common to a good UI. It doesn't matter if the UI is on a digital watch or is the Mac OS X interface, they all still apply. ·         Follows standards and guidelines ·         Intuitive ·         Consistent ·         Flexible ·         Comfortable ·         Correct ·         Useful If you read a UI design book, you may also see other traits being listed as important. Most of them are inherent or follow from these seven. For example, "easy to learn" isn't listed above, but if something is intuitive and consistent, it's probably easy to learn. As a tester, if you concentrate on making sure your software's UI meets these criteria, you'll have a darn good interface. Each trait is discussed in detail in the following sections. Follows Standards and Guidelines The single most important user interface trait is that your software follows existing standards and guidelinesor has a really good reason not to. If your software is running on an existing platform such as Mac or Windows, the standards are set. Apple's are defined in the book Macintosh Human Interface Guidelines, published by Addison-Wesley, also available online at developer.apple.com/documentation/mac/ HIGuidelines/HIGuidelines-2.html. Microsoft's are in the book Microsoft Windows User Experience, published by Microsoft Press, with the online version at msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwue/html/welcome.asp. Each book goes into meticulous detail about how software that runs on each platform should look and feel to the user. Everything is defined from when to use check boxes instead of an option button (when both states of the choice are clearly opposite and unambiguous) to when it's proper to use the information, warning, and critical messages as shown in Figure 11.1. Figure 11.1. Did you ever notice that there are three different levels of messages in Windows? When and how to use each one is defined in the user interface standards for Windows.   NOTE If you're testing software that runs on a specific platform, you need to treat the standards and guidelines for that platform as an addendum to your product's specification. Create test cases based on it just as you would from the product's spec.   These standards and guidelines were developed (hopefully) by experts in software usability. They have accounted for a great deal of formal testing, experience, and trial and error to devise rules that work well for their users. If your software strictly follows the rules, most of the other traits of a good UI will happen automatically. Not all of them will because your team may want to improvise on them a bit, or the rules may not perfectly fit with your software. In those cases, you need to really pay attention to usability issues. It's also possible that your platform doesn't have a standard, or maybe your software is the platform. In those situations, your design team will be the ones creating the usability standards for your software. You won't be able to take for granted the rules that someone else has already figured out, and the remaining traits of a good user interface will be even more important for you to follow. Intuitive In 1975 the MITS (Micro Instrumentation Telemetry Systems) Altair 8800 was released as one of the first personal computers. Its user interface (see Figure 11.2) was nothing but switches and lightsnot exactly intuitive to use. Figure 11.2. The MITS Altair 8800 and its less-than-intuitive user interface. (Photo courtesy of the Computer Museum of America, www.computer-museum.org.)   The Altair was designed for computer hobbyists, people who are a lot more forgiving of user interface issues. Today, users want much more out of their software than what the Altair 8800 provided. Everyone from grandmothers to little kids to Ph.D.s is using computers in their daily lives. The computers with the most intuitive UIs are the ones that people don't even realize they're using. When you're testing a user interface, consider the following things and how they might apply to gauging how intuitive your software is: ·         Is the user interface clean, unobtrusive, not busy? The UI shouldn't get in the way of what you want to do. The functions you need or the response you're looking for should be obvious and be there when you expect them. ·         Is the UI organized and laid out well? Does it allow you to easily get from one function to another? Is what to do next obvious? At any point can you decide to do nothing or even back up or back out? Are your inputs acknowledged? Do the menus or windows go too deep? ·         Is there excessive functionality? Does the software attempt to do too much, either as a whole or in part? Do too many features complicate your work? Do you feel like you're getting information overload? ·         If all else fails, does the help system really help you? Consistent Consistency within your software and with other software is a key attribute. Users develop habits and expect that if they do something a certain way in one program, another will do the same operation the same way. Figure 11.3 shows an example of how two Windows applications, which should be following a standard, aren't consistent. In Notepad, Find is accessed through the Search menu or by pressing F3. In WordPad, a very similar program, it's accessed through the Edit menu or by pressing Ctrl+F. Figure 11.3. Windows Notepad and WordPad are inconsistent in how the Find feature is accessed.   Inconsistencies such as this frustrate users as they move from one program to another. It's even worse if the inconsistency is within the same program. If there's a standard for your software or your platform, follow it. If not, pay particular attention to your software's features to make sure that similar operations are performed similarly. Think about a few basic items as you review your product: ·         Shortcut keys and menu selections. In a voicemail system, pressing 0, not other numbers, is almost always the "get-out" button that connects you to a real person. In Windows, pressing F1 should always get you help. ·         Terminology and naming. Are the same terms used throughout the software? Are features named consistently? For example, is Find always called Find, or is it sometimes called Search? ·         Audience. Does the software consistently talk to the same audience level? A fun greeting card program with a colorful user interface shouldn't display error messages of arcane techno babble. ·         Placement for buttons such as OK and Cancel. Did you ever notice that in Windows, OK is always on the top or left and Cancel on the right or bottom? The Mac OS places OK on the right. Keyboard equivalents to onscreen buttons should also be consistent. For example, the Esc key always does a cancel and Enter does an OK. Flexible Users like choices not too many, but enough to allow them to select what they want to do and how they want to do it. The Windows Calculator (see Figure 11.4) has two views: Standard and Scientific. Users can decide which one they need for their task or the one they're most comfortable using. Figure 11.4. The Windows Calculator shows its flexibility by having two different views.   Of course, with flexibility comes complexity. In the Calculator example you'll have a much larger test effort than if there's just one view. ·         State jumping. Flexible software provides more options and more ways to accomplish the same task. The result is additional paths among the different states of the software. Your state transition diagrams can become much more complex and you'll need to spend more time deciding which interconnecting paths should be tested. ·         State termination and skipping. This is most evident when software has power-user modes where a user who's very familiar with the software can skip numerous prompts or windows and go directly to where they want to go. A voicemail system that allows you to directly punch in your party's extension is an example. If you're testing software that allows this, you'll need to make sure that all the state variables are correctly set if all the intermediate states are skipped or terminated early. ·         Data input and output. Users want different ways to enter their data and see their results. To put text into a WordPad document, you can type it, paste it, load it from six different file formats, insert it as an object, or drag it with the mouse from another program. The Microsoft Excel spreadsheet program allows you to view your data in 14 different standard and 20 different custom graphs. Who even knew there were that many possibilities? Testing all the different ways to get data in and out of your software can very quickly increase the effort necessary and make for tough choices when creating your equivalence partitions. Comfortable Software should be comfortable to use. It shouldn't get in the way or make it difficult for a user to do his work. Software comfort is a pretty touchy-feely concept. Researchers have spent their careers trying to find the right formula to make software comfortable. It can be a difficult concept to quantify, but you can look for a few things that will give you a better idea of how to identify good and bad software comfort: ·         Appropriateness. Software should look and feel proper for what it's doing and who it's for. A financial business application should probably not go crazy with loud colors and sound effects. A space game, on the other hand, will have much more leeway with the rules. Software should neither be too garish nor too plain for the task it's intended to perform. ·         Error handling. A program should warn users before a critical operation and allow users to restore data lost because of a mistake. People take the Undo/Redo feature for granted today, but it wasn't long ago that these features didn't exist. ·         Performance. Being fast isn't always a good thing. More than one program has flashed error messages too quickly to read. If an operation is slow, it should at least give the user feedback on how much longer it will take and show that it's still working and hasn't frozen. Status bars, as shown in Figure 11.5, are a popular way to accomplish this. Figure 11.5. Status bars show how much of the work has been completed and how much is left to go.   Correct The comfort trait is admittedly a bit fuzzy and often can be left to interpretation. Correctness, though, isn't. When you're testing for correctness, you're testing whether the UI does what it's supposed to do. Figure 11.6 is an example of a UI that isn't correct. Figure 11.6. This software has a completely useless Abort button.   This figure shows a message box from a popular page-scanning program for Windows. The box appears when a scan is started and is supposed to provide a way for the user to stop the scan mid-process. Unfortunately, it doesn't work. Note that the cursor is an hourglass. An hourglass means (according to the Windows standard) that the software is busy and can't accept any input. Then why is the Abort button there? You can repeatedly click the Abort button during the entire scan, which can take a minute or more, and nothing happens. The scan continues uninterrupted until it completes. If clicking the Abort button with the hourglass cursor did stop the scan, would that be a bug? You bet it would! Correctness problems such as this are usually obvious and will be found in your normal course of testing against the product specification. You should pay attention to some areas in particular, however: ·         Marketing differences. Are there extra or missing functions, or functions that perform operations different from what the marketing material says? Notice that you're not comparing the software to the specificationyou're comparing it to the sales information. They're usually different. ·         Language and spelling. Some programmers are poor spellers and writers and often create very interesting user messages. The following is an order confirmation message from a popular e-commerce websitehopefully fixed by the time you read this: If there are any discrepancies with the information below, please contact us immediately to ensure timely delivery of the products that you ordered. ·         Bad media. Media is any supporting icons, images, sounds, or videos that go with your software's UI. Icons should be the same size and have the same palette. Sounds should all be of the same format and sampling rate. The correct ones should be displayed when chosen from the UI. ·         WYSIWYG (what you see is what you get). Make sure that what the UI displays is really what you have. When you click the Save button, is the document onscreen exactly what's saved to disk? When you load it back, does it perfectly compare with the original? When you print it, does the output perfectly match what's previewed on the screen? Useful The final trait of a good user interface is whether it's useful. Remember, here you're not concerned with whether the software itself is useful, just whether the particular feature is. A popular term used in the software industry to describe unnecessary or gratuitous features is dancing bologna. Think of a sausage bouncing around on the screen completely unnecessary! When you're reviewing the product specification, preparing to test, or actually performing your testing, ask yourself if the features you see actually contribute to the software's value. Do they help users do what the software is intended to do? If you don't think they're necessary, do some research to find out why they're in the software? It's possible that there are reasons you're not aware of, or it could just be dancing bologna. Those superfluous features, whether they be in a solitaire program or a heart monitor are bad for the user and mean extra testing for you.

 

Testing for the Disabled: Accessibility Testing A serious topic that falls under the area of usability testing is that of accessibility testing or testing for the disabled. A 1997 government Survey of Income and Program Participation (SIPP) used by the U.S. Census Bureau found that about 53 million people (nearly 20% of the population) in the country had some sort of disability. Table 11.1 shows a more detailed breakdown. Table 11.1. People with Disabilities Age Percentage of People with Disabilities 024 18% 2544 13% 4554 23% 5564 36% 6569 45% 7074 47% 7579 58% 80+ 74%   Cutting the data another way, reveals that 7.7 million people have difficulty seeing the words and letters in a newspaper. 1.8 million People are legally blind and 8 million people have difficulty hearing. With our aging population and the penetration of technology into nearly every aspect of our lives, the usability of software becomes more important every day. Although there are many types of disabilities, the following ones make using computers and software especially difficult: ·         Visual impairments. Color blindness, extreme near and far sightedness, tunnel vision, dim vision, blurry vision, and cataracts are examples of visual limitations. People with one or more of these would have their own unique difficulty in using software. Think about trying to see where the mouse pointer is located or where text or small graphics appear onscreen. What if you couldn't see the screen at all? ·         Hearing impairments. Someone may be partially or completely deaf, have problems hearing certain frequencies, or picking a specific sound out of background noise. Such a person may not be able to hear the sounds or voices that accompany an onscreen video, audible help, or system alerts. ·         Motion impairments. Disease or injury can cause a person to lose fine, gross, or total motor control of his hands or arms. It may be difficult or impossible for some people to properly use a keyboard or a mouse. For example, they may not be able to press more than one key at a time or may find it impossible to press a key only once. Accurately moving a mouse may not be possible. ·         Cognitive and language. Dyslexia and memory problems may make it difficult for someone to use complex user interfaces. Think of the issues outlined previously in this chapter and how they might impact a person with cognitive and language difficulties. Legal Requirements Fortunately, developing software with a user interface that can be used by the disabled isn't just a good idea, a guideline, or a standard it’s often the law. In the United States, three laws apply to this area and other countries are considering and adopting similar laws: ·         The Americans with Disability Act states that businesses with 15 or mores employees must make reasonable accommodations for employees, or potential employees, with disabilities. The ADA has recently been applied to commercial Internet websites, mandating that they be made accessible to the public who uses them. ·         Section 508 of the Rehabilitation Act is very similar to the ADA and applies to any organization that receives federal funding. ·         Section 255 of the Telecommunications Act requires that all hardware and software that transfers information over the Internet, a network, or the phone lines be made so that it can be used by people with disabilities. If it's not directly usable, it must be compatible with existing hardware and software accessibility aids. Accessibility Features in Software Software can be made accessible in one of two ways. The easiest is to take advantage of support built into its platform or operating system. Windows, Mac OS, Java, and Linux all support accessibility to some degree. Your software only needs to adhere to the platform's standards for communicating with the keyboard, mouse, sound card, and monitor to be accessibility enabled. Figure 11.7 shows an example of the Windows accessibility settings control panel. Figure 11.7. The Windows accessibility features are set from this control panel.   If the software you're testing doesn't run on these platforms or is its own platform, it will need to have its own accessibility features specified, programmed, and tested. The latter case is obviously a much larger test effort than the first, but don't take built-in support for granted, either. You'll need to test accessibility features in both situations to make sure that they comply. NOTE If you're testing usability for your product, be sure to create test cases specifically for accessibility. You'll feel good knowing that this area is thoroughly tested.   Each platform is slightly different in the features that it offers, but they all strive to make it easier for applications to be accessibility enabled. Windows provides the following capabilities: ·         StickyKeys allows the Shift, Ctrl, or Alt keys to stay in effect until the next key is pressed. ·         FilterKeys prevents brief, repeated (accidental) keystrokes from being recognized. ·         ToggleKeys plays tones when the Caps Lock, Scroll Lock, or NumLock keyboard modes are enabled. ·         SoundSentry creates a visual warning whenever the system generates a sound. ·         ShowSounds tells programs to display captions for any sounds or speech they make. These captions need to be programmed into your software. ·         High Contrast sets up the screen with colors and fonts designed to be read by the visually impaired. Figure 11.8 shows an example of this. Figure 11.8. The Windows desktop can be switched to this high contrast mode for easier viewing by the visually impaired.   ·         MouseKeys allows use of keyboard keys instead of the mouse to navigate. ·         SerialKeys sets up a communications port to read in keystrokes from an external non-keyboard device. Although the OS should make these devices look like a standard keyboard, it would be a good idea to add them to your configuration testing equivalence partitions. For more information about the accessibility features built into the popular OS platforms, consult the following websites: ·         www.microsoft.com/enable ·         www.apple.com/accessibility ·         www-3.ibm.com/able ·         www.linux.org/docs/ldp/howto/Accessibility-HOWTO

 

Summary The software is difficult to understand, hard to use, slow, or in the software tester's eyes will be viewed by the end user as just plain not right. As a software tester checking the usability of a software product, you're likely the first person to use the product in a meaningful way, the first person to see it all come together in its proposed final form. If it's hard to use or doesn't make sense to you, customers will have the same issues. Above all, don't let the vagueness or subjectivity of usability testing hinder your test effort. It's vague and subjective by nature. Even the experts who design the user interfaces will admit to that well, some of them will. If you're testing a new product's UI, refer to the lists in this chapter that define what makes for a good one. If it doesn't meet these criteria, it's a bug, and if it's a usability bug, it might just be the law.

 

Quiz 1: True or False: All software has a user interface and therefore must be tested for usability. 2: Is user interface design a science or an art? 3: If there's no definitive right or wrong user interface, how can it be tested? 4: List some examples of poorly designed or inconsistent UIs in products you're familiar with. 5: What four types of disabilities could affect software usability? 6: If you're testing software that will be accessibility enabled, what areas do you need to pay close attention to?