User-friendly Documentation Testing

Functional testing, security testing, load testing and requirements testing are important aspects of the product’s behavior, so they require sufficient amount of attention. The testing of the documentation supplied with the software (the particular form doesn’t matter – book, PDF file or a knowledgebase on the website) is often carried out on the leftover principle.

Normally, the documentation isn’t refreshed entirely. As the product gets new features, their descriptions are added to the documentation, and then the assigned person ensures that everything “is in order”. It is a good option, when testing is performed by an experienced person who is familiar with the software and the other context (e.g. the subject domain; the documentation’s language; the target audience, etc.).

However, the problem is that often there is no formal approach to implement such checks. A person reading the documentation is in fact checking it for strange parts. If he finds any, good – he discusses the problem with teammates and reports a bug. However, if he doesn’t see anything strange, it doesn’t guarantee the documentation’s accuracy.

Sometimes the manual's proofreading is the special activity for newbies. This practice allows introducing new members to the product and testing documentations, albeit at a surface grammar level. Although, it's not a very user-friendly approach, right?

Sometimes there are changes that affect the entire documentation, like:

• New interface (so we need to change all the screenshots).
• Rebranding (so we replace the old products’ name with new one everywhere in the documentation).
• Documentation’s structure update (we read and check if the new structure is implemented).

In this case, the tester usually focuses on specific changes, leaving aside other aspects like spell check. For example, if the name of the product changed from “On” to “Off”, it is essential to make sure that the documentation reflects this modification correctly and doesn’t show new words like “cofftract” or “doffe”. Also, not a word about user-friendly experience.

Understanding the User

We have only the skilled and well-trained testers checking our manuals, because they should be able to walk in users’ shoes, be down with their needs. It is doubtful that the users are having time of their lives reading all the manuals to each product they get in their hands.

It is due to the one of the classic problems of the documentation: programmers write it in their usual manner, with technic terms and descriptions of software languages, functions and settings.

User needs the plain use cases, including description of what, where and in what sequence he has to do to get the desired result.

If, instead, he sees the settings' description, it does not necessarily mean that he will be able to achieve the goal. No wonder that so many questions on forums are related to the simplest software functions.

Documentation Testing Criteria

The main problem of manuals' testing is in the requirements and criteria. What do we expect from the documentation? What are the criteria to decide if there are any defects in the text? If the requirements are not set, you can (or have to) define them in the testing process. The main thing is to make sure all parties agreed on the criteria.

To analyze your manuals profoundly, try seven aspects we consider important when testing documentation:

1. The Accuracy and Fullness of Information

Everything you write in the manual must be true. If possible, the description should be complete; the user may need instructions on any of your product's features. Moreover, if the documentation doesn't provide the info about some options, the user may never get to know them (unless he accidentally stumbles upon them while working with the software).

The manual shouldn’t contain any outdated information. One of the prove-the-rule exception is the API description including some references to the obsolete functions with related notes. Just remember that the irrelevant information about your product in the manual can cost the user a lot of time and money.

2. The Replicated Information

In complex products, the different chapters of the documentation might use the same long, detailed instructions. You want to find the right option for your manual. You can either describe everything in detail wherever needed or write the guidance notes once and refer to these notes in other chapters.

Both approaches have their pros and cons. If in the first case the instruction changes, we might forget to enter the updates into the one of the chapters. In the second, we force the user to scroll the documentation back and forth, which is often not very convenient.

3. The Terminology

If the documentation contains any specific terms and abbreviations, make sure to have the relevant definitions and interpretations too. The terms should be used the same way in all documents and chapters. Avoid the too vague concepts with various meanings. And certainly don’t name the different entities of the product with the same (or very similar) terms.

4. The Native Navigation

Ideally, the user should never think about how to find the required data. The documentation itself must lead him by the hand to the needed information. This is, of course, an example of unbelievable happiness, but let’s at least take care of the clear links between the parts, the table of contents structure and the alphabetical index.

If the manual is the PDF document, and it contains bookmarks, then make sure that the panel with tabs shows up immediately when opening the document. Not every user is a daily PDF reader – make his life easier. In general, just think about how you search the information in the documentation, and how would you simplify or speed up this process.

5. The Text Formatting

Inspect the documentation and try to answer some questions. Is it easy to find the important places in text? Is it comfortable to find a place where you left off, if you turn away for a couple of seconds and then return to the document? It’s not a very simple point. Concentrate on easy-to-read fonts, line spacing, paragraphs, numeration and other design nuances to make your texts readable.

Sometimes the user can’t say, why he is trudging through your documentation. If the cause of inconvenience is not quite obvious, try at least to describe the problems arising when reading. Your thoughts might help you to find out the key issue.

6. The Text Copying

User might want to copy a part of the manual’s text. This could be an info to save for later or the script code to perform right now, for example.

Firstly, make sure it is possible to copy the data from the documentation. Then watch what happens. If the desired text was placed on multiple pages, were the header and footer copied with it? Does the copied part contain any non-printable characters? Or maybe they are lost when copying?

7. The Role Separation

If the product provides a variety of roles and access permissions levels, and there’s a separate manual for each of them describing the work with the system, you should make sure that the document for the lower level doesn’t contain information allowing to know something specific and important for the higher level.

For example, the average user doesn’t need to know how to configure the system. Clearly, if he sets a goal to get this info, he’ll most likely find a relevant document. However, he doesn’t accidentally stumble upon it, at least. In addition, the extra information not related to his level of access might be confusing.

It would seem, like the documentation doesn't matter at all. However, we strongly advise spending some time with the manuals’ testing, because the relevant and clear information:

1. Could become a timesaver for both end user and your company, cause the person might just read the documentation and figure out the problem by oneself.

2. It shows your thoughtful attitude to clients' needs. This surely might gain you some loyalty points.

So have fun with your documentation testing!