18.4 Checklists
Checklists are a core element of classic inspection. They summarize the experience accumulated in previous projects, and drive the review sessions. A checklist contains a set of questions that help identify defects in the inspected artifact, and verify that the artifact complies with company standards. A good checklist should be updated regularly to remove obsolete elements and to add new checks suggested by the experience accumulated in new projects. We can, for example, remove some simple checks about coding standards after introducing automatic analyzers that enforce the standards, or we can add specific semantic checks to avoid faults that caused problems in recent projects.
Checklists may be used to inspect a large variety of artifacts, including requirements and design specifications, source code, test suites, reports, and manuals. The contents of checklists may vary greatly to reflect the different properties of the various artifacts, but all checklists share a common structure that facilitates their use in review sessions. Review sessions must be completed within a relatively short time (no longer than two hours) and may require teams of different size and expertise (from a single junior programmer to teams of senior analysts). Length and complexity of checklists must reflect their expected use. We may have fairly long checklists with simple questions for simple syntactic reviews, and short checklists with complex questions for semantic reviews.
Modern checklists are structured hierarchically and are used incrementally. Checklists with simple checks are used by individual inspectors in the early stages of inspection, while checklists with complex checks are used in group reviews in later inspection phases. The preface of a checklist should indicate the type of artifact and inspection that can be done with that checklist and the level of expertise required for the inspection.
The sidebar on page 346 shows an excerpt of a checklist for a simple Java code inspection and the sidebar on page 347 shows an excerpt of a checklist for a more complex review of Java programs.
A common checklist organization, used in the examples in this chapter, consists of a set of features to be inspected and a set of items to be checked for each feature. Organizing the list by features helps direct the reviewers' attention to the appropriate set of checks during review. For example, the simple checklist on page 346 contains checks for file headers, file footers, import sections, class declarations, classes, and idiomatic methods. Inspectors will scan the Java file and select the appropriate checks for each feature.
The items to be checked ask whether certain properties hold. For example, the file header should indicate the identity of the author and the current maintainer, a cross reference to the design entity corresponding to the code in the file, and an overview of the structure of the package. All checks are expressed so that a positive answer indicates compliance. This helps the quality manager spot possible problems, which will correspond to "no" answers in the inspection reports.
Inspectors check the items, answer "yes" or "no" depending on the status of the inspected feature, and add comments with detailed information. Comments are common when the inspectors identify violations, and they help identify and localize the violations. For example, the inspectors may indicate which file headers do not contain all the required information and which information is missing. Comments can also be added when the inspectors do not identify violations, to clarify the performed checks. For example, the inspectors may indicate that they have not been able to check if the maintainer indicated in the header is still a member of the staff of that project.
Checklists should not include items that can be more cost-effectively checked with automated test or analysis techniques. For example, the checklist at page 346 does not include checks for presence in the file header of file title, control identifier, copyright statement and list of classes, since such information is added automatically and thus does not require manual checks. On the other hand, it asks the inspector to verify the presence of references to the author and maintainer and of cross reference to the corresponding design entities, since this checklist is used in a context where such information is not inserted automatically. When adopting an environment that automatically updates author and maintainer information and checks cross references to design entities, we may remove the corresponding checks from the checklist, and increase the amount of code that can be inspected in a session, or add new checks for reducing different problems experienced in new projects.
Properties should be as objective and unambiguous as possible. Complete independence from subjective judgment may not be possible, but must be pursued. For example broad properties like "Comments are complete?" or "Comments are well written?" ask for a subjective judgment, and raise useless and contentious discussions among inspectors and the authors of an artifact undergoing inspection. Checklist items like "Brief comment on each import with the exception of standard set: java.io.*, java.util.*" or "One sentence summary of class functionality" address the same purpose more effectively.
Items should also be easy to understand. The excerpts in the sidebars on pages 346 and 347 list items to be checked, but for each item, the checklist should provide a description, motivations, and examples. Figure 18.1 shows a complete description of one of the items of the sidebars.
Figure 18.1: Detailed description referenced by a checklist item.
Checking for presence of comments is easy to automate, but checking contents for meaning and usefulness is apt to require human judgment. For example, we can easily automate a check that there is a comment for each import section or class, but we cannot automatically check that the comment is meaningful and properly summarizes the need for the import section or the functionality of the class.
Descriptions, motivations and examples may be lengthy, and displaying them directly in the checklist reduces an inspector's ability to quickly scan through the checklist to identify items relevant to each feature under review. Therefore, explanatory material is typically provided separately, linked to the inspection checklist by reference (e.g., a page number for printed documentation or a hyperlink in a Web-based display). Inexperienced inspectors and teams reviewing complex items may access the details to resolve questions and controversies that arise during inspection, but frequency of reference declines with experience.
Checklists can be used in many different contexts. The sidebar on page 350 shows an excerpt of a checklist for comprehensive review of test plans. That checklist assumes that the document includes a standard set of sections, whose presence can be easily checked automatically, and is suited for experienced testers.
Ref. Checklist D1A, page 1/1.
FEATURE: CLASS DECLARATION Are the following requirements satisfied?
ITEM The visibility of the Class is consistent with the design document
Detailed checklist item reference:
Description The fields and methods exported by a class must correspond to those in the specification, which may be in the form of a UML diagram. If the class specializes another class, method header comments must specify where superclass methods are overridden or overloaded. Overloading or overriding methods must be semantically consistent with ancestor methods. Additional public utility or convenience methods may be provided if well documented in the implementation.
The class name should be identical to the name of the class in the specifying document, for ease of reference. Names of methods and fields may differ from those in the specifying document, provided header comments (class header comments for public fields, method header comments for public methods) provide an explicit mapping of implementation names to specification names. Order and grouping of fields and methods need not follow the order and grouping in the specifying document.
Motivations Clear correspondence of elements of the implementation to elements of the specification facilitates maintenance and reduces integration faults. If significant deviations are needed (e.g., renaming a class or omitting or changing a public method signature), these are design revisions that should be discussed and reflected in the specifying document.
Examples The code implementing the following UML specification of class CompositeItem should export fields and methods corresponding to the fields of the specification of class CompositeItem and its ancestor class LineItem. Implementations that use different names for some fields or methods or that do not redefine method getUnitPrice in class CompositeItem are acceptable if properly documented. Similarly, implementations that export an additional method compare that specializes the default method equal to aid test oracle generation is acceptable.
No comments:
Post a Comment