3.6 Feedback

3.6 Feedback

Feedback is another classic engineering principle that applies to analysis and testing. Feedback applies both to the process itself (process improvement) and to individual techniques (e.g., using test histories to prioritize regression testing).

Systematic inspection and walkthrough derive part of their success from feedback. Participants in inspection are guided by checklists, and checklists are revised and refined based on experience. New checklist items may be derived from root cause analysis, analyzing previously observed failures to identify the initial errors that lead to them.

Summary

Principles constitute the core of a discipline. They form the basis of methods, techniques, methodologies and tools. They permit understanding, comparing, evaluating and extending different approaches, and they constitute the lasting basis of knowledge of a discipline.

The six principles described in this chapter are

• 
  

  Sensitivity: better to fail every time than sometimes,

  
  


• 
  

  Redundancy: making intentions explicit,

  
  


• 
  

  Restriction: making the problem easier,

  
  


• 
  

  Partition: divide and conquer,

  
  


• 
  

  Visibility: making information accessible, and

  
  


• 
  

  Feedback: applying lessons from experience in process and techniques.

  
  

Principles are identified heuristically by searching for a common denominator of techniques that apply to various problems and exploit different methods, sometimes borrowing ideas from other disciplines, sometimes observing recurrent phenomena. Potential principles are validated by finding existing and new techniques that exploit the underlying ideas. Generality and usefulness of principles become evident only with time. The initial list of principles proposed in this chapter is certainly incomplete. Readers are invited to validate the proposed principles and identify additional principles.

Further Reading

Analysis and testing is a relatively new discipline. To our knowledge, the principles underlying analysis and testing have not been discussed in the literature previously. Some of the principles advocated in this chapter are shared with other software engineering disciplines and are discussed in many books. A good introduction to software engineering principles is the third chapter of Ghezzi, Jazayeri, and Mandrioli's book on software engineering [GJM02].

Exercises

3.1	Indicate which principles guided the following choices: Use an externally readable format also for internal files, when possible. Collect and analyze data about faults revealed and removed from the code. Separate test and debugging activities; that is, separate the design and execution of test cases to reveal failures (test) from the localization and removal of the corresponding faults (debugging). Distinguish test case design from execution. Produce complete fault reports. Use information from test case design to improve requirements and design specifications. Provide interfaces for fully inspecting the internal state of a class.
3.2	A simple mechanism for augmenting fault tolerance consists of replicating computation and comparing the obtained results. Can we consider redundancy for fault tolerance an application of the redundancy principle?
3.3	A system safety specification describes prohibited behaviors (what the system must never do). Explain how specified safety properties can be viewed as an implementation of the redundancy principle.
3.4	Process visibility can be increased by extracting information about the progress of the process. Indicate some information that can be easily produced to increase process visibility.