< Day Day Up > |
9.6 Control System Test Techniques
After the control system implementation has been completed and initial checkout of the system indicates correct operation, it is time to perform thorough system testing. This testing phase must exercise all controller modes throughout the system's entire operating envelope. The test process must ensure that the system and its embedded controller operate properly under nominal conditions, as well as in extreme situations.
I discuss two basic approaches for performing system testing in this section: system simulation and operational tests of the actual system. These are complementary techniques that enable different types of testing to be performed. By effectively combining simulation and system testing, it is possible to thoroughly test a complex embedded control system under a complete range of operating conditions while keeping testing costs to a minimum.
9.6.1 System Simulation
System simulation [1] is focused at the level of the entire system as it operates in its intended environment. This type of simulation permits testing of the system in its intended operational modes as well as under dangerous emergency conditions without risking loss of life or valuable assets. Environmental conditions that could be difficult or impossible to access for system tests (such as icy roads in the middle of summer or the conditions of outer space) can be simulated by computers running appropriate software algorithms. With simulation, intricate test sequences can be executed quickly and repeatably at comparatively low cost.
A system simulation is a software program developed with a simulation package like Simulink or in a programming language such as C++. The simulation contains mathematical models of the system components and their interactions with the environment. These models must possess sufficient fidelity to enable the execution of realistic tests.
Simulation development often starts early in the project life cycle with fairly simple models of the system components and the environment. As the project continues, the data from tests of hardware components and complete system prototypes enables the refinement of the simulation models to more accurately represent the real system.
The software-only simulation described above can form the basis for a hardware-in-the-loop (HIL) simulation. Prototype hardware and software can be tested in a HIL simulation long before a testable system prototype becomes available. HIL simulations run at real-time speeds and perform I/O operations with the system or subsystem under test so that the test item "thinks" it is operating as part of a real system in its operational environment. This enables the system or subsystem to be tested under simulated nominal conditions as well as at (and beyond) its intended operational boundaries.
HIL simulation provides the ability to thoroughly test subsystems early in the development process. This can greatly reduce the debugging time and project risk compared to the alternative approach of waiting until prototype hardware is available before performing system integration and testing.
Simulation verification and validation must be addressed carefully to gain the full range of potential benefits from a simulation effort. Verification is the process used to demonstrate that a simulation has been implemented according to its specifications. The validation process demonstrates that the simulation is a sufficiently good representation of the actual system it attempts to simulate. The primary goal of the verification and validation processes is to provide sufficient convincing evidence of the correctness of the simulation so that even skeptical observers will agree that it is credible and sufficiently accurate for its intended purposes.
Simulation will never completely replace tests of the actual system. However, in many cases, the thoughtful application of simulation techniques to an embedded control system development process can increase the overall test coverage while significantly reducing the test time required and the cost of the test program.
9.6.2 Operational Testing
Operational testing places the actual system into its intended operating environment for test execution. In this type of testing, you set up test scenarios that represent real-world situations the embedded system might encounter. You then observe the system's response to the situation, recording the data necessary to provide a complete picture of the system's behavior, as well as relevant attributes of the environment. Analysis of the test results leads to conclusions about whether the system performed properly and what type of performance improvements might be necessary.
Example 9.1: Testing an autonomous ground vehicle.
Suppose you are developing a small autonomous ground vehicle. This vehicle must be able to navigate over difficult terrain using a set of sensors to identify a path and with steering and drive systems to move along the chosen path. How would you go about setting up a set of operational tests for this vehicle?
First, you should identify the scope of the testing to be performed. In this case, the scope might be limited to having the vehicle travel between selected starting and ending points across various types of terrain. The types of terrain to be used for the testing must be identified carefully. External influences on the terrain, such as the presence of rain, must be understood and addressed.
A set of tests must be developed to exercise the system across its range of capability. Some of the dimensions explored in the testing might include
terrain difficulty, from smooth and level to steep and rocky;
time of day and corresponding light conditions;
weather conditions: hot, cold, windy, rainy, foggy, etc.; and
degraded modes of system operation (e.g., you might disable a sensor or actuator and observe how the system compensates).
As is clear from this short list, it is not a trivial job to set up and execute a comprehensive series of operational tests. Testing this system could involve long periods of waiting for the weather to cooperate. Alternatively, it might be necessary to use a water truck to provide a simulated rainstorm, for example.
Although operational testing is unquestionably a highly realistic way of testing the system, the method does present some problems. For one thing, operational testing can take a long time and cost a lot of money. Commercial airliners receive airworthiness certification after passing a rigorous series of flight tests. These tests often take over a year to complete. Great patience and deep pockets are required to conduct an operational test series of this magnitude. It makes sense to replace operational tests with quicker and lower cost forms of testing such as simulation whenever appropriate.
Another problem with operational testing is that it can be difficult to perform the tests repeatably. In a flight test, for example, the wind conditions and air temperature are factors that cannot be duplicated from one day to the next. When performing a regression test in an operational environment, it might be difficult to determine whether deviations from previously observed behavior are a result of system problems or differences in test conditions.
A combination of operational testing and simulation testing can help alleviate both of these problems. The operational tests should be designed to provide the maximum possible amount of data for use in validating the simulation under a variety of operating conditions. After the system simulation has been developed and validated, test runs can be performed with the simulation quickly and at low cost. This can eliminate the need for some operational tests, though not all of them. A number of real-world tests will always be required to validate the simulation.
No comments:
Post a Comment