Use-Case-Driven Testing
The classical view of testing, from a quality control perspective, is use-case driven. That is, test cases are derived from use cases. For example, the following is a use case for the Mini HR "Search by Name" feature.
Precondition:
The user has browsed to the Mini HR main page.
Primary Flow:
Click the Search link on the main page.
On the Search page, choose to Search by Name.
Enter all or part of an employee's name.
Submit the form. If a username was not entered, a validation error is displayed.
All employees that match the input name are displayed on the page. For each user, the employee name and social security number are listed. If no employees match, a message is displayed indicating that no matches exist.
This use case is straightforward and easy to understand. It is complete in that it specifies what must happen prior to the scenario (preconditions), all steps that the user must take, and the results of those steps.
Creating Test Cases
Specific test cases can be easily derived from this use case. A test case is generally more explicit than the use case in that it may describe specific test data to be used. In addition, alternate use-case flows are often modeled as separate test cases. The test explicitly states the steps that must be taken to test the functionality described by the use case. In addition, it explicitly states what the results should be. Think of a test case as a set of steps that you would hand to someone who is testing a certain function or feature of the application. The following are the two test cases for Search by Name-the key differences from the use case are in bold.
Test Case: Search by Name with Matches
Preconditions:
The employee database is preloaded with test data.
The browser location is the Mini HR main page.
Primary Flow:
Click the Search link on the main page.
Verify that the Search page is displayed.On the Search page, choose to Search by Name.
Enter "Jim" in the username field.Submit the form.
Verify that the results are displayed with a table like the following:Jim Smith
123-45-6789
Will Jimmerson
987-65-4321
Test Case: Search by Name with No Matches
Preconditions:
The employee database is preloaded with test data.
The browser location is the Mini HR main page.
Primary Flow:
Click the Search link on the main page.
Verify that the Search page is displayed.On the Search page, choose to Search by Name.
Enter "Fred" in the username field.Submit the form.
Verify that the "No Employees Found" message is displayed.
These test cases now provide the test engineer with a script to follow for testing. While manual testing is important, it would also be beneficial if test cases like these could be automated. Automated testing would permit regression tests of an application anytime such testing is needed. For example, these tests would be run whenever a new version of the application was deployed to the QA server. The next section demonstrates a tool for automating these types of tests.
Using Canoo WebTest
Canoo WebTest, developed by Canoo Engineering, was designed to test Web applications. It is distributed as free open-source software. WebTest uses the Ant build utility as a test driver and HttpUnit to interact with an HTTP server. It works off an XML-based test specification. Test cases can be easily modeled using this test description. These tests are functional tests, not unit tests. The expected behavior is specified as a set of steps. These steps closely match the steps in a test case. Ant is used to execute the tests. You can create these tests in your normal Ant build script. However, it is better if they are separated out into separate files. They can always be integrated into a build, say for use by Quality Control, using Ant's ant task (which can execute a target of another build file). To demonstrate WebTest, a functional test for the "Search by Name" test case from the preceding section will be created.
First, download Canoo WebTest (http://webtest.canoo.com). Extract the distribution into a directory and create the environment variable WEBTESTHOME that refers to that directory. You can test the installation by running the WEBTESTHOME\doc\samples\ installTest.xml file with Ant as follows:
cd WEBTESTHOME\doc\samples
webtest –buildfile installTest.xml
However, this test may fail on certain systems. As suggested in the WebTest installation instructions, HtmlUnit (which is used internally by WebTest) may not handle the file protocol. To verify the installation, make a copy of installTest.xml. In the copied file, change the host, port, and protocol to valid values for your application server. For example, if you are running Tomcat locally on port 8080, you would change the checkWebTest element to the following:
<target name="checkWebTest">
<testSpec name="check calling and parsing a local file">
<config host="" port="8080" basepath="/" summary="false"
verbose="true" saveresponse="false" haltonfailure="true"
protocol="http" />
<steps>
<invoke stepid="get local file" url="/index.jsp" />
<verifytitle stepid="check the title is parsed correctly"
text="Apache Tomcat" regex="true" />
</steps>
</testSpec>
</target>
An important item to note here is that the checkWebTest target is a regular Ant target. The first nested element of the testSpec element is config. This element is required. It defines the location of the application under test. Host defaults to localhost if not specified. Following the config element are the steps of the test specification. The steps consist of a sequential set of nested elements that define the actions that the test is to take. These steps can generally be categorized as either action steps or verification steps. Action steps include such things as accessing a URL, clicking a link or button, and typing text on a form. Verification steps are used like assertions in a JUnit unit test. They define steps that evaluate to true or false.
Once WebTest is up and running, you will want to create the test specifications for the test cases. The following test specification is used to test the two Search by Name test cases. A target is created for each test case. These test targets can be run by simply calling Ant, passing the build filename using the –buildfile testfile.xml (or –f testfile.xml) option. Likewise, if your IDE provides Ant integration (as most IDEs do), you can run the tests from your IDE in the same manner that you run an Ant build script.
<?xml version="1.0"?>
<!DOCTYPE project SYSTEM "WebTest.dtd"[
<!ENTITY definition SYSTEM "includes/definition.xml">
<!ENTITY config SYSTEM "includes/config.xml">
<!ENTITY getSearchPage SYSTEM "includes/getSearchPage.xml">]>
<project name="MiniHR-SearchTest" basedir="." default="all">
<property name="webtest.home" value="../../canoo"/>
&definition;
<target name="all" depends="nameSearchWithResults, nameSearchNoResults"/>
<target name="nameSearchWithResults">
<testSpec name="Perform a Search by Name">
&config;
<steps>
&getSearchPage;
<setinputfield stepid="Enter 'Jim' into Name field"
name="name" value="Jim"/>
<clickbutton stepid="Submit the 'search by' form"
label="Submit"/>
<verifyxpath stepid="Verify xpath 'Jim Smith'"
xpath="/html/body/table/tr/td[1]"
text="[Jj][Ii][Mm]"
regex="true"/>
</steps>
</testSpec>
</target>
<target name="nameSearchNoResults">
<testSpec name="Perform a Search by Name with No Results">
&config;
<steps>
&getSearchPage;
<setinputfield stepid="Enter 'Morty' into Name field"
name="name" value="Morty"/>
<clickbutton stepid="Submit the 'search by' form"
label="Submit"/>
<verifytext stepid="Verify 'No Employees Found'"
text="No Employees Found"/>
</steps>
</testSpec>
</target>
</project>
As you can see in the DOCTYPE declaration, this test makes extensive use of XML entities to bring in shared XML fragments. The first entity, definition, defines an entity that contains the task definitions for Canoo.
includes/definition.xml:
<taskdef file="${webtest.home}/webtestTaskdefs.properties">
<classpath>
<pathelement path="${webtest.home}/lib"/>
<fileset dir="${webtest.home}" includes="**/lib/*.jar"/>
</classpath>
</taskdef>This fragment is generic to most any use of Canoo WebTest.
The second entity, config, is specific to the application being tested. In this case, the XML fragment defines the config element specifically for accessing the Mini HR application running on Tomcat.
includes/config.xml:
<config host="" port="8080" basepath="/MiniHR" summary="false"
verbose="true" saveresponse="false" haltonfailure="true"
protocol="http"/>
You can reuse this element for any test specification of your application.
The final entity, getSearchPage, is specific to this test set. It defines the steps that are necessary to get the Search page. These steps include accessing the application's main page, clicking the Search link, and then verifying that the Search page is displayed. While the steps could have been duplicated for each target, creating the entity allows reuse of this functionality for additional tests that need to get the Search page. Those tests may be added to this set of specifications or to new sets of specifications.
includes/getSearchPage.xml:
<invoke stepid="Get main page" url="/index.jsp"/>
<verifytitle stepid="Check that title is correct"
text="ABC, Inc. Human Resources Portal"/>
<clicklink stepid="Click Search for Employees link"
label="Search for Employees"/>
<verifytext stepid="Find 'Employee Search' text"
text="Employee Search"/>
The first step of this fragment, Get main page, uses the invoke task to access the page of the application. The URL is relative to the settings in the config element. Next, the title of the displayed page is verified. Following that, the Search link is clicked. Finally, the text "Employee Search" is searched for and verified. This XML fragment is used in both of the test targets.
The test case to Search by Name uses these entities. First, the config entity is referenced to indicate the basic configuration. Next, the getSearchPage entity is used. At this point in the test, the Search page should be displayed. Next, the setinputfield element is used to emulate a user typing "Jim" into the name field. The clickbutton task is then used to submit the form. The last step is the most interesting. Canoo allows you to use XPath expressions to identify and verify portions of the expected response. In this case, the XPath attribute of the verifyxpath task refers to the first table data cell of the first row in the table. The text attribute indicates what the expected value should be. The text attribute can be either normal text to search for or a regular expression. Here, a regular expression is used to indicate that the table data cell referenced by the XPath expression should contain the text "JIM" regardless of case.
The second test target, nameSearchNoResults, verifies proper behavior when the search results in no matches for the entered name. This target reuses the getSearchPage steps. Also, it uses the simpler verifytext task to check if the resulting response contains the expected message-No Employees Found.
As you can see, Canoo WebTest is very powerful. Test cases map to automated test specifications in a natural and easy-to-understand fashion. The test specification tasks are well documented and can be written by nondevelopers. WebTest also provides the capability to generate test reports for a test specification. These reports, generated in XML, can then be rendered in HTML using an XSLT stylesheet. Check the Canoo WebTest samples for examples of how to utilize this functionality. Finally, the use of Ant as the underlying framework means easier integration with cross-platform build processes.
The one aspect that WebTest does not easily provide verifications for is presentation and layout. These verifications can be made using XPath; however, wholesale use of XPath to verify the presentation of a page can get extensive and difficult to maintain.
Depending on the size of your application, you may want to consider a testing tool that provides record and playback capability. Such tools are usually not open source or free; however, they are usually quite powerful and may be necessary for large sites- particularly if those sites use a lot of JavaScript and other Dynamic HTML techniques. Selenium (http://www.openqa.org/selenium), from the Open QA quality assurance project repository, is one such open-source test tool for Web applications that provides record and playback functionality.
No comments:
Post a Comment