Univeristy of PhoenixDAT/305Individual: Testing and Debugging, Section 3.2Resource: Ch. 3, "Testing and Debugging", of Data Structures: Abstraction and Design Using Java.Complete the Exercises for Sec
Chapter 3
Testing and Debugging
Chapter Objectives
To understand different testing strategies and when and how they are performed
To introduce testing using the JUnit test framework
To show how to write classes using test-driven development
To illustrate how to use a debugger within a Java Integrated Development Environment (IDE)
This chapter introduces and illustrates some testing and debugging techniques. We begin by discussing testing in some detail. You will learn how to generate a proper test plan and the differences between unit and integration testing as they apply to an object-oriented design (OOD). Next we describe the JUnit Test Framework, which has become a commonly used tool for creating and running unit tests. We also describe test-driven development, a design approach in which the tests and programs are developed in parallel. Finally, we illustrate the debugger, which allows you to suspend program execution at a specified point and examine the value of variables to assist in isolating errors.
Testing and Debugging
3.1 Types of Testing
3.2 Specifying the Tests
3.3 Stubs and Drivers
3.4 Testing Using the JUnit Framework
3.5 Test-Driven Development
Case Study: Finding a target in an array.
3.6 Testing Interactive Methods
3.7 Debugging a Program
3.1 Types of Testing
Testing is the process of exercising a program (or part of a program) under controlled conditions and verifying that the results are as expected. The purpose of testing is to detect program defects after all syntax errors have been removed and the program compiles successfully. The more thorough the testing, the greater the likelihood that the defects will be found. However, no amount of testing can guarantee the absence of defects in sufficiently complex programs. The number of test cases required to test all possible inputs and states that each method may execute can quickly become prohibitively large. That is often why commercial software products have different versions or patches that the user must install. Version n
usually corrects the defects that were still present in version n−1
Testing is generally done at the following levels and in the sequence shown below:
Unit testing refers to testing the smallest testable piece of the software. In OOD, the unit will be either a method or a class. The complexity of a method determines whether it should be tested as a separate unit or whether it can be tested as part of its class.
Integration testing involves testing the interactions among units. If the unit is the method, then integration testing includes testing interactions among methods within a class. However, generally it involves testing interactions among several classes.
System testing is the testing of the whole program in the context in which it will be used. A program is generally part of a collection of other programs and hardware, called a system. Sometimes a program will work correctly until some other software is loaded onto the system and then it will fail for no apparent reason.
Acceptance testing is system testing designed to show that the program meets its functional requirements. It generally involves use of the system in the real environment or as close to the real environment as possible.
There are two types of testing:
Black-box testing tests the item (method, class, or program) based on its interfaces and functional requirements. This is also called closed-box testing or functional testing. For testing a method, the input parameters are varied over their allowed range and the results compared against independently calculated results. In addition, values outside the allowed range are tested to ensure that the method responds as specified (e.g., throws an exception or computes a nominal value). Also, the inputs to a method are not only the parameters of the method, but also the values of any global data that the method accesses.
White-box testing tests the software element (method, class, or program) with the knowledge of its internal structure. Other terms used for this type of testing are glass-box testing, open-box testing, and coverage testing. The goal is to exercise as many paths through the element as possible or practical. There are various degrees of coverage. The simplest is statement coverage, which ensures that each statement is executed at least once. Branch coverage ensures that every choice at each branch (if statements, switch statements, and loops) is tested. For example, if there are only if statements, and they are not nested, then each if statement is tried with its condition true and with its condition false. This could possibly be done with two test cases: one with all of the if conditions true and the other with all of them false. Path coverage tests each path through a method. If there are n
if statements, path coverage could require 2n test cases if the if statements are not nested (each condition has two possible values, so there could be 2n possible paths).
EXAMPLE 3.1
Method testMethod has a nested if statement and displays one of four messages, path 1 through path 4, depending on which path is followed. The values passed to its arguments determine the path. The ellipses represent the other statements in each path.
public void testMethod(char a, char b) {
if (a < 'M') {
if (b < 'X') {
System.out.println("path 1");
…
} else {
System.out.println("path 2");
…
}
} else {
if (b < 'C') {
System.out.println("path 3");
…
} else {
System.out.println("path 4");
…
}
}
To test this method, we need to pass values for its arguments that cause it to follow the different paths. Table 3.1 shows some possible values and the corresponding path.
TABLE 3.1 Testing All Paths of testMethod
a b Message
‘A’ ‘A’ path 1
‘A’ ‘Z’ path 2
‘Z’ ‘A’ path 3
‘Z’ ‘Z’ path 4
The values chosen for a and b in Table 3.1 are the smallest and largest uppercase letters. For a more thorough test, you should see what happens when a and b are passed values that are between A and Z. For example, what happens if the value changes from L to M? We pick those values because the condition (a < 'M') has different values for each of them.
Also, what happens when a and b are not uppercase letters? For example, if a and b are both digit characters (e.g., '2'), the path 1 message should be displayed because the digit characters precede the uppercase letters (see Appendix A, Table A.2). If a and b are both lowercase letters, the path 4 message should be displayed (Why?). If a is a digit and b is a lowercase letter, the path 2 message should be displayed (Why?). As you can see, the number of test cases required to test even a simple method such as testMethod thoroughly can become quite large.
Preparations for Testing
Although testing is usually done after each unit of the software is coded, a test plan should be developed early in the design stage. Some aspects of a test plan include deciding how the software will be tested, when the tests will occur, who will do the testing, and what test data will be used. If the test plan is developed early in the design stage, testing can take place concurrently with the design and coding. Again, the earlier an error is detected, the easier and less expensive it is to correct it.
Another advantage of deciding on the test plan early is that this will encourage programmers to prepare for testing as they write their code. A good programmer will practice defensive programming and include code that detects unexpected or invalid data values. For example, if the parameter n
for a method is required to be greater than zero, you can place the if statement
if (n <= 0)
throw new IllegalArgumentException("n <= 0: " + n);
at the beginning of the method. This if statement will throw an exception and provide a diagnostic message in the event that the parameter passed to the method is invalid. Method exit will occur and the exception can be handled by the method caller.
Testing Tips for Program Systems
Most of the time, you will be testing program systems that contain collections of classes, each with several methods. Next, we provide a list of testing tips to follow in writing these methods.
Carefully document each method parameter and class attribute using comments as you write the code. Also, describe the method operation using comments, following the Javadoc conventions discussed in Section A.7.
Leave a trace of execution by displaying the method name as you enter it.
Display the values of all input parameters upon entry to a method. Also, display the values of any class attributes that are accessed by this method. Check that these values make sense.
Display the values of all method outputs after returning from a method. Also, display any class attributes that are modified by this method. Verify that these values are correct by hand computation.
You should plan for testing as you write each module rather than after the fact. Include the output statements required for Steps 2 and 3 in the original Java code for the method. When you are satisfied that the method works as desired, you can “remove” the testing statements. One efficient way to remove them is to enclose them in an if (TESTING) block as follows:
if (TESTING) {
// Code that you wish to "remove"
. . .
}
You would then define TESTING at the beginning of the class as true to enable testing,
private static final boolean TESTING = true;
or as false to disable testing,
private static final boolean TESTING = false;
If you need, you can define different boolean flags for different kinds of tests.
EXERCISES FOR SECTION 3.1
SELF-CHECK
Explain why a method that does not match its declaration in the interface would not be discovered during white-box testing.
During which phase of testing would each of the following tests be performed?
Testing whether a method worked properly at all its boundary conditions.
Testing whether class A can use class B as a component.
Testing whether a phone directory application and a word-processing application can run simultaneously on a personal computer.
Testing whether a method search can search an array that was returned by method buildArray that stores input data in the array.
Testing whether a class with an array data field can use a static method search defined in a class ArraySearch.
3.2 Specifying the Tests
In this section, we discuss how to specify the tests needed to test a program system and its components. The test data may be specified during the analysis and design phases. This should be done for the different levels of testing: unit, integration, and system. In black-box testing, we are concerned with the relationship between the unit inputs and outputs. There should be test data to check for all expected inputs as well as unanticipated data. The test plan should also specify the expected unit behavior and outputs for each set of input data.
In white-box testing, we are concerned with exercising alternative paths through the code. Thus, the test data should be designed to ensure that all if statement conditions will evaluate to both true and false. For nested if statements, test different combinations of true and false values. For switch statements, make sure that the selector variable can take on all values listed as case labels and some that are not.
For loops, verify that the result is correct if an immediate exit occurs (zero repetitions). Also, verify that the result is correct if only one iteration is performed and if the maximum number of iterations is performed. Finally, verify that loop repetition can always terminate.
Testing Boundary Conditions
When hand-tracing through an algorithm using white-box testing, you must exercise all paths through the algorithm. It is also important to check special cases called boundary conditions to make sure that the algorithm works for these cases as well as the more common ones. For example, if you are testing a method that searches for a particular target element in an array testing, the boundary conditions means that you should make sure that the method works for the following special cases:
The target element is the first element in the array.
The target element is the last element in the array.
The target is somewhere in the middle.
The target element is not in the array.
There is more than one occurrence of the target element, and we find the first occurrence.
The array has only one element and it is not the target.
The array has only one element and it is the target.
The array has no elements.
These boundary condition tests would be required in black-box testing too.
EXERCISES FOR SECTION 3.2
SELF-CHECK
List two boundary conditions that should be checked when testing method readInt below. The second and third parameters represent the upper and lower bounds for a range of valid integers.
/** Returns an integer data value within range minN and maxN inclusive
* @param scan a Scanner object
* @param minN smallest possible value to return
* @param maxN largest possible value to return
* @return the first value read between minN and maxN
*/
public static int readInt (Scanner scan, int minN, int maxN) {
if (minN > maxN)
throw new IllegalArgumentException ("In readlnt, minN " + minN + " not <= maxN " + maxN) ;
boolean inRange = false; // Assume no valid number read.
int n = 0;
while (!inRange) { // Repeat until valid number read.
System.out.println("Enter an integer from " + minN + " to " + maxN + ": ") ;
try {
n = scan.nextlnt();
inRange = (minN <= n & & n <= maxN) ;
} catch (InputMismatchException ex) {
scan.nextLine();
System.out.println("not an integer - try again");
}
} // End while
return n; // n is in range
}
Devise test data to test the method readInt using
white-box testing
black-box testing
PROGRAMMING
Write a search method with four parameters: the search array, the target, the start subscript, and the finish subscript. The last two parameters indicate the part of the array that should be searched. Your method should catch or throw exceptions where warranted.
3.3 Stubs and Drivers
In this section, we describe two kinds of methods, stubs and drivers, that facilitate testing. We also show how to document the requirements that a method should meet using preconditions and postconditions.
Stubs
Although we want to do unit testing as soon as possible, it may be difficult to test a method or a class that interacts with other methods or classes. The problem is that not all methods and not all classes will be completed at the same time. So if a method in class A calls a method defined in class B (not yet written), the unit test for class A can't be performed without the help of a replacement method for the one in class B. The replacement for a method that has not yet been implemented or tested is called a stub. A stub has the same header as the method it replaces, but its body only displays a message indicating that the stub was called.
EXAMPLE 3.2
The following method is a stub for a void method save. The stub will enable a method that calls save to be tested, even though the real method save has not been written.
/** Stub for method save.
@pre the initial directory contents are read from a data file.
@post Writes the directory contents back to a data file.
The boolean flag modified is reset to false.
*/
public void save() {
System.out.println("Stub for save has been called");
modified = false;
}
Besides displaying an identification message, a stub can print out the values of the inputs and can assign predictable values (e.g., 0 or 1) to any outputs to prevent execution errors caused by undefined values. Also, if a method is supposed to change the state of a data field, the stub can do so (modified is set to false by the stub just shown). If a client program calls one or more stubs, the message printed by each stub when it is executed provides a trace of the call sequence and enables the programmer to determine whether the flow of control within the client program is correct.
Preconditions and Postconditions
In the comment for the method save, the lines
@pre the initial directory contents are read from a data file.
@post Writes the directory contents back to a data file.
The boolean flag modified is reset to false.
show the precondition (following @pre) and postcondition (following @post) for the method save. A precondition is a statement of any assumptions or constraints on the method data (input parameters) before the method begins execution. A postcondition describes the result of executing the method. A method's preconditions and postconditions serve as a contract between a method caller and the method programmer—if a caller satisfies the precondition, the method result should satisfy the postcondition. If the precondition is not satisfied, there is no guarantee that the method will do what is expected, and it may even fail. The preconditions and postconditions allow both a method user and a method implementer to proceed without further coordination.
We will use postconditions to describe the change in object state caused by executing a mutator method. As a general rule, you should write a postcondition comment for all void methods. If a method returns a value, you do not usually need a postcondition comment because the @return comment describes the effect of executing the method.
Drivers
Another testing tool for a method is a driver program. A driver program declares any necessary object instances and variables, assigns values to any of the method's inputs (as specified in the method's preconditions), calls the method, and displays the values of any outputs returned by the method. Alternatively, driver methods can be written in a separate test class and executed under the control of a test framework such as JUnit, which we discuss in the next section.
EXERCISES FOR SECTION 3.3
SELF-CHECK
Can a main method be used as a stub or a driver? Explain your answer.
PROGRAMMING
Writer a driver program to the test method readInt in Self Check exercise 1 of Section 3.2 using the test data derived for Self-Check Exercise 2, part b in Section 3.2.
Write a stub to use in place of the method readInt.
Write a driver program to test method search in programming exercise 1, Section 3.2.
3.4 The JUnit Test Framework
A test harness is a driver program written to test a method or class. It does this by providing known inputs for a series of tests, called a test suite, and then compares the expected and actual results of each test and an indication of pass or fail.
A test framework is a software product that facilitates writing test cases, organizing the test cases into test suites, running the test suites, and reporting the results. One test framework often used for Java projects is JUnit, an open-source product that can be used in a stand-alone mode and is available from junit.org. It is also bundled with at least two popular IDEs (NetBeans and Eclipse). In the next section, we show a test suite for the ArraySearch class constructed using the JUnit framework.
JUnit uses the term test suite to represent the collection of tests to be run at one time. A test suite may consist of one or more classes that contain the individual tests. These classes are called test harnesses. A test harness may also contain common code to be executed before/after each test so that the class being tested is in a known state.
Each test harness in a test suite that will be run by the JUnit main method (called a test runner) begins with the two import statements:
import org.junit.Test;
import static org.junit.Assert.*;
The first import makes the Test interface visible, which allows us to use the @Test attribute to identify test cases. Annotations such as @Test are directions to the compiler and other language-processing tools; they do not affect the execution of the program. The JUnit main method (test runner) searches the classes that are listed in the args parameter for methods with the @Test annotation. When it executes them, it keeps track of the pass/fail results.
The second import statement makes the methods in the Assert class visible. The assert methods are used to determine pass/fail for a test. Table 3.2 describes the various assert methods that are defined in org.junit.Assert. If an assert method fails, then an exception is thrown causing an error to be reported as specified in the description for method assertArrayEquals. If one of the assertions fail, then the test fails; if none fails, then the test passes.
TABLE 3.2 Methods Defined in org.junit.Assert
Method Parameters Description
assertArrayEquals [message,] expected, actual Tests to see whether the contents of the two array parameters expected and actual are equal. This method is overloaded for arrays of the primitive types and Object. Arrays of Objects are tested with the .equals method applied to the corresponding elements. The test fails if an unequal pair is found, and an AssertionError is thrown. If the optional message is included, the AssertionError is thrown with this message followed by the default message; otherwise it is thrown with a default message
assertEquals [message,] expected, actual Tests to see whether expected and actual are equal. This method is overloaded for the primitive types and Object. To test Objects, the .equals method is used
assertFalse [message,] condition Tests to see whether the boolean expression condition is false
assertNotNull [message,] object Tests to see if the object is not null
assertNotSame [message,] expected, actual Tests to see if expected and actual are not the same object. (Applies the != operator.)
assertNull [message,] object Tests to see whether the object is null
assertSame [message,] expected, actual Tests to see whether expected and actual are the same object. (Applies the == operator.)
assertTrue [message,] condition Tests to see whether the boolean expression condition is true
fail [message] Always throws AssertionError
EXAMPLE 3.3
Listing 3.1 shows a JUnit test harness for an array search method (ArraySearch.search) that returns the location of the first occurrence of a target value (the second parameter) in an array (the first parameter) or −1 if the target is not found. The test harness contains methods that implement the tests first described in Section 3.2 and repeated below.
The target element is the first element in the array.
The target element is the last element in the array.
The target is somewhere in the middle.
The target element is not in the array.
There is more than one occurrence of the target element and we find the first occurrence.
The array has only one element and it is not the target.
The array has only one element and it is the target.
The array has no elements.
Method firstElementTest in Listing 3.1 implements the first test case. It tests to see whether the target is the first element in the 7-element array {5, 12, 15, 4, 8, 12, 7}. In the statement
assertEquals("5 is not found at position 0", 0, ArraySearch.search(x, 5));
the call to method ArraySearch.search returns the location of the target (5) in array x. The test passes (ArraySearch.search returns 0), and JUnit remembers the result. After all tests are run, JUnit displays a message such as
Testsuite: KW.CH03New.ArraySearchTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.111 sec
where 0.111 is the execution time in seconds. The Netbeans IDE also shows a Test Results window as shown in Figure 3.1. (The gray box at the end of Section 3.5 shows how to access JUnit in Netbeans.)
image
FIGURE 3.1 Test Results
If instead we used the following statement that incorrectly searches for target 4 as the first element
assertEquals("4 is not found at position 0", 0, ArraySearch.search(x, 4));
the test would fail and AssertionErrorException would display the messages
Testcase: firstElementTest: FAILED
4 not found a position 0 expected:<0> but was:<3>
If we omitted the first argument in the call to assertEquals, the default message
Testcase: firstElementTest: FAILED
expected:<0> but was:<3>
would be displayed instead where 3 is the position of the target 4.
LISTING 3.1
JUnit test of ArraySearch.search
import org.junit.Test;
import static org.junit.Assert.*;
/**
* JUnit test of ArraySearch.search
* @author Koffman and Wolfgang
*/
public class ArraySearchTest {
// Common array to search for most of the tests
private final int[] x = {5, 12, 15, 4, 8, 12, 7};
@Test
public void firstElementTest() {
// Test for target as first element.
assertEquals("5 not at position 0",
0, ArraySearch.search(x, 5));
}
@Test
public void lastElementTest() {
// Test for target as last element.
assertEquals("7 not at position 6",
6, ArraySearch.search(x, 7));
}
@Test
public void inMiddleTest() {
// Test for target somewhere in middle.
assertEquals("4 is not found at position 3",
3, ArraySearch.search(x, 4));
}
@Test
public void notInArrayTest() {
// Test for target not in array.
assertEquals(-1, ArraySearch.search(x, -5));
}
@Test
public void multipleOccurencesTest() {
// Test for multiple occurrences of target.
assertEquals(1, ArraySearch.search(x, 12));
}
@Test
public void oneElementArrayTestItemPresent() {
// Test for 1-element array
int[] y = {10};
assertEquals(0, ArraySearch.search(y, 10));
}
@Test
public void oneElementArrayTestItemAbsent() {
// Test for 1-element array
int[] y = {10};
assertEquals(-1, ArraySearch.search(y, -10));
}
@Test
public void emptyArrayTest() {
// Test for an empty array
int[] y = new int[0];
assertEquals(-1, ArraySearch.search(y, 10));
}
@Test(expected = NullPointerException.class)
public void nullArrayTest() {
int[] y = null;
int i = ArraySearch.search(y, 10);
}
The last test case in Listing 3.1 does not implement one of the tests in our earlier list of test cases. Its purpose is to test that the ArraySearch.search method fails as it should when it is passed a null array. To tell JUnit that a test is expected to throw an exception, we added the parameter expected = exception-class to the @Test attribute. This parameter tells JUnit that the test should cause a NullPointerException (the result of calling ArraySearch.search with a null array). Without this parameter, JUnit would have indicated that the test case did not pass but reported an error instead. If the exception is not thrown as expected, the test will fail.
EXERCISES FOR SECTION 3.4
SELF-CHECK
Modify the test(s) in the list for Example 3.3 to verify a method that finds the last occurrence of a target element in an array?
List the boundary conditions and tests needed for a method with the following heading:
/**
* Search an array to find the first occurrence of the
* largest element
* @param x Array to search
* @return The subscript of the first occurrence of the
* largest element
* @throws NullPointerException if x is null
*/
public static int findLargest(int[] x) {
PROGRAMMING
Write the JUnit test harness for the method described in Self-Check Question 1.
Write the JUnit test harness for the tests listeded in Self-Check exercise 1.
3.5 Test-Driven Development
Rather than writing a complete method and then testing it, test-driven development involves writing the tests and the method in parallel. The sequence is as follows:
Write a test case for a feature.
Run the test and observe that it fails, but other tests still pass.
Make the minimum change necessary to make the test pass.
Revise the method to remove any duplication between the code and the test.
Rerun the test to see that it still passes.
We then repeat these steps adding a new feature until all of the requirements for the method have been implemented.
We will use this approach to develop a method to find the first occurrence of a target in an array.
Case Study: Test-Driven Development of ArraySearch.search
Write a program to search an array that performs the same way as Java method ArraySearch.search. This method should return the index of the first occurrence of a target in an array, or −1
if the target is not present.
We start by creating a test list like that in the last section and then work through them one at a time. During this process, we may think of additional tests to add to the test list.
Our test list is as follows:
The target element is not in the list.
The target element is the first element in the list.
The target element is the last element in the list.
There is more than one occurrence of the target element and we find the first occurrence.
The target is somewhere in the middle.
The array has only one element.
The array has no elements.
We start by creating a stub for the method we want to code:
/**
* Provides a static method search that searches an array
* @author Koffman & Wolfgang
*/
public class ArraySearch {
/**
* Search an array to find the first occurrence of a target
* @param x Array to search
* @param target Target to search for
* @return The subscript of the first occurrence if found:
* otherwise return -1
* @throws NullPointerException if x is null
*/
public static int search(int[] x, int target) {
return Integer.MIN_VALUE;
}
}
Now, we create the first test that combines tests 1 and 6 above. We will screen the test code in gray to distinguish it from the search method code.
/**
* Test for ArraySearch class
* @author Koffman & Wolfgang
*/
public class ArraySearchTest {
@Test
public void itemNotFirstElementInSingleElementArray() {
int[] x = {5};
assertEquals(-1, ArraySearch.search(x, 10));
}
And when we run this test, we get the message:
Testcase: itemNotFirstElementInSingleElementArray: FAILED
expected:<-1> but was:<-2147483648>
The minimum change to enable method search to pass the test is
public static int search(int[] x, int target) {
return -1; // target not found
}
Now, we can add a second test to see whether we find the target in the first element (tests 2 and 6 above).
@Test
public void itemFirstElementInSingleElementArray() {
int[] x = new int[]{5};
assertEquals(0, ArraySearch.search(x, 5));
As expected, this test fails because the search method returns −1. To make it pass, we modify our search method:
public static int search(int[] x, int target) {
if (x[0] == target) {
return 0; // target found at 0
}
return -1; // target not found
}
Both tests for a single element array now pass. Before moving on, let us see whether we can improve this. The process of improving code without changing its functionality is known as refactoring. Refactoring is an important step in test-driven development. It is also facilitated by TDD since having a working test suite gives you the confidence to make changes. (Kent Beck, a proponent of TDD says that TDD gives courage.1)
The statement:
return 0;
is a place for possible improvement. The value 0
is the index of the target. For a single element array this is obviously 0
, but for larger arrays it may be different. Thus, an improved version is
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
return -1; // target not found
}
Now, let us see whether we can find an item that is last in a larger array (test 3 above). We start with a 2-element array:
@Test
public void itemSecondItemInTwoElementArray() {
int[] x = {10, 20};
assertEquals(1, ArraySearch.search(x, 20));
}
The first two tests still pass, but the new test fails. As expected, we get the message:
Testcase: itemSecondItemInTwoElementArray: FAILED
expected:<1> but was:<-1>
The test failed because we did not compare the second array element to the target. We can modify the method to do this as shown next.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
if (x[index] == target)
return index; // target at 1
return -1; // target not found
}
However, this would result in an ArrayOutOfBoundsException error for test itemNotFirstElementInSingleElementArray because there is no second element in the array {5}. If we change the method to first test that there is a second element before comparing it to target, all tests will pass.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
if (index < x.length) {
if (x[index] == target)
return index; // target at 1
}
return -1; // target not found
}
However, what happens if we increase the number of elements beyond 2?
@Test
public void itemLastInMultiElementArray() {
int[] x = new int[]{5, 10, 15};
assertEquals(2, ArraySearch.search(x, 15));
}
This test would fail because the target is not at position 0 or 1. To make it pass, we could continue to add if statements to test more elements, but this is a fruitless approach. Instead, we should modify the code so that the value of index advances to the end of the array. We can change the second if to a while and add an increment of index.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
while (index < x.length) {
if (x[index] == target)
return index; // target at index
index++;
}
return -1; // target not found
}
At this point, we have a method that will pass all of the tests for any size array. We can group all the tests in a single testing method to verify this.
@Test
public void verificationTests() {
int[] x = {5, 12, 15, 4, 8, 12, 7};
// Test for target as first element
assertEquals(0, ArraySearch.search(x, 5));
// Test for target as last element
assertEquals(6, ArraySearch.search(x, 7));
// Test for target not in array
assertEquals(-1, ArraySearch.search(x, -5));
// Test for multiple occurrences of target
assertEquals(1, ArraySearch.search(x, 12));
// Test for target somewhere in middle
assertEquals(3, ArraySearch.search(x, 4));
}
Although it may look like we are done, we are not finished because we also need to check that an empty array will always return −1:
@Test
public void itemNotInEmptyArray() {
int[] x = new int[0];
assertEquals(-1, ArraySearch.search(x, 5));
Unfortunately, this test does not pass because of an ArrayIndexOutofboundsException in the first if condition for method search (there is no element x[0] in an empty array). If we look closely at the code for search, we see that the initial test for when index is 0 is the same as for the other elements. So we can remove the first statement and start the loop at 0 instead of 1 (another example of refactoring). Our code becomes more compact and this test will also pass. A slight improvement would be to replace the while with a for statement.
public static int search(int[] x, int target) {
int index = 0;
while (index < x.length) {
if (x[index] == target)
return index; // target at index
index++;
}
return -1; // target not found
}
Finally, if we pass a null pointer instead of a reference to an array, a NullPointerException should be thrown (an additional test not in our original list).
@Test(expected = NullPointerException.class)
public void nullValueOfXThrowsException() {
assertEquals(0, ArraySearch.search(null, 5));
JUnit in Netbeans
It is fairly easy to create a JUnit test harness in Netbeans. Once you have written class ArraySearch.java, right click on the class name in the Projects view and then select Tools −> Create/Update Tests. A Create Tests window will pop up. Select OK and then a Select JUnit Version window will pop up: select the most recent version of JUnit (currently JUnit 4.x). At this point, a new class will be created (ArraySearchTest.java) that will contain prototype tests for all the public functions in class ArraySearch. You can replace the prototype tests with your own. To execute the tests, right click on class ArraySearchTest.
EXERCISES FOR SECTION 3.5
SELF-CHECK
Why did the first version of method search that passed the first test itemNotFirstElementInSingleElementArray contain only the statement return −1?
Assume the first JUnit test for the findLargest method described in Self-Check exercise 2 in section 3.4 is a test that determines whether the first item in a one element array is the largest. What would be the minimal code for a method findLargest that passed this test?
PROGRAMMING
Write the findLargest method described in self-check exercise 2 in section 3.4 using Test-Driven Development.
3.6 Testing Interactive Programs in JUnit
In this section, we show how to use JUnit to test a method that gets an integer value in a specified range from the program user method readInt is defined in class MyInput:
/**
* Method to return an integer data value between two
* specified end points.
* @pre minN <= maxN.
* @param prompt Message
* @param minN Smallest value in range
* @param maxN Largest value in range
* @throws IllegalArgumentException
* @return The first data value that is in range
*/
public static int readInt(String prompt, int minN, int maxN) {
if (minN > maxN) {
throw new IllegalArgumentException("In readInt, minN " + minN + "not <= maxN " + maxN);
}
// Arguments are valid, read a number
boolean inRange = false; //Assume no valid number read
int n = 0;
Scanner in = new Scanner(System.in);
while (!inRange) {
try {
System.out.println(prompt + "\nEnter an integer between " + minN + " and " + maxN);
String line = in.nextLine();
n = Integer.parseInt(line);
inRange = (minN <= n && n <= maxN);
} catch (NumberFormatException ex) {
System.out.println("Bad numeric string - Try again");
}
}
return n;
}
The advantage of using a test framework such as JUnit is that tests are automated. They do not require any user input, and they always present the same test cases to the unit being tested. With JUnit, we can test that an IllegalArgumentException is thrown if the input parameters are not valid:
@Test(expected=IllegalArgumentException.class)
public void testForInvalidInput() {
int n = MyInput.readInt("Enter weight", 5, 2);
}
We also need to test that the method works correctly for values that are within a valid range and for values that are outside without requiring the user to enter these data. So we need to provide repeatable user input, and we need to verify that the output displayed is correct.
System.in is a public static field in the System class. It is initialized to an InputStream that reads from the system-defined standard input. The method System.setIn can be used to change the value of System.in. Similarly, System.out is initialized to a PrintStream that writes to the operating system-defined standard output, and the method System.setOut can be used to change it.
ByteArrayInputStream
The ByteArrayInputStream is an InputStream that provides input from a fixed array of bytes. The array is initialized by the constructor. Calls to the read method return successive bytes from the array until the end of the array is reached. Thus, if we wanted to test what readInt does when the string "3" is entered, we can do the following:
@Test
public void testForNormalInput() {
ByteArrayInputStream testIn = new ByteArrayInputStream("3".getBytes());
System.setIn(testIn);
int n = MyInput.readInt("Enter weight", 2, 5);
assertEquals(n, 3);
}
ByteArrayOutputStream
The ByteArrayOutputStream is an OutputStream that collects each byte written to it into an internal byte array. The toString method will then convert the current contents into a String. To capture and verify output written to System.out, we need to create a PrintStream that writes to a ByteArrayOutputStream and then set System.out to this PrintStream.
To verify that the prompt is properly displayed for the normal case, we use the following test:
@Test
public void testThatPromptIsCorrectForNormalInput() {
ByteArrayInputStream testIn = new ByteArrayInputStream("3".getBytes());
System.setIn(testIn);
ByteArrayOutputStream testOut = new ByteArrayOutputStream();
System.setOut(new PrintStream(testOut));
int n = MyInput.readInt("Enter weight", 2, 5);
assertEquals(n, 3);
String displayedPrompt = testOut.toString();
String expectedPrompt = "Enter weight" + "\nEnter an integer between 2 and 5" + NL;
assertEquals(expectedPrompt, displayedPrompt);
}
For the data string “3”, the string formed by testOut.toString() should match the string in expectedPrompt. String expectedPrompt ends with the string constant NL instead of \n. This is because the println method ends the output with a system-dependent line terminator. On the Windows operating system, this is the sequence \r\n, and on the Linux operating system it is \n. The statement
private static final String NL = System.getProperty("line.separator");
initializes the String constant NL to the system-specific line terminator.
Finally, we need to write additional tests (4 in all) that verify that method readInt works properly when an invalid integer string is entered and when the integer string entered is not in range. Besides verifying that the value returned is correct, you should verify the prompts displayed. For each of these tests, your test input should include a valid input following the invalid input so that the method will return normally. These tests are left to programming exercises 1 and 2. For example, the statement
ByteArrayInputStream testIn = new ByteArrayInputStream("X\n 3".getBytes());
would provide a data sample for the first test case ("X" is an invalid integer string, "3" is valid).
EXERCISES FOR SECTION 3.6
SELF-CHECK
Explain why it is not necessary to write a test to verify that readInt works properly when the input consists of an invalid integer string, followed by an out-of-range integer, then followed by an integer that is in range.
PROGRAMMING
Write separate tests to verify the result returned by readInt and the prompts displayed when the input consists of an invalid integer string followed by a valid integer. Verify that the expected error message is presented followed by a repeat of the prompt.
Write separate tests to verify the result returned by readInt and the prompts displayed when the input consists of an out-of-range integer followed by a valid integer. Verify that the expected error message is presented followed by a repeat of the prompt.
3.7 Debugging a Program
In this section, we will discuss the process of debugging (removing errors) both with and without the use of a debugger program. Debugging is the major activity performed by programmers during the testing phase. Testing determines whether you have an error; during debugging you determine the cause of run-time and logic errors and correct them, without introducing new ones. If you have followed the suggestions for testing described in the previous section, you will be well prepared to debug your program.
Debugging is like detective work. To debug a program, you must inspect carefully the information displayed by your program, starting at the beginning, to determine whether what you see is what you expect. For example, if the result returned by a method is incorrect but the arguments (if any) passed to the method had the correct values, then there is a problem inside the method. You can try to trace through the method to see whether you can find the source of the error and correct it. If you can't, you may need more information. One way to get that information is to insert additional diagnostic output statements in the method. For example, if the method contains a loop, you may want to display the values of loop control variables during loop execution.
EXAMPLE 3.4
The loop in Listing 3.2 does not seem to terminate when the user enters the sentinel string ("***"). The loop exits eventually after the user has entered 10 data items but the string returned contains the sentinel.
LISTING 3.2
The Method getSentence
/**
* Return the individual words entered by the user.
* The user can enter the sentinel *** to terminate data entry.
* @return A string with a maximum of 10 words
*/
public static String getSentence() {
Scanner in = new Scanner(System.in);
StringJoiner stb = new StringJoiner(" ");
int count = 0;
while (count < 10) {
System.out.println("Enter a word or *** to quit");
String word = in.next();
if (word == "***") break;
stb.add(word);
count++;
}
return stb.toString();
To determine the source of the problem, you should insert a diagnostic output statement that displays the values of word and count to make sure that word is receiving the sentinel string ("***"). You could insert the line
System.out.println("!!! Next word is " + word + ", count is " + count);
as the first statement in the loop body. If the third data item you enter is the sentinel string, you will get the output line:
!!! next word is ***, count is 2
This will show you that word does indeed receive the sentinel string, but the loop body continues to execute. Therefore, there must be something wrong with the if statement that tests for the sentinel. In fact, the if statement must be changed to
if (word == null || word.equals("***")) break;
because word == "***" compares the address of the string stored in word with the address of the literal string "***", not the contents of the two strings as intended. The strings’ addresses will always be different, even when their contents are the same. To compare their contents, the equals method must be used. Note that we needed to do the test word == null first because we would get a NullPointerException if equals was called when word was null.
Using a Debugger
If you are using an IDE, you will most likely have a debugger program as part of the IDE. A debugger can execute your program incrementally rather than all at once. After each increment of the program executes, the debugger pauses, and you can view the contents of variables to determine whether the statement(s) executed as expected. You can inspect all the program variables without needing to insert diagnostic output statements. When you have finished examining the program variables, you direct the debugger to execute the next increment.
You can choose to execute in increments as small as one program statement (called single-step execution) to see the effect of each statement's execution. Another possibility is to set breakpoints in your program to divide it into sections. The debugger can execute all the statements from one breakpoint to the next as a group. For example, if you wanted to see the effects of a loop's execution but did not want to step through every iteration, you could set breakpoints at the statements just before and just after the loop.
When your program pauses, if the next statement contains a call to a method, you can select single-step execution in the method being called (i.e., step into the method). Alternatively, you can execute all the method statements as a group and pause after the return from the method execution (i.e., step over the method).
The actual mechanics of using a debugger depends on the IDE that you are using. However, the process that you follow is similar among IDEs, and if you understand the process for one, you should be able to use any debugger. In this section, we demonstrate how to use the debugger in NetBeans, the IDE that is distributed by Sun along with the software development kit (SDK).
Before starting the debugger, you must set a breakpoint. Figure 3.2 is the display produced by the debugger at the beginning of debugging the GetSentence.main method. In NetBeans, you set a breakpoint by clicking in the vertical bar just to the left of the statement that you want to select as a breakpoint. The small squares and highlighted bars indicate the breakpoints. You can click again on a small square to remove the breakpoint. The source editor window displays the code to be debugged. The Debug pull-down menu shows the options for executing the code. The selected item, Step Into, is a common technique for starting single-step execution, as we have just described. A window (such as window Variables in the center left) typically shows the values of data fields and local variables. In this case, there is one local variable for method main: the String array args, which is empty. The arrow to the left of the highlighted line in the source editor window indicates the next step to execute (the call to method getSentence). Select Step Into again to execute the individual statements of method getSentence.
image
FIGURE 3.2 Using the Debugger for NetBeans
Figure 3.3 shows the editor and Variables windows after we have entered "Hello", "world", and "***". The contents of sentence is "Hello world", the value of count is 2, and the contents of word is "***". The next statement to execute is highlighted. It is the if statement, which tests for the sentinel. Although we expect the condition to be true, it is false (why?), so the loop continues to execute and "***" will be appended to sentence.
image
FIGURE 3.3 Editor and Debugging Windows
EXERCISES FOR SECTION 3.7
SELF-CHECK
The following method does not appear to be working properly if all data are negative numbers. Explain where you would add diagnostic output statements to debug it, and give an example of each statement.
/** Finds the target value in array elements x[start] through x[last].
@param x array whose largest value is found
@param start first subscript in range
@param last last subscript in range
@return the largest value of x[start] through x[last]
@pre first <= last
*/
public int findMax(int[] x, int start, int last) {
if (start > last)
throw new IllegalArgumentException("Empty range");
int maxSoFar = 0;
for (int i = start; i < last; i++) {
if (x[i] > maxSoFar)
maxSoFar = i;
}
return maxSoFar;
}
Explain the difference between selecting Step Into and Step Over during debugging.
Explain the rationale for the position of the breakpoints in method getSentence.
How would the execution of method getSentence change if the breakpoint were set at the statement just before the loop instead of at the loop heading?
PROGRAMMING
After debugging, provide a corrected version of the method in Self-Check Exercise 1. Leave the debugging statements in, but execute them only when the global constant TESTING is true.
Write and test a driver program to test method findMax in Self-Check exercise 1.
Chapter Review
Program testing is done at several levels starting with the smallest testable piece of the program, called a unit. A unit is either a method or a class, depending on the complexity.
Once units are individually tested, they can be tested together; this level is called integration testing.
Once the whole program is put together, it is tested as a whole; this level is called system testing.
Finally, the program is tested in an operational manner demonstrating its functionality; this is called acceptance testing.
Black-box (also called closed-box) testing tests the item (unit or system) based on its functional requirements without using any knowledge of the internal structure.
White-box (also called glass-box or open-box) testing tests the item using knowledge of its internal structure. One of the goals of white-box testing is to achieve test coverage. This can range from testing every statement at least once, to testing each branch condition (if statements, switch statements, and loops) to verify each possible path through the program.
Test drivers and stubs are tools used in testing. A test driver exercises a method or class and drives the testing. A stub stands in for a method that the unit being tested calls. This can be used to provide test results, and it can be used to enable a call of that method to be tested when the method being called is not yet coded.
The JUnit test framework is a software product that facilitates writing test cases, organizing the test cases into test suites, running the test suites, and reporting the results.
Test-Driven development is an approach to developing programs that has gained popularity among professional software developers. The approach is to write test cases one at a time and the make the changes to the program to pass the test.
Interactive programs can be tested by using the ByteArrayInputStream to provide known input and the ByteArrayOutputStream to verify output.
We described the debugging process and showed an example of how a debugger can be used to obtain information about a program's state.
Java API Classes Introduced in This Chapter
ByteArrayInputStream
ByteArrayOutputStream
User-Defined Interfaces and Classes in This Chapter
class ArraySearch
class MyInput
Quick-Check Exercises
______ testing requires the use of test data that exercises each statement in a module.
______ testing focuses on testing the functional characteristics of a module.
______ determines whether a program has an error: ______ determines the ______ of the error and helps you ______ it.
A method's ______ and ______ serve as a ______ between a method caller and the method programmer— if a caller satisfies the, the method result should satisfy the ______.
Explain how the old adage “We learn from our mistakes” applies to test-driven development.
Review Questions
Indicate in which state of testing (unit, integration, system) each of the following kinds of errors should be detected:
An array index is out of bounds.
A FileNotFoundException is thrown.
An incorrect value of withholding tax is being computed under some circumstances.
Describe the differences between stubs and drivers.
What is refactoring? How it is used in test-driven development?
Modify the list of tests for finding the first occurrence of a target in an array shown in Section 3.4 to finding the last occurrence.
Use the list of tests for review question 4. To develop a JUnit test harness.
Use test-driven development to code the method for finding the last occurrence of a target in an array.
Programming
Design and code a JUnit test harness for a method that finds the smallest element in an array.
Develop the method in project 1 using test-driven development.
Design and code the JUnit test cases for a class that computes the sine and cosine functions in a specialized manner. This class is going to be part of an embedded system running on a processor that does not support floating-point arithmetic or the Java Math class. The class to be tested is shown in Listing 3.3. You job is to test the methods sin and cos; you are to assume that the methods sin0to90 and sin45to90 have already been tested.
You need to design a set of test data that will exercise each of the if statements. To do this, look at the boundary conditions and pick values that are
Exactly on the boundary
Close to the boundary
Between boundaries
LISTING 3.3
SinCos.java
/** This class computes the sine and cosine of an angle expressed in degrees. The result will be an an integer representing the sine or cosine as ten-thousandths.
*/
public class SinCos {
/** Compute the sine of an angle in degrees.
@param x The angle in degrees
@return the sine of x
*/
public static int sin(int x) {
if (x < 0) {
x = -x;
}
x = x % 360;
if (0 <= x && x <= 45) {
return sin0to45(x);
} else if (45 <= x && x <= 90) {
return sin45to90(x);
} else if (90 <= x && x <= 180) {
return sin(180 - x);
} else {
return -sin(x - 180);
}
}
/** Compute the cosine of an angle in degrees
@param x The angle in degrees
@return the cosine of x
*/
public static int cos(int x) {
return sin(x + 90);
/** Compute the sine of an angle in degrees
between 0 and 45
@param x The angle
@return the sine of x
@pre 0 <= x < 45
*/
private static int sin0to45(int x) {
// In a realistic program this method would
// use a polynomial approximation that was
// optimized for the input range
// Insert code to compute sin(x) for x between 0 and 45 degrees
/** Compute the sine of an angle in degrees
between 45 and 90.
@param x - The angle
@return the sine of x
@pre 45 <= x <= 90
*/
private static int sin45to90(int x) {
// In a realistic program this method would
// use a polynomial approximation that was
// optimized for the input range
// Insert code to compute sin(x) for x between 45 and 90 degrees
}
Answers to Quick-Check Exercises
White-box testing requires the use of test data that exercises each statement in a module.
Black-box testing focuses on testing the functional characteristics of a module.
Testing determines whether a program has an error: debugging determines the cause of the error and helps you correct it.
A method's precondition and postcondition serve as a contract between a method caller and the method programmer—if a caller satisfies the precondition the method result should satisfy the postconditon.
In test-driven development, failing a test informs the method coder that the method developed so far needs to be modified. By repeated failure and adaptation of the code to pass the test just failed while still passing the rest of the tests in the test harness, the method coder develops a correct version of the desired method.
Note
1 Beck, Kent. Test-Driven Development by Example. Addison-Wesley, 2003.
