Resource: Ch. 3, "Testing and Debugging", of Data Structures: Abstraction and Design Using Java.Complete the Exercises for Section 3.2, Programming #1, in Ch. 3, "Testing and Debugging", of Data Struc
3.2 Specifying the Tests
In this section, we discuss how to specify the tests needed to test a program system and its components. The test data may be specified during the analysis and design phases. This should be done for the different levels of testing: unit, integration, and system. In black-box testing, we are concerned with the relationship between the unit inputs and outputs. There should be test data to check for all expected inputs as well as unanticipated data. The test plan should also specify the expected unit behavior and outputs for each set of input data.
In white-box testing, we are concerned with exercising alternative paths through the code. Thus, the test data should be designed to ensure that all ifstatement conditions will evaluate to both true and false. For nested if statements, test different combinations of true and false values. For switchstatements, make sure that the selector variable can take on all values listed as case labels and some that are not.
For loops, verify that the result is correct if an immediate exit occurs (zero repetitions). Also, verify that the result is correct if only one iteration is performed and if the maximum number of iterations is performed. Finally, verify that loop repetition can always terminate.
Testing Boundary Conditions
When hand-tracing through an algorithm using white-box testing, you must exercise all paths through the algorithm. It is also important to check special cases called boundary conditions to make sure that the algorithm works for these cases as well as the more common ones. For example, if you are testing a method that searches for a particular target element in an array testing, the boundary conditions means that you should make sure that the method works for the following special cases:
The target element is the first element in the array.
The target element is the last element in the array.
The target is somewhere in the middle.
The target element is not in the array.
There is more than one occurrence of the target element, and we find the first occurrence.
The array has only one element and it is not the target.
The array has only one element and it is the target.
The array has no elements.
These boundary condition tests would be required in black-box testing too.
EXERCISES FOR SECTION 3.2
SELF-CHECK
List two boundary conditions that should be checked when testing method readInt below. The second and third parameters represent the upper and lower bounds for a range of valid integers.
/** Returns an integer data value within range minN and maxN inclusive
* @param scan a Scanner object
* @param minN smallest possible value to return
* @param maxN largest possible value to return
* @return the first value read between minN and maxN
*/
public static int readInt (Scanner scan, int minN, int maxN) {
if (minN > maxN)
throw new IllegalArgumentException ("In readlnt, minN " + minN + " not <= maxN " + maxN) ;
boolean inRange = false; // Assume no valid number read.
int n = 0;
while (!inRange) { // Repeat until valid number read.
System.out.println("Enter an integer from " + minN + " to " + maxN + ": ") ;
try {
n = scan.nextlnt();
inRange = (minN <= n & & n <= maxN) ;
} catch (InputMismatchException ex) {
scan.nextLine();
System.out.println("not an integer - try again");
}
} // End while
return n; // n is in range
}
Devise test data to test the method readInt using
white-box testing
black-box testing
PROGRAMMING
Write a search method with four parameters: the search array, the target, the start subscript, and the finish subscript. The last two parameters indicate the part of the array that should be searched. Your method should catch or throw exceptions where warranted.
A test harness is a driver program written to test a method or class. It does this by providing known inputs for a series of tests, called a test suite, and then compares the expected and actual results of each test and an indication of pass or fail.
A test framework is a software product that facilitates writing test cases, organizing the test cases into test suites, running the test suites, and reporting the results. One test framework often used for Java projects is JUnit, an open-source product that can be used in a stand-alone mode and is available from junit.org. It is also bundled with at least two popular IDEs (NetBeans and Eclipse). In the next section, we show a test suite for the ArraySearchclass constructed using the JUnit framework.
JUnit uses the term test suite to represent the collection of tests to be run at one time. A test suite may consist of one or more classes that contain the individual tests. These classes are called test harnesses. A test harness may also contain common code to be executed before/after each test so that the class being tested is in a known state.
Each test harness in a test suite that will be run by the JUnit main method (called a test runner) begins with the two import statements:
import org.junit.Test;
import static org.junit.Assert.*;
The first import makes the Test interface visible, which allows us to use the @Test attribute to identify test cases. Annotations such as @Test are directions to the compiler and other language-processing tools; they do not affect the execution of the program. The JUnit main method (test runner) searches the classes that are listed in the args parameter for methods with the @Test annotation. When it executes them, it keeps track of the pass/fail results.
The second import statement makes the methods in the Assert class visible. The assert methods are used to determine pass/fail for a test. Table 3.2describes the various assert methods that are defined in org.junit.Assert. If an assert method fails, then an exception is thrown causing an error to be reported as specified in the description for method assertArrayEquals. If one of the assertions fail, then the test fails; if none fails, then the test passes.
TABLE 3.2Methods Defined in org.junit.Assert
| Method | Parameters | Description |
| assertArrayEquals | [message,] expected, actual | Tests to see whether the contents of the two array parameters expected and actual are equal. This method is overloaded for arrays of the primitive types and Object. Arrays of Objects are tested with the .equals method applied to the corresponding elements. The test fails if an unequal pair is found, and an AssertionError is thrown. If the optional message is included, the AssertionError is thrown with this message followed by the default message; otherwise it is thrown with a default message |
| assertEquals | [message,] expected, actual | Tests to see whether expected and actual are equal. This method is overloaded for the primitive types and Object.To test Objects, the .equals method is used |
| assertFalse | [message,] condition | Tests to see whether the boolean expression condition is false |
| assertNotNull | [message,] object | Tests to see if the object is not null |
| assertNotSame | [message,] expected, actual | Tests to see if expected and actual are not the same object. (Applies the != operator.) |
| assertNull | [message,] object | Tests to see whether the object is null |
| assertSame | [message,] expected, actual | Tests to see whether expected and actual are the same object. (Applies the == operator.) |
| assertTrue | [message,] condition | Tests to see whether the boolean expression condition is true |
| fail | [message] | Always throws AssertionError |
EXAMPLE 3.3
Listing 3.1 shows a JUnit test harness for an array search method (ArraySearch.search) that returns the location of the first occurrence of a target value (the second parameter) in an array (the first parameter) or −1 if the target is not found. The test harness contains methods that implement the tests first described in Section 3.2 and repeated below.
The target element is the first element in the array.
The target element is the last element in the array.
The target is somewhere in the middle.
The target element is not in the array.
There is more than one occurrence of the target element and we find the first occurrence.
The array has only one element and it is not the target.
The array has only one element and it is the target.
The array has no elements.
Method firstElementTest in Listing 3.1 implements the first test case. It tests to see whether the target is the first element in the 7-element array {5, 12, 15, 4, 8, 12, 7}. In the statement
assertEquals("5 is not found at position 0", 0, ArraySearch.search(x, 5));
the call to method ArraySearch.search returns the location of the target (5) in array x. The test passes (ArraySearch.search returns 0), and JUnitremembers the result. After all tests are run, JUnit displays a message such as
Testsuite: KW.CH03New.ArraySearchTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.111 sec
where 0.111 is the execution time in seconds. The Netbeans IDE also shows a Test Results window as shown in Figure 3.1. (The gray box at the end of Section 3.5 shows how to access JUnit in Netbeans.)
FIGURE 3.1 Test Results
If instead we used the following statement that incorrectly searches for target 4 as the first element
assertEquals("4 is not found at position 0", 0, ArraySearch.search(x, 4));
the test would fail and AssertionErrorException would display the messages
Testcase: firstElementTest: FAILED
4 not found a position 0 expected:<0> but was:<3>
If we omitted the first argument in the call to assertEquals, the default message
Testcase: firstElementTest: FAILED
expected:<0> but was:<3>
would be displayed instead where 3 is the position of the target 4.
LISTING 3.1
JUnit test of ArraySearch.search
import org.junit.Test;
import static org.junit.Assert.*;
/**
* JUnit test of ArraySearch.search
* @author Koffman and Wolfgang
*/
public class ArraySearchTest {
// Common array to search for most of the tests
private final int[] x = {5, 12, 15, 4, 8, 12, 7};
@Test
public void firstElementTest() {
// Test for target as first element.
assertEquals("5 not at position 0",
0, ArraySearch.search(x, 5));
}
@Test
public void lastElementTest() {
// Test for target as last element.
assertEquals("7 not at position 6",
6, ArraySearch.search(x, 7));
}
@Test
public void inMiddleTest() {
// Test for target somewhere in middle.
assertEquals("4 is not found at position 3",
3, ArraySearch.search(x, 4));
}
@Test
public void notInArrayTest() {
// Test for target not in array.
assertEquals(-1, ArraySearch.search(x, -5));
}
@Test
public void multipleOccurencesTest() {
// Test for multiple occurrences of target.
assertEquals(1, ArraySearch.search(x, 12));
}
@Test
public void oneElementArrayTestItemPresent() {
// Test for 1-element array
int[] y = {10};
assertEquals(0, ArraySearch.search(y, 10));
}
@Test
public void oneElementArrayTestItemAbsent() {
// Test for 1-element array
int[] y = {10};
assertEquals(-1, ArraySearch.search(y, -10));
}
@Test
public void emptyArrayTest() {
// Test for an empty array
int[] y = new int[0];
assertEquals(-1, ArraySearch.search(y, 10));
}
@Test(expected = NullPointerException.class)
public void nullArrayTest() {
int[] y = null;
int i = ArraySearch.search(y, 10);
}
}
The last test case in Listing 3.1 does not implement one of the tests in our earlier list of test cases. Its purpose is to test that the ArraySearch.searchmethod fails as it should when it is passed a null array. To tell JUnit that a test is expected to throw an exception, we added the parameter expected =exception-class to the @Test attribute. This parameter tells JUnit that the test should cause a NullPointerException (the result of calling ArraySearch.search with a null array). Without this parameter, JUnit would have indicated that the test case did not pass but reported an error instead. If the exception is not thrown as expected, the test will fail.
EXERCISES FOR SECTION 3.4
SELF-CHECK
Modify the test(s) in the list for Example 3.3 to verify a method that finds the last occurrence of a target element in an array?
List the boundary conditions and tests needed for a method with the following heading:
/**
* Search an array to find the first occurrence of the
* largest element
* @param x Array to search
* @return The subscript of the first occurrence of the
* largest element
* @throws NullPointerException if x is null
*/
public static int findLargest(int[] x) {
PROGRAMMING
Write the JUnit test harness for the method described in Self-Check Question 1.
Write the JUnit test harness for the tests listeded in Self-Check exercise 1.
3.5 Test-Driven Development
Rather than writing a complete method and then testing it, test-driven development involves writing the tests and the method in parallel. The sequence is as follows:
Write a test case for a feature.
Run the test and observe that it fails, but other tests still pass.
Make the minimum change necessary to make the test pass.
Revise the method to remove any duplication between the code and the test.
Rerun the test to see that it still passes.
We then repeat these steps adding a new feature until all of the requirements for the method have been implemented.
We will use this approach to develop a method to find the first occurrence of a target in an array.
Case Study: Test-Driven Development of ArraySearch.search
Write a program to search an array that performs the same way as Java method ArraySearch.search. This method should return the index of the first occurrence of a target in an array, or −1−1 if the target is not present.
We start by creating a test list like that in the last section and then work through them one at a time. During this process, we may think of additional tests to add to the test list.
Our test list is as follows:
The target element is not in the list.
The target element is the first element in the list.
The target element is the last element in the list.
There is more than one occurrence of the target element and we find the first occurrence.
The target is somewhere in the middle.
The array has only one element.
The array has no elements.
We start by creating a stub for the method we want to code:
/**
* Provides a static method search that searches an array
* @author Koffman & Wolfgang
*/
public class ArraySearch {
/**
* Search an array to find the first occurrence of a target
* @param x Array to search
* @param target Target to search for
* @return The subscript of the first occurrence if found:
* otherwise return -1
* @throws NullPointerException if x is null
*/
public static int search(int[] x, int target) {
return Integer.MIN_VALUE;
}
}
Now, we create the first test that combines tests 1 and 6 above. We will screen the test code in gray to distinguish it from the search method code.
/**
* Test for ArraySearch class
* @author Koffman & Wolfgang
*/
public class ArraySearchTest {
@Test
public void itemNotFirstElementInSingleElementArray() {
int[] x = {5};
assertEquals(-1, ArraySearch.search(x, 10));
}
}
And when we run this test, we get the message:
Testcase: itemNotFirstElementInSingleElementArray: FAILED
expected:<-1> but was:<-2147483648>
The minimum change to enable method search to pass the test is
public static int search(int[] x, int target) {
return -1; // target not found
}
Now, we can add a second test to see whether we find the target in the first element (tests 2 and 6 above).
@Test
public void itemFirstElementInSingleElementArray() {
int[] x = new int[]{5};
assertEquals(0, ArraySearch.search(x, 5));
}
As expected, this test fails because the search method returns −1. To make it pass, we modify our search method:
public static int search(int[] x, int target) {
if (x[0] == target) {
return 0; // target found at 0
}
return -1; // target not found
}
Both tests for a single element array now pass. Before moving on, let us see whether we can improve this. The process of improving code without changing its functionality is known as refactoring. Refactoring is an important step in test-driven development. It is also facilitated by TDD since having a working test suite gives you the confidence to make changes. (Kent Beck, a proponent of TDD says that TDD gives courage.1)
The statement:
return 0;
is a place for possible improvement. The value 00 is the index of the target. For a single element array this is obviously 00, but for larger arrays it may be different. Thus, an improved version is
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
return -1; // target not found
}
Now, let us see whether we can find an item that is last in a larger array (test 3 above). We start with a 2-element array:
@Test
public void itemSecondItemInTwoElementArray() {
int[] x = {10, 20};
assertEquals(1, ArraySearch.search(x, 20));
}
The first two tests still pass, but the new test fails. As expected, we get the message:
Testcase: itemSecondItemInTwoElementArray: FAILED
expected:<1> but was:<-1>
The test failed because we did not compare the second array element to the target. We can modify the method to do this as shown next.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
if (x[index] == target)
return index; // target at 1
return -1; // target not found
}
However, this would result in an ArrayOutOfBoundsException error for test itemNotFirstElementInSingleElementArray because there is no second element in the array {5}. If we change the method to first test that there is a second element before comparing it to target, all tests will pass.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
if (index < x.length) {
if (x[index] == target)
return index; // target at 1
}
return -1; // target not found
}
However, what happens if we increase the number of elements beyond 2?
@Test
public void itemLastInMultiElementArray() {
int[] x = new int[]{5, 10, 15};
assertEquals(2, ArraySearch.search(x, 15));
}
This test would fail because the target is not at position 0 or 1. To make it pass, we could continue to add if statements to test more elements, but this is a fruitless approach. Instead, we should modify the code so that the value of index advances to the end of the array. We can change the second if to a while and add an increment of index.
public static int search(int[] x, int target) {
int index = 0;
if (x[index] == target)
return index; // target at 0
index = 1;
while (index < x.length) {
if (x[index] == target)
return index; // target at index
index++;
}
return -1; // target not found
}
At this point, we have a method that will pass all of the tests for any size array. We can group all the tests in a single testing method to verify this.
@Test
public void verificationTests() {
int[] x = {5, 12, 15, 4, 8, 12, 7};
// Test for target as first element
assertEquals(0, ArraySearch.search(x, 5));
// Test for target as last element
assertEquals(6, ArraySearch.search(x, 7));
// Test for target not in array
assertEquals(-1, ArraySearch.search(x, -5));
// Test for multiple occurrences of target
assertEquals(1, ArraySearch.search(x, 12));
// Test for target somewhere in middle
assertEquals(3, ArraySearch.search(x, 4));
}
Although it may look like we are done, we are not finished because we also need to check that an empty array will always return −1:
@Test
public void itemNotInEmptyArray() {
int[] x = new int[0];
assertEquals(-1, ArraySearch.search(x, 5));
}
Unfortunately, this test does not pass because of an ArrayIndexOutofboundsException in the first if condition for method search (there is no element x[0]in an empty array). If we look closely at the code for search, we see that the initial test for when index is 0 is the same as for the other elements. So we can remove the first statement and start the loop at 0 instead of 1 (another example of refactoring). Our code becomes more compact and this test will also pass. A slight improvement would be to replace the while with a for statement.
public static int search(int[] x, int target) {
int index = 0;
while (index < x.length) {
if (x[index] == target)
return index; // target at index
index++;
}
return -1; // target not found
}
Finally, if we pass a null pointer instead of a reference to an array, a NullPointerException should be thrown (an additional test not in our original list).
@Test(expected = NullPointerException.class)
public void nullValueOfXThrowsException() {
assertEquals(0, ArraySearch.search(null, 5));
}
JUnit in Netbeans
It is fairly easy to create a JUnit test harness in Netbeans. Once you have written class ArraySearch.java, right click on the class name in the Projects view and then select Tools −> Create/Update Tests. A Create Tests window will pop up. Select OK and then a Select JUnit Version window will pop up: select the most recent version of JUnit (currently JUnit 4.x). At this point, a new class will be created (ArraySearchTest.java) that will contain prototype tests for all the public functions in class ArraySearch. You can replace the prototype tests with your own. To execute the tests, right click on class ArraySearchTest.
EXERCISES FOR SECTION 3.5
SELF-CHECK
Why did the first version of method search that passed the first test itemNotFirstElementInSingleElementArray contain only the statement return −1?
Assume the first JUnit test for the findLargest method described in Self-Check exercise 2 in section 3.4 is a test that determines whether the first item in a one element array is the largest. What would be the minimal code for a method findLargest that passed this test?
PROGRAMMING
Write the findLargest method described in self-check exercise 2 in section 3.4 using Test-Driven Development.
