Introduction/Code Coverage
•	Introduction to testing and quality analysis
•	Testing in intro programming
•	Blackbox vs. Whitebox testing
•	Sample code coverage techniques
Introduction
•	Your first job in a software development company
•	Goal of "good" software
•	How can testing help?
Quality analysis
•	Not only produce good software, but do the job "well"
•	Want it to be profitable to maintain the software
•	ex: Excite search engine
•	Want to improve the process of software development (see next class)
Testing So Far
•	How do you test your homework assignments?
•	Are you sure that your program will work on data that you have not tried yet?
•	Do you think your approach will work on larger programs?
•	Why or why not?
•	Why is it difficult for big companies to produce bug-free software?
General principles of testing
•	Testing can show bugs are present, but can never show there are no bugs (Dijkstra)
•	All serious testing, even "freestyle" testing, requires a plan
•	There is not enough time and resources to test all possibilities
•	Need to decide when to stop testing
•	Want to find the max. # bugs using the least time/people
Blackbox testing
•	Blackbox testing: Emphasis is on functionality of code
•	results visible to user
•	What can the user see?
•	Blackbox testing usually covers most common paths through code
•	In any single execution of a program, many statements will not be executed
•	As a tester, why might this be a concern?
 Example -- queue of strings
•	Inputs: "e" to add string to end of queue
•	"d" to remove and print first string
•	"q" for quit
•	[case does matter]
•	Error messages if adding to full queue or removing from empty one
•	need to know capacity of queue
Whitebox testing
•	Whitebox testing: tests are based on structure of code
•	Assumption: failures related to control flow
•	Trying to test program units: single functions, classes
•	Try to ensure existing code is tested
•	Cannot detect if needed code is missing
•	Ex: Are all statements executed in at least one test case?
Example (slide 1 of 2)
•	template <class item, int max_items>
•	class queue {
•	    item items[max_items];
•	    int next_free;
•	    int next_full;
Example (slide 2 of 2)
•	void enqueue(item it) { // public
•		if (next_free < max_items) {
•			items[next_free] = it;
•			next_free++;
•		}
•	} // what situations do we want to test?
Whitebox and Blackbox testing
•	For simple programs, blackbox and whitebox testing are similar
•	Need both approaches
•	Programmers usually do whitebox testing of their code
•	Separate team often does blackbox testing
•	When the customer tries your software, what kind of testing are they doing?
Structural Testing Techniques
•	Statement Coverage
•	Decision Coverage
•	Condition Coverage
•	Path coverage
•	[more]
Statement Coverage
•	Ensure every statement executed in at least one test case
•	Statement = line of code?
•	What if a statement is not reachable?
•	Should we care?
•	Does Statement Coverage guarantee correctness?
Example
•	For enqueue(), enough to have one case where next_free < max_items
•	Ex: next_free = 0, max_items = 1
•	OR next_free = 1, max_items = 5 OR ...
•	Why "OR"?
Decision (Branch) Coverage
•	Ensure every decision is covered by at least one test case
•	Ex: if (condition) ... else ...
•	At least one test case makes condition true 
•	At least one test case makes condition false 
Decision Coverage
•	What if there's no else?
•	How does this compare with Statement coverage?
•	Does Decision coverage guarantee Statement coverage?
•	How does "short-circuit" logic affect decision coverage?
Example
•	For enqueue(), need one case with next_free < max_items AND one case where next_free >= max_items
•	Ex: next_free = 0, max_items = 1
•	and next_free = 1, max_items = 1
•	Further cases do not increase decision coverage
Condition Coverage
•	Ensure every condition is covered by at least one test case
•	enqueue() example: decision and condition coverage are equivalent
•	Ex: if ((x > 0) && (y > 0)) ...
•	test cases need to include x <= 0, x > 0, y <= 0, y > 0
•	What would decision coverage require?
•	Give an example where test cases satisfy condition coverage but not statement coverage
Path Coverage
•	Ensure every path through code executed in at least one test case
•	Ex: if (x > 0) { j++; } else { j--; }
•	        if (y > 0) { k++; }
•	What are the possible paths?
Path Coverage
•	Path coverage implies decision coverage
•	Why is Path Coverage not practical, in general?
Basis Path Testing
•	Compromise between decision and path coverage
•	Similar to concept of basis vectors
•	All paths can be formed as linear combination of basis paths
•	Find basis paths from control flow graph
Control flow graph
•	Graph for paths through block of code
•	Each node represents an indivisible set of statement(s)
•	cannot jump into/out of middle of node
•	source node: must enter block here
•	sink node: leave block here
•	Each edge represents a possible following statement(s)
Number of basis paths
•	Upper bound: # edges in control flow graph - # nodes + 2
•	Ex: single statement => #nodes? #edges? #basis paths?
•	This bound is also called the cyclomatic complexity
•	used to measure complexity of code
Basis path example
•	Ex: if (x > 0) { j++; } else { j--; }
•	        if (y > 0) { k++; }
•	Draw the graph
•	Possible basis paths?
•	Show how you can generate the other paths
What about loops?
•	Ex: for (i = 0, sum = 0; i < n; i++) sum += i;
•	Give an example test case for statement coverage
•	Decision coverage?
•	Basis paths?
Doing Coverage testing
•	Typical goal: 80% to 90% of statements/decisions/etc. covered by test cases
•	Why not 100%?