Process, Equivalence Classes
•	Reading: Kaner et al., ch. 1
•	Informal testing process
•	Sample problem
•	Equivalence classes
•	Boundary values
Random testing
•	Random testing
•	Choose random inputs to test software
•	What could go wrong?
•	[What could go right?]
What could go wrong with random testing
•	May miss important cases
•	May duplicate tests
•	May not be able to repeat tests
•	Debugging often requires running program to produce same result
Informal testing process
•	Test cycle 1 -- explore -- 5 steps
•	Start with obvious and simple test
•	Note what else needs testing
•	Check valid cases
•	Exploratory tests
•	Summarize program, problems
Informal testing process
•	Test cycle 2 -- adapt
•	Review responses to problem reports
•	Create new tests based on explanations of what will not be fixed
•	Goal: understand goals of developer
Informal testing process
•	Later test cycles -- refine, confirm
•	Formalize testing
•	Create stricter tests
•	(Last cycle) repeat all tests (why?)
•	General: try to get bugs fixed
Sample problem
•	Adding 2 numbers (each has 1-2 digits)
•	Start with 2 + 3 (step 1)
•	Program starts with "?" prompt for each number, then prints "5"
•	What are the problems so far?
•	a lot of programs just don't work -- that's why you're testing them!
Test Cycle 1 problems
•	No title
•	What program is this? [esp. consider multiple open windows]
•	No help
•	How do I know what to do?
•	How do I quit?
•	Try the "emacs" editor in Unix
Test Cycle 1 problems
•	5 is not lined up under 2 and 3
•	Format does matter (large numbers, real numbers)
•	Speaking up for end user
•	One problem report per error
•	Not all will be fixed
•	If fixed, may not be fixed during same cycle
Problem Report format (fig. 1.3)
•	Documentation helps protect you
•	How to reproduce problem
•	Need to identify version of software
•	Try to control random behavior (load the dice!)
•	Developer may be using different system/configuration
•	Try to use sys/config that will be used
Problem Report format (fig. 1.3)
•	Classify problem
•	Helps prioritize resolution
•	Helps direct report to appropriate person(s)
•	Testing is not just about the code
•	Space for response from developer(s)
Designing tests (step 2)
•	Goals
•	Thorough testing -- include cases most likely to fail
•	Efficiency: avoid wasted effort
•	Should be able to describe/justify test
•	One approach is to pick boundary values from equivalence classes
Equivalence Classes (Partitions)
•	Process:
•	Divide possible input into ranges or sets
•	Select at least one value in each range or set to test
General cases
•	Acceptable range of values: n1 to n2
•	What are the classes?
•	In general, how many classes will there be?
•	Acceptable set of values: {n1, n2, ... nm}
•	What are the classes?
•	In general, how many classes will there be?
General cases
•	Acceptable data type: must be of type T
•	What are the classes?
•	In general, how many classes will there be?
•	Special values
•	Zero, negative numbers, very small/large real numbers, control characters, "dummy" values/records
What are the classes?
•	Ranges:
•	0-100
•	'm'-'p'
•	Sets:
•	positive integers
•	'a', 'e', 'i', 'o', 'u'
•	"apple", "orange", other fruit -- give answer with 3 classes [ex: names of fruit, other words, strings not in the dictionary]
Other issues
•	Not all classes are obvious
•	Check the code
•	Anything that causes different behavior can create classes
•	How do you choose from the class?
•	Remember goal: pick "most likely to fail"
Boundary values
•	For deciding which value from a class to use
•	pick values from both sides of the boundary
•	pick values close to the boundary, if not on it
•	boundaries don't always exist (ex: vowels)
•	representation limits can create additional boundaries/classes
•	ex: 2**32 is not within the range of signed integers that can be represented with only 4 bytes, recursive function with certain parameter may cause stack overflow
Sample problem
•	Do this as a group exercise
•	What are classes for adding two numbers, each with one or two digits?
Classes (from Kaner)
•	Largest pair of numbers (99+99)
•	also fits as sum is 3 digits and positive
•	Smallest pair of numbers (-99 + -99)
•	also fits as sum is 3 digits and negative
Classes (from Kaner)
•	Large first number (99 + -14)
•	also fits as Positive + Negative
•	or Large 1st num, sum is 2 digits
•	Negative + positive (-38 + 99)
•	Large 2nd number (56+99)
Classes (from Kaner)
•	Largest pair of single digit numbers (9 + 9)
•	Zeros (0 + 0)
•	0 + nonzero (0 + 23)
•	nonzero + 0 (-78 + 0)
Step 3
•	In example, program does not allow entry of more than 2 digits
•	even for -78
Some error checks (Step 4)
•	100+100
•	boundary test to catch "if x <= 100", etc.
•	<Enter> + <Enter> (null value)
•	A + b (invalid type)
•	1.2 + 5 (invalid type?)
•	<F1> + <Ctrl-C> (control characters)
Some error checks (Step 4)
•	Use your knowledge of data storage, input representation
•	-128, -127, 127, 128 are boundary values (why?)
•	Slight hint: -129, -128 might be better for first pair
•	Also boundary values: /, 0, 9, : (ASCII) -- other possible problems with input?
Summarize (Step 5)
•	Ex: (p. 8)
•	hard to understand how to do things
•	doesn't handle negative numbers
•	doesn't handle 3-character input
•	doesn't check for invalid character
Test cycle 2
•	Review responses
•	there may be reasons why a cases is not handled
•	helps you understand requirements, developer
Test cycle 2
•	Creating new tests based on responses
•	ex: if negative numbers handled, but not 3-character input, -9 + -9
•	still checks addition of negative numbers
•	Special case: "No user would do that"
•	try to think of what a user might do that would cause the same problem
•	ex: instead of 3-digit numbers, what about spaces before/after the number?
•	ex: editing number (backspace)
Test cycle 2
•	Do tests that failed in cycle 1
•	Just because developer says it's fixed does not make it so
•	(May run on developer's system, but not yours)
•	Fixes can create new problems
Later cycles
•	As deadline approaches, not all problems will be addressed
•	keep documentation for testing next release
•	try to present accurate picture of severity of problems