Limitations and Purpose of testing
•	Reading: Kaner, ch. 2
•	Blackbox, Whitebox (Glassbox) testing
•	Coverage
•	Limitations
Purpose of terminology
•	Terminology can help communication
•	Between different parts of organization
•	Between manufacturer and consumer
•	Between applicant and interviewer
•	Terminology doesn't make you a better tester
Blackbox testing
•	Blackbox testing: Emphasis is on functionality of code
•	results visible to user
•	What can the user see?
•	Blackbox testing usually covers most common paths through code
•	In any single execution of a program, many statements will not be executed
•	As a tester, why might this be a concern?
Whitebox testing
•	Whitebox (Glassbox) testing: tests are based on structure of code
•	Assumption: failures related to control flow
•	Trying to test program units: single functions, classes
•	Try to ensure existing code is tested
•	Cannot detect if needed code is missing
•	Ex: Are all statements executed in at least one test case?
Some comments
•	Blackbox and whitebox testing are not mutually exclusive
•	Graybox testing : mixes the two
•	Whitebox testing is often automated
•	Blackbox tests usually require more design
•	but can be "recorded"
Concept of coverage
•	Coverage: for testing, ensuring that tests cover all possibilities for some criteria
•	Statement coverage: cover all statements
•	Coverage is applicable to many areas of testing
•	Ex: GUI -- was every button pressed in at least one test?
Limitations
•	Does statement coverage handle all cases?
•	Path coverage : cover all possible paths through the code
•	How many paths for 1 if-else statement? 2 in a row? 3? n?
•	Path coverage is not feasible for most large programs
•	Even statement coverage is often not practical
Limitations
•	Path coverage cannot catch case where a different path should exist
•	Even for the same value, how you enter the value can matter (ex: backspace)
•	Path coverage does not account for timing
User interface issues
•	Many issues other than correctness
•	Include ergonomic, psychological issues
•	Affected by user environment
•	Individual user preferences
•	General GUI design principle is to allow user to customize interface
Other sobering data
•	Beizer: Tester's get code with 1-3 bugs per 100 executable statements
•	These are public bugs
•	Private bugs : found by programmer
•	Consider also frequency of updates for Windows
•	Most security updates fix the same bug
•	"Beta-release" software
Open source projects
•	Raymond, The Cathedral and the Bazaar
•	"Release early, release often"
•	"Most bugs are shallow"
•	Implication 1: do not expect to release perfect software
•	Implication 2: most bugs are easy to find (why didn't the developer find them?)
How this affects testing
•	Developer may see testing as proving the code works
•	Successful testing should find errors
•	Tester should assume code does not work
•	You find more errors when you expect them
Quality
•	Kaner describes how testers can do their best under current circumstances
•	What (if anything) do you think could be done to change the circumstances?
•	Do you think your idea is feasible?