Testing and the Software Process, cont'd
•	Reading: Kaner et al., ch. 3
•	More on design testing
•	Incremental testing
•	Other types of testing
•	Maintenance
Weaknesses of prototyping
•	What kinds or problems are not revealed by prototypes?
•	If design does not handle all requirements (esp. error handling)
•	If standards are satisfied
•	If software performs poorly
Weaknesses of prototyping
•	If design is done poorly
•	duplicate elements
•	tight coupling between components (many dependencies)
•	poor cohesion within a component (parts do not belong together)
Review meetings
•	Held to find problems
•	Often best to have nonparticipating (neutral) facilitator
•	Advance preparation to make meeting productive
•	ex: Checklist for each area of concern (error handling, standards, etc.)
•	ex: Reviewers examine design in advance and bring issues to meeting
Review meetings
•	Not held to fix problems
•	Example of static checking -- code is not executed
•	(Dynamic checking tests code execution)
Testing and implementation
•	Glassbox (also known as Whitebox or Clearbox) testing
•	Testing based on actual code
•	Statement coverage ("line coverage" in Kaner), decision coverage ("branch coverage in Kaner), condition coverage are examples
Incremental testing
•	Testing larger and larger parts of the software
•	Unit (module or element) testing for individual parts
•	Integration testing tests combinations of units, then combinations of combinations, and so on
Incremental testing
•	System tests for entire software
•	Testing is done on development side
•	Acceptance tests for entire software
•	Testing for approval by customer
Incremental testing
•	Testers usually start working with System and Acceptance tests
•	Acceptance tests can be designed from customer requirements
•	System tests can be designed after developer requirements are formed
•	where standards, etc. are usually decided
Incremental testing
•	Even unit tests can be partly designed before coding
•	Helps avoid wasting time writing code that will need to be rewritten
•	JUnit is an example of helping developer do this
Unit testing
•	Unit testing does require writing driver to execute
•	Creates any data needed for test
•	Calls functions to test
•	May produce output for evaluating results or compare results with expected results
Testing units with dependencies
•	Unit may depend on unfinished unit -- what can be done?
•	Write a stub to do that unit's job
•	Goal : when that unit is finished, can replace stub with that unit and code should work
Testing units with dependencies
•	Ex: Testing unit which includes:
•	int result = myComputation( ) * OtherUnit.compute( );
•	define class for OtherUnit, including
•	int compute() { return 5; }  // assuming 5 is a valid value to return
Top-down vs. Bottom-up
•	Top-down tests "highest level" first
•	Stubs replace lower level modules
•	Can find system-wide design problems easier
•	Bottom-up tests "lowest level" first
•	Only test higher level when all lower-level modules are done
•	Tests actual code :)
Other types of tests
•	Compare product against documentation: specification, manual/help
•	Performance tests
•	Security -- unauthorized access, potential damage by user, viruses in deliverable?
•	Compatibility with other products
Other types of tests
•	Configurations (what does the customer use?)
•	Load tests
•	how does the software handle reaching resource limits?
•	ex: Size of data, number of users, memory/disk usage
•	Want graceful degradation of performance
Regression testing
•	Repeating tests that have already passed
•	When would this be necessary?
•	Rate of success in actually fixing a bug without creating more problems is quite low
•	one of the reasons we want low coupling
•	Maintenance phase uses a lot of regression tests
Port testing
•	Porting your software: building it to run on a different system
•	Many of the "other types of tests" may be affected
•	Even compilation is frequently a problem
•	Ex: new system uses different word-size
•	look through some Linux source code some day to see how many "#ifdef"s there are