SOFTWARE METRICS ================ Reference: Software Metrics: A Rigorous & Practical Approach by Fenton, Pfleeger Software Engineering: A Practitioner's Approach by Pressman. The Mythical Man-Month by Brooks Measurements/Metrics Goals: UNDERSTAND what is happening during development and maintenance. CONTROL what is happening to our projects. IMPROVE our processes and products. Metrics Scope: 1. Cost and Effort Estimation: COCOMO: E = a*S^b where E = effort in person months a,b = parameters based on the type of software S = size measured in KLOC (thousand lines of source code) Mythical Man-Month: Are men (and women) interchangeable with months? Valid for perfectly partitionable task (picking cotton). L = LOC = 200,000 p = productivity (lines/m-m) = 1000 n = number of men (persons) = 10 E = effort (m-m) D = duration/development time (months) E = L/p = 200m-m D = E/n = 20m Some tasks cannot be partitioned (bearing of children): E = constant = 9 w-m and n = 1 D = E/1 = 9m Some tasks partitioned after initial training/start-up: D = E/n + c Partitioning creates intercommunication overhead: D = E/n + c * n(n-1)/2 Overhead reduced by formation of t teams. Communication (reviews) improves quality and productivity. Scheduling: 12 man-month task = 3 people x 4 months milestones at end of months 1, 2, 3, 4 problem: first milestones not reached for 2 months solution 1: trim the task (happens anyway) solution 2: reschedule and allow plenty of time solution 3: complete task ON-TIME by adding people assume only first milestone was wrong need 4.5 people to do 9 m-m in 2 remaining months add 2 people new problem 1: training/learning curve new problem 2: repartition into 5 parts result : just as late solution: add more people? 2. Productivity Measures and Models Productivity is a function of Value and Cost: For example: Productivity | +---------------+---------------+ | | v v Value Cost | | +------+------+ +-------+-------+ | | | | Quality Quantity Personnel Resources | | | | +--+--+ +-----+------+ +--+--+ +----+----+ | | | | | | | | | | v v v v v v v v Functionality Size Time Money Hardware Software Defects Reliability Another example: KLOC = thousands of lines of code Productivity = KLOC/man-month Quality = defects/KLOC Cost = $/LOC But the cost has some problems: language dependent person dependent is $120,000/yr programming twice as fast as $60,000/yr? or are they more or less than twice? Another example: Length: NCLOC = non-comment lines of code CLOC = comment lines of code Density of Comments = CLOC/LOC Predicting Length: Expansion Ratio = Size of Design / Size of Code This ratio can be recorded from experience with different types of applications. 3. Data Collection The quality of any metric is dependent on careful data collection. Metrics must be defined unambiguously with consistent and complete collection. Data must then be shown cleary, with graphs and charts, to management. Resources # Faults ^ x Actual | |x| | x | |x| |x| | x o Planned | |x| |x| |x| | x o | |x| |x| |x| | x o | |x| |x| | | |x| CLOSED | x o | |x| |x| | | |x| | xo | | | |x| | | | | OPEN | xo | | | | | | | | | +---------------------> +--------------------------- Time Require Design Code Test 4. Quality Models and Measures For example: Quality Factor Criteria Metric -------------- -------- ------ Product operation: +------> COMMUNICATIVENESS ----> a | USABILITY --+ | +------> ACCESSIBILITY --------> b +----> ACCURACY -------------> c | RELIABILITY --+ | +----> COMPLETENESS ---------> d EFFICIENCY --> Product Revision: REUSEABILITY MAINTAINABILITY PORTABILITY TESTABILITY Note that a criteria could be used by more that one factor. Example: Both RELIABILITY and REUSABILITY depend on ACCURACY. 5. Reliability Models defects/KLOC may not really affect user, instead: MTTF = Mean Time To Failure MTTR = Mean Time To Repair MTBF = Mean Time Between Failures (which may be very long) MTBF = MTTF + MTTR Metric: AVAILABILITY = (MTTF / MTBF) * 100% 6. Performance Evaluation and Models BIG-OH: Bubble Sort: O(n*n) Quicksort : O(nlogn) 7. Structural and Complexity Measures CYCLOMETRIC MEASUREMENT = Predicates + 1 Determines the number of independent paths through the code and measures control flow structure. A high metric indicates the likelihood of: more faults more difficult testing more difficult maintenance In classical (structured) code, we might have a control hiearchy of modules: A | +-------+-------+ | | v v B C +-----+-----+ +-----+-----+ | | | | v | v v D +-> E F Fan-In(X) : number of modules that call X Fan-Out(X): number of modules called by X Example: Fan-In(B) = 1 Fan_Out(B)= 2 Fan_In(E) = 2 INFORMATION FLOW COMPLEXITY(X) = Size(X) + (Fan-In(X) * Fan-Out(X))^2 In object-oriented code, we have other metrics: Depth of Inheritance Tree (DIT) Number of Children (NOC) Coupling Between Object classes (CBO): how many other classes does it "use"? 8. Capability-Maturity Assessment Example: IEEE Software Maturity Index (SMI) (indirect measure of quality) M_T = number of modules in current release F_c = number of modules changed F_a = number of modules added (new) F_d = number of modules deleted SMI = (M_T - (F_a + F_c + F_d)) / M_T As SMI -> 1.0 this implies stability Other Concepts: FAILURE REPORT: Location : where did the problem occur? [installation where failure was observed] Timing : when did it occur? [clock time or some temporal measure] Symptom : what was observed? [type of error message or indication of failure] End Result: which consequences results? [OS crash, loss of data, wrong output, no output] Mechanism : how did it occur? [chain of events including keyboard commands] Cause : why did it occur? [reference to possible fault(s) leading to failure] Severity : how much was the user affected? [critical, major, minor] Cost : how much did it cost? [cost to fix plus lost of potential business]