1. Calculations of values of typical quality metrics on the basis of the developed software artifacts 2. Use of metric theory of programs for conducting test experiments 3. Prepare report
Laboratory work N2 2 Subject "Software testing" Theme Testing by "white box" methods" 1 Work objective To make the acquaintance of methods of testing of program logics (by coverage methods), creation of formal description of testing results and providing testing session based on real example of program code. 2 Theoretical fundamentals In general software testing includes: - Formulation of test problem, - Test design, - Preparation of test, - Testing the tests, - Test session performance, - Analysis of testing sessions. Test design plays the main role in this process. There are a lot of approaches to design a test. The first is that a test is being designed based on or external requirements and specifications of a program and modules, or specifications of integration of programs and modules. In this case we consider a program as a "black box". The goal of this approach is to check that a program corresponds to the specifications. But we don't analyze program logics. The second approach is based on the analysis of program logic (strategy of "white box"). The idea is to check each path of each variants of algorithm performance. In this case we don't use external specifications of a program. Any approaches are not optimal. The first approach realization leads us to testing all possible combinations of initial values of input program parameters. But to finish such test fully is not possible, because there is an infinite set of initial values of the parameters. Therefore, we must estimate the maximum benefit from a test with comparison with the expenses to create a test. The benefit can be calculated as a possibility of error occurrence, when we perform a test. Of course if we predict that a program has errors. The expenses can be measured by time of test performance and costs of test preparation. The second approach also does not guarantee that if all possible paths of a program were realized, then a program has not an error. Because the performance of all paths in a test does not guarantee us that a program corresponds to a specification. Let us assume that we write a program to calculate a cube root. But in fact a program calculates a square root. Therefore, a program is wrong in spite of we have tested all possible paths of its performance. Next problem is lost paths. If a program satisfies a specification not fully (for example, it does not include special function to check negative values of input parameters of a program), then via testing we can not find out an error. At last, we have a problem that testing results depend on input values. Because testing path can be passed through on one data but can not be realized for another data. For example, we write the program to define the equality of three numbers via the following expression: IF(A+B+C)/3=A. It is right not for all values of A,B and C (we have the error when one value B or C more then A, but at the same time another value less than A ). Therefore, if we pay attention to testing the paths only, then we can not guarantee that we find a mistake of a program. Thus, full testing of a program is impossible. The strategy of testing is to decrease an incompleteness as far as possible. 2. Methods of 'white box' The principle of 'white box' testing characterizes by a degree of coverage of program logic (program code). 2.1.1 Method of operator coverage The goal is to perform each operator of a program at least one time. Let us discuss the sample on Figures 2.12.2. Samnle Figure 2.1 Figure 2.2 Here we can apply only one test to the program to cover the path ace. If input parameters are: A=2,B=0,X=3, then each operator is being performed once. But this criterion is bad. If in the first condition we replace "and" by "or" and in the second condition we replace " x>1 " by " x1,B=0},{A=2,X>1}. Therefore, it is necessary to perform the number of tests which is enough to realize the situations, where A>1,A1,B=0 and B=0 in the point a and A=2,A=2,X>1 and X1 in the point B. Tests satisfied to the criterion of covering of conditions and corresponded them paths are: a) A=2,B=0,X=4 ace ) A=1,B=1,X=0abd Table 2.3 - The testing results done by method of covering of conditions 2.4 Criteria of covering of decisions (conditions) The criterion of a covering of decisions/conditions requires such amount of tests when all possible results of each condition in the decision are being carried out at least once, all results of each decision are being carried out at least once and, control are being transferred to each input point at least once. There are two tests of this methods (see on Figure 2.1) a) A=2,B=0,X=4 ace b) A=1,B=1,X=0abd which are corresponded to the criterion of covering of decisions. This fact is a consequence of the problem, when one conditions hide another in the decisions. If the condition A>1 is false, the compiler may not check the condition B=0, because any results of the condition B=0, when the decision of ((A>1)&(B=0)), lead us to false value. Therefore, the disadvantage of the criterion is that it is impossible to apply the criterion to get all possible values of all possible conditions in a program. Other variant of realization of the sample is showed on Figure 2.3. The decisions of a program done by a series of conditions are divided into separate solutions and transitions. The most full coverage by tests can be done, when all possible results of each simple decision were being performed. Therefore, it is necessary to cover the paths: HILP (test I: A=2,B=0,X=4), HIMKT (test II A=3,B=1,X=0 ), HJKT (test III A=0,B=0,X=0 ), HJKR (test IV A=0,B=0,X=2 )... Having tested the algorithm on Figure 2.3, we see that the criteria of covering of conditions and decisions are not sensitive to the errors of the logical expressions. 2.5 Method of a combinatory covering of conditions. To solve the problem with error sensitiveness in logical expressions we use the criterion of the combinatory covering of conditions. It requires to create the number of tests when all possible combinations of results of a condition in each decision are being carried out at least once. The set of the tests, satisfying to criterion of a combinatory covering of conditions, satisfies as well to criteria of a covering of decisions, coverings of conditions and a covering of decisions/conditions According to this criterion for the sample we must prepare the following eight combinations: 1) A>1,B=0;2)A>1,B=0;3)A1,B=0;4)A1,B=0;5)A=2,X>1;6)A=2,X1; 7) A=2,X>1;8)A=2,X1. To test all these combinations it is not necessary to use all eight tests. In a fact, they can be covered by four tests: A=2,B=0,X=4{ covers 1,5}; A=2,B=1,X=1{ covers 2,8}; - A=0,5,B=0,X=2{ covers 3,6}; A=1,B=0,X=1{covers4,7}. Table 2.4 - The testing results done by method of a combinatory covering of conditions. 3 Task of laboratory work 1. As a basis for task performance to take the program code realised in laboratory work 1 according to task of the teacher. 2. As the program scheme to take the UML activity diagram and also the control graph, realized during performance of laboratory work 1 . 3. To designate by letters or figures the branches of algorithm on the control graph or on UML activity diagram. 4. To select the testing method, which is the most efficient to detect an error in a program: 4.1. Method of operator coverage; 4.2. Method of covering of decisions; 4.3. Method of covering of conditions; 4.4. Criterion of decisions; 4.5. Method of a combinatory covering of conditions. 5. To choose the paths of algorithm that must be checked by selected testing method. 6. To choose the tests, which allow You to cover fully the selected paths. 7. To test the program created by You. The results must be represented in a form of table (see Tables 2.1-2.4). 8. To prepare report. 4 Content of report 1. Title list. 2. Control graph or UML activity diagram with marked paths of algorithm. 3. Explanation of selection of testing method. 4. The list of paths and tests selected for testing session. 5. Table of testing results. 6. Summary: a) the amount of detected errors in the program; b) time of testing; c) which test allowed You to detect an error
