Answers may be incorrect...

Notes are at the bottom

Concepts Failure any deviation of the observed behavior from the specified behavior Erroneous State the system is in a state such that further processing of the system will result in a failure. Fault the mechanical or algorithmic cause of the erroneous state. AKA defect, bug Fault leads to erroneous state which leads to failure Test component part of the system that can be isolated for testing Testing Testing is the systematic attempt to find faults/bugs in a planned way in the implemented software Not the process of demonstrating that faults aren't present A successful tests shows us a failure There is no way to demonstrate that faults are not present through testing unless we can test every possible input Similar to falsification of scientific theories Design experiments where the goal is to falsify the theory if the theory is not falsified, you now have more confidence in the theory Our goal in testing is to find the errors so we can correct them If we are unable to find any, we are more confident in our software Central Limitation of Testing "Program testing can be used to show the presence of bugs, but never to show their absence!" Edsger W. Dijkstra (1972) So we will discuss formal reasoning also, later. Test Case A Test Case contains: A set of inputs Includes the state of the object! Expected results for those input We use them to detect the existence of faults by causing failures A failure is a deviation from the expected behavior, so we need to know the expected results Unit Testing: Dealing with Scale Best practice is to test individual units or components of software (one class, one method at a time) This is known as unit testing Testing what happens when multiple components are put together into a larger system is known as integration testing Testing a whole end-user system is known as system testing Unit Tests Just focus on one small part of the system at a time Reduces complexity of the tests Focuses the test Easier to pinpoint faults Independence of testing components Unit Test Techniques Equivalence testing Boundary testing Path Testing we will discuss Test Stub and Test Driver Test Stub Partial implementation of components on which the tested components depend Replaces lower level of code Test Driver Partial implementation of component that depends on the test component Simulates the part of the system that uses the test component Call's the test component and then checks the input Replaces the user interface and higher levels of code Can be used to isolate possible locations of the fault Test Stub and Test Driver Test Stub Partial implementation of components on which the tested components depend Replaces lower level of code Test Driver Partial implementation of component that depends on the test component Simulates the part of the system that uses the test component Call's the test component and then checks the input Replaces the user interface and higher levels of code Can be used to isolate possible locations of the fault

Designing a Test Plan A Test Plan is the set of test cases for a specific unit. To make testing most likely to succeed in revealing defects, best practices include: Test boundary cases: "smallest", "largest", "special" values based on the contract Test routine cases Test challenging cases, i.e., ones that, if you were writing the code (maybe you didn't write the code being tested!), you might find difficult or error-prone We want distinct test cases Is there a scenario where you would expect one test case to pass and one to fail? Our Test cases should follow design by contract We don't use inputs that go against the preconditions. Our post conditions help us identify our expected output Also don't give test cases that compilers would prevent Example Method Contract /** * Returns some factor of a number. * * @precondition * n > 0 @postcondition * aFactor > 0 and * n mod aFactor = 0 */ private static int aFactor(int n) {...) Psychology of Testing Design and coding are creative activities Testing is a destructive activity The primary goal is to "break" the software, i.e., to show that it has defects Very often the same person does both coding and testing (not a best practice) You need a "split personality": when you start testing, become paranoid and malicious It's surprisingly hard to do: people don't like finding out that they made mistakes Blackbox vs Whitebox testing Blackbox Focus on the input and output behavior Ignore internal details or structure of the component Use the contracts to create our test plans Whitebox Focuses on internal structure of the component. Every state of the object and interactions are also tested Every possible path in the code can be tested 100% code coverage Look at the code while developing test cases Blackbox Testing If we are doing blackbox testing, we don't use implementation details We just rely on the interface to test our code Pros One test plan can work for any implementation of the same Interface If implementation changes, or a new implementation is developed, we don't have to change our test plan Cons We can't guarantee 100% code coverage We don't look at the code, so we don't not if we are hitting every path Techniques for developing good test cases are more conceptually difficult Whitebox testing We look at the implementation details Pros We can guarantee 100% code coverage Conceptually easier to develop Cons We can miss errors of omission Can't develop before the code is written When the code changes, we may need to change our test plan New implementations of the same interface require new test plans Tracing and Debugging Objective locate errors using test inputs Approach Analyze the code on sample test inputs to understand why code fails to behave as specified in its contract Tracing and Debugging Tracing: Analyze, but do not execute code Debugging: Execute code on selected inputs and follow the execution Test Driven Development (TDD) A common practice in Software Engineering Create the automated test cases before writing the code based on the contracts Then write the code needed to pass your test cases Pros You are ready to test your code immediately after writing it Reducing the time between writing the code and discovering the existence of a fault makes it easier to locate the fault Developing test cases first forces you to consider challenging inputs and how to handle them Cons Only works with blackbox testing Coding to the test plan relies on having a great test plan

Why does testing matter? Engineering comes with responsibility You're responsible for the product you produce In Software Engineering, you're responsible for the software you produce You want it to be high quality software Bad software can lead to Angry customers lost money and time Even loss of life Let's consider some historical examples Software Engineering Mistakes Mariner 1 Supposed to do a flyby of Venus Cost 18.5 million in 1962 ($146,476,299 today) NASA lost control of the craft, and had to destroy it 294.5 seconds after launch. Apparent Cause? Incorrectly transcribed a hand written equation into the code. Software Engineering Mistakes Lt. Col. Stanislav Petrov 1982 Soviet Officer monitoring the missile detection system Alarms went off indicating incoming missiles from America NATO involved in provocative exercises Russians had recently shot down a South Korean airliner that went into its air space Petrov thought it was odd there were only 5 missiles, and told his superiors it was a false alarm, instead of telling them to fire the retaliatory missiles False alarm caused by the reflection of sunlight off the top of the clouds Software Engineering Mistakes Ariane 5 rocket First test launch ended in automatically triggered self destruct Velocity was computed as a 64-bit float Tried to store velocity in a 16-bit memory register The number was too large for the register, creating an overload error. Overload error caused a drastic and unnecessary flight correction Flight correction caused damage to booster rockets Damage to the booster rockets resulted in self destruct sequence being activated. Software Engineering Mistakes Mars Climate Orbiter Two teams worked on the software for the orbiter Lockheed Martin's team used English Units NASA's JPL team used Metric units When the orbiter tried to calculate the thrust to enter the orbit of Mars the unit difference caused a miscalculation The orbiter crashed into Mars The orbiter cost $125 million dollars

Question 1 1 pts _has occurred when the observed output did not match the expected output failure Question 2 1 pts An is when any further processing will result in the observed output not matching the expected output erroneous state Question 3 is the algorithmic cause of of the observed output not matching the expected output. fault D Question 4 pts In order to have a complete test case, you need to have .. A set of specific input values A set of specific output values A testdriver A test stub The observed output Testing is used to prove that your code has no faults. D Question 5 True False leads to a which leads to a 1 pts D Question 8 erroneous state, failure, fault erroneous state, fault, failure Which type of testing allows you to look at the code while developing test cases? failure, fault, erroneous state Greybox Testins o fault, erroneous state, failure D Greenbox Testing fault, erroneous state, failure Redbox testing (only used to test movie rental vending machines) Whitebox testing Blackbox testing 1 pts D Question 6 A successful test case shows us... Question 9 a faillite What type of testing can you use while following Test Driven Development that a fault exists Dl the location of the fault Whitebox Testing that the code has no faults Greybox testing the identity of the hacker who infiltrated the system Greenbox Testing Blackbox Testing 1 pts Question 10 The best practice of breaking our system into smaller components to test individually is known as.... Integration Testing Blackbox Testing Whitebox Testing Unit Testing System Testing