Important Terminologies in Software Testing

Traceability Matrix

Traceability matrix/Requirement Traceability matrix (RTM)Traceability matrix is a matrix which is used to keep track of the requirements. It is a mapping between the requrements and test cases, we will do this for identify missing test cases. It is prepared by trhe either test lead or test engg along with test lead.Exact requirements from the requirement doc given by the client are copied in this matrix. These requirements are assigned a unique number and the remark as testable or not. Against each testable requirement test objective and test case is identified. It is highly possible that for one req there could be multiple test objectives and test cases. For each of the test objective and test case unique number is assigned. Number flow is usually like Requirement Id >> test obj Id>> test case id.Advantages:a. We can trace the missing test cases.b. Whenever requirements changes then we can easily refer to matrix document, change the usecase and go to corresponding testcases and change them. c. Easy to test any functionality. Only we need to refer matrix document and we can reach to related test cases.d. We can trace the impact of functionalities on one another. Because different functionalities can have same test cases.

Bug Density

1. Bug Density: Bug Density is nothing but the number of bugs found in 1000 lines of code. Now every organization has this 1000 set to their requirement and need it can be 100 or any number based on the scale of the project.2. what is defect density?Number of defects divided per unit time. Defect density is one of the metric which is equal to the ratio of number of defects to the number of lines of code .Defect Density = Defect/unit sizeDD=Total Defect/KLOC( Kilo lines of Code)Ex: Suppose 10 bugs are found in 1 KLOCTherefore DD is 10/KLOC (Kilo lines of code)3. what is defect matrix?Time at which defects were discovered relative to when they wereinserted into the software.

System Testing!!

What is System testing? Testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements [IEEE 90]. System testing is actually done to the entire system against the Functional Requirement Specification(s) (FRS) and/or the System Requirement Specification (SRS).Types of system testing:-a. User interface testingb. Usability testingc. Performance testingd. Compatibility testinge. Error handling testingf. Load testingg. Volume testingh. Stress testingi. User help testingj. Security testingk. Scalability testingl. Capacity testingm. Sanity testingn. Smoke testingo. Exploratory testingp. Adhoc testingq. Regression testingr. Reliability testings. Recovery testingt. Installation testingu. Idempotency testingv. Maintenance testing

Severity and Priority

Severity: Severity determines the defect's effect on the application. Severity is given by Testers

Priority: Determines the defect urgency of repair.Priority is given by Test lead or project manager

1. High Severity & Low Priority : For example an application which generates some banking related reports weekly, monthly, quarterly & yearly by doing some calculations. If there is a fault while calculating yearly report. This is a high severity fault but low priority because this fault can be fixed in the next release as a change request.

2. High Severity & High Priority : In the above example if there is a fault while calculating weekly report. This is a high severity and high priority fault because this fault will block the functionality of the application immediately within a week. It should be fixed urgently.

3. Low Severity & High Priority : If there is a spelling mistake or content issue on the homepage of a website which has daily hits of lakhs. In this case, though this fault is not affecting the website or other functionalities but considering the status and popularity of the website in the competitive market it is a high priority fault.

4. Low Severity & Low Priority : If there is a spelling mistake on the pages which has very less hits throughout the month on any website. This fault can be considered as low severity and low priority.Priority is used to organize the work. The field only takes meaning when owner of the bugP1 Fix in next buildP2 Fix as soon as possibleP3 Fix before next releaseP4 Fix it time allowP5 Unlikely to be fixedDefault priority for new defects is set at P3

Bug, Error, Defect and Issue

a. Bug:A software bug is an error, flaw, mistake, failure, or fault in a program that prevents it from behaving as intended (e.g., producing an incorrect result). Most bugs arise from mistakes and errors made by people in either a program's source code or its design, and a few are caused by compilers producing incorrect code.

b. Error: The mistake made by developer in coding.

c. Defect: Defect is something which is in the requirement document and it is not implemented or it is implemented in a wrong way.

d. Issue: Issue is something which is not all above, Some issues are there like site is slow, session related problems, security problems etc.

Waterfall Model

This is the most common and classic of life cycle models, also referred to as a linear-sequential life cycle model. It is very simple to understand and use. In a waterfall model, each phase must be completed in its entirety before the next phase can begin. At the end of each phase, a review takes place to determine if the project is on the right path and whether or not to continue or discard the project.
Requirement

a. Design
b. Implementation & Unit Testing
c. Integration & System Testing
d. Operation
Advantages
a. Simple and easy to use.
b. Easy to manage due to the rigidity of the model – each phase has specific deliverables and a review process.
c. Phases are processed and completed one at a time.
d. Works well for smaller projects where requirements are very well understood.
Disadvantages
a. Adjusting scope during the life cycle can kill a project
b. Poor model for complex and object-oriented projects.
c. Poor model for long and ongoing projects.
d. Poor model where requirements are at a moderate to high risk of changing.

Spiral Model

The spiral model gives more emphases placed on risk analysis. The spiral model has four phases: Planning, Risk Analysis, Engineering and Evaluation. A software project repeatedly passes through these phases in iterations (called Spirals in this model). The baseline spiral, starting in the planning phase, requirements are gathered and risk is assessed. Each subsequent spirals builds on the baseline spiral.
Requirements are gathered during the planning phase. In the risk analysis phase, a process is undertaken to identify risk and alternate solutions. A prototype is produced at the end of the risk analysis phase.
Software is produced in the engineering phase, along with testing at the end of the phase. The evaluation phase allows the customer to evaluate the output of the project to date before the project continues to the next spiral.
In the spiral model, the angular component represents progress, and the radius of the spiral represents cost.
Advantages
a. High amount of risk analysis.
b. Good for large and mission-critical projects.
c. Software is produced early in the software life cycle.
Disadvantages
a. Can be a costly model to use.
b. Risk analysis requires highly specific expertise.
c. Project’s success is highly dependent on the risk analysis phase.
d. Doesn’t work well for smaller projects.

Software Testing

Black Box Testing

Black Box testing refers to the technique of testing a system with no knowledge of the internals of the system. Black Box testers do not have access to the source code and are oblivious of the system architecture. A Black Box tester typically interacts with a system through a user interface by providing inputs and examining outputs without knowing where and how the inputs were operated upon. In Black Box testing, target software is exercised over a range of inputs and the outputs are observed for correctness.

Advantages
a. Efficient Testing — Well suited and efficient for large code segments or units.
b. Unbiased Testing — clearly separates user's perspective from developer's perspective through separation of QA and Development responsibilities.
c. Non intrusive — code access not required.
d. Easy to execute — can be scaled to large number of moderately skilled testers with no knowledge of implementation, programming language, operating systems or networks.
Disadvantages
a. Localized Testing — Limited code path coverage since only a limited number of test inputs are actually tested.
b. Inefficient Test Authoring — without implementation information, exhaustive input coverage would take forever and would require tremendous resources.
c. Blind Coverage — cannot control targeting code segments or paths which may be more error prone than others.

White Box Testing

White Box testing refers to the technique of testing a system with knowledge of the internals of the system. White Box testers have access to the source code and are aware of the system architecture. A White Box tester typically analyzes source code, derives test cases from knowledge about the source code, and finally targets specific code paths to achieve a certain level of code coverage. A White Box tester with access to details about both operations can readily craft efficient test cases that exercise boundary conditions.

Advantages
a. Increased Effectiveness — Crosschecking design decisions and assumptions against source code may outline a robust design, but the implementation may not align with the design intent.
b. Full Code Pathway Capable — all the possible code pathways can be tested including error handling, resource dependencies, and additional internal code logic/flow.
c. Early Defect Identification — Analyzing source code and developing tests based on the implementation details enables testers to find programming errors quickly.
d. Reveal Hidden Code Flaws — access to source code improves understanding and uncovering unintended hidden behavior of program modules.
Disadvantages
a. Difficult To Scale — requires intimate knowledge of target system, testing tools and coding languages, and modeling. It suffers for scalability of skilled and expert testers.
b. Difficult to Maintain — requires specialized tools such as source code analyzers, debuggers, and fault injectors.
c. Cultural Stress — the demarcation between developer and testers starts to blur which may become a cultural stress.
d. Highly Intrusive — requires code modification has been done using interactive debuggers, or by actually changing the source code. This may be adequate for small programs; however, it does not scale well to larger applications. Not useful for networked or distributed systems.

Gray Box Testing

Gray Box testing refers to the technique of testing a system with limited knowledge of the internals of the system. Gray Box testers have access to detailed design documents with information beyond requirement documents. Gray Box tests are generated based on information such as state-based models or architecture diagrams of the target system.

Advantages
a. Offers Combined Benefits — Leverage strengths of both Black Box and White Box testing wherever possible.
b. Non Intrusive — Gray Box does not rely on access to source code or binaries. Instead, based on interface definition, functional specifications, and application architecture.
c. Intelligent Test Authoring — Based on the limited information available, a Gray Box tester can author intelligent test scenarios, especially around data type handling, communication protocols and exception handling.
d. Unbiased Testing — The demarcation between testers and developer is still maintained. The handoff is only around interface definitions and documentation without access to source code or binaries.
Disadvantages
a. Partial Code Coverage — Since the source code or binaries are not available, the ability to traverse code paths is still limited by the tests deduced through available information. The coverage depends on the tester authoring skills.
b. Defect Identification — Inherent to distributed application is the difficulty associated in defect identification. Gray Box testing is still at the mercy of how well systems throw exceptions and how well are these exceptions propagated with a distributed Web Services environment.

Difference between Black Box and White Box Testing

1. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box.
2. Synonyms for white-box include: structural, glass-box and clear-box.
3. Generally black box testing will begin early in the software development i.e. in requirement gathering phase itself. But for white box testing approach one has to wait for the designing has to complete.
4. We can use black testing strategy almost any size either it may be small or large. But white box testing will be effective only for small lines of codes or piece of codes.
5. In white box testing we can not test Performance of the application. But in Black box testing we can do it.