Regression Testing...

Regression Testing Guidelines

1. REGRESSION TESTING – PROCESS 3
1.1 PERFORM IMPACT ANALYSIS 3
1.2 ESTABLISH TEST OBJECTIVES 3
1.3 CREATE REGRESSION TEST PLAN 3
1.4 IDENTIFY/CREATE REGRESSION TEST CASES 3
1.5 ENSURE REGRESSION TESTING ENVIRONMENT 4
1.6 EXECUTE REGRESSION TESTING 4
1.7 REVIEW REGRESSION TEST RESULTS 4
1.8 PREPARE TEST REPORT 4
1.9 EXIT CRITERIA 4
1.10 MEASUREMENTS 5
2. TOOLS FOR REGRESSION TESTING 5
3. METRICS FOR REGRESSION TESTING 5


1. Regression Testing – Process
A. Perform Impact Analysis
B. Establish Test Objectives
C. Create Regression Test Plan
D. Identify/Create Regression Test Cases
E. Ensure Regression Testing Environment
F. Execute Regression Testing
G. Review Regression Test Results
H. Prepare Test Report
I. Exit Criteria
J. Measurements

1.1 Perform Impact Analysis

The Team shall understand new requirements and analyze the impact of the added functionality to the existing features. Based on the analysis the impacted functionalities and new functionalities shall be identified for testing.

1.2 Establish Test Objectives

The Team shall Study and review each new requirement and Identify “key" System functions that need to be tested. Test objectives for each function shall be specified, Automation scope and tool shall be identified.

1.3 Create Regression Test Plan

The TL shall prepare the Regression Test Plan based on the new functionality as a result of the enhancements and the impacted areas. Scope shall be specified (the features shall be tested), Test Criteria gives the Entry and Exit criteria shall be defined. The resources requirements shall be identified. Responsibilities to the resources shall be assigned. Metrics to be captured shall be identified.

1.4 Identify/Create Regression Test Cases

The Test Designer shall identify a representative sample / subset of test cases to test impacted functionalities and prepare test cases for new functional scenarios.

1.5 Ensure Regression Testing Environment

The TL shall ensure the required test environment for Regression testing as identified in RTP. It is recommended that this setup is separate from the development environment. The smoke test shall be executed to ascertain if the build is stable and it can be considered for further testing.

1.6 Execute Regression Testing

Regression Test Cases shall be executed by the Tester manually and/or automated based on Project Requirements. All Regression Test runs shall be recorded in the Regression Test Cases/Test Report. If the observed behavior matches with the expected behavior, then “P” (Pass) shall be entered in the observed behavior column; otherwise “F” (Fail) shall be entered.

For every run, observed behavior shall be recorded. Test runs shall be repeated until all the defects are closed. All defects shall be recorded by the concerned tester. Defect Reporting and Closing shall be done using Defect tracking Tool.

1.7 Review Regression Test Results

The Regression Test results shall be reviewed by the TL. The TL may assign the task of review within the team. Reviewer may take some sample test cases and conduct testing to verify the correctness. The reviewer shall verify the following:

• All the relevant test cases are executed
• The choice of test cases have covered the main scenarios
• The testing is as per the corresponding test plan
• Identified defects are captured and analyzed

1.8 Prepare Test Report

The TL shall prepare the Test Summary Report after evaluation. Metrics specified in the test plan shall be prepared and analyzed.
Test Summary Report shall cover Test Execution and Defect Data summary. Based on the completion criteria, testing shall be signed off by the Test Manager.

1.9 Exit Criteria

The regression testing is completed when all the criteria mentioned in the agreement is met.

The basis on which regression testing can considered to be finished are
• When all the testing cycles are covered
• When the enhancements/changes are found to be met

A proper Signoff document should be prepared.

1.10 Measurements

Use the following metrics of Regression Testing to know effectiveness of this procedure:

• Defect Slippage
• Defect removal Efficiency
• Effort Estimation Variance
• Schedule Estimation Variance
• Productivity
• Defect Detection Rate

2. Tools for Regression Testing

The following tools are recommended for automated Regression.
• Mercury WinRunner
• Rational Robot
• Mercury Quick Test Professional
• Silk Test

3. Metrics for Regression Testing

Capture the following metrics for Regression Testing.
• Phase wise Defect removal efficiency and Total defect removal efficiency
• Failure rate/ Test accuracy/ Defect density
• Completeness/ Test coverage/ Defect indices
• Run reliability/ Error distribution/ Fault-days number
• Requirements traceability/ Cause and effect graphing
• No. of conflicting requirements/ No. of entries and exits/ module
• Review Effectiveness/ Delivered Defect Density
• Testing Effectiveness/ Testing Efficiency Test Case Design Effectiveness
• Phase wise Defect Density and Cumulative defect density
• Cost of Poor Quality/ Cost of Quality