TIPS

A Simple 12 Steps Guide to Write an Effective Test Summary Report [with a Sample Report for Download]

Posted In | Software Testing Templates, Testing Tips and resources

Several documents and reports are being prepared as part of Testing. Some are Test Strategy doc, Test Plan doc, Risk management Plan, Configuration management plan etc. Among these Test Summary Report is one such report which is prepared after the Testing is completed.

I have tried to explain the purpose of ‘Test Summary Report’ and provided a sample Test Summary Report template along with an actual report for download.

What is a Test Summary Report?

As we know, Software Testing is an important phase in SDLC and also it serves as the “Quality Gate” for the application to pass through and certified as “Can Go Live” by the Testing Team.

Test Summary Report is an important deliverable which is prepared at the end of a Testing project, or rather after Testing is completed. The prime objective of this document is to explain various details and activities about the Testing performed for the Project, to the respective stakeholders like Senior Management, Client etc.

As part of Daily status reports, daily testing results will be shared with involved stakeholders every day. But Test Summary Report provides a consolidated report on the Testing performed so far for the project.

Recommended reading => How to Report Test Execution Smartly (Status Report Template download)

Assume that if the Client who sits in a remote location need to understand the results and status about a Testing project which was performed for a period of, say for example – four months, Test Summary Report will solve the purpose.

This is also an artifact required to be prepared as part of CMMI process.

What Test Summary Report contains?

A typical Test Report template will contain the below information, however based on each Company’s format & practice, the contents may vary. I have also provided real examples for better understanding.

At the end of this article you can download a test summary report sample.

12 Steps Guide to writing an effective test summary report:

Step #1: Purpose of the document

Example: This document explains the various activities performed as part of Testing of ‘ABCD transport system’ application.

Step #2: Application Overview

Example:‘ABCD transport system’ is a web based Bus ticket booking application. Tickets for various buses can be booked using the online facilities. Real time passenger information is received from a ‘Central repository system’, which will be referred before booking is confirmed. There are several modules like Registration, Booking, Payment and Reports which are integrated to fulfill the purpose.

Step #3: Testing Scope

1. In Scope
2. Out of Scope
3. Items not tested

Example: A functionality verification which needs connectivity to a third party application cannot be tested, as the connectivity could not be established due to some technical limitations. This section should be clearly documented, else it will be assumed that Testing covered all areas of the application.

a) In Scope
Functional Testing for the following modules are in Scope of Testing

• Registration
• Booking
• Payment

b) Out of Scope
Performance Testing was not done for this application.

c) Items not tested
Verification of connectivity with the third party system ‘Central repository system’ was not tested, as the connectivity could not be established due to some technical limitations. This can be verified during UAT (User Acceptance Testing) where the connectivity is available or can be established.

Step #4: Metrics

a) No. of test cases planned vs executed
b) No. of test cases passed/failed

c) No of defects identified and their Status & Severity

d) Defects distribution – module wise

Step #5: Types of testing performed

1. Smoke Testing
2. System Integration Testing
3. and Regression Testing

Note: If several rounds of Testing were done, the details can also be included here.>

Example: 
a) Smoke Testing
This testing was done whenever a Build is received (deployed into Test environment) for Testing to make sure the major functionality are working fine, Build can be accepted and Testing can start.

b) System Integration Testing

• This is the Testing performed on the Application under test, to verify the entire application works as per the requirements.
• Critical Business scenarios were tested to make sure important functionality in the application works as intended without any errors.

c) Regression Testing

------------

• Regression testing was performed each time a new build is deployed for testing which contains defect fixes and new enhancements, if any.
• Regression Testing is being done on the entire application and not just the new functionality and Defect fixes.
• This testing ensures that existing functionality works fine after defect fix and new enhancements are added to the existing application.
• Test cases for new functionality are added to the existing test cases and executed.

Step #6: Test Environment & Tools

Example:

Step #7: Lessons Learned

Example:

Step #8: Recommendations

Example:

• Admin control for defect management tool can be given to Offshore Test manager for providing access to Testing team.
• Each time the onsite Admin need not be contacted for requests whenever they arise, thereby saving time due to the geographical time zone difference.

Step #9: Best Practices

Example:

• A repetitive task done manually every time was time consuming. This task was automated by creating scripts and run each time, which saved time and resources.
• Smoke test cases were automated and the scripts were run, which ran fast and saved time.
• Automation scripts were prepared to create new customers, where lot of records need to be created for Testing.
• Business critical scenarios are separately tested on the entire application which are vital to certify they works fine.

Step #10: Exit Criteria

(i) All planned test cases are executed;
(iI) All Critical defects are Closed etc.>
Example: 
a) All test cases should be executed – Yes
b) All defects in Critical, Major, Medium severity should be verified and closed – Yes.
c) Any open defects in Trivial severity – Action plan prepared with expected dates of closure.

No Severity1 defects should be ‘OPEN’; Only 2 Severity2 defects should be ‘OPEN’; Only 4 Severity3 defects should be ‘OPEN’. Note: This may vary from project to project. Plan of Action for the Open defects should be clearly mentioned with details on when & how they will be addressed and closed.>

Step #11: Conclusion/Sign Off

Example: As the Exit criteria was met and satisfied as mentioned in Section 10, this application is suggested to ‘Go Live’ by the Testing team. Appropriate User/Business acceptance testing should be performed before ‘Go Live’.

Step #12: Definitions, Acronyms, and Abbreviations

=> Download Sample Test Summary Report:
Click here to download a sample test report template with an example.

Few points to note while preparing the Test Summary Report:

• As part of Test Execution, collect all required information on the Testing performed. This will help to prepare a sound Test summary report.
• Lessons learned can be explained in detail, which will convey the Responsibility which was taken to solve these issues. Also this will be a reference for upcoming projects to avoid these.
• Similarly, mentioning the Best Practices will portray the efforts taken by the team apart from regular testing, which will also be treated as a “Value Addition”.
• Mentioning the Metrics in graphics form (Charts, Graphs) will be a good way to visually represent the status & data.
• Remember, Test summary report shall mention and explain the activities performed as part of the Testing, to the recipients to understand better.
• Few more appropriate sections can be added if required.

Conclusion:

Test summary report is an important deliverable and focus should be to prepare an effective document, as this artifact will be shared with various stake holders like senior management, client etc.

After performing an exhaustive testing, publishing the test results, metrics, best practices, lessons learnt, conclusions on ‘Go Live’ etc. are extremely important to produce that as an evidence for the Testing performed and the Testing conclusion.

We have also made available the test report sample for download. It is a perfect example of how to prepare an effective test summary report!

About the author: This is a guest post by Baskar Pillai. He is having around 14 years of experience in Test management and end to end software testing. CSTE certified Testing professional, trainer, worked in IT majors like Cognizant, HCL, Capgemini and currently working as Test Manager for a large MNC.