How to write an effective performance test report?

You might be new to performance testing arena or you are an intermediate level of experience or you might be an expert in performance test scripting, and awesome in designing the workload models and capacity planning. But above points will be null, if you are not showcasing beautiful and intuitive performance test report. In this article, you will learn about how to write an effective performance test report and get applause from your clients. It doesn’t matter which performance testing tool or monitoring tool you are using right now. All we need is metrics and statistics.

Prerequisites

Before you start writing performance test report, make sure that below data points are available with you and you well aware of it.

1. Target audience
2. Project/Release/Build details
3. Test Environment and the version details
4. Type of testing (load/stress/endurance/volume/spike)
5. Workload model
6. Protocols
7. Time zone/timestamp
8. Business flows
9. Test data
10. Inputs from Performance monitoring tools

If you are using HP LoadRunner, by default it generates the report with all the details. But it is not ideal to share it as-it-is with your clients. I prefer to customize the default report so that you can highlight the important statistics and metrics.

Before start writing performance report

Before you start writing performance test report it is important to do homework in spreadsheet program. Most of us use Microsoft Excel program which has the flexibility and robustness. Copy and paste all the metrics, statistics, and raw data; categorize it and then analyze it. E.g. Fund Transfer is the business scenario. Following would be the business transactions:

1. Launch Bank Website
2. Login
3. Check Balance
4. Fund Transfer Submit
5. Check Balance
6. Secure Logoff

There are six critical transactions in the above flow. Assume that target load is 10000 transactions per hour with the SLA of 2 seconds 95 percentile response time.

Apart from above details, assume that you have Mean, Median, Standard Deviation, CPU, Memory and Disk utilization.

During writing performance test report

This is very critical phase where you are going to write like a professional. Every organization will have specific template to record the performance test reports. I am not going to tell to edit those, but you can tailor it for effectiveness. It is ideal to incorporate the following details to the performance test report.

1. Objective of the test
2. Project Overview
3. Business transaction overview
4. Workload model distribution
5. Run-time settings
6. Build/version details
7. Iteration
8. Results from the testing tool
9. Graphs from the MS Excel or testing tool
10. SLA violation details
11. Comparison of response time with production and non-production infrastructure
12. Comparison with last release results
13. Conclusion and Recommendations

In above details, last four data is very important to incorporate in the performance test report.

SLA violation details

In this section, you need to mention which transactions violated the SLA and why. If you see the above example, Fund Transfer Submit transaction, violated the SLA of 2 seconds. In that page, the data will get submitted to the server, where server will validate the transaction by the rules defined in the code. Whether the fund transfer occurring within the bank or external bank or foreign money transfer etc. Before it triggers the fund transfer, it will check whether sufficient balance is present in the account or not. Beneficiary details also will get validated. So, there will be lots of web service calls will fly back and forth. By considering above points, definitely there will be more response time.

You need to analyze the transaction in detail to understand how the request travels between client and server. It is must for any performance test engineer to understand the technical design documents. If you well aware of design, your job is half done.

Comparison of response time with production and non-production infrastructure

Most of the performance test engineers will forget about comparing the statistics with the current non-production infrastructure and production infrastructure. It is not ideal to inject the same load in non-production, if the non-production is scaled-down environment.

Majority of the fortune companies will have identical infrastructure in production and non-production (pre-production) environment. But if the IT budget is not available for small or medium business, they usually will have scaled-down environment. If the production has 64GB RAM and non-production has 6 GB RAM, then definitely there will be difference in the performance metrics. In this case, it is ideal to prorate the target load in non-production environment.

Though the architects insist you to execute the same load as in production, explain them the logic behind it.

Comparison with last release results

It is always best practice to compare the current results with the previous iteration and with the last release results. This will ensure that how much percentage improvement been done in the code tuning. It is ideal to store all the artifacts and its versions in the content or document management portal like SharePoint.

Always have pictorial representation instead of columns and rows of data. Add beautiful graphs/charts, illustrations et cetera to showcase the raw result. Correlate and merge different types of data to plot multi-dimensional graphs which will reveal the breaking point of the system.

Conclusion and Recommendations

Conclusion is the most critical part of any report; if architects are referring previous release results, they directly go to conclusion, instead of going thru all the pages. It is much important to articulate the conclusion and recommendation precisely. It should not contain more data points; instead it should describe how the performance was and what your recommendation is. You can mention the risk (if any) and its impact.

After writing

After completing the performance test report, I would suggest to review at least thrice: your review, peer review, and review by your manager or team lead. Once everyone aligned to the test report, upload it in the project folder or SharePoint location. Maintain all the version history in the document with date and time stamp. Attach all the reference emails, documents, zip files, excel files, screenshots, and other miscellaneous documents.

When it is time, draft a concise email and share it across to your client partners; CC all your team members and project managers.

I hope you would have learnt better way of writing an effective performance test report. I like to hear if I missed any points above. Please do share it in the comments section.

If you like my articles, please show me you love by sharing it in your favorite social media, and do subscribe to free weekly newsletter.