In this blog post we will address the two important question that comes to every performance tester about Improve the Performance Testing Process. Below are the questions that I got via email from one of our reader. He is working as an independent performance tester from past one and half years. I thank him for his patience and enthusiasm.
Before addressing to his queries, I would like to throw few ideas and steps to follow in Performance Testing. Following are the important factors that every performance tester must know.
- Crystal clear non-functional requirements (NFR)
- Knowledge of Current/Past known issues
- Which type of performance testing is required?
- Critical data collection
- Network and Infrastructure knowledge
- Performance Testing strategy
- Result Analysis and recommendation
 1. Crystal clear non-functional requirements (NFR)
Crystal clear non-functional requirements i.e. NFR elicitation is mandatory task for performance testing. It defines the inception phase. This can be Microsoft Word / Excel document which consists of Non-Functional requirements’ detailed/overview of application under test (AUT).
E.g. Application purpose, Number of daily/monthly visitors, risks associated with the project, performance monitoring etc. It could seek the technical data from the clients such as technology of AUT, types of servers deployed, infrastructure model, protocols details etc. If the business analysts are not able to respond, seek the response from the developers. This will help performance testers to plan the performance testing.
2. Knowledge of Current/Past known issues
Once NFR is in our hands, next step is to seeking the knowledge of current and past known issues from the client / developers. This will help performance tester to analyze the application for performance testing.
E.g. any memory lock issues faced in live environment, application being hacked due to lack of security layers, unexpected growth in the daily visitors, more response time which leads to time out error etc. Different issue can be fixed by different type of performance testing.
3. Which type of performance testing is required?
Only Load Testing and Stress testing might not help to fix the issues. From the knowledge of current and past issues, selection of right type of performance testing helps performance testers to analyze and provide recommendations.
Refer my previous blog post on more about Types of Performance Testing.
4. Critical data collection
It is important to collect all critical data like expected response time, throughput, transactions per second etc. This data can be collected from the server logs. Following are the important parameters must collect from clients.
- Expected response time
- Throughput
- Transactions per second
- Concurrent users
- CPU utilization
- Memory Utilization
- Disk Utilization
If no data available for expected response time, you can refer industry standards which has been listed in SPEC. Usually the expected response time of website load page can be 5 seconds.
5. Network and Infrastructure knowledge
It is important to gather about network and infrastructure details from the developers. LAN/WAN details, terminal servers, bandwidth link, load balancing techniques, batch transactions, and disaster recovery details are the important details must to know before you start performance testing process.
6. Performance Testing strategy
Performance testing strategy defines how to approach performance testing and accomplish the mission to identify the performance issues. It defines the project in-scope, out-scope, risk and mitigations, tools license, protocols license, number of engineers to work etc.
7. Result Analysis and recommendation
Performance testers must be capable to reading the graphs between various parameters and provide his inferences and recommendations. Also correlating the graph reveals more details which helps to identify the bottlenecks. Experiences make a man perfect. This proverb fits for performance testing analysis as well.
Now we will address below questions.
Table of Contents
How to improve the current process to make it result oriented?
Executing the identified test scripts repeatedly will not give any inferences. Picking up the right kind of performance test at right time and comparing with the baselines results will give effective results.
E.g. consider a website which has daily visits of 5000 unique visitor, considering the peak load of 500 users from 11AM to 12PM. It is not required to load test the server with 500 users. It will ideal to achieve the throughput alone using Calculating Virtual Users (VUsers) for Load/Stress Testing.
This will save the number of VUsers license procured.
It is recommended to execute at least three runs each script and calculate the average, which gives best value for baseline and comparison.
Creating knowledge artifacts and documenting the experiences that learnt will acts as a reference for other projects and learning center for new joiners
How to get an idea what road map I need to cover to become a good performance tester?
From my experiences, I would like to list out the road map for effective performance testing.
- Learn at least one commercial tool like HP LoadRunner, IBM Rational Performance Tester etc. and open source tools like Apache Jmeter etc.
- Create effective workload models
- Certifications is optional
- Learn to design workload mix models (different combinations of workload model)
- Result analysis and recommendations
Apart from this, please try to follow below tasks
- Sign up in online performance testing websites
- Participate and watch software testing forums
- Join LinkedIn, Google, Yahoo groups etc.
If you are enjoying our articles and blog post, please subscribe for our free newsletter.
Hi Naveen,
Thanks for providing this useful information.
Hi Naveen,
I went through both of these questions answers in details and I still feel few areas are still not covered. Let me explain them here,
As far first question is concerned, the question was how one can improve the performance testing process when no information is provided prior to application testing?
I did some exploration on this and following is the conclusion of my findings,
• First of all testing an application without performance goals and application information is very difficult and can only produce limited results
• If no information is provided before starting the performance testing activity, you try to educate the client and send him a questionnaire to get the information of as many performance parameters as possible.
• Even then if you don’t get the required information, try to explore the application and extract maximum information for performance test and run load and stress tests to give an idea to your client how this application is behaving under expected and unexpected loads
Moreover I believe performance factors you mentioned at top needs to be covered in under performance testing process.
As far second question is concerned, I think answer is right for a performance tester but as you know in small organizations, performance tester role is seen as performance engineer who will not only test the application and will find its bottlenecks but will also help in diagnosing these performance issues. So for being a performance engineer knowledge of follow domains is also very important,
• Architecture Knowledge
• Application Knowledge
• Database Knowledge
• Monitoring capabilities
• Performance Modeling
• Troubleshooting and Tuning