Today’s World Wide Web (WWW) application built in advance platforms and technologies such as HTML 5, Web 2.0, CMS, and Cloud etc. It is crucial and challenging to ensure application speed, stability and scalability. Performance Testing is a process to validate system’s non-functional parameters such as response time, throughput, CPU utilization, memory etc. In last blog post, we have seen about Performance Testing Life Cycle (PTLC). Now we will see what all the challenges in performance testing that we face.
Here I am listing out the challenges that I faced in my tenure in Performance Testing projects.
1. Environment or Test Bed
Not all clients have dedicated test environments for performance testing. It is ideal to have replica of production environment. But due to budget constraints, client might not be able to provide. In this scenario, performance testers should use scale-up and scale-down process for effective result analysis and to identify bottlenecks.
2. Performance Testing Tool Selection
I would not say selecting right tool as a challenge, but it is not easy task. There are commercials tools like HP LoadRunner, IBM Rational Performance Tester etc. as well as open source tools like Apache JMeter etc. are available in the market. Selecting right tool for right need saves your cents. Following are the criteria need to be considered before investing in tool(s).
- Budget
- Technology or Platform
- Protocols
- Tool training or Trained Resources availability
- Learning level or easiness to learn the tool
- Tool support / forums
- Tool upgrades and patches
- Browser / OS Compatibility
- Virtual Users (VUsers) License Package
- Result generation options
- Bottleneck or diagnostics option availability
3. Non-Functional Requirement Elicitation
Ambiguous, unclear, unknown, not-sure kind of requirements from clients is the major challenge in Performance Testing. In this scenario, I would suggest you to start with the minimum load and by consulting with development / architecture team; we can increase the load eventually.
4. Unavailability of license
Consider your client has 500 VUsers for load testing. Now client wants you to perform stress testing. But for Stress Testing one should load beyond the VUsers which was used for Load Testing. During inception phase itself, this should be considered.
5. Unavailability of data
Unavailability of valid data leads to failures during script execution. During design phase, tester should collect valid data and implemented in the script by parameterization.
- Incorrect implementation
During workload pattern design, tester should consider the following important parameters like
- Types of Performance Testing
- Think time
- Pacing Time
- Iterations
- Types of logs
- Duration
- Mandatory graphs and charts, secondary graphs and charts
- Correlation techniques
- Mode of recording
- Rules sets
- Transactions
- Actions
- Parameterization
- Load balancing techniques
- Browsers
Any incorrect implementation will affect the testing which yields unrealistic results.
6. Process
It is mandatory that every project should follow process for better result and continuous improvement (beckon Kaizen rule). Any incorrect or non-standard implementation of process or no process implementation in performance testing produces bad results.
7. Performance Monitoring
If the application is in distributed network, then monitoring the performance of each server will be challenging and it is mandatory activity during bottleneck analysis.
Above listed items is purely based on my tenure at various projects. If I missed out anything, please do mention in comments sections.
If you are enjoying my articles, please do subscribe our newsletter for weekly free updates or our feeds.
Image Courtesy: Microsoft Clipart