Close

November 8, 2012

Challenges in Performance Testing

Today’s World Wide Web (WWW) application built in advance platforms and technologies such as HTML 5, Web 2.0, CMS, and Cloud etc. It is crucial and challenging to ensure application speed, stability and scalability. Performance Testing is a process to validate system’s non-functional parameters such as response time, throughput, CPU utilization, memory etc. In last blog post, we have seen about Performance Testing Life Cycle (PTLC). Now we will see what all the challenges in performance testing that we face.

Here I am listing out the challenges that I faced in my tenure in Performance Testing projects.

1. Environment or Test Bed

Not all clients have dedicated test environments for performance testing. It is ideal to have replica of production environment. But due to budget constraints, client might not be able to provide. In this scenario, performance testers should use scale-up and scale-down process for effective result analysis and to identify bottlenecks.

2. Performance Testing Tool Selection

I would not say selecting right tool as a challenge, but it is not easy task. There are commercials tools like HP LoadRunner, IBM Rational Performance Tester etc. as well as open source tools like Apache JMeter etc. are available in the market. Selecting right tool for right need saves your cents. Following are the criteria need to be considered before investing in tool(s).

  1. Budget
  2. Technology or Platform
  3. Protocols
  4. Tool training or Trained Resources availability
  5. Learning level or easiness to learn the tool
  6. Tool support / forums
  7. Tool upgrades and patches
  8. Browser / OS Compatibility
  9. Virtual Users (VUsers) License Package
  10. Result generation options
  11. Bottleneck or diagnostics option availability

3. Non-Functional Requirement Elicitation

Ambiguous, unclear, unknown, not-sure kind of requirements from clients is the major challenge in Performance Testing. In this scenario, I would suggest you to start with the minimum load and by consulting with development / architecture team; we can increase the load eventually.

 4. Unavailability of license

Consider your client has 500 VUsers for load testing. Now client wants you to perform stress testing. But for Stress Testing one should load beyond the VUsers which was used for Load Testing. During inception phase itself, this should be considered.

5. Unavailability of data

Unavailability of valid data leads to failures during script execution. During design phase, tester should collect valid data and implemented in the script by parameterization.

  • Incorrect implementation

During workload pattern design, tester should consider the following important parameters like

  1. Types of Performance Testing
  2. Think time
  3. Pacing Time
  4. Iterations
  5. Types of logs
  6. Duration
  7. Mandatory graphs and charts, secondary graphs and charts
  8. Correlation techniques
  9. Mode of recording
  10. Rules sets
  11. Transactions
  12. Actions
  13. Parameterization
  14. Load balancing techniques
  15. Browsers

Any incorrect implementation will affect the testing which yields unrealistic results.

6. Process

It is mandatory that every project should follow process for better result and continuous improvement (beckon Kaizen rule). Any incorrect or non-standard implementation of process or no process implementation in performance testing produces bad results.

7. Performance Monitoring

If the application is in distributed network, then monitoring the performance of each server will be challenging and it is mandatory activity during bottleneck analysis.

Above listed items is purely based on my tenure at various projects. If I missed out anything, please do mention in comments sections.

If you are enjoying my articles, please do subscribe our newsletter for weekly free updates or our feeds.

Image Courtesy: Microsoft Clipart

About NaveenKumar Namachivayam

Hi there! is a passionate and experienced Performance Test Analyst based in Chicago, USA with skill sets in Manual Testing, Test Automation, Performance Testing, and Test Estimation. He loves to learn and experiment new trends and models in Software Testing. Please stop by at his personal blogs: Excel Blog, DealsBrook, NaveenKumarN.in, and Affiliate Insights blog and Software Testing Memes.

Leave a Reply

Your email address will not be published. Required fields are marked *