In this blog post we are going to see about Right Tail Outlier Detector in JMeter. Analyzing results is one of the critical phase in any performance testing project. Most of the results you encounter might have outliers. In commercial tools such as NeoLoad has a built-in feature to remove the noise. In JMeter, you can isolate the outliers using the Right Tail Outlier Detector plugin.
Table of Contents
Installing Right Tail Outlier Detector in JMeter
Prerequisites
- Stable version of JMeter
- Plugins Manager
In this demonstration, I am using JMeter 5.4.1 and Plugins Manager 1.6. Download the Plugins Manager jar from here and place it into JMETER_HOME\lib\ext folder.
(Re)Start JMeter to install the Right Tail Outlier Detector plugin.
Launch Plugins Manager by clicking on the icon at the top right.
Search for “Right Tail” under the “Available Plugins” as shown below.
Hit Apply Changes and Restart JMeter. This will download the relevant jars and place it in the appropriate locations and restart JMeter.
Usage
Launch JMeter and add the Right Tail Outlier Detection element which is located under Non-Test Elements as shown below.
You can see the element configuration as shown below.
What is an Outlier?
From performance test result perspective, an Outlier is a data point which is significantly different from the other data points.
E.g. following are the response time of a page load during the test run. The data point at 17:19:00 has 1.061 seconds which is different from the other data points. Here the outlier is 1.061 seconds.
| Time Stamp | Response Time (s) |
| 17:15:00 | 0.05 |
| 17:16:00 | 0.07 |
| 17:17:00 | 0.06 |
| 17:18:00 | 0.045 |
| 17:19:00 | 1.061 |
| 17:20:00 | 0.046 |
About Right Tail Outlier Detector Plugin
This plugin uses Tukey’s technique to find the upper bounds using two options: 1.5 and 3 Tukey’s constant k.
k 1.5 will identify all the outliers and k 3 will identify only the extreme values.
To detect the outliers, first you need the performance test results either in jtl, csv or tsv format.
I have the sample test result which I ran against my local app. First let us detect all the outliers using k 1.5.
Select 1.5 (detect all outliers) and browse the results file as shown below and then click on Detect.
For my test result, I have got 5 Outliers and the Outliers and Trimmed files have been generated in the bin folder.
You could discard the Outliers file and consider only the Trimmed file for further analysis. In case if you want to analyze the root cause of potential performance problems, you can consider Outliers files.
You can also see the Upper fence and the Trimmer count in the table view. To save the table data, click on Save Table Data.
For k 3, there are no Outliers detected for my test results, as it will detect only the extrement values.
Command Line Usage
You can leverage command line to identify the outliers on the fly. Issue the below command to detect the outliers for k 1.5.
java -jar lib\cmdrunner-2.2.jar --tool RightTailOutlierDetector --input-file bin\Run2.jtl --tukey-k 1.5
Conclusion
Detecting outliers helps you to analyze the performance test results. Outlier detection should be your first step in your checklist. This plugin saves time and helps you to detect in few clicks. Do you use any other method to detect the outliers? Please let me know in the comments.