How to setup Realistic Test in JMeter using Concurrency Thread Group
What is a real-world performance test
- Simulate actual user actions with timings/delays
- Control ramp-up and down of virtual users
- Control timing between iterations
- Achieve x iterations in y mins/sec
We will learn about Pacing in this article, we'll have to follow below steps to setup a realistic test
Step 1 - Download JMeter plugin manager
We cannot create realistic environment using JMeter only, so we'll need a plugin that will enable JMeter to create a realistic environment.
- Download JMeter Plugin Manager from here.
- Put plugins-manager.jar into lib/ext directory.
- Restart JMeter.
- After restart you can see plugin manager in the toolbar.
Step 2 - Open Plugin Manager and select CTG
In plugin manager we can see all the pre-installed plugins as well as available plugins. We can select the plugins we need for our test.
- Open Plugin Manager and go to Available Plugins
- Type Custom Thread Group and select Custom Thread Group(follow white arrow)
- Click on Apply changes and restart JMeter(follow red arrow)
- I have already added it. That's why it is showing in Installed plugins
Step 3 - Create Testplan
TestPlan is created to check performance of different pages under a single project
- Name the test plan. I have given Test
Step 4 - Add Concurrency Thread Group
This thread group offers simplified approach for configuring threads schedule. It is intended to maintain the level of concurrency, which means starting additional during the runtime threads if there's not enough of them running in parallel. Unlike standard Thread Group, it won't create all the threads upfront, so extra memory won't be used.
- Add Concurrency Thread Group Test(right click)>Add>Thread Group>bzm- Concurrency Thread Group
- Consider the test case scenario, there are some parameters, we have to set them accordingly.
- The given scenario explains the concurrency that means every 0.4 seconds, 50 users will be added until we reach 200 users. (20 seconds divided by 50 steps equals 0.4 seconds per step. 200 users divided by 50 steps equals 4 users per step. Totaling 50 users every 0.4 seconds).
- Hold target rate time means After reaching 200 threads all of them will continue running and hitting the server together for 5 seconds.
- We can see the graphical representation of our desired test.
Step 5 - Add a Sampler (HTTP)
Sampler is used to run the test. We will use HTTP sampler as we'll be testing the performance of a website
- Follow this to create a sampler Concurrency Thread Group(right click) >Add>Sampler>HTTP request
- Suppose we have to test the page https://qaprovider.com/project/1/execute/118
- Server name or IP for this page is qaprovider.com . Don't write https:// as it takes http as default protocol.
- Path is the exact page of the website we want to test i.e. /project/1/execute/118
- Keep the rest of the parameters as it is
Step 6 - Add Listeners
To see results of the test we'll have to use listeners. Here we will use Aggregate Report and Results Tree as Listeners. Aggregate report will show us the total no. of samples, their success rate and error rate. Results tree will show the detailed result of each sample.
- Add listeners Concurrency Thread Group(right click) >Add>Listeners>Aggregate Report/View Results Tree
Step 7 - Run & Validate the test
Now we'll check results in both the listeners,
In Aggregate report , we can see that 18.75% error rate is captured as the load is more for a very short period of time
Now we will check Results tree for detailed error of samples
- We can see that some of the samples give error code 502 i.e. Bad Gateway.
- This is because the load is very high i.e. 50 users every 0.4 seconds, so some of the users will get error like this
In this way we can check how much load the server can bear for a given period of time under realistic testing environment. We can change parameter values in Concurrency Thread group to find optimum load the server can bear without involving a single real user. This type of tests can be executed using Stepping Thread Group also but it is deprecated, so we use Concurrency Thread Group.
This is all about Pacing -- A Realistic Test Environment.
Thanks for reading.
Cool! We have checked our home page with your data.
Why did you advice to use default http here?
"Don't write https:// as it takes http as default protocol"
We have http -> https redirect, so the server will run this redirect every request here. Any purpose on this? It is valid case, but may be some reason why you have mentioned this?
Actually you don't need to write the protocol. Be it http or https , after the test is executed you can see in the result that the protocol will be automatically retrieved i.e. if it is https then in request bodbody of result tree you will see https before the url.
From the logs we found resource that cause this problem (it is php-fpm):
2019/07/04 00:47:25 [error] 15697#15697: *10783 connect() to unix:/run/php/php7.2-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: (ip), server: qaprovider.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "qaprovider.com"
We have nginx as web server that proxies all php requests to php-fpm. From error above we see that problem after nginx, so the web server is performing correctly.
We had settings for maximum PHP-FPM processes defined as value 5.
pm.max_children = 5 // this was in www.conf
One PHP process takes about 20 MB per request. We have 2 GB server. We decide to setup pm.max_children = 50. This value was used in JMeter test settings for Ramp-Up Steps count. So, in total PHP processes will take around 1000 MB ~ 1GB.
After some playing with settings we decide to change pm=dynamic on pm=ondemand
And we see new results for your settings (Error is 0%):
Really it is a cool instrument for fast testing of server settings.
Raja, thank you very much for this post!