For many of us, who are not working as performance test engineers, performance testing is an uncharted territory. And it’s a common issue because there are so many performance test types and even more ways to measure them. Depending on the application that you want to test the scope might be quite large and sometimes too large to even cover basic performance testing needs. Also, it’s very hard to know where to start.
In addition, reporting is also an issue, even if you have the right method or tool, it might be hard to integrate that with CI (continues integration) and compare your performance test runs with previous data to measure changes.
Performance testing types:
After going through many different types of performance tests I compiled a simple list for myself which I use to determine and implement a basic performance test coverage for the application/software. I know that there a few more test types, but I believe those are very specific, made for niche software and complicates coverage definition quite a bit. These test type can be applied to most of the software/applications and provide a basic test coverage which can be used as a performance threshold for later testing.
|Performance test||To determine or validate speed, scalability, and/or stability|
|Load test||To verify application behaviour under normal and peak load conditions|
|Stress test||To determine or validate an application’s behaviour when it is pushed beyond normal or peak load conditions.|
|Throughput test||To determine how many users and/or transactions a given system will support and still meet performance goals.|
Next step in this process is to select tools that you’ll be using to run and report. This is just an example and can be extended by adding more tools which suite your application.
Now when we have test types and tools ready we should generate out performance testing scope. Do we only want to check application performance or we want to dig more and also check DB/Services?
|Application||Testing application level system performance|
|SOAP services||Testing performance of SOAP services|
|Database||Testing performance of DB connection|
Performance Test Table:
At this point, there is only one table left to finish off the process and it depends on your application/website. You just write down the component/application/website name that you want to test and add the data from the tables above. It’s a pretty straightforward process and as soon as you begin, more ideas will pop up and you will be able to cover the majority of your application.
|Uploading Files on website||Load test||Application||Uploading||Jenkins||With this test we will check how normal/peak load affect the application|
|Uploading Files on website||Stress test||Application||Uploading||Jenkins||This test will push the application beyond supported levels (Uploading 100k files)|
|Login into the system||Performance test||Application||Uploading||Jenkins|
All in all…
I personally found this to be a great way to determine the scope of performance testing, in addition, it’s easy to demonstrate your intentions and thought the process to colleagues/management/business which makes things easier for everyone.
All in all, it’s quite hard and confusing to efficiently define a basic performance testing scope and it’s hard to know where to begin, this method worked for me quite well, allowed me to create a large number of tests and then consolidate them into one basic performance test suite which was a great threshold for the future runs. And as I mentioned it’s quite easy to share/demonstrate the last table to everyone who is interested to get as much feedback and suggestions as possible.