Designing Performance Tests

For many of us, who are not working as performance test engineers, performance testing is an uncharted territory. And it’s a common issue because there are so many performance test types and even more ways to measure them. Depending on the application that you want to test the scope might be quite large and sometimes too large to even cover basic performance testing needs. Also, it’s very hard to know where to start.

In addition, reporting is also an issue, even if you have the right method or tool, it might be hard to integrate that with CI (continues integration) and compare your performance test runs with previous data to measure changes.

Performance testing types:

After going through many different types of performance tests I compiled a simple list for myself which I use to determine and implement a basic performance test coverage for the application/software. I know that there a few more test types, but I believe those are very specific, made for niche software and complicates coverage definition quite a bit. These test type can be applied to most of the software/applications and provide a basic test coverage which can be used as a performance threshold for later testing.

Performance test To determine or validate speed, scalability, and/or stability
Load test To verify application behaviour under normal and peak load conditions
Stress test To determine or validate an application’s behaviour when it is pushed beyond normal or peak load conditions.
Throughput test To determine how many users and/or transactions a given system will support and still meet performance goals.

Reporting/Tools:

Next step in this process is to select tools that you’ll be using to run and report. This is just an example and can be extended by adding more tools which suite your application.

Splunk
  • Could be used very detail report generation
  • Performance measuring, calculations and reporting are being done on Splunk side
  • Pretty much real-time reporting
  • Splunk keeps 30 days of logs
JMeter
  • Used to measure SOAP and DB performance
  • Easy to integrate with Jenkins
  • Good reports
Jenkins
  • Test NG Pass or Fail test

Scope:

Now when we have test types and tools ready we should generate out performance testing scope. Do we only want to check application performance or we want to dig more and also check DB/Services?

Application Testing application level system performance
SOAP services Testing performance of SOAP services
Database Testing performance of DB connection

Performance Test Table:

At this point, there is only one table left to finish off the process and it depends on your application/website. You just write down the component/application/website name that you want to test and add the data from the tables above. It’s a pretty straightforward process and as soon as you begin, more ideas will pop up and you will be able to cover the majority of your application.

Example:

Test Name
Type
Scope
Component
Tool
 Notes
Uploading Files on website Load test Application Uploading Jenkins   With this test we will check how normal/peak load affect the application
Uploading Files on website Stress test Application Uploading Jenkins  This test will push the application beyond supported levels (Uploading 100k files)
Login into the system Performance test Application Uploading Jenkins

All in all…

I personally found this to be a great way to determine the scope of performance testing, in addition, it’s easy to demonstrate your intentions and thought the process to colleagues/management/business which makes things easier for everyone.

All in all, it’s quite hard and confusing to efficiently define a basic performance testing scope and it’s hard to know where to begin, this method worked for me quite well, allowed me to create a large number of tests and then consolidate them into one basic performance test suite which was a great threshold for the future runs. And as I mentioned it’s quite easy to share/demonstrate the last table to everyone who is interested to get as much feedback and suggestions as possible.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s