Volume Testing Tutorial: Examples And Volume Testing Tools

 What is Volume Testing?

Volume Testing is a type of non-functional testing. This testing is done to check the data volume handled by the database. Volume testing also called as flood testing is non-functional testing that is done to check the software or app for its performance against huge data of the database.

The database is stretched to a threshold point by adding a large amount of data to it and then the system is tested for its response.

This was the theory part, let me explain you with a few practical examples to help you understand the ‘when’ part of volume testing.

When is This Testing Imperative?

Ideally every software or an app should be tested for data volume but in some cases where the data will not be heavy, we tend to avoid this testing. But in some cases where data is dealt with in MBs or GBs on a daily basis then definitely, volume test should be performed.

Following are a few examples out of my own experience of 8 years that explain the ‘when’ part:

Example 1:

One of my ventures was a big system which comprised of both web app and a mobile app. But the web app itself had 3 modules handled by 3 different teams.

At times, even with us, the database used to become slow when we all ‘together’ added data for our testing. It was annoying and the work used to get hampered because of the huge volume of data and to ease the work we had to clean up the DB quite frequently.

The data that the ‘live’ system was handling was around a GB, hence when compared to the mobile app, the web app was very frequently tested for the volume of data. The web app QA teams had their own automation scripts that would run at night and perform this testing.

Example 2:

Another example of my venture was an ecosystem which not only had a web app but also a SharePoint app and even an installer. All these systems were communicating to the same database for data transfers. The data handled by that system was also very huge and if for any reason the DB becomes slow even the installer would stop working.

Hence, volume test was done on a regular basis and the DB performance was observed minutely for any issues.

Similarly, we can take Examples of few apps that we use on a daily basis for shopping, booking tickets, financial transactions etc which deal with heavy data transactions and hence need a volume test.

On the flipping side, an ideal volume testing may not always be achievable as it has its own limitations and challenges.

Few of its limitations and challenges include:

• It is difficult to create the exact fragmentation of the memory.
• Dynamic key generation is tricky.
• Creating an ideal real environment i.e. the replica of the live server can be tricky.
• Automation tools, network, etc., also affect the test results.

Now, we have understood when we need to do this type of testing. Let us also understand ‘why’ we should do this testing as in, the objective or aim of performing this testing.

Why should I Aim for Volume Testing?

Volume testing can help you to understand how fit your system is for the real world and it also helps to save your money which will later be spent on for maintenance purposes.

Following are few possible reasons for performing this testing:

• The most basic need is to analyze your system’s performance against increased data. Creating a huge volume of data will help you to understand the performance of your system in terms of response time, data loss, etc.
• Identify the issues that will occur with huge data and the threshold point.
• Beyond the sustainable or threshold point, the system behavior i.e. if the DB crashes becomes irresponsive or timing out.
• Implementing solutions for DB overload and even verifying them.
• Finding out the extreme point of your DB (which can’t be fixed) beyond which the system will fail and thus precaution needs to be taken.
• In case of more than one DB servers, finding out the issues with DB communication, i.e. the most prone to failure out of them, etc.

Now we know the importance and reason for performing this testing.

One experience that I would like to share here is that in terms of mobile apps, volume testing may not be needed because only one person uses the app at a time and mobile apps are designed to be simple.

So unless you have a very complex app with a lot of data involvement, volume testing can be skipped.

Once you know what has to be verified for your system or app, the next thing to do is to make a checklist for your app to define ‘what’ needs to be tested.

What is my Checklist for this testing?

Before we step into some examples for creating a checklist for your app or a system, let us first understand few pointers to keep in mind while creating a checklist for volume testing or the approach before starting the testing.

Points to be remembered:

• Keep the developers in the loop about your testing plan because they know a lot about the system and can provide you inputs and even bottlenecks.
• Understand the physical aspect as in the server configurations, RAM, processor etc well before strategizing the testing.
• Understand the complexities of the DB, the procedures, DB scripts etc to the possible extent so that you can outline your system’s complexity on a whole.
• Prepare informatics i.e. graphs, datasheet, etc., if possible for the normal volume of data and how well is the system, this will help you to make sure that before you stress the DB, the performance is fine for normal data load. This will also help you to ensure before you move on to the stressing part, that there are no issues which will require a fix for your volume test.

Following are some Examples that you can add or use in your checklist:

• Check for the correctness of the data storage methods.
• Check if the system has the necessary memory resources or not.
• Check if there’s any risk of data volume greater than a specified limit.
• Check and observe the system’s response to the data volume.
• Check if the data is getting lost during volume testing.
• Check that if data is overwritten, then it is done so with prior information.
• Identify the areas that extend beyond the normal range like a lot of attributes (searchable), huge no. of lookup tables, a lot of location mappings, etc.
• As mentioned earlier, create a baseline first by getting results for the normal volume and then move ahead with stressing.

Before we move on to the other examples, test cases, and tools, let us first understand how this testing differs from load testing.

Volume Testing Vs Load Testing

Given below are some of the prime differences between Volume and Load Testing:

S.No.	Volume Testing	Load Testing
1	The volume testing is done to verify the database performance against a large volume of data in the DB.	The load testing is done by changing the user loads for the resources and verifying the performance of the resources.
2	The primary focus of this testing is on ‘data’.	The primary focus of this testing is on ‘users’.
3	The database is stressed to the maximum limit.	The server is stressed to the maximum limit.
4	A simple example can be creating a huge sized file.	A simple example can be creating a large number of files.

How to Perform This Testing?

This testing can be done both manually or by using any tool. In general, using tools will save our time and efforts but in case of volume test, as per my experience using tools can give you more accurate results when compared to manual testing.

Before starting your test case execution make sure that:

• The team has agreed to the testing plan for this testing.
• Other teams of your project are well informed about the database changes and its impact on their work.
• The testbeds are set for the specified configurations.
• The baseline for testing is prepared.
• The specific data volumes for testing (data scripts or procedures etc) are ready. You can read about data creation tools on our data generation page.

Let’s see few sample test cases that you can use in execution:

Verify this for all the selected data volumes for Volume testing:

1. Verify if adding data can be done successfully and if it reflects in the app or website.
2. Verify if deleting data can be done successfully and if it reflects in the app or website.
3. Verify if updating data can be done successfully and if it reflects in the app or website.
4. Verify that there is no data loss and all the information is displayed as expected in the app or website.
5. Verify that the app or web pages are not timing out due to high data volume.
6. Verify that crashing errors are not shown due to high data volume.
7. Verify that data is not overwritten and proper warnings are shown.
8. Verify that other modules of your website or app are not crashing or timing out with high data volume.
9. Verify that the response time of the DB is within the acceptable range.

Volume Testing Tools

As discussed earlier that automation testing saves time and even gives accurate results when compared to manual testing. Another benefit of using tools for volume testing is that we can run the tests at night and that way work of the other teams or team members won’t be affected by the data volume of the DB.

We can schedule the tests in the morning and the results will be ready.

Following is a list of few open-source volume test tools:

#1) DbFit:

This is an open-source tool that supports test-driven development.

DbFit testing framework is written on top of Fitness, the tests are written using tables and can be executed using any Java IDE or CI tool.

#2) HammerDb:

HammerDb is also an open-source tool which can be automated, multi-threaded and even allows the run-time scripting. It can work with SQL, Oracle, MYSQL etc.

#3) JdbcSlim:

JdbcSlim commands can be easily integrated into Slim Fitness and it supports all the databases which have a JDBC driver. The focus is on keeping the configuration, test data and SQL queries separately.

#4) NoSQLMap:

This is an open-source Python tool that is designed to automatically inject attacks and disrupt the DB configurations to analyze the threat. It works only for MongoDB.

#5) Ruby-PLSQL-spec:

The PLSQL can be unit tested using Ruby as Oracle is available as an open-source tool. This basically uses two libraries: Ruby-PLSQLand Rspec.