performance testing Archives - Testbytes Software

What is Software Performance Testing?

Posted on December 24, 2019 by tbsupadminnxtbackendacc

Software performance testing checks the actual software with specifications mentioned before it is being built. Its most basic function is to work as a diagnostic aid to locate problems in communication. Scott Barber, a famous testing expert, said that it is difficult to pinpoint an exact definition for performance testing because it works differently in different situations and other various companies.

What is software performance testing?
Usually, software performance testing is performed to validate and verify quality metrics of software such as robustness, reliability, scalability, etc.
The software will be subjected to various load conditions and its response and output will be measured to find out various factors.

Typed of software performance testing

However, every company uses different types of performance tests depending upon the testing environment. Here’s a look at some of the popular performance testing:

1. Load Testing

One of the most basic types of performance testing, load testing method checks the increasing load of the software until the threshold is reached. Increasing load means an increasing number of simultaneous users, transactions, and behavior of the software under test situations.

The testing’s primary purpose is to observe the response time to ensure that the software can withstand a certain amount of load at any given time, which is why it is also called Volume Testing.

2. Stress Testing

Stress testing is another common performance testing method that checks the stability of the software when hardware including CPU, disk space, and memory isn’t stable. It is negative testing in which software is loaded with a large number of concurrent users that can’t be handled by the software.

It is a type of non-functional testing because it tests the non-functional aspects of a website or application, which is carried under a controlled situation so that it can capture uncertain scenarios and rectify them.

3. Throttle Testing

Throttle testing is a limited testing situation in which a load is tested at a limited speed to check the speed at which a website will load for users who have low network connectivity including those living in distant areas, 3G and 2G data, and using mobile internet. This testing makes use of certain tools like traffic sharpers and speed stimulators.

4. Endurance Testing

In endurance testing, the software is tested with a fixed amount of load for a long period of time to observe the system’s behavior. For instance, if the software is designed to work properly for four hours, its endurance is to be tested for double the time, which is eight hours.

This type of performance testing is used to check if there are any system fails or memory leaks. Endurance testing is often referred to as soak testing.

5. Spike Testing

Spike testing is similar to stress testing but the software is tested against load volumes and workload models for a longer time with a simultaneous increase in concurrent load.

The test is conducted for a time longer than its anticipated functioning time. Any unknown behavior is tapped and corrected at the very moment.

6. Smoke Testing

In smoke testing, the software is run on a very small load, which is much less than it can usually handle. If the test generates smoke, then the software has an error in its script.

It is one of the simplest tests used and usually the verification test that is run before the software is launched or larger tests are run.

7. Capacity Testing

As simple as it sounds, a capacity test is one in which the software is tested for the load it takes handle at any given time. When you know how many users your software can handle, you have better visibility of future events that might limit your application or website. It is a confirmatory test to avoid any future problems especially when you want to increase the volume of data.

8. Component Testing

Component testing checks the functionality of a particular part of your application or website. It tests simple but important aspects including file upload, shopping cart, email function, chart feature, or search function.

9. Peak Testing

In peak testing, you expose the software with increased load for a small time and regular peak in concurrent users to analyze how the website or application will behave in a real-life situations.

10. Scalability Testing

This kind of performance testing is usually carried out during the development of the software and just before launch. In scalability testing, you check how the website or application will program is another software is added to the server or find ways to improve loading time. The test’s primary aim is to find out at what peak the software will prevent more scaling.

Common Issues Observed during Performance Testing:

Performance tests are conducted to make sure the system is free of the fault so that it is ready to be launched in the market. Let’s take a look at some of the common issues that you might encounter during performance testing.

1. Poor Scalability

The system isn’t able to handle the desired number of load that delays loading time leading to unexpected benefits that can affect memory, CPU usage, disk usage, network configuration, and operating system.

2. Bottlenecking

This issue occurs when data flow is halted or interrupted because the website or application isn’t capable enough to handle the load.

3. Insufficient Hardware Resource

This means that there is a lack of hardware that results in memory shortage and CPUs don’t perform well.

4. Software Configuration Problem

It is a minor issue which occurs when configuration settings are set lower than the system can handle. All you have to do is adjust software configuration settings, and it’ll work perfectly fine.

Here are Some of the Best Software Performance Testing Practices:

Run performance tests as early as possible in the development process, so that you can conduct small trials that are successful rather than one big that will have many problems.
You should remember that performance testing is not only for the entire system but can also be used to test individual sections.
You must conduct multiple tests on one part to ensure that test results are consistent.
Make sure you always involve IT developers and testers in the process so that it is an all-around process and you don’t need to keep going between two departments.
Before the test, make sure that you require internal/external or a combination of both is required.
Devise a high-level plan before the test
Develop script for proof-of-concept for application under test
Configure router configuration, database test sets, quiet network, test environment, etc.
Before the actual test run a dry test to check the script is correct or not

Software performance testing process

Test environment identification- Identify the physical test environment, tools required and the goals that need to be achieved.
Identify performance acceptance criteria – Charestrisics such as response time, throughput, and resource-use goals and constraints
Plan and design tests- Prepare test environment, select the desired tools, devise strategy, etc.
Configure the test environment – Prepare the required resources to execute the tests
Implement test design – Develop test design in accordance with test design
Execute the test – Run, monitor, validate and collect the test and test data
Analyze test results, make reports, and retest – Consolidate the test data and analyze it. If tuning is required do that and go for a re-test.

Conclusion
Software Performance testing is a key process that can determine the success of your software. If you wish to know how your app can perform in the real world talk to us!

A Quick Guide To Load Testing: Why is it Crucial For Your Mobile App?

Posted on April 25, 2018 by tbsupadminnxtbackendacc

The app that you are about to launch lately is not just a product. It holds the beacon of your company.

It has been estimated that 26% of the app that has been download from various app stores are uninstalled within an hour itself.

So how do you ensure, that your app does not suffer the same fate?

Performance!

What better way to test it than performing a load test on your app.

Let’s have a look at it in detail.

Load Testing – What is it?

Load testing process involves artificially creating various load environments for your app, so that the performance of your app will be monitored closely to determine it is stable or not.

How is it Performed?

Check the Parameters and Creating a Work Load Model

First of all there are certain parameters that need to considered before the test begins. They are,

Response Time:

The time in which your app responds to certain inputs

Communication Rate:

Rate at which the software can send and receive requests

Resource Utilization:

While performing the test it has to be closely monitored that how much load the system is taking while interacting with the app.

User Load:

Concurrent user load the app can withstand

The Work Load Model

A work load model will make sure that the app is being injected with correct load parameters at different point of time.

For better understanding here is the detailed video regarding how to construct a work load model.

The test is basically done in definite intervals,

Exactly like that of the diagram below

Creating The Test Cases

While creating test cases it has to be kept in mind that expected result should have a fail or pass section for marking as it would be useful later.

While creating test cases make sure that,

Actions and scenarios of the testing procedure
Customise the test case for various inputs
Replay test cases for verifying the simulation

Execution of Test

When you run the test make sure that the load is increased slowly against the user profile. Between tests make sure that the system has enough breathing time so that it can stabilize. Last but not least, while incrementing make sure that the output of the app is being monitored and recorded.

Slow increase of the load will make sure that the threshold point of the app is marked well.

It is important that the load test is performed in cycles so that each cycle should have a load increment. After each analysis results should be verified and analysed.

Report Analysis:

Indentifying the hindrance for the good performance of any app application is an art. For effective analysis a well experienced testing engineer or the help of

an established software testing company is required as it requires chart analysis and various other forms of mathematical data interpretation.

The result obtained should be compared to that of respective industrial standard benchmark.

Benefits of Performing Load Test on Mobile Apps

All the functions of the app can be validated
Speed accuracy and stability of a app can be measured
User experience can be guaranteed once the app is deployed
Issues can be rectified before the deployment
Load capability can be improved

Factors That Need To Be Checked While Performing Load Test On Apps

First of all performance of the app should be impeccable and for that to happen these factors has to be perfect.

App Loading time
Power consumption
Memory consumption
Hardware /software integration
Integration with other apps
How app behaves if its retrieved from background

API/ Server Interaction

While an app interacts with Server via API these factors needs to be checked

Data transfer between app and the server
Number of app calls generated to the server
Server switch time

Network Performance

Delay time in receiving information from network
In case of packet loss, the request to the server should be resend
Network speed

Challenges That Can Come Up in Performing Load Test

Since Mobile phone comes in various size, shape and OS it’s hard to organize them
Device fragmentation
Different app types such as native, web app and hybrid app
Security of cloud server which performs testing

Reliable Tools Used For Load Testing Apps

Jmeter:

An open source modular GUI oriented tool that’s out to use the most now

Gatling:

Written in Scala based DSL the tool has been download over million times

Locust:

Written in Python the tool is developer friendly tool renowned for its user friendliness

Performance Testing – Types, Stages, and Advantages

Posted on November 24, 2017 by tbsupadminnxtbackendacc

Performance testing has proved itself as a crucial success factor of projects with an aim to demonstrate an application which works as per benchmarked specification within a given response time and on a practical database.

It actually enables developers to elicit the diagnostic information needed to eliminate bottlenecks in the performance of a system or software or application in users’ system. In otherwise it determines the readiness of the system to third party use.
Type of Performance tests
Performance testing is a comprehensive terminology. It entails five types of tests such as performance testing, load testing, scalability test, volume testing and stress testing.
1) Performance testing
This is the first task and it includes testing each part of the system to detect which part is slow. Based on the results of this test, efforts are made to make the response faster.
2) Load testing
This test is done to test whether the application or system is user ready in terms of expected load or not. The applications performance is examined by the response time it takes for a specified user load.
3) Stress testing
The test is done to understand at what level of stress or user number, the system starts generating errors and whether it is able to come back online after a massive spike in user number or a crash. The time taken to come back online is another factor to reckon with in this test.
4) Scalability testing
Scalability testing is done to ensure that the software is handling increasing workload effectively. This is done by gradually adding work load or users and continuously monitoring performance of the software or application.
5) Volume testing
This ensures how effectively the system is able to handle a large amount of data. That is why, it is also known as flood testing. The data is then gradually increased with the apps usage.
Performance Testing Stages
There are basically 7 important steps in performance testing.
1) Identifying the testing environment:
The testing team has to identify the hardware, software, network configurations and tools needed to start the test. This may include the actual production system or a replica of the same. Sometimes a subset of the production system with same or lower specification is also used.
2) Identifying the performance metrics:
Performance metrics is very important which determines the parameters on which the performance has to be tested. The metric must include response time, wait time, average load time, peak response time, error rate, concurrent users, throughput, request per second, CPU utilisation, memory utilisation, constraints etc. The success criterion differs from application to application. Therefore, corresponding success criteria must be included in the metrics.
3) Planning and designing the test:
At this stage the test team has to identify test scenarios taking into account variability of users, test data and testing parameters. This may lead to creation of more than one model for testing.
4) Configuring the testing environment:
At this stage the testing environment consisting of hardware, software and testing instruments needs to be prepared to monitor the performance of the application or system.
5) Capturing the data:
When everything is ready testing is begun and the data captured on different parameters is captured for analysis.
6) Analysis and report generation:
The data captured in the previous step is analysed and shared with the team. Based on the analysis corrective action is taken to rectify the problems.
7) Retesting:
The test is again performed on the same parameters to test if the rectification work is sufficient or needs fine tuning. Once this is found to be okay testing is carried out using different parameters and rectification, if needed, conducted.

Advantages of Performance Testing
Performance testing helps to –

Assess the amenability of the system to growth
Identify weak points in the architecture
Detect bugs in the software that could not have been identified otherwise
Tune up the system
Verify resilience and reliability of the system

When Should Performance Testing be Performed?
To save money and time, performance testing should be done early in the application or system development life cycle. This helps in testing the basic technology such as network performance, load balancing, application server performance, database server performance and web server performance. However, this should be done with expected load levels.
Detecting and correcting a performance problem after the website has gone live can be a prohibitively costly proposition for you. As soon as all the web pages are built and working load testing should be initiated. Thereafter, every time a module or part of software or application is implemented it should be tested for performance.
Conclusion
To avoid delay and cost overrun the best practice to be adopted is testing the application early in its development cycle. So individual units or modules should be tested for performance as and when these are complete.
Moreover, a single test does not resolve all queries. So repeated, but smaller tests need to be performed to come up with a robust system.

5 Tips to Setup a Better Performance Testing Environment

Posted on August 22, 2017 by tbsupadminnxtbackendacc

Performance testing is the process during which a product’s quality or its ability to function in the required environment is evaluated. As a non-functional testing technique, performance testing is conducted to evaluate a systems ability in terms of responsiveness and stability under workload. This process is conducted on three major attributes, which includes scalability, reliability and resource usage.

A number of techniques can be used to check performance of a software or hardware in a performance testing environment setup. This includes:

Load testing is the simplest form of testing and is basically conducted to understand the behaviour of the system under a specific load.
Stress testing is conducted to check a system ability to cope with the increased load, if any. It is performed to determine the maximum capacity of the existing system.
Soak testing, also known as endurance testing, is a type of testing done to verify a system’s ability to perform in situations of continuous load.
Spike testing is conducted by abruptly increasing the number of users of a system and determining how the system performs under such load.

There are a number of simple ways in which one can ensure accuracy and better results in the tests. This can be done using a better performance testing environment setup, which can be done in the following ways:
1. Detailed knowledge of AUT Production and Test Environment
It is the responsibility of a performance testing engineer to have complete knowledge and awareness about the AUT production environment such as server machines, load balancing as well as a number of other system components. These details, once known, should be properly documented and well-understood before initiating the initial stage of the performance testing
It is also important for an engineer to keep himself/herself aware about the complete details of the AUT architecture and to make sure that the same architecture is being executed in the test environment. This is because, having any sort of difference between the two can lead to the wastage of time, cost of production and effort.
2. Isolating the test environment
It is important to ensure that there is no activity which is being carried out on the performance test environment when there is someone already using the system. This is because the results of every performance test are different and it might get difficult to implement a new bottleneck every time in a test environment when there are also other users currently active on the system.
Apart from this, heavy load on any application server affects its performance. This, in turn, might not allow the other real time application users to successfully complete their tasks when a performance task is already under execution.
3. Network Isolation
It is important to ensure that a sufficient network bandwidth is available for your test as network bandwidth is essential to achieve accurate performance test results. In case the network bandwidth is low, the user requests begin to produce timeout errors. Therefore, one should make sure to provide maximum network bandwidth to the test environment by isolating the test network from other users.
4. Test Data Generators
Database records have an important role to play in the validation of any tests. Therefore, database reading, writing, deletion and updating are the most performance intrusive actions in any application.
Since there is a high probability of an application’s test failure in the production environment if it is conducted on lesser database records as compared to the test records. Therefore, it becomes the responsibility of the performance engineers to make sure that the number of test records are same in both the test environment system and database. In case, the database is small, it is recommended to pick some tool from the available ones and generate the required test data for better accuracy.
5. Removing proxy servers from network path
Performance results can be highly affected if there is a proxy server between the client and the web server. If the mid of client and web server has a proxy server, it is certainly true that the client will be served with data in cache and will stop sending any requests to web server. This, in turn, can lead to a lower AUT response time.

A performance engineer can deal with such issue by transferring the web server in a secluded environment. Another way is to strike directly to the web server, which can be done by editing the HOSTS file by including server IP address.
Performance Testing Production Environment
There are a number of advantages and disadvantages of conducting a performance test in production environment. Some of these include the following:
Advantages:

There is no need to reproduce the production site data set
Validating the performance test results performed on test environment is possible.
There is a reduced cost and time involved in test infrastructure.
Application recovery process and its complexities are well known.

Disadvantages:

Real application users end up receiving slower application and errors.
It gets difficult to identify the bottleneck root cause in the presence of real application users.
It is very likely that the access to real users might have to be blocked so as to properly achieve the performance test results.
In case of generating lots of data on production database, database may become very slow even after the test.

Once the test performance testing environment setup is ready, one can always compare it with the production environment on the basis of several factors such as the number of servers, load balancing strategy, hardware and software resources, application components, to name a few.
Above all, it is important that the required tests are conducted properly so that there are no faults pending in an application when making its use in real-life situations.

11 Guidelines for Mobile App Performance Testing

Posted on June 30, 2017 by tbsupadminnxtbackendacc

Performance testing services is done to provide information about apps regarding speed, stability, reliability and scalability. It also determines speed & degradation of systems with different configurations under both controlled and uncontrolled standards. More importantly, it unfolds what needs to be improved within an app before the product is launched in the market.

The General Guidelines for Performance Testing
The methodology for software performance testing can vary in different steps, but the objective of the whole process remains the same. The following are the list of general guidelines for performance testing of mobile apps:
1) Plan for Test: Once we are ready for a test, we need to have a proper planning and check for available resources. The performance test plan should describe the overall strategy for testing the complete mobile app.
2) Identification of Testing Environment: While testing, it is important to identify the software, hardware, tools and network configurations. This will allow the design and development team to identify the performance test errors at an early stage.
Different types of testing environment:
a) Replica of production system
b) Actual production system
c) Production system with less number of servers of low specifications
d) Production system with less number of servers of same specifications
3) Define Necessary Formats for Displaying System Usage: Necessary formats need to be defined for measuring the performance of an application. Some of the possible formats are:

Task Distribution Diagram: This gives information on the number of users for a particular process and the time taken to load. The peak and off usages are determined through this.
Transaction Profile Sheet: This provides information about the transaction names, a summary of a number of transactions at a particular point of time, screen navigation, etc.
User Profile Diagram: This depicts the user load time and pages accessed by the users

4) Deciding what to test first: The following are some criteria for decision-making about which process to be tested first. It is a very important step as it guides the total performance testing process.
·         Processes having the highest business risks are tested
·         Processes that are most frequently used are tested
·         Applications’ peak usage hours of the day are analysed
·         Peak days of the month, quarter, and year
·         Processes that are linked
·         Processes which are resource intensive
5) Documenting: The purpose of documenting the process is to obtain information necessary to create virtual users such as:
·         User Actions
·         Expected results of each action
·         Input data fields
·         Valid data for input
Identifying these properties shall give a clear scope of performance test against each user action.

6) Establish which data to use: Before starting, it’s important to establish the data to be used. Any real-time data can also be imported from the development servers and the same is used to analyse results and reports.
7) Define what resources are to be tested: Performance testing involves defining the resources that are involved in testing. This includes app servers like WebSphere, Weblogic, Tomcat, IIS Server; database server like DB2, Oracle and network resources. These resources form the total system and therefore defining them is critical.
8) Performance testing should be done after functional testing: There is no question of doing performance testing when functional testing is not done. Functional testing confirms the working of the system as expected with all aspects of the UI, multilingual navigation, navigation between pages and off-site, etc.
9) Done at the time of system-testing phase: System testing is a phase of software testing in which the complete system is tested. So, performance testing should be taken up during system testing phase only.
10) Tools should be identified: There are plenty of tools available for doing performance testing. But, we need to identify tools that may be useful for system testing.
11) Execution and Communicating Results: Once the performance testing is completed, we need to summarize the findings and test results. Later, it is distributed to respective resource owners for making corrections if any.
Once the corrective measures/fixes are done, the entire tests need to be repeated and the final summary report is taken.
Conclusion
While an inclusive performance testing strategy will continue to kill bugs for all tech innovations, this will certainly help you to bring the best out of your testing efforts. With perspectives and tools emerging each day, testers need to leverage on this testing mechanism for their mobile app testing services.