Month: October 2023

What is Compliance Testing? How to do it?

Posted on by tbsupadminnxtbackendacc

Compliance testing evaluates and assesses whether your software fulfills all the regulations, standards, requirements of specifications, etc. that it has to stand true on.

The process can be considered more as an auditing task to ensure that it fulfills required standards.

It is many times also referred to as conformance testing.

Attributes of compliance testing

  • Robustness
  • Performance
  • Interoperability
  • Functions
  • Behavior of system

What are the prerequisites of compliance testing?

  • The product development should be complete with all the features working as expected.
  • The documentation and user manuals for the product should be available to help understand and recheck for compliance.
  • The online support and documentation, if applicable should be the latest version.
  • Functional and integration testing should be complete and should satisfy the exit criteria.
  • Escalation matrix should be available along with the point of contact for development, testing, and management teams
  • All licenses should be up to date.

Importance of Compliance Testing

Here are a few points that will help you understand its utility.

  • To validate if your software fulfills all the system requirements and standards.
  • To assess if all the related documentation is complete and correct.
  • To validate the software design, development, and evaluation are carried out as per specifications, standards, norms, and guidelines.
  • To validate if system maintenance is determined as per specified standards and recommended approach.
  • To assure that your software is free from any sort of complaints from regulatory bodies, regulatory compliance testing is performed.

Who executes Compliance testing?

Many companies do not consider it mandatory. Why? executing the test largely depends on the management.

However, If they consider a need to execute compliance testing, they hire or ask the in-house team to conduct compliance testing.

Many organizations also deploy a panel of experts or a regulatory body to assess and validate various regulations, specifications, policies, and guidelines.

Vulerability assessment protection

What to test in Compliance testing?

The process is initiated by the management taking care of the complete understanding of the team about various regulations, specifications, guidelines, etc.

To ensure the best results and quality assurance, all the regulations and standards should be clearly mentioned to the team to avoid any ambiguities.

  • Requirement objectives
  • Scope of requirements
  • Standards that rule the implementation
  • Call of the software to be developed

What are the examples of compliance testing?

Some of the examples of compliance testing are:

  • User Access Rights and Security Regulations
  • Program change and control procedures
  • The procedure and guidelines for documentation
  • The guidelines for program documentation
  • Logs review
  • Audit of the software artifacts including licenses

What is not tested in compliance testing?

Some teams consider system and integration testing to be part of compliance testing as well. But that is not true.

Compliance does not mean re-running the system or functional tests.

On the contrary, compliance tests are a set of specifically designed tests that are carried out at the end of the software development cycle before rolling out the software product to production.

When to perform Compliance Testing?

There are some countries where compliance testing is mandatory and they have specific guidelines as well to accomplish this testing.

In most other countries, it is purely a management call. If the management wants to strictly follow the set guidelines, rules, and best practices, it will be pushing for a compliance test.

For the compliance tests to be carried out, the first step would be to chart out a detailed document with the procedures, standards, and methodology. It will be based on these that the compliance tests are designed.

Also, the compliance test would differ from one domain to another. Thus these tests need to be designed as per the industry and domain needs.

How to perform compliance testing?

it is more like an audit and follows no specific testing methodology.

You can simply carry it out like other general testing methods.

Here is an overview of the generic compliance testing methodology that may help you in performing it.

  • The first step is to collect precise details about all specified standards, norms, regulations, and other relevant criteria.
  • In the next step, you are required to document all the norms and standards clearly and precisely.
  • In the third step, you will have to keenly assess all the development phases against the documented standards and norms to identify and detect any deviations or flaws in the implemented process.
  • The next step includes creating a report and reporting all the flaws to the concerned team.
  • Lastly, you are required to re-verify and validate the affected areas post-fixation to ensure conformance to the required standards.
  • If required certification is provided to the system for the compliance of required norms and standards.

What is the need for compliance testing?

Here are the reasons

  • Safety: The safety of the customers and the safety of the product are the primary reasons for conducting compliance tests. Compliance tests are designed to find negligence issues and to ensure all safety standards are met.
  • Quality: Improved and proven quality is another reason why we should push for compliance testing for the products. Apart from the compliance test, it is also important to conduct periodic audits.
  • Legal Requirements: In some cases, the companies are legally bound to conduct compliance tests before releasing the products. If these tests are not performed legal action can be taken against the company and their license can also be canceled.
  • Customer Satisfaction: Customers would have more confidence in a product that is tested and is marked compliant. It is thus good for the company and its reputation as well.
  • Conformance: Compliance with the physical standards ensures conformance and compatibility with other products in the market that might be from different manufacturers.

Who sets the standards for compliance testing?

Most commonly, there are external organizations that come with the standards in compliance testing for various industries and are then accepted by a majority of the industries.

Some organizations are

Based upon the required standards and your system type there are many compliance testing tools that are available in the market.
Here are the names of a few commonly used compliance testing tools.

  • EtherCAT conformance testing tool
  • MAP2.1 conformance testing tool
  • Software Licence Agreement OMS Conformance Tester 4.0
  • CANopen Conformance test tool

Advantages of Compliance Testing

Unfortunately, compliance testing has not yet become a widely accepted part of STLC, but it is advisable to carry around to assure better performance and compliance of your software.

Listed below are a few points that might help you to better understand the advantages of carrying out the process

  1. It assures proper implementation of required specifications
  2. It validates portability and interoperability
  3. It validates whether the required standards and norms are properly adhered to
  4. Validate that the interfaces and functions are working as expected
  5. Can help you identify the areas that are to be confirmed with those which are not to be confirmed such as syntax and semantics

Disadvantages of Compliance Testing

Here are some challenges that you might incur while doing compliance testing

  1. To get the best results, you need to identify the class of the system, and then the testing has to be carried out based on the class following a suitable methodology
  2. You will have to specific specifications into Profiles, Levels, and Modules
  3. You will need to have the complete know-how of different standards, norms, and regulations of the system to be tested.

What is the need for compliance testing?

One may wonder why they need compliance testing when functional, system, and integration testing are already done.
Here are the reasons, why we need compliance testing.

  • Safety: The safety of the customers and the safety of the product are the primary reasons for conducting compliance tests. Compliance tests are designed to find negligence issues and to ensure all safety standards are met.
  • Quality: Improved and proven quality is another reason why we should push for compliance testing for the products. Apart from the compliance test, it is also important to conduct periodic audits.
  • Legal Requirements: In some cases, the companies are legally bound to conduct compliance tests before releasing the products. If these tests are not performed legal action can be taken against the company and their license can also be canceled.
  • Customer Satisfaction: Customers would have more confidence in a product that is tested and is marked compliant. It is thus good for the company and its reputation as well.
  • Conformance: Compliance with the physical standards ensures that conformance and compatibility with other products in the market that might be from different manufacturers.

Types of compliance testing?

  1. Mandatory Testing: In some countries for security-related software products, compliance testing is legally mandatory. This testing is either performed by a govt agency or a third party appointed by the govt. For the product to be released it requires certifications from the govt. Failing to comply with tests could mean withdrawing the product from the market, fines, payment of damages, or more.
  2. Obligatory Testing: When 2 companies are working with each other, one company may ask for a compliance test report from the other. Failure to perform the tests could lead to contract termination and subsequent loss of business.
  3. Voluntary Testing: To ensure that the process is carried out in an unbiased manner, companies may engage third parties to do compliance testing. The company may not be legally bound to do the test but want to perform the tests to ensure the best product rollout.
  4. Internal Testing: Companies can also engage the teams internally to perform compliance tests to improve the performance of their products and services. This is not a regulation but is done based on the directive from the management.

Standards in compliance testing

  1. SO 9001 (Quality Management System)
  2. ISO/IEC 27001 (Information Security Management)
  3. ISO 13485 (Medical Devices)
  4. HIPAA (Health Insurance Portability and Accountability Act)
  5. PCI DSS (Payment Card Industry Data Security Standard)
  6. GDPR (General Data Protection Regulation)
  7. Sarbanes-Oxley Act (SOX)
  8. COBIT (Control Objectives for Information and Related Technologies)
  9. IEEE 829 (Software Test Documentation)
  10. OWASP Top Ten (Web Application Security)

Forms of compliance testing

 Internal Testing

This is performed internally by the organization to ensure that the software and processes adhere to the policies, standards, and best practices of the business. It contributes to the quality and consistency of software development.

External or legally required testing for compliance:

Compliance testing of this nature is mandated by law by governmental authorities or industry-specific regulatory organizations. It guarantees compliance of the software with obligatory regulations, laws, and standards. There may be legal repercussions for noncompliance.

Testing for mandatory or obligatory compliance:

Comparable to testing that is mandated by law, this is necessary to comply with particular industry standards and regulations. Instances of such adherence encompass healthcare software conformity with the Health Insurance Portability and Accountability Act (HIPAA) and payment processing applications’ adherence to the Payment Card Industry Data Security Standard (PCI DSS).

Testing for Voluntary Compliance:

Organizations may elect to undergo voluntary compliance testing as a means of showcasing to clients or business partners their dedication to quality and safety. Compliance with industry-recognized standards, even in the absence of legal requirements, may be required.

Compliance testing in various forms is of the utmost importance in guaranteeing that software satisfies the mandatory criteria, be they those mandated by legislation, industry standards, or internal quality assurance processes. They aid in ensuring that software is dependable, secure, and conforms to stakeholders’ expectations.

Conclusion:

Delivering glitch-free software enhances your customer’s trust in you. Compliance testing is another step that assures that your system is free from any flaws and glitches…

What is Boundary Value Analysis?

Posted on by tbsupadminnxtbackendacc

BVA (Boundary Value Analysis) is a software testing technique that focuses on testing values at the extreme boundaries of input domains. It is based on the observation that defects frequently occur on the outskirts of valid input ranges rather than in the center. Testers hope to identify potential issues and errors more effectively by testing boundary values. BVA is widely used in black-box testing and is especially useful for detecting off-by-one errors and other boundary-related issues.

Here’s an example of Boundary Value Analysis:

Consider the following scenario: You are testing a software application that calculates discounts for online purchases. The application provides discounts based on the amount of the purchase and has predefined discount tiers.

  • Tier 1: 0% discount for purchases less than $10.
  • Tier 2: 5% discount for purchases from $10 (inclusive) to $50 (exclusive).
  • Tier 3: 10% discount for purchases from $50 (inclusive) to $100 (exclusive).
  • Tier 4: 15% discount for purchases of $100 or more.

In this scenario, you want to apply Boundary Value Analysis to ensure the discount calculation works correctly. Here are the boundary values and test cases you would consider:

  • Boundary Value 1: Testing the lower boundary of Tier 1.
    • Input: $9.99
    • Expected Output: 0% discount
  • Boundary Value 2: Testing the upper boundary of Tier 2.
    • Input: $10.00
    • Expected Output: 5% discount
  • Boundary Value 3: Testing the lower boundary of Tier 3.
    • Input: $50.00
    • Expected Output: 10% discount
  • Boundary Value 4: Testing the upper boundary of Tier 3.
    • Input: $100.00
    • Expected Output: 10% discount (Tier 3)
  • Boundary Value 5: Testing the lower boundary of Tier 4.
    • Input: $100.01
    • Expected Output: 15% discount
  • Boundary Value 6: Testing the upper boundary of Tier 4.
    • Input: $1,000.00
    • Expected Output: 15% discount (Tier 4)

By testing these boundary values, you ensure that the software handles discounts at the tier’s edges correctly. If there are any flaws or issues with the discount calculation, this technique will help you find them. Boundary Value Analysis improves software robustness and reliability by focusing on critical areas where errors are likely to occur.

Boundary Value Analysis Diagram

 

What are the types of boundary value testing?

Boundary value testing is broadly classified into two types:

Normal Boundary Value Testing: This type is concerned with testing values that are precisely on the boundary between valid and invalid inputs. Normal boundary value testing, for example, would examine inputs like 1, 100, and any values in between if an input field accepts values between 1 and 100.

Robust Boundary Value Testing: This type of testing includes values that are slightly outside of the valid boundary limits. Using the same example, robust boundary value testing would use test inputs such as 0, 101, -1, and 101 to see how the system handles them.

While these are the two most common types of boundary value testing, there are also variations and combinations based on the specific requirements and potential risks associated with the software being tested.

What is the difference between boundary value and equivalence testing?

Aspect Boundary Value Testing Equivalence Testing
Focus Concerned with boundary values Focuses on equivalence classes
Objective To test values at the edges To group similar inputs
Input Range Tests values at boundaries Tests values within classes
Number of Test Cases Typically more test cases Fewer test cases
Test Cases Includes values on boundaries Represents one from each class
Boundary Handling Checks inputs at exact limits Tests input within a class
Risk Coverage Addresses edge-related issues Deals with class-related issues
Applicability Useful for validating limits Suitable for typical values

The goal of boundary value testing is to discover issues related to boundary conditions by focusing on values at the edges of valid ranges. Equivalence testing, on the other hand, groups inputs into equivalence classes in order to reduce the number of test cases while maintaining effective test coverage. Both techniques are useful and can be used in tandem as part of a comprehensive testing strategy.

Advantages and DIsadvantages of Boundary Value Analysis

Benefits of Boundary Value Analysis:

  • BVA focuses on the edges or boundaries of input domains, making it effective at identifying issues related to these critical points.
  • It provides comprehensive test coverage for values near the boundaries, which are often more likely to cause errors.
  • BVA is simple to understand and implement, making it suitable for both experienced and inexperienced testers.
  • It can detect defects in the early stages of development, lowering the cost of later problem resolution.

The following are the disadvantages of boundary value analysis:

  • BVA’s scope is limited to addressing boundary-related defects and potentially missing issues that occur within the input domain.
  • Combinatorial Explosion: BVA can result in a large number of test cases for systems with multiple inputs, increasing the testing effort.
  • Overlooking Class Interactions: It fails to account for interactions between different input classes, which can be critical in some systems.
  • BVA makes the assumption that system behavior near boundaries is linear, which may not be true for all applications.
  • BVA may not cover all possible scenarios or corner cases: While it is effective in many cases, BVA may not cover all possible scenarios or corner cases.

 

FAQs

What’s boundary value analysis in black box testing with an example

BVA is a black-box testing technique that is used to test the boundaries of input domains. It focuses on valid and invalid input ranges’ edges or boundaries to test values. The primary goal is to ensure that a system correctly handles input values at its limits, as this is frequently where errors occur.

Here’s an illustration of Boundary Value Analysis:

Consider the following scenario: You are testing a simple calculator application, and one of its functions is to add two numbers. The application accepts integers from -100 to +100.

Boundary Values: The following are the boundary values in this scenario:

Lower Boundary: -100 Upper Boundary: +100 BVA Test Cases:

Test with the smallest valid input possible:

Input 1: -100
Input 2: 0
-100 is the expected outcome. (At least one valid input)
Test with the most valid input possible:

Input 1: 100
Input 2: 50
150 (Maximum valid input) is the expected result.
Just below the lower boundary, perform the following test:

Input 1: -101
Input 2: 50
Expected Outcome: Error (outside of the valid range)
Just above the upper limit, perform the following test:

Input 1: 101
Input 2: 50
Error (outside valid range) is the expected outcome.
By using Boundary Value Analysis in this example, you ensure that the calculator application handles edge cases at the input range’s minimum and maximum boundaries, as well as values just outside the boundaries, correctly. This assists in identifying potential boundary value errors or issues.

Equivalence Partitioning and Boundary Value Analysis, What’s the difference?

Aspect Equivalence Partitioning Boundary Value Analysis
Definition Divides the input domain into groups or partitions, where each group is expected to behave in a similar way. Focuses on testing values at the edges or boundaries of the input domain.
Objective Identifies representative values or conditions from each partition to design test cases. Tests values at the extreme boundaries of valid and invalid input ranges.
Usage Suitable for inputs with a wide range of valid values, where values within a partition are expected to have similar behavior. Effective when values near the boundaries of the input domain are more likely to cause issues.
Test Cases Typically, one test case is selected from each equivalence class or partition. Multiple test cases are created to test values at the boundaries, including just below, on, and just above the boundaries.
Coverage Provides broad coverage across input domains, ensuring that different types of inputs are tested. Focuses on testing edge cases and situations where errors often occur.
Example For a password field, you might have equivalence partitions for short passwords, long passwords, and valid-length passwords. In a calculator application, testing inputs at the minimum and maximum limits, as well as values just below and above these limits.
Applicability Useful when you want to identify a representative set of test cases without focusing solely on boundary values. Useful when you want to thoroughly test boundary conditions where errors are more likely to occur.

Both Equivalence Partitioning and Boundary Value Analysis are valuable black-box testing techniques, and the choice depends on the specific characteristics of the input data and where potential issues are expected to arise.