Software Testing - Part 2

Testing Methodologies

1. Testing Techniques

Manual Testing

The first testing we experience when we learn to program is manual testing: try the program and see if it works!

In manual testing, the tester plays the role of a user and check to see if there is any unexpected or undesirable behavior.

Often, manual testers will use a test plan with specified test cases to ensure a thorough exploration of the project.

The tester may or may not be part of the programming team that created the code.

Automated Testing

Automated tests use testing software to control and track one or more automatically executed tests.

Automated tests can be created and configured to run each time a new version of the project is created.

Microsoft Test Manager will provide detailed reporting on the test results for each automated test.

Manual vs. Automated Testing

Manual and automated test cases are complementary, both types of tests are important for ensuring high quality software.

Automation is quick and can test many subtle variations in data, it can also easily repeat tests as software evolves. And because it is executed by a computer, the fatigue and mistakes that sometimes accompany repetitive tasks is negated.

Although manual testing typically takes longer to execute (since it’s conducted by a person), it often requires far less set-up time. It is a good choice for tests that only need to be run occasionally, or cases when cost/time of automation setup would outweigh the benefits.

Black Box Testing

Black box testing is testing conducted without knowledge of the internal workings of the system that is being tested. This type of testing simulates end-user experience.

In general, the tester does not know how the code works, she or he is providing input and examining the output. This person does not necessarily need to know how to program.

Example scenarios for black box testing include:

• Testing that the user interface meets all requirements and is functional
• Testing for a variety of input types (including input outside the expected range, such as entering a negative number for a weight)
• Load or stress testing a system
• Testing the security of a project or system.

White Box Testing

White box testing is conducted by examining code for potential failure scenarios.

White box test cases are created by someone who analyzes the code of the application block and prepares test cases to ensure that the class is behaving in accordance with the specifications.

White box scenarios include:

• Testing internal subroutines that are used "behind the scenes"
• Testing loops and conditional statements for accuracy
• Performance testing of a code path or algorithm

2. Testing Levels

Unit Testing

Unit tests are automated tests that verify functionality at the component, class, method, or property level.

The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect.

Each unit is tested separately before integrating them into components to test the interfaces between units.

Unit tests should be written before (or very soon after) a method is written. Often, developers building the class or method designs the unit test themselves.

Component and Integration Testing

From a testing perspective, individual units are integrated together to form larger components. In its simplest form, two units that have already been tested are combined into an integrated component and the interface between them is tested. This testing is called integration testing (or “component testing”).

Integration testing identifies problems that occur when units are combined. New errors that arise are likely related to the interface between units rather than within the units themselves—this simplifies the task for finding and correcting the defects.

3. Testing Types

Regression Testing

Whenever any changes are made to a project, it is possible that existing code may no longer work properly, or that previously undiscovered bugs will present themselves. This kind of bug is called a regression.

To catch these defects, the entire project must be regression tested: a complete retesting of a modified program, rather than a test of only the modified units, to ensure that no errors have been introduced with the modifications.

Stress Testing

Testing on a small scale, such as a single user running a web application or a database with only a handful of records, may not reveal problems that may occur when the application is used in “real world” conditions.

Stress testing pushes a system’s functional limits. It is performed by subjecting the system to extreme conditions, such as peak volumes of data or a large number of simultaneous users.

These tests are also referred to as load tests, since they test a system under heavy loads.

Test automation allows rigorous stress testing without a minimal amount of manual labor.

Performance Testing

Performance testing determines responsiveness, throughput, reliability, and/or scalability of a system under a given workload.

In web applications, performance testing is often closely related to stress testing, measuring lag and responsiveness under a heavy load.

In other applications (desktop and mobile apps, for example), performance testing measures speed and resource utilization, such as disk space and memory.

Security Testing

Security testing validates an application's security services and identifies potential security flaws.

Many projects use a black box approach to security testing, allowing security experts with no knowledge of the software to probe the application for holes and weaknesses.

Usability Testing

Usability testing evaluates a project by studying how real users actually use the software.

Examples include:

• Measuring how long it takes a user to complete a task
• Tracking how many “clicks” or user actions it takes to complete a task or access a feature.

Localization Testing

Localization translates the product UI and occasionally changes some initial settings to make it suitable for another region.

Accessibility Testing

Accessibility testing validates an application’s support for users with disabilities.

Accessibility testing may include:

• Compliance: Does it comply with legal requirements regarding accessibility?
• Effectiveness: Can users with disabilities use the application?
• Usefulness: Does the application expose adequate functionality for users with disabilities?
• Satisfaction: How is the application perceived by users with disabilities?
• Accessibility testing may include usability tests with disables users and assistive technology devices.