Chapter 3: TestNG Test Configuration

TestNG XML Configuration

TestNG provides a powerful XML-based configuration system that allows you to organize and control test execution without modifying your test code. This separation of test logic from test configuration makes your tests more maintainable and flexible.

Basic XML Structure

A TestNG XML file typically has the following structure:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="TestSuite">
    <test name="TestModule1">
        <classes>
            <class name="com.example.TestClass1"/>
            <class name="com.example.TestClass2"/>
        </classes>
    </test>
    <test name="TestModule2">
        <classes>
            <class name="com.example.TestClass3"/>
        </classes>
    </test>
</suite>

This XML file defines a test suite named "TestSuite" with two test modules, each containing specific test classes.

Running Tests with XML Configuration

You can run tests using the XML configuration file in several ways:

1. Using Maven:

   mvn test -DsuiteXmlFile=testng.xml

2. Using TestNG directly:

   java -cp "path/to/classpath" org.testng.TestNG testng.xml

3. Using an IDE like IntelliJ IDEA or Eclipse, which provide built-in support for running TestNG XML files.

Test Groups and Suites

TestNG allows you to organize tests into groups and suites, providing more flexibility in test execution.

Defining and Running Test Groups

You can define test groups using the groups attribute in the @Test annotation:

import org.testng.annotations.Test;

public class GroupsExample {

    @Test(groups = {"smoke"})
    public void smokeTest1() {
        // Test logic
    }

    @Test(groups = {"regression"})
    public void regressionTest1() {
        // Test logic
    }

    @Test(groups = {"smoke", "regression"})
    public void bothTest() {
        // Test logic
    }
}

To run specific groups, you can configure your TestNG XML file:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="GroupsSuite">
    <test name="SmokeTests">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.GroupsExample"/>
        </classes>
    </test>
</suite>

Organizing Tests into Suites

TestNG allows you to organize tests into suites, which can be useful for running different types of tests separately:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="MasterSuite">
    <suite-files>
        <suite-file path="smoke-tests.xml"/>
        <suite-file path="regression-tests.xml"/>
        <suite-file path="performance-tests.xml"/>
    </suite-files>
</suite>

This approach allows you to maintain separate XML files for different test categories and combine them as needed.

Test Dependencies

TestNG provides mechanisms to define dependencies between tests, ensuring they run in the correct order and handling failures appropriately.

Method Dependencies

You can specify that a test method depends on other test methods using the dependsOnMethods attribute:

import org.testng.annotations.Test;

public class DependencyExample {

    @Test
    public void createUser() {
        // Test user creation
        System.out.println("User created");
    }

    @Test(dependsOnMethods = {"createUser"})
    public void updateUser() {
        // Test user update
        System.out.println("User updated");
    }

    @Test(dependsOnMethods = {"updateUser"})
    public void deleteUser() {
        // Test user deletion
        System.out.println("User deleted");
    }
}

In this example, updateUser will only run if createUser passes, and deleteUser will only run if updateUser passes.

Group Dependencies

You can also specify dependencies between groups using the dependsOnGroups attribute:

import org.testng.annotations.Test;

public class GroupDependencyExample {

    @Test(groups = {"setup"})
    public void setupEnvironment() {
        // Set up test environment
        System.out.println("Environment set up");
    }

    @Test(groups = {"data"}, dependsOnGroups = {"setup"})
    public void createTestData() {
        // Create test data
        System.out.println("Test data created");
    }

    @Test(dependsOnGroups = {"data"})
    public void runTest() {
        // Run the actual test
        System.out.println("Test executed");
    }
}

Handling Dependency Failures

By default, if a test method that others depend on fails, the dependent methods are skipped. You can change this behavior using the alwaysRun attribute:

import org.testng.Assert;
import org.testng.annotations.Test;

public class AlwaysRunExample {

    @Test
    public void setupTest() {
        // This test will fail
        System.out.println("Setup test running");
        Assert.fail("Deliberate failure");
    }

    @Test(dependsOnMethods = {"setupTest"}, alwaysRun = true)
    public void cleanupTest() {
        // This test will run even if setupTest fails
        System.out.println("Cleanup test running");
    }
}

Test Priorities and Execution Order

TestNG allows you to control the order in which tests are executed using priorities.

Setting Test Priorities

You can set the execution priority of test methods using the priority attribute:

import org.testng.annotations.Test;

public class PriorityExample {

    @Test(priority = 1)
    public void testA() {
        System.out.println("Test A running");
    }

    @Test(priority = 2)
    public void testB() {
        System.out.println("Test B running");
    }

    @Test(priority = 0) // Lowest priority, runs first
    public void testC() {
        System.out.println("Test C running");
    }
}

In this example, the tests will run in the order: testC, testA, testB.

Combining Priorities and Dependencies

You can combine priorities with dependencies to have fine-grained control over test execution:

import org.testng.annotations.Test;

public class PriorityAndDependencyExample {

    @Test(priority = 1)
    public void testA() {
        System.out.println("Test A running");
    }

    @Test(priority = 2, dependsOnMethods = {"testA"})
    public void testB() {
        System.out.println("Test B running");
    }

    @Test(priority = 0) // Runs first due to priority
    public void testC() {
        System.out.println("Test C running");
    }

    @Test(priority = 3, dependsOnMethods = {"testB"})
    public void testD() {
        System.out.println("Test D running");
    }
}

Parallel Execution

TestNG supports parallel execution of tests, which can significantly reduce test execution time.

Configuring Parallel Execution

You can configure parallel execution in the TestNG XML file:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ParallelSuite" parallel="methods" thread-count="5">
    <test name="ParallelTest">
        <classes>
            <class name="com.example.ParallelTestClass"/>
        </classes>
    </test>
</suite>

The parallel attribute can have the following values:

• methods: Run test methods in parallel
• classes: Run test classes in parallel
• tests: Run <test> tags in parallel
• instances: Run the same test method in different instances in parallel

Thread Count Configuration

The thread-count attribute specifies the number of threads to use for parallel execution:

<suite name="ParallelSuite" parallel="methods" thread-count="10">
    <!-- Test configuration -->
</suite>

Data Provider Parallel Execution

You can also run data provider invocations in parallel:

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class ParallelDataProviderExample {

    @DataProvider(name = "testData", parallel = true)
    public Object[][] createData() {
        return new Object[][] {
            {"Test 1", 1},
            {"Test 2", 2},
            {"Test 3", 3},
            {"Test 4", 4}
        };
    }

    @Test(dataProvider = "testData")
    public void testMethod(String name, int value) {
        System.out.println("Running " + name + " with value " + value +
                           " on thread " + Thread.currentThread().getId());
        // Test logic
    }
}

Java 21 Virtual Threads for Parallel Execution

Java 21 introduces virtual threads, which are lightweight threads that can significantly improve the performance of parallel test execution. Let's see how to leverage virtual threads with TestNG.

Custom Executor for Virtual Threads

You can create a custom executor service using virtual threads and integrate it with TestNG:

import org.testng.IExecutor;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.internal.IConfiguration;
import org.testng.internal.ITestResultNotifier;

import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class VirtualThreadExecutor implements IExecutor {

    private ExecutorService executorService;

    public VirtualThreadExecutor() {
        // Create an executor service using virtual threads
        this.executorService = Executors.newVirtualThreadPerTaskExecutor();
    }

    @Override
    public List<ITestResult> execute(List<ITestNGMethod> methods, ITestResultNotifier notifier,
                                    IConfiguration configuration) {
        // Implementation of test execution using virtual threads
        // This is a simplified example
        for (ITestNGMethod method : methods) {
            executorService.submit(() -> {
                // Execute the test method
                // Report results to notifier
            });
        }

        // Return test results
        return null; // Actual implementation would return real results
    }

    @Override
    public void shutdown() {
        executorService.shutdown();
    }
}

Using Virtual Threads for Test Setup

You can use virtual threads to perform parallel setup operations:

import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.ArrayList;
import java.util.List;

public class VirtualThreadSetupExample {

    private List<String> setupResults = new ArrayList<>();

    @BeforeClass
    public void parallelSetup() throws Exception {
        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
            List<Future<String>> futures = new ArrayList<>();

            // Submit multiple setup tasks to run concurrently
            futures.add(executor.submit(() -> {
                // Setup database
                Thread.sleep(1000); // Simulate work
                return "Database setup complete";
            }));

            futures.add(executor.submit(() -> {
                // Setup test data
                Thread.sleep(1000); // Simulate work
                return "Test data setup complete";
            }));

            futures.add(executor.submit(() -> {
                // Setup external services
                Thread.sleep(1000); // Simulate work
                return "External services setup complete";
            }));

            // Collect all results
            for (Future<String> future : futures) {
                setupResults.add(future.get());
            }
        }

        System.out.println("All setup tasks completed concurrently");
    }

    @Test
    public void testAfterSetup() {
        System.out.println("Setup results: " + setupResults);
        // Test logic using the setup results
    }
}

Timeouts and Performance

TestNG allows you to set timeouts for test methods, which can help identify performance issues.

Setting Timeouts

You can set a timeout for a test method using the timeOut attribute:

import org.testng.annotations.Test;

public class TimeoutExample {

    @Test(timeOut = 1000) // Timeout in milliseconds
    public void fastTest() {
        // This test should complete within 1 second
        System.out.println("Fast test running");
    }

    @Test(timeOut = 5000)
    public void slowTest() throws InterruptedException {
        // This test has up to 5 seconds to complete
        System.out.println("Slow test running");
        Thread.sleep(3000); // Simulate work
    }

    @Test(timeOut = 2000, expectedExceptions = {TimeoutException.class})
    public void timeoutTest() throws InterruptedException {
        // This test will timeout and throw an exception
        System.out.println("Timeout test running");
        Thread.sleep(3000); // Will cause timeout
    }
}

Measuring Test Performance

You can use TestNG listeners to measure and report test performance:

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(PerformanceListener.class)
public class PerformanceExample {

    @Test
    public void testMethod1() throws InterruptedException {
        // Test logic
        Thread.sleep(100); // Simulate work
    }

    @Test
    public void testMethod2() throws InterruptedException {
        // Test logic
        Thread.sleep(200); // Simulate work
    }
}

class PerformanceListener implements IInvokedMethodListener {

    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
        testResult.setAttribute("startTime", System.currentTimeMillis());
    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
        long startTime = (Long) testResult.getAttribute("startTime");
        long endTime = System.currentTimeMillis();
        long duration = endTime - startTime;

        System.out.println("Method " + method.getTestMethod().getMethodName() +
                           " took " + duration + " ms");
    }
}