Selenium Grid: what it is and how to set it up

What is Selenium Grid?

Selenium Grid is a powerful tool that enhances the efficiency of Selenium test automation by allowing tests to be executed in parallel across multiple machines and web browsers. It acts as a test execution environment where tests can be distributed and run on various Selenium Grid Nodes simultaneously.

What are the benefits of using Selenium Grid?

Its distributed testing capability makes Selenium Grid an invaluable resource for reducing test execution time and achieving faster feedback in the development cycle.

1. Reduced test execution time: with parallel test execution, Selenium Grid significantly reduces the time required to execute test suites, as multiple tests run concurrently on different Nodes.
2. Improved test coverage: Selenium Grid enables testing on various browser and operating system combinations, ensuring better test coverage and identifying cross-browser compatibility issues.
3. Cost-effective: by leveraging existing infrastructure and reusing test scripts, Selenium Grid helps optimize resource utilization, making it cost-effective for large-scale test automation.
4. Efficient test feedback: faster test execution and parallelization provide quicker feedback to developers, enabling them to identify and fix issues promptly.
5. Scalability: Selenium Grid's distributed architecture allows easy scaling by adding more Nodes, accommodating increased testing demands as projects grow.

When to use Selenium Grid?

Selenium Grid becomes particularly advantageous in scenarios where its distributed testing capabilities can significantly enhance test automation efficiency and effectiveness.

Large test suites and parallel execution

When dealing with extensive test suites that take a long time to execute sequentially, Selenium Grid can parallelize test execution across multiple Nodes. This dramatically reduces the overall test execution time and provides faster feedback to the development team.

Cross-browser and cross-platform testing

To ensure your web application functions correctly across different browsers and operating systems, Selenium Grid allows you to execute tests on a variety of browser configurations concurrently. This helps identify compatibility issues early in the development process.

Scaling test infrastructure

As your test automation requirements grow, Selenium Grid facilitates horizontal scaling by adding more nodes to the grid. This scalability ensures that your test infrastructure can accommodate increased testing demands without sacrificing execution speed.

Continuous integration (CI) pipelines

In CI/CD pipelines, where frequent code changes trigger automated testing, Selenium Grid's parallel execution capability becomes indispensable. It allows you to execute multiple tests simultaneously on various Nodes, speeding up the testing process and ensuring rapid feedback.

Geographically distributed testing

When your application caters to users from different regions, Selenium Grid can set up Nodes on geographically distributed machines. This approach allows you to verify the application's functionality and performance in different network conditions and locations.

Example scenario: e-commerce website testing

Consider an e-commerce website with a large number of test cases to validate its functionalities. Without Selenium Grid, running these tests sequentially could take hours. However, by leveraging Selenium Grid, we can divide the test suite across multiple Nodes, each capable of testing on different browser and OS combinations. As a result, we can significantly reduce test execution time, enabling faster feedback for developers and testers.

Selenium Grid architecture

Selenium Grid's architecture is designed to facilitate parallel test execution across multiple Nodes, enabling efficient distribution of test cases and providing faster results. The architecture consists of two main components: the Hub and the Nodes.

The Hub

The Selenium Grid Hub serves as the central control point for test execution. It receives test requests from clients (test scripts) and manages the distribution of these tests to available Nodes. The Hub acts as a mediator between clients and Nodes.

Nodes

Nodes are the execution environments where tests run. Each Node registers itself with the Hub, indicating its availability for test execution. Nodes can be configured with various browser and OS combinations, offering a diverse testing environment.

Communication flow

1. A test script (client) requests a new session from the Hub by specifying the desired browser and platform (e.g., Chrome on Windows).
2. The Hub examines its registry of available Nodes and forwards the test request to an appropriate Node capable of fulfilling the desired capabilities.
3. The selected Node launches the specified browser with the desired configuration and establishes a new WebDriver session.
4. The test script communicates with the browser via the WebDriver session on the Node for test execution.
5. Test results and status are reported back to the Hub, which forwards them to the client.

Load balancing

Selenium Grid employs a load balancing mechanism to ensure efficient utilization of available Nodes. When multiple Nodes with similar desired capabilities are present, the Hub distributes test requests across these Nodes, optimizing resource usage and reducing test execution time.

Handling failures

Selenium Grid provides robust mechanisms to handle Node failures gracefully. If a Node becomes unresponsive during test execution, the Hub reassigns the affected test cases to other available Nodes, ensuring that the overall test suite continues running smoothly.

Web browser drivers

Each Node in Selenium Grid must have the corresponding web browser driver installed. For example, if a Node is configured to run tests on Chrome, it should have the ChromeDriver installed and properly configured.

How to set up Selenium Grid

Prerequisites

• Java 11 or higher installed (download link)
• Browser(s) installed (e.g., Chromium, Firefox, Safari…)
• Browser drivers (e.g., ChromeDriver, GeckoDriver)
• Add the browsers’ location to the system PATH or place them in a directory accessible to the Python scripts.
• Download the Selenium Server jar file from the latest release (at the time of writing of this article, the latest release was Selenium Server version 4.10.0)

Start Selenium Grid Hub

Open a terminal or command prompt and run the following command to start the Selenium Grid Hub, making sure to start the grid from the same folder where the jar file is located:

java -jar selenium-server-<version>.jar hub

Once the command finishes running, we will receive a message indicating that the Hub has been successfully started:

To ensure that everything worked as intended, visit the local URL where the Selenium Grid Hub started. Since we have not registered any Nodes yet, you should see a screen similar to the one below:

Add Nodes to the Hub

At startup, the Node scans the System PATH to identify and make use of the available drivers. This allows the Node to access the necessary browser drivers for test execution. Note that the provided command assumes that both the Node and the Hub are running on the same machine.

java -jar selenium-server-<version>.jar node

You can register multiple Nodes with different desired capabilities to test on various browser configurations. For example, we can register an additional Node with the —port 6666

java -jar selenium-server-4.10.0.jar node --port 6666

Verify Hub and Nodes

To verify what Nodes we have open on our Hub, we can open a web browser and navigate to http://localhost:4444/grid/ui. This will display the Selenium Grid console, showing the registered Hub and Nodes.

Running tests using Selenium Grid

Now, let's create a simple test script in Python to demonstrate how to run tests using Selenium Grid. In this example, we will use the Selenium WebDriver with Python to open Apify Store and extract the text content of its title, and description on multiple browsers simultaneously.

from selenium import webdriver
from selenium.webdriver.common.by import By

# Define the URL for the Selenium Grid hub
hub_url = '<http://192.168.1.221:4444>'

# Create browser options for Chrome
chrome_options = webdriver.ChromeOptions()

# Create browser options for Firefox
firefox_options = webdriver.FirefoxOptions()

# Connect to the Selenium Grid hub and create a remote WebDriver instance

# Chrome
driver_chrome = webdriver.Remote(
    command_executor=hub_url,
    options=chrome_options
)

driver_chrome.get("<https://apify.com/store>")

chrome_data = {
    "page_title": driver.find_element(By.CSS_SELECTOR, "header > div > h1").text,
    "page_description": driver.find_element(By.CSS_SELECTOR, "header > div > p").text
}

print(chrome_data)

# Firefox
driver_firefox = webdriver.Remote(
    command_executor=hub_url,
    options=firefox_options
)

driver_firefox.get("<https://apify.com/store>")

firefox_data = {
    "page_title": driver.find_element(By.CSS_SELECTOR, "header > div > h1").text,
    "page_description": driver.find_element(By.CSS_SELECTOR, "header > div > p").text
}

print(firefox_data)

driver.quit()

The code snippet above demonstrates a test scenario where the page title and description are extracted from the Apify Store website using both Chrome and Firefox browsers.

With this code, we can run the same test on different browsers concurrently by creating separate WebDriver instances for each browser and connecting them to our Selenium Grid Hub.

Handling test failures

In distributed testing scenarios, it is crucial to handle test failures efficiently. If a Node becomes unresponsive during test execution, the Hub automatically reassigns the affected test cases to other available Nodes, ensuring the overall test suite continues running smoothly.

Parallel test execution tips

• Organize test cases in a way that allows for easy parallelization and avoids dependencies between tests.
• Divide your test suite into smaller chunks to distribute the load evenly across Nodes.
• Consider setting up a dedicated test infrastructure for Selenium Grid to ensure stability and optimal performance.

If you're not sure that Selenium is the right testing framework for you, check out this detailed post on Cypress vs. Selenium. Or find out whether Selenium is the best choice for web scraping in Playwright vs. Selenium.

Learn more about Selenium

🔖 Selenium documentation
🔖 Playwright vs. Selenium: which one to use for web scraping?
🔖 Puppeteer vs. Selenium for automation
🔖 Cypress vs. Selenium: choosing the right web testing and automation framework
🔖 Selenium page object model: what is POM and how can you use it?
🔖 Selenium Webdriver: how to handle popups
🔖 Selenium Webdriver: how to handle iframes
🔖 Web scraping with Selenium