If you're automating the web with Selenium, you're bound to stumble across iframes. These elements add a layer of complexity that demands special attention and techniques. Iframes create isolated environments where standard Selenium commands fall short, making interacting with them a puzzle.
In this tutorial, we’ll explore why these elements defy conventional automation methods and explain how to use Selenium's tools to navigate within iframes.
What is an iframe – and what makes them “special”?
Iframes, short for inline frames, are HTML elements that allow us to embed external content within a parent HTML document, such as web pages or media. Using Selenium to interact with elements on the main web page is relatively simple; however, when working within iframes, it requires specific handling. This is because iframes create isolated environments that can't be accessed directly using normal Selenium commands.
In a typical scenario, if we attempt to interact with elements inside an iframe using regular methods like find_element.(By.ID, ...) or find_element.(By.XPATH, ...), we'll run into a NoSuchElementException. This is because Selenium's default context is the main page, and it cannot directly see elements within iframes.
Identifying and switching frames
To interact with elements inside iframes, we need to switch the context of the WebDriver to the iframe. To do that, we can use Selenium’s switch_to.frame() method.
To illustrate that, let’s write some code and see how we can switch frames without disrupting our automation flow. We’ll be using this simple CodePen page where we have a dropdown button inside an iframe.
Our goal is to click the green button to get the text content of the dropdown menu links.
1. Go to the target page
The first step, as in any automation job, is to go to the page where the iframe is located. Nothing new at this stage, just initialize your driver and go to the target website.
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
# Navigate to the target website
driver.get("https://codepen.io/pervillalva/full/abPoNLd")2. Switching frames
Next, let's target and switch to the iframe we want to interact with. As previously mentioned, we can achieve this using Selenium's switch_to.frame() method.
Selenium provides three approaches to select and switch to the desired frame: utilizing a WebElement, specifying a name or ID, or indicating an index.
The differences between each approach are minimal and the choice of method ultimately depends on personal preference and the specific context of the situation.
In this example, we will use the WebElement approach to select and switch the frame. But, for a quick overview, here is how we could employ each of the different methods in this particular scenario:
- Using WebElement
iframe = driver.find_element(By.CSS_SELECTOR, "#result")
driver.switch_to.frame(iframe)
- Using an index
iframe = driver.find_elements(By.TAG_NAME,'iframe')[0]
driver.switch_to.frame(iframe)
- Using a name or ID
driver.switch_to.frame('result')
If you're interested in taking a deeper look at each method, check out the Selenium documentation for working with iframes. Now, let’s proceed with our example.
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
driver = webdriver.Chrome()
# Navigate to the target website
driver.get("<https://codepen.io/pervillalva/full/abPoNLd>")
# Wait a few seconds for the page to load
sleep(3)
# Print the title of the main frame
main_page_title = driver.find_element(By.CSS_SELECTOR, 'title').get_attribute('innerHTML')
print(main_page_title)
# Switch to the iframe
iframe = driver.find_element(By.CSS_SELECTOR, "#result")
driver.switch_to.frame(iframe)
# Print the title of the page inside the iframe
iframe_page_title = driver.find_element(By.CSS_SELECTOR, 'title').get_attribute('innerHTML')
print(iframe_page_title)
In the script above, we navigate to the target web page and retrieve titles from both the main frame page and the iframe.
Note that to obtain the title of the iframe's page, we had to first shift from the main frame to the iframe. Upon executing the script, two distinct titles will be printed to the terminal, proving that we have successfully switched frames.
3. Interacting with elements inside the iframe
Having successfully switched to the iframe, all that's left to do is use Selenium’s conventional syntax to target the dropdown button, click it, and get the text content of the links enclosed in the menu. Let's modify our earlier script accordingly to accomplish these actions:
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
driver = webdriver.Chrome()
# Navigate to the target website
driver.get("<https://codepen.io/pervillalva/full/abPoNLd>")
# Wait a few seconds for the page to load
sleep(3)
# Switch to the iframe
iframe = driver.find_element(By.CSS_SELECTOR, "#result")
driver.switch_to.frame(iframe)
# Click the dropdown button
driver.find_element(By.CSS_SELECTOR, '.dropbtn').click()
sleep(3)
# Select the links inside the dropdown menu
dropdown_links = driver.find_elements(By.CSS_SELECTOR, '.dropdown-content a');
# Print the link's text
for link in dropdown_links:
print(link.text)
Finally, just run the script and voilà, watch as an automated browser spawns, navigate to the target webpage, switch frames, click the button, and get the text we want.
Bonus: handling nested iframes
Congratulations, you now know how to interact with elements within iframes! However, in specific situations, this process can become more intricate. For instance, dealing with nested iframes can pose challenges if you don’t know what to look for. The fundamental principle remains the same, but navigating the nested layers can be confusing.
To illustrate this, let's explore a different CodePen example. Here, we'll interact with a Wikipedia page embedded within an iframe that resides inside yet another iframe.
Our goal is to correctly switch frames so we can click the dropdown button and extract the text from the links highlighted below.
The most challenging aspect of this specific scenario is recognizing the presence of nested iframes and effectively traversing the DOM tree. Once this is accomplished, the subsequent steps remain consistent with what we've already learned. The key difference lies in the need to switch frames twice, rather than just once.
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
driver = webdriver.Chrome()
# Navigate to the webpage with an iframe
driver.get("https://codepen.io/pervillalva/full/gOQVBWe")
# Wait a few seconds for the page to load
sleep(3)
# Switch to the first iframe
iframe = driver.find_element(By.CSS_SELECTOR,"#result")
driver.switch_to.frame(iframe)
# Switch to the second iframe
iframe2 = driver.find_element(By.CSS_SELECTOR, "body > iframe")
driver.switch_to.frame(iframe2)
# CLick the dropdown menu
driver.find_element(By.CSS_SELECTOR, '#vector-main-menu-dropdown-checkbox').click()
sleep(3)
# Select the links inside the dropdown menu
dropdown_links = driver.find_elements(By.CSS_SELECTOR, '#p-navigation .vector-menu-content-list li a');
# Print the link's text
for link in dropdown_links:
print(link.text)And that’s it, we can now just sit back, relax and watch our script run without any issues 🤖
What’s next after learning about iframes?
Mastering the nuances of handling iframes in Selenium WebDriver is an essential skill to have when implementing complex web automation flows. Now, you've learned the tools Selenium offers to seamlessly navigate within iframes and interact with their elements.
If you've enjoyed this guide on handling iframes with Selenium, you're in for a treat. You can learn more in our comparison between Selenium and Puppeteer. Or, if you're interested in web scraping, check out our tutorial on web scraping with Selenium and Python.
