Handle Dynamic content with Selenium and Jsoup!
Today almost every website has some element of user-interaction and content that changes dynamically based on inputs. This calls for a different approach while designing test automation scripts with Selenium or any other automated testing tool. And the standard tools used to automate web applications generally fall short of meeting these challenges. These dynamic websites require special care when designing test code so as not to break them again and again every time we make a change. There are some simple ways to get started with your first tests, but it’s an ongoing process that requires keeping up with the changes made by designers and developers. Luckily, there are some great libraries out there that can help us conquer these challenges.
Jsoup for Selenium Automation
Jsoup is a Java library for working with real-world HTML. It allows you to select and extract the parts of a web page that you are interested in, as either raw text or XML, and construct new pages from these components. As a library for working with real-world HTML, Jsoup is primarily concerned with solving the problems that you will actually face when dealing with HTML in the wild. JSoup is a lightweight Java library that lets you extract and select data from HTML documents using CSS selectors. In this example we will be using CSS selector method to extend it to be used with Selenium.
Getting Started with Jsoup
Make sure that Jsoup is available on the classpath before you begin. I use Maven, so I have included a Maven dependency in my pom.xml that you will need to include in the dependencies section.
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.15.1</version>
</dependency>For latest version refer to:
mvnrepository.com : https://mvnrepository.com/artifact/org.jsoup/jsoup
Example : Jsoup + Selenium
We will click on a few links on a website without coding for locators (xpath or css selectors) in this example. In real testing scenarios, these would be dynamic text such as a transaction id.
In this example we are opening a page and clicking on links, however you can click on any webelement like span or div as long as you provide a text.
Git repo for you to play
This article discussed how Selenium can be used for testing dynamic websites. It also described how Jsoup can be used as a better alternative for xpath in testing dynamic websites. With the increasing complexity of applications, it’s critical to ensure that your test automation code is designed with challenges in mind. Automated tests can be very useful in making sure that your code works correctly and spots any issues that might be present. However, if the tests are not stable, they can actually cause more problems than they solve. Therefore, it’s important to make sure you are using the right techniques to create your test automation code.
