0

I'm trying to find all broken links on a webpage with Selenium for Java. I tried it with a simple tutorial and it seems to work on very basic websites but in my case, the DOM seems to change and I often get a StaleElementReferenceException.

I'm not quite sure how I can prevent getting that Exception or how to handle it. In the case of such an exception, I basically want to skip that URL and keep on going with my loop. But since ignoring exceptions is a really bad practice I want to ask you guys how I can change my code to handle the exception well.

public class BrokenLinkCollector {
    WebDriver driver;
    String webPage;
    public BrokenLinkCollector(WebDriver driver, String webPage) {
        this.driver = driver;
        this.webPage = webPage;
    }
    public void giveMeAllTheBrokenLinks(){
        String url = "";
        HttpURLConnection huc = null;
        int respCode = 200;

        driver.get(webPage);

        List<WebElement> links = driver.findElements(By.tagName("a"));

        Iterator<WebElement> it = links.iterator();

        while(it.hasNext()){

            WebElement we = it.next();
            url = we.getAttribute("href");

            if(url == null || url.isEmpty()){

                System.out.println("URL is either not configured for anchor tag or it is empty");
                continue;
            }

            try {
                huc = (HttpURLConnection)(new URL(url).openConnection());

                huc.setRequestMethod("HEAD");

                huc.connect();

                respCode = huc.getResponseCode();

                if(respCode >= 400){
                    System.out.println(url+" is broken");
                }
                else{
                    System.out.println(url+" works");
                }

            } catch (MalformedURLException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }

    }
}
questionasker
  • 2,536
  • 12
  • 55
  • 119
cepi1993
  • 35
  • 5
  • https://stackoverflow.com/questions/23414150/how-to-find-broken-links-using-selenium-webdriver-with-java This link will help you to solve – Justin Lambert Jun 22 '20 at 12:14
  • @JustinLambert those examples don't handle the exception I am speaking about – cepi1993 Jun 22 '20 at 12:35
  • What line is returning the stale element? – Jortega Jun 22 '20 at 12:45
  • url = we.getAttribute("href"); - The problem is that the DOM changes and some elements from the links-list are stale when I read them a few milliseconds later in that line. This doesn't happen everytime. – cepi1993 Jun 22 '20 at 13:21

0 Answers0