0

I am trying to read youtube chat messages using HtmlUnit

@Test
void test() throws IOException {
    try (final WebClient webClient = new WebClient()) {
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

        final HtmlPage page = webClient.getPage("https://www.youtube.com/live_chat?v=" + id);
        webClient.waitForBackgroundJavaScriptStartingBefore(2000);


        DomNodeList<DomElement> elementsByTagName = page.getElementsByTagName("yt-live-chat-text-message-renderer");

        System.out.println(elementsByTagName.size());
    }
}

String id is a youtube livestream containing (!) chat messages

Running this will never return any elements with the tag yt-live-chat-text-message-renderer. I have tried various wait() methods without any success. I have noticed that in the browser the page seems to load twice and only at the end starts displaying the chat.

isADon
  • 3,433
  • 11
  • 35
  • 49
  • Any log output? – RBRi Mar 13 '19 at 19:33
  • No error but hundreds of other warning and info output as is normal with htmlUnit. I am pretty sure the page is stuck at the initial opening page and never loads any of the chat elements. – isADon Mar 13 '19 at 20:14

0 Answers0