In playwright, locating elements that are lazy loaded in HTML

Question

I have an interesting element to locate in playwright.

There is a list of rows that comes up when I visit a URL. The number of records that comes from an API called on that page is say 400 but only first 100 rows from those records are shown in the page. Also there are empty divs created in HTML, like for example in this case since total records were 400 so code will create say 4 empty divs and fill the first div with 100 row elements and rest of the 3 divs will be empty. When user scrolls then following happens -

A. Initially only 1-100 records are shown in the first div. Please note that the other records are not even there in the HTML.

B. When user scrolls till the point the 101th element just enters the viewport, then the elements from 101 till 200 gets populated and displayed in second div.

C. When user scroll such that the last element of the first set i.e. 100th element goes beyond the top of the viewport then the first div becomes empty. So now the first div is empty and second one has 100 elements from 101-200 and rest of the 2 divs are empty.

And so on D, E, F and G...See below pattern for better understanding :

A.

<div class="gridRowContainer">
 <div>100 elements here(1-100)...</div>
 <div></div>
 <div></div>
 <div></div>
</div>

B.

<div class="gridRowContainer">
 <div>100 elements here(1-100)...</div>
 <div>100 elements here(101-200)...</div>
 <div></div>
 <div></div>
</div>  

C. elements from 1-100 are NOT visible because they just passed the top of viewport

<div class="gridRowContainer">
 <div></div>
 <div>100 elements here(101-200)...</div>
 <div></div>
 <div></div>
</div> 

D.

<div class="gridRowContainer">
 <div></div>
 <div>100 elements here(101-200)...</div>
 <div>100 elements here(201-300)...</div>
 <div></div>
</div>

E. elements from 101-200 are NOT visible because they just passed the top of viewport

 <div class="gridRowContainer">
   <div></div>
   <div></div>
   <div>100 elements here(201-300)...</div>
   <div></div>
 </div>

F.

 <div class="gridRowContainer">
   <div></div>
   <div></div>
   <div>100 elements here(201-300)...</div>
   <div>100 elements here(301-400)...</div>
 </div>

G. elements from 201-300 are NOT visible because they just passed the top of viewport

 <div class="gridRowContainer">
   <div></div>
   <div></div>
   <div></div>
   <div>100 elements here(301-400)...</div>
 </div>

. . .

What I want to do : Each element that are displayed inside each of these 4 divs looks like this <div>Some random text</div> except for one div <div>Target text</div>. I want to extract the text from this div.

Currently I have a locator that successfully extract this text by getByText() method. It is able to do that because this element comes in the list of first 100 elements. However if this element was present somewhere in the list starting from 101-200 or 201-300 or 301-400, then it won't be visible when we land on the page unless we scroll past the point where we display the div that has the target element.

What I want to know: I want an approach that could help me tackle this problem where I am able to locate the required target element no matter in which div it lies.

Answer 1

How about keep on scrolling till you find your target:

 const sizes = await page.evaluate(() => {
    const browserHeight = window.innerHeight;
    const pageHeight = document.body.scrollHeight;
    return { browserHeight, pageHeight };
  });

  for (let i = 0; i < sizes.pageHeight; i += sizes.browserHeight) {
    await page.mouse.wheel(0, i);
    console.log("scrolled to", i);
    await page.waitForTimeout(1000);
    if (await page.getByText("Target text").isVisible()) 
      break;
  }