scrolling down in selenium while more Data loads- Lazy loading

Hi I have written a code to scroll down a page using the following :

JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("window.scrollTo(0, document.body.scrollHeight)");


However, it scrolls only once, while more data loads on my page. The data is enclosed inside a div tag.

When it scrolls down, The page shows loading and loads more data (Lazy loading).
How do i implement scrolling for lazy loading in it?

1 Answer
1

Here example how you can scroll until size of data elements not change. To get loadingLocator open chrome devtools, scroll and press F8, it will pause and you'll able to get selector.

WebDriverWait wait = new WebDriverWait(driver, 5);
JavascriptExecutor js = (JavascriptExecutor) driver;

int dataSize = driver.findElements(dataLocator).size();
while (true)
js.executeScript("window.scrollTo(0, document.body.scrollHeight)");
wait.ignoring(NoSuchElementException.class)
.until(ExpectedConditions.invisibilityOfElementLocated(loadingLocator));

if (driver.findElements(dataLocator).size()== dataSize)
break;
dataSize = driver.findElements(dataLocator).size();



Also you can implement your own wait like below:

By dataLocator = By.cssSelector(".save_ride_container");
WebDriverWait wait = new WebDriverWait(driver, 5, 500);
JavascriptExecutor js = (JavascriptExecutor) driver;

AtomicInteger dataSize = new AtomicInteger(driver.findElements(dataLocator).size());
boolean complete = false;
while (!complete)
js.executeScript("window.scrollTo(0, document.body.scrollHeight)");
complete = wait.ignoring(TimeoutException.class)
.until(d ->
int size = driver.findElements(dataLocator).size();
if (size== dataSize.get())
return true;
dataSize.set(size);
return false;
);


List<WebElement> passengers = driver.findElements(dataLocator);
passengers.forEach(p -> System.out.println(p.findElement(By.xpath("./div[1]/div[3]/p[5]/strong"))));


Thanks for contributing an answer to Stack Overflow!

But avoid …

To learn more, see our tips on writing great answers.

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.