Dfyjkt

I don't understand why the list I'm trying to extract the text from is returning blanks when I'm definitely using the correct Xpath. Here is my code:

driver = webdriver.Firefox()
driver.get("https://www.omegawatches.com/watch-omega-specialities-first-omega-wrist-chronograph-51652483004001")
betweenLugs = driver.find_elements(By.XPATH, "/html/body/div[2]/main/div[3]/div/div/div[2]/div/div[2]/div[3]/div/ul/li[1]")])
print(betweenLugs.text)

This should grab the first list item and measurement

Between lugs: 20 mm 

I have also tried other methods, but the fact that Xpath doesn't pick it up tells me something is wrong and it doesn't matter how I do it, I won't be able to extract the text inside the lists. Does anyone know what am I doing wrong? This is the first time I've ran into this problem.

OK, try this and see if it solves the problem:

between_lugs = driver.find_element_by_xpath("//*[contains(text(), 'Between lugs')]").get_attribute("innerHTML")
between_lugs_value = driver.find_element_by_xpath("//*[contains(text(), 'Between lugs')]/../span").get_attribute("innerHTML")

final_text = between_lugs + " " + between_lugs_value

The xpath is wrong. It fails in /div[2], it doesn't match anything. This is an example why you shouldn't use absolute path.

The section has id attribute, use it

betweenLugs = driver.find_elements(By.XPATH, "//*[@id='product-info-data-5bea7fa7406d7']/ul/li[1]")[0]

You might also want to add some wait for the loading

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions

betweenLugs = WebDriverWait(driver, 10).until(expected_conditions.visibility_of_element_located((By.XPATH, "//*[@id='product-info-data-5bea7fa7406d7']/ul/li[1]")))

That page already has jQuery on it so you can just:

driver.execute_script("return jQuery('li:contains(Between lugs)').text().trim().replace(/s+/g, ' ')")

You can fiddle with selectors in the chrome selectors, it makes it much easier.

Another simpler approach might be the following one:

from contextlib import closing
from selenium import webdriver
from selenium.webdriver.support import ui

url = "https://www.omegawatches.com/watch-omega-specialities-first-omega-wrist-chronograph-51652483004001"

with closing(webdriver.Chrome()) as wd:
 wait = ui.WebDriverWait(wd, 10)
 wd.get(url)
 item = wait.until(lambda wd: wd.find_element_by_xpath("//*[contains(@class,'technical-data')]//li")).get_attribute('textContent')
 print(' '.join(item.split()))

Output:

Between lugs: 20 mm

Using a scroll down and a wait with a css selector to target the parent li

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions

driver = webdriver.Chrome() #Firefox()
driver.get("https://www.omegawatches.com/watch-omega-specialities-first-omega-wrist-chronograph-51652483004001")
driver.execute_script("window.scrollTo(0, 2000)") 
betweenLugs = WebDriverWait(driver, 10).until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, "#product-info-data-5beaf5497d916 > ul > li:nth-child(1)")))

print(betweenLugs.text)