1

I need to scrape price of certain listed food items basis different locations in the country. There's an input text box that allows me to enter the name of the city & pressing "Enter" shows me the list of items available in that city.

Here's how I am trying to automate this:

driver.get("https://grofers.com/")
ele = driver.find_element_by_xpath("//input[@data-test-id='area-input-box']")`
ele.send_keys(area)
ele.send_keys(Keys.RETURN)

Here's the HTML I'm working with:

 

   <div style="margin-left: 51px; height: 36px;">
   <div style="display: flex; height: 100%;">
      <button class="btn location-box mask-button">Detect my location</button>
      <div class="oval-container">
         <div class="oval">
            <span class="separator-text">
               <div class="or">OR</div>
            </span>
         </div>
      </div>
      <div style="width: 220px;">
         <div class="modal-right__input-wrapper">
            <div class="display--table full-width">
               <div class="display--table-cell full-width">
                  <div id="map-canvas"></div>
                  <div class="Select location-search-input-v1 is-searchable Select--single">
                     <div class="Select-control">
                        <div class="Select-multi-value-wrapper" id="react-select-2--value">
                           <div class="Select-placeholder">Type your city Society/Colony/Area</div>
                           <div class="Select-input" style="display: inline-block;">**<input data-test-id="area-input-box" aria-activedescendant="react-select-2--value" aria-expanded="false" aria-haspopup="false" aria-owns="" role="combobox" value="">**</div>
                        </div>
                        <span class="Select-arrow-zone"><span class="Select-arrow"></span></span>
                     </div>
                  </div>
               </div>
            </div>
         </div>
      </div>
   </div>

The problem is - after send_keys, the website takes time to autofill the input box AFTER WHICH I need to press enter.

I tried using time.sleep(2) after send_keys but this leads to pop-up disappearing & a StaleElementException when I do Keys.RETURN.

Have been stuck on this for quite some time now. Any help/pointers would be appreciated.

DaWanderer
  • 75
  • 7
  • You can read about it in the [documentation](https://selenium-python.readthedocs.io/waits.html). – Ananth Jan 26 '21 at 17:05
  • Please [edit] your question to include a [repro], which for selenium questions requires both enough python code and HTML code to answer the question. – Greg Burghardt Jan 26 '21 at 17:17
  • @GregBurghardt Added HTML & the popup is the first thing that appears (asking for the city). When I manually enter say, "Mumbai", the list auto populates and I'm able to select the city by pressing "Enter". However, send_keys("Mumbai") followed by send_keys(keys.RETURN) doesn't work. – DaWanderer Jan 27 '21 at 16:13
  • Still not able to get a working solution. If anyone could help please? – DaWanderer Jan 30 '21 at 09:10

2 Answers2

0

Selenium actually has an article on this with Explicit and Implicit waits, I think this is the one you're looking for:

# Wait until an element with id='myNewInput' has class 'myCSSClass'
wait = WebDriverWait(driver, 10)
element = wait.until(element_has_css_class((By.ID, 'myNewInput'), "myCSSClass"))

https://selenium-python.readthedocs.io/waits.html That's the article

Salad
  • 36
  • 6
  • Thanks but somehow I get a timeout using this approach as well. Almost seems like automation doesn't trigger the autofill at all. I've edited the question to give you a better idea. – DaWanderer Jan 27 '21 at 16:21
0

You can also create custom wait conditions when none of the previous convenience methods fit your requirements. A custom wait condition can be created using a class with call method which returns False when the condition doesn’t match.

class element_has_css_class(object):
  """An expectation for checking that an element has a particular css class.

  locator - used to find the element
  returns the WebElement once it has the particular css class
  """
  def __init__(self, locator, css_class):
    self.locator = locator
    self.css_class = css_class

  def __call__(self, driver):
    element = driver.find_element(*self.locator)   # Finding the referenced element
    if self.css_class in element.get_attribute("class"):
        return element
    else:
        return False

# Wait until an element with id='myNewInput' has class 'myCSSClass'
wait = WebDriverWait(driver, 10)
element = wait.until(element_has_css_class((By.ID, 'myNewInput'), "myCSSClass"))
Anshuman
  • 11
  • 3