1

I am trying the below code in webharvest

 <var-def name="url">https://www.athome.com/on/demandware.store/Sites-athome-Site/default/Stores-FindByZip?</var-def>

<var-def name="urlResult">
 <html-to-xml>
 <http url="${url}" method="GET"  >
     <http-param name="Content-Type">text/html;charset=UTF-8</http-param>   
    <http-param name="dwfrm_storelocator_countryCode">US</http-param>
    <http-param name="dwfrm_storelocator_distanceUnit">mi</http-param>
    <http-param name="dwfrm_storelocator_postalCode">07054</http-param>
    <http-param name="dwfrm_storelocator_maxdistance">999999</http-param>
    <http-param name="dwfrm_storelocator_findbyzip">Search</http-param>
</http> 
</html-to-xml>
</var-def>

I get the error stating There was an error processing your request.

Screenshot of the error is Error Screenshot

But when i define the url with all the parameters, then i get the intended result

 <var-def name="url"><template>https://www.athome.com/on/demandware.store /Sites-athome-Site/default/Stores-FindByZip?dwfrm_storelocator_countryCode=US&amp;dwfrm_storelocator_distanceUnit=mi&amp;dwfrm_storelocator_postalCode=07054&amp;dwfrm_storelocator_maxdistance=999999&amp;dwfrm_storelocator_findbyzip=Search</template></var-def>

<var-def name="urlResult">
     <html-to-xml>
     <http url="${url}" method="GET">
        </http> 
    </html-to-xml>
    </var-def>

Why we have this kind of differing behavior ?

Thank you !

Here s Toimprove
  • 11
  • 3
  • I used Web-Harvest and can execute your first config file successfully. Can you attach the error screenshot? – shaochuancs Sep 05 '17 at 06:24
  • Hi Shaochuancs, i edited my question to include the variable definition in the first case and below is the snippet of the response

    An Error Occurred

    There was an error procesing your request.  You can try searching below or click hereto return to the Home Page.
     – Here s Toimprove Sep 05 '17 at 07:25
  • Only in the second case where i have all the parameter in the url itself then i get the whole listing in the response – Here s Toimprove Sep 05 '17 at 07:28
  • I can still execute your first case successfully. Can you attach the error screenshot picture? It's hard to read code in comment. – shaochuancs Sep 05 '17 at 07:31
  • @shaochuancs , i have edited the post and included the error screenshot for the first time run with just the base url and other parameters defined using the http-param – Here s Toimprove Sep 05 '17 at 07:44
  • I tried again and can successfully scrape the web page using your 1st case. – shaochuancs Sep 05 '17 at 07:59
  • @shaochuancs can you share your config as i fail to understand why the same command work so differently for two users ? i will try using your config and see if i can receive a successful message. Thanks. – Here s Toimprove Sep 13 '17 at 02:39
  • Actually, my config is exactly the same as yours. The one below "I am trying the below code in webharvest" in the question. – shaochuancs Sep 13 '17 at 02:43

0 Answers0