0

I am writing text content from a div element to a csv file using the ruby csv gem. When I output this div's text content to the console, I get all the text content including that from child divs. When I write to the csv file, it only outputs the text content from 'a' child elements. I need it to also include all the text from a child div and it's 'a' elements. Here is the HTML:

<div class="bodytag" style="padding-bottom:30px; overflow:visible"> 
<h2>Search Results for "serialnum3"</h2>
<div id="results_banner">
Products
<span>Showing 1 to 2 of 2 results</span>
</div>
<div class="pg_dir"></div>
<div class="results_row">
<a href="/products/fuji/" title="Fuji, Inc.">FUJI</a>
<a href="/products/fuji/50mm lens/" title="Fuji, Inc.">50mm lens</a>
<div class="results_subrow">
    <p>more product info</p>
</div>
</div>
<div class="results_row">
<a href="/products/fuji/" title="Fuji, Inc. 2">FUJI</a>
<a href="/products/fuji/50mm lens/" title="Fuji, Inc.">50mm lens</a>
<div class="results_subrow">
    <p>more product info 2</p>  
</div>

Here is my code:

 if
browser.div(:class => "results_row").exists?
search_results = browser.divs(class: "results_row").map(&:text)
csv << search_results
else
csv << %w("no search results")
end

My end goal is to use the csv data as an excel column so if anyone knows an easier way I'd love to hear it! Right now I am successfully using roo to parse an excel column as an array for search terms. I would like to use roo to write the div element's text content in an adjacent column instead of writing to csv but couldn't figure out how to do it with roo.

djangodev
  • 363
  • 5
  • 15
  • I'm not sure I understand the question. The code should already return all of the text. At least it works when I run against the sample HTML - eg `search_results` equals `["FUJI 50mm lens\nmore product info", "FUJI 50mm lens\nmore product info 2"]`. Is there more to the page that would cause the code not to work? – Justin Ko Sep 11 '18 at 19:09
  • Hey @justin-ko I actually just saw your comment on my other question after I posted this and that gave me what I needed to know. I had to include brackets around search_results `search_results = browser.divs(class: "results_row").map(&:text) csv << [search_results]`. For some reason I do not get all the div's text content when writing to csv unless the variable is specified in array format. Before I had `csv << search_results` – djangodev Sep 11 '18 at 19:23
  • That's weird. I would only expect that to change how it gets written to CSV (not what was retrieved from the page). `csv << [search_results]` should write all the text to a single cell, where as `csv << search_results` would have split across multiple columns. – Justin Ko Sep 11 '18 at 19:39
  • 2
    Possible duplicate of [How to write method to csv not just string? Ruby csv gem](https://stackoverflow.com/questions/52280347/how-to-write-method-to-csv-not-just-string-ruby-csv-gem) – engineersmnky Sep 11 '18 at 20:23
  • yeah @JustinKo it does get split across multiple columns without the array, but it only write the first two 'a' tags text content – djangodev Sep 12 '18 at 01:34

0 Answers0