Replace markup (as a string) including certain inline elements

Question

My intent is to modify a sentence within a tag.

For example change:

<div id="1">
  This is text in the TD with <strong> strong </strong> tags
  <p>This is a child node. with <b> bold </b> tags</p>
  <div id=2>
      "another line of text to a <a href="link.html"> link </a>"
     <p> This is text inside a div <em>inside<em> another div inside a paragraph tag</p>
  </div>
</div>

To this:

<div id="1">
  This is modified text in the TD with <strong> strong </strong> tags
  <p>This is a child node. with <b> bold </b> tags</p>
  <div id=2>
      "another line of text to a <a href="link.html"> link </a>"
      <p> This is text inside a div <em>inside<em> another div inside a paragraph tag</p>
   </div>
</div>

Which would mean I need to traverse the nodes grabbing a tag and getting all the text & style nodes, but not grabbing the children tags. Modifying the sentences and putting them back. I would need to do this for each tag with full text until all the content was modified.

For example grabbing the text and style nodes for div#1 would be: "This is text in the TD with strong tags" but as you can see, none of the other text underneath would be grabbed. It should be accessible and modifiable through a variable.

div#1.text_with_formating= "This is modified text in the TD with <strong> strong </strong> tags"

The below code removes all content, not just the children tags, keeping content leaves all content even the tags under div#1. Therefore, I'm not sure how to proceed.

Sanitize.clean(h,{:elements => %w[b em i strong u],:remove_contents=>'true'})

How would you recommend solving this?

Answer 1

If you want to find all the text nodes underneath an element, use:

text_pieces = div.xpath('.//text()')

If you want to find only the text that is an immediate child of an element, use:

text_pieces = div.xpath('text()')

For each text node, you can change the content any way you like. You must, however, just be sure you use my_text_node.content = ... instead of my_text_node.content.gsub!(...).

# Replace text that is a direct child of an element
def gsub_my_text!( el, find, replace=nil, &block )
    el.xpath('text()').each do |text|
        next if text.content.strip.empty?
        text.content = replace ? text.content.gsub(find,replace,&block) : text.content.gsub(find,&block)
    end
end

# Replace text beneath an element.
def gsub_text!( el, find, replace=nil, &block )
    el.xpath('.//text()').each do |text|
        next if text.content.strip.empty?
        text.content = replace ? text.content.gsub(find,replace,&block) : text.content.gsub(find,&block)
    end
end


d1 = doc.at('#d1')
gsub_my_text!( d1, /[aeiou]+/ ){ |found| found.upcase }

puts d1
#=> <div id="d1">
#=>   ThIs Is tExt In thE TD wIth <strong> strong </strong> tAgs
#=>   <p>This is a child node. with <b> bold </b> tags</p>
#=>   <div id="d2">
#=>       "another line of text to a <a href="link.html"> link </a>"
#=>      <p> This is text inside a div <em>inside<em> another div inside a paragraph tag</em></em></p>
#=>   </div>
#=> </div>


gsub_text!( d1, /\w+/, '(\\0)' )
puts d1
#=> <div id="d1">
#=>   (ThIs) (Is) (tExt) (In) (thE) (TD) (wIth) <strong> (strong) </strong> (tAgs)
#=>   <p>(This) (is) (a) (child) (node). (with) <b> (bold) </b> (tags)</p>
#=>   <div id="d2">
#=>       "(another) (line) (of) (text) (to) (a) <a href="link.html"> (link) </a>"
#=>      <p> (This) (is) (text) (inside) (a) (div) <em>(inside)<em> (another) (div) (inside) (a) (paragraph) (tag)</em></em></p>
#=>   </div>
#=> </div>

---

Edit: Here is code that allows you to extract runs of text+inline markup as a string, run a gsub on that, and replace the result with new markup.

require 'nokogiri'

doc = Nokogiri.HTML '<div id="d1">
  Text with <strong>strong</strong> tag.
  <p>This is a child node. with <b>bold</b> tags.</p>
  <div id=d2>And now we are in <a href="foo">another</a> div.</div>
  Hooray for <em>me!</em>
</div>'

module Enumerable
  # http://stackoverflow.com/q/4800337/405017
  def split_on() chunk{|o|yield(o)||nil}.map{|b,a|b&&a}.compact end
end

require 'set'
# Given a node, call gsub on the `inner_html` 
def gsub_markup!( node, find, replace=nil, &replace_block )
  allowed = Set.new(%w[strong b em i u strike])
  runs  = node.children.split_on{ |el| el.node_type==1 && !allowed.include?(el.name) }
  runs.each do |nodes|
    orig   = nodes.map{ |node| node.node_type==3 ? node.content : node.to_html }.join
    next if orig.strip.empty? # Skip whitespace-only nodes
    result = replace ? orig.gsub(find,replace) : orig.gsub(find,&replace_block)
    puts "I'm replacing #{orig.inspect} with #{result.inspect}" if $DEBUG
    nodes[1..-1].each(&:remove)
    nodes.first.replace(result)
  end
end

d1 = doc.at('#d1')

$DEBUG = true
gsub_markup!( d1, /[aeiou]+/, &:upcase )
#=> I'm replacing "\n  Text with <strong>strong</strong> tag.\n  " with "\n  TExt wIth <strOng>strOng</strOng> tAg.\n  "
#=> I'm replacing "\n  Hooray for <em>me!</em>\n" with "\n  HOOrAy fOr <Em>mE!</Em>\n"

puts doc
#=> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
#=> <html><body><div id="d1">
#=>   TExt wIth <strong>strOng</strong> tAg.
#=>   <p>This is a child node. with <b>bold</b> tags.</p>
#=>   <div id="d2">And now we are in <a href="foo">another</a> div.</div>
#=>   HOOrAy fOr <em>mE!</em>
#=> </div></body></html>

Answer 2

The easiest way would be:

div = doc.at('div#1') 
div.replace div.to_s.sub('text', 'modified text')