4

I have a few strings containing a variant of Hexadecimal strings (source is framemaker if one would care). Strings could therefore look like

this is some sentence with some hex code\x27 s , and we need that fixed.

and will need to be changed to

this is some sentence with some hex code's , and we need that fixed.

In reality there can be a few of these in a single string, so I'm looking on the best way to walk through the text, capture all hex codes (looking like \x## ) and replace all of these codes with the correct character. I have made a xml list / lookup table containing all the characters as follows :

<xsl:param name="reflist">
    <Code Value="\x27">'</Code>
<Code Value="\x28">(</Code>
<Code Value="\x29">)</Code>
<Code Value="\x2a">*</Code>
<Code Value="\x2b">+</Code>
    <!-- much more like these... -->
</xsl:param>

For now I used a simple replace argument but there are simply too many characters to make this workable.

What's the best way to do this?

Mads Hansen
  • 63,927
  • 12
  • 112
  • 147
Wokoman
  • 1,089
  • 2
  • 13
  • 30

2 Answers2

5

One can completely avoid using any "reference table" -- like this:

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 xmlns:xs="http://www.w3.org/2001/XMLSchema"
 xmlns:my="my:my" exclude-result-prefixes="my xs">
 <xsl:output omit-xml-declaration="yes" indent="yes"/>

 <xsl:template match="node()|@*">
  <xsl:copy>
   <xsl:apply-templates select="node()|@*"/>
  </xsl:copy>
 </xsl:template>

 <xsl:template match="text()[matches(.,  '\\x(\d|[a-f])+')]">
   <xsl:analyze-string select="." regex="\\x(\d|[a-f])+" >
     <xsl:matching-substring>
       <xsl:value-of select=
       "codepoints-to-string(my:hex2dec(substring(.,3), 0))"/>
     </xsl:matching-substring>
     <xsl:non-matching-substring>
      <xsl:value-of select="."/>
     </xsl:non-matching-substring>
   </xsl:analyze-string>
 </xsl:template>

 <xsl:function name="my:hex2dec" as="xs:integer">
  <xsl:param name="pStr" as="xs:string"/>
  <xsl:param name="pAccum" as="xs:integer"/>

  <xsl:sequence select=
   "if(not($pStr))
     then $pAccum
     else
      for $char in substring($pStr, 1, 1),
          $code in
            if($char ge '0' and $char le '9')
              then xs:integer($char)
              else
                string-to-codepoints($char) - string-to-codepoints('a') +10
       return
          my:hex2dec(substring($pStr,2), 16*$pAccum + $code)
   "/>
 </xsl:function>
</xsl:stylesheet>

When this transformation is applied on the following XML document:

<t>
 <p>this is some sentence with some hex code\x27 s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x28 s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x29 s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2a s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2b s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2c s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2d s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2e s ,
    and we need that fixed.</p>
 <p>this is some sentence with some hex code\x2f s ,
    and we need that fixed.</p>
</t>

the wanted, correct result is produced:

<t>
   <p>this is some sentence with some hex code' s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code( s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code) s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code* s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code+ s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code, s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code- s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code. s ,
    and we need that fixed.</p>
   <p>this is some sentence with some hex code/ s ,
    and we need that fixed.</p>
</t>

Do note:

This transformation is generic and can correctly process any hexadecimal unicode code.

For example, if the same transformation is applied on this XML document:

<t>
 <p>this is some sentence with some hex code\x0428\x0438\x0448 s ,
    and we need that fixed.</p>
</t>

the correct result (containing the Bulgarian word for "grill" in Cyrillic) is produced:

<t>
   <p>this is some sentence with some hex codeШиш s ,
    and we need that fixed.</p>
</t>
Dimitre Novatchev
  • 240,661
  • 26
  • 293
  • 431
  • 1
    Thanks Dimitri, though this is a good and great solution I can't use it as such because of the way Adobe 'abused' the hex code values inside Framemaker. Adobe uses 3 default Fonts, and for each they give different values, hence why I am forced to use a lookup table. But thanks again for the efforts, I can probably use the way of thinking in a later stage. – Wokoman Nov 11 '12 at 15:39
3

Use analyze-string as in

<xsl:template match="text()">
  <xsl:analyze-string select="." regex="\\x[0-9a-f]{{2}}" flags="i">
    <xsl:matching-substring>
      <xsl:value-of select="$reflist/Code[@Value = .]"/>
    </xsl:matching-substring>
    <xsl:non-matching-substring>
      <xsl:value-of select="."/>
    </xsl:non-matching-substring>
  </xsl:analyze-string>
</xsl:template>

I would also suggest to use a key e.g.

<xsl:param name="reflist" as="document-node()">
  <xsl:document>
    <Root>
      <Code Value="\x27">'</Code>
      <Code Value="\x28">(</Code>
      <Code Value="\x29">)</Code>
      <Code Value="\x2a">*</Code>
      <Code Value="\x2b">+</Code>
      <!-- much more like these... -->
    </Root>
  </xsl:document>
</xsl:param>

<xsl:key name="code-by-value" match="Code" use="@Value"/>

then the lookup can be improved to

<xsl:template match="text/text()">
  <xsl:analyze-string select="." regex="\\x[0-9a-f]{{2}}" flags="i">
    <xsl:matching-substring>
      <xsl:value-of select="key('code-by-value', ., $reflist)"/>
    </xsl:matching-substring>
    <xsl:non-matching-substring>
      <xsl:value-of select="."/>
    </xsl:non-matching-substring>
  </xsl:analyze-string>
</xsl:template>

I have found some time to morph the suggestions made into working code, with the input being

<root>
  <text>this is some sentence with some hex code\x27 s , and we need that \x28and this\x29 fixed.</text>
</root>

and the complete stylesheet being

<xsl:stylesheet 
  version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  exclude-result-prefixes="xs">

<xsl:param name="reflist" as="document-node()">
  <xsl:document>
    <Root>
      <Code Value="\x27">'</Code>
      <Code Value="\x28">(</Code>
      <Code Value="\x29">)</Code>
      <Code Value="\x2a">*</Code>
      <Code Value="\x2b">+</Code>
      <!-- much more like these... -->
    </Root>
  </xsl:document>
</xsl:param>

<xsl:key name="code-by-value" match="Code" use="@Value"/>

<xsl:template match="@* | node()">
  <xsl:copy>
    <xsl:apply-templates select="@* , node()"/>
  </xsl:copy>
</xsl:template>

<xsl:template match="text/text()">
  <xsl:analyze-string select="." regex="\\x[0-9a-f]{{2}}" flags="i">
    <xsl:matching-substring>
      <xsl:value-of select="key('code-by-value', ., $reflist)"/>
    </xsl:matching-substring>
    <xsl:non-matching-substring>
      <xsl:value-of select="."/>
    </xsl:non-matching-substring>
  </xsl:analyze-string>
</xsl:template>

</xsl:stylesheet>

Saxon 9.4 transforms the input as follows:

<root>
  <text>this is some sentence with some hex code' s , and we need that (and this) fixed.</text>
</root>
Martin Honnen
  • 160,499
  • 6
  • 90
  • 110