How to remove the duplicate nodes from xml using xslt

Question

I tried n number of options to remove the duplicate node from xml but not able to achieve the correct result.Below is the input xml

<root>
  <paymentFlag>true</paymentFlag>
  <cardFlag>true</cardFlag>
  <pinFlag>true</pinFlag>
  <paymentFlag>true</paymentFlag>
  <outputFlag>false</outputFlag>
  <pinFlag>true</pinFlag>
  <cardFlag>true</cardFlag>
  ...
</root>

Inside root element I have a n number of nodes all in same level but some of them are duplicate. I want to remove them and expecting a below output.

<root>
  <paymentFlag>true</paymentFlag>
  <cardFlag>true</cardFlag>
  <pinFlag>true</pinFlag>
  <outputFlag>false</outputFlag>
</root>

Please provide your input.

Possible duplicate of [Removing duplicate elements with XSLT](http://stackoverflow.com/questions/10912544/removing-duplicate-elements-with-xslt) — Stavr00, May 06 '16 at 13:43
Take a look at https://www.safaribooksonline.com/library/view/xslt-cookbook/0596003722/ch04s03.html — Stavr00, May 06 '16 at 13:44

zx485 · Accepted Answer · 2016-05-06T17:37:40.670

You can use <xsl:key> to generate a map that separates the first occurrences from the others:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
  <xsl:output method="xml" indent="yes" />
  <xsl:key name="firsts" match="/root/*" use="local-name()" />    
  <xsl:template match="/root"> 
    <root>
      <xsl:for-each select="*[generate-id() = generate-id(key('firsts',local-name()))]">
        <xsl:copy-of select="." />
      </xsl:for-each>
    </root>
  </xsl:template>
</xsl:stylesheet>

Output is:

<?xml version="1.0"?>
<root>
    <paymentFlag>true</paymentFlag>
    <cardFlag>true</cardFlag>
    <pinFlag>true</pinFlag>
    <outputFlag>false</outputFlag>
</root>

To determine the full name of the element I used local-name(), because most beginners do not care about namespace s and do not consider the part before the : in a QName like abc:Element. So if you need to include namespaces into your consideration, just replace local-name() with name().

Thanks for the solution @zx485 – VJ THAKUR May 06 '16 at 14:54 — VJ THAKUR, May 06 '16 at 14:54

score 0 · Answer 2 · edited May 23 '17 at 10:28

Here's an XSLT 1.0 option that uses xsl:key that is very similar to the currently accepted answer.

The difference is that if you change from XSLT 1.0 to 2.0, this answer won't break. It will work in both versions. (The other answer will fail with the error "A sequence of more than one item is not allowed as the first argument of generate-id()" if run as version 2.0.)

Another difference is that I use name() instead of local-name(). By doing this I treat elements with namespace prefixes as different (which they are). If you want to treat them the same, use local-name(). (Also, if you want to treat elements in different default namespaces (unprefixed), you could create a compound key using namespace-uri(). Let me know if you'd like an example.)

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output indent="yes"/>
  <xsl:strip-space elements="*"/>

  <xsl:key name="firsts" match="/*/*" use="name()"/>

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="/*">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:for-each select="*[count(.|key('firsts',name())[1])=1]">
        <xsl:apply-templates select="."/>
      </xsl:for-each>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>

If you're already using 2.0, here's another 2.0 only option using xsl:for-each-group...

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output indent="yes"/>
  <xsl:strip-space elements="*"/>

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="/*">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:for-each-group select="*" group-by="name()">
        <xsl:apply-templates select="current-group()[1]"/>
      </xsl:for-each-group>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>

Note that I use xsl:apply-templates in both options to make the stylesheet easier to extend in the future.

score -1 · Answer 3 · answered May 06 '16 at 14:04

same level but some of them are duplicate

Question is what is to be considered as duplicated. If only the element name should be checked you may try something like this:

<xsl:stylesheet  
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>  

  <xsl:template match="root/*">
    <xsl:variable name="this" select="." />
    <xsl:if test="not(preceding-sibling::*[name() = name($this)])" >
     <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
    </xsl:if>
  </xsl:template>

</xsl:stylesheet>

Thanks for the solution @hr_117 – VJ THAKUR May 06 '16 at 14:54 — VJ THAKUR, May 06 '16 at 14:54

score -1 · Answer 4 · answered May 06 '16 at 14:39

With xsl stylesheet version 2

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes" />
  <xsl:template match="/">
    <root>
      <xsl:for-each select="root/*[not(name()=preceding::*/name())]">
        <xsl:element name="{name()}">
              <xsl:value-of select="text()"/>
        </xsl:element>
      </xsl:for-each>
    </root>
  </xsl:template>
</xsl:stylesheet>

Output :

<?xml version="1.0" encoding="UTF-8"?>
<root>
   <paymentFlag>true</paymentFlag>
   <cardFlag>true</cardFlag>
   <pinFlag>true</pinFlag>
   <outputFlag>false</outputFlag>
</root>

How to remove the duplicate nodes from xml using xslt

4 Answers4

Linked