Here's a start for an XSLT that will replace the party href with the corresponding party element.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:fpml="http://www.example.com">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*" />
</xsl:copy>
</xsl:template>
<xsl:template match="fpml:partyReference[@href]">
<xsl:variable name="href" select="@href" />
<partyReference>
<party>
<xsl:apply-templates select="//party[@id=$href]" mode="dereference" />
</party>
</partyReference>
</xsl:template>
<xsl:template match="party" />
<xsl:template match="party" mode="dereference">
<xsl:element name="{@id}">
<xsl:apply-templates select="node()|@*[not(local-name()='id')]" />
</xsl:element>
</xsl:template>
</xsl:stylesheet>
Seeing how I don't know what your fpml prefix binds to, I put in some sample URI for that namespace.
The first template matching node()|@* is some standard approach that will just copy anything that doesn't match any other templates.
The second template matching fpml:partyReference[@href] will take any partyReference with a href attribute (in the given namespace), extract the value of @href to a variable, and then applies templates to any party element where the id attribute matches that href value. Notice how it introduces a mode named "dereference". That name is arbitrary and something I chose.
Next is an empty template that matches all party elements and does nothing. They won't get copied. This avoids copying the party again after it had already been placed in the reference earlier.
Finally is a template that matches all party elements, but only when in mode dereference. This will create a new element with as name the value of the id attribute and then applies templates to the attributes and child nodes, with the exception of the id attribute (since you don't want it copied in the output). This will just default to copying the contents.
Since I don't have enough information about what those partyIdScheme attributes in your input do, I haven't transformed those contents. The above should give you some indications of how to solve this. Note that you will need to alter the XSLT with the right namespace for prefix fpml, and that you may need to alter the namespace usage since your XML extracts leave some ambiguity about what is in which namespace (we'd need to see well-formed XML document to figure that out rather than extracts).
Also when i try to do a template match for Party1 which is the XSLT transformed element, the parser is not able to recognize it. But when i match the element party (which is the original one) the parser is able to recognize it.
That is because XSLT only works on the input document. It traverses the input, matches its parts to templates and executes the instructions in those templates. It's a declarative language. So the output being generated is not part of the input and won't affect it. You'd need multiple XSLT transformations for that.
It's possible that the XML you're provided makes use of some technologies such as XInclude or other referencing schemes. You may be able to get your desired results using parsers that support the right technology or some library that implements such referencing schemes, so before you continue using XSLT, see if there's something that already does what you're trying.
EDIT: an example matching multiple elements in the same amount of templates as above. Note that this will only work if the id attribute in the input XML is unique for every element that can get referenced by href. Otherwise the results might be incorrect.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
xmlns:fpml="http://www.example.com">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*" />
</xsl:copy>
</xsl:template>
<xsl:template match="fpml:partyReference[@href]|fpml:accountReference[@href]|fpml:payerPartyReference[@href]">
<xsl:variable name="href" select="@href" />
<xsl:element name="{local-name()}" namespace="{namespace-uri()}">
<xsl:apply-templates select="//*[@id=$href]" mode="dereference" />
</xsl:element>
</xsl:template>
<xsl:template match="party|account|payerParty" />
<xsl:template match="party|account|payerParty" mode="dereference">
<xsl:element name="{local-name()}" namespace="{namespace-uri()}">
<xsl:element name="{@id}">
<xsl:apply-templates select="node()|@*[not(local-name()='id')]" />
</xsl:element>
</xsl:element>
</xsl:template>
</xsl:stylesheet>
