It's not clear why you want to convert this to a custom binding - usually a custom binding is used when one of the out-of-the-box bindings does not meet the needs of the requirements.
A quick way to do this is to use Yaron Naveh's WCF BindingBox (found via this answer here on SO). Using the BindingBox with your supplied config yields the following:
<customBinding>
<binding name="NewBinding0">
<security authenticationMode="CertificateOverTransport"
messageSecurityVersion="WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10"
requireDerivedKeys="false" securityHeaderLayout="Lax" />
<textMessageEncoding MessageVersion="Soap11" />
<httpsTransport />
</binding>
</customBinding>
When developing a custom binding, you need to keep in mind the following order of elements:
- TransactionFlow element (Optional)
- ReliableSession element (Optional)
- Security (Optional, can be Symmetric, Asymmetric, Transport-level)
- CompositeDuplex element (Optional)
- Transport upgrades (Optional, can be SSL stream, Windows stream, Peer Resolver)
- Text Encoding (Required, can be Text, Binary, MTOM, Custom)
- Transport (Required, can be TCP, Named Pipes, HTTP, HTTPS, flavors of MSMQ, Custom)
You can get more information from MSDN - CusotmBinding Class
EDIT
Since you are using a custom binary encoding, you would change the <textMessageEncoding>
element to <binarMessageEncoding MessageVersion="Soap11">
.