ASP.NET to handle JWT tokens from Azure ACS (Access Control Service)

Question

I am trying to make my ASP.NET application to handle JWT token from ACS instead of SAML 2.0 - which works perfectly.

I have configured my web.config to use the default Jwt handler from System.IdentityModel.Tokens.Jwt as follows

<system.identityModel>
<identityConfiguration saveBootstrapContext="true">
  <audienceUris>
    <add value="http://localhost:57547/" />
  </audienceUris>
  <securityTokenHandlers>
         <add type="System.IdentityModel.Tokens.JwtSecurityTokenHandler, System.IdentityModel.Tokens.Jwt" />
           <securityTokenHandlerConfiguration>
              <certificateValidation certificateValidationMode="PeerTrust"/>

    </securityTokenHandlerConfiguration>

  </securityTokenHandlers>
  <issuerNameRegistry type="System.IdentityModel.Tokens.ValidatingIssuerNameRegistry, System.IdentityModel.Tokens.ValidatingIssuerNameRegistry">

    <authority name="https://nithish.accesscontrol.windows.net/">

      <keys>
        <add thumbprint="xxxx" />
        <add symmetricKey="xxxx" />
      </keys>
      <validIssuers>
        <add name="https://nithish.accesscontrol.windows.net/" />
      </validIssuers>
    </authority>
  </issuerNameRegistry>

  <issuerTokenResolver type="System.IdentityModel.Tokens.NamedKeyIssuerTokenResolver, System.IdentityModel.Tokens.Jwt">
    <securityKey symmetricKey="xxxx"
                  name="https://nithish.accesscontrol.windows.net/" />
  </issuerTokenResolver>
  <!--certificationValidationMode set to "None" by the the Identity and Access Tool for Visual Studio. For development purposes.-->
  <certificateValidation certificateValidationMode="None" />
</identityConfiguration>

I am getting this error as follows

[NotSupportedException: IDX11008: This method is not supported to validate a 'jwt' use the method: ValidateToken(String, TokenValidationParameters, out SecurityToken).]
   System.IdentityModel.Tokens.JwtSecurityTokenHandler.ValidateToken(SecurityToken token) in c:\workspace\WilsonForDotNet45Release\src\System.IdentityModel.Tokens.Jwt\JwtSecurityTokenHandler.cs:641
   System.IdentityModel.Tokens.SecurityTokenHandlerCollection.ValidateToken(SecurityToken token) +73
   System.IdentityModel.Services.TokenReceiver.AuthenticateToken(SecurityToken token, Boolean ensureBearerToken, String endpointUri) +118
   System.IdentityModel.Services.WSFederationAuthenticationModule.SignInWithResponseMessage(HttpRequestBase request) +489
   System.IdentityModel.Services.WSFederationAuthenticationModule.OnAuthenticateRequest(Object sender, EventArgs args) +361
   System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +136
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +69  

The more weird thing is that System.IdentityModel.Tokens.JwtSecurityTokenHandler.ValidateToken(SecurityToken token) in c:\workspace\WilsonForDotNet45Release\src\System.IdentityModel.Tokens.Jwt\JwtSecurityTokenHandler.cs:641
Is not even a valid file location of my own source code. I don't know where this error is coming from, seems like its from the framework itself.

What am I doing wrong?