AWS SecretsManager works in Eclipse, can't connect to Service Endpoint in ColdFusion

Question

I have the following class written in Java using Eclipse on my Amazon EC2 instance.

import java.nio.ByteBuffer;

import com.amazonaws.auth.*;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.secretsmanager.*;
import com.amazonaws.services.secretsmanager.model.*;

public class SMtest {
    public static void main(String[] args) {
        String x = getSecret();
        
        System.out.println(x);
    }     
    
  public SMtest()
  {
      
  }
  

public static String getSecret() {

      String secretName = "mysecret";
      String endpoint = "secretsmanager.us-east-1.amazonaws.com";
      String region = "us-east-1";
      String result = "";
      
      //BasicAWSCredentials awsCreds = new BasicAWSCredentials("mypublickey", "mysecretkey");
      

      AwsClientBuilder.EndpointConfiguration config = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
      AWSSecretsManagerClientBuilder clientBuilder = AWSSecretsManagerClientBuilder.standard()
              .withCredentials(new InstanceProfileCredentialsProvider(false));
              //.withCredentials(new AWSStaticCredentialsProvider(awsCreds));
      clientBuilder.setEndpointConfiguration(config);
      AWSSecretsManager client = clientBuilder.build();

      String secret;
      ByteBuffer binarySecretData;
      GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
              .withSecretId(secretName).withVersionStage("AWSCURRENT");
      GetSecretValueResult getSecretValueResult = null;
      try {
          getSecretValueResult = client.getSecretValue(getSecretValueRequest);

      } catch(ResourceNotFoundException e) {
          System.out.println("The requested secret " + secretName + " was not found");
      } catch (InvalidRequestException e) {
          System.out.println("The request was invalid due to: " + e.getMessage());
      } catch (InvalidParameterException e) {
          System.out.println("The request had invalid params: " + e.getMessage());
      }

      if(getSecretValueResult == null) {
          result = "";
      }

      // Depending on whether the secret was a string or binary, one of these fields will be populated
      if(getSecretValueResult.getSecretString() != null) {
          secret = getSecretValueResult.getSecretString();
          result = secret;
          //System.out.println(secret);
      }
      else {
          binarySecretData = getSecretValueResult.getSecretBinary();
          result = binarySecretData.toString();
      }
      
      return result;

  }

}

When I execute it from within Eclipse, it works just fine. When I compile the class to use it in ColdFusion (2021) from the same EC2 with the following code:

<cfscript>
    obj = CreateObject("java","SMtest");
    obj.init();
    result = obj.getSecret();
</cfscript>
<cfoutput>#result#</cfoutput>

I get a "Failed to connect to service endpoint" error. I believe I have all the IAM credentials set up properly since it is working in straight Java. However, when I change the credentials to use the Basic Credentials with my AWS Public and Secret Key (shown in comments in the code above), it works in both Java and ColdFusion.

I created an AWS Policy that manages the Secrets Manager permissions. I also added AmazonEC2FullAccess. I also tried to create a VPC Endpoint, but this had no effect.

Why would it be working in Java and not in ColdFusion (which is based on Java) ? What roles/policies would I have to add to get it to work in ColdFusion when it is already working in Java?

STACK TRACE ADDED:

com.amazonaws.SdkClientException: Failed to connect to service endpoint: 

at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)

at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)

at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)

at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)

at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsEndpoint(InstanceMetadataServiceCredentialsFetcher.java:58)

at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsResponse(InstanceMetadataServiceCredentialsFetcher.java:46)

at com.amazonaws.auth.BaseCredentialsFetcher.fetchCredentials(BaseCredentialsFetcher.java:112)

at com.amazonaws.auth.BaseCredentialsFetcher.getCredentials(BaseCredentialsFetcher.java:68)

at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:165)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)

at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)

at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.doInvoke(AWSSecretsManagerClient.java:2454)

at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2421)

at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2410)

at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.executeGetSecretValue(AWSSecretsManagerClient.java:943)

at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.getSecretValue(AWSSecretsManagerClient.java:912)

at SMtest.getSecret(SMtest.java:52)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:568)

at coldfusion.runtime.java.JavaProxy.invoke(JavaProxy.java:106)

at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4254)

at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4217)

at cfsmtest2ecfm1508275519.runPage(D:\ColdFusion2021\cfusion\wwwroot\smtest.cfm:4)

at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:257)

at coldfusion.tagext.lang.IncludeTag.handlePageInvoke(IncludeTag.java:749)

at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:578)

at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)

at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:573)

at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:43)

at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)

at coldfusion.filter.PathFilter.invoke(PathFilter.java:162)

at coldfusion.filter.IpFilter.invoke(IpFilter.java:45)

at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:30)

at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:97)

at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:81)

at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)

at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)

at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:60)

at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)

at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)

at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)

at coldfusion.CfmServlet.service(CfmServlet.java:231)

at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:311)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)

at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:46)

at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)

at coldfusion.inspect.weinre.MobileDeviceDomInspectionFilter.doFilter(MobileDeviceDomInspectionFilter.java:57)

at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)

at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:373)

at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:462)

at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)

at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)

at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

at java.base/java.lang.Thread.run(Thread.java:833)

Caused by: java.net.MalformedURLException: Unknown protocol: http

at java.base/java.net.URL.<init>(URL.java:708)

at java.base/java.net.URL.fromURI(URL.java:748)

at java.base/java.net.URI.toURL(URI.java:1139)

at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:83)

at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)

... 80 more

Caused by: java.lang.IllegalStateException: Unknown protocol: http

at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:373)

at java.base/java.net.URL.<init>(URL.java:703)

... 84 more

After noticing that the bottom end of the Stack Trace says "unknown protocol: http" and "Illegal State Exception: Unknown Protocol", I changed the endpoint in my class above to:

String endpoint = "https://secretsmanager.us-east-1.amazonaws.com";

and I am running the web-site via https now. It still works with no problem running it in Eclpse, and I am still getting the same error when using ColdFusion (i.e., when using a browser)

Are they both using the same JVM? I’ve run into issues with Java http requests that use certain types of SSL certs. — Redtopia, Mar 14 '22 at 23:59
Yes I changed the JVM of ColdFusion to the same JVM that I am using for the Java class. — RickInWestPalmBeach, Mar 15 '22 at 00:32
What's the full stack trace message? What is your classpath in java compared to CF? — SOS, Mar 15 '22 at 02:41
Classpath is identical in both. I added stack trace and additional comments to my original question. — RickInWestPalmBeach, Mar 15 '22 at 12:40
@RickInWestPalmBeach "Yes I changed the JVM of ColdFusion to the same JVM that I am using for the Java class." Is that the same major version of JVM? What would be the negative impact of changing the CF server to use a different version? Did you load your `SMTest.jar` file to the CF class path? Any chance the network is rejecting the outbound request from the CF instance? — Adrian J. Moreno, Mar 15 '22 at 15:43
I had a whole different set of issues when I used different JVM's. Now they are using the same exact version. I copied the .jar to the CF class path, and it had no effect. My security settings for my EC2 has all outbound ports open. — RickInWestPalmBeach, Mar 15 '22 at 16:22
I realize the top level error is still the same, but what does the rest of the stack trace look like now that you switched to https? — SOS, Mar 19 '22 at 04:34
@RickInWestPalmBeach - Before the question bounty expires tomorrow, did you have any luck with enabling logging to get more info OR trying cfhttp just as a test? — SOS, Mar 31 '22 at 19:43
So it works on your local machine, but not on the instance? Is that correct? — gshpychka, Apr 01 '22 at 13:12
It works from the Eclipse IDE, but not from a ColdFusion web page or the AWS CLI — RickInWestPalmBeach, Apr 06 '22 at 23:11

score 2 · Answer 1 · answered Mar 23 '22 at 02:33

2

Since the basic credentials are working and since the error is not access denied, it suggests there is no issue with your IAM setup. Instead, likely your process failed to fetch the EC2 instance metadata. For example, timeout when calling the metadata endpoint, hence the Failed to connect to service endpoint error.

One way around this is to retrieve the accessKey and secretKey manually by calling the instance metadata API. For example, using cfhttp to populate variables.

Example curl command from docs: retrieve /api/token and use it to retrieve /meta-data/iam/security-credentials/<rolename>.

[ec2-user ~]$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access

Output:

{
  "Code" : "Success",
  "LastUpdated" : "2012-04-26T16:39:16Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAIOSFODNN7EXAMPLE",
  "SecretAccessKey" : "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "Token" : "token",
  "Expiration" : "2017-05-17T15:09:54Z"
}

From here, if you still face any errors, at least it should be more explicit to you which step has failed and why.

Just a note, AWS recommends caching the credentials until near expiry instead of querying for every transaction to avoid throttling.

answered Mar 23 '22 at 02:33

Register Sole

3,206
1
14
22

I shouldn't need to specify any credentials at all because I am accessing the AWS API from an Amazon EC2 and I established the policies in IAM. And as my question explains, it works every time when I execute the java code within Eclipse. Also, I am able to successfully retrieve my secret from the AWS CLI. It is just when I try to retrieve the secret from the ColdFusion page that it fails. – RickInWestPalmBeach Mar 23 '22 at 16:13
@RickInWestPalmBeach You shouldn't need to specify any credentials because the SDK does it for you automatically by fetching temporary credentials from the instance metadata API. It failed though, so I'm suggesting you fetch it manually. Also I'm suggesting to use ColdFusion `cfhttp` to call the fetch credentials API, to see if it works. I just gave the example in CLI because I don't have a CF environment to test. – Register Sole Mar 24 '22 at 02:50
`likely your process failed to fetch the EC2 instance metadata` Yeah, I was wondering about that myself. You might try adjusting the [log settings](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html) for the aws classes, to try and confirm the theory or at least get some additional information about the issue. – SOS Mar 24 '22 at 19:18
Can you connect to http://169.254.169.254/latest/meta-data/ from your EC2 instance? See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html. – Tony BenBrahim Apr 01 '22 at 22:20
Yes I can. I get a list of directories. – RickInWestPalmBeach Apr 06 '22 at 23:12

AWS SecretsManager works in Eclipse, can't connect to Service Endpoint in ColdFusion

1 Answers1

Linked