Connect Java Google AppEngine Local Standard Server to Cloud DB | appengine-api-1.0-sdk-1.9.84.jar | IntelliJ & Cloud Code

Question

EDIT2: I have managed to get past the GlobalDatastoreConfig has already been set error. I managed to pinpoint all the locations that were getting called before the init function. They were in static space in some weird files.

I have now pointed ALL DatastoreServiceFactory.getDatastoreService() to a new static function I've created in a file called Const.java.

private static boolean hasInit = false;
public static DatastoreService getDatastoreService() {
    if(!hasInit) {
        try {
            CloudDatastoreRemoteServiceConfig config = CloudDatastoreRemoteServiceConfig
                        .builder()
                        .appId(CloudDatastoreRemoteServiceConfig.AppId.create(CloudDatastoreRemoteServiceConfig.AppId.Location.US_CENTRAL, "gcp-project-id"))
                        .build();
            CloudDatastoreRemoteServiceConfig.setConfig(config);
            hasInit = true;
        } catch (Exception ignore) {}
    }
    return DatastoreServiceFactory.getDatastoreService();
}

This returns no errors on the first initialisation. However, I am getting a new error now!

Dec 08, 2022 6:49:56 PM com.google.appengine.api.datastore.dev.LocalDatastoreService init
INFO: Local Datastore initialized: 
    Type: High Replication
    Storage: C:\Users\user\dev\repo\Celbux\core\Funksi179_NSFAS_modules\classes\artifacts\Funksi179_NSFAS_modules_war_exploded\WEB-INF\appengine-generated\local_db.bin
Dec 08, 2022 6:49:56 PM com.google.appengine.api.datastore.dev.LocalDatastoreService load
INFO: Time to load datastore: 20 ms
2022-12-08 18:49:56.757:WARN:oejs.HttpChannel:qtp1681595665-26: handleException / java.io.IOException: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
2022-12-08 18:49:56.762:WARN:oejsh.ErrorHandler:qtp1681595665-26: Error page too large: 500 org.apache.jasper.JasperException: com.google.apphosting.api.ApiProxy$RPCFailedException: I/O error

Full stacktrace: https://pastebin.com/YQ2WvqzM

Pretty sure the first of the errors is invoked from this line:

DatastoreService ds = Const.getDatastoreService();
Key ConstantKey = KeyFactory.createKey("Constants", 1);
Entity Constants1 = ds.get(ConstantKey) // <-- This line.

---

EDIT1: I am not using Maven. Here are the .jars I have in WEB-INF/lib

appengine-api-1.0-sdk-1.9.84.jar
appengine-api-labs.jar
appengine-api-labs-1.9.76.jar
appengine-api-stubs-1.9.76.jar
appengine-gcs-client.jar
appengine-jsr107cache-1.9.76.jar
appengine-mapper.jar
appengine-testing-1.9.76.jar
appengine-tools-sdk-1.9.76.jar
charts4j-1.2.jar
guava-11.0.2.jar
javax.inject-1.jar
json-20190722.jar

---

Original Question: The company that I'm working at have a legacy GCP codebase written in Java. This codebase uses the appengine-api-1.0-sdk.jar libary. Upon running this CloudDatastoreRemoteServiceConfig code in the very first place that our DatastoreService gets initialised, it says that the config has already been set.

If someone can shed light on how to get this outdated tech connected to the Cloud via localhost, I'll be most grateful!

web.xml

    <filter>
        <filter-name>NamespaceFilter</filter-name>
        <filter-class>com.sintellec.funksi.Filterns</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>NamespaceFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

Code

public class Filterns implements javax.servlet.Filter {

    public void init(FilterConfig filterConfig) {
        try {
            CloudDatastoreRemoteServiceConfig config = CloudDatastoreRemoteServiceConfig
                    .builder()
                    .appId(CloudDatastoreRemoteServiceConfig.AppId.create(CloudDatastoreRemoteServiceConfig.AppId.Location.US_CENTRAL, "gcp-project-id"))
                    .build();
            CloudDatastoreRemoteServiceConfig.setConfig(config);
            DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
        } catch (Exception e) {
            System.out.println(e);
            return;
        }

        this.filterConfig = filterConfig;
    }

}

I got this code snippet from here.

Was thinking a few ideas:

1. Perhaps there's GCP code that's called before our Java code which initialises the Local DB
2. Perhaps I need to set a global environment variable to point this old emulator to a Cloud Configuration instead

Only problem is I have no idea what to do from here, hoping someone has experience on the legacy Java library here.

To clarify; I am trying to get this outdated GCP Java codebase (appengine-api-1.0-sdk.jar) to connect to Cloud Datastore, NOT use the Local Datastore Emulator. This is so I can debug multiple applications that all access the same Cloud DB

Answer 1

It is very difficult to say especially with that amount of code and we can only guess but, as you indicated, probably some code is initializing your DataStore configuration, probably the SDK itself. You could try setting a breakpoint in the setConfig method of CloudDatastoreRemoteServiceConfig and analyze the call stack.

In any way, one think you could also try is not performing such as initialization in your code, delegating to Application Default Credentials the authentication of your client libraries.

For local development you have two options to configure such as Application Default Credentials.

On one hand, you can use user credentials, i.e., you can use the gcloud CLI to authenticate against GCP with an user with the required permissions to interact with the service, issuing the following command:

gcloud auth application-default login

Please, don't forget to revoke those credentials when you no longer need them:

gcloud auth application-default revoke

On the other, you can create a service account with the necessary permissions and a corresponding service account key, and download that key, a JSON file, to your local filesystem. See this for instructions specific to DataStore. Then, set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the downloaded file with your service account key.

Again, a word of caution: take care of the downloaded service account key file and never put it under version control because anyone with that file could assume the permissions granted to the service account.

You code should work without further problem when running in GCP because probably you will be using a service that supports attaching a service account which means that Application Default Credentials are provided by the GCP services per se.