9

I am trying to test google's cloud datastore locally. My test steps are:

  1. Crawl data using Scrapy
  2. Save data to the cloud datastore using local emulator

I followed all the steps in order to use the local emulator

  1. start local emulator: gcloud beta emulators datastore start
  2. set local environment variables: gcloud beta emulators datastore env-init

However, in python, when use the following command to access cloud datastore, it always save the data directly to google cloud instead of saving them to the local emulators

#Imports the Google Cloud client library
from google.cloud import datastore

# Instantiates a client
datastore_client = datastore.Client()

sample_entry = some_data

# Saves the entity
datastore_client.put(sample_entry)

It seems like you cannot specify the library to use the local datastore emulator, just like what they offer in their Node.js client

var datastore = gcloud.datastore({
        apiEndpoint: "http://localhost:8380"
});

My question is, How can I ask the google cloud datastore python library to use local emulator instead of using the cloud directly

JSNoob
  • 1,477
  • 2
  • 18
  • 31
  • 1
    What's the value of the `DATASTORE_EMULATOR_HOST` environment variable after you call `env-init`? – Ed Davisson May 23 '17 at 18:44
  • 1
    While I'm always one for local testing and dev serving, the Google Cloud environment is so hopelessly convoluted that I definitely wouldn't recommend using their platform emulation packages. Bugs like this, as well as inconsistencies in behavior with the actual deployment environment, make it a headache not worth having; unless there's a reason you can't, I'd suggest just using a staging project in the cloud environment. – kungphu May 24 '17 at 07:27
  • Probably, you may want to double check your emulator setup following steps given in here: https://cloud.google.com/datastore/docs/tools/datastore-emulator – Jadav Bheda May 25 '17 at 23:35
  • Not sure if this is going to help you either: https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1837 – Jadav Bheda May 25 '17 at 23:48

2 Answers2

1

You need to eval $(gcloud beta emulators datastore env-init).

gcloud beta emulators datastore env-init only prints the commands that set the necessary environment variables.

brocoli
  • 542
  • 3
  • 16
0

You can try something like 

if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/'):
  # Production
else:
  # Local development server

You can follow more here, https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server

skjagini
  • 3,142
  • 5
  • 34
  • 63