Chalice on_s3_event trigger seems to not work

Question

I have a Chalice app that reads config data from a file in an S3 bucket. The file can change from time to time, and I want the app to immediately use the updated values, so I am using the on_s3_event decorator to reload the config file.

My code looks something like this (stripped way down for clarity):

CONFIG = {}
app = Chalice(app_name='foo')

@app.on_s3_event(bucket=S3_BUCKET, events=['s3:ObjectCreated:*'],
    prefix='foo/')
def event_handler(event):
    _load_config()

def _load_config():

    # fetch json file from S3 bucket

    CONFIG['foo'] = some item from the json file...
    CONFIG['bar'] = some other item from the json file...

_load_config()

@app.route('/')
def home():
    # refer to CONFIG values here

My problem is that for a short while (maybe 5-10 minutes) after uploading a new version of the config file, the app still uses the old config values.

Am I doing this wrong? Should I not be depending on global state in a Lambda function at all?

have you turned on the notification? [Doc](https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html) — Lamanus, Oct 17 '19 at 14:45
It's my understanding that Chalice does that for you, when it sees the `on_s3_event` decorator. I also have a log message (that I didn't show) inside the event handler function, and that message does appear in my logs. — John Gordon, Oct 17 '19 at 14:55

score 0 · Answer 1 · answered Dec 16 '19 at 20:44

So your design here is flawed.

When you create an S3 Event in chalice it will create a separate Lambda function for that event. the CONFIG variable would get updated in the running instance of that Lambda function and all new instances of the Lambda function. However any other Lambdas in your Chalice app that are already running would just continue on with their current settings until they were cleaned up and restarted.

If you cannot live with a config that is only changeable when you deploy your Lambda functions you could use redis or some other in memory cache/db.

score 0 · Answer 2 · answered Jan 28 '20 at 20:40

You should be using the .config/config.json file to store your variables for your chalice application. Those variables are stored in the os library and can be called:

URL = os.environ['MYVAR']

Your config.json file might look like this:

{
  "version": "2.0",
  "app_name": "MyApp",
  "manage_iam_role": false,
  "iam_role_arn": "arn:aws:iam::************:role/Chalice",
  "lambda_timeout": 300,
  "stages": {
    "development": {
      "environment_variables": {
        "MYVAR": "foo"
      }
    },
    "production": {
      "environment_variables": {
        "MYVAR": "bar"
      }
    }
  },
  "lambda_memory_size": 2048
}

Chalice on_s3_event trigger seems to not work

2 Answers2