DynamoDB w/ Serverless, using Fn::GetRef to reference global secondary index

Question

I have an API/service I'm defining with a DynamoDB table. I have a couple indexes (defined as Global Secondary Index) to support a couple queries. I have the table designed, with the GSI definitions, and what looks like correct queries. However, I get this exception when making the query:

{ AccessDeniedException: User: arn:aws:sts::OBSCURED:assumed-role/chatroom-application-dev-us-east-1-lambdaRole/chatroom-application-dev-getRoomMessages is not authorized to perform: dynamodb:Query on resource: arn:aws:dynamodb:us-east-1:OBSCURED:table/messages-table-dev/index/roomIndex
at Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:48:27)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
message: 'User: arn:aws:sts::OBSCURED:assumed-role/chatroom-application-dev-us-east-1-lambdaRole/chatroom-application-dev-getRoomMessages is not authorized to perform: dynamodb:Query on resource: arn:aws:dynamodb:us-east-1:OBSCURED:table/messages-table-dev/index/roomIndex',
code: 'AccessDeniedException',
time: 2018-06-02T22:05:46.110Z,
requestId: 'OBSCURED',
statusCode: 400,
retryable: false,
retryDelay: 30.704899664776054 }

At the top of the exception it says the ARN for my getRoomMessages method is not authorized to perform: dynamodb:Query on resource: and shows the ARN for the global secondary index.

It seems clear, I need to define the policy to grant rights to access the global secondary index. But it's not at all clear how. I've seen other StackOverflow questions about DynamoDB complain about the fragmented documentation and how difficult it is to find anything. I have to agree. The word "fragmented" is putting it too mildly.

I am using the Serverless Framework. The provider section shows this policy/role definition:

provider:
  name: aws
  runtime: nodejs8.10
  stage: dev
  region: us-east-1
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource:
        - { "Fn::GetAtt": ["MessagesDynamoDBTable", "Arn" ] }
        - { "Fn::GetAtt": ["#roomIndex", "Arn" ] }
        - { "Fn::GetAtt": ["#userIndex", "Arn" ] }
  environment:
    MESSAGES_TABLE: ${self:custom.tableName}

In the Resource section, I believe I'm supposed to list the resources for which the permissions are declared. The first references the table as a whole. The last two I just added, and reference the indexes.

EDIT: When I run serverless deploy the following message is printed:

The CloudFormation template is invalid: Template error: instance of Fn::GetAtt references undefined resource #roomIndex

I tried several variations on this only to get the same error. What this boils down to is - how do I, in the serverless.yml, using Cloudfront syntax, get the ARN for the indexes. The ARN does exist because it is shown in the exception.

The DynamoDB table definition:

resources:
  Resources:
    MessagesDynamoDBTable:
      Type: AWS::DynamoDB::Table
      Properties:
        AttributeDefinitions:
          - AttributeName: messageId
            AttributeType: S
          - AttributeName: room
            AttributeType: S
          - AttributeName: userId
            AttributeType: S
        KeySchema:
          - AttributeName: messageId
            KeyType: HASH
        GlobalSecondaryIndexes:
          - IndexName: roomIndex
            KeySchema:
              - AttributeName: room
                KeyType: HASH
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
          - IndexName: userIndex
            KeySchema:
              - AttributeName: userId
                KeyType: HASH
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:custom.tableName}

The query being used corresponding to the above exception:

{
    "TableName": "messages-table-dev",
    "IndexName": "roomIndex",
    "KeyConditionExpression": "#roomIndex = :room",
    "ExpressionAttributeNames": {
        "#roomIndex": "room"
    },
    "ExpressionAttributeValues": {
        ":room": {
            "S": "everyone"
        }
    }
}

And the Lambda function code snippet which generates the query:

app.get('/messages/room/:room', (req, res) => {
    const params = {
        TableName: MESSAGES_TABLE,
        IndexName: "roomIndex",
        KeyConditionExpression: '#roomIndex = :room',
        ExpressionAttributeNames: { '#roomIndex': 'room' },
        ExpressionAttributeValues: {
            ":room": { S: `${req.params.room}` }
        },
    };
    console.log(`QUERY ROOM ${JSON.stringify(params)}`);
    dynamoDb.query(params, (error, result) => {
        if (error) {
            console.log(error);
            res.status(400).json({ error: 'Could not get messages' });
        } else {
            res.json(result.Items);
        }
    });
});

David Herron · Accepted Answer · 2018-06-05T05:46:20.993

I found a better answer than was posted by jake.lang. EDIT: I didn't see his second comment in which he suggested the following.

As he noted, his was incorrect since the ARN can change for valid reasons. However, the solution arose because the ARN for a Global Secondary Index is to append "/INDEXNAME" to the ARN for the table. This means the policy statement can be:

iamRoleStatements:
  - Effect: Allow
    Action:
      - dynamodb:Query
      - dynamodb:Scan
      - dynamodb:GetItem
      - dynamodb:PutItem
      - dynamodb:UpdateItem
      - dynamodb:DeleteItem
    Resource:
      - { "Fn::GetAtt": ["MessagesDynamoDBTable", "Arn" ] }
      - { "Fn::Join": [ "/", [ 
          { "Fn::GetAtt": ["MessagesDynamoDBTable", "Arn" ] }, "index", "roomIndex" 
        ]]}
      - { "Fn::Join": [ "/", [
          { "Fn::GetAtt": ["MessagesDynamoDBTable", "Arn" ] }, "index", "userIndex" 
        ]]}

The "Fn::Join" bit is from CloudFormation, and is a "join" operation. It takes an array of strings, concatenating them using the first argument. Hence it is a rather convoluted and overly complex method to calculate the ARN's required in this policy statement.

For documentation, see: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-join.html

score 11 · Answer 2 · answered May 17 '19 at 07:29

Instead of Fn::Join you can use !Sub "${MessagesDynamoDBTable.Arn}", it's simpler. Moreover if you want to access all indexes (that's usually my case), then /index/* is all you need.

Example:

...
      Policies:
        - Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - dynamodb:Query
              Resource:
                - !Sub "${MessagesDynamoDBTable.Arn}"
                - !Sub "${MessagesDynamoDBTable.Arn}/index/*"
...

This approach worked the best for me. Clean and readable and carries a dynamic reference to the resource — ChrisRich, Oct 30 '22 at 13:03

score 1 · Answer 3 · answered Jun 02 '18 at 23:04

1

Here's how I've declared my working GSI permissions in serverless. I'm not sure but maybe the issue is you need to declare actions for each resource independently?

iamRoleStatements: 
  - Effect: Allow
    Action:
      - dynamodb:Query
      - dynamodb:GetItem
      - dynamodb:PutItem
    Resource: "arn:REDACTED:table/TABLENAME”
  - Effect: Allow
    Action:
      - dynamodb:Query
    Resource: "arn:REDACTED:table/TABLENAME/index/INDEXNAME”

This hardcoding of the arn is probably not the best thing to do but it's worked fine for my development thus far.

answered Jun 02 '18 at 23:04

Jacob Lange

1,299
14
22

Hardcoding the ARN in the serverless.yml doesn't seem like the right way to go. The ARN can change for valid reasons, yes? I tried `{ "Fn::GetAtt": ["MessagesDynamoDBTable/index/roomIndex", "Arn" ] }` -- the benefit of Fn::GetAtt is it dynamically retrieves the value. But I got this error: `The CloudFormation template is invalid: Template error: instance of Fn::GetAtt references undefined resource MessagesDynamoDBTable/index/roomIndex` – David Herron Jun 04 '18 at 00:03
Have you tried `- { "Fn::Join" : [ { "Fn::GetAtt" :["MessagesDynamoDBTable", "Arn" ] }, "index/roomIndex"] }` ? you can get the MessagesDynamoDBTable ARN right? then the GSI ARN will just be that string + "index/roomIndex" – Jacob Lange Jun 04 '18 at 05:12

DynamoDB w/ Serverless, using Fn::GetRef to reference global secondary index

3 Answers3