Querying a Global Secondary Index in dynamodb Local

Question

I am creating a table and GSI in DynamoDB, using these parameters, as per the documentation:

configId is the primary key of the table, and I am using the publisherId as the primary key for the GSI. (I've removed some unnecessary configuration parameters for brevity)

var params = {
    TableName: 'Configs',
    KeySchema: [ 
        {
            AttributeName: 'configId',
            KeyType: 'HASH',
        }
    ],
    AttributeDefinitions: [
        {
            AttributeName: 'configId',
            AttributeType: 'S',
        },
        {
            AttributeName: 'publisherId',
            AttributeType: 'S',
        }
    ],
    GlobalSecondaryIndexes: [ 
        { 
            IndexName: 'publisher_index', 
            KeySchema: [
                {
                    AttributeName: 'publisherId',
                    KeyType: 'HASH',
                }
            ]
        }
    ]
};

I am querying this table using this:

{ TableName: 'Configs',
  IndexName: 'publisher_index',
  KeyConditionExpression: 'publisherId = :pub_id',
  ExpressionAttributeValues: { ':pub_id': { S: '700' } } }

but I keep getting the error:

"ValidationException: One or more parameter values were invalid: Condition parameter type does not match schema type"

In the docs it specifies that the primary KeyType can either be HASH or RANGE, and that you set the AttributeType in the AttributeDefinitions field. I am sending the publisherId as String, not sure what I am missing here.

Is the issue in the way I am creating the table, or the way I am querying? Thanks

I would make a describeTable call on the 'Configs' to double check that the secondary index has been created as expected. — Ryan Fitzgerald, Nov 24 '15 at 01:02

score 103 · Accepted Answer · edited Sep 17 '20 at 11:14

103

Try to change ExpressionAttributeValues from this

{ 
 TableName: 'Configs',
 IndexName: 'publisher_index',
 KeyConditionExpression: 'publisherId = :pub_id',
 ExpressionAttributeValues: { ':pub_id': { S: '700' } } 
}

to

{ 
 TableName: 'Configs',
 IndexName: 'publisher_index',
 KeyConditionExpression: 'publisherId = :pub_id',
 ExpressionAttributeValues: { ':pub_id': '700'} 
}

From { ':pub_id': { S: '700' } } to { ':pub_id': '700'}.

I had the same problem and I spent 2 days for this. The official documentation in misleading.

edited Sep 17 '20 at 11:14

Gopesh

3,882
11
37
52

answered Apr 23 '16 at 11:07

gior91

1,745
1
16
19

Sorry, I tried this but it does not work for me. When I do not provide something with the datatype, like in: { ':pub_id': { S: '700' } }, I get a validation error. Are you sure you are not talking about ExpressionAttributesNames instead of ExpressionAttributesValues? – Hatim Khouzaimi Apr 29 '16 at 17:53
The correct keyword is `ExpressionAttributeValues` and not AttributeS. Are you sure that your ValidationException is not dued to this little mistake? In my case I don't used the `ExpressionAttributeNames` parameter and that worked. – gior91 Apr 30 '16 at 19:13
1

@gior91 Cool! This really helped me. Thanks! – ckuijjer Jun 07 '16 at 19:48
@RobertOschler you're welcome! I'm surprised that after ~3 years this error is still present!!! – gior91 Feb 16 '19 at 20:05
@gior91 Sign of the times. Seeing the same thing in blockchain. The platforms change so fast, the docs can't keep up or worse, there's multiple sets for multiple version, but the old docs naturally rise to the top of the search engines over the new ones due to their larger more mature link networks. – Robert Oschler Feb 17 '19 at 02:25
4

This is because when you use the `DocumentClient`, you no longer need the "S" to indicate that it's a string. Under the hood the API takes that data, but the `DocumentClient` abstracts it away from you in Node. https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html – Jack Ryan Oct 14 '20 at 05:22

score 22 · Answer 2 · edited Jun 20 '20 at 09:12

Turns out it depends on whether you use AWS.DynamoDB or AWS.DynamoDB.DocumentClient.

Check the difference in the documentation: AWS.DynamoDB.query vs. AWS.DynamoDB.DocumentClient.query

In the DocumentClient the doc clearly states:

The document client simplifies working with items in Amazon DynamoDB by abstracting away the notion of attribute values. This abstraction annotates native JavaScript types supplied as input parameters, as well as converts annotated response data to native JavaScript types.

...

Supply the same parameters as AWS.DynamoDB.query() with AttributeValues substituted by native JavaScript types.

Maybe you where also referring to the DynamoDB API Reference which in fact makes no assumptions about the used SDK but uses plain HTTP Requests in the examples.

Thus using the AWS.DynamoDB.DocumentClient you would just provide a simple key-value map for ExpressionAttributeValues as suggested by @gior91.

score 7 · Answer 3 · edited May 22 '17 at 22:01

As stated above, you need to use the DynamoDB Document Client if you want to abtract away the type casting.

var docClient = new AWS.DynamoDB.DocumentClient();

...then you can just use the above listed object notation to call the API.

{ ':pub_id': '700'}

Ran into this issue myself, I was using DynamoDB() in some places and the docClient in others, couldn't figure it out for awhile, but that'll solve it.

score 2 · Answer 4 · answered Jun 13 '21 at 22:14

2

I found the other answers very helpful, but I still couldn't get the ValidationExceptions to go away... eventually I realized that I'd been using the DocumentClient's .get instead of .query

(Technically not an answer to the question, but posting this here rather than in a comment to make it more visible to anyone else struggling with such a marvellously unhelpful error message.)

answered Jun 13 '21 at 22:14

therightstuff

833
1
16
21

1

Well you actually helped me there, that was the origin of my error. Thanks ! – AirOne Apr 24 '22 at 21:31

score 2 · Answer 5 · answered Oct 01 '22 at 21:27

Maybe I am late, but I'd like to share a method that will guarantee a correct query param.

So,

go to your aws console, select dynamodb service,
in the left, it's the dynamodb menu and click "Explore items".
And then, choose the table you are interested in.
after selecting the table, in the right, the top will show "Scan/Query items". Please expand it
choose "Query" tab and in the dropdown, select your index
then, input a value, and click the button "Run"
ATTENTION: In the network of chrome, the "Fetch/XHR" tab will show a http request. Please see the payload. It's the query you need. Of course, you can try to remove some fields you don't need, such as "Limit"
How to use the query param? await documentClient.query(params).promise();

If you are still confused, see the screenshot below:

score 0 · Answer 6 · answered Jan 12 '23 at 09:42

This is technically not an answer to the question asked, but I am posting this here to make it visible to anyone else dealing with the validation error.

I was getting the "query condition missed key schema element id" ValidationError.

The problem was that my QueryCommand was missing "IndexName" field. After adding it, the command worked.

What helped me figure it out was the documentation - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#query-property

score -3 · Answer 7 · answered Apr 29 '16 at 17:46

Try replacing your query JSON with this:

{
  TableName: 'Configs',
  IndexName: 'publisher_index',
  KeyConditionExpression: '#publisherId = :pub_id',
  ExpressionAttributeNames: { '#publisherId': 'publisherId' },
  ExpressionAttributeValues: { ':pub_id': { 'S': '700' } }
}

Querying a Global Secondary Index in dynamodb Local

7 Answers7

Linked