How to retrieve more than 100 Records from Amazon Simple DB

Question

I am Creating an application in MVC and using Amazon cloud service for backend. I need the Data in Bulk from Database like so I am using the Query like this-

  SelectResponse response = simpleDBClient.Select(new SelectRequest()
  {
      SelectExpression =  "select * from survey1 limit 2400"

  });

which is now working fine and returning 2400 Records. Now I want to Apply Search on these Records So I must have to use where clause but when I am using where clause then its returning only 10 Records in any Valid Condition.

Please Help me Any Help will be Appreciated.

Answer 1

You can use limit with the where clause. For more detail here is the Syntax for the Select Query-

select output_list from domain_name [where expression] [sort_instructions] [limit limit]

The output_list can be: , itemName(),count(), list of attributes

1. * for all attributes.
2. `itemName()` for the item name only.
3. count(*) for the total count of items matches the query expression. It will return the number of items in a result set instead of returning the items.
4. An explicit list of attributes (attribute1,..., attributeN)

The domain_name is the domain from which you want to search items.

The expression is the match expression for items. You can use select expressions like =, <=, <, > =, like, not like, between, is null, is not null etc.

The sort_instructions sorts the results on a single attribute, in an ascending or descending order.

The limit is the maximum number of results to return (default: 100, max. 2500).

Please Note-

The total size of the response cannot exceed 1 MB. Amazon SimpleDB automatically adjusts the number of items returned per page to enforce this limit. For example, even if you ask to retrieve 2500 items, but each individual item is 10 KB in size, the system returns 100 items and an appropriate next token so you can get the next page of results.

Note: Operations that run longer than 5 seconds return a time-out error response or a partial or empty result set. Partial and empty result sets contain a NextToken value, which allows you to continue the operation from where it left off.

Source

Answer 2

SimpleDB can only retrieve only 1MB of data at a time, SimpleDB puts a nextToken in response to indicate that there is more data to be retrieved.

Here is how do it in Java.

public List<Item> getItems(){
    AmazonSimpleDBClient client = new AmazonSimpleDBClient(...);
    List<Item> items = new ArrayList<>();
    String nextToken = null;

    do {
        final SelectRequest request = new SelectRequest();
        request.setSelectExpression("MY SELECT QUERY");

        // SimpleDB can paginate the result. For paginated result NextToken value will be not null
        request.setNextToken(nextToken);
        nextToken = null;

        SelectResult result = client.select(request);

        if (result != null) {
            nextToken = (result.getNextToken() != null) ? result.getNextToken() : null;
            items.addAll(result.getItems());
        }
    } while (nextToken != null);

    return items;
}