DynamoDB Global Secondary Index with Exclusive Start Key

OK I'm super late to the party but I have figured out what is going on. This isn't a bug, it's working as it should, but I've never seen it in the documentation.

It turns out that in global secondary indexes, the primary indexes are used as "tiebreakers." That is, if two objects have the same GSI hash+sort keys, then the primary indexes are used to order them in the GSI. That means that when you query a GSI with an exclusive start key, you need both the GSI indexes and the primary indexes in order to start at the exact right place.

Maybe this will help out somebody. I know it stumped me for a while!

Had the same issue and just got sorted. :) Too late to answer the question but hope someone will find helpful.

When you query or scan table with secondary indexes and pagination, you should include primary keys of the table and the index (as key), with last evaluated values (as attribute value) when you setting ExclusiveStartKey.

Just Sysout the LastEvaluatedKey from the query or scan result to see the format.

// let's just assume that we have a table to store details of products
Map<String, AttributeValue> exclusiveStartKey = new HashMap<String, AttributeValue>();
// primary key of the table
exclusiveStartKey.put("productId", new AttributeValue().withS("xxxx"));
exclusiveStartKey.put("produtSize", new AttributeValue().withS("XL"));
// primary key of the index
exclusiveStartKey.put("categoryId", new AttributeValue().withS("xx01"));
exclusiveStartKey.put("subCategoryId", new AttributeValue().withN("1"));

Per an Amazonian, this is not possible: https://forums.aws.amazon.com/thread.jspa?threadID=146102&tstart=0

A workaround that worked for my use case, though, was to just specify a RangeKeyCondition greater than the last retrieved object's timestamp. Here's the idea:

Condition hashKeyCondition = new Condition();
hashKeyCondition.withComparisonOperator(ComparisonOperator.EQ).withAttributeValueList(new AttributeValue().withS(hashKeyAttributeValue));

Condition rangeKeyCondition = new Condition();
rangeKeyCondition.withComparisonOperator(ComparisonOperator.GT).withAttributeValueList(new AttributeValue().withN(timestamp.toString()));

Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put(MappedItem.INDEXED_ATTRIBUTE_NAME, hashKeyCondition);
keyConditions.put(MappedItem.TIMESTAMP, rangeKeyCondition);


QueryRequest queryRequest = new QueryRequest();
queryRequest.withTableName(tableName);
queryRequest.withIndexName(MappedItem.INDEX_NAME);
queryRequest.withKeyConditions(keyConditions);

QueryResult result = amazonDynamoDBClient.query(queryRequest);

List<MappedItem> mappedItems = new ArrayList<MappedItem>();

for(Map<String, AttributeValue> item : result.getItems()) {
    MappedItem mappedItem = dynamoDBMapper.marshallIntoObject(MappedItem.class, item);
    mappedItems.add(mappedItem);
}

return mappedItems;

Note that the marshallIntoObject method is deprecated in favor of a protected method in the DynamoDBMapper class, but it's easy enough to write a marshaller were a future upgrade to break the mapping.

Not as elegant as using the mapper but it accomplishes the same thing.