Finding certain messages in SQS

I do not think the short or long answers are "No".

Here are two counterpoint solutions that are "Yes".

  1. Traversing the Queue, maintaining a visited list
  2. Using Enterprise Integration Patterns (message routing) to split your messages into downstreams based on criteria

1. Traversing the Queue, maintaining a visited list

Consider the case of a queue with N messages, with no messages being added or deleted. Without additional information (e.g. if you knew how many messages should match your criteria), you need to traverse all N messages.

The key point here is knowing when you've traversed all N messages. There are some issues here.

  1. To know exactly, you'd have to track messages as they are added to the queue
  2. To know approximately, you can get the ApproximateNumberOfMessages attribute of the queue
  3. Or you could receive messages in a loop, maintaining a visited list, and assume that you will eventually sample and exhaust messages from each server your queue is sharded over

To maintain the visited list, as you receive messages and evaluate your match criteria, you could store the message_id of all visited messages.

Message ID's are nearly unique. See this thread

https://forums.aws.amazon.com/message.jspa?messageID=76119

If you went with (3), you wouldn't be certain how many iterations would be required to exhaust the queue. However, if you performed this indefinitely, you'd be guaranteed to exhuast the queue so long as the weighted random distribution over the SQS shard servers gives them all non-zero probability.

2. Using Enterprise Integration Patterns (message routing) to split your messages into downstreams based on criteria

If you have control over your messaging architecture you could use a Message Router as a "front-end" message processor that dispatches messages to various recipients based on criteria.

And specifically you'd use a Content-Based Router:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/ContentBasedRouter.html


Short answer: no.

Queues are designed for things like tasks. A machine grabs a new task (i.e., message) from the queue, executes the task, then deletes the task.

If you're trying to search the messages to filter them, I can't help but wonder if you're using the wrong tool for the job…