Pop multiple values from Redis data structure atomically?

Use LRANGE with LTRIM in a pipeline. The pipeline will be run as one atomic transaction. Your worry above about WATCH, EXEC will not be applicable here because you are running the LRANGE and LTRIM as one transaction without the ability for any other transactions from any other clients to come between them. Try it out.

Starting from Redis 3.2, the command SPOP has a [count] argument to retrieve multiple elements from a set.

See http://redis.io/commands/spop#count-argument-extension

Here is a python snippet that can achieve this using redis-py and pipeline:

from redis import StrictRedis

client = StrictRedis()

def get_messages(q_name, prefetch_count=100):
    pipe = client.pipeline()
    pipe.lrange(q_name, 0, prefetch_count - 1)  # Get msgs (w/o pop)
    pipe.ltrim(q_name, prefetch_count, -1)  # Trim (pop) list to new value
    messages, trim_success = pipe.execute()
    return messages

I was thinking that I could just do a a for loop of pop but that would not be efficient, even with pipeline especially if the list queue is smaller than prefetch_count. I have a full RedisQueue class implemented here if you want to look. Hope it helps!

To expand on Eli's response with a complete example for list collections, using lrange and ltrim builtins instead of Lua:

127.0.0.1:6379> lpush a 0 1 2 3 4 5 6 7 8 9
(integer) 10
127.0.0.1:6379> lrange a 0 3        # read 4 items off the top of the stack
1) "9"
2) "8"
3) "7"
4) "6"
127.0.0.1:6379> ltrim a 4 -1        # remove those 4 items
OK
127.0.0.1:6379> lrange a 0 999      # remaining items
1) "5"
2) "4"
3) "3"
4) "2"
5) "1"
6) "0"

If you wanted to make the operation atomic, you would wrap the lrange and ltrim in multi and exec commands.

Also as noted elsewhere, you should probably ltrim the number of returned items not the number of items you asked for. e.g. if you did lrange a 0 99 but got 50 items you would ltrim a 50 -1 not ltrim a 100 -1.

To implement queue semantics instead of a stack, replace lpush with rpush.