RabbitMQ - How many queues can RabbitMQ handle on a single server?
There are not any hard-coded limits inside RabbitMQ broker. The broker will utilize all available resources (unless you set limits on some of them, they are called watermarks in RabbitMQ terminology).
There are some limitations put by Erlang itself, like maximum number of concurrent processes, but if you theoretically can reach them on single node then it is always good idea to use distributed features.
There are a lot of discussions about RabbitMQ resource usage and limits,
- How many queues can one broker support on RabbitMQ mailing list
- Max messages allowed in a queue in RabbitMQ? on RabbitMQ mailing list
- Rabbitmq - Reasonable performance/scale expectations on Server Fault
- Is there a limit to the number of exchanges for rabbitmq? on Stack Overflow
P.S. There are AMQP protocol limit though. They are described in section 4.9 Limitations
The AMQP specifications impose these limits on future extensions of AMQP or protocols from the same wire-level format:
- Number of channels per connection: 16-bit channel number.
- Number of protocol classes: 16-bit class id.
- Number of methods per protocol class: 16-bit method id.
The AMQP specifications impose these limits on data:
- Maximum size of a short string: 255 octets.
- Maximum size of a long string or field table: 32-bit size.
- Maximum size of a frame payload: 32-bit size.
- Maximum size of a content: 64-bit size.
The server or client may also impose its own limits on resources such as number of simultaneous connections, number of consumers per channel, number of queues, etc. These do not affect interoperability and are not specified.
This post can help you:
http://rabbitmq.1065348.n5.nabble.com/Max-messages-allowed-in-a-queue-in-RabbitMQ-tp26063p26066.html
- Max queues allowed in RabbitMQ?
Thousands (or even tens of thousands) of queues should be no problem at all, though each object (e.g., queues, exchanges, bindings, etc) will take up some memory and/or disk space. By default, Erlang will enforce a maximum number of concurrent processes (i.e., lightweight threads) at around 32768 IIRC. Each queue is managed by its own process and each connection can result in several more, so if you're planning on having a very large number of active queues in a single node (?) and using them all at the same time, then you may need to tweak the emulator arguments rabbit passes the VM by setting +P <new limit> to a higher limit.
You're also likely to use up many Gb just with the overhead for each queue / connection pretty fast, so you're going to need a pretty meaty server to handle millions of both. Tens of thousands should be no problem at all, providing they fit into RAM.