Why would you use a message based system?

A message-based architecture de-couples producers and consumers of messages, both in time and space. This has a lot of benefits:

• producers and consumers can run on different machines
• producers and consumers can run at different times.
• producers and consumers can run on different hardware/software platforms (they only need to understand the same message protocol)
• it's easy to coordinate multiple producers / consumers (e.g. for compute-intensive jobs that need multiple machines, as Dave Markle has described)
• higher stability when services are temporarily unavailble (e.g. when doing order processing, using a messaging system can help avoid "dropping" orders)

You lose most of these benefits when you do RPC-style communication (i.e. when you block while waiting for service responses)

There are multiple advantages to using message based systems.

1. Messages form a well defined technology neutral interface between applications.
2. Enables loose coupling of applications.
3. Lots of options for performance, tuning and scaling:
   • Deploy requester and service process on different hardware
   • Multiple requesters sharing single server
   • Multiple requesters sharing multiple servers
4. The various messaging middlewares implement the common messaging patterns independently from you application.
   • Request/Reply
   • Fire and Forget offline updates
   • Publish/Subscribe
5. Many of the middleware products handle message transformation (e.g. SWIFT to SWIFTXML).
6. Many of the middleware products can decompose a single large request into several smaller requests.
7. They nearly all support multiple platforms.

Incidentally the two market leaders in this area are IBM with their Websphere MQ and related products, and, TIBCO with their Enterprise Service Bus.

the benefits are really down to decoupling the parts of your application. Once you have the bus set up, and applications are added, you can easily extend your app by adding new pieces that you can guarantee will not affect the other parts. Its a very good way of continually adding to a system over time.

eg. we have a system like this, each command is implemented as a part to the overall GUI, if we want to add a new feature, or modify an existing one, we just write a new part and add it to the system. When it gets called, it has no dependencies on the rest. It means we can extend our app very easily. Also, we have a message passing between nodes on our network - when something gets changed in 1 computer, a message is sent to all others so they can update themselves. We have a hundred different messages for different events, so we can extend the system by dropping a new service in that reacts to the appropriate messages.

Message passing architectures typically have the same features as web services, you have discrete services you can call, you can add new ones easily.

Don't think that message passing architectures require fancy (and expensive!) middleware products though, Windows runs on a message passing architecture - every WM_* message passed to a window is.. well, a message, and I think that shows the best example of the architecture - no part of a system needs to know about any other part, you can extend it infinitely as you can handle as many controls as you like on any dialog, etc etc.

Message passing is a fabulous architecture, though it can be slower than tightly coupling your application together, that's not much of a reason not to use it nowadays especially if you're already using scripting or .net apps.

One use case is when you have a pool of resources that can work on a given item, and a list of work which needs to be distributed in a scalable fashion.

I once had a project where I had to do mainframe integration with a number of 3270 screen scrapers (all slow). I could have at most 10 of these processes open on a box at a time. I had thousands of accounts to screen-scrape and update, so I put the work in a queue, and my machines (I had about 3 of em) just picked up work items off the message queue (MSMQ) and did their screen scraping, and that was it. I could easily spin up a new machine, or deactivate old ones without interrupting the flow of work, so that was nice.