nServiceBus vs Mass Transit vs Rhino Service Bus vs other?
a potential con of anything MSMQ based is the restriction on maximum message size. IIRC it is approximately 4MB, which you might easily run into if you're dealing with large files and storing the file content within the message.
An update to the state of Rhino vs NServicebus:
http://www.infoq.com/news/2012/04/nservicebus3-0
InfoQ to Ayende: You have previously written a service bus for .NET yourself, namely the Rhino Service Bus. Should users of Rhino Service Bus now reconsider and move to NServiceBus?
Ayende: I built Rhino Service Bus around 2008. I built it mostly because I wasn't happy with the state of the other service buses at the time. I have had different concerns and direction when building my service bus, but that was 4 years ago. In that time, I think that NServiceBus made great strides in becoming an easier to use product and having a much better out of the box development story. If I was starting out with service buses today, I strongly doubt that I would be building my own.
NServiceBus is a good product but beware of licensing problems. It has a tendency to change it licensing policy as the authors wishes. Take a look for example at old license information.
It could happen that in the middle of you project development you will find out that you have to pay lot's of money for NServiceBus.
Also free version have performance limitations.
MassTransit is absolutely free open source, it has no limitations, and is under Apache 2.0 license.
I have not used Rhino Service Bus.
I'd recommend staying away from hand-rolled solutions as there is a bunch of somewhat difficult stuff that needs to be gotten just right - like how transactions are handled, how exceptions cause rollbacks, how to stop rolling back endlessly (poison messages), how to integrate with long-running workflows so that the state management boundaries line up, and more.
You will probably want some kind of durable/transactional messaging infrastructure, so not using MSMQ you'd be left with Service Broker on the Microsoft platform, or some other alternative like ActiveMQ. MSMQ has the benefit of already being installed on all Windows machines, as opposed to Service Broker which isn't.
In terms of choosing between NServiceBus, Mass Transit, and Rhino Service Bus - this Stackoverflow answer comparing NServiceBus to MassTransit would be a good place to start..
In our 3.1 release, we're introducing NSB Studio - a set of Visual Studio integrated modeling tools that enable you to model your system at a higher level of abstraction and have much of the configuration and initialization of NServiceBus be done for you automatically. I'd say that this really tips the scales in favor of NServiceBus.
Hope that helps.
Disclaimer: I am the author of NServiceBus.