Message Oriented Middleware and Open Protocols

Something that I was recently reminded to post on (due to a comment on my post on Progress and Iona) was the work that I had done into AMQP recently (encompassing RabbitMQ, Qpid, et al).

As some of you might have gathered, I'm a SonicMQ user. I'm a big fan of SonicMQ as a core product. The work that they've done on the core message broker has been phenomenal, and compared to anything else in the market, it's amazing. I'm quite pleased with many parts of the commercial decision that my company made (based on my technical input) to make a very substantial investment in SonicMQ. So don't let anything I'm saying act as SonicMQ bashing: if you are in the market for a commercial, centralized-broker-based MOM system, buy SonicMQ. With the caveats you'll get below.

The one thing that I will pretty openly criticize SonicMQ for (aside from their installers, which are openly toxic and should be approached with nothing other than open hostility) is their non-Java client libraries.

Quite simply put, their C++ client support is precisely what I would expect from a Java-based company: it's sporadic, not well engineered, looks exactly like Java translated to C++ (rather than idiomatic C++ using things like Smart Pointers and the like), and the porting is pretty abysmal (they seem to think that "Solaris" is one unified thing, rather than properly understanding the difference between hardware platforms, OS versions, compilers and versions). Their C# support ain't much better to be fair.

But it shouldn't have to be that way. Quite frankly, I'm shocked at this point that there isn't some type of IETF-supported protocol for MOM in its various forms.

So when I was alerted to AMQP, I was quite excited. The basic premise of AMQP is to provide a vendor neutral wire-level binary protocol for asynchronous, broker-based message oriented middleware. Huzzah! It's all good! We're saved! SonicMQ supports this, C++ domain experts write the client libraries, I can get Python native libraries, and I get the best of both worlds: client libraries in all the languages and environments I have to support, and the best broker-based middleware I can find.

Alas, it's not to be. While it might be time for a vendor-neutral middleware protocol, it doesn't appear that AMQP is it.

To start with, a little birdie who knows many of the LShift people (who are partially responsible for RabbitMQ) indicates that the protocol discussions are degenerating into SQL-99-level vendor arguments, before 1.0 of the protocol has even been ratified. All the vendors of pre-1.0 products are already jockeying for position. And I thought SQL was bad; but at least they all had commercial products out when they started fighting in committee! So the protocol itself seems stillborn by its own vendor quibbles.

But then there's the fact that Merrill LynchJPMorgan (one of the major client drivers of the protocol) pushed for something that doesn't really make sense. It's almost as though the protocol requires an implementation that's half bog-standard JMS, and half Tibco EMS, and all confusing. SonicMQ, for example, has IMHO, the best approach to overcoming the insanity of the JMS specification: Shared Subscriptions. But rather than allowing for vendor differentiation through features like this (and if you're evaluating MOM, you have to fully understand the power this gives you, and is unique to SonicMQ), the AMQP defines a MOM model that nobody currently implements, and is highly unintuitive.

Why, oh, why, couldn't they have just taken something like JMS, taken the edges off it (introspectible queues with online reordering? W-T-F?), and done that as a 1.0? Then at least they'd have a basis for something that would be commonly understood by MOM professionals, and would have had a scope for a vendor community to build implementation support.

Instead we face a situation where we have, simultaneously:

• A protocol nobody supports; based on


• A programming model nobody is used to; based on


• A conceptual model nobody has worked with; with


• No existing functioning reference implementations

This is not the way to build a community! It's classic chicken-and-egg. Give some existing stakeholder some reason to work with it, whether it's a vendor or an MOM professional or just an existing J2EE developer. You can't go from nothing to functioning ecosystem by redefining the world around you.

For example, I grabbed RabbitMQ and the Qpid JMS client (because it was the only one I could find). You'd think I'd be able to point the Qpid JMS client (built just on AMQP) at RabbitMQ and all would be well, right? Nope. Didn't work at all, never figured out why, nobody in the LazyWeb had ever tried such an insane thing. (You use the Qpid clients against Qpid! Not another AMQP product! That would be crazy!)

So my point is that while I think that open protocols for things like this are critical (I'll have a post on open protocols for RDBMS connectivity), this isn't it, for a variety of reasons. I wish it was, and I've pushed for Progress Software to support AMQP as an in-broker connector (entirely possible with their internal architecture), but I doubt it's going to happen.

Do I think SonicMQ is the end-all, be-all of MOM? Heck no. There's a lot of stuff they do badly. But I wish I could evaluate them as a broker rather than having to keep thinking in the back of my mind "no native cPython support; no GCC 4 Intel OpenSolaris x86 64-bit support; ...". Crack THAT, and you've got something.

UPDATED to clarify which other Financial Industry company was originally driving AMQP.