Real Time Linux Kernel Smacktown Comments

I read El Reg's writeup of Red Hat and Novell attempting to out-gun each other on their RT-esque kernel patch versions of their main Linux distros. Just a few quick comments.

First of all, they're testing against RMDS 6 (something I have some experience with). When you see anything about an RMDS 6 P2PS, what you're really looking at is an ancient branch of RVD running the TCP form of the Rendezvous protocol. That means that a P2PS esssentially acts as a middleware broker in a bizarre mixture of a protocol designed for unreliable broadcast/multicast usage (Rendezvous) but running on a single machine. It's a quite strange beast to be fair. Just a bit of background detail for you there.

Secondly, where things get really strange in this is when Infiniband comes in on the Novell SLERT side. Uhm, huh? They're combining RMDS with IB? In what form? Are they doing IP/IB? Uhm, why? I'm sorry, but if you're going to roll out IB, without using something IB specific like OpenFabrics that allows you to leverage all that nice IB RDMA action, there's something seriously wrong. Yes, you might shave off a few microseconds off your latency compared to IP/Ethernet, but you'd get much better by coding raw against IB. Based on that, I'm actually surprised that Novell included this at all, since I really doubt that anybody is going to attempt to improve the performance of their RMDS infrastructure by running it over IB.

So all that being said, this type of benchmark is really just a proxy for something testing how fast the kernel patches can get networking packets off the wire and into user-space for processing. That's a pretty interesting point and good to know, but if you really care about that and don't have to run RMDS for everything in the first place, why not use something like Solace which doesn't have user-space in the first place? Why not just code against IB using OpenFabrics? This is far more interesting to me in the client side (where you're going to have user-space code ultimately doing your processing) than on the broker (e.g. P2PS) side.

So that being said, I think a very interesting benchmark would be:

• Similar workload in general (the STAC workloads actually are representative of the general workload you get in financial market data processing);
• Hardware distribution to the clients (Tervela, Solace, or OpenFabrics/IB);
• Clients different between the RT-patch version of the kernel and stock kernel.

The client is where I think the action needs to be here, not the broker.