Next Open Protocol Failure: Perforce

I've been using Perforce for years (professionally since early 1998), and I like it. It's a great product. Not perfect, but the combination of the server, the community, and the tools are pretty hard to beat from the perspective of a centralized SCM system. But it's not the end-all, be-all that it once was (when the only real alternative was CVS, which really just needs to be put out of its misery for good).

Subversion, for example (originally started as a Perforce replacement according to the Collab.net people that I spoke with at the time, as I was working on my Open Source Database startup), has come along quite a bit. It's a pretty good system, and you could in theory replace Perforce with Subversion for a lot of uses. I use it for non-commercial work, and it's a fine system.

Perforce realizes that part of the real benefit of an SCM system is that you can build a development ecosystem around it, with alternative GUIs and Continuous Integration systems and bug tracking and automated scripts and all kinds of interesting tidbits designed to save engineers time and encourage proper development methodology. That's great.

But there has been a whole category of systems that have come into being that Perforce absolutely, positively, supremely refuses to support: Java developers.

There are a lot of people making tools that attempt to integrate with Perforce but using Java: Hudson (Continuous Integration), Ant (build tools), Maven (build tools), JIRA (bug tracking), FishEye (SCM introspection), Crucible (Code Reviews), Bamboo (Continuous Integration), Eclipse/NetBeans/IntelliJ (IDE). All of them are Java. All of them have very valid reasons to integrate with Perforce. And Perforce Software categorically refuses to help them in any way. Well, no, that's not entirely true, Perforce finally realized that they couldn't refuse to support Eclipse anymore and the existing p4eclipse plugin wasn't that great, so they released P4WSAD specifically for that.

Perforce supports a C++ client library, which they build for you on every platform they support (which, to be fair, is pretty much any platform that you're ever likely to ever want to do development these days), and they've built wrappers on top of it for the various scripting languages that usually just wrap C/C++ libraries: Python, Perl, Ruby. But there's no Java there, and there never has been. And while in theory you could JNI/SWIG wrap the C++ API, nobody who is realistic about supporting Java applications ever does that without a fallback that's native if they can possibly avoid it. It's just a world of fail down that path.

So what all tools do is just wrap the p4 executable and run that. Heck, that's what P4WSAD appears to do. And to be honest, that's fine for occasional use, but it's not really ideal for long-running processes that have to kick off a massive number of p4 interactions. Is that really the best they can do? And the existing libraries for wrapping the p4 binary in Java really aren't that great and have some serious performance implications. Sure, you could make them better, but the generic problem is that shelling out to a subprocess, executing a simple command, and then parsing a bunch of textual results really just isn't ideal. What you want is something that natively speaks the protocol in your programming language of choice. And for that, you need the protocol.

The core of the problem here is that Perforce refuses to document their binary protocol in any form. This means that if you want to write a purely native client for a Perforce server in any language that isn't C++, you're stuck. The only supported integration path is to link against their C++ client library, and if you don't want to have to deal with native dependencies in all your various programming languages, you're completely out of luck: you have to wrap the p4 binary. You want Erlang, Haskell, Jython, IronPython, JRuby, F#, OCaml, Lisp, Scheme, whatever, you're out of luck. You have to wrap the p4 binary.

From the API, it appears that their protocol is almost stupidly simplistic: send over a list of strings (the command and its arguments); server blasts back commands you should execute. Is your critical intellectual property wrapped up in such a simple protocol? One that I could packet-sniff and reverse engineer in an afternoon (while of course violating my license agreement). Really? That's how you're going to stop the advance of open source alternatives?

If I worked at Perforce, I would give up the whole thing and document my wire protocol. I might not make it completely open. I might open it up to key people initially over NDA (short-term fix), but I sure wouldn't force everybody in the world who wants to help your ecosystem grow but doesn't use your favorite programming languages to go through such serious efforts to wrap a stupid platform-specific application. I'd be bending backwards to make people want to pay me money for what I make money on, which is the p4d server application. Anything that keeps people paying me money is great. Anything that keeps people paying me money while actually reducing the number of things I have to provide them with to keep them paying me money is even better.

It's just like my SonicMQ problems. Closed binary protocols are bad if you ever want to code in a programming language your vendor doesn't really like.