Sunday, May 23, 2010

Don't Host Crowd and Jira in the same Servlet Container

This took up quite a bit of my Saturday figuring out, so I figured I'd add some pointers for other people to find.

Atlassian doesn't recommend that you host their applications in the same Tomcat instance, rather encouraging you to deploy them in different Tomcat instances and JVMs through the "Standalone" distributions. However, recommendations never stopped me before, so at OpenGamma we have two different sets of Atlassian infrastructure:

  • One set of Confluence, Crowd and Jira for the FudgeMsg project running in one Tomcat container on one VM
  • One set of Crowd and Jira for our corporate use running in one Tomcat container on one VM
  • Bamboo and FishEye for OpenGamma corporate use running behind our firewall in their own standalone implementations in their own VMs.

Yesterday I tried to upgrade Crowd and Jira for the OpenGamma corporate installation. It wasn't pretty.

First of all, I ran into this KnowledgeBase issue, where Confluence, Crowd, and Jira all ship with different versions of the Felix jar for plugin management. This stopped the plugin system from starting for Jira (since Tomcat launches the Crowd application first), so Jira was pretty borked.

Then I ran into something far more pernicious, which other people should be aware of.

Reindexing Requires Crowd

If you're running your applications with Crowd, the application delegates user-related information to Crowd and uses a RESTful approach to loading the data. So far, so good.

However, when Jira attempts to in-situ upgrade an installation (which again they don't recommend), it will do its database wrangling, and then reindex the system with the new functionality (in our case, going from 4.0 to 4.1.1 to allow searches based on votes and watches on issues). All this upgrading it does in the application startup logic, which happens in the Tomcat main thread.

When it gets to reindexing, it then attempts about 10 different RESTful calls to your Crowd instance, as it doesn't have any caches populated on user data. However, while Tomcat has opened up port 80 (or 8080 or whatever) for Crowd, it hasn't enabled application dispatch to the Crowd servlets yet.

This means that all 10 remote calls (which are actually local to the single Tomcat instance since both apps are co-hosted) hang, and the entire server startup process fails.

The only workaround is to launch Crowd in its own servlet container, change your Jira's crowd.properties to refer to the new one, startup, and then undo what you've done.

The Moral Of The Story

When your software vendor recommends that you don't do something, don't do it unless you have an exceptional reason to do so.

If you're a software vendor and you support something but don't recommend it, still test it as there will be customers who ignore your recommendations and do it anyway.

Monday, May 10, 2010

The Difference Engine: Give Up And Move To London

So I came about an article about The Difference Engine, which is attempting to be the YCombinator of Europe. I'm going to have to completely and utterly disagree with Mike Butcher here when I give some pretty pointed advice: they should give up right now, this minute, this very round of startups and escape Middlesbrough and move to London.

Editorial Note: If you're offended by some soft Southerner capping on The North, or, worst of all, an American bashing everywhere in the UK that Isn't London, just skip ahead to the comments and start bashing. You won't like the rest of this article, and you'll just start flaming the comments anyway, so save yourself some time.

Mike, you're 100% wrong that Europe is somehow Exceptional when it comes to siting of startups.

First of all, let's deal with the whole "YCombinator Of Europe" thing. YCombinator started out in Cambridge, Massachusetts, which already had a pretty large startup tech cluster. Then they started doing the program half-and-half with Silicon Valley. And then they finally gave up and moved the whole thing to Silicon Valley. They already had experience with several rounds of startups, and several rounds of exits. They saw that the #2 tech cluster in the US, and possibly the world, simply didn't have a big enough ecosystem. Yet somehow Middlesbrough does?

Second, let's deal with the whole "Don't Have To Be In London" thing. You're right, there are a number of European startups that are coming from NotLondon. Mike references Dopplr (Helsinki, but then also has a big London-based office) and Spotify (Stockholm, though how much of a scrappy startup they can be with the most opaque ownership and funding sources in the world is debatable). I'd add a number of Baltic-based startups to that list (like Erply). None of them started out in London, that's very true. But they all did start out in their country's gravity well for top talent.

One thing that strikes me as an American about Europe is that the vast majority of countries have one super-dominant central city that acts as a major pull for talent throughout the country, and which is usually the capital city: London (the UK); Dublin (Ireland); Paris (France); Copenhagen (Denmark); Prague (Czech); Stockholm (Sweden); Helsinki (Finland); Amsterdam (Netherlands, though this almost deserves an asterisk since the whole Utrecht-Rotterdam-Amsterdam cluster is one massive metropolitan area). Probably the place with the least sort of concentration is Germany, with its multiple nearly co-equal metropolitan areas.

All of these places are the dominant suck of talent from their respective nations, and produce the types of network effects that you need for a wide variety of knowledge-based industrial sectors. To me, it's no wonder that you're getting non-London-based startups out of these cities, rather than also-ran cities in each of the countries (how many startups are coming from Marseilles, Brno, Arhus, Den Haag, Valencia? Probably not more than a few). You need to be where the best pool of talent is.

So let's focus on what you need for a successful startup:

  • A Pool of Skilled Talent: While The Difference Engine may be able to get these people on the ground in Middlesbrough, what happens when the team needs to go from 2 to 3? What happens when it needs to go from 3 to 10? Where are they going to get the talent? They'll have to leave Middlesbrough, that's what.
  • Sources of Funding: This is a longer-term thing for an incubator, but you need to have the sources involved relatively early and often to make it a success; YCombinator flew the Cambridge guys to Silicon Valley just to give them experience with the funding scene, and Cambridge already has quite a few big name VCs. Middlesbrough? How many European VCs have ever been there?
  • Entrepreneurial Culture and Mentors: You've got the guys directly involved with The Difference Engine, but what about other people who have run the whole cycle a few times and can act as passive or active mentors?

I contend that none of this is present in Middlesbrough, and quite simply never will be. All of them are present in London today. The single best thing that The Difference Engine could do in order to help its startup founders is to move to London next week.

And here's the best part: it's not going to stay in Middlesbrough. That's right, it's moving to Sunderland. Which means it'll end up with the types of back-and-forth that mean neither place ever sets up any type of cluster effects. Smacks to me of the type of thinking that's led to the European Parliament splitting its time between Brussels and Strasbourg.

Here's what this is, plain and simple: Yet Another Regional Development Agency Thinking One Industrial Park Will Rebuild Its Economy. How many cities and regions say "well, if we just setup an industrial park, we'll all of a sudden be the Silicon Valley Of X"? How many cities, regions, and countries say "well, if we just setup a small venture capital fund we'll build a new tech cluster?" I've seen it all before. And it turns out that the top three sponsors of The Difference Engine are all councils and a regional development agency.

Tech clusters happen organically. They can't be willed into existence by any government or development agency.

Look, I give Middlesbrough and Sunderland councils props for trying to convert their cities from the Grim Northern spots that they're known to be. But they're being, I think, disingenuous to their founding participants.

If you care about the actual founders, rather than just about wasting development funds, get your founders out of the North and into London immediately.

If you're a founder considering The Difference Engine, my personal recommendation is to ignore it. Move to London, get yourself a desk in a shared office in the Shoreditch or Bankside area, and go to town. If you're the type of person ready to do a startup, you'll probably learn more hanging around the vibe than you would sitting in the Grim North for a few months.