Thursday, January 29, 2009

Fannie Mae Employee Attempts Network Armageddon

Yet again, Thanks to El Reg.

A few quick points here:
  • If you don't have an air-gapped (or WORM) backup/archival solution, you're at risk of this. Having archive solutions on your network won't be enough, because there will be someone who can override your internal controls.
  • Your termination protocols should lock down every single thing connected to anyone in IT the moment they're in the room with HR. True story: I once had to do this manually at my startup as we didn't have any automated system to deal with this at all, as our only systems administrator was being let go.
  • Even if your developers aren't disgruntled, a single poorly written multiple-ssh script could have done the same thing (even with the monitoring system running, how fast could they have shut down an inadvertant worm?).
  • 10 years? For that? Really? Man, the US justice system is messed up.

Monday, January 26, 2009

More on London vs. Valley

Just a quick follow up on this morning's epic rant about London and Funding.

Read this from the Futurist: "Why Government Is Set to Extinguish Silicon Valley."

Whether or not you agree with the argument, he makes four key points here:
  • SOX compliance makes it difficult to justify an IPO. No worries about that here, and the London Stock Exchange's AIM still has some IPO activity.
  • Immigration Torture. Again, not only do we have all of Europe here, the UK actively encourages anybody who has excellent skills to relocate here from wherever you are under the Highly Skilled Worker program (hint: if you can work in the Valley at all, you qualify).
  • California Taxation. Yep, Cali is completely ungovernable and taxes are going to go up a lot. We don't have a split between federal and state taxes here.
  • Federal Taxation. Yep, your taxes are going up. Ours are as well, but much less on current projections (if nothing else, no matter how broke UK.Gov is, our debt as a percentage of GDP is much less). We also don't have an AMT here to contend with.
Still think London isn't a desirable place to locate a startup?

Want a London Tech Sector? Invest In Startups

I've written previously about moves to set up a London TechHub, and I've spoken disparagingly (some might say "ranted") about it. Since then I've been made redundant and spent my time helping out an Open Source project and generally embedding myself in the local AMQP community. I'm a lot more optimistic than I once was.

Basically, I've been hanging out every day in Silicon Roundabout, and it's starting to feel pretty interesting. It's got some of the things that the rest of London lacks:
  • Public third-spaces have free WiFi and power points all over the place. Contrast this with a standard central London Starbucks. [1] The only big difference between London and San Francisco is that in London the wired third-spaces are pubs; in Silicon Valley they're coffee shops.
  • Lots of Class-Z office space. Nobody's trying to tear down everything they see and put in Marc Jacobs boutiques and Class-A office space. The fact that there's office space that's actually affordable is a big plus.
  • Lots of startups. There are all kinds of small tech and media companies around seeding each other with ideas. People working here know each other and can meet up easily for beer after work and weekday meetings.
  • Lots of Vietnamese restaurants. Maybe it's my personal experience, but grabbing the team and going to get Pho for lunch is part of the Valley experience to me. [2]
  • People doing deals. Everywhere around there are people obviously getting Deals done. Funding deals, consulting deals, hiring deals, it's all happening. You can feel it.
  • Non-traditional-appearance people. Artists, piercings, nobody wearing business attire, it's all there. Much more techie than the traditional City/Wharf culture.
And here's the thing: none of this has happened with any type of TechHub coming into being. It just happened organically.

Fund Firms To Build A Tech Cluster
You want to know what would make this really gel? More funded startups.

There's a general sense these days that any new tech startup requires like 2 guys, some Merb/Rails, a couple of MacBook Pros, EC2, and you'll have a million users overnight. Why take funding? Why invest? Why not wait for commercialization?

Because you take the long view. Because you realize that creating a culture of early stage startups requires a culture of entrepreneurship. And because you know that only funding companies leads to spreading that culture of entrepreneurship to new people. And, even more importantly, because there are a lot of ideas that actually do require more than 2 guys + laptops; they require some pretty substantial up-front investment to pop, but the pop is probably going to be bigger as a result.

But that takes ideas that require funding, and a willingness to invest in early stage companies. There aren't a lot of angels around at the moment (if nothing else, the financial implosion hits them first), and London's been so expensive that most techies of sufficient quality don't have enough savings to bootstrap for a year. So you need a bunch of people looking at interesting IdeaCos, and investing in them.

The Kirk Story
Perhaps the most important thing that happened to me to incubate my personal culture of entrepreneurship was taking a startup job straight out of University (Go Bears!). That changed my worldview so dramatically that when I then went on to work for WebLogic (just after BEA acquisition), I thought that 2000 person company was monstrous and bureaucratic. I couldn't wait to get back to a startup.  That's how you build a tech community: you infect people with the startup bug. [3]

When I moved to the UK from Silicon Valley for personal reasons, I wanted to continue that culture of startups, that constant entrepreneurship. But when I looked around, I realized that the only way to live with a reasonable standard of living was to dive into finance. [4] VCs I met with tried their hardest to get me on board with one of their portfolio companies, and even asked me whether I had an idea they could fund (which I didn't, and even if I did, I didn't have the local network at the time to put a deal together). I viewed working in finance as the best chance I had to build a life for myself in London working in software development. [5]

I think things have changed. I think London is the place, and now is the time.

Why London Now?
So if you're reading this far, and you have a few quid to invest, here's why I think this is exactly the time to push hard-core into technology companies that require up-front investment in London:
  • People Are Available. The Credit Implosion has really started to hit financial IT. [6] This makes the calibre of people that you need to do startups available, and it's only going to get worse/better from here. Infect them with the bug, make it clear that they can do tech startups and be able to live in London and maybe have something pop, and you've got a culture going.
  • Sterling Is Worthless. $1.366 as of Friday close, even worse in Euros. Nobody denominates VC funds in Sterling. Invest here while you can. Seriously, even alpha developers here are earning at market exchange rates monstrously less than they are in the Valley or New York.
  • We Have The Basics. WiFi/Power Points, Vietnamese Restaurants, Cheap Offices, Public Piercings, it's all here. [7]
  • We're Not USAia. We can draw on skills (language skills, cultural affinity) from throughout Europe [8], we understand European culture isn't the same as California, and we can better target consumers and businesses in Europe/Middle East/Africa. North America is roughly 350MM people, EMEA is circa 1000BN (maybe more).
  • We Speak English. Even the Americans here (guilty!) speak some bastardized form of the Queen's Tongue. For global businesses, where you want some real language skills and native English, we can put it all together in one location.
  • GMT Rocks. We have business hours overlapping with San Francisco, New York, Moscow, China, and (at a stretch) Japan. It's one reason why finance firms flocked here, and tech firms can leverage it too.
So if you're thinking that this might be a good time to invest, trust me, it is. If you're thinking of looking into firms here, do it. If you're thinking of opening the taps, you're thinking along the right lines. You don't even need to fund entrepreneurs here immediately, just getting your existing startups to open offices in London and giving the employees equal equity stakes their their Valley cohorts will go a long way towards equalizing the situation, and you'll still be able to take advantage of all the advantages above.

I fervently believe that if you infect enough Londoners with the startup bug, we won't get cured. We'll be just as driven as the Valley folk. You just have to infect us.

But don't think that you're best served in making a big PR splash, opening a fancy office, and then not investing. It's still embryonic here, and you're going to have to help entrepreneurs by funding people you feel have talent (but maybe not just yet...) just as you would in the Valley. Make a long term commitment, and take advantage of current troubles to make it a cleaner short-term bet. You'll find that you have a stable of entrepreneurs and experienced techies that can really make great things happen.

Footnotes:
[1] This sounds utterly trivial and stupid, but it's a real sense of the types of people you expect to be hanging around. If you've got free (not wireless-company-paid, not pay-per-day) wifi, and power points, you expect people to be hanging around and doing stuff. Stuff that involves the intertubes.
[2] My tips: Song Que in the Kingsland Road Vietnamese strip; Green Papaya if you're willing to voyage way out to Deepest, Darkest Hackney for fantastic food; Cay Tre if you want to stay close to the office.
[3] People from Radik have gone on to Yahoo!, Google, Intel, Duke University, VMWare, Nuova, TellMe Networks, Gunderson Dettmer, Intuit, MobSpin, LucidEra, Required Technologies, ..... A good startup has a fantastic alumni legacy.
[4] Yes, I've told this story before. Forgive me for becoming my mother.
[5] Have I mentioned I'm unemployed yet? Sorry, that mother thing again.
[6] Hey, still unemployed here!
[7] Just to point out, this is largely from my recent experiences in Silicon Roundabout/Hoxton. There are a lot of similarities with Bermondsey, which is where I'd personally put a firm I founded (mostly because Hoxton Square takes much longer to get to from where I live), and has a pretty similar vibe (if smaller). If only it had a reasonable Vietnamese restaurant...
[8] EU Free Movement of Labour doesn't just apply to Polish Plumbers, it's also applies to software engineers and business guys from everywhere in the EU.

Thursday, January 22, 2009

I Use Product X, AMQP Is Irrelevant To Me

As I've gotten more involved with the greater AMQP community, and as I've talked more and more about it with people who already "get" messaging, I keep hearing the following comments:
I've made a significant investment in Product X [MQ Series (or whatever IBM's calling it these days), SonicMQ, Tibco EMS/Rendezvous, FioranoMQ] and I use it for as much as I can. I'm happy with it. I'm happy with vendor support, all my apps "just work." Why in the world should I care at all about this AMQP thing?
It's a legitimate sentiment. A lot of the AMQP hype has been driven by next-generation pure-play AMQP broker platforms trying to eat into those existing install bases. I still think it's relevant to you, the Happy Commercial MOM Broker Customer.

It's The Ecosystem, Stupid!
Remember back when I initially ranted about AMQP. One point I made there, quite aside from my "My commercial vendor is annoying me" ranting, was about the overall ecosystem. Here's where I make my AMQP pitch: AMQP Will Help You Leverage Your Existing Investment.

Let's assume that you're completely happy with your WebSonicFioEMQ platform and you've changed all your code to be able to use it. You're still not leveraging it to its maximum level, because no matter which one you chose (or if you chose them all), you can't always get third party products to integrate properly.

Consider one pretty Enterprise Software product: Sybase Replication Server. Last time I checked, it could pump out event messages on "interesting" events through three mechanisms: JMS, Rendezvous, and MQSeries. But I'm sure it's actually a lot more specific than that (I can't check directly as it's firewall protected, but could someone who has access fill us in in the comments), down to the version of individual JMS Brokers and versions of Rendezvous and MQSeries and all that that it supports.

Consider some more emerging products like Continuous Integration platforms. All the modern ones can send alerts using email and Jabber.

They're astute in choosing the most dominant messaging mechanisms in their target environment: commercial MOM systems in the case of Sybase Rep Server, and open communications standards in the case of Continuous Integration. And therein lies the rub:
  • Unless your MOM platform is truly ubiquitous in your domain, every piece of software you want to integrate into it will result in custom effort;
  • Your MOM platform isn't ubiquitous in every domain.
It doesn't matter what you're using for asynchronous communications: it's not ubiquitous enough.

AMQP Promises Ubiquity
Let's assume that the promise of AMQP really gets going. I wholeheartedly believe that the sheer engineering effort that application vendors can save through only supporting one protocol, and thus one client library package, without any interoperability concerns, will force it through.

Consider someone like Atlassian. You're an Atlassian customer, and you want your Confluence and Jira and Bamboo and FishEye instances to fire off messages to your existing MOM infrastructure when "interesting" things happen. So you lobby and say "Hey, can you support SonicMQ for message publication?" and they say "hey, that looks pretty interesting."

But the moment someone else catches wind of this, pretty soon they have a many-to-many compatibility story as every customer with a different infrastructure pops up and says "if you support X, you should also support my MOM infrastructure as well." Math is Hard, Atlassian Goes Shopping and says "write your own plugin, yo."

What if Atlassian could satisfy every single major MOM infrastructure with exactly one protocol? They'd do it. Interop testing by the broker developers (and not Atlassian) will take care of everything else.

AMQP Helps Your Commercial Investment
If you think this all seems like it makes sense (and if it doesn't, let me know precisely what about it doesn't in the comments), your immediate response should be to get on the phone to your salesperson and say:
  • I want AMQP support in a (near-)future release of your MOM Broker;
  • If you don't do it, I will roll out someone else in parallel and bridge them.
Trust me, this will get them thinking. Because I'll guarantee that within 6 months of AMQP 1.0 coming out, there will be open source bridges between every major commercial MOM infrastructure and AMQP. And the absolute last thing your sales person wants to hear is that you're thinking of adding something to reduce your existing lockin.

Once this is done, you get the best of both worlds:
  • You can leverage the entire organizational and technical infrastructure you've already built up to do MOM properly without adding more "products" to manage; and
  • You can integrate yet more applications and participate in a far richer ecosystem than Your Product could ever support on its own, helping drive that investment farther.
So get on the phone to your sales guy. Now.

Tuesday, January 20, 2009

Extprot: A Very Cool Idea

Another thing that I ran across today is the release of Extprot. Very quick rundown:
  • Packed binary encoding of message data (a la Thrift, GPB, ASN.1, blah blah blah).
  • Backwards compatibility with certain schema changes.
  • Self-Describing: permits runtime introspection of message contents without access to the message definition (currently not in GPB for example).
  • Message length encoded in the message (to avoid the streaming problem that GPB has).
  • Compiler in OCaml.
  • Reference implementation in OCaml, apparently Ruby is in the works.
  • If they can get the big-three enterprise/MOM-friendly languages (C/C++, Java, C#/.Net) supported, this could be a winner.
I have a long-edited blog post that I'll flesh out in the next couple of days about this very subject (MOM is all well and good, but what do you put inside the messages?). This makes it far more relevant.

Great Rundown On Key-Value Stores

In the event that you're interested in the general area of key-value stores (Amazon SimpleDB, CouchDB), you should read this article by Richard Jones where he gives a pretty great rundown on all the current players, most of which I'd never heard of. Go read it. It will make you smarter.

Anti-RDBMS: A list of distributed key-value stores

Friday, January 16, 2009

Open Source: It's All About The Community

I wanted to take a look at an open source project/product yesterday evening (I'm not going to name them for obvious reasons). In order for me to look at anything other than marketing-ware provided by the company looking to commercialize it, I had to:
  • Register with an account in their Confluence instance.
  • Run through an email validation step so that they were sure I gave a real email address.
  • Register again, giving them my full contact details (name, email address, and a valid phone number), a company name, company sector, and title.
  • I then got an email from their system that they had automatically added my email address to a mailing list so that they could, presumably, spam me.
  • I expect a phone call from a sales rep any day now (or else why would they have required a valid phone number in the first place?).
Once I had done this, I could look at their documentation. Yay!

Next step: I want to grab it and take a look. Oops, only binary download format is a Windows-only installer (seriously? You couldn't just do a tarball? I don't run Windows. At all.).

Okay, so maybe I can take a look at their source code quickly. Oops, no online source code browsing. Rather, I had to do a search on their confluence, and then that showed me how to do an SVN pull.

Really? You think this is the way to build a community? By putting this many barriers in front of someone who just wants to have a poke around?

A firm like this, to me, would be much better suited as a closed-source, source-code-available firm (and lots of firms do this; for example, at an Enterprise price point, you get the source code to most Atlassian products, which helped me to debug some problems I had with an early Bamboo 2.0 build). Building a community requires that you be more than willing to actually embrace Open Source as a way of life, with an emphasis on the Open side.

They should be doing everything they can to get code in the hands of developers, and to get binaries in the hands of potential evaluators. They shouldn't make me feel like I'm going to have to talk to an Enterprise Sales Guy just to see what their system actually does.

Focus on the community and the rest should follow. Don't make people feel like Open Source is just a marketing angle.

Help in Vacating Blogger/BlogSpot?

Dear LazyWeb,

I've been on Blogger and BlogSpot for a while now, and I'm sick of it. After a pretty minor configuration change yesterday locked everybody out of the comments for a number of hours, I've finally decided that it's time to move on. (More importantly, like many Google acquisitions, after the Big-G bought them out, innovation large seems to have ceased; I hate being that far behind the curve).

After looking at quite a few options out there, it looks like all the cool kids are using WordPress, so whether I like it or not I think I might be stuck with it.

So with that in mind, LazyWeb, can I ask you for advice:
  • Should I go with WordPress or with your favorite platform of choice (please, plug away!)
  • Should I host myself (on some type of cloud environment, probably FlexiScale)? What do I get out of this?
  • If I choose a hosted provider, which one do I go with? What am I trading off on this decision?
Please, LazyWeb, help me!

Love and Rants Always,

Kirk

Thursday, January 15, 2009

Qt Going LGPL: The Java Angle

(One of Many articles on Qt allowing LGPL licensing). Assuming that this includes the Jambi components (Java/Qt bindings), which I can't find any reference to positively or negatively, but since it's part of the language bindings and not the value-add tools I can't imagine they wouldn't, I think this is a great move for the Java community in terms of rich GUIs.

No, I'm not going to talk JavaFX. Ignore that as a distraction at the moment.

Right now, if you want to roll a thick/rich client (and yes, there are still a lot of applications that are (shock! horror!) not delivered through the browser, and that's going to stay that way) in Java, you've got a few choices:
  • Swing. Gag. Even with Nimbus (only available if you've got 6u10 pushed to the desktop), it's ugly, and no matter how good the Windows L&F gets, it's never going to look native in any way. Swing is also an example of precisely why Java needs closures.
  • SWT. SWT would be much better if JFaces was more advanced (I don't want my apps to look like ass, but I don't want to go back to AWT-level programming either), and SWT was more documented, and there were really people using it for things other than Eclipse plugins. Yes, I know there are some other uses, but not a ton to be fair, and so not a lot of example code to copy.
  • Qt/Jambi. Very well done MVC programming model (signals and slots, but with no preprocessor, so it's all Java toolchain friendly), looks native. Oh, and you can embed WebKit.
Honestly, I think this is a really positive thing for Java to have a rich GUI framework that's easy to work with, and looks good. Lord knows we've been waiting long enough for it....

Saturday, January 10, 2009

Ordinary People, Super-Stars, Compensation and XP

(Background: "We Don't Pay You to Work Here" on Venture Hacks).

I'm going to completely disregard every single point in there about incentivising (yes, I just typed that word; you may hereby assume I've lost all cred) employees without money, and urge everybody to leave their cynicism at the door (because ultimately, if you have to be taught to do that stuff, and you have to make a conscious effort, you will fail at it and should resort to just paying people more; a quality working environment is an intrinsic part of culture and can't be bolted on by paying consultants).

I'm with the article, and at least finding it a useful POV, until this:
Practices like Extreme Programming, that were designed for programmers with ordinary skills, work even better with extraordinary programmers.
No. No, no, no, no. No, no, a thousand times no. Stop right there.

Whoever typed that should be aware that their chances of actually employing extraordinary programmers or working with them are approximately 0. Maybe of the group of programmers with which the author has experience, the extraordinary ones of that group work well with XP. Not in the real world.

I do not know a single extraordinary programmer who likes or would even be willing to tolerate XP.
  • Pair programming just doesn't work with extraordinary programmers (and extraordinary ones are usually the ones who literally shove the other part of the pair to the side and pretend it's normal programming).
  • 40-hour strict workweek just doesn't work, because many extraordinary programmers can't switch off voluntarily, and go through manic high and low phases of productivity.
  • The phrase "YAGNI" makes extraordinary programmers violent with rage, because they are always more skilled than the person uttering it, and always know that Yes, You F**king ARE Gonna Need It, Moran! (YYFAGNIM? [Welsh?]) They understand agile, but at the same time they also understand overall lifecycles and how difficult it is to retrofit Lack Of Design Up-Front f**k-ups later on.
  • Perhaps most importantly, extraordinary programmers understand anyone selling any type of Silver Bullet is lying or naive.
And remember, XP is based on an irreducable model of software development methodology: it's the Three Stooges Syndrome of Agile Methodologies. Proponents explicitly state that without doing every single part, the whole edifice crumbles. I do not know of a single programmer I would consider extraordinary who doesn't have at least a few problems with XP, and who would voluntarily join an XP project (and everything these guys do is voluntary: fear of unemployment doesn't do anything for them).

Perhaps the author of the article needs to meet some real extraordinary programmers.

Thursday, January 08, 2009

RabbitMQ/AMQP OSGi Integration

I had a chance to meet up with Neil Bartlett today, and got to talk to him about his RabbitMQ OSGi integation (note that I believe that everything he's done thus far is actually applicable to anyone using an AMQP broker that is compatible with the RabbitMQ Java client, so I'm treating it as a generic AMQP/OSGi module). While it's a work in progress, it's incredibly cool. It allows you to:
  • Declare production/consumption parameters (exchanges, queues, hosts, credentials) in a declarative way;
  • Have your OSGi container connect the dots so that your declarative bundles automatically match up with concrete connections to your AMQP broker of choice;
  • (If you're using Equinox) Use the OSGi shell to interact with the RabbitMQ/AMQP client library.
The first two parts were the types of things that I had worked on in the past (having a declarative way to specify messaging endpoints, down to the broker connectivity level) so that you could change them at execution-time. The third was the real WOW-worthy point.

Assume that you're coding against an OSGi runtime. Assume that you're working with bundles, and dynamically loading and unloading them as you're compiling your code (meaning that you're probably at an osgi> prompt). Assume that want to see very quickly whether your code is interacting properly. Now you can easily setup your bindings and publish and consume messages as your real code is in flight, and see the effects in your debugger.

How amazing is that? Imagine the human cycles that you'll save being able to do all this from one area, that you're already familiar with?!?!

This is a work in progress, and hopefully Ben Hood and I had a chance to give him some places to go from here, but I think this is exactly what the promise of AMQP offers: the chance to do amazing things based on not having to support a million different client libraries, but to code against a standard and make great stuff happen.

If you're using the RabbitMQ Java Client and any type of OSGi container, I highly encourage you to take a look at this and get involved. I see great things happening here.

Appearing at Powered By Cloud

Those of you who are in the UK and interested in Cloud/Utility Computing might find it interesting that I'll be appearing on a panel at the Powered By Cloud conference in London.

I'll be on a panel addressing a subject close to my heart, regulatory issues in cloud computing in finance. I'll be part of Session 7: Privacy, Regulation, Security and Cloud Computing (no worries, my name isn't there because I haven't given them a bio and face pic yet).

Key Details
Date:
2-3 February, 2009 (I'll be on day 2, 3 February 2009)
Location:
Lewis Media Centre
Millbank
London
SW1P 4RS

2009 Goals

Following on from my predictions for 2009, let me follow Joel's lead and try to turn Seth's comment into a meme. So without further ado, here are my goals for 2009 (whether they're SMART or not is an exercise to the reader and yours truly).

Find Gainful Employment
As those of you who are faithful readers know, I found myself affected by the Credit Earthquake and am currently seeking gainful employment. My #1 goal is thus to find myself once again part of the gainfully employed section of society (and if you think you can help that goal, feel free to contact me).

Extend Asynchronous Communications To The Masses
My #2 goal is to seek an end to polling; it's inefficient, it doesn't scale, it doesn't smell right, and it's not necessary. Expect more posts here about this, as well as ceaseless and boring arguments elsewhere about how we, as software engineers, have alternatives, and should seek and embrace them.

Help AMQP 1.0 Along

Many of you found me through my ramblings on about AMQP 0-10. I believe in AMQP, and believe proprietary protocols are doomed to the dustbin of history. As such, I am setting my #3 goal to be to do everything in my effort to give us, in the community, a workable, interoperable AMQP specification that we can use to achieve my #2 goal above. This means that I'm actually being less ranty and more constructive in my relationship with people helping to get AMQP 1.0 along; have no fears, my innate lack of tact hasn't disappeared, I'm just subsuming it towards the greater good.

Fix My Flat
Usually this would go on my (much-neglected) non-technical blog, but most of the readers there read this as well. My flat is a vast collection of Fail. We've spent two years (since we moved in) trying to resolve the Fail into a cascade of Win. My #4 goal is thus to have a fully functioning flat, with all major services working.

Get Zoo to Cash-Flow Positive
Again, a non-technical subject, but many readers know that we setup and run a registered UK animal conservation charity (the Rare Species Conservation Trust), in part to operate the zoo that my partner operates, the Rare Species Conservation Centre. After a massive, multi-year financial and time investment, I believe that this is the year that the zoo will pay for itself and (as we originally envisaged):
  • Help us fund the in-situ wildlife conservation work that we've devoted so much of our lives to;
  • Continue our mission of educating the UK public in the plight of the animals that have touched our lives and are worthy of protection in the wild.
We're trying to help undo the trauma that man has inflicted on some of the rarest, most endangered, most misunderstood animals on Earth, and I hope that this is the year that our efforts to educate the UK public are rewarded and we can turn back to our main goal: helping in-situ wildlife conservation efforts.

It's only a few goals, and several of them are under way, so maybe I'll actually hit my goals this year after all.

Monday, January 05, 2009

NYT on The State of Finance Yesterday

Originally picked up on via a BoingBoing post, the NYT has a pair (designed to be one read) of Op-Eds about the state of the financial meltdown:
If you've been reading things like the FT and Economist all this time, none of it is going to seem new in any way, but it's a very concise (for 2,000 words each that is) overview of some of the salient issues.

In terms of their prescriptions, a few very brief comments:
  • They specifically mention the Spanish regulatory policy on capital (more in good times, less in bad). This is a quite common prescription at the moment.
  • Regulate CDS contracts. I personally believe that this is due (having done software development for a pretty major player in this space, I'm familiar with quite a few failures in the way the market itself works) in some form (where that form does not mean that the NY State Department of Insurance should be in charge of a global market). Just don't assume that "regulation" means "market minimization." The market needs to be more efficient, and regulated, but not just plain-old shut down.
  • Apparently, regulators need to get smarter. The only country that I know of that has really attempted to tackle the public/private skills gap is Singapore, which has an explicit policy of paying civil servants what they would earn in private firms. This will never happen in the US (can you imagine someone justifying paying a 25-year-old working for the federal government $300,000+ per year? I didn't think so). Smart people follow money, and you can either give them the money up front or have a pot-of-gold at the end of the rainbow. Arguing specifically to eliminate the pot-of-gold at the end of the rainbow by shutting off SEC->Wall Street migration just makes SEC work even less desirable for people with the skill set to be able to do it effectively.
That's all. Seriously. Go read the articles. They're pretty good writing even if I disagree with some points here and there.

Saturday, January 03, 2009

Amazon SDB: "Eventually Consistent" Applies to Credentials

I wanted to get my head around using Amazon SimpleDB for a couple of projects, so I grabbed Typica, the Java library for accessing AWS services. I was having some massive problems with authentication validation errors, but because the Typica distribution doesn't include source code, I realized that the source line numbers don't match up with the binary for 1.4.1.

Even updating to a SVN pull of head didn't solve things. So I waited while I ate my dinner. Some requests went just fine, some failed. Since SDB involves a scatter/gather approach, this meant that my system wasn't working properly as a whole and I couldn't get my aggregate results correctly.

After waiting a while, the problem went away on its own. I think it's a result of the Eventually Consistent principle at work. I had enabled my AWS account so recently it hadn't had a chance to propagate my AuthN/AuthZ credentials to all the SDB nodes. When it finally did, everything was fine. And I can now quantify the time it takes: One Pizza. That's how long it takes. Push data, eat pizza straight out of oven, data consistent.

By the way, now is a perfect time to try out SDB. It's essentially free for the time being, so no reason not to try it out. I'll have more postings on it, because it's pretty cool.