Billy Newport is an Distinguished Engineer at IBM and creator of IBM WebSphere eXtreme Scale. JAOO Today met him in Musikhuset for a chit chat.

Where are you from?

I was born in Ireland and grew up there. Since graduating, I did consulting for 13 years. I was making really good money, but as the kids grew older, we got tired of moving around and decided to settle down.

This brought me to the United States, where I’m now doing my daily work at the IBM Site in Minnesota.

Many people – this interviewer included – see IBM as a monstrous corporation with no ability to produce anything worth using – and yet interesting pieces of software continues comes out of it. It’s almost like two different cultures within this one company. How do you see this?

You know I’m really new to IBM – I’ve only been working there for 8 years, so I haven’t been indoctrinated yet.

Before I joined the company, I was probably their most angry customer. And I guess in many ways I do things very differently. It sure gets me into trouble sometimes – but as a company we need that to keep evolving.

Regarding eXtreme Scale, what we try to do is create a sort of viral middleware, trying to mimic the ways open source projects produce software, using Wikis and other informal collaboration tools.

Part of the success of middleware products like Spring is that they’re just a jar that you drop into you app server, and we tried to borrow from those kinds of practices. There’s no need for bosses to sign off on new strategic updates or system administrators to download and install patches and service packs for WebSphere app servers. Our product is as easy as Spring to slip into a project, because it integrates with every app server out there. And it can evolve on its own independent of the rest of the middleware platform.

IBM Marketing is centered around themes like e.g. SOA, where things like Extreme Scale has a tendency to drown, so we’re trying to sell it in a more viral
way too.

How would you compare Websphere eXtreme Scale to the offerings of e.g. GridSpaces?

It’s very similar technology, so we’re competing head to head. I see our main competitors as Tangosol, Coherence, Gigaspaces. And when they eventually come out, Microsoft Velocity.

As I see it, the really interesting thing about Grid Based computing is that by forcing the programmer to acknowledge the distribution, not making it transparent, middleware can achieve all sorts of optimizations. Do you see this dripping down from Extreme Transaction Processing down to more “middle of the road” enterprise computing?

I really do. For example we had a project with a customer doing a sports website using Hibernate and relational database for storager. They handled 10.000 transaction per second. We helped them switch to a grid based solution, backed by the same storage layer, but with the grid in between. Now they’re executing requests 10 times faster – in six weeks from start to production. So it’s very much relevant for more traditional systems too.

But most of all, I think what’s happening now is that grid technology is lowering the cost of deploying solutions. In some cases, disk or databases are neither the best nor the cheapest answer – things like http sessions that last maybe a couple of hours can be stored much more efficiently in a grid: Still fault tolerant, but in a way that is distributable and shared. The grid provides a more economical solution.

This is especially the case when deploying to virtualized servers. People see 10 servers with 5% cpu load and decide to consolidate them to get 1 server with 50% cpu load. But many times, you forget the memory – when you need 10 times more ram for the single server, it might end up being more expensive in the end!

When you look at it, a lot of the memory used in the virtual servers are caches. Middleware will give us ways centralize this memory usage.

How does trends like languages, immutability and focus on concurrency fit in with the kind of middleware you are talking about?

Concurrency within a single box only gets you so far. When the need to distribute arises, you need to take into account the cost of moving data around.

It’s all about partitioning the job. As an industry, we need higher lever abstractions for this; closures etc.

I think we should ban the for loop, and give people a very easy way to handle iteration in a way that scales and distribution. The kind of Fork/Join constructions that are developing as part of for example Java 7, to get those working on grids, might be interesting.

From following you on Twitter, I guess I’ve found out you have quite a passion for racing cars. Where did that come from?

I always liked go-carting, and I did a lot of it in Europa. In the states, one of my friends had a Ferrari, which made my buy a Porsche. We got speeding tickets.

So we thought we’d rather learn how to drive those cars right. But when you race street cars on speed tracks, they tend to brake down – not even Porches are built for the kind of stress put on the car when racing it at maximum speeds. Along the way, buying spare parts got too expensive, so I switched to a real race car.

The racing has a nice side effect: Minnesota is very cold and icy for 4 to 6 months a year, so driving is almost like if you live in Scandinavia. When those are the conditions, it’s a good thing to have the driving skills to keep the car on the road.

Also, my driving helps me fight one of the challenge of modern life: multitasking. When you’re always doing several things at once, switching tasks and skills, it’s hard to be really good at anything.

To drive a racing car as different. Once you’re in the race, it requires complete focus. Racing is one of the few things I do, where when I’m doing it, everything else goes away.

If you had not become a software engineer, what would your career have looked like?

God, I don’t know. I did electronics at 10. Started programming at 11. Did video games commercially at 16. My brain has become hardwired to do software.

As a kid, I wanted to be a soldier, but then computers came along. I think it is a little bit safer that way.

At JAOO Aarhus 2009, Billy Newport gave the talk “Evolving the Key/Value Programming Model to a Higher Level” on monday and will present “How Cloud is working as a disruptor to shake up middleware design” aka “Challenges for elastic scaling in cloud environments” aka “how cloud computing is forcing middleware to evolve or die” in Store Sal on 14.45 on wednesday.

And for the record, Billy’s pretty sure the last title is the one going to lure the most people into the talk. Hope to see you there!

Monday, October 5th, 2009 at 22:19 in JAOO Århus 2009.