Grid Meter | April 2006

MORE ENTRIES

April 2006

April 26, 2006 | Comments: (0)

IBM's latest forays into the SMB market are really interesting to watch. "If IBM seems too big for your mid-sized business ... call 1-877-IBM-ACCESS to resize us," reads their online storefront today. It's clear that they are making a very concerted effort to dispel the notion that IBM is for large enterprises / deep pockets only.

In yesterday's announcement of three new SMB offerings, the third was the latest "Grid and Grow Express" release.

"IBM Grid and Grow Express -- a complete package, including hardware, software and services, that enables first-time customers to take advantage of grid technology. The offering consists of BladeCenter with seven blades, along with a choice of server architectures, operating systems, schedulers and services that tie these technologies together. This package positions SMBs for future growth, as well as provides the technology to address business challenges associated with mission critical, compute-intensive applications. Available at $1,369 per month for a 36 month term. Available in the US, the UK, Canada, Germany, France, Belgium, Netherlands, Italy, Spain, Portugal, Greece, Israel, Turkey, and Luxembourg."

So it sounds like the equivalent of leasing a car. You get all the hardware and middleware to start running a Grid in your SMB environment -- without having to build it from scratch yourself, and presumably with IBM services to help you out if you screw anything up.

At $1,369 per month over three years, you're paying $50,000 for the use of 7 blade servers and the accompanying job schedulers and middleware. If you bought 7 SUSE Linux enterprise blades from Novell at $899 apiece, you could run Altair's OpenPBS for free, use the Globus Toolkit as the middleware for free, and be ahead >$40k.

However, how much time would your sys admin(s) spend on the home grown open source Grid if they went that route? Over three years -- probably more than $40k.

So IBM has packaged up something that does seem to make economic sense for SMBs ... and there's no question that there are plenty of mid-market customers that need the performance of Grid, but want to avoid cobbling together everything on their own.

According to IDG News' China Martens, who broke the story on Grid and Grow Express yesterday:

"Sanjeev Aggarwal, senior analyst for Yankee Group's SMB strategies decision service, described the pricing as "very aggressive" and likely to appeal to SMBs weighing the cost of buying additional PCs versus moving to share high-performance computing resources via grid technology. 'There's definitely an interest [in grid] among midmarket verticals including education and pharmaceuticals,' he said."

Expect other vendors to come out with similar bundled Grid solutions on lease programs ... we're going to see some interesting price models and competition as the big iron guys keep duking it out for mid-market revenues.

Posted by Greg Nawrocki on April 26, 2006 09:13 AM

April 24, 2006 | Comments: (0)

Grid 2.0 discussions premature ...

Tom Gibbs from Intel has an interesting write-up on GRIDtoday that draws comparisons between Web 1.0 / Web 2.0 and what we've seen thus far from the Grid, versus what we will see with "Grid 2.0."

He says (get the full version at GRIDtoday):

"I think that the Grid community is going through a similar metamorphosis from "Grid 1.0," which was born on the Web in the early 1990s along with Web 1.0. Interestingly, many of the same folks and the organizations they worked for were involved with the birth of both. The benchmark application for Web 1.0 is Netscape, which had its roots in Mosaic, which was developed at NCSA. Similarly, the benchmark application for Grid 1.0 is Globus, which has its roots with the distributed computing project at Argonne National Laboratory. Maybe it's the water in Illinois.
As with Web 1.0 evolving to Web 2.0, the fundamental concepts remain the same. It is a natural evolution that is happening in a very organic way, largely stimulated by the open source community. The focus of Web 1.0 for the ISV was still to sell software or specialized services and provide access to the Web. With Web 2.0, the ISV is selling a service, and the Web is packaged as part of the service. Similarly, Grid 1.0 included a vision along with the basic building blocks for ubiquitous computing and communications, but its focus was on sharing compute cycles. Grid 2.0 will usher in a new world of distributed ubiquitous virtual computing, networking and storage in the enterprise that will allow a whole raft of new rich services. I started to see this coming with the demographics at SC'05 in Seattle. The scientists were being outnumbered by folks usually associated with content or content delivery."

I agree with many of Gibbs' points. But where the analogy falls short for me is that it presupposes that Grid 1.0 has actually happened. Web 2.0 discussions really got off the ground AFTER the commercial Internet had truly gelled with mainstream businesses. And before we start talking about Grid 2.0, we need to see Grid 1.0 through the final stages.

As Irving Wladawsky-Berger from IBM put it:

"If I compared Grid computing to the early days of the Internet and the Worldwide Web, I would say we are still pre-'95. Lots of people know about Grids, there are already quite a number of successful commercial pilots of Grids, and everybody in the research and supercomputing community is involved with Grids already. But the problems being tackled by Grid computing, the whole general area of distributed systems based on open standards over the Internet, is an incredibly complicated set of technical problems. So it's not surprising that it's taken time to develop the needed architectures and the needed software for Grids to be acceptable for general production use. There's lots of work being done, lots of good progress, but we are still in the development phase, in my opinion."

Grid 1.0 is still happening largely in research / science. Grid 1.0 does not yet have the pervasive enterprise use that even remotely resembles commercial Internet use in the late 90's / early 00's. Yes, there are signs that we're moving in that direction. However, IMO, Grid 2.0 is a bit of a red herring.

Posted by Greg Nawrocki on April 24, 2006 08:17 AM

April 20, 2006 | Comments: (0)

Network World face-off on app-aware networks raises interesting questions

Earlier this week, Network World ran one of its "Face-offs" -- between an engineer from Cisco's Application Delivery Business Unit, and a technical director from the office of the CTO at Ciena. The debate -- on whether users need application-aware networks -- extends the discussion of where intelligence should reside (and not reside) in the network.

In Issy Ben-Shaul's (of Cisco) argument FOR application-aware networks, he concludes that "by embedding intelligence and common functions, application-aware network products offer a more easily managed and more affordable way to deliver higher levels of performance and add greater functionality across an IT infrastructure."

Meanwhile, in Ranghu Ranganathan's (of Ciena) argument AGAINST application-aware networks, he says that "Coupling ever-evolving applications with stable network elements doesn't make economic or operational sense."

This particular debate isn't just interesting from an application performance perspective ... it's also a really interesting continuation of the blurring of the boundaries between network and systems management. Networking pros have long pointed to this "blurring of the boundaries" (http://weblog.infoworld.com/gridmeter/archives/2005/08/systems_managem.html) between operating systems, networks and middleware -- and now with the heavy web services flavor of today's enterprise applications, the networking players have more opportunity to interact with the applications and control them directly.

As Cisco's Application Oriented Network business unit continues to grow, it's likely that we'll hear additional debate on how far network intelligence should extend its hooks into the applications.

Posted by Greg Nawrocki on April 20, 2006 08:47 AM

April 19, 2006 | Comments: (0)

The 451 Group: Financial Sector Grid Adoption Growing Beyond Mere Compute

The 451 Group yesterday announced their latest "Grid Adoption Research Service Report" -- this time providing an update on the financial services industry's fast-evolving Grid computing usage patterns. The study itself is available to clients only, but in the preview it appears that they have canvassed a very comprehensive list of all the big names on Wall Street that one would hope to see in such an update (Bank of America, Bank of Montreal, BNP Paribas, Bowne, Citigroup, Genworth, HSBC, JPMorgan Chase, Markit, MassMutual, Nationwide, Royal Bank of Scotland, Societe Generale, TD Bank Financial Group, UBS, Wachovia and WestLB, etc.).

The abstract suggests that Grid adoption in financial services is quickly moving beyond just the compute Grids that support complex simulations (like Monte Carlo) -- and creeping into more all-encompassing resource and data sharing requirements. The abstract also suggests a synergy between the financial services industry's SOA investments and their Grid interests, which is not surprising, given Grid and SOA's common ground in web services standards.

Financial services' aggressive push towards using Grid and SOA as a "fabric"
for tying together IT islands is very consistent with the feedback that Deborah Williams -- leader of the capital markets practice at research firm, Financial Insights -- shared recently with Ian Foster (http://www.computerworld.com/hardwaretopics/hardware/gridcomputing/story/0,
10801,101350,00.html):

"We have huge integration issues in our industry, I think because of the natural best-of-breed predilection of most of the firms. Over the last 20 years, we've picked the best solutions for the problems."
"So you end up with a very disparate, wide-ranging, siloed kind of architecture. And this over time has led to huge integration issues."

In yesterday's blog entry, I referenced that underutilization is not necessarily considered to be a huge "problem" in enterprise IT. With the commoditization of hardware -- in particular -- folks are generally ok with the fact that they only get X-percent utilization out of existing equipment. Integration, on the other hand, is a HUGE problem that really speaks much more directly to the wallets of IT managers at Global 2000 companies.

Posted by Greg Nawrocki on April 19, 2006 06:43 AM

April 17, 2006 | Comments: (0)

Univa announces general availability of enterprise Grid computing software

Enterprise Grid start-up Univa has announced the general availability of its "Univa Globus Enterprise" (UGE) product, an "open standards platform for deploying enterprise Grid solutions." The official G.A. follows the beta announcement back in December.

There are a few particularly interesting aspects to how Univa is describing its product.

First, it's a commercial open source platform for deploying Grid solutions (i.e., the benefits of open source, minus the headaches of installation / configuration that usually accompany raw open source ... AND lets you pick and choose from other best-of-breed open source Grid solutions and easily plug them in for your specific needs). Second, Univa has really emphasized "data availability" as a key differentiator (CTO Steve Tuecke previously discussed the large enterprise data management issues that Univa is tackling).

Another interesting element to Univa's positioning is that it's emphasizing "what's possible" with Grid ... rather than presenting it as a fix to enterprise problems. I think that too many Grid solutions today are being presented as a fix, usually to underutilization of resources. What's slippery about that is that underutilization is generally NOT perceived as a mission critical problem. And realistically -- how many enterprise end users are going to stick their neck out and choose Grid computing as the fix to their overall data and resource integration problems (current market numbers suggest that's not happening at a very rapid pace right now)?

So Univa is instead appealing to the imaginations of IT decisionmakers by suggesting the competitive edge that Grid can provide -- in getting a product to market faster, discovering or designing something better or sooner, etc. This is really what's going to push enterprise adoption. It's going to happen first in R&D (which most closely resembles the research / scientific world where Grid's origins are) and then slowly start creeping into more mainstream production environments.

Posted by Greg Nawrocki on April 17, 2006 10:29 AM

April 17, 2006 | Comments: (0)

Looking Ahead to GridWorld 2006

There's good news for serious enterprise IT professionals who are interested in Grid, but annoyed by the prospect of keeping tabs on all of the different Grid bodies and making sense of their relevance to enterprise. At this year's GridWorld in September, they'll all be in the same place at the same time.

The GridWorld event has officially kicked off the program info and call for Speakers (GlobusWORLD)
(GridWorld) for the first ever Grid computing event that's rolled GlobusWORLD, the Global Grid Forum, and the Enterprise Grid Alliance into one event under a single roof.

The GGF will be there to provide the latest info on Grid standards and implementation best practice. The Globus Toolkit folks will be there to provide the most current technical tutorials for leveraging Grid's most pervasive open source middleware. And with the Enterprise Grid Alliance's participation, you get a large roster of vendors that will be announcing and discussing the latest commercial Grid solutions.

Posted by Greg Nawrocki on April 17, 2006 08:36 AM

April 13, 2006 | Comments: (0)

Getting the Internet Ready for TV ...

Earlier this week, Richard Waters from the Financial Times broke the story that the public Internet's performance may not be enough to sustain the sort of traffic demands that might be created when the large networks start pushing ad-supported shows over the Internet. This was in response to the Disney / ABC announcement that it plans to offer some select shows on the Internet in a streaming, non-downloadable format. According to Waters:

"[S]treaming video, which relies on delivering "bits" of data to viewers in real time, has exposed weaknesses in the fundamental design of the internet, which is based on being able to send data through a series of "hops" across the network between two distant computers."

Last year, Joe Touch, director of the Postel Center, told me that with the public Internet, "everybody gets an Acura ... you can tune things so they don't break, but you don't build a Maserati on the Internet." In other words, the public Internet was architected for the greatest common good, and NOT necessarily for the streaming data demands of the major television networks.

So is it possible that the networks will start laying dedicated pipes to ensure the performance of content delivery? It doesn't seem like that big of a stretch of the imagination. However, it's also possible that the television networks will look to Grid pros like the Belfast e-Science Centre in the UK, whose Gridcast project is already helping the BBC manage its enormous media file transfer issues and would certainly present some interesting scenarios in helping US networks push content out to viewers over IP.

The streaming implementation that ABC has chosen was selected in part so that viewers are prevented from fast forwarding through sections (read: commercials) as one would be able to do with media that was available in a conventional download. But if there were some hybrid system where media could be downloadable but secured in a way to prevent rebroadcast, multiple uses, or even fast forwarding through sections, bandwidth concerns of streaming methods would be mitigated. Not that Grid is a plug and play solution for this, but these topics are certainly familiar to those in the Grid space.

One thing's for sure ... over the last 20 years, consumers don't seem to mind how they get their TV content, as long as there's some sort of pipe that comes in from somewhere outside (a satellite, the local cable company, etc.) that they can plug into and become one with their couches.

Posted by Greg Nawrocki on April 13, 2006 07:56 AM

April 11, 2006 | Comments: (0)

Storage Virtualization or Too Good to Be True?

It's no secret that there are rigorous storage requirements for apps running on a Grid. As XenSource CTO Simon Crosby previously said:

"When you deal with a broader set of enterprise applications, you'll find that there are dependencies which are extremely difficult to resolve in terms of solving the allocation problem. For example, a particular app may need access to a particular dataset that lives on a particular SAN on a particular LUN, and you've got to deal with the storage virtualization world to solve that problem. The Grid's I've seen in action don't deal with complex storage architectures -- the data sets are local to the computation or are made available from a centralized server. But for a broader class of enterprise applications, the storage problem must be addressed. Many applications are now interlinked, for example a Web service today is composed of a Web server, an app server, and a database. And creating an instance of the service and placing it on the infrastructure requires the placement of all three running components."

As Crosswalk, Inc. officially launched the iGrid Intelligent Storage Grid System today, they described the Holy Grail for Grid storage demands (according to the release): "iGrid is a vendor agnostic grid technology that enables any application to access any authorized storage resource, any iGrid node to access any and all file systems, and all users to concurrently access any data they need for collaborative projects."

But as Chris Mellor pointed out in his write up this morning, Crosswalk's arm-waving does not provide any substantive technical details that would indicate whether this is real (as Mellor puts it, "There is no information on how existing storage products can be linked into iGrid, nor about how iGrid relates to other storage grid concepts, such as HP's smart cells or the GridFTP file transfer protocol. CrossWalk's iGrid information gruel leaves you asking for more").

Two or three years ago, if you recall, there was an onslaught of new systems management start-ups describing the future of "dynamic provisioning" in "scale-out, commodity environments." When reporters tried to dig a little deeper and test these vendors' stories, critical missing pieces tended to be immediately evident ... for example, many did not support Windows environments (no trivial absence for the majority of business users).

Crosswalk describes the finish line in their company launch press release, but I concur with Mellor that there's not nearly enough information about how they're going to take us there.

Posted by Greg Nawrocki on April 11, 2006 08:21 AM

April 07, 2006 | Comments: (0)

Ian Foster and IBM's Irving Wladawsky Berger Catch Up Re. Grid ...

Grid pioneer Ian Foster recently caught up Irving Wladawsky-Berger (a key On-Demand guru at IBM) for his latest perspectives on the Grid evolution.

What's particularly interesting about the discussion (concluding my other entries this week re. the Linux / Grid relationship) are Wladawsky-Berger's thoughts about the synergies between the two communities that could potentially take shape over the near future. Wladawsky-Berger also provides some interesting insights about how open source Grid projects might take advantage of existing open source "stacks" such as LAMP to gain broader adoption. It's an elaboration on similar perspectives that we've heard from Grid pundits such as William Fellows from The 451 Group in the past, and quite interesting ...

Posted by Greg Nawrocki on April 7, 2006 07:09 AM

April 07, 2006 | Comments: (0)

A Quick Primer on Linux Clustering from Donald Becker

I recently had the opportunity to catch up with clustering pioneer Donald Becker, who started the Beowful project, and is now the CTO of Penguin Computing. I asked him a few questions about the history of Linux clustering, and here's what he had to say ...

Q: Tell us a little bit about the genesis of the Beowulf Project, and how Linux clustering has evolved since those early days.

Becker: Beowulf started out as a way for people to use collections of commodity, off-the-shelf Linux machines for high-performance computing ... as an alternative to using purpose-built, specialized machines.

The real key to doing that is providing a software layer that hides, as much as possible, the "ugliness" of machines not designed for high-performance computing. So it's about a software system and a methodology to put together machines that can be used effectively in high-performance computing.

From the beginning of the Beowulf project in 1994, we targeted Linux as the platform for the software we were deploying. At the time Linux had a very small presence in high-performance computing and the market in general. I like to think Beowulf had a strong influence with Linux becoming popular for doing clustering for the purpose of high-performance computing. This eventually led to Linux being a popular platform for doing all sorts of things in the HPC realm.

Q: So what were some of the specific challenges for clustering commodity Linux boxes for HPC?

Becker: In the early days, the challenge was as simple as getting the machines to talk to each other - so my background on the Linux side was contributing to the networking side of the Linux kernel. Getting the machines to communicate meant figuring out a lot of the high-throughput, low-latency communication requirements for clusters. From there, the focus turned to communication libraries and managing large sets of machines. One of the things about scientists is they'll put up with quite a bit in terms of complex systems. The rest of the world, however, wanted to put complexity in the background -- to minimize the complexity -- because they're much more focused on their own specific applications.

Q: What are some of the unique management issues in Linux cluster environments?

Becker: Our focus on the cluster management side is consistency. We're trying to make it look like a single system from the point of view of the end user and the administrator. We want to guarantee that a process being run remotely will return the same results as one left running locally, even in the face of library updates, application updates, user setup updates.

To accomplish this you must administratively control what's installed on the machine. Inside of a cluster, we focus on dynamically provisioning machines, making certain that we control every detail of how the machines are installed. So we go the whole way down to loading kernels and managing device drivers and up to the level of making sure the right versions of libraries are there.

One of the things we do within a cluster is we try to guarantee consistency -- that an application you run on a remote machine will run exactly the same as it does on a local machine. It's a lot easier to guarantee consistency within a cluster than it is over a Grid. When you have local-area high-bandwidth communication and have administrative control over the machines, you have a lot more opportunities for consistency. The challenges are the same for a Grid, but we were able to pick an easier set of problems to solve within a local cluster.

Q: What sorts of applications are better suited for a cluster than a Grid?

Becker: One of the reasons to select a cluster instead of a Grid is to run applications that require low-latency communication, and you're pretty much constrained to do that on local machines. You can get very low-latency interconnects for clusters. That's difficult to accomplish with a Grid. So there are some application characteristics that preclude them being run effectively over a Grid unless you're doing it at the coarsest grain level.

One class of applications, "spectral methods," requires all-to-all communication with each time step. And the length of the time steps often are determined by the latency of that communication, so many of these applications really only effectively run on machines that are local to each other.

Now, on the other side of that, a surprising amount of computation today is parametric execution. I think a decade ago, people didn't really expect this. Today you have machines powerful enough to run pretty big simulations. What you need is to run hundreds of thousands of similar simulations, but with different input parameters. For workloads like that, wide schedulers - wide-area scheduling systems in Grids - are very effective. Of course these are also very easy tasks for clusters to do.

Posted by Greg Nawrocki on April 7, 2006 07:05 AM

April 04, 2006 | Comments: (0)

Why Linux is the preferred OS for Grid ...

What is it about the Linux operating system that makes it particularly well-suited for Grid computing? I put that question to Carl Drisko -- Service Line Principal of Open Source and Linux at Novell -- and here's what he had to say:

"From a performance standpoint, when you're working with Linux, you get very close to the metal. So there isn't a lot of systems overhead, and you can take full advantage of the hardware that you've got, as well as support customization. For example, there are Grid users out there that are accessing a bunch of sensor data. They need to be able to very easily write their own device driver to go and grab some data off of potentially unique devices.

Grid users often aren't just buying solutions off the shelf, they're building them themselves. Linux allows much more flexibility to do what you want, to create custom installations. If something isn't there, you can more easily add it in and make it work better on your own - which is something that's attractive to a lot of people, particularly researchers. Despite all the talk about Grids from some of the major hardware players, there's not a heck of a lot of good software out there, so you still have to do a lot of work yourself to fill the gaps.

Another obvious factor for Linux's popularity in Grid environments is cost. Anytime you're deploying 100 or 1,000 or 10,000 boxes, licensing costs become a big consideration. Cost may not be a big consideration for the financial services world - but for any research institution, where Grid traction has been strongest to date, cost is a huge consideration. Beyond the O/S, it is just as big a factor, if not bigger, for other "up-the-stack" software. And in many cases, the proprietary software (and even some open source) licensing models have not adjusted to Grids."

Posted by Greg Nawrocki on April 4, 2006 08:29 AM

April 03, 2006 | Comments: (0)

Room for More than One Default Hypervisor in the Linux Kernel?

A year ago, the news that Xen virtualization support in the Linux kernel was just around the corner grabbed headlines.

For the IT community -- and for Grid technologists such as Katarzyna Keahey at the Globus Alliance, who is working with Xen hypervisors in her "Virtual Workspace" Grid projects -- this was big news. After all, "If you use open source software, your choice of available versions and scalable support may in practice be limited," as she said in an interview last October.

Indeed, for any organization to get the benefits of Xen virtualization, they need to "roll their own" Linux -- meaning take a standard Fedora Core or Novell SUSE or other distro and recompile with the Xen patch. Patched kernels, however, are typically not supported by the major Linux distros, or, more critically, third party ISVs such as Oracle, SAP or IBM. Naturally this tends to make end users a little uneasy. So Linux kernel support of Xen created interesting speculation about the end user confidence it might inspire in open source virtualization.

But today, the Xen hyperpatch has still not yet been merged with the Linux kernel. And in the meantime, some believe there may now be an opportunity for more than one hypervisor in the Linux kernel.

"Architecturally, there are increasing discussions in the industry about the equivalent of an API on a hypervisor level, within the Linux kernel," said Andrew Morton, maintainer of the Linux 2.6 kernel, whose full-time work on the Linux kernel is sponsored by the Open Source Development Labs. "Theoretically, different hypervisors could play in there, without it being restricted to one default hypervisor. Different, competing hypervisors would be able to provide the virtualization functionality, and ultimately the virtualization itself would become so transparent and good that users would not need to re-certify the same version of the kernel when running on top of a hypervisor."

A year ago, the headlines suggested that Xen had won the Linux virtualization battle. But has the delay in the Xen patch for Linux kernel support left the door open for VMware?

With a common API to hypervisors in the Linux kernel, the hypervisors themselves would become a commodity - shifting the power struggle higher up the stack to the management functions on top of the hypervisors. That's a game that VMware would rather play. Perhaps they'd even open source their underlying hypervisor?

Only time will tell. But you can be sure that the next year should continue to be an interesting one to watch in the battle for Linux virtualization market share between Xen and VMware. And keep an eye on Virtual Iron. They may provide yet another option under that possible Linux kernel API.

As the LinuxWorld expo kicks off, I'll be focusing on Linux / Grid directions throughout the week.

Posted by Greg Nawrocki on April 3, 2006 07:29 AM

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert