Grid Meter | May 2006

MORE ENTRIES

May 2006

May 31, 2006 | Comments: (0)

European Grids More Advanced, but Similar Problems as US

Yesterday I commented on Derrick Harris' theory as to why European grid users are more advanced than their North American counterparts. As Derrick alludes, to it really comes down to cultural issues, and as far back as I've been doing GlobusWORLD surveys, cultural issues are one of the biggest barriers to grid adoption.

In the US the car is king. We all have our own little silo-ed modes of transportation. Our roads are many and do a fine job of getting us around our own country using a set of standards that we are all familiar with. Although our roads do connect to bordering countries and our cars work on those roads, there are differences in standards of maintenance, units of measurement and units of currency for tolls.

My first trip to Europe I was fascinated by the interconnecting mesh of rail lines that allowed one to travel between and within the various countries. Rail lines within different countries have different ownership, but one can use them all with a single type of ticket. Decentralized cooperative systems at their best both in the shared nature of many passengers in a rail car and the network on which those cars travel.

I've used the roads with no cars" analogy to describe grids in the US and the lack of real useable applications available. The same concept applies to Europe. Elaborate rail infrastructures don't really mean a whole lot if there are no trains traveling on them ...

Posted by Greg Nawrocki on May 31, 2006 07:14 AM

May 30, 2006 | Comments: (0)

GRIDtoday: European Grid use more advanced, but why?

Derrick Harris -- editor of GRIDtoday -- yesterday weighed in on the fact that European Grid users are, on average, 'more advanced than their North American counterparts.'

Harris goes on to pose a theory for Europe's advanced usage: a 'community spirit' where sharing and cooperation seems to be a more common social norm between the countries (whereas we tend to think much more in terms of the individual countries - U.S., Canada, Mexico - than as a single "North America" entity).

I think Harris' theory has merit. But another possible theory I'd introduce is that some European countries' governments are specifically creating incentives that reward technology efforts that promote the national trade agenda. This results in academic and research oriented projects that are more applicable to enterprise. Indeed, European research institutions actively partner with commercial entities.

For example, the UK's Department of Trade and Industry gives government funding to technology initiatives such as the Belfast e-Science Centre, which (among other initiatives) helped build the BBC's Gridcast grid computing framework for large broadcast file transmission.

Posted by Greg Nawrocki on May 30, 2006 09:51 AM

May 26, 2006 | Comments: (0)

Grid and Licensing

A while back I commented on a plan by Cablevision to offer a digital video recording service in lieu of a piece of hardware. The plan is for subscribers to be able to "record" content to the network as opposed to needing a set top box with a comparatively large amount of computational and storage resources. Although the word "grid" was never used in the article, with distributed resources and a valuable commodity as a service, it is easy to draw parallels.

Not surprising, some of the big players in media aren't too happy about this plan and are looking for special licensing agreements for this type of use of their content.

When grids start moving from small, private quanta to larger wide-use grids and content and services from third parties start coming into play multi dimensional use licensing is going to become increasingly important. This is a topic we touch upon from time to time in the grid arena, then those discussions seem to go quiet for periods of time. I believe that those of us in the grid space need to pop this issue up a bit higher in the stack as licensing and rights management issues in the entertainment industry could be the canary in the coal mine of things to come for grid.

If it quacks like a duck it might just have the same licensing issues as a duck.

Posted by Greg Nawrocki on May 26, 2006 08:38 AM

May 24, 2006 | Comments: (0)

Microsoft joins huge list of vendors charting course for virtual machine management

Earlier this year, we singled out Microsoft as one of the interesting vendors to watch in Grid in '06 (ok, I didn't exactly go out on a limb by singling out the biggest software player as an 'interesting one to watch,' but it is true that the Grid industry at large is seriously anticipating more specifics around how Microsoft intends to play in the space).

Yesterday's news on Microsoft's virtualization intentions drew more questions than answers, and not a whole lot of meat on the bone for the Grid watchers.

What I found pretty interesting was that within the announcement, Microsoft sort of unceremoniously announced that the virtualization roadmap will include a new "System Center Virtual Machine Manager." For a company that's NOT known for systems management, it's just sort of interesting how little fanfare accompanies the "how do I manage VM's" question that customers would invariably have (Robert Mitchell's recent Computerworld article on the unique challenges of managing virtual machines is great reading on the topic).

On a related note, it's pretty amazing to consider how many vendors are scrambling to carve out ownership of this 'managing virtual machines' discussion. Among the vendors touting management solutions for virtual machines:

> VMWare VirtualCenter Control

> Levanta Intrepid M

> IBM Virtual Machine Manager

> HP Proliant

> Cassatt Collage

> ToutVirtual Inc. Virtual IQ

> Platform VM Orchestrator

> BMC Virtualizer

> Opsware Server Automation System

> Blade-Logic Operations Manager

> Red Hat, XenSource, Intel and AMD are building interesting virtual machine management capabilities into Red Hat Enterprise Linux 5.0

Those are just some of the bigger ones, off the top of my head. I'm sure I missed at least a 1/2 dozen to a dozen.
But it's a real sign of maturity for virtualization that the IQ of the industry discussion has evolved from the "what is virtualization" to the now "people are buying virtualization, and now they want tools to manage virtual environments."

Posted by Greg Nawrocki on May 24, 2006 06:44 AM

May 23, 2006 | Comments: (0)

Are grid computing contests the right driver for enterprise?

Although I don't consider myself to be an economist by any means, I have always admired John Kenneth Galbraith who passed away late last month at the age of 97. I have used his quotes in this blog before, drawing parallels between the science and of economics and the science of the of Grid.

In my last blog entry I commented on Sun Microsystem's contest to find practical applications for their Sun Grid infrastructure. Currently a "road without cars", the point of my comments were that if Sun wants to see Grid succeed, they need to be a bit more proactive and start creating applications themselves as opposed to sponsoring contests and hoping the killer application falls out of the sky.

Recently the Department of Trade and Industry and the British Computer Society in the UK launched a similar contest to find practical applications as to how Grid computing can be used to enhance existing processes and solve existing problems.

Both of these efforts seem a bit academic to me and have me questioning the business sense of these efforts in attracting true enterprise Grid end users. While this approach may be less risky I just don't see Grid succeeding in enterprise unless big players like Sun actually start producing useable Grid enterprise applications themselves.

The contest approach reminds me of another Galbraith quote: "Economists are economical, among other things, of ideas; most make those of their graduate days last a lifetime."

Economists and the science of economics certainly made the transition from graduate school to the enterprise. I have great faith that Grid can do the same but it has to involve some risk taking and a lot of real work.

Posted by Greg Nawrocki on May 23, 2006 09:47 AM

May 18, 2006 | Comments: (0)

Sun Grid Seeking Application Developers

According to Jeff Burt's article today on eWeek, Sun is working on incentives to entice enterprise developers to build applications that can be hosted on the Sun Utility Grid.

In the article, Aisling MacRunnels, senior director of utility computing at Sun, says the following.

"A key to attracting more users to the grid is building up the applications available on it. We are the infrastructure; the compute is a service. It enables ISVs to offer their apps as a service. Once these apps are there, that's when the users really get to engage."

While I agree that the key to adoption are useable apps, why would anyone want to contract for a service they generally already own, and if they don't it can be had fairly cheap with the potential of a return on that investment. Witness Salesforce.com's tremendous financial performance -- would any other would be 'software as a service' ISV turn over the hosting of the services to a third party? Hasn't Salesforce.com proved out that the hosting and integration is as much of a revenue stream as the actual service itself?

The fact that Sun is cultivating application development for the Grid is also further indication of what enterprise Grid analysts have been pointing out for some time now -- that the mainstream enterprise Grid apps are still M.I.A. But cultivating by dangling a carrot and ever hopeful optimism ain't gonna' make it so. If Sun wants this thing to fly they need to take the bull by the horns and "Carpe Gridem."

I've heard it said by others and repeated it myself. The killer app for Financial Services, a vertical market that has traditionally been the most pro Grid is MS Excel. Doesn't Sun own Star Office, which includes Calc, a MS Excel compatible spreadsheet? Gridify that and I think you've got something. I really do like Sun Microsystems, so they can have that idea for free.

I believe I've said this before, so forgive me if I repeat myself, but what if Henry Ford decided to build roads instead of cars, and just left the car building up to someone else? That's sort of where enterprise Grid is today -- there's been a ton of activity and involvement in building the underlying infrastructure, but no one is coming to the party with the apps that are uniquely equipped to take advantage of this new performance. Party time is over, it's time to buckle down and start producing those apps.

Posted by Greg Nawrocki on May 18, 2006 08:24 AM

May 17, 2006 | Comments: (0)

Nortel proving out virtual mash-ups on the Grid

In previous blog entries, we've discussed the common conundrum faced by organizations with large, distributed data sets ... do you move the data to the compute, or do you move the compute to the data? There's been a fair bit of discussion about GridFTP and related protocols for bulk file transfers for Grid environments. And, of course, the very nature of Grid computing is to provide resource pools that can be dynamically provisioned to jobs in the queue.

But a very exciting and NEW, related discussion is taking place around "mashups" of virtual machines and virtual network resources. This new research area is being driven by Dr. Franco Travostino -- who heads Grid activities in the capacity of Director for Advanced Technology and Research at Nortel and is co-Director for the Infrastructure Area at the GGF -- and who is without question one of the most interesting players on the networking side of the enterprise Grid discussion.

Travostino and his team have made some fascinating breakthroughs with Xen hypervisors and Nortel's own DRAC (PDF: 'Dynamic Resource Allocation Controller').

"In Seattle, at Supercomputer 2006, we created a demo by which we took some Xen-based virtual machines that were crunching some particular, computation-intensive tasks ... and we moved them to Amsterdam," said Travostino. "And then from Amsterdam, we moved them to Chicago, and them from Chicago back to Seattle. And in spite of the tens of thousands of miles, the impact on the applications was less than one second. So that's pretty mind-boggling, to think about having a fully featured Linux environment running lots of applications, and teleporting all that across the world with such minimal disruption. DRAC is the "network middleware" that makes this long-haul migration possible at the network level. Specifically, DRAC puts in place a short-lived deterministic network service, on demand. As well, it preserves the sessions with any remote client."

Later this year, Travostino is running an essay chronicling the experiment in the Elsevier Future Generation Computer Systems publication ... but to paraphrase a few of the specific reasons he sites for why virtual network resources + virtual machines is so exciting to the Grid community:

* "[Often] it is impossible to bring data (or devices) closer to the computation engines. This may result from policy limitations (e.g., a data set that is embargoed from export) or capacity limitations (e.g., a data set is exceedingly large, thus adding an unwieldy preamble to the computation phase). An application's style of operation may exacerbate the latter characteristic (e.g., an application that infers from intermediate computation results which data sets are required next, lending itself to traveling salesman optimizations);
* It is desirable to load balance computation workloads with a scope that transcends the confines of individual data centers. One policy may dictate that during off hours the workload be consolidated into fewer data centers over a regional area to limit operating expenses or power consumption. Another policy might dictate that the workload "tracks the moon" to harvest spare computational power in different geographical areas that happen to be off business hours;

* It is required for operational business continuance, disaster recovery capabilities while meeting regulations for geographic diversity. The migration of virtual machines enables orderly evacuation of computation out of data centers that experience or anticipate failures, security compromises, or storms in their access network."

For all the discussion and excitement around virtualization, relatively little has been said to date about the specific network resource management requirements that are specific to Grid. In addition to these recent R&D proof of concept efforts by Travostino and the team at Nortel, many of these issues will be addressed in his upcoming book later this year: "Grid Networks: Enabling Grids with Advanced Communication Technology." The book -- edited by F. Travostino, J. Mambretti, and G. Karmous-Edwards with a dream team of chapter authors -- is a comprehensive look at the network implications / requirements of Grid -- from the different layers, down to the packets, circuits and optical (and even wireless).

Posted by Greg Nawrocki on May 17, 2006 06:57 AM

May 15, 2006 | Comments: (0)

GGF launches Grid standards wiki

The SCRM working group within the GGF has launched a Wiki that attempts to corral the work done by the many different organizations that have their fingers in the cake of emerging grid computing standards.

SCRM stands for 'Standards Development Organization Collaboration on Networked Resources Management' which in and of itself is a mouthful. The Wiki is a collection of descriptions with links that leads the user to text on the actual standards and specifications.

This is a non-trivial task and the list of standards and specifications is quite lengthy. Included is a landscape section that is a proposal for a taxonomy to help define how this mass of specifications and standards organizations all relate to one another.

While I do believe that this is a noble effort, at a high level it seems to me that this is a whole lot of cooks to fit into one kitchen, especially given that there is more recipe writing than actual cooking going on ...

Posted by Greg Nawrocki on May 15, 2006 08:36 AM

May 10, 2006 | Comments: (0)

IBM developerWorks: Managing Big Data Sets on TeraGrid

IBM developerWorks ran an article yesterday that looks at the two major data management approaches in the TeraGrid. TeraGrid is a massive Grid delivering computational resources to support scientific discovery. The Grid delivers 50 teraflops of computing power, 2 petabytes of rotating storage -- and supports seven different operating systems and a dozen distinct system architectures.

Even before TeraGrid secured an additional $150 million in funding earlier last year, the community at large has been following TeraGrid closely to observe how it reconciles the huge data demands of the research scientists tapping into its power. The top disciplines using it are chemistry, physics and molecular biology -- all of which have tremendous data sets under their purview, and all of which heavily rely on the ability to transfer, compile and analyze these data sets to drive scientific discovery.

In another recent TeraGrid press release, it's not just big science using TeraGrid, at least not directly. Scientists recently used TeraGrid to create a graphic simulation of the collision of two galaxies that will be used in the finale of a new planetarium show for the American Museum of Natural History.

The IBM article drills down on TeraGrid's use of GridFTP -- the popular extension to the FTP protocol that specifically enables large data transfer for Grids. The article also discusses TeraGrid's use of IBM's GPFS-WAN file system, which has introduced some interesting new features and security capabilities for providing a parallel file system on a Grid.

Posted by Greg Nawrocki on May 10, 2006 08:38 AM

May 09, 2006 | Comments: (0)

Mash-ups on the Grid ...

By definition, Grid applications must be consumable across a range of heterogeneous platforms and environments. But as Peter Yared noted yesterday, "if you look at any J2EE application, it wasn't written to scale beyond four to eight machines ... and most hard core enterprise applications were simply not written to perform in scale-out, heterogeneous Grid environments."

And as Martin Brown from MCslp puts it: "A perennial problem with grid applications is making them flexible enough to be used across a range of potential platforms and environments. While older grids used a dedicated solution with rigidly controlled hardware and environments, it has recently become clear that making your grid application run on a wider range of platforms enables you to easily expand the scope and power of your grid simply by adding more machines."

So as the community works to figure out how to make applications scale better in Grid environments -- we've been hearing a ton of encouraging discussions over the last couple of years about how service-oriented architectures and convergence on Web services standards are breaking monolithic apps into interoperable components for use across heterogeneous environments.

Now we're also starting to hear some experts point out that SOA development directions are going to start surfacing the types of "mash-up" applications in enterprise that we've been seeing sprout up in the consumer world over the last year.

"Naturally, enterprises have taken notice of the rise of 'mash-up' applications on the Internet that integrate data from a variety of sources in new and useful ways," said Yared. "Large organizations also want to deliver composite apps, like CRM applications that can call other services. That way they can show, for example, that a customer's last five orders have been delayed before they make a sales call and get ambushed by an irate person on the other line. The days where each enterprise application is an island are coming to an end--even things as simple as an employee directory now need to integrate the HR systems of multiple divisions, accommodate cross-reporting and virtual teams, and integrate outsourced third parties."

And as Matt Haynos, program director for Grid computing strategy and technology at IBM, points out, the enterprise mash-up is increasingly within reach as SOA continues to gel in enterprise.

"As companies are moving towards SOA and they're more comfortable, you're going to start to see mash-up application development. We're at the very early stages, but it's sort of in the outside communities where developers are taking these well-exposed services and developing the innovative new composite applications. Once you have all of these services created, you can much more easily write applications that utilize these services and do interesting things and choreograph these services in interesting ways. So I'm a big believer that as SOA continues to mature to a services-based view, running on a virtualized infrastructure - we're going to see a lot more mainstream enterprise mash-ups being developed."

It may turn out that the enterprise mash-up becomes the Grid's first "killer app" in the commercial world. When you talk about breaking a monolithic app down into thousands of services -- that could require access to a certain quanta of data, storage, network bandwidth, etc. at any given time -- you start to introduce new resource management issues that Grid is specifically equipped to deal with.

Posted by Greg Nawrocki on May 9, 2006 08:06 AM

May 08, 2006 | Comments: (0)

The enterprise Grid application challenge

It's no secret that in the commercial world -- with the exception of niche vertical market examples (Monte Carlo simulations in financial services, et al.) -- there are precious few 'enterprise-class' Grid applications to point to.

So why is it that it's so tough to 'Grid-enable' transactional enterprise apps? In the words of ActiveGrid CEO, Peter Yared:

"The Grid community -- before they solve world hunger with the 'big pool of virtualized resources that you can tap into' vision -- needs to concentrate on making enterprise-class applications that can scale horizontally in their own dedicated clusters. That's step one, because if you look at any J2EE application, it wasn't written to scale beyond four to eight machines.
Back in the late 90's, customers wanted to run small clusters of SMP machines. That's why Sun made so much money. Now that Google and Yahoo! and Amazon have started running large clusters of commodity machines, everybody wants to do that. But you can't take the software that you wrote on small clusters of SMP machines and expect it to scale horizontally to 200 or more machines in Grid environments.

When you're running transactional applications, you typically need to run commodity machines that are identical. For example, you want them to be Linux and x86, and on one subnet or on one set of network controls. And that's the only way it's going to scale. So there's the Grid vision that you can have free resource pools -- of any sort of heterogeneous combinations -- available to jump in on the fly. But for transactional applications, forget about it.

So that's the fundamental application challenge that the Grid community faces -- most hard core enterprise applications were simply not written to perform in scale-out, heterogeneous Grid environments."

Yared is also a big evangelist on the benefits of integrating services on the front end, on LAMP ... and he sees LAMP challenging Java and .Net as the preferred application infrastructure for running service-oriented applications.

For the Grid community's push towards commercial adoption -- I've often thought that a Grid-enabled version of Apache might be a step in the right direction. As an application itself, the benefits of Grid for Apache load balancing and data virtualization resonates with me. And as an application platform, Apache already has the benefit of the "P" languages as an established platform for development.

The good news is that there are indeed efforts underway in this direction such as pyGlobus, the Python interface to the Globus Toolkit. As this matures it will be interesting to see if this will be a catalyst for the development of innovative Grid applications.

Posted by Greg Nawrocki on May 8, 2006 08:23 AM

May 02, 2006 | Comments: (0)

Chambers Talks Network Intelligence @ Interop

John Chambers kicked off Interop this morning, using his exuberant (per the usual) morning keynote to remind us that the IQ of the network is only going to continue to rise to support the next wave of networked IT solutions and collboration / unified communications.

Chambers also outlined Cisco's view that the "Network is the Platform" -- and that virtualization of storage / applications / processors is leading us to a new era where the applications themselves are relatively thin, with the heavy lifting of security and performance requirements abstracted into the network. Essentially, a "services-oriented network achitecture" -- where the network determines what type of device is accessing what type of data, and the right info is presented to the right folks at the right time.

A lot of Chambers' fodder at the top was focused on generic agility / business flexibility discussions. But he did get into some interesting discussions about how collaboration across IT groups (and 'breaking down silos') is pushing the need for this service-oriented network platform.

He called out transportation - a $2.9 trillion annual industry - and showed attendees how IP sensors throughout a railway system could bring real time monitoring of engine performance, routes, security, maintencance, etc. into a single view. Similarly, he pointed to the trucking industry (60 percent of US freight is carried by trucks, according to Chambers) and ran the audience through a fleet management application that shows where clogs are occuring in the highway system, allows real-time re-routing ("smart highways"), and provides the driver with what looked like a GPS system on steroids (all over IP).

The idea that applications will be dumbed down -- with the critical details abstracted into the network, is one that has varying industry opinions).

Posted by Greg Nawrocki on May 2, 2006 11:06 AM

May 01, 2006 | Comments: (0)

Rising Telco Interest in Grids ...

Xchange ran an interesting editorial today about the telco industry's increasing interest in Grid computing.

The author cited billing systems (where "Application virtualization can significantly improve performance for billing and statement generation"), business intelligence (where data processing demands are intense), and product development (where Grid can increase speed-to-market) as three core areas where telcos can put Grids to work to tackle some of their hariest problems.

There are some very meaningful efforts underway today to support the budding telco market for Grid computing. Franco Travostino from Nortel Networks (and one of the heads of GGF's infrastructure group) heads a Telecommunications Community Group, which is scoping out some of the key technical directions / considerations that must be addressed for full-blown telco Grid computing adoption.

Some of those areas (according to GRIDTODAY) include):

1. How network providers/telcos will need to evolve their networks and services in order to support large-scale Grid traffic and other novel requirements posed by Grids? Some participants felt that point-to-point connections were initially required (rather than LAN multipoint-to-multipoint) to facilitate the deployment of large-scale, reliable and available Grids.
2. How network providers/telcos might use Grid technologies to improve their own internal operations (e.g., billing, event analysis, network planning and modeling). In particular, how telcos might virtualize their data centers/ operations support centers? This could provide huge improvements in total cost of ownership/operational expenses and return on investment (ROI).

3. How can network providers/telcos become suppliers of Grid managed services while adopting profitable new business models? This could be a managed network service alone or combined with some type of IT resources (e.g., storage, server hosting, application specific computing).

As Interop kicks off, I'll be on the hunt for other network-specific Grid themes and considerations throughout the week.

Posted by Greg Nawrocki on May 1, 2006 12:33 PM

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert