Grid Meter » December 2005

December 22, 2005 | Comments: (0)

Monitoring and Discovery in Grid environments

TAGS: None

By nature, Grids have exponentially more moving parts than the typical, silo'd architecture that today's enterprises are moving away from, where dedicated hardware stacks support specific applications in a very static way. In Grids, resources come online into the production environment, and they can just as quickly go back offline. Jobs start, take up a certain quanta of resource to run, and then that resource is freed up again for the next job.

So how in the heck do Grid pros get real-time insight into what's happening, where and when?

In Globus Toolkit Grid environments, this need is satisfied by the Monitoring & Discovery System (MDS), the "information services component of the Globus Toolkit [that] provides information about the available resources on the Grid and their status." On the client side, users reference the MDS user interface to get the real-time view they need of their Grid resources. On the "provider" end, MDS allows Grid participants to create the necessary interfaces that allow other users to get access to their resources.

This is done using meta data constructs in XML format.

According to Jeffrey Hollingsworth and Brian Tierney (in Grid 2: "A major focus of MDS's design is achieving scalability in a system with large numbers of information providers and consumers."

"MDS uses a variety of plug-in modules to ingest XML data," added Mike D'Arcy, programmer at ISI. "From processes, to files, to remote servers, to connections over a socket to a non-WSRF service -- the system is very flexible on ingest methods. Anything that's a WSRF service and that publishes its own resource properties can very easily be pulled into MDS using its mechanisms. You can also advertise any kind of data that's expressed in XML by creating your own provider."

One of the key differences between research and enterprise grid computing is the need for accountability. MDS presents an interface where this data, critical to enterprise, can be ingested and aggregated. MDS may indeed be the catalyst the Globus Toolkit needs to spark its evolution from science and academics into a key player in enterprise Grid.

Posted by Greg Nawrocki on December 22, 2005 07:29 AM

December 21, 2005 | Comments: (0)

Acxiom drawing headlines in hostile takeover scenario

TAGS: None

A regional newspaper in Arkansas cited Grid computing as a "key to future growth" for Little Rock-based Acxiom, a data processing powerhouse that's been a recent target for a $2-billion hostile takeover by ValueAct Capital partners.

Acxiom has a massive, in-house Linux Grid that they use to process more than 50 billion customer AbiliTec (their popular transactional processing application) transactions per month. Acxiom calls its Grid "Hive for Hire," and even built a home-grown management tool called "Apiary" (an apiary is a place where bees are kept, for all you non-Apiologists out there). You can read more about Acxiom's efforts in an Ian Foster op-ed on Computerworld that ran earlier this year.

Posted by Greg Nawrocki on December 21, 2005 07:57 AM

December 19, 2005 | Comments: (0)

Thanking the SETI Project for 10 years of free PR for Grid ...

TAGS: None

Did it find alien life forms? No (at least, not yet). Did it raise the mindshare of Grid computing? Absolutely.

Even with the news that the project is being folded up into BOINC ("Berkeley Open Infrastructure for Network Computing") -- SETI@Home project has been an absolute PR machine for Grid computing for the last ten years or so.

Some in the Grid community would argue that while SETI ushered Grid lexicon into the mainstream -- it also relegated the mainstream understanding of Grid to a very narrow "CPU scavenging" type of definition. While so-called "compute-Grids" are in fact still the most prevalent types of Grids to date, the Grid community sees CPU scavenging to be just a subcomponent of the overall value proposition for Grids (which encompasses much broader capabilities in data virtualization, service-oriented infrastructures, and coordinated resource sharing between 'virtual organizations').

E.T. phone home jokes aside, SETI really appealed to the imagination of the IT community, and officially put Grid on the map.

Posted by Greg Nawrocki on December 19, 2005 08:51 AM

December 15, 2005 | Comments: (0)

more thoughts on networking vendors' Grid opportunity

TAGS: None

Cisco is not the only networking vendor making forays into areas that have typically been understood as "systems management." Networking is not just about 'pipes' and 'plumbing' anymore, it's much smarter than that. While Cisco's "application-oriented networking" product line launch yesterday is a great example -- it's just the tip of the iceberg in terms of what we're going to be hearing about from the networking vendors for the foreseeable future.

As we try to understand networking vendors' role in the Grid computing evolution, in particular, we have to start thinking of the network more anatomically and less like hardware. According to Dr. Franco Travostino, Consulting Engineer and Dir. in the Adv. Tech/CTO Office at Nortel:

"It's not just a matter of creating bigger pipes and expecting that we're going to have greater bandwidth. That will be one key ingredient, but not the only one. In addition to needing to blast as much data as possible, equally important there is the aspect of control of the network, so you can gain virtualized control and access to the network and do, for example, time of the day reservations if you need to - or be able to change the security properties, or the routing properties, or the service discovery properties. It's not about brute force and how many bits per second you can send - but also about how to finesse the network experience. We want to remove all sorts of GUI-driven and operator-driven elements so that the network can be controlled by software, and there are specific feedback loops going on between network and software, without operators involved. We want an addressing schema capable to support hitless joins in a virtual organization without collisions and impact from intervening network address translation boxes or firewalls. So there are very important features above and beyond the basic capability of sending a lot of data through a pipe."

What it is really coming down to is that we are going to have to start thinking of the network more like neurons in a biological sense and less like pipes in a plumbing sense.

Posted by Greg Nawrocki on December 15, 2005 08:54 AM

December 14, 2005 | Comments: (0)

Cisco's rolling thunder in systems management

TAGS: None

2005 has been a banner year for service-oriented architectures ... and the shift from monolithic apps on silod architecture to services in dynamic compute has introduced very compelling new business opportunities for the networking vendors. For that reason, networking giant Cisco should be one of the most important players to watch in systems management in 2006.

In August, Bob Aiken -- Director of Academic Research and Technology Initiatives at Cisco -- noted that we're seeing a "blurring of the boundaries between operating systems, networks and middleware." Earlier this week, Network World's management beat journalists Denise Dubie and Phil Hochmuth broke the news that Cisco is announcing new 'application aware' management products, which would allow customers to "monitor and measure application performance on a network."

Even the novice networking enthusiast is familiar with Cisco's "intelligence in the network" mantra. Well, today's convergence of virtualization, loosely-coupled services and dynamic provisioning capabilities could accelerate the network's role from mere transport of IP packets -- to central nervous system for the IT infrastructure. Cisco's "Application-Oriented Networking" product line compliments a "Server Networking and Virtualization" group that had a very busy year in '05 -- and which is also making forays into system management.

I think it's really interesting to consider the consolidation that will likely take place in the systems management realm in the next year. Over the last couple of years, there's been an explosion of vendors with dynamic server provisioning capabilities -- Opsware, Cassatt, Platform, Data Synapse, Levanta, Egenera, Qlusters, BladeLogic, etc., etc. -- the environment is ripe for acquisitions, and Cisco is one of the looming giants that's up to the task. To me, the question is not 'if' Cisco will continue to beef up its systems management capabilities -- but 'when' they will officially acknowledge that they intend to compete head to head with the Tivolis, Openviews, etc. of the world. As highly distributed applications become normalized in enterprise -- the value of "intelligence in the network" keeps getting more compelling for enterprise customers.

Today's industry trends could be playing right into Cisco's hands.

Posted by Greg Nawrocki on December 14, 2005 07:57 AM

December 12, 2005 | Comments: (0)

Lack of consolidation in Grid standards concerns enterprise end users

TAGS: None

GridTODAY kicked off this week's industry discussion with an op-ed that highlights emerging interoperability threats in enterprise Grid
computing environments. According to author Brooklin Gore, enterprise end users lose when Grid vendors fail to converge on common standards:

"How many distributed resources managers (DRMs) does an enterprise need? I argue one. But what happens when you deploy that nifty new Grid version of SAS? Well, no problem because that SAS implementation comes with Platform's Grid software. But what if you're already running a DataSynapse grid that has a perfectly fine DRM? Unfortunately, those two DRMs don't talk very well to each other. But they do both talk Globus. So, maybe you could have SAS schedule jobs with Platform and run those jobs on DataSynapse-managed resources through a Platform-to-Globus-to-DataSynapse gateway? Now, if that is your idea of fun, go for it. Personally, I'd rather see a minimal set of standards that ALL Grid software adheres to, eliminating the need for intermediate Grid gateways. Then, an enterprise grid manager could more easily configure multiple job schedulers to leverage resources managed by a single DRM."

Gore's battle cry for interoperable vendor Grid solutions is a reminder of the nascence of enterprise Grid adoption. And the Grid development community itself in 2005 has remained predominately academics / scientific. So beyond financial and pharma (two industries which, BTW, have much broader integration and interoperability issues than just those ushered in by Grid) -- there hasn't been enough actual Grid usage for vendors to evaluate and consolidate standards in the context of real-world enterprise experiences. Even in the financial and pharma verticals, the examples beyond compute grid cycle stealing are few and far between.

Would greater enterprise end user participation in Grid standards group be welcome? Absolutely. But today, Grid standards are being predominately played out by the groups that have the most expertise -- which are the research / science community, and the vendors that have been heavily investing in R&D for how Grid will fit into their broader product strategies.

Gore makes a good point that "SMTP and HTTP emerged as de facto standards providing 99.9 percent of what folks needed to make the Internet what it is today." I'd argue that of these, HTTP was the most influential as e-mail had been around for a few decades, with very limited use, before web browsing came along. I'd also point out that, in the case of this particular standard, is that it was application driven. People really didn't pay it a whole lot of attention until there were web browser applications up and running.

The enterprise Grid end user community hasn't really shown up at the party yet, and I don't see Grid standards consolidation accelerating until there is broader Grid usage (read "applications") in production environments.

Posted by Greg Nawrocki on December 12, 2005 09:42 AM

December 09, 2005 | Comments: (0)

Slug Grid

TAGS: None

To keep myself from forgetting the few coding skills I can still remember (from my embedded systems / control systems days), and to let me think about something other than Grid for a while, I dabble in a few home automation projects. One such project had me looking for a suitable replacement for my home control Linux box that moved on to a new life as a music server.

There really isn't a lot of heavy lifting that my home monitoring box has to do. Interface to my alarm system with a serial port, run a few cron tasks from time to time, aggregate the information and build a web page. That, and it also needs to act as my file server for backups.

I started poking around and stumbled upon this. The NSLU2 is fondly referred to as “Slug” by those that have "unslung" it -- and there is a whole community that has come into existence to take advantage of the fact that the Slug comes with some pretty cool Linux firmware that can be extended on. I bought one a while back (for something like $70 as I recall!), "unslung" it, added a few packages, and away I went.

So what does this have to do with Grid ... ?

On the nslu2-linux Yahoo group that I try to stay up on, someone mentioned the possibility of a NSLU2 "Slug Cluster distributed over the Internet." Well I don't think the processing power of the Slug is going to win any "my cluster has more CPU power than yours contests," but with the amount of storage only limited by the size of the USB drives (it has ports for two) that you can plug in, there may be some real potential here in the data grid category. That, and it shows that Grid ideas are permeating to ever smaller, and cheaper targets.

Sorry, I gotta go, by the pattern flashing on the LEDs of my Slug, I think it just discovered the existence of extraterrestrial life...

Posted by Greg Nawrocki on December 9, 2005 09:07 AM

December 06, 2005 | Comments: (0)

IBM developerWorks: Grid Pushing IT Automation Boundaries

TAGS: None

IBM developerWorks editor Mary McCommon is maintaining probably the most informative knowledge base for enterprise Grid developers out there. If you're trying to track Grid computing and the most important technical issues -- do yourself a favor and plug into the RSS feed. No, I'm not on the payroll ;)

This week, IBM DW's lead story comes from Matt Haynos, program director for Grid Technology and Strategy at IBM. Haynos highlights that automation is key to Grid environments, but notes that automation is inherently at odds with many IT manager's predisposition for proactively managing and monitoring their systems:

"I hark back to my programmer days when I spent many hours with UNIX shell scripts. I'd spend weeks perfecting a script to automate some development or administration activity that promised to make my job -- and my company's -- life easier. Every time it ran, I'd check that the results were expected. I babysat the script for weeks on end until I was comfortable that it was running correctly. I was inherently distrustful until I assured myself that the script was doing what I had intended it to do. It was clear sailing from there."

So Haynos calls out this "human element" involved with getting comfortable with automation. A Grid service requires a certain quanta of resource (storage / compute / etc.) to run ... and there are a wide range of commercial tools that come into play for the provisioning and orchestration in the Grid environment. Getting comfortable with the automation of managing your Grid environment, however, is a tough proposition for many IT managers. This is an excellent read for understanding the psychology behind that uncertainty.

Posted by Greg Nawrocki on December 6, 2005 10:15 AM

December 05, 2005 | Comments: (0)

open source's new target: systems management

TAGS: None

When we look back at the early milestones of Linux's success in enterprise, one thing really sticks out to me: the value proposition was clear from the onset. A free operating system equaled tremendous cost saving potential ... that was enough to capture the imagination of enterprise developers, and the "community" snowballed from the very clear economic implications of the technology.

With Grid Computing, and Open Source Grid Computing in particular, it's still unclear to the enterprise development community exactly what the upside is, what exactly the technology is going to displace, and what the upside is to contributing code.

The Globus Toolkit is an excellent example of this. For all of the success that the Globus Toolkit has had in science and academia, (last week, the National Science Foundation announced an additional $13.3 million in funding for the Globus Toolkit), this is an open source effort that has not yet fully captured the enterprise development community from the standpoint of end user commercial deployments -- but I expect the pendulum to start to swing in 2006.

Why? Open source Grid computing and the components of the Globus Toolkit in particular are highly complimentary to other open source efforts that are changing the game with IT systems management. From open source network monitoring player Nagios, to Xen's Linux virtualization breakthroughs, to Levanta's Linux virtual filesystem MapFS code -- there are numerous projects popping up today that provide open source alternatives to proprietary systems management tools.

While the value proposition for these projects is not quite so crystal clear as Linux (displacing a very expensive OS, with a free OS) -- I believe this trickle of open source systems management initiatives underway today will clear the path for a broader enterprise developer community interest in replacing expensive systems management tools with open source. First open source targeted the OS, then the application -- now it's poised to change the game with how we manage our systems.

Grid computing is an ideal catalyst to help write this next chapter in the Open Source story.

Posted by Greg Nawrocki on December 5, 2005 11:06 AM