Grid Meter | August 2005

MORE ENTRIES

August 2005

August 31, 2005 | Comments: (0)

When children are younger they tend to engage in parallel play. Watching and often mimicking each other but not necessarily playing with or interacting with others. Sometime around their second birthday they start to develop relationships and friendships with others. They recognize other children familiar to them and actually play together. Having fun, solving puzzles, and yes, sometimes engaging in conflicts that only seem to last a few seconds.

I can't help but wonder if the Grid Computing revolution is an anthropomorphic equivalent of the social evolution of a two year old.

I remember when I first brought my computer home, it seemed so small and helpless. It needed me for everything. As it learned more applications it would often do similar things as the computers around it. Repeating or doing similar tasks in a similar way, communicating from time to time but not really working with the other computers around it.

Now that it is on the Grid I find that it is learning things without my guiding hand. It is capable of so much more. It is developing complex relationships with other computers and often works with others to accomplish tasks. Sure there are conflicts from time to time, but they resolve themselves quickly and are soon forgotten.

As I look at my computer I cant help but wonder what the future will bring. Who will its friends be? Where will they meet? Will they be friends for a long time? What will my computer want to be when it grows up? Will I be able to survive the teenage years?

*Sigh* computers these days, they grow up so fast...

Posted by Greg Nawrocki on August 31, 2005 07:42 AM

August 30, 2005 | Comments: (0)

The Chicken or the Egg?

One of the hairiest challenges in modern computing has been finding ways for organizations to take better advantage of the explosion of data over the last twenty years. With the proliferation of eCommerce, CRM, devices at the "edge" -- the sheer amount of information that's being generated has seen a drastic increase, but it has also created new challenges in the movement, assimilation, and analysis of that data, specifically in the speeds in which these tasks must be done.

When we look back on the last decade of Grid computing (from the mid- 90's through today), we often see them pigeonholed in the taxonomies of data and compute Grids. However, modern Grid computing has a less distinct division between the two.

The Google searches that we're all so dependent upon to quickly return our queries are powered by a massive Grid. Financial services shops use Grid to run complex Monte Carlo simulations that help them predict capital market risks. These simulations are based on increasingly complex models, relying on an increasing number of data sets ... the list goes on and on.

While many organizations started leveraging Grid to achieve better control over larger data sets, they are now tapping into Grid to catch up to the evolution of the actual devices that are producing the data. Take the LEAD (Linked Environments for Atmospheric Discovery) project, for example. Mesoscale meteorologists are seeing huge breakthroughs in the Doppler radar systems at their disposal. The next generation of Doppler radar is pushing the envelope in terms of sensitivity. The volume of data they are retrieving and the granularity of that data is continually on the rise. Thus, the sheer amount of data that meteorologists must crunch / analyze on the back end is getting greater every day. In addition, this next-gen Doppler technology have both increased configurability and the ability to react to new weather patterns on the fly. Hence the need to analyze data quickly and simulate possible outcomes, with this data fed back to the instrumentation itself so it can make active adjustments is also a concern. The LEAD project is developing a new Grid cyber infrastructure capable of moving that data to compute resources, scheduling data mining jobs -- and enabling faster, more dynamic interpretation of real-time weather patterns.

Another area where devices are generating an exponential growth curve in terms of generated data is 3D vision. "Stereo Vision" uses cameras to gather environmental data -- and on the back-end, various data mining techniques are used to help machines "see" the real world in a similar fashion to how humans see the world. They can sense depth and color, they can differentiate one object from the next -- and it's a huge growth area in terms of autonomic systems.

In stereo vision, there's actually a new law (called "Woodfill's Law") -- which says that "the computational complexity of computing a 3D image is cubed relative to side of the image's edge." As the camera sensor technology that's used to produce these images becomes more complex (via processing power, Moore's Law trends) and image resolution increases -- the amount of data produced which must be analyzed to direct the next events of these autonomous systems is also growing at an exponential rate. One can see how this balance of accommodating for larger data sets and the need to process that data faster will keep feeding back upon itself.

All that's left is to coin a clever term for this "chicken and the egg" relationship.

Posted by Greg Nawrocki on August 30, 2005 12:36 PM

August 29, 2005 | Comments: (0)

Systems Management and Network Management Convergence

You just can't open an IT trade rag today without seeing a virtualization story. It's gotten to the point where I had to disable my Google news alert for "virtualization" -- because it was yielding too many darn results and distracting me from my full time job. It's the buzzword du jour that's crept into so many companies' boilerplates and press releases, that it's sometimes tough to tell where the real action is, versus who's just trying to get caught in the current of the industry momentum around virtualization.

One of the trends that I've been keeping an eye on as it relates to Grid is the amount of acquisitions and activity on the storage vendor side, targeting the virtualization of network-attached storage. It was big news last week that EMC is going to acquire Rainfinity -- and there's a good read today on Computerworld about clustered storage approaches to addressing throughput / scalability requirements.

A few months ago, Bob Aiken, Director of Academic Research and Technology Initiatives at Cisco Systems, gave me something to think about in Grid's role in the convergence of systems management and network management. He said, "There are two trends that are happening. One is that there's going to see a lot more intelligence in the network -- which has been going on already for a long time, but will continue to accelerate. Applications are going to have to become more 'network aware,' and the network is going to have to become more 'application aware' -- and as a result there will be a blurring of boundaries between operating systems, networks and middleware."

In the enterprise's early Grid discussions, most of the attention has been focused on the systems management implications of Grid -- how it will affect provisioning, job scheduling, etc. But clearly, the networking side of the discussion is going to be very interesting to keep an eye on. How will networking policy, switching and routing directions drive Grid's evolution? How will fire-walling and security issues be accommodated? Can the public Internet adequately carry Grid traffic, will organizations need to build dedicated networks for IntraGrids, or will "smart networking" devices be created to more effectively route Grid traffic? To what extent are today's bandwidth, latency and I/O constraints being addressed for Grid environments? There are numerous, network-specific Grid efforts that have been taking place for years -- but these topics are just now starting to creep into the media dialogue.

Sun's famous prediction "The Network is the Computer" is looking more spot-on every day -- as enterprise continues to move from silo'd compute stacks to distributed resources that are networked to behave as a more cohesive system. It would appear that Cisco's "Intelligence in the Network" mantra will continue to manifest itself in Grid discussions -- as virtualization concerns continue to move further down the stack.

I predict that the next year of Grid discussions will feature an increased involvement of companies in the traditional networking hardware markets and a new importance in issues surrounding the marriage of network and systems management. It will be fun to watch it unfold.

Posted by Greg Nawrocki on August 29, 2005 09:28 AM

August 26, 2005 | Comments: (0)

Mainframes, Dinosaurs and IT Evolution Theories

Hiawatha Bray (Boston Globe) put out an interesting story today on the survival / resurgence of the mainframe in IT. The story includes various perspectives from the mainframe audience gathered at the SHARE event this week in Boston.

Reading this story reminded me about a discussion that we're increasingly seeing in this new wave of Grid, virtualization, and commodity computing. Industry pundits are increasingly pointing out a "back to the future" trend in IT -- where instead of mainframes, we're building "scale-out" architectures that employ the same virtualization and symmetric multiprocessing techniques that mainframes use, albeit at a lower cost, and with finer-grained management capabilities.

Andreas Antonopoulos from Nemertes Research describes the IT architecture progression in terms of a pendulum "swinging every few decades from centralized to-decentralized and back again." In a recent column, he suggests that the mainframe isn't coming back -- next generation IT architectures are simply borrowing the most attractive qualities of mainframes. It's a great read for additional clarity on the "centralized vs. decentralized" and "mainframe vs. scale-out" discussions.

Regardless of where you think this is all headed, one can't help but note the explosion of management vendors over the last few years, employing new virtualization techniques to tackle the new "scale-out" environment. BladeLogic, Cassatt, DataSynapse, Egenera, Levanta, Opsware, Platform, VMWare ... the list goes on and on.

Posted by Greg Nawrocki on August 26, 2005 09:25 AM

August 25, 2005 | Comments: (0)

More on the SOA / Grid Connection

I just finished reading an interesting IBM developerWORKS column titled "Build applications based on SOA." The author draws various comparisons between Grid and SOA -- highlighting the convergence of the two technology movements.

It's a good sign that folks are beginning to speak about the two trends in the same breath. Up until this point, SOA has often been discussed in a vacuum -- strictly in the context of new application development trends. But when we're talking about this new way of developing applications -- shifting from monolithic applications sitting on dedicated hardware to "services" being shared / re-used across a broader environment -- that begs the question of how the underlying physical resources will support the model. And as Ian Foster pointed out earlier in the year, to be truly successful, SOA environments are going to require a much better command of the underlying physical resources than the typical IT shop has today:

"SOA can use common Web services protocols (XML, SOAP, WSDL and UDDI) to distribute processes, but it in and of itself doesn't describe the distribution and management of the resources. This is where the grid computing and SOA movements converge. Grid enables an effective collection of distributed resources. Grid defines the virtualization of data and resources as well as mechanisms for resiliency, including monitoring, resource and data discovery, and security. In order to execute a computing function, you need a process and a resource. Until the enterprise gets better command of its resources, the promised land of "services when and where you need them" will remain just an idea of where applications are heading. The unique combined capabilities of grid technology, including management of resources and information services -- all built on a foundation of security -- create the framework within which SOAs can succeed."

Here's a link to another IBM developerWORKS column that looks at Grid's legacy with Web services, this time in the context of Grid security directions. As Grids and SOAs continue to mature -- we'll also hear a lot of discussion about their compatibility in terms of security requirements / frameworks.

Posted by Greg Nawrocki on August 25, 2005 10:16 AM

August 24, 2005 | Comments: (0)

Grid Mobility -- Can You Hear Me Now?

Yesterday I had the opportunity to catch up with Raj Kumar and Xiang Song from HP's research labs. They're working on an effort called "GridLite" -- a proof of concept Grid project that's exploring the boundaries of what it means for "lite" mobile devices to participate in a Grid. Mobile devices have legitimate resource constraints (limited storage, memory, compute cycles and so forth) that preclude them from participating in Grids, so it's a tricky problem these guys are trying to solve.

"But if businesses can access the Grid from a laptop or desktop -- they should also be able to access the Grid from a PDA, smartphone or other handheld device," said Kumar.

What Kumar's team has figured out over the last year-and-a-half is a new set of services (sitting above core services) that enable mobile devices to tap into the Grid infrastructure.

In the GridLite architecture, the clients (which include any ubiquitous device like iPaqs or cell phones) contact the server through a portal called GridLite Resource Manager (GRM). If the mobile user is working on an application, the data is brought to the user device, with a master copy kept in the Grid infrastructure. If the user needs to free up some storage on the device, they can flush some extra files to the Grid infrastructure in a seamless fashion. There is a policy engine that allows for thresholds where data moves back and forth from the Grid to the handheld devices, all based on the user's prompting. Predictive algorithms anticipate use of applications (based on recurring usage patterns) and potentially download applications on devices before the user request even occurs.

These are just a few of the really interesting new ways that the GridLite folks are Grid-enabling mobile devices. A big part of what they're doing is essentially providing an abstraction mechanism that gives the illusion of unlimited storage on the client (mobile device) side. One of the ongoing discussions / debates in Grid -- of course -- is where the intelligence should reside and where the compute should take place. When you're talking about "lite" clients (with the resource constraints mentioned above), the ultimate goal is to make data and storage resources appear local to the user, and to remove any latency or performance issues.

I think these guys are onto something really cool ... I'm very impressed by the progress they've made. We also had some preliminary discussions about some work that they're doing with the open source Globus Toolkit on Grid-enabled mobile devices. I hope to circle back and discuss that further within the next few weeks -- and also to touch on the authentication / authorization issues that the GridLite team is working on with Grid-enabled mobile devices.

In the meantime, if you want to learn more about the GridLite project, here's a link to a research paper you might find helpful.

Posted by Greg Nawrocki on August 24, 2005 09:22 AM

August 23, 2005 | Comments: (0)

Cool Grid Links Worth Checking Out

A few recommended resources for enterprise IT folks interested in Grid ...

The University of Edinburgh in the UK has a "NextGRID" initiative that's really focused on commercial application of Grid computing technologies. They have a series of videos that explain how various industries are using (or plan to use) Grid. This link launches a movie that explains how Audi sees benefit in leveraging Grid to share designs and simulations with end users and sub-contractors. Click here to view other links to movies about Grid use in pharma, financial services and predictive modeling applications.

Another Grid organization that's doing some very cool stuff is the Belfast e-Science Centre in the UK. They're using Grid (and open source, like the Globus Toolkit) to help organizations like the British Broadcasting Corporation (BBC) manage large file transfer. Follow this link to check out some of the interesting Grid projects they have cooking (the GridCast link goes to more info about the BBC effort).

Ever wonder about the networking requirements for Grid computing? Nortel Networks' advanced labs have a project called the 'Dynamic Grid Allocation Controller' (or DRAC), which looks at the next generation of orchestrating compute + data + network resources. Click here to check out an overview on the project. Similarly, Cisco has some very interesting technical overviews of the networking requirements and / or directions for Grid. This 'Network Service Interfaces to Grid' paper shows how Grid networks might be built over different types of networks, including L3/2/1 networks, MPLS networks, optical transport networks and more.

Posted by Greg Nawrocki on August 23, 2005 07:42 AM

August 22, 2005 | Comments: (0)

VC's Back Univa for Enterprise Grid Push

This morning, Grid start-up Univa announced that it has secured $8-million in funding (from ARCH Ventures, Appian Ventures, New World Ventures and OCA Ventures). Univa CEO, Steve Tuecke, was one of the original pioneers of the open source Globus Toolkit effort (along with Ian Foster out of Argonne National Lab, and Carl Kesselman out of ISI).

Univa is addressing the services and support requirements for enterprises building open source, open standards-based Grids. Red Hat proved that services and support for enterprise open source can be a profitable business -- and Univa hopes to replicate their success for the emerging enterprise Grid market. According to Univa COO, Rich Miller:

"Professional services will be very important to the uptake of Grid infrastructure. And, it's pretty easy to understand why. Grid infrastructure isn't something that comes ready-made. It's not a 'turnkey solution,' since almost every enterprise has a very unique collection of resources, a unique combination of applications that need to run on the infrastructure, and very different levels of performance or service levels that are acceptable.
Thus, for the foreseeable future, most implementations of Grid infrastructure will be consultative sales, and require the expertise of professional services to design and then implement the Grid infrastructure. Having once put Grid infrastructure in place, any enterprise which depends on that infrastructure will also demand that there be a high-quality level of technical support. No one wants to 'bet the company' on an infrastructure for which you can't find a 'mechanic.'

Why would someone tap Univa, a startup, for Globus-oriented services and support? Well, the founders of Univa are truly the inventors and the world experts in Globus. Working for our company are perhaps the most skilled people on the planet when it comes to understanding what Globus can and cannot do. We envision Univa's professional services to be sought out by leading enterprise users of Globus, particularly for the assessment, design, customization and integration of Globus components. We also expect to partner with the major systems vendors and professional services companies in executing and supporting the infrastructure for major projects."

Earlier this year, Univa announced its first three projects ... and later announced its first customer, Raytheon.

Posted by Greg Nawrocki on August 22, 2005 07:58 AM

August 19, 2005 | Comments: (0)

Grid Credit Where Credit is Due

There's been a ton of discussion this week about the whopping $150-million in funding that the TeraGrid effort secured. So where's all that money going? Accoring to TeraGrid director, Charlie Catlett ...

"The $150 million just secured is for a five year period and it is primarily to operate, manage, and evolve the TeraGrid system so the funding is primarily for staffing. Two thirds of the funding will go toward operation and management of the resources at the eight resource provider sites, along with user support and related services provided by those partners. The remaining $50 million [over five years] is for software integration and engaging the user community through programs such as the Science Gateways initiative and a coordinated support program called Advanced Support for TeraGrid Applications, or ASTA.
The Science Gateways program involves partnerships with projects that are putting infrastructure in for entire communities, where we are working on a set of rapid prototypes aimed at developing a capability that will allow any number of such 'science gateways' to integrate with TeraGrid. In some cases, that would be a portal project such as the Nanohub at Purdue University. This is a portal that is offering tools and data and applications to a community of people interested in microelectronics and nanotechnology. That community includes about 40 or 50 university courses at six or seven universities. We're putting the TeraGrid behind the nanoHUB as a provider of computing and storage and other resources. And by doing that, we get to work with the nanoHUB group, and provide services to an entire community.

Another example of a Science Gateway involves an NIH project at the University of Chicago, called the National Microbial Pathogen Data Resource Center. They're providing desktop applications to several hundred pathogen researchers, funded by NIH. And while it's not a portal, we're embedding within those applications the capability to reach out and use TeraGrid resources from the desktop. So that's another kind of a science gateway."

I think it's been great for enterprise IT pros to hear about TeraGrid, because so much of the groundwork for Grid has in fact originated in e-Science / academia -- and enterprise IT hasn't had a whole lot of visibility into the early history of Grid. Muddying the waters even further, as Ian Foster recently discussed, special interest Grid groups have been extremely divisive recently, suggesting that e-Science Grid pros don't understand the rigors of enterprise environments.

When you look back to the early days of the Internet (ARPANET --> USENET -->extensive DoE, DoD contributions --> TCP/IP standards), much of the critical infrastructure and design was accomplished in government and academia before it was ultimately picked by enterprise and commercialized. The history of Grid is very similar. e-Science and academic Grid pros have been hard at work for years on the various technical challenges that Grid and distributed computing are introducing -- and their progress and findings are certainly making their way into the commercial-level Grid offerings being rolled out by enterprise Vendors.

So lets give credit where credit is due. There are great Grid things happening in the academic and research world. Scientific advances, new discoveries and even more jobs being created.

It seems to me that in order to execute on a project of this scale one must absolutely understand the rigors of enterprise environments. Perhaps it is the enterprise community that doesn't understand the resolve and determination of those in research.

Posted by Greg Nawrocki on August 19, 2005 02:31 PM

August 19, 2005 | Comments: (0)

Meet the Meter

Greetings InfoWorld readers. Thanks for stopping by Grid Meter.

So why the heck would InfoWorld (such a great source for ENTERPRISE pros) let a former propellerhead from the Department of Energy do a blog about enterprise-level Grid computing, you might ask? Allow me to offer just the cursory intro, so you know where I'm coming from and what value I'm hoping to bring to you with this new blog.

Just over two years ago, I left the consumer products embedded systems world and started working with Ian Foster at Argonne National Lab in Illinois. Ian, of course, is one of the original pioneers of Grid -- and played a key role in the creation of the most widely used open source Grid computing standard, the Globus Toolkit. Ian and his colleagues first kicked off the Globus Toolkit project about ten years ago, with the intention of enabling large-scale compute resource sharing among e-Scientists (for things like advanced particle phsyics, earthquake simulations, etc.).

In the recent past, the Globus Toolkit has branched out from just e-Science, and is now also used in commercial settings -- like in financial, where many large institutions use it as the standard Grid middleware for doing Monte Carlo simulations and the like. Recently, a set of enterprise vendors (IBM, HP, Intel, Sun, Nortel Networks, Univa) kicked off the Globus Consortium, to continue to support the evolution of the Globus Toolkit open source in enterprise settings (I'm the President of the Globus Consortium). We believe that open source is the key to mainstream enterprise Grid adoption -- and that open standards in Grid will be every bit as important as TCP/IP was to the commercial Internet.

So now that there's been some fair disclosure about who I am (and my axe to Grind) -- I'm looking forward to writing about the most compelling open source Grid projects, personalities, issues and debates out there in enterprise today. I believe that one of the reasons why enterprise IT pros are having such a tough time making sense of Grid computing is the degree of vendor hype around the technology. The goal of this blog is to get beyond that vendor hype and address the real issues and barriers to adoption out there.

Email me anytime.

Posted by Greg Nawrocki on August 19, 2005 02:20 PM

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert