IT Troubleshooter | Harper Mann » June 2006

June 27, 2006 | Comments: (0)

Man-Made Security Woes

TAGS: Security

In enterprise IT, all sorts of suffering happens in the good name of security. As an end user, it's really hard to know what you need to do and don't need to do -- and often the processes / technology in place are merely illusory feel-goods.

For example, most login systems will kick a user off after three tries. But what's the difference between getting kicked out after three tries, after four tries, or after 100 tries? There's no sound statistical reasoning behind the number three, and it's a perceived security rather than a real security. But what's very real is the inconvenience for fumbley-fingered people like me that get locked out all the time after three mis-types in a row, and the help desk guy/gal's time as they reset the password and get the user back on the system.

Then you have the joy of the random-generated password. Yes, it is the strongest password that you can create. But I can't remember a randomly-generated password with tildes and other weird diacritical marks. So I've got 300 passwords on a PC, and I guess if anyone ever guesses the password to that one, I'm hosed.

Almost all of us have experienced virus attacks on email systems (either directly, or via a bunch of spam sent by an associate's email system going bonkers). But I've never heard of anyone actually breaking into a firewall that didn't already have the password to the firewall. And doesn't it seem like every time you hear about a massive customer data theft (like AIG's recent one), it's a matter of a burglar going into the physical location and stealing the actual hardware?

And for all the firewalls and intrusion detection systems that are built, there's a definite pain in managing and fine-tuning these systems.

In a recent VARBusiness article, George Hulme teed up the market opportunity for solution providers targeting security configuration. It would seem that customers (even with as few as ten or so servers and a few routers) are finding it prohibitively complex to configure and maintain their systems over time.

Hulme quotes one Gartner analyst who says that 99 percent of external hacks are exploiting system configurations, and quotes another who says: "Customers are always changing and adopting new applications ... [S]ome have thousands and thousands of services, and they just can't get their hands around how many servers they actually have deployed, or the functional relationships between servers."

Posted by Harper Mann on June 27, 2006 10:43 AM

June 19, 2006 | Comments: (0)

Are SANs really as infallible as vendors would have you believe?

TAGS: Networking

The original rational behind the enterprise SAN and fiber channel fabric market was to put in place a world-class infrastructure for protecting data. By dissolving the direct-attached relationship between servers and storage, the idea was that when servers failed, the storage would still be accessible.

But studies about downtime causes are steadily surfacing data that suggests that SANs themselves go on the fritz relatively frequently. This recent vendor-produced white paper ("Why Email Fails") by MessageOne, for example, cites SAN failures as one of the leading overall causes of messaging system failures, and suggests that compared to other causes of failures, SAN outages tend to lead to a considerable amount of down time.

So what are some of the common afflictions in SAN technology? Jon Toigo, principal of Toigo Partners recently enlightened me:

"There are a number of problems with SAN technology that are giving enterprise IT pros problems.

One is the poor set of standards upon which they are based. You could fill a whole library with fiber channel standards. They've been developed primarily by players in the industry. And the really interesting thing about their standards is that you can build a switch to the letter of the standard with absolute certainty that the switch will not work with competitor switch, which is also built to the letter of the standard. Now, you figure it out. When is a standard not a standard? When has the storage standards process been hijacked by the vendor community? I once referred to a fiber channel standards group as the Taliban of the storage industry, and I got in trouble for it. Somebody responded that the comment was over the top, but someone else noted that it was also a little unfair to the Taliban.

Secondly, SAN technology is prone to a lot of human factors in the configuration and set-up ... failures that are based on the fact that people may not understand how to create a zone or they may not understand how to properly configure their SAN. There are lots of configurations and complexities that require ongoing fine-tuning and optimization, and create a lot of dependencies. Which, of course, means lots of little things that can break.

The other common criticism of SANs is that they are extremely difficult to manage. If you have a heterogeneous environment -- meaning you've got disk arrays from multiple vendors -- it is almost impossible to manage a SAN. And there have been ongoing efforts, like SMI, at the storage networking industry association, to try to come up with the universal holy grail of management, except that it's all held hostage by the vendors. To a vendor, making management easy means that you also make it easy to deploy his competitor's product in the same storage fabric as his, and he doesn't like that idea. So the storage vendors' participation in management standards has been halting at best. For end users, it's tough to see when a SAN problem is building, and it's really tough to quickly troubleshoot and rectify the problem when it occurs. So the only way around it, to get a little more efficiency and a little more resiliency out of the SAN, is to buy all the pieces from a single vendor. So while the price of compute has dropped (on a per gigabyte basis) at the rate of 50% per year since the mid- 90's ... the price of a storage array made up of a bunch of commodity disks has actually accelerated at a rate of 125% per year."

I completely agree with Jon's points. SAN vendors have sold a "bill of goods." SAN complexity is through the roof ... possibly two orders of magnitude over conventional storage, which makes them nearly impossible to manage. Most failures are due to configuration changes, not hardware or software bugs or breakage. Add that to the extra virtual file system layer needed to manage SAN requests -- which is on the order of a whole other OS with the concomitant problems with races and resources -- and you've got even more trouble.

The big SAN vendors want it complex so they can make the case for how expensive it all is. If you look at the per disk cost for SAN approved disks, they are on the order of $3000 or $4000 each, plus the extra charges for going > 1TB and similar. As an IT director, I don't see any value in paying more just because I went over some arbitrary value like 1TB. It's also galling to pay thousands of dollars for the same damn disk that cost $150 bought raw from the manufacturer.

Posted by Harper Mann on June 19, 2006 07:39 AM

June 15, 2006 | Comments: (0)

Open Source As Much About The People As The Code

TAGS: Open Source

One of the most interesting (and relatively subjective) business discussions in IT today is around what exactly constitutes "Intellectual Property" in open source start-ups.

From the VC perspective ... before delivering a term sheet to a potential open source start-up, it's about crossing off potential risk areas at every step of the way. Can the company build a product around the technology? Is there a market for the product? How do you market it, scale it, execute on the plan? Are the multipliers on the original investment attractive enough to justify the investment risk? These are all the sorts of well-known questions that the VC's examine, pre-investment.

But the muddier waters are around the personalities and commitment of the engineers who created the code. How long do they intend to stay? What is their level of commitment? These are fuzzy types of questions - but we know from history that when the core team of engineers that best understands the code up and walks out ... it tends to send a company into a death spiral.

At that point the company must find new developers / engineers that understand the technology, and it can take months and months for them to decipher from a code perspective how the product works -- meanwhile, no one's at the wheel with respect to evolving the product to keep ahead of the competition. You basically have complete technological stasis when key engineers leave, and often there's no recovering.

"The code without the people is worth nothing," according to Phillipe Cases, partner at VC firm Partech International. "A million lines of code is like a million problems that you have to solve. So the risk on any open source investment project is that the 2-3 guys that created it and maintain it could leave. The commitment of the developers is often the IP -- not the code itself."

And this doesn't just apply to tech start-ups and their VC's ... it applies to big companies that acquire open source start-ups. If you acquire the company, only to have the core engineers split after a short period of time, what's your return going to be on that acquisition? Just ask Computer Associates, who shortly after acquiring Ingres, experienced a 'mass exodus' of developers immediately thereafter, and from there certainly struggled to pick up the pieces and profit from the acquisition.

Posted by Harper Mann on June 15, 2006 07:38 AM

June 05, 2006 | Comments: (0)

Open Source Network Management Tool You Should Care About: NeDi

TAGS: Networking

Documenting the network and keeping tabs on the equipment in your environment is the equivalent of doing the dishes, taking out the trash -- i.e., the everyday household chore minutia that gets pretty tiresome over the long haul. It's even worse when network hardware fries and you don't have your configuration backed up. What's needed is an automated way to keep your network inventory and collect the configurations so recover from problems can be just a few clicks away.

In recent blog entries, I've discussed some compelling open source tools for network monitoring and management -- and without question, one of my favorites is one called NeDi ("Network Discovery Suite"). NeDi was created by Remo Rickli, and pulls SNMP values, using the Cisco Discovery Protocol -- and makes it really easy to visualize the environment and present it on the front-end (with PHP).

I've been working with this tool in customer environments for a number of years, and it's as solid at retrieving and displaying info about what's on the network as any commercial product out there. It's also faster than most. It runs at night and picks up any network node configuration changes and stores them.

About a year ago, Paul Venezia did a nice curtain raiser article that explained what makes NeDi so good. When Venezia wrote this article, the 1.0 version of Nedi hadn't come out, so I'd just add that it has now added new features:

* Improved the calendar usage in the frontend. Date and Time can be selected in Devices- and Nodes-List now.
* Monitoring overhauled
* Devices-Table is grouped by rooms (if available).
* New Vlan Information in Devices-Status
* Changed some icons to be more intuitive

Posted by Harper Mann on June 5, 2006 04:47 PM