May 30, 2007 | Comments: (0)
3Ware's 9650SE and the Sun Ultra 40 M2
For the past few months, I've been running a Sun Ultra 40 M2 coupled with a 3Ware 9650SE SATA RAID controller. It would seem that this is a marriage made in heaven.
As I've remarked before, the Ultra 40 M2 is simply the most powerful workstation available from a mainstream vendor today. Armed with two AMD Opteron 2218 dual-core CPUs, up to 16GB RAM, eight hot-swap SAS or SATA drive bays, two PCI-X slots, built-in 7.1 sound, S/PDIF optical input and output, a dual-layer DVD burner, and (in my case) an nVidia Quadro 5500 graphics card, this system is the creme de la creme of the workstation world. The only downside is the relatively anemic nVidia SATA RAID controller built into the mainboard. The performance of this controller isn't terrible, but the Linux driver support simply isn't there. Enter the 3Ware 9650SE.
The 3Ware 9650SE-8LPML I have running in this system is a full-on 8-port SATA RAID controller with 256MB RAM and an optional battery-backup unit. There are two four-port SATA multilane connectors on the card, which can be used to marry the 9650SE to a multilane-capable disk array, or to individual SATA drives with multi-lane to discrete cabling. In the case of the Ultra 40 M2, however, multilane to SAS cabling is needed. Fortunately, the built-in nVidia controller uses multilane connectors to feed the disk backplanes within the Ultra 40 chassis, but the included cables aren't long enough to reach the 9650SE. Sun can supply cables of appropriate length to reach the card, however.
Once I had the right cables, it was simply a matter of cable routing to each backplane connector and then back into the 9650SE. The fan tray that sits to the left of the disk bays can get in the way here, but some creative cable management within the case made everything fit and look like it was meant to be there. I placed eight 250GB SATA drives in the disk cages, and powered the system on. The 9650SE posted, found all the drives, and all was well.
I configured the eight drives into a RAID50 set, giving me high throughput on 1.36TB of usable space while providing significant fault-tolerance. The configuration through the 3Ware BIOS tools is quick and easy. Unfortunately, installing and running Fedora Core 6 (or any reasonably recent distro) on the 3Ware 9650SE isn't as simple. The 9650SE and the more recent cards from 3Ware aren't supported in the included 3w-9xxx driver found in stock 2.6 kernels. Historically, 3Ware has been extremely good at providing support for Linux and FreeBSD, so I would think that this problem will be rectified shortly, but in the interim, there are a few steps involved in getting everything working right on Fedora and RedHat. The first is to download the right install disk from 3Ware. You can find the files for just about every major distro on their site. These are just zipfiles with driver sets. Format a floppy with mformat (mformat a:), and unzip the installdisk file to the floppy. Then, boot the system as you would for a normal installation. At the boot: line, enter linux dd and the installer will prompt for a boot disk. Select the floppy drive, and it should load the appropriate driver. Continue the installation normally. On the Ultra 40 M2, I had to use a USB floppy drive, which appears as /dev/sda.
Following the initial boot, the system needs to be updated. Be aware that updating the kernel may result in a non-bootable system since the new kernel will not have the right driver for the 3650SE. Fortunately, it's easy to remedy this problem. Run the yum update to pull in all the new packages, including a new kernel and kernel-devel package. Then, download the upstream driver for the 2.6.19+ kernels from 3Ware's download site. Extract the driver source into a new directory, such as /usr/local/src/3ware, (mkdir -p /usr/local/src/3w-9xxx; cd /usr/local/src/3w-9xxx; tar zxf /path/to/source.tgz; tar zxf ./3w-9xxx.tgz) move into the driver directory, and edit the Makefile to pull in the right kernel path. In my case, the SRC:= line at the top of the Makefile should be modified to SRC := /lib/modules/2.6.20-1.2948.fc6/source/. This will tell the compiler to build the driver with the source tree of the new kernel, not the running kernel. Then, simply run /lib/modules/2.6.20-1.2948.fc6/updates and you should be all set.
Once this was done, I rsync'd 190GB to the fresh install (yes, my home directory is 190GB), and saw write throughput to the RAID50 set at around 100MB/s. Reads were slightly higher than that at 110MB/s. I've been beating up the 9650SE and the Ultra 40 M2 with my normal brand of workstation torture -- cyclic MD5 sums on multi-gigabit files, kernel recompilations, DVD ripping, MP3 encoding, and two virtual systems running under VMware Workstation 6, all while playing movies from NFS shares and running Beryl with all the widgets enabled. Between the stellar performance of the 9650SE and the calm and steady power of the Ultra 40 M2, all of these tasks were handled with aplomb. Suffice it to say, you'd be hard-pressed to equal or surpass the performance of this box with any computing hardware available today.
As far as longevity and survivability goes, the 9650SE has been running for a few months without a problem, and my several-year-old 9500 8-port SATA RAID controller has been driving a four-disk RAID5 set without a hiccup. If history is any indicator, reliability isn't an issue with 3Ware cards. I'll be posting more on this power duo as time and events warrant, but for right now, I'm a very happy guy.
Posted by Paul Venezia on May 30, 2007 10:17 AM
May 10, 2007 | Comments: (0)
Yesterday, I posted about six things that need to change. One of them was entitled "Broadband Bandits", where I basically denounced broadband companies' artificially limited bandwidth options. After re-reading it, I think I need to clarify a few things.
Certainly, these companies aren't in this business for wholly altruistic purposes -- they're in it to make money. That's the whole idea. The problem that I have with most broadband offerings is that they're specifically designed to limit end-user options without any reasonable alternative. Most areas with broadband access have one or two options, and they're generally both playing this game.
One of the major issues is the ridiculously limited upstream bandwidth provided in most residential packages. For $50 a month, I would expect to get better than 39KB/s uploading images to Flickr, videos to YouTube, pictures to my eBay auctions, and when sending email attachments. Unfortunately this is rarely the case, since upstream bandwidth has been squeezed as low as possible.
Even RoadRunner, a company that does not generally limit users' bandwidth, nor block well-known ports, delivers 5Mb/384k service with their standard package. I tested a freshly-installed RoadRunner line the other day, and found that it's just barely possible to get 5Mb down, with the 384k upstream completely maxed out with TCP ACKs. Other companies do the same thing, offering a 15:1 up/down ratio service that can just barely reach those levels, hampered by the upstream caps.
The DOCSIS cable standard isn't synchronous. Current DOCSIS installations based on the 2.0 standard are capable of delivering 38Mbit/s downstream and 27Mb/s upstream to a group of modems. A small neighborhood would have this bandwidth split between any number of modems, and using the law of averages, most users will get their rated download speeds. But notice that the 2.0 standard's down/up ratio is roughly 5:3. This doesn't coincide with the 15:1 ratio found in most broadband plans. Some offerings in the US and Canada are nearly 20:1. This doesn't jive with the capabilities of DOCSIS, so there's no technical reason why these plans exist. Upstream data is subjected to higher noise levels across a cable plant, but that doesn't justify the low caps found nearly everywhere.
The new DOCSIS 3.0 standard is very new and hasn't been widely adopted yet, but was designed to give FIOS a run for its money, offering 160Mb/s downstream and 120Mbit/s upstream to the same number of modems. Again, we see a similar down/up ratio in play.
I've seen many commercials for broadband service showing a fellow sitting in his kitchen with a laptop, telling his wife he can't go to the mall because he has to finish some work. Suddenly, we see a screenshot showing a "Done" dialog box, and voila, due to the power of XYZ's broadband service, the lucky fellow can go to the mall and relax on the hard wooden benches outside Bed Bath and Beyond. The problem here is that the ad specifically targets those people that can telecommute, without mentioning that if he was uploading a PowerPoint presentation, he'd be sitting there for a long, long time... assuming that the provider hasn't blocked IPSec and he can actually connect to the corporate network in the first place.
Consumer broadband needs to change. It needs to provide at least a 5:3 down/up ratio as part of the standard package for a reasonable price. I know dozens of broadband users that would gladly trade a few Mb downstream for a few Mb upstream, and this trend is only going to grow. Fears of illicit filesharing and copyright infringement be damned -- you can't penalize a captive audience for something they might do.
Posted by Paul Venezia on May 10, 2007 11:42 AM
May 09, 2007 | Comments: (0)
Six things that need to change
Although I'm generally able to see both sides of an argument, there exists a short list of issues that I just can't comprehend. These are those issues.
1) The RIAA's war on its customers
This one has been going on so long as to almost be accepted. Of course, that's their plan. The vast amount of money being poured into lawyers, lobbyists, and scare tactics by the RIAA would have been more than enough to rework their long-deceased business model into something for the next generation. For an industry that was built upon pushing the envelope, they certainly can't seem to think outside the CD case. The heavy lobbying in Florida that has resulted in the used CD market there receiving stricter controls than the gun market is just one tiny example.
The RIAA is certainly under attack from every angle -- piracy, slowing CD sales, a massive increase in self-produced music, and flagging interest in marquee acts -- but nearly all of that is their own fault. Instead of embracing the new market, they've been trying to kill it by shipping CDs with rootkits masquerading as DRM schemes, producing lawsuits by the bushel, apparently destroying Internet radio, and projecting an overall public persona that falls somewhere between Al Capone and Stalin. It's just ludicrous.
But then, this is the industry best described in a misquote to Hunter S Thompson: "The music business is a cruel and shallow money trench, a long plastic hallway where thieves and pimps run free, and good men die like dogs. There's also a negative side." His original words were actually describing TV broadcasting, but the sentiment prevails.
2) Broadband Bandits (Update: More on this topic can be found here)
Comcast is the easy target on this one, but there are many perpetrators of this travesty. You know who you are. More importantly, your customers know who you are, and will jump ship in an instant if given the chance. With most of the competition buried in the backyard, and a weakened FCC sitting idly by, Comcast, Verizon, and many other providers are ramping up prices and dropping service levels. They're also applying voodoo AUP interpretations to cut off paying customers that go over some amorphous limit. Many of these companies come from a delivery-only background, where they deliver the signal, and the customer passively accepts it, such as cable TV. Back in the day, this was largely true of the Internet -- Web servers existed in datacenters, ISPs, and universities, and most content was text and the occasional picture. With Flickr, YouTube, MySpace, and the advent of simple videoconferencing, end users are much more apt to be sending nearly as much as they receive, yet most broadband connections are still ridiculously asynchronous. I just ordered Verizon DSL to provide a backup circuit. $30/mo for a 3Mb/768k circuit. This means that uploading a few 5 megapixel photos will take me roughly 3 minutes, and completely obliterate that 3Mb/s download rate due to upstream congestion, even though I'm not downloading anything.
There are a few reasons that most of these wildly unbalanced plans exist. Contracts with peering partners generally dictate up/down ratios to be maintained (eg, saving the ISP money). They also prevent customers from using videoconferencing and VoIP technologies to their full potential, resulting in poor performance. This forces the customer to only use approved methods of communication (eg, paying the carrier more per month). And lastly, they've always been that way, right?
As a sideline to all this nonsense, many carriers go so far as to block well known ports, such as Web, IPSec, and SMTP ports to residential lines. True, most people aren't running Web servers from their house, but lots of them are just trying to connect to the corporate VPN. To do that, you need a business-level contract for way more money per month and usually lower bandwidth. What a bargain.
Certainly, not all carriers act this way. Comcast and Verizon DSL are famous for it, but Time Warner's RoadRunner seems to be above this chicanery, at least so far. If AT&T wasn't dismantled nearly 25 years ago, we'd still be renting our phones from Ma Bell for $20 a month, and our telecommunications infrastructure would be the best the third-world had to offer. At least Verizon is offering FIOS in some areas, yet I know of entire communities that have no broadband whatsoever. Wasn't there a Universal Service initiative started over a decade ago? Note as you read that page, you see "The Federal-State Joint Board on Universal Service recommended that the Federal Communications Commission take immediate action to rein in explosive growth in high-cost universal service support disbursements. The Joint Board is also seeking comment on proposals for long-term, comprehensive reform of the high-cost program. 5/1/07." This is because we've gotten nothing for a whole lot of something.
Just ask a South Korean how much they spend on the 100Mbit Internet circuit in their house. CNet was talking about 20Mbit links, universal video-on-demand on the cheap back in 2004. Not much has changed in three years, except their average bandwidth has increased fivefold. Heck, just ask them about the Internet service to their mobile phones -- it beats anything in the US by far. This brings me to number three.
3) The US is a mobile communications wasteland
Crazy, indecipherable "plans", "anytime minutes", $0.10 per text message, $0.003 per KB (read that any way you want), and current phones that were cutting edge in Europe when John Paul II was still wandering around the Vatican. That's the state of mobile connectivity in the US today. I've heard more than a few foreigners describe a trip to a T-Mobile store as "like visiting a cellphone museum". Given what they're used to in Europe and Asia, I have little doubt this is true. Wireless carriers in the US have been raking in money hand over fist for the past five years, riding the cellphone boom as high as it will go. During all this, they've been slowly doling out features to their users like cake to the starving, while the rest of the world runs circles around us.
The pending release of Apple's iPhone may spark something here, just as the iPod blew the portable MP3 player market apart. Hey, has Steve Jobs ever made a mistake?
4) Airport Wifi
This one's personal. I understand that fleecing business travelers for $10 or so during a flight delay is part of the business model, but even crack dealers give away the first few tastes. Can't we get 30 minutes free, and a reasonable hourly rate thereafter? I can't believe that any airport Wifi installation hasn't already paid for itself a hundred times over. I'll continue to hold Manchester Airport up as a shining example -- wide coverage, free service, no splash page. It's just beautiful.
5) Spam and the Windows Protection Racket
This one will never disappear, but it can be marginalized. If thousands and thousands of compromised Windows systems were to be patched, replaced, or burned in effigy, the volume of spam worldwide would be drastically reduced. Couple this to viruses, adware, malware, and so on, and there's very little that your PC can't do -- your taxes, spreadsheets, Web surfing, and spamming the bejeezus out of thousands of people. I think we may be near the top of a Bell curve on that one. Vista is more secure than XP (which isn't saying much) but the sheer numbers of wide-open Windows systems on the Internet will necessarily begin to decline due to hardware failure, if nothing else. If the replacements are tougher to compromise, then the spam levels will abate somewhat, as will other nefarious afflictions of the digital age, and we'll all be a little safer and saner.
Of course, if Windows were suddenly secure, it would directly affect the revenue of hundreds of smaller software vendors hawking Windows protection applications, but I can't feel too bad for Symantec or McAfee -- they'll survive.
6) Oops! I lost your ID. My bad.
Every week or so, we hear about the theft of another million identities from a laptop or network intrusion. Sometimes it's a corporation, sometimes a university, or sometimes the federal government. Sometimes it's your ID, sometimes it's mine. Pretty soon, it'll be nearly everyone that's ever had a credit card, applied for a loan, opened a bank account, or was simply assigned a social security number.
There are no formal penalties for this invasive personal intrusion, and some companies simply don't tell anyone that the event occurred. If a company doesn't have adequate security and lets a few hundred thousand database records flap in the wind, the victim will at best spend days straightening out a credit mess and changing all their accounts to new numbers. At worst, they'll lose money, their credit rating, and maybe even their job through no fault of their own. If a department store chains' physical security was so lax as to have their customers violently mugged en masse simply for being in one of their stores, you can bet they wouldn't be in business any more. What would be worse would be the poor people that got mugged because they were in a different store, but that store told the muggers they were there. Identity theft isn't much different -- since your ID is bought and sold to whomever, without your approval.
We need accountability for data security lapses of this magnitude, plain and simple. We only get one identity, and when it has been dragged through the mud it can take years to recover, and sometimes it's impossible. Unfortunately, it will take new laws and stiff penalties to see any change here, since it's apparently more cost effective to throw your customers under the bus (see number one, above).
It's obvious that the US is going through a period of massive change, largely related to the presence of the Internet and the forces that can exert some influence on it. Some of these issues may be just growing pains, but some of them may be cancer. Thus, it's very important that we not shortchange our technological future for short-term economic and bureaucratic issues. We've sold our society to the electron, and we'll be beholden to anyone who wields it better than we do.
Posted by Paul Venezia on May 9, 2007 02:58 PM
May 02, 2007 | Comments: (0)
Check it out: Deep into APC hardware management
I just barely finished turning up two new datacenters in two different states within two weeks. Exhausting? Definitely. On the plus side, however, I wrote several new tools and plugins to manage all of the APC gear that went into both sites with Nagios and Cacti.
First, a little background. Both datacenters were built to be nearly identical to each other -- from rack layout to equipment, to color-coded patch cabling. The major difference is that one site is cooled with APC ACSC100 In-row air units, and the other cooled with ACRC100 In-row water-cooling units. Both sites are powered from APC Symmetra PX UPSes and PDUs, and use APC racks and 3-phase zero-U rackmount PDUs. In addition, several NetBotz WallBotz 500 units were implemented to provide external environmental monitoring and surveillance of the rooms. Basically, it's all APC gear. I'll be posting more on the build process over the next few weeks, but I wanted to get some of the code out there first.
I wrote two main plugins for Nagios and Cacti to assist in monitoring all this new stuff. The Nagios plugin checks the most pertinent data on the ACRC and ACSC units, as well as the main sensors on the NetBotz units, and the load on each phase on the PDUs. It's come in very handy since the sites were turned up, since I have a easily-digested central view of all PDUs, or all AC units on one page. Tweaking parameters on the AC units becomes very simple when you have all the data in one place, versus having to log into each unit to get status info, or even using APC's Infrastruxure Central Console.
I've released the Nagios plugin, check_apcext, and will be posting the Cacti templates soon. Here's the overview of the Nagios plugin, and a link to the NagiosExchange page. Enjoy.
Usage: ./check_apcext.pl -H <hostip> -C <community> -p <parameter> -w <warnval> -c <critval>Parameters:
APC NetBotz
nbmstemp NetBotz main sensor temp
nbmshum NetBotz main sensor humidity
nbmsairflow NetBotz main sensor airflowAPC Metered Rack PDU (3 phase)
rpduamps Amps on each phaseAPC ACSC In-Row
acscstatus System status (on/standby)
acscload Cooling load
acscoutput Cooling output
acscsupair Supply air
acscairflow Air flow
acscracktemp Rack inlet temp
acsccondin Condenser input temp
acsccondout Condenser outlet tempAPC ACRC In-Row
acrcstatus System status (on/standby)
acrcload Cooling load
acrcoutput Cooling output
acrcairflow Air flow
acrcracktemp Rack inlet temp
acrcsupair Supply air
acrcretair Return air
acrcfanspeed Fan speed
acrcfluidflow Fluid flow
acrcflenttemp Fluid entering temp
acrcflrettemp Fluid return tempThus, in checkcommands.cfg, place the following:
define command{
command_name check_apcext
command_line $USER1$/check_apcext.pl -H $HOSTADDRESS$ -C $ARG1$ -p $ARG2$ -w $ARG3$ -c $ARG4$
}and in services.cfg, you'll have something similar to the following:
define service{
use generic-service
hostgroup_name acsc
service_description ACSC Status
is_volatile 0
contact_groups admins
check_command check_apcext!public!acscstatus
}
define service{
use generic-service
hostgroup_name acsc
service_description ACSC Rack Temps
is_volatile 0
contact_groups admins
check_command check_apcext!public!acscracktemp!90!95
}... and so on, for all parameters you wish to inspect. There are two special cases:
1) ACSC and ACRC status has no warn/critical values -- it's OK if the unit is operating, and WARNING if it's on standby
2) Rack PDUs will flag as WARNING or CRITICAL if any of the three phases is beyond the threshold.TODO:
1) NetBotz external sensor monitoring
2) Other rack PDUs (although I don't have any to test)
3) Bugfixes?
Posted by Paul Venezia on May 2, 2007 11:29 AM
TOP STORIES
IBM boosts BlackBerry accessIntel to develop PC with Alibaba
Adobe refreshes Flash Player
Cybercriminals can rent a botnet
Comcast to buy Plaxo social network
Rootkit for Cisco routers
Leopard interface tweaks
Icahn to launch proxy fight
Office VBA and Mac IT
Test your Geek IQ
ADDITIONAL RESOURCES
- Virtualization: A Step by Step Approach to Success
- Dialing up Agility with Business Transformation
- 5 Things You Need to Know About Storage Virtualization
- Is your smaller organization ready for High Availability?
- Is system maintenance doing more harm than good?
- Virtual Test Lab Automation: Manage development infrastructure
