May 30, 2006 | Comments: (0)
Just a few days ago I put the finishing touches on the homegrown Ultra40 project. In response to many folks who mentioned that while the Ultra40 was quite an impressive workstation, you could build one for far less. So I did.
I chose solid server-class parts, since that's really what this system required, with the obvious exception of the video card. Antec's Titan550 case seemed the best bet to fit the large Tyan Thunder K8WE mainboard, and included the TruePower2 550W power supply. Two dual-core AMD Opteron 285s fit the bill, along with two Zalman CNPS9500LED CPU coolers. Disk was handled by a pair of Western Digital 250GB SATA drives and a Sony 16x DVD-RW drive. Following this I threw in two gigs of DDR400 registered ECC RAM and a SoundBlaster Audigy2 Platinum Pro soundcard to provide the 5.1 and SPDI/F support. Completing the picture was a brand-spanking new nVidia Quadro 3500 PCI-X video card. Mix well with a fresh installation of Fedora Core 5 and serve.
I've been running this system for a few days and it's been perfectly stable and incredibly responsive. The nVidia Quadro 3500 is the newest in the Quadro line, and simply blew me away. It's driving two 21" Sun GDM-5410 CRTs at 1920x1440, and pushes glxgears at 1,200fps at 1920x1440x75. At 640x480, it's well over 6,000fps. Using the nVidia Linux x86_64 driver version 1.0-8756, I ran into a little bit of trouble, since they deprecated IgnoreEDID in favor of UseEDID, causing the card to step down to the max resolution claimed by the monitors, which is 1600x1200. That just wouldn't do. Setting Option "UseEDID" "false" cleaned that up nicely. The only other problem here was the cursor animation flicker that's a known bug in the nVidia Linux driver. Setting Option "SWCursor" "true" helped a bit, but that's not a great solution. Hopefully this will be fixed in a later rev of the driver.
The Tyan Thunder K8WE uses the nVidia chipset and is actually quite similar to the Sun Ultra40 in this respect, including the use of the nVidia SATA RAID chipset, which doesn't have native Linux support. Since I'm actually running mirrored 250GB SATA drives in the box, I opted for the Linux software mirroring, which is extremely fast -- and more responsive than the 3ware 8002 SATA RAID controller I'd been using in a previous workstation. In fact, I consistently see buffered reads from the RAID1 device at over 60MB/s. I liked the parallel CPU layout on this mainboard as well, since in the Titan550 the CPU sockets line up with the 120mm rear fan, and using the simply enormous Zalman CPU coolers, it's possible to get all the CPU heat to flow directly out the rear of the case in a straight line. I did put an 80mm intake fan in the front, which was a bit of a challenge given the front-loading nature of the Titan550 case, but the overall result was a system that runs remarkably cool. With the mainboard ambient temp at 93F, the CPUs run around 103F each, slightly higher under extreme load. The downside was that one of the Zalman fans quit about 12 hours after I installed it. It was on the rear CPU, and I didn't notice that it had happened for an hour or so. Normally, a fan dying on a CPU cooler is a recipe for disaster, but when I noticed that the fan wasn't spinning, I checked the temps; that CPU was running 118F, even without the fan running. Undoubtedly this was due to the working fan on the front CPU and the large exhaust fan right behind the bad unit, but it's still quite impressive that a passive heatsink could work so well, even if it's not supposed to be passive.
Of course one of the most dangerous tasks in building custom systems is replacing a CPU cooler. Since the thermal paste hardens after use, it's not a great idea to just wrench the heatsink off the CPU cold, so powering up the system to melt the paste somewhat generally works, but you have to hit that window perfectly between melting the paste and removing the cooler or cooking the CPU. Otherwise, you'll have a nifty Opteron 285 keychain. Added to that possibility was the fact that I had to do this while the mainboard was installed in the case, since I'd already finished building the whole system, and I wasn't about to remove everything including the mainboard to replace this part. After reapplying the paste with a foam peanut (by far the best tool for the job), I fired everything back up. The replacement cooler has been working well since it was installed, but I've been watching the fan RPMs and set gkrellm to trigger a warning notification if the RPMs on that fan drop below 2,000. While I was quite unpleased that one of the Zalman units failed almost immediately, I like them overall -- they look great and function very well in this system... so far.
The Titan550 is really a server case, so there's relatively little in the way of bells and whistles, but it's a solid platform with plenty of internal space, and the front-loading 3.5" drive bays are a nice touch since you don't have to drag hard drives across a crowded interior, they slide out the front on rails. The only problem there is that the intake fan mounts are on the hinged door covering the disks, and the distance from the fans to the mainboard fan headers is quite far. With a few modifications to the fan and power cabling, I made it work, but you have to disconnect the fan from the mainboard to open the front cage, and retrieving the cable after this can be a pain. I do like the quiet nature of the case though, with shock-mounted rails for the disks, a quiet powersupply with internal fan RPM leads, and a large 120mm rear exhaust fan. The rear fan doesn't provide a tach though, so I'll be replacing it with one that does shortly.
The installation of FC5 went smoothly and very quickly, as you might expect. Following the first boot, I installed yum repos for livna, RPMForge, FreshRPMS, dag, and ATrpms.net, took a quick list of RPMs installed on my current FC3 system with rpm -qa --qf '%{NAME}\n' > origrpms.txt and did the same on the new system. Running them though comm to find the differences, a quick manual perusal of the final list, and then yum install `cat ./rpmlist.txt | tr '\n' ' '`. Within a few minutes, all my apps and their dependencies were installing, including mplayer, xmms-mp3, easytag, and so forth. What's better is that pirut, the software installation manager, searches enabled yum repos, so firing up that tool will let you search for packages across multiple repositories. Nicely done.
As far as raw performance, I no longer have the Ultra40 in the lab, but I can say with certainty that this box is actually faster than the Ultra40. This is due in no small part to the Opteron 285s vs the 280s in the Ultra40, but regardless, even if it measured up equally, it's still cheaper. You forego the Sun tools, support, grid applications and so forth, but if you don't need them, you can build your own Ultra40 and save yourself the equivalent of a new MacBook Pro.
| Part | Cost |
|---|---|
| Antec Titan550 | $180 |
| Tyan Thunder K8WE | $500 |
| AMD Opteron 285 CPUs | $2,000 |
| nVidia Quadro 3500 | $1,500 |
| Western Digital 250GB SATA drive | $100 |
| Crucial 8GB DDR400 RAM | $800 |
| SoundBlaster Audigy2 Platinum Pro | $175 |
| Sony 16x DVD-RW | $50 |
| Zalman CNPS9500LED CPU cooler | $100 |
| Misc Parts | $15 |
| Total | $5,430 |
| Sun Ultra40 | $7,000 |
all prices approximate
Posted by Paul Venezia on May 30, 2006 01:50 PM
May 25, 2006 | Comments: (0)
If you have a MacBook or PowerBook with the embedded motion sensor, you have to see Erling Ellingsen's SmackBook. Desktop paging with a tap of the hand; so very cool. I normally see things like this and appreciate the inventive nature of the author, but rarely do I bother to actually implement them. This was an exception.
If you read the comments you'll find patched binaries of Desktop Manager (a great app that I've been using for eons) and some hints on getting everything working. In my case, I'm running 1.67Ghz 15" PowerBook G4 and I had to do some fiddling with the thresholds after building the patched Desktop Pager. I'm still working on getting the settings just right, but if you're having trouble, try this modified smack.pl:
#!/usr/bin/perl
use strict;
my $stable;
open F,"./AMSTracker -s -u0.01 |";
while(
my @a = /(-?\d+)/g;
print, next if @a != 3;
# we get a signed short written as two unsigned bytes
my $x = $a[0];
if(abs($x) < 10) {
$stable++;
}
if(abs($x) > 15 && $stable > 15) {
$stable = 0;
my $foo = $x < 0 ? 'Prev' : 'Next';
system "./notify SwitchTo${foo}Workspace\n";
}
}
It's a bit trying to find the line between breaking your screen hinges to shift desktops and having them switch too easily. The easiest way to gauge what's happening is to run AMSTracker -s -u0.01 > test and tap each side of the screen at an appropriate level, then take a look at the resulting values. Nice work, Erling!
Posted by Paul Venezia on May 25, 2006 03:52 PM
May 23, 2006 | Comments: (0)
Since I was playing around with my IPCop firewall anyway to do the gkrellmd work, I decided to upgrade it to 1.4.10 and install the ZERINA OpenVPN addon. Even though this isn't an official IPCop addon, it works very well, has a simple installer, and integrates very nicely with the IPCop Web UI. After generating all the PKI information, including the client certs, I installed Tunnelblick 3.0RC2 for OS X on my PowerBook. The OpenVPN addon is so complete that it will actually generate a zipfile containing a valid OpenVPN configuration for connecting to the firewall as well as the client PKS12 certificate right from the IPCop Web UI. I pulled this down, tossed it in ~pvenezia/Library/openvpn and fired up Tunnelblick. No go on the first try with a rather bizarre error claiming "unroutable packet received" from the IPCop system. Then I realized that the time on my firewall was off by over an hour, which would cause problems with the certs. I set the time and configured NTP time sync, and tried again. Bam -- instant secure access with more than a bit of panache. For those running Windows, check out the nicely detailed howto, including Windows client setup.
Posted by Paul Venezia on May 23, 2006 12:09 PM
May 22, 2006 | Comments: (0)
| The thought occurred to me the other day that it might be cool to have a gkrellm monitor on my main workstation displaying throughput on my IPCop firewall. I couldn't find a gkrellmd addon for IPCop, so I put one together. This is based on gkrellm-daemon 2.2.5 and includes the necessary glib2 2.4.7 libraries.
Instructions are in the tarball, but essentially you just |  |
Posted by Paul Venezia on May 22, 2006 04:14 PM
May 18, 2006 | Comments: (0)
Another little tweak for dealing with MIMEDefang problems. Recently, someone sent me a large attachment via email. As we all know, this is a clear violation of the Geneva Convention, but there it was. Or rather, wasn't.
Large attachments haven't been a problem in the past, so I was a little surprised to see this one being bounced by sendmail due to apparent MIMEDefang errors. The culprit turned out to be Sendmail's milter timeouts coupled with a higher-than-normal load on the server. Scanning the attachment took longer than the timeout set on that milter, and thus, the message was rejected with a 451 error. The fix was to increase those timeouts in the sendmail.mc file thusly:
INPUT_MAIL_FILTER(`mimedefang', `S=local:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=C:15m;S:10m;R:10m;E:15m')dnl
The timeouts are in the flags at the end of the string:
C - Timeout for connecting to a milter. If this is set to zero, use the system connect timeout
E - Overall timeout between sending end-of-message to milter and waiting for the final response
R - Timeout for reading a reply from the milter
S - Timeout for sending information from the MTA to a milter
A quick fix, and perhaps a bandaid to be sure. I think it might be time to drop some new hardware into that mailserver. In the past week, over 190,000 incoming emails were rejected by the DNSBL filters alone, not to mention the tens of thousands that weren't, but were caught by greylisting, SpamAssassin, and clamav. That's a lot of work for an elderly HP Kayak XU500...
Posted by Paul Venezia on May 18, 2006 09:32 AM
May 11, 2006 | Comments: (0)
Due to the responses that I got from the Sun Ultra40 entry, I've undertaken a side project to replicate the same system with commodity parts and see what happens. So far, AMD has kicked in two Opteron 285 CPUs, Tyan has supplied a Thunder K8WE mainboard, and Antec sent a Titan 550 case and 500w PS. I'm going to toss in a couple gigs of DDR400, a few SATA 3.0Gb disks, hopefully a comparable nVidia video card, and a DVD-RW drive and bring to a boil. Stay tuned.
Posted by Paul Venezia on May 11, 2006 03:20 AM
May 07, 2006 | Comments: (0)
Although I've been using MIMEDefang as the primary component in my email filtering path for years, I find that once in awhile it will mysteriously stop functioning. When this happens, it usually spontaneously coughs up an error state with no apparent changes to the configuration or filtering code. Now I've spent very little time actually working with MIMEDefang, though I've built several dozen mail filtering servers based on it. This is due to the fact that except for these incidents, it Just Works™ 99% of the time.
On my primary mail server a problem such as this might happen once or twice a year, which prompts me to upgrade to the newest version which I had probably been lax in doing anyway. Nine times out of ten, this "fixes" the problem and it's all but forgotten until next time.
The other day I ran into a situation where I would see MIMEDefang go to an error state with the message MIMEDefang-2.56: accept() returned invalid socket (Result too large), try again. There was a dearth of information on this error in the MIMEDefang forums or on Google, and I was running the most recent version. So, I had to dig a little deeper.
Using md-mx-ctrl histo I could see that of the 10 slave processes that I had configured as the max number, the last few were hardly touched, so it didn't seem to be a concurrency problem. I did note that earlier in the day there were some spurious memory-related errors on the server, and I thought it possible that a large message or batch of messages had exhausted the memory limits for the processing slave, or even the whole group, so I upped MX_MAX_RSS to 20MB and MX_MAX_AS to 100MB in the MIMEDefang startup script. Upon restarting MIMEDefang and leaving a daemon to watch the maillog for errors, the problem appears to be resolved.
As always, YMMV.
Posted by Paul Venezia on May 7, 2006 09:21 AM
May 03, 2006 | Comments: (0)
In order to test some security gear, I'm in the process of collecting samples of worms and viruses... which isn't as easy as you might think. It's simple enough to put an unprotected Windows XP system live on the 'net for a few minutes to catch any number of bugs, but to be able to handle them properly, they need to be distilled back into their transmitted form, which is easily done with Ethereal.
Email-borne critters are a bit of a different story. In order to catch a few of these, I altered my MIMEDefang filter to quarantine any discovered viruses in email, which results in the message being dumped in the MD-Quarantine folder. In order to turn the base64-encoded files into a regular executable or zipfile, it's simplest to use openssl: openssl enc -d -base64 -in ./ENTIRE_MESSAGE -out ./test.zip.
Peeling out these files from a TCP stream is slightly more difficult, as you have to find the conversation that actually contains the bug, which could be a TFTP, FTP, or HTTP transaction, and using the "Follow TCP Stream" functions in Ethereal, decode the stream as raw and save it to a file.
Oh, and that unprotected Windows XP system I left out as a honeypot? It took all of 30 seconds to get hit, and about 5 minutes to catch three different viruses and two bot control programs.
Posted by Paul Venezia on May 3, 2006 07:54 PM
TOP STORIES
Microsoft's post-Yahoo optionsNet neutrality bill introduced
MS adds $3 million to Big Easy
AMD's Java improvement efforts
Leopard at 6 months
Intel still investing in WiMax
Yahoo tests aggregated search
Developers vs designers
Sun defends JavaFX Script
Botnet spams 60B a day
ADDITIONAL RESOURCES
- Virtualization: A Step by Step Approach to Success
- Dialing up Agility with Business Transformation
- 5 Things You Need to Know About Storage Virtualization
- Virtual Test Lab Automation: Manage development infrastructure
- Improve Resource Utilization and Lower Operating Costs
- Protect Your Data with SSL
