Google Search » Off The Record | Anonymous » Is it hot in here or is it me?

June 03, 2008 | Comments: (0)

Is it hot in here or is it me?

Like a lot of IT groups we don't manage the physical plant (building, physical security, power, and such). We are lucky enough to work very closely with our facilities department, though, and have them set up with our paging system. When our temperature sensors register high temperatures, the facilities department gets paged along with us. We also have a monitor set up in the guard shack that shows the temperature so they can monitor it too.

One night all the pagers went off with an overtemp alarm. While we were still scrambling, however, it magically cleared. A few admins lived close enough to check on things anyway. When they got into the datacenter, the heat was overpowering: at least 85 degrees, not counting the hot spots.

The temperature sensor read a perfect 62 degrees. What could cause such a discrepancy?

One of the main 30-ton AC units had failed and a temporary portable put into service. When the 3-ton portable didn't make the temperature alarm go away, the probe was moved in front of the running unit to "fix" the problem.

Now before you all start saying how facilities departments just don't understand, we had a similar problem with a former server admin. One night the pager went off because one of the remote DCs failed. Not a big deal: Performance was degraded but still working. Nevertheless, we still worked to resolve the issue. A few minutes later the alarm cleared, but performance was still poor. It turns out the hardware had failed but rather than go through the hassle of fixing the hardware after hours, the admin simply brought the IP up on another machine.

Surely you've seen some interesting "fixes" in your time ... ?

Posted by Anonymous on June 3, 2008 03:00 AM

RATE THIS ARTICLE: