Off The Record | Anonymous

How to make a DBA blush

Anonymous — 2008-05-06T03:00:00-08:00

As the person who keeps the local instant messenger gateway server running, I have worked with our DBAs to ensure the database is backed up and regularly purged of old records. With the occasional update and patch, it has been running more or less problem-free for about six months. I usually only have to touch it when HR or Legal needs to pursue a issue. The oldest records are supposed to be purged weekly to keep the size down. A week or so ago, one of the DBAs called and said the purge wasn't running successfully and he was seeing data corruption. He exported the database and tried to restore it. Still corrupt. He tried to restore from an earlier backup. Still corrupt. He even tried to truncate it without any success. As a last ditch effort, the DBA looked through the export file to see if the data might be salvageable. He came back a few minutes later and asked if I knew what people did in IM. He seemed a little embarrassed. I explained that the system popped up a notice every 15 minutes to remind people that their IM conversations were being logged. In theory, that notification... READ MORE

Don't do this in Windows

Anonymous — 2008-04-29T03:00:00-08:00

Our own Test Center guru Brian Chee encountered quite a few setbacks in a recent installation of Office Communications Server. His blog, Geeks in Paradise, often offers insight -- both amusing and useful -- into the working life of an IT professional. We're posting an example of those insights here in Off the Record this week not only to give you a taste of Brian's candor but also to provide some tips in case you're planning your own foray into Office Communications Server land. As Windows ages and evolves, it's sometimes too easy to forget those little gotchas that creep up and become the bane of sysadmins. My scenario is working on the Interop iLabs (technology demonstration areas) on Unified Communication. I'm responsible for bringing up Office Communications Server so that our group can get both voice and presence status to transit a gateway to a Jabber server through a federation interface. But I lost a bunch of time on little nit-picky issues as I burned the midnight oil rushing to meet our ship deadline.... So for the newbies, take note and save some bookmarks. For the old timers, have a good chuckle and remember the pain you went through... READ MORE

Congratulations, you have been integrated

Anonymous — 2008-04-22T03:00:00-08:00

I work as a senior manager for a small high-technology engineering firm. We were recently acquired and integration is now being inflicted upon us. One of the early integration targets was corporate travel. For the first time since the genesis of Travelocity, Expedia, Orbitz, and their ilk, we have a corporate travel agent whose use is mandated for all travel. If there is anything more useless to a bunch of brilliant engineers with Web access and smartphones than an old-fashioned travel agent, it's hard to imagine what it is. Our accounting and timekeeping systems are now being integrated. By integration, the parent company means ditching our systems and force-fitting us into their less appropriate systems. The timekeeping affects the line engineers the most. We are subject to total time accounting and therefore everyone must record their time daily. Our old system was junked in the name of "integration" because a different system that was already in use by the parent company could import time charges into their accounting system directly. Our old system required one person to perform two steps, both of which were almost completely automated. The cost for removing these two steps is the almost complete lack of... READ MORE

Our security is secretly secure

Anonymous — 2008-04-15T03:00:00-08:00

I make no bones about being a bigot when it comes to routing gear. I like the company that has the bridge on the box. I also manage the world wide WAN resources of a little company that had a few billion dollars in sales last year. Even converted to euros, that's a chunk of change. We bought a division that is outsourced to one of the big three-letter acronym companies. There is a firewall between the parent company and the new division until the turnover process is completed. As the day for turnover approached, we started asking for details on firewalls, routers, and switches. We were inundated with pictures, spreadsheets, and procedure documents but no actual useful information. I think this particular group adhered to the baffle-them-with-bull-stuff rule. In one of our weekly time-waisting conference calls, feeling incredibly frustrated at the impending turnover with nary a solid piece of documentation, the outsourcers finally agreed to run some commands on the firewall and routers and send us the output. Being the wise guy that I am, I told them I only needed one command executed. Mind you, on this call are people who supposedly work in networking for a living.... READ MORE

Laptop follies

Anonymous — 2008-04-08T03:00:00-08:00

I did tech support for a company with a few dozen account managers scattered throughout the U.S. Most of them had many years of experience in a business with a multitude of nuance and detail that had to be attended to with the initial contract, and on a continuing basis. Millions of dollars a month were at stake with each account. Most of the account managers were salespeople at heart. Top management had a long history of encouraging their sometimes outsized ideas of entitlement. Hotshots were hired. Their sense of entitlement, and lack of technical ability, may have been larger all of the others combined. In fairness, most of them were intelligent, conscientious, and appreciated the extra effort it took for us to support them by phone and e-mail. We gave thanks for Apple Remote Desktop often, as it saved us hours of trying to control wayward mouse clicks and explaining, for the tenth time, how to get to the Apple menu. Then there were the others. One was a little ditzy, but those in the know reported she was good at her job. Unfortunately, she was one of those who did not believe that her job included a minimum... READ MORE

Bait and switch IT

Anonymous — 2008-04-01T03:00:00-08:00

My company, U, acquired another company, V. The entire IT department at V was outsourced to Company TLA (Three Letter Acronym). In the process of trying to discoveri the extent of the network we had recently acquired, we asked TLA reps for diagrams and any other relevant information such as network size, IP addresses, and numbers of PC desktops. Let's just say that my dog leaves more legible information on a fire hydrant than what we got from the TLA people. Most of the diagrams looked more like a game of pick-up-sticks than networking. I was unaware you could cross that many lines in one drawing or that you could intentionally loop traffic through a firewall more than once. We decided they were either brilliant geniuses or blooming morons. You can guess which camp I put them in. One particular site was particularly sparse on information. When we asked for additional material, we were told an investigation would be required. Several days, even a week passed before we got the results back. The diagrams and information were minimally improved but it came with a nice cover letter detailing the deficiencies of this site and how it was not up to... READ MORE

How to keep your servers moist

Anonymous — 2008-03-25T03:00:00-08:00

I've done a lot of contracting but this one tops them all. I worked for a company as a contractor to design and oversee the building and migration of a new server room. This was a major move up for them and they wanted the works with waterless fire suppression UPS, generator cooling, and so on. They liked the design and all the features, and planned to go ahead with it. After the demolition was done on the space and the new work started, the CIO left his position and the CFO stepped in for the interim. He had no understanding of IT. Based on recommendations from other sources, he decided the company could save money by replacing the fire suppression system with good old standard sprinklers. And that they could reduce the fire rating required for the room, thereby saving even more money. Yes, my blood ran cold, too. I protested the changes and explained the risks and issues but he would have none of it. He'd made his decision. Well, I get paid either way so his wishes were carried out. The project was completed on time and well under budget. A bonus was paid to the CFO... READ MORE

How fast can you move a datacenter?

Anonymous — 2008-03-18T03:00:00-08:00

We had been with our datacenter provider for a little over two years and were planning to move out. We believe in partnering with our vendors so our intended move was no surprise to them. We agreed to go month to month rather than sign up for a 6-month extension. Everything seemed good; we had plans to upgrade our power, cooling, fire suppression, and build the room. We were on track to move in July, but in February got notice that they colocation facility was kicking us out with 45 days notice. Luckily the team I have is top notch and rather than panic, we aggressively planned. We came up with contingencies for power (we didn't have enough) but could retire some systems, replace some older 21-inch CRT monitors, and virtualize a lot of systems in the short amount of time we had. We didn't have cooling but were able to secure temporary cooling units in time for the move. We needed backups but luckily had a really good relationship with our vendor who agreed to expedite new hardware for us. We met with our executives and explained the risks: limited redundancy on power and cooling, none on our... READ MORE

Finding my own level of incompetence

Anonymous — 2008-03-11T12:01:33-08:00

I am temperamentally a lone or small-group software developer, and have always generated frequent -- usually weekly -- builds for my customer to review. I was so successful at this at one company that I was promoted to manager. I had taken on the contract development of a DOS application for PCs in C from a regular customer. There was just too much work for me to deliver it on time by myself so I hired a programming manager -- let's call him Bill -- and he hired three more C programmers and a technical writer. He had a great track record, so I gave him the management of the project. I took on the database design and the assembly language programming, but otherwise tried to stay out of Bill's way and manage the business. We started work in the summer. Our first deliverable build was due in October. I booked a flight to the customer's site for myself and Bill two months ahead of time. I was very impressed by the progress everyone was reporting until it got to 90 percent and stayed there. Uh, oh. And where was the integrated build? Not to worry, said Bill: Everything was... READ MORE

Manual? What manual?

Anonymous — 2008-03-04T03:00:00-08:00

We have all had our own personal "learning experiences." I experienced one back in 1991 when I was the second-shift computer operator responsible for backing up our Novell servers, including payroll. My job was to run the backups and any other little jobs that I could do, such as run reports, change printer ribbons, and the like. It was a great job for a college student learning about computers and IT. I'd been doing it for 6 months when we got a new version of backup software. Upgrades seemed easy enough and I was looking for more to do, so I offered to do it after hours. The install went fine, so fine in fact, that I didn't even need to read the manual. That night the backups worked perfectly. I was proud of the way I handled it and pretty excited that my first real IT project went so smoothly. All was good for three months or so, when I got a call from the first shift admin, Al. "Hey, did backups finish OK last night?" Al asked. I knew they had worked since I had watched them, so I replied, "Yup sure did." I looked at the log... READ MORE

Readers: Keep your pants on!

Anonymous — 2008-02-26T03:00:00-08:00

Last week's Off the Record showcased a junior IT guy caught in a tough situation. Readers sounded off with their own advice. On the one hand, there's your pride; on the other, your job -- and your trousers. Last week's Off the Record told the story of a young fellow in IT support just trying to make his way in the datacenter when he accidentally pushed the wrong button. To make a long story short, if he released the button, a critical server would have shut down while people were using it, and he'd be history. Yet he couldn't reach the help line some 10 feet away. And so he took off his pants and attempted to lasso the phone, dragging it to him. Well, you get the picture. (For the down-and-dirty details, read “Don’t lift a finger.”) Our IT readers, problem-solvers by nature, had plenty of alternative solutions for this fellow. Some of them were pretty creative. Practicality goes a long way, as John writes: "Why not use his belt to loop around the rack and possibly tie the button down?" Or, for that matter, shoelaces would work, too. If irony is what you seek, check out Sachin's advice... READ MORE

Don't lift a finger

Anonymous — 2008-02-19T03:00:00-08:00

This incident dates back to 1998 when I was an employee of a very large IT company on the North Side of Dublin. My position there was as a junior member of the IT support team: a team supporting 650 users. One day one of the other guys on the team asked me to head down to one of the server rooms and shut down a particular server. It was a noncritical server, so there was no issue with doing this, at least I didn't think there was. A brief amble later I arrived at my final destination, and entered the server room. It seemed more like a scene from a NASA film with huge fans and blinking lights everywhere. The Server I was to shut down was called "Dub06." It was connected along with 5 other servers to a single keyboard, mouse and monitor via a switch box. I selected the corresponding button on the switch box to give me control over Dub06. All fine there. I then told the computer to shut down, and after about two minutes I got the message saying "It is now safe to switch of your computer." Grand, I thought. I reached over... READ MORE

Nothing cheeky about safety

Anonymous — 2008-02-12T03:00:00-08:00

The incident you are about to read about is entirely true in every single respect. There is no exaggeration anywhere, as any would be pointless. About four years ago, I was contracting to a large insurance firm in their IT department in City Centre Dublin (Dawson Street, as it happens, for those who wish to guess what the firm is). Anyway, my boss, Harry for the sake of this account, had asked me to go to Server Room Two and repatch some of the panels, as the cables were in an awful state. After about an hour on the job, alone in the server room, I started to get a little bored and so decided to go get a cup of coffee. I began to untangle myself from the cables around my feet and in the process turned around in order to make my exit from the room. Unfortunately, untangling myself suddenly became a problem as my foot slipped through a gap in the floor. I instantly realized my foot was jammed and so I bent down to attempt to free it. This is where things went haywire. During the action of bending down, my bum extended toward the patch... READ MORE

Avoiding dot-bomb 2.0

Anonymous — 2008-02-05T03:00:00-08:00

With an increasing wave of Web-based applications in development, it may be worth remembering some of the follies from the dot-com boom era so that we don't repeat some of their mistakes in the new enthusiasm. [Submit your own tales of IT woe! Email them to offtherecord@infoworld.com and receive a $50 American Express gift cheque for your trouble.] I was a VP of development at a site trying to attract a specific demographic slice, one that we all thought would be highly valued by advertisers and individuals alike. With several tens of millions of dollars from investors, we had a huge staff, lots of partner contracts, and a completely unscalable architecture. And calling it architecture is kind; it was a hodgepodge of tools and hand-coding that meant implementing anything took inordinate time and human resources. I came in after launch and, having worked at more traditional companies, tried to bring some process order to the chaos. The young wunderkinds who largely staffed the company believed in the popular Silicon Valley mantra that the old rules didn't apply on the Web, and they were determined to do the opposite of "established" technology and business practice. Little things such as budget approval... READ MORE

Black plasma

Anonymous — 2008-01-29T03:00:00-08:00

Nobody challenged the loud, overconfident marketing exec who thought an outdated idea was hip A high-profile exec was hired to do marketing, and find or create new ways to drum up business. He was smart, fairly good looking, and thought highly of every word he spoke. Although a little bombastic, he made sense, or appeared to, often enough to have some credibility. He had not been in our business in many years, but apparently recognized it as an easy entry to bigger and better things. He hired assistants. He bought new lights and office accessories. He hired a fancy-pants and probably very expensive cohort to help him hatch new and brilliant schemes. He presumed to know better than those with more time in the business than he had birthdays. His clothes were expensive, his German cars were new and large. His grandiose house was known to all who would listen, and a few who would rather they hadn't. He was the Big Cheese. He didn't care what kind, as long as it was rare and expensive. It turned out to be a little stinkier than anticipated. After the assistants, all of whom adopted his self-important airs, were outfitted with the... READ MORE