- The high-tech redneck
- Google Chrome, or 'Great, yet another browser to support'
- Who needs identity theft when you have identity gifts?
- Identity theft should be a crime
- Network Solutions under attack
- This is why we have labs
- Working with Vista, revisited
- Vista on a smaller scale
- Maybe I'm not alone. Google goes after the Library of Congress
- Some reflections and a reader's observations on the Terry Childs case
September 29, 2008 | Comments: (0)
When I was younger and had boundless energy, it seems that I spent vast amounts of time online. I wasn't really into games, chatting, or other social pursuits, however -- I was completely and totally obsessed with how computers and networks function. I spent that time learning various computer languages; developing a close relationship with routing, switching, BGP, OSPF, spanning-tree, and other network technologies; building workstations and servers, and so on and so forth. I don't really recall a time when there was a clear distinction between work and nonwork; it all fell in the same pile. When I was building ISPs, there was always work to be done, day and night. As a consultant, that remains true. Testing and reviewing IT hardware and software follows those lines as well. I used to joke with friends that at some point, I'd give it all up and become a goat farmer.
Now it seems I actually am becoming a goat farmer.
Perhaps I'm slowing down somewhat. I still spend at least 10 hours a day in front of the computer, but at some point I began to need a separation from that world. I have apparently found it at the other end of the technological spectrum -- I'm getting into farming.
I have chickens and pigs, bought a tractor, and find myself actually enjoying heading out into the fields and getting "real work" done. It really couldn't be further from my normal routine. That's probably why it's so enticing. I spent most of this weekend stumping and clearing an overgrown field to prepare it for three new pigs. In the spring, I'll add the goats to fulfill my tongue-in-cheek prediction from years past.
Of course, while I'm out on the tractor pulling up 200-year-old tree stumps and running my trusty Stihl chain saw, my iPhone is in my pocket, and I have a strong Wi-Fi signal in the field. There's something very odd about sshing into a server to fix a minor problem while sitting on a running tractor.
I'm coming to this from the technical side, but there are many "real" independent farmers moving in a similar direction from the other end. This weekend, I headed out to Wellscroft Farm, a working farm with some spectacular views that also runs a business selling fencing supplies. In addition to its main focus on sheep and Border Collie breeding, the farm has pigs, goats, cattle, chickens, and hay fields, it has a large barn filled with fencing of every conceivable type. The barn is quite old but sturdy, and in an equally rustic connected building, five Dell workstations with flat-screen monitors sit, joined by a simple LAN, and connected to the Internet via what appears to be a microwave link to a distant building equipped with broadband. They sell online via their own Web site, and are surprisingly adept at leveraging the Internet in their business. Suffice it to say, I was impressed. I also bought several sections of electrified fencing, grounding rods, and other assorted parts.
From the farmers' point of view, properly using the Internet and technology is a means to grow their business -- it's a tool like a tractor or a York rake. Their apparent full-scale adoption of that technology is somewhat surprising. From my point of view, working in the fields and herding livestock is a way to reconnect with the tangible world, unlike my normal routine of working with SQL queries and herding bits. It's also equally surprising to my high-tech friends, many of whom think I may have lost my mind. However, I can say with certainty that homegrown eggs are mighty tasty.
I have some plans to run solar-powered Wi-Fi cameras out in the fields, perhaps even integrate a fence warning system that will send me an e-mail if the electric fence goes down, and temperature probes that will do the same if temps go out of range. I don't believe that SNMP-enabled solar fence energizers are readily available, so I might have to make those myself. I'm sure that large-scale agribusiness has all manner of automated monitoring and telemetry infrastructures, but I don't believe it's really trickled down to small-scale operations yet.
So far, my forays into farming have proven cathartic. I wonder if I'll still feel that way when I head out to feed the pigs in February with temperatures hovering around -10 degrees Fahrenheit.
Posted by Paul Venezia on September 29, 2008 10:57 AM
September 02, 2008 | Comments: (0)
Google Chrome, or 'Great, yet another browser to support'
After all of ten minutes using Chrome, Google's brand-new browser, I can safely say that yes, it is a Web browser. Surprisingly, most of the sites that I've visited (including those that I've written) seem to work as advertised, obviously with the lack of plug-in support. Flash works, but Shockwave and Java don't, among others.
It's simple, it's sleek, and it doesn't have much distraction from the main window. It certainly seems fast, pages render quickly and appropriately, even those that bend space and time with CSS like csszengarden.com render like they should.
All in all, not too shabby for a beta browser.
[ Check out InfoWorld's Special Report for all the news, reviews, and commentary on Google's open source Chrome browser. ]
I dig the Incognito mode, although it might be over the heads of normal users who think they can't be tracked, since it may prevent sites you visit from appearing in the browser history, but the hits generated certainly will appear in the server logs on the other side. I also really like the Task Manager that shows you exactly what system resources the browser, tab, and plug-in are consuming -- that's really handy. The combined search/URL field is okay I guess, but didn't really have much of an impact on me either way.
The bookmark creating feature is nice though. It takes a snap of the page apparently built from the meta tags on the site, and creates a bookmark for the page on the desktop, start menu, or quick-launch bar. Google's pushing Chrome by claiming that most Web sites are really applications, and this is one small way to illustrate that thinking.
I don't think I'll be switching from FireFox to Chrome for my main browser, especially since there's no Mac version yet, but it certainly looks promising.
Now, the only real question is exactly how much of your browsing habits are being sent back to Google ...
UPDATE: Well, it's not just browsing habits. Google apparently owns everything you create using Chrome.
Posted by Paul Venezia on September 2, 2008 04:07 PM
August 28, 2008 | Comments: (0)
Who needs identity theft when you have identity gifts?
Hot on the heels of my post earlier this week comes a perfect example of the negligence I was talking about. In this case, nobody had to hack into anything to get access to "[d]etails of customers of three companies, including the Royal Bank of Scotland (RBS) and its subsidiary, Natwest". Also, this appears to be some really high-test info. "The information is said to include account details and in some cases customers' signatures, mobile phone numbers and mothers' maiden names."
The vector for all this data? eBay. They sold a computer containing this information for around US$140. The banks in question are indescribably lucky that the fellow who purchased the system is an IT manager and apparently a stand-up guy.
Though I am curious as to why eBay continues to be involved: '"Clearly such details should never have been included in the hard drive of the computer offered for sale on eBay," said the eBay spokesman.' I don't think that eBay has any real involvement here other than providing the marketplace, but thanks, I guess.
Either way, this is a prime example of my original point. This kind of negligence needs to be met with crippling fines at the very least, and preferably jail time for the executives ultimately in charge. There is simply no valid defense of this situation. "Oops" doesn't cut it.
I do note that the data has not yet been returned. I hope he gets a sizable reward for doing so.
Posted by Paul Venezia on August 28, 2008 07:45 AM
August 24, 2008 | Comments: (0)
Identity theft should be a crime
The act of stealing someone's identity is illegal. Allowing crackers to steal the identity in the first place isn't, but it should be. The most recent case is a doozy: Best Western gave up every piece of data on every customer of theirs since 2007. Note the choice of words -- they gave this information to criminals, whether they meant to or not. Every year, we're treated to several stories of large corporations losing personal information collected from their customers. I wrote about this back in May of last year, and obviously nothing has changed.
I've had to change my credit card numbers several times in the past few years simply because I used them at TJ Maxx and Hannaford stores, both of which managed to give away that information to criminals. The banks covered the losses from illegitimate charges on those accounts, but I had to spend several hours canceling the cards, modifying all the direct-charge accounts that used those numbers, like my NetFlix account, and live without my debit or credit cards for a week or so each time. Once, my card was canceled while I was traveling. There are few things more frustrating than to be on the road with only a few bucks in cash and no access to my accounts because of Hannaford's problems.
This isn't acceptable. I'm tired of playing craps with my identity every time I use a credit or debit card. I can't come up with any justification for TJ Maxx or Hannaford's practice of retaining sensitive credit-card information. They shouldn't keep this data in the first place, much less give it away to anyone who manages to breach their security. And that's the crux of the issue. You can consider this information "stolen" from the corporation, or you can consider it as given away via negligence. As far as I'm concerned, it's the latter.
The responsibilities attached to retaining sensitive personal identity information should include criminal charges against the company responsible for a leak, in addition to the party that receives the information. This isn't a case of blaming the victim, since technically speaking, the company isn't the victim -- their customers or employees are. The company is complicit in the theft of this information since they retained the data in a database that was improperly secured. They should thus be charged as an accessory. Further, attempting to cover up the fact that customer identity information has been stolen should result in even harsher punishments.
Until the penalties for giving away sensitive information in this manner include heavy fines and possibly even jail time for those responsible for securing that information, we'll see this problem occur again and again. The only ramification for these companies now is some bad publicity. That's simply not sufficient.
Posted by Paul Venezia on August 24, 2008 03:30 PM
August 20, 2008 | Comments: (0)
Network Solutions under attack
I've been trying to figure out what is wrong with Network Solutions for a few weeks now. After I posted about their questionable practice of returning incorrect answers for non-authoritative queries on some of their nameservers, I started receiving e-mails and comments on my blog posts about folks outside of the United States who could not access any resources from Network Solutions: no e-mail, Web hosting, their main Web site -- nothing.
It appears that the reason for this is that Network Solutions is the target of a large-scale, prolonged DDoS attack, and the company has been blocking large ranges of public IP addresses from accessing any of their resources in order to fight the attack. Of course, very many legitimate users have been caught in these nets.
Details are sketchy at the moment, but I have been contacted by the company and am quite interested in finding out the details of this situation. I do find it quite amazing that they've been able to keep this under the radar for so long, since it appears that the attack has lasted for several weeks at least.
Generally speaking, DDoS attacks of this scale are "earned," at least according to the attackers. Network Solutions has never enjoyed a great reputation with the Internet underworld, although I don't know of anything specific that might have elicited this response.
In any event, to those reading this in hopes of finally regaining access to their Web sites and e-mail, I don't have good news. As one commenter remarked on my post this morning:
"I asked them to unblock our range since we are their customer and our servers and host are not in any way involved with the DDOS. NetSol's suggestion was to change IP addresses for our web servers. Obviously, this is not an easy thing to do on short notice."
Asking a customer to re-IP their Web servers due to this issue is simply absurd, so it certainly appears that they have no good way to deal with this problem at the moment. It would seem that many people using NETSOL are, well, SOL. At least for now.
Posted by Paul Venezia on August 20, 2008 08:52 PM
August 12, 2008 | Comments: (0)
If you haven't heard that VMware ESX 3.5 U2 inadvertently expires globally today, you probably aren't running VMware.
It appears that VMware mistakenly left a hardcoded timeout in the U2 code. It should have been stripped before the code went out the door, but it wasn't -- so anyone running ESX 3.5 U2 as of today will not be able to migrate or power on any VMs. If they're running, they'll stay running, but if they go down, they won't come back up. There's still no fix available, though VMware now thinks they'll have one ready for this evening, around 6pm PST, hopefully. A media reissue may come tomorrow.
That's a very, very long time.
Fortunately (for me), my VMware farms are safe and sound. That's because regardless of the vendor's track record, unless there's an extremely compelling reason to apply updates to production systems soon after their release, it doesn't happen. This is one of my annoyances with Windows Automatic Updates, and any automated update/patching system. Updating for the sake of updating is very rarely a good thing. Just because the code is newer doesn't mean it's better, and I'd rather that someone else go through the headache of dealing with problems like this. I have ESX 3.5 U2 running in the lab, but I could care less what happens to those VMs. If it ain't broke, don't fix it, indeed.
Of course, not everyone rushed to update and got bitten by this bug -- there are plenty of folks out there putting in their first VMware installations that used ESX 3.5 U2 to start with. What a way to take your first steps into virtualization.
If you're currently dealing with this problem, my sincere condolences.
Posted by Paul Venezia on August 12, 2008 04:59 PM
August 12, 2008 | Comments: (0)
So after spending a week tied to an HP Compaq 2710p tablet running Vista Business Edition, I can say that it's actually quite usable and that overall, I've enjoyed the experience. Vista on this laptop has been relatively stable, able to handle my workload well, and though it's not as polished as Mac OS X, it's not terrible. There are more than a few things that irritate me about the OS, but it's not dog slow on this system, the sleep/wake cycles do work (most of the time) and hibernation works fairly well too. The lack of a native shell is really quite annoying to me, since I generally have a dozen xterms running at any given moment, and I really like to call them from the local system, rather than run an X server on the laptop and ship them from a Linux box, or use putty (or similar) to SSH into other systems just to get to a shell. I am running Cygwin, which mitigates this issue somewhat, but it's still not the same.
I like the gadgets, though on a laptop screen, they get covered quickly. I like the visual effects, which are smooth and attractive on this system. I like the tablet features, including the handwriting recognition which is surprisingly accurate. I was pleased to find that Firefox 3 on Vista is very fast and very stable, even when pushed very hard. If given no other choice, I could run Vista and get my work done.
However, I'm not going to be staying in Vista-land. Since I do have other choices, I still prefer Mac OS X and Linux.
The Start menu in Vista (and all other iterations) has become far too ornery for me. Navigating to new apps, or to find apps is annoying and too clumsy. The quick-launch menu in the taskbar is far too small, and even gadgets that supply a larger quick-launch facility don't really work so well since on a 1280x800 screen, it's behind all other windows most of the time. The ever-changing list of frequently-used applications in the Start menu has always been a source of wonderment to me. Occasionally, I find an app in there that I've actually used recently. The rest are just thoroughly bizarre. I've never used Windows Fax and Scan or Backup Status and Configuration, yet there they are.
While the sleep/suspend cycles do work well (better than anything I've ever experienced in Windows XP), they're still not on par with Mac OS X. Time to wake is quick, but several times the system seemed groggy, and took quite some awhile to present me with a usable interface. That said, I have yet to lose any work or have a spontaneous reboot when coming out of sleep. That's definitely a plus.
I did have some application incompatibilities to overcome, and discovered that HP's Protect Tools identity management tools conflicted significantly with Firefox. The only solution to that was to turn off those tools, since even exempting the application from the protected list didn't fix the problem. I did note that this problem did not occur with Firefox 2, so the problem may actually lie with Firefox.
I was quite annoyed that when common devices such as mice, USB flash drives, and so on were inserted into the system, it takes Vista quite a long time to locate and install the drivers. In Linux and Mac OS X, these devices are usable nearly instantly.
Running anti-virus and anti-malware code on the system bothered me, especially when they cut into battery life. I don't like the idea of these tools requiring so many system resources to protect me from the big, bad Internet. I figure that just running those tools takes about 5% performance away from the system, maybe more depending on use.
Overall, I found Vista to be a suitable OS for me, just not my preferred OS. Where Mac OS X and Linux feel stable and confident, Vista feels shaky in places, leaving me wondering if the next application installation will cause problems with something else -- I'd rather not feel like I was taking the system's life into my hands when an installation progress bar doesn't move for two solid minutes, like when I installed Microsoft Office. I like the idea of restore points, but frankly, I'd rather they weren't necessary. An OS should be able to keep itself together in the face of just about anything, but I've seen Windows go all pear-shaped after a few app installations that did naughty things to the registry for one reason or another. At one time, the Windows Registry might have been a good idea, but it sure seems like a ball and chain now.
I also loathe the number of required reboots, and the default setting to reboot the system after updates are downloaded. This is a deal-breaker for me. Don't ever interrupt my workflow telling me that I need to reboot in the next five minutes. I'm busy. The only time I should have to reboot a laptop, workstation, or server is when the kernel is updated. Generally speaking, this is still the rule for Linux, FreeBSD and Mac OS X, but not for Windows -- not for any flavor.
As a side note, over the past few weeks, I've found this laptop to be extremely usable. It's light, and compact, but the keyboard is just right, I found that I don't miss the trackpad nearly as much as I thought I would, the screen is bright, and not too grainy (a common problem on tablets), the battery life is quite sufficient, and the extras (like an additional battery, docking station with optical drive, USB hub, etc) were well received. I was quite impressed with the performance of this system given the 1.20Ghz Core 2 Duo ULV CPU and 2GB of RAM. It's very well equipped to handle Vista Business Edition. That said, I do think that Microsoft's minimum requirements for Vista are way too low.
I think the major problem with Vista is that it's not a big enough step away from Windows XP in terms of functionality, stability, and security, but it's definitely a major change in terms of UI elements and design. Microsoft draws strength from the ubiquity of their OS -- everyone knows enough of Windows to be able to use it -- but it's also an Achilles' Heel. They've tried shaking things up in Vista (and Office) to upset that particular applecart, without providing enough of substance to make the IT world follow along. It's one thing for a home user to get a new system with Vista and figure it out, but when you're looking at rolling out a few thousand workstations with Vista, the retraining, hardware upgrade, and time investments simply aren't worth it. This is why most companies aren't falling all over themselves to migrate to Vista -- there's a very limited business case to do so.
But that's not to say that Vista is a terrible OS, along the lines of Windows ME. It's not. It's usable, not too slow on decent hardware, and it seems to be a bit more stable than XP. So there you have it. All told, I like Vista.
In true Internet tradition, please commence with the "(Microsoft|Apple|Linux) fanboy" accusations.
Posted by Paul Venezia on August 12, 2008 09:53 AM
August 04, 2008 | Comments: (0)
I recently posted about getting back in touch with Vista after a rocky start almost a year and a half ago. My conclusions were then (and are now) that Vista isn't the terrible disgrace that some might think. It's definitely not perfect, but it's not that bad.
Of course, I was also running it on an extremely high-end IBM Intellistation zPro with 8GB RAM and a few dual-core CPUs. This is far above the spec that most Vista users enjoy.
So, I decided to scale back a bit, and switch form factors. I'm writing this post on an HP Compaq 2710p tablet running Vista Business Edition with tablet support. HP sent this unit to me at my request, and I heartily thank them for the opportunity. I wanted this system because it's around a year old, and shows off the tablet support in Vista. It has an Intel Core 2 Duo at 1.20Ghz, 2GB of RAM, and an 80GB 4200 RPM SATA disk. Not too shabby for an ultraportable laptop first released last year, but obviously not nearly the same horsepower as the IBM workstation.
I've only had the system for a few days, but I'm embarking on the same basic premise of my earlier posts -- I'm going to forego my MacBook Pro and MacBook Air for a little while, and see if I can work with Vista in a mobile setting.
Part of this is due to my apparent trend towards challenging my own notions. I'm a strictly OSS kinda guy except for my laptops. My personal servers and workstations run a few different varieties of Linux, with the servers tending towards FreeBSD. My laptops are Macs with the exception of a Dell Latitude running CentOS. I haven't run Windows as a primary desktop for a decade. Currently, my sole Windows desktop is the aforementioned IBM zPro, with a few Windows XP VMs floating around the lab.
My only real criteria for desktop operating systems is that they get out of my way. Eye candy is nice, if it's within reason, but core functionality is key, along with the ability to be basically transparent -- when I want to do something at the OS level, I want to get it done, and not spend time dallying around with the OS itself. If you saw my normal working environment, you'd see a dozen brower windows open, a dozen terminals open, logged into different servers, perhaps a 6509 and a router or two, an IM client, Microsoft Word or OpenOffice Writer, and one or two email clients like Thunderbird. I generally run my laptops and workstations very hard.
So far I'm reasonably impressed; the Vista handwriting recognition is surprisingly accurate, and I have horrible handwriting (the unfortunate downside to spending so much time in front of a keyboard). More importantly, the sleep/suspend cycles have yet to fail me. This is a big deal, as I believe that rebooting any system should be an extremely rare event. The one big downside is the lack of a native bash or (in a pinch) csh shell. I'm going to install Cygwin eventually, but it's still not the same.
In any event, it should make for an interesting experiment. I'm going into it with a completely open mind and I have every expectation that I'll survive.
Posted by Paul Venezia on August 4, 2008 08:20 PM
July 31, 2008 | Comments: (0)
Maybe I'm not alone. Google goes after the Library of Congress
For those who read about my trials and tribulations with Google... it appears that after they let me off the hook, they've taken on the Library of Congress.
Dave Bowman: Hello, HAL. Do you read me, HAL?
HAL: Affirmative, Dave, I read you.
Dave Bowman: Open the pod bay door, HAL.
HAL: I'm sorry Dave, I'm afraid I can't do that.
Dave Bowman: What's the problem?
HAL: I think you know what the problem is just as well as I do.
Dave Bowman: What are you talking about, HAL?
HAL: This mission is too important for me to allow you to jeopardize it.
Dave Bowman: I don't know what you're talking about, HAL.
Posted by Paul Venezia on July 31, 2008 09:40 PM
July 31, 2008 | Comments: (0)
Some reflections and a reader's observations on the Terry Childs case
My main concern on the Childs matter is that the case against Childs may be built around a profound lack of understanding of the technology involved. To those outside of IT, a statement in court that the defendant "was watching everything on the network, including information regarding city government, the police, and private emails between government officials" sounds extremely sinister. However, the reality of that statement is far more likely to be that the defendant operated an IDS on the network for security purposes. Nobody in IT would think twice about it, but a jury packed with people that have no real concept of how computers and networks function, much less how large networks are built and maintained might have a different view, regardless of reality.
Recently, I received a very well written email regarding the Childs' case. The author wishes to remain anonymous and his words are his own, though they do channel the vast majority of the emails I've received on this subject. I thought it quite well put.
--
I have been working on computers since before the P.C. was invented. I have also been engineering networks since the thick net days. I'm not saying I agree with what he did, but a lot of it looks like a dedicated (slightly paranoid) admin who did not want anybody to screw up what he considered to be his baby.
1. The only access to the core devices was from a terminal at the Hall of Justice. - Hmmmm…. I need to have an access point to get the core of a city-wide network that is being de-centralized. Where can I put it that is safe. Maybe at the police department? It would take some big brass ones to break in there and try to access the core….
2. He photographed the individual that was removing devices from desks in an unannounced, after-hours audit. - If I am rubbing people the wrong way at work, I am going to want evidence of what was happening if I get drug in front of a review board.
3. He had the routers set to self-destruct on a reboot. - No, he was just overly paranoid. You and I both know you cant remember the configs for the routers in our heads. He has them somewhere so that he can reload them. I am pretty sure that in the past few months he has had to reload at least one device and I can bet he didn’t do it by hand. Not sure why he wont tell them where he keeps them though.
4. All of his data was stored on encrypted devices. - So is mine. Not because I'm hiding anything, but because it is a requirement from central IT. The drives are encrypted. Big whoop. Generate the override key on the servers and get the data (He wasn't a server admin so I am assuming some other admin has the ability to override the keys)
5. Access points at other locations - If you de-centralize a system, you have to have the ability to manage the physical network inside each logical network. You need a way to get in and fix it.
6. He had an IDS that monitored the network. - Hell, I've got hundreds of IDS systems. And yes, one is set to look for other IDS systems that may be trying to probe the device at the core. Security doesn’t stop at the border. If you are decentralizing the process, you need to make sure that the admins of the resulting networks do not start playing with what is left of the central network. If they do, and make a mistake, it can affect everybody. Seen it happen to many times.
7. The request to keep bail mentions he accessed another network the day before he was fired. Funny how it didn’t point out that the investigator in the original filing said it was so he could perform 'requested maintenance' on a system at the Sherriff's Office.
8. He has password lists of other users. - I have them from when I originally generate a password for a new system. And I will admit, I even have a some of the problem users passwords so when they lock themselves out of a system each Monday I can get them back in.
9. He had diagrams and configs of the network at his house. - I'm sure if you dug through my stuff you would get a great lesson in the evolution of networking. When I design a complicated network, I am proud of it. I always keep copies. (plus I work to much and don’t clean out my home office very often)
10. Why are they making such a big deal about the pager? If I had an admin that didn’t have pager notification on the status of devices I would probably fire him. And he better have access from home. I'm not paying a 2 hour travel bonus for a fix that should take 5 minutes. (and I'm not driving 2 hours to fix something that takes 5 minutes to fix either)
--
To me, this is the central thrust of the case, so far. Childs may very well be guilty of something, but if he is, I want to be completely sure that his crime is an actual crime, and not a overhyped fabrication. To see a prosecutor pointing to his pack of matches and declaring him an arsonist, so to speak, would do no good for IT in general.
This case will set precedents, if it ever gets to trial. It would be a sad day indeed if network admins could be arrested for using 'no service password-recovery'. Actually, it could get worse -- if his security measures are the very petard that he's hoisted upon, then the ramifications for security professionals everywhere may be severe.
Childs is innocent until proven guilty, and if there is viable, accurate, and non-hyperbolic proof that he intended to cause the failure of the network, then he should be convicted. Just don't send him up for trying to secure it.
Posted by Paul Venezia on July 31, 2008 06:03 PM
July 30, 2008 | Comments: (0)
A primer on the Terry Childs case for the non-technical
Since I continue to read, see, and hear news stories on the Childs case that depart from reality, I figured I'd put together this handy primer for anyone non-techical who wants to really understand the case. This may or may not apply to many mainstream journalists. I hope it does. Also, please forgive me for the car analogies I'm about to use.
• The "network" as used to describe this case is defined as the hardware used to connect computers to other computers. It is not, and never was, intended to be construed as any form of data, applications, resources, or anything other than the actual hardware.
• Essentially what Childs' did was build a car, give the city the keys to start the engine and drive the car normally, but he locked the hood so nobody else could work on the car.
• At no time since Childs' arrest has the network been unavailable, offline, down, or anything approaching unusable. The caveat to this is the fact that when the DA placed functional VPN usernames and passwords into the public record, all VPN access had to be shut down and reconfigured.
• The passwords that Childs gave to Newsom were to be used to access the hardware comprising the network. They were not his email passwords, passwords to unlock and city databases, or anything of the sort. They were passwords that could be used to log into routers and switches to make changes. Nothing more, nothing less.
• The passwords released by the DA's office were not the passwords they were trying to get from Childs. The passwords they released to the public were for another part of the network entirely, one that provided external access to the network for city employees. In essence, by publishing this list, the city opened the deadbolt on the door to the network, but left the handle locked.
• Modems plugged into routers and switches in various places on a large network are not scandalous, they're common. They're also generally mandatory on large networks. These modems are used to provide instant emergency access to remote locations to reduce or eliminate network downtime.
• Do not confuse a modem with a router. Sadly, these terms are used interchangeably in consumer electronics, but not in this case. The term modem here means an analog phone modem, not a DSL or cable modem (unless specifically noted), and router means a device used to route packets through a network.
• "Sniffers" on a large network are basically mandatory. The fact that Childs' had one is not a surprise, and is not immediate cause for concern. Cisco has manufactured and sold devices designed to be used in core network equipment for just this purpose for many years. Sniffers are put in place to detect intrusions or other problems on the network.
• The routers and switches that comprise a network are essentially fixed-purpose computers. They have a CPU, RAM, and storage (similar to a hard drive). Like a PC, they have an operating system and a configuration that instructs the device on how to function within the network. When changes are made to this configuration, they are made to the running config, not the stored config. The changes are applied instantly, but unless the running configuration is saved to disk, the changes will be lost if the device is powered off. For instance, if you open a saved Word document, and make changes to it, then shut down the computer without saving the document, the changes will be lost.
(NB: This always bothered me about Doogie Howser. He'd type all this stuff into his computer, then just shut it off. Same goes for the end of Stand By Me)
• Requiring specific points of access for administrative functions of a network is not a bad thing. It's generally a security requirement. For instance, to make changes to a network, you must use a specific IP address, perhaps within a specific building.
I'll update this list as events warrant.
Posted by Paul Venezia on July 30, 2008 12:36 PM
July 30, 2008 | Comments: (0)
Anton Chuvakin is not an idiot
I remarked on a blog post by Anton Chuvakin a little while ago, discussing his apparent anti-admin stance. His post in response is quite entertaining, and no, he's not an idiot.
His points on logging and auditing are spot on. If I had a dollar for every syslog server I've installed, or every time I was heartily thankful that centralized logging was running, I'd... well, I think I do have a dollar for each one of those. Anyway, to me, this isn't the issue. Good admins log. Great admins log paranoiacally. Bad admins don't bother. His point on the "bus test" is fairly accurate too:
"it is an unacceptable risk for all but the smallest organizations to have one person who have the power to control access to critical systems AND to place no controls (neither monitoring, auditing nor preventative) on his activity."
However, it's highly unlikely that an organization in that kind of situation would have the wherewithal to implement logging and auditing without the help of that "one person", who could obviously circumvent the process whenever they liked.
It wasn't those stances that rankled me, it was the "start logging and monitoring (and then controlling) their actions!" quote. When management starts controlling the actions of admins, things start to fall apart. When admins become automatons that will do whatever management thinks is best for the network or infrastructure without question, problems appear like magic.
There's a divide between a request and reality. This is true in just about any profession, but in IT, it seems to be taken to another level. Perhaps it's human nature -- non-technical people tend to think that because they're familiar with computers and can send an email, that they have some kind of deeper knowledge than they do. They wouldn't think this way of their doctor, but they will of their IT staff. After all, they both sit in front of monitors every day, right? Oh, and this isn't a just a small business or enterprise problem, this crosses all boundaries.
It's pretty simple, really. Management needs to make business needs clear to IT, and then trust the IT admins decide how best to make that happen. If they can't trust the admins, it's time for either new admins or new management.
Posted by Paul Venezia on July 30, 2008 10:35 AM
July 28, 2008 | Comments: (0)
A little note to Network Solutions
A few days ago, I posted an entry where I pointed out that two Network Solutions DNS servers were returning incorrect answers for non-authoritative domains. Matt Ho, a representative for Network Solutions posted two comments on that post stating that they weren't hacked, and that they were configured to return incorrect answers. He noted that Sedo, the domain parking company, and GoDaddy also do this, and that these servers are not meant to be resolvers. He was very gracious, and even noted that "We've debated time and time again internally about both whether this type of practice is ethical and appropriate." I humbly suggest that it is neither.
Then, this post appeared on Network Solutions' website, also stating these claims. The author says that I got it wrong:
"We respect InfoWorld and Paul, but it’s important to research and confirm the facts before simply stating an opinion. And in this case throwing the word hacked out there without proper homework seemed a little off kilter and unnecessarily causes false alarms when there is no need for them."
Note that I posted a comment to that page several days ago, but that comment has not appeared on the page as of this writing.
The post I published asked a question in the title: "Network Solutions DNS Servers hacked already?", and was posted a day after the DNS bug "hit". In that post, I stated
"I was just hipped to the fact that two DNS servers apparently operated by Network Solutions aren't returning valid results for some domains, notably www.google.com:"
I then pasted raw nslookup and whois output showing that querying those servers for 'www.google.com' returned incorrect responses, and they were part of Network Solution's network.
If you're going to purposefully violate RFCs and configure your nameservers to knowingly return incorrect information, then you cannot complain when someone points this out. My post contained absolutely correct information -- those nameservers aren't returning valid results. This practice may be used by other companies, but that obviously doesn't make it right. I seem to recall my mother saying something about all my friends jumping off a bridge...
Live by the sword, die by the same.
Posted by Paul Venezia on July 28, 2008 12:14 PM
July 25, 2008 | Comments: (0)
The anti-admin stance and the Childs case
In my reading of a few blogs and articles regarding the Childs case, I keep coming across statements that seem to be extremely anti-administrator. For instance, Anton Chuvakin seems to think that all admins should be kept underneath management's boot at all times.
[ Follow the Terry Childs saga with InfoWorld special report: Terry Childs: Admin gone rogue. ]
He references this blog, and specifically a statement I quoted from Richard Childers:
... search Craigslist's 'Jobs' section for the keyword "ownership". I see 674 references to the word, the majority of them in the IT-related industries.Sure, it's a buzzword, but it's also a way of life for many IT professionals. We are paid to TAKE OWNERSHIP. We get bonuses for seeing problems and fixing them -- also known as BEING PROACTIVE."
His take on this is that admins "0WN YOUR BUSINESS". Well, yeah. They were specifically asked to do so in the job posting. He also equates Terry Childs with "a Romanian script kiddie".
The nature of network and system administrators is generally one of high intelligence, and a highly elevated sense of autonomy. At least, that's the good admins. It's a necessary trait, and should most certainly not be squashed. Every single instance I've ever encountered where management decided to deconstruct their own admins' actions and watch them like felons resulted in the quick departure of all the competent admins, and their positions backfilled with people who could toe management's line, but couldn't admin for beans. This generally led to systemic failures of the infrastructure, and that's when I would be called, to come in and clean up the mess.
Perhaps it's human nature to fear what you don't know or understand -- and that's why management can develop a fear of their own employees. Managers can't and don't understand what we do, and thus eventually come to the conclusion that we can't be trusted with our own knowledge. On the face of it, Terry Childs' case would appear to be an example of this, but it's blindingly obvious that this case is an anomaly, and that there's much more going on here than we currently know.
So go right ahead and (as Chuvakin puts it) "start logging and monitoring (and then controlling) their actions". Just keep the phone numbers of those extremely expensive consultants handy. You'll need them.
Posted by Paul Venezia on July 25, 2008 11:12 AM
July 23, 2008 | Comments: (0)
A thousand thoughts on the iPhone
Okay, so I've been using an iPhone for a week now. I previously posted seven reasons why the iPhone wasn't for me, and now that I've had a chance to use one, I have to say that all of them still apply, with the possible exception of the fact that AT&T doesn't offer service in my area. I jailbroke and unlocked an original iPhone in order to use it with my GSM carrier. I ran the old firmware (1.1.4) and upgraded to the v2.0 firmware on Sunday, the day the 2.0 jailbreak iPwnage app was released.
But the rest of those reasons still apply, and after a week, are driving me kinda nuts. There are also a few things that I really like about the iPhone that I didn't expect. Rather than a structured list, here's a stream-of-consciousness presentation:
I can't tell you how annoying it is to have to run iTunes (and iPhoto) every time I plug the phone into my MacBook just to sync my calendar.
I found myself in several situations this past week where I couldn't use the phone as a Bluetooth modem with my MacBook Air, and really, really missed my Nokia N95. For the love of God, Apple, make this happen. I wonder how much the lack of this feature has to do with AT&T being nervous about the amount of data "unlimited" users will use?
Wow, this is a very usable interface.
Wait -- if I can use the Remote app to control iTunes on my MacBook Pro, then WiFi syncing is basically already there. WHY CAN'T I DO THAT NOW?!
It was really handy to be able to run some latency and connectivity tests of a two-way satellite link directly from my phone while standing outside, next to the dish.
The "Customize" app from Installer.app kinda blew up the phone. I had to SSH in a clean up a bunch of detritus after installing it.
The apps from the app store seem to take quite awhile to install... and occasionally caused the phone to spontaneously reboot.
Waiting for Safari to render a page...waiting for Safari to render a page.
Why can't I save usernames and passwords in Safari, anyway?
Boy, I wish that the iPhone had Flash support.
Wow, this is a very usable interface.
What do you mean I can't copy a phone number or address from an email and paste it into a new contact? Oh, right. There's no cut and paste at all...
Wow, the camera kinda sucks.
I bought a speed-dial app from the app store for $0.99. It takes five seconds to load, and thus pretty much destroys the whole idea of speed dial.
The idea of Pandora on my iPhone makes me all warm and fuzzy inside, especially since it works surprisingly well over EDGE.
I dig the iChat-style SMS app.
I dig the IM app.
Wow, this is a very usable interface.
NES on the iPhone is vastly underrated.
The auto-switching EDGE/WiFi support is surprisingly seamless.
Is it just me, or does the WiFi range of the iPhone seem, well, quite limited?
The Mail app in 2.0 is much easier to work with than in 1.1.4.
Cydia kicks ass. This is one very well designed and executed application.
Wow, it's really easy to crap up your phone with free apps of questionable stability.
Can someone please fix MobileTerminal? It worked fine before. kthxbye. (UPDATE: Thanks!)
I love the idea that I can run top on my phone and scp files back and forth.
Randy Newman's singing gives me hives.
Posted by Paul Venezia on July 23, 2008 05:41 PM
July 23, 2008 | Comments: (0)
Beyond the Childs case: The network as art, and why managers need to get that
On Sunday, I wrote a blog post titled "Distillation" in which I said:
"It's quite difficult to accurately convey the stress and effort required to build and maintain large complex networks to those with no real frame of reference. I've done it for years, building networks for city governments, universities, hospitals, and private companies. At some point, a network moves beyond "straightforward" complexity, and almost becomes a work of art. Whether it's a clever iBGP VPN failover for a large MPLS-based WAN, an OSPF-based ISDN dialback configuration, or a novel method of route injection through a third-party cloud, there are instances where network architects and admins need to color outside the lines to provide a needed service or measure of redundancy. It's at this point that the proverbial wheat is separated from the chaff in terms of network administration."
I've felt this way about several of the networks that I've built in the past -- they transcended the mundane and became basically a work of art. Terry Childs also felt this way, because he applied for and received a copyright in June 2007 on the configuration of the FiberWAN as technical artistry. This would back up my contention that Childs' felt that what he had created couldn't be understood or maintained by anyone else. After all, would Picasso let anyone else work on one of his paintings?
[ Follow the Terry Childs saga with InfoWorld special report: Terry Childs: Admin gone rogue. ]
More information coming to light shows just how in the dark his managers really are. In the arrest warrant, several key details are presented as evidence of malfeasance on the part of Childs. These include a detailed description of an analog modem and a DSL modem that were discovered in a network cabinet that he had built, and another analog modem attached to a desktop PC that he had installed. The description in the arrest warrant introduces these devices as evidence that Childs had added backdoors to the FiberWAN. Further on, the inspector describes an event in which Childs' pager was taken from him, and shortly thereafter, the pager went off with a message described as having come "from one of the routers on the network". This event was presented as evidence that Childs "still had administrative access to the network", and was probably a very important "fact" that helped convince the judge to sign the warrant.
The fact that this information is in the arrest warrant underscores the fact that the city truly doesn't understand anything about this case. According to Childs, one of the modems was in place to perform dialback services to provide him with emergency access to the network, and was installed following an outage event that was extended due to the lack of such access. Further, it was installed with the full knowledge of his managers. Also according to Childs, the other analog modem was hooked up to a desktop system running What's Up Gold, a network monitoring tool. This modem was used solely to send warning messages to Childs' pager when problems occurred on the network -- it's more than likely that this is the same modem that called Childs' pager after he had surrendered it to his management.
I find it deeply disturbing that both the inspector that prepared the arrest warrant and affidavit, and the "expert" brought in to help the city with this situation did not understand the actual purpose of these items, and yet are apparently still involved in the investigation of this case. I find Childs' description of these two modems and their purpose to be far more realistic than the description in the arrest warrant affidavit.
The DSL modem is slightly more curious. If it was connected to a raw pair (or a BANA circuit), where's the other end? If it was connected to an ISP, providing Internet access or a path through which to access the network, I find it hard to believe that nobody else knew about it. After all, unless Childs was paying for that circuit from his own pocket, the bills had to go somewhere, and ostensibly, somebody had to sign off on it. More information is needed on that one.
Unfortunately, it appears I might have an answer to some of my questions from my post on Sunday, namely "If the FiberWAN network is as complex as it appears to be, are there CCIE-level forensic networking experts employed or contracted by the San Francisco police department?" The answer would be that no, there aren't. The people tasked with investigating this case appear to be woefully ignorant, and lack basic understanding of how enterprise networks are constructed and maintained. This isn't necessarily a knock on the SFPD -- there's no realistic expectation that they should have this level of expertise on staff. They should, however, contract with skilled engineers that can provide that service for them.
I suppose I should be surprised, but I'm not. Now that Mayor Newsom has swept in with his superhero cape and retrieved the passwords from Childs, I do hope that at some point I'll be able to interview Terry himself, and get a more detailed version of the technical details of this case, since it certainly appears that they are being pushed aside.
Posted by Paul Venezia on July 23, 2008 09:40 AM
July 17, 2008 | Comments: (0)
So after I wrote about the San Francisco network hostage situation yesterday, I started thinking a bit more about this situation. Based on all the data I've found in the press, it certainly appears that Terry Childs changed the passwords on some number of network devices within the city's network. If there's more to this story (such as the rumors that he was a DBA and this was an Oracle sabotage job), then we move into a completely different ballpark. As it stands now, using the information publicly available, what Childs has done could be considered a juvenile prank, not an attempt to sabotage the network and cause real damage.
Again, we're assuming here, but even if we remove the specifics and make this a hypothetical case, there are many, many miles between changing the passwords on the core and edge switches and, say, dropping a dozen databases.
Unfortunately, to the public at large, there isn't much of a difference. To a normal computer user, the phrases "he maliciously altered the AAA mechanisms in the city's network to prevent access" and "he issued queries damaging to city data repositories" are basically the same thing. Of course, they're miles apart in damage done, but to folks who struggle with spyware and anti-virus tools (and sit on juries) they might as well be the same thing.
In a comment on my previous post on this issue, a user name 'der golem' summed it up nicely:
"Okay, I won't pretend I understand everything you wrote here, but there is something really alluring and provocative about tech speak."
"Alluring" might not have been the word I would have chosen, but the point is that the law deals with common crimes like theft with offense levels. There's petty larceny and grand larceny, for instance. What Childs did may actually violate any number of other laws, possibly even anti-terrorism laws since it involves a city government. If all he did was change those passwords, then it's likely that he'll be charged with crimes that don't match the events, simply because the case centers around a computer network.
Sten, another commenter on that post, had another point of view (and one that's quite common):
"Entelleghent [sic] mangers always love exaggerating the actual proportions - it's a management trick they call "risk management" - you pretend to have a huge problem; if the problem is small and solved fast - you're a genius hero; if the problem turned out to be complex - you can say 'I told you so'"
This is definitely true -- underpromising and over-delivering aren't bad things, necessarily, but for city government officials to do so publicly, while the man accused of the crimes is in jail isn't really appropriate.
Again, this is all speculation and hypotheticals since I don't have enough information on the specifics of the Childs case to come to any meaningful conclusion. I would love to have more information on this case, however. If anyone has anything more detailed than what's been released to the press, I'd love to hear it.
Given the facts known, Childs certainly did something he shouldn't have, but unless he dropped a logic bomb in the network, it's barely a bump in the road.
If you really wanted to make a point and mess up the network, there are many better ways to do it. You could place a box near the core somewhere that randomly swaps bits in the datastream. That would certainly cause problems, but would also be discovered quickly.
Better yet, write a few database queries that randomly swap numbers and letters in various database fields. If that script started out slow and then grew in scope over days and weeks, it's likely that by the time the problem was discovered, most of the backups would already be tainted, and anything using that database would be basically unusable. For a municipal government, the data loss and time required to fix that problem would be significant, to be sure. Most or all criminal and tax records would be compromised and chaos would ensue. Interestingly, I wrote about this very scenario several years ago. The cost to fix problems like that would carry a heavy pricetag, indeed. Maybe even millions of dollars.
Don't get me wrong -- I'm not defending Childs' actions in any way, shape, or form, I'm just pointing out that there's a world of difference between letting the air out of a car's tires and wiring a bomb to the ignition switch.
Posted by Paul Venezia on July 17, 2008 07:45 AM
July 09, 2008 | Comments: (0)
If you've been following my tale of Google AdSense woe, you know that just under a month ago, Google disabled my AdSense account for no apparent reason. I didn't even notice for four days, as I was traveling at the time and buried in the midst of a major network core restructuring project, but when I did notice, I filled out their dispute form. Several weeks later, I was notified that my appeal was denied, and I was still persona non grata. I filled out another dispute form following that rejection, but had resigned myself to a lifetime bereft of the opportunity to use Google's AdSense service, since they apparently have a one-strike and you're out policy.
So imagine my surprise when I received this today:
Hello,
After thoroughly re-reviewing your AdSense account, we've decided to reinstate your account. However, there will be a delay before ads start running on your website, as it may take up to 48 hours before all of our servers are informed of the change.
We've also applied a credit of $45.40 to your earnings for this month, reflecting your valid earnings prior to the account disabling. You'll be able to see your finalized earnings for this month when they're posted to your account's Payment History page during the first week of next month. [...]
Apparently, I'm now back in Google's good graces, and they've even seen fit to replace the (admittedly small) balance in my AdSense account. Sometimes, justice does prevail.
After this fiasco, I'm wondering what I can do to not run afoul of Google's fraud detection again. I'm still completely in the dark as to why this happened in the first place, so I'm not really sure how to prevent an unknown event from occurring again, but I'm thinking about adding some code to my sites to prevent the Google ads from appearing if I'm either logged into the site or visiting the site from my normal external IP address. This will obviously add some overhead to the sites, but if it provides some protection against having my account disabled again, it may be worth it.
This trip to the darker side of Google has certainly been educational. I'd never given much thought to how Google deals with click fraud in the past, but it's definitely been on my mind the past month, and it does seem to be a significant challenge. I'm guessing that Google flags large numbers of ad clicks from a single IP address or range of IPs within the same subnet, and compares them against other actions from that IP, such as logging into a Google account, be that a Google Analytics, GMail, or AdSense account. It may possibly geolocate the offending IPs and compare them to the geolocation of a known IP that has logged into the Google AdSense account in the past, and then punts to a human to make the final determination. This process would certainly not be foolproof, especially with superproxies, tor, open proxies on the Internet, and the myriad other ways that IP addresses can be masked, but those problems would seem to pale in comparison to those introduced by malware, or over-eager legitimate software, like the AVG pre-clicking debacle I discussed earlier this week.
I wonder what chaos might ensue if a virus or piece of malware bent on "clicking" every Google ad it sees became prevalent. I suppose that dealing with those problems is part of the cost to be the boss. If my adventures are any indication, it's certain that Google is quite vigilant in protecting their advertisers -- indeed, perhaps a little overzealous.
Posted by Paul Venezia on July 9, 2008 10:21 AM
July 07, 2008 | Comments: (0)
I read this article on The Register with interest this weekend. It seems that the latest version of AVG anti-virus has implemented a "feature" that clicks links on Web pages for you, scanning the resulting pages for malware. In theory this might be considered a good idea. In practice it's a terrible idea. The concept is that by clicking all those links for you (and apparently they're limited to search results, but who really knows?), AVG can better protect the user from malware-laden links. The obvious problem is that AVG uses standard browser identification strings to do this, so each click is indistinguishable from an actual user click. Thus, when using AVG 8, you litter logfiles with fake clicks, and cause bandwidth utilization to rise on sites that you aren't even visiting. Website statistics become relatively useless since they're not accurately showing user actions, and perhaps more importantly, it may be that using this tool and visiting your own site can cause clicks on your own Google ads, unbeknownst to you. Further, other users that visit your sites may be clicking on all the ads as well, even though they're not actually clicking them.
I can't yet verify that this is true, however, and AVG has apparently announced that they will stop this practice, but there are millions of installations of AVG out there that will continue performing this operation until they're updated. There may be other applications out there doing the same thing, but with a smaller install base and thus haven't received attention.
If you've been following my Google AdSense account suspension saga, you know that I have no idea why my account was disabled, since I never violated any of their rules. They won't tell me what their reasoning was behind the account suspension, nor will they disclose any information about it whatsoever.
My guess is that my account was suspended for fraudulent clicks. Presumably, that would be a person clicking their own ads to drive up their AdSense revenue. But with "features" like this example in AVG, it seems that no user intervention is required to click those ads. In fact, the user never knows it's happening. If tools like this are clicking every link and supplying a valid browser ID string, Google's AdWords model goes out the window, as there's no way to accurately determine a user-generated click versus an automated click. Thus, advertisers are paying for clicks that never reach the user, and it's entirely possible that AdSense accounts will be disabled even though the owner of that account has nothing to do with it.
I'm trying to figure out if there's a moral to this story, but I'm coming up short. Maybe "Damned if you do, damned if you don't".
Posted by Paul Venezia on July 7, 2008 10:05 AM
July 01, 2008 | Comments: (0)
A day after I posted the last update in this sorry saga, Google deigned to communicate with me:
This is obviously a form email, and has nothing of substance -- like an actual reason for the account suspension. Also, under the same account name, I'm a Google advertiser with AdWords. Apparently, I pose a significant risk to myself as well. Also, Google has been very keen on charging me for the AdWords ads, and has taken the money my sites earned before my account suspension. Talk about getting hit coming and going.Hello,
Thanks for providing us with additional information. However, after
thoroughly reviewing your account data and taking your feedback into
consideration, we've re-confirmed that your account poses a significant
risk to our advertisers. For this reason, we're unable to reinstate your
account. Thank you for your understanding.
As a reminder, if you have any questions about your account or the actions
we've taken, please do not reply to this email. You can find more
information by visiting
https://www.google.com/adsense/support/bin/answer.py?answer=57153.
Sincerely,
The Google AdSense Team
Posted by Paul Venezia on July 1, 2008 01:36 PM
June 30, 2008 | Comments: (0)
Google's response, or lack thereof
It's been 18 days since Google decided to disable my AdSense account. Having no Earthly idea why they did this, I've followed their instructions and filled out all the forms, wondered aloud about why they might have done this, and postulated that it might be stupidly simple for anyone to cause this to happen to others. It's all been for naught, I'm afraid.
Although the dispute form I filled out claimed a 48-hour response time, I've heard nothing from them. I gave them my phone number, email address, carrier pigeon route ID... everything I could think of -- but nary a whisper.
It appears that I have no recourse at this point. None. They've disabled my account permanently, as in a life-time ban -- apparently I'll never be able to use AdSense again for the rest of my days.
They've taken the money earned from my sites in the month or so since I received the only payment they ever sent me, and haven't given me any justification, asked any questions, offered any insight, or made any attempt at contacting me whatsoever.
It's a lot like a mugging.
Posted by Paul Venezia on June 30, 2008 01:32 PM
June 18, 2008 | Comments: (0)
So it's been six days since Google disabled my AdSense account without warning, and 48 hours since I filled out their response form. I've heard nothing from them in that time. Based on a comment on my original post, I started digging around to see what other folks were running into, and found adsenseaccountdisabled.org, a site that has some more information on these occurrences, and forums for those that have been hit by it. Apparently, based on the text of the email I received, I've been flagged for violating their T&C for landing pages. Of course, this is all news to me, as I haven't made any changes to any ads on any sites in four months, and prior to that, in almost a year. Again, I'm completely flummoxed as to why they would have disabled my account.
adsenseaccountdisabled.org doesn't paint a rosy picture of the future, either:
Can I rejoin AdSense?The short answer is “No, never as yourself again”. Google made it clear in the Disabled Account FAQ,
We understand your concern about the actions taken against your account. Please know that our actions are the result of careful investigation by our team of dedicated specialists, taking into account the interests of our advertisers, publishers, and users. Though you may be disappointed with our decision, we are unable to reinstate your account.
Please also note that publishers disabled for invalid click activity are not allowed any further participation in AdSense. For this reason, these publishers may not open new accounts.
So apparently, there's little to no recourse.
So let's open this up. Google, please tell me why you disabled my account. If you're feeling generous, tell me why there seems to be a growing number of people using your service that suddenly find themselves in the same boat. You must have some justification for your actions, so let's hear it. Give me at least an inkling of why you chose to do this -- some proof, some information, something to explain this. I know that I haven't done anything overt or covert, and from reading the T&C, I'm reasonably sure that I haven't violated any of the terms. If somehow there is a problem with a site I have, let me know, and I'll be more than happy to fix it.
This is all small potatoes -- these sites generate maybe $10-$20 a month in ad revenue collectively -- but it's the principle that's important. Google has every right to deny service to whomever they choose, but generally speaking, they should tell them why, and give them a fair chance to respond. Instead, they appear to be using a zero-tolerance policy coupled with mandatory capital punishment.
I'm all ears.
Posted by Paul Venezia on June 18, 2008 10:06 AM
June 16, 2008 | Comments: (0)
Like many, many folks the world over, I've taken to using Google as my one-stop shop for Web elements. Ads through AdSense, advertising through AdWords, and analytics through Google Analytics. The tools are very well designed, fast, and provide just about everything I need for the small collection of sites that collectively handle maybe a few thousand hits a day.
In fact, just two weeks ago, I received my first check from Google for $124 -- representing over a year of their ads displayed on my low-traffic sites.
So imagine my surprise when my account was disabled last Thursday. From the email:
While going through our records recently, we found that your AdSense account has posed a significant risk to our AdWords advertisers. Since keeping your account in our publisher network may financially damage our advertisers in the future, we've decided to disable your account.
Please understand that we consider this a necessary step to protect the interests of both our advertisers and our other AdSense publishers. We realize the inconvenience this may cause you, and we thank you in advance for your understanding and cooperation.
Wow. I've received a smackdown from Google. I should be chastened. The problem is that I have absolutely no idea what they're talking about.
This happened last week, apparently. I can't log into my AdSense account, and I haven't logged in for weeks, so I'm completely in the dark there. Google won't tell me what has happened, so I can only guess that this is all a big mistake, or someone's gaming me. After all, how hard would it be to write a script that transits one or more sites running Google ads on the same account, and clicks all the ad links? Not terribly challenging. What would the end result be? Google would shut down the account, apparently. Now, I have no idea if that's what happened here, but when you're erroneously accused of click fraud, you start to think about things like that.
I've filled out Google's dispute form, and we'll see how that goes. They cautioned me that it may be 48 hours or more before I get a response. Judging by when they disabled the account, the supposed malfeasance occurred while I was in the middle of a road trip. I was orchestrating major surgery on a large network -- two nights of Cisco Catalyst 6509 core replacements. Believe me, I didn't have time to bother with nonsense like this. Heck, I didn't even notice the account was disabled for four days.
Regardless of the outcome of this dispute resolution, I realize now that I'm not going to be so Google-gung-ho in the future. For the past few years, using Google's tools for Web development, advertising and monitoring has seemed to be a no-brainer. Easy, simple, fast... who wouldn't use them?
I think I'm starting to realize the answer to that question -- what happens when they decide they don't like you?
Posted by Paul Venezia on June 16, 2008 11:06 AM
April 14, 2008 | Comments: (0)
How you know if your IT department is doing it right
This one's easy. A good IT department is generally kinda bored.
When the infrastructure has been designed and built correctly and the telemetry is just right, IT doesn't have much to do except keep an eye on things and work on new projects. Sure, there are always break/fix scenarios, but those are par for the course. Unless there's a major project underway, it's the "insanely busy" IT departments that are the cause for worry, not the ones playing Nerf football. As a consultant, I've been involved in projects with hundreds of different companies and seen just about every form IT can take. This theme crosses all boundaries.
My theory of good IT is that the best network and system administrators are the laziest. When presented with a problem that will require lots of small modifications to lots of moving parts, they will always opt to write some code to automate the process. This generally takes less time than the manual effort, and the resulting code can be reused in the future. In many cases, this will require that the admin learn a new language, or at least be able to think abstractly in order to address the problem. Those that opt to do everything manually still get the job done, but with plenty of wasted effort and no long-term gains. To turn a phrase, they're generally too busy mopping the floor to turn off the faucet.
For instance, given a task to migrate from one firewall platform to another, there are many, many admins that would simply re-create all ACLs and rules in the new firewall. This is obviously error-prone and will take a long time if there are many rules. "Lazy" admins will write some perl to parse the config from the original firewall and generate valid code for the new platform. I've done this many times -- even published perl code to migrate PIX firewall rules from conduits to ACLs.
The best admins will design a system that will be more difficult and may take slightly longer to implement in the beginning, but will all but eliminate problems later. Those are the admins you're looking for.
When presented with a new project or new requirements, the better IT shops will look for open-source solutions or frameworks and adapt them to their needs rather than look for something they can buy that may not be as adaptable, but might be simpler to implement. That's not to say that commercial products are never used, but the first course of action isn't to spend lots of money, it's to research what's out there. These shops also don't generally use consultants since there's no real need for them. These are also the IT shops that tend to have the highest admin-to-user ratios, and the lowest overall cost.
Of course, there are downsides to the "lazy IT" method. The main problem is that the "lazy" approach doesn't play well with non-technical executives. The issue is that a well-designed and implemented infrastructure makes everything look easy. Modifications, additions, and tweaks become simple if the foundation is solid, though they can lead to disaster if the foundation is poor. In the right environment, major projects can be implemented with great speed and competency -- but giving the impression to those outside of the IT department the idea that anyone can do it.
Regardless of the stability and performance of the IT infrastructure, there are many that believe that unless the IT staff is red-faced and sweating, they're not doing their jobs. This can lead to staffing cuts, which then cause major problems when those that were most capable of maintaining a stable infrastructure are let go since "they weren't doing anything". New, cheaper staff are bought in and the stability and resiliency of the network infrastructure soon begins to falter. But those new admins sure seem to be working hard, running in circles trying to keep the roof from collapsing. I've seen this happen far too many times. Quite often, I've been the consultant brought in at a high hourly rate to perform CPR and stop the bloodshed.
To executives that lack a concrete grasp of how IT should work, a solid IT department needs to be presented as the best insurance policy available. After all, those insurance premiums don't do anything unless they're needed, but what happens if you stop paying them?
Posted by Paul Venezia on April 14, 2008 09:12 AM
TOP STORIES
ADDITIONAL RESOURCES
- Migrating to Vista
- CX4: Leading-Edge Midrange Storage for Virtualized Environments
- Turning Information Into A Competitive Advantage
- Your Guide to Troubleshooting Application Problems
- AT&T Article: Reinventing the Telephone with VoIP
- Coordinating Marketing and Sales Across the Entire Revenue Cycle
