March 13, 2008 | Comments: (0)
I'm a big fan of TED (www.ted.com), inspired talks by the world's greatest thinkers and doers. Many of the talks have amazing visualizations. This one by Hans Rosling has been around for a while, but I like it a lot.
The TED talks are fascinating. While perhaps not strictly IT, they are so creative that after watching them I feel like I can solve any IT problem. At least they put IT perspective which is often helpful when solving problems. The talks have been great to share with my team and have made for great discussions and problem solving, IT or otherwise.
This one is a mash-up of Google Maps and Wikipedia, showing who is doing updates across the world. It's cool that it's real-time and updates. There seems to have been a critical mass a little time back between data collection and display. I'm seeing so many flash displays in addition to AJAX that are really re-defining how data is displayed. They are also surprisingly fast. I remember working on Motif and X Windows years ago. The web is now faster than local windows or at least the insanely great sites are.
We've always known logs are more interesting than monolithic text entries scrolling by in tail. Logs have always had relationships in them but you had to imagine them cause you couldn't see them. This really cool Real-time Log Visualization fixes that. It's fascinating to watch. It displays data linked up in ways not previously seen. I'm seeing this stuff a lot and I think it's changing how we view logs.
Here's another display of Web page usage using a different visual syntax. What interests me is the interconnectivity of various systems that can now been regarded on a single display.
Well, I'm going to go find my Tufte book and review.
Posted by Harper Mann on March 13, 2008 10:27 AM
November 03, 2006 | Comments: (0)
New Special Interest Group Focuses on Open Source Monitoring
Events like LinuxWorld highlight the capabilities of hot open source projects, but what of the times in-between these big gatherings? Smaller, more consistent special interest groups allow for focused, idea-driven interactions and offer a deeper look into relevant issues. Frankly, they are more effective at contributing to and preserving particular segments of the open source culture for which intricacies crucial to advancement might be overlooked in larger settings. Open source monitoring now has its own special interest group, Monitoring SIG, organized by GroundWork Open Source in conjunction with BayLISA, the Bay Area's Large Installation Systems Administrators chapter.
Monitoring the network is difficult -- the potential for capturing too much or too little information is high, and without the right tools, it's easy to flood the system or to miss crucial elements. Word of the flexibility, ease, clarity, and lower cost of open source monitoring projects has spread fast, so open source tools are establishing themselves as desirable alternatives to traditional proprietary solutions. RRDTool and MRTG allow for easier collection of the right amount of data from a higher number of network devices. Ganglia monitors different aspects of the network with little strain to the network. Cacti offers clear and sophisticated graphing and charting capabilities. The list of quality open source monitoring projects goes on and on.
A monitoring-oriented group provides a forum for thought leaders to brainstorm, storyboard and plot the future path of development,
The well-attended inaugural Monitoring SIG event in San Francisco last month included IT staff from Cisco and the Stanford Linear Accelerator Center among others.
"What's exciting is that interested people can meet and explore issues in the field itself as well as the possibilities of their own technology and the technology of others," said Peter Mui, community manager of GroundWork Open Source. "This next meeting is focused around mapping out the monitoring space. It will be a kind of 'show-and-tell' of monitoring tools that will give attendees a chance to champion or rant about monitoring projects that have had experience with."
The next Monitoring Sig event takes place in San Francisco on November 8. The invitation to attend is open to all and most importantly, food will be provided. For more information, visit BayLISA's website.
Posted by Harper Mann on November 3, 2006 11:23 AM
February 01, 2006 | Comments: (0)
How Hard is it to Troubleshoot IT Anyway?
So how hard is it to troubleshoot anyway? Last month Bay Lisa, LOPSA (league of professional system administrators), NaSPA (network and system professionals association), Splunk, SysAdmin Magazine and Usenix surveyed the attendees at Camp SysAdmin in San Francisco about their troubleshooting activities and found some things that surprised everyone.
Campers represented large Fortune 100 companies and small organizations and consultants who had responsibilities for all types of IT infrastructures and technologies including enterprise applications ( J2EE, .NET, LAMP, Web Services), email, on-demand services, VoIP, and more.
Here is what the campers had to say.
#1. Even when troubleshooting cutting edge technologies, we are still doing things the old-fashioned way. Campers were asked to name their Top 3 Troubleshooting Tools in an open ended question.
The most frequently mentioned tools were:
a. grep
b. manually scanning logs
c. tailing data sources
These were just a few of the command line utilities that were mentioned. Others included traceroute, ping, sort, awk, swatch, vim, topdump, strace, and more.
There were frequent mentions of opensource or free tools, primarily Nagios, Ethereal, and MRTG. Surprisingly, given the amount of money corporations spend on these, very few commercial tools were mentioned. Keynote, HP OpenView, NetIQ App Manage, Tivoli, and BMC PATROL were all mentioned just once each.
#2. The hardest thing about troubleshooting is that there is too much data in too many places and in too many formats.
In an open-ended question, participants were asked to identify the hardest thing about troubleshooting, 25% of respondents replied with an answer related to dealing with the huge volume of data they use for troubleshooting, the inconsistency of formatting, or the number of locations where the data resides. These data-related concerns were far and away the top answer, with lack of documentation and lack of expertise tied for number two.
#3. The most aggravating thing about troubleshooting is too much data.
In a follow-on open-ended question asking about the most aggravating aspect of troubleshooting, 30% of respondents replied similarly saying that the amount and inconsistency of data was aggravating. In this case, however, the number two answer was related to people issues, interrupting calls from management, vague user reports, or unresponsive members of other teams (developers in particular).
#4. The amount and types of data available for troubleshooting is overwhelming.
More than half (54%) of participants had data stored in more than 50 locations, with 19% having data in 1000+ locations. 60% of participants said that each of these locations generates more than 25 MB of data a day. 96% of participants said they deal with more than 3 formats of troubleshooting data, 38% said they have more than 15 formats and 23% said they deal with over 30 formats.
#5) System administrators are very proactive in their search for problems.
The good news in this survey is system administrators are generally not so overwhelmed that they can not be proactive. 60% proactively search through data either daily or weekly. 35% proactively search through data occasionally.
The sponsors are planning to run a broader survey of troubleshooting issues targeted at more specific categories at the next Camp SysAdmin. Stay tuned and I'll let everyone know when its announced. If you are interested in participating or have specific questions you'd like asked as part of the next survey email me at thebaum@splunk.com.
Posted by Michael Baum on February 1, 2006 06:53 PM
January 20, 2006 | Comments: (0)
This weekend I attended Camp SysAdmin, a get together of leading IT professionals in the Bay Area. The event was sponsored by Bay Lisa, LOPSA (league of professional system administrators), NaSPA (network and system professionals association), Splunk, SysAdmin Magazine and Usenix. What all of the sponsors of the event share in common is a commitment to improving the everyday lives of system administrators by fostering a dialog around the growing challenges facing IT professionals in an increasingly complex world.
There is no doubt, as the world becomes more on-demand, the demand for system administration is on the rise. This year more than $100B will be spent managing the worlds data centers and that number is growing at more than 2.5 times the rate of new hardware spend. The current or imminent deployment of new technologies like VOIP, Web Services, Virtualization, competing Open Source solutions just to name a few will mean solving more difficult problems and working longer hours.
I've always been surprised at the lack of dialog regarding system administration challenges, both in the popular IT press, but also among the system administration community itself. Camp SysAdmin was a useful forum to spark more conversations, connections and solutions for us all.
Campers were able to participate in two of five potential tracks discussing a wide range of system administration topics. We had Brian Aker of MySQL fame discussing troubleshooting Open Source technologies. Ethan Galstad, lead Nagios contributor, joined the campers in exploring operations collaboration in the data center, including the impact of outsourcing data center operations. Richard Whitehead, CTO of Clarus Systems, led a roundtable on IP Telephony and Eric Allman, author and Chief Science Officer at Sendmail helped campers explore all aspects of administering messaging infrastructures.
The day was filled with incredible conversation and intense exchanges of opions and ideas. Thank you to everyone who worked so hard to make the event possible and to all the campers who participated. If you were not able to attend, the sponsors will be making the recorded audio of the roundtables and the results of the camp survey available soon.
If you are interested in getting involved in Camp SysAdmin email me at thebaum@splunk.com.
Posted by Michael Baum on January 20, 2006 07:48 AM
December 14, 2005 | Comments: (0)
The IT Troubleshooter @ Interop NYC
This week the IT Troubleshooter is hanging out at the InteropNET network operations center (NOC) in New York. You can check out the NOC through one of the four live webcams. InteropNET is the the world's largest temporary network, built as a living lab of best-of-breed technologies. Fourteen vendors including AGN, APC, Avaya, Citrix, Computer Associates, Cyclades, Extreme Networks, Fluke Networks, Infoblox, Network General, Gigamon Systems, Juniper, Splunk and Quest have commited gear, software and people to run internet access for the Interop trade show.
The NOC consists of twenty crack system administrators and a handful of help desk support staff working together to set-up, manage and troubleshoot the InteropNET systems. A total of 60 people make up the complete InteropNET team.
The network installation starts on-site at 6:00 am Friday and is delivered by Monday in time for the beginning of the show. Troubleshooting in this type of just-in-time environment can be pretty arduous given the timeframes and the range of technologies to integrate. The network uses 6000 feet of fiber and 20,000 feet of CAT 5 cable. The 802.11a/b/g wireless network has to cover about 200,000 square feet. Internet access to the outside world is a 45Mb DS3 circuit. To read more about the InteropNET mission and the journey towards it's exisitence read an interview with Glenn Evans, Interop's Lead Network Engineer.
These savvy system administrators have some great stories about cutting edge troubleshooting problems and techniques. Over the next few days I'll be bringing you some of their tales. Stay tuned.
Posted by Michael Baum on December 14, 2005 09:27 AM
December 14, 2005 | Comments: (0)
The IT Troubleshooter @ Interop NYC
This week the IT Troubleshooter is hanging out at the InteropNET network operations center (NOC) in New York. You can check out the NOC through one of the four live webcams. InteropNET is the the world's largest temporary network, built as a living lab of best-of-breed technologies. Fourteen vendors including AGN, APC, Avaya, Citrix, Computer Associates, Cyclades, Extreme Networks, Fluke Networks, Infoblox, Network General, Gigamon Systems, Juniper, Splunk and Quest have commited gear, software and people to run internet access for the Interop trade show.
The NOC consists of twenty crack system administrators and a handful of help desk support staff working together to set-up, manage and troubleshoot the InteropNET systems. A total of 60 people make up the complete InteropNET team.
The network installation starts on-site at 6:00 am Friday and is delivered by Monday in time for the beginning of the show. Troubleshooting in this type of just-in-time environment can be pretty arduous given the timeframes and the range of technologies to integrate. The network uses 6000 feet of fiber and 20,000 feet of CAT 5 cable. The 802.11a/b/g wireless network has to cover about 200,000 square feet. Internet access to the outside world is a 45Mb DS3 circuit. To read more about the InteropNET mission and the journey towards it's exisitence read an interview with Glenn Evans, Interop's Lead Network Engineer.
These savvy system administrators have some great stories about cutting edge troubleshooting problems and techniques. Over the next few days I'll be bringing you some of their tales. Stay tuned.
Posted by Michael Baum on December 14, 2005 09:27 AM
December 09, 2005 | Comments: (0)
The Ultimate Troublshooting Meet-up
This week marks the 19th annual meeting of ubber systems administrators at LISA '05, the Large System Installation Administration Conference.
Talk about a troubleshooter's delight. For those of you who have never been, it really is the best gathering of bright IT minds on the planet. Five days of technical sessions, demonstrations, BOFs and constant geek talk. And of course the weather in California is really a treat given what it's like in Boston and New York this week.
This year LISA will has "Solve My Problem Boards" distributed throughout the conference site in a real world version of constant collaboration that goes on within the data center.
Posted by Michael Baum on December 9, 2005 11:54 AM
TOP STORIES
Hyperconnected users growingSteve Jobs to keynote WWDC
CSC settles kickbacks case
MS previews SMB software
What does HP-EDS really mean?
Mac Office 2008 SP1 released
HP buys EDS for $13.9 billion
Corporate IT spending slows
MS targets smartphone market
Sun to clarify JavaFX plan
ADDITIONAL RESOURCES
- Virtualization: A Step by Step Approach to Success
- Dialing up Agility with Business Transformation
- 5 Things You Need to Know About Storage Virtualization
- Is your smaller organization ready for High Availability?
- Is system maintenance doing more harm than good?
- Virtual Test Lab Automation: Manage development infrastructure
