Tech's Bottom Line | Bill Snyder » Is counting open source code contributions really useful?

February 21, 2008 | Comments: (0)

Is counting open source code contributions really useful?

TAGS: Open Source

If you're active in the open source world, Ohloh probably knows you. The Bellevue, Wash., software company has a database of some 70,000 developers working on nearly 11,000 of the world's major open source projects.

After a year of building up its community of open source developers, Ohloh (whose backers include former Microsoft big wig Paul Maritz) has begun trying to cash in on a database that attempts to measure the productivity of open source projects -- and evaluate the developers working on them.

Is Ohloh measuring developers in a useful way?

Ohloh's developer evaluation approach raises a significant question: Is there an accurate way beyond word of mouth to measure the importance and skill of a developer?

"It's intellectually interesting, but I'm skeptical," says Savio Rodrigues, an InfoWorld contributor (and IBM project manager) active in the open source community. His concern: "I'm not sure the rankings are really useful."

Ohloh ranks contributors by measuring the number of "commits" contributed by developers, as well as "kudos" received from other developers in the community. Commits are the lines of code actually added to the code base of a particular project.

If you are interested in the Eclipse platform project, for example, a search of the database returns a list of 186 contributors. The top developer -- who uses the alias "Darin" -- has more than 8,000 commits over six years. Drilling deeper, you can see the length of time he has used various programming languages, the number of lines of code he has changed, and other information.

That could be useful, but it's not clear whether Darin wrote all his commits -- perhaps they are the work of other developers he has cleared to enter contributions into the code base, says Rodrigues.

Critics say that it's questionable whether counting lines of code -- whatever their source -- is a meaningful measure. Verbose code would count more than elegant but simple code. The kudos that Ohlo also tracks help balance the commits metric, but still.

Ohloh co-founder and CEO Scott Collison says he's aware of the issue about commits and the authorship of code. "We understand the limitations of the data, but commits are generally uniform across a project." It’s rare, he says, that the data would greatly understate or overstate someone’s real contributions.

Ohloh has taken this commits-and-kudos counting a step further, releasing an open source command-line tool and library called Ohcount that lets individuals count commits (but not kudos) themselves. Ohcount counts lines of source code and analyzes the language of the source code. Ohcount count lines of code contributed by a developer. It supports 35 languages, including C/C++, C#, Java, Javascript, Ruby, HTML and XML. Ostensibly, having this information will help users decide which open source code to adopt.

Ohloh has raised the hackles of some open source contributors in two areas. One, some argue it undercounts contributions that don't take the form of actual code, such as steering development efforts. Two, some complain that it could compromise contributors' privacy.

Ohloh's business proposition may not fly

Ultimately, if Ohloh doesn't have a viable business proposition, these controversies could be short-lived.

To get revenue, Ohloh will offer classified developer ads, broker support for open source software, and sell data subscriptions.

For example, Ohloh will use widgets to place its classified ads on technical job recruiting sites such as Monster.com and CareerBuilder.com, and split the revenue. It will sell quarterly data subscriptions for information on developer and project activity. The subscriptions will cost $25,000 to $50,000, depending on customization and other needs.

Ohlo CEO Collison says the data available on the free site now represents only a portion of the information Ohloh has collected. The company will do tailored searches for clients, and give them tools to find the most useful information in the database.

Such data mining approaches form a creative and potentially useful idea, but can it really make money?

Jay Lyman, an analyst with the 451 Group, is not so sure: "Its business model is largely untested." But he credits Ohloh for being "novel in its breadth across open source software teams and cross-project communication and collaboration," which Lyman describes as "a cross of SourceForge and Facebook," connecting the developers of different projects to each other.

Ohloh is not the first to company to mine and commercialize data from open source projects such as SourceForge, but results were "lackluster," Lyman says. Competitors include include Koders, which has classifieds for developers, and Krugle, which has an open source code search engine, although the goals of those projects are somewhat different, notes Lyman.

The use of Ohloh to comb through code and find software license information also means competition with intellectual property scanning and governance players Black Duck and Palamida, he says.

Bernard Golden, published of the Navica open source newsletter asks, "Does this mesh with the way people behave when they want to find a service provider?" He doesn't think so, arguing that it's more likely they would directly to the project, or to a user they trust.

There are reasons to be skeptical about Ohloh, but every new social force -- and open source is surely such a force -- goes through many iterations and takes many different turns before maturity. Even if individual implementations fail, the next guy will have something to learn from.

I welcome your comments, tips and suggestions. Reach me at bill_snyder@infoworld.com.

Posted by Bill Snyder on February 21, 2008 03:00 AM

RATE THIS ARTICLE: