Free Newsletters

   All InfoWorld Newsletters
Open Sources | Rodrigues & Urlocker » The open source download heist

March 27, 2007 | Comments: (0) | TrackBacks: (13)

The open source download heist

Javier Soltero (CEO, Hyperic) has a great post taking the download bean counters to task for...counting downloads. Actually, his qualm is not with those who tout their download counts so much as those who a) inflate their download counts (so that they can flaunt them), b) assume that download counts substitute for community/value/revenues, c) both "a" and "b."

The correct answer, of course, is "c."

This means that while [an open source project's download counts are] somewhat indicative, and mildly amusing, tracking download counts is in no way a real diagnostic of a project’s success. When there’s money involved (as there always is in Commercial Open Source companies), there’s a built-in incentive to use raw download metrics as evidence that one project is ‘taking off’.
Javier then goes on to suggest a few ways that companies inflate their download statistics, disparaging both the outcome and the purpose behind doing it. (Btw, at Alfresco we felt 'less than' for a long time because we didn't have any tricky ways to game the numbers, but now that Javier has shown us how to do it...watch our downloads skyrocket!!! :-)

He therefore suggests - in the same way that grocery stores do for products with a "$0.00 per ounce" metric - that someone invent a way to measure real utility (which he suggests is best measured with community data points, rather than download numbers) in a project:

This is what the open source community needs. Price per ounce for downloads. A normalized download counter. The first step will be to defraud the numbers as they stand. The second, and more important one, is to start measuring the heartbeat of the community.
I agree. I'd actually suggest that a useful (though hidden to all but those who work for the companies in question) metric would be dollars-per-download. Or, in other words, how much revenue does your company have compared to downloads. It's not a good way of measuring community, but it is a very good way of measuring how much the buying community values one's software. I know from some experience that you don't need to have millions of downloads to have millions of dollars in sales.

In fact, as I wrote last year about JBoss, that company's downloads remained relatively flat while sales rose dramatically. It wasn't that downloads mattered more, but that the company figured out how to wring more dollar value out of them.

Isn't that what companies should be measured on? Money? I don't think shareholders care very much how many downloads one has if they're not feeding little Nathan....

Posted by Matt Asay on March 27, 2007 08:35 PM


RATE THIS ARTICLE:





 

  •  
  • COMMENTS




Heh, the OpenNMS project was recently on Slashdot and it took out our five year old server. I moved the website to a new, shiny server and since we use Mediawiki we added a Squid front end to improve the speed.

However, this totally pooches our "page view" counter on Slashdot since the image that Slashdot uses to count such things is cached.

I'd rather our community have a faster wiki than to have a higher ranking on Sourceforge, so I guess our project sucks now. (grin)

Posted by: Tarus Balog at March 28, 2007 06:38 AM

Downloads are like crack, addictive and ultimatly not very satisfyingly. The real metric is how well you convert those community members, downloaders and lurkers into paying customers. We've recently begun our "early adopters" program for our Enomalism Virtual Server management dashboard (LGPL). Our initial sales have been very strong including some major fortune 500 companies. We've just about recouped all our initial development costs, which in my opinion is the clearest quantitative measurement that our open source product is succeeding.

Posted by: Reuven Cohen at March 28, 2007 07:47 AM

Thanks for raising this in InfoWorld, Matt.

Tarus brings up another point...download numbers are one thing, but Sourceforge rankings are entirely different. Most projects don't use all the functionality of Sourceforge.net, but the project rankings get scored as if you do. If you reset something like Tarus did, or don't use their bug tracker or mirror your subversion checkins, then your project, while performing relatively well in downloads is stuck way down the dogpile. Course, the over-eager project manager who actually has time on his hands for trivial games like this, could spend the time manually dumping in the occasional orphaned, out of process items, and suddenly their project is in the double digits.

Yet another way to put lipstick on a pig.

Hyperic has discussed this with Sourceforge before about this, and I know they are very actively trying to figure out a way to defraud this process in a fair manner. Until then though, the games will go on...

I agree with you, Matt. Customers are king. Usage traction balanced with revenues reflecting customer value is how all open source projects should be measured.

Posted by: Javier Soltero at March 28, 2007 09:32 AM

Traffic (e.g. pageviews) and downloads do not translate directly to revenue, but there are ways on coverting one into the other. Returning visitors and dependent (or loyal) users of your S/W give you influence. Influence, unlike traffic, can be assigned monetary value. You can buy influence... well, unless it's the Linux Foundation. ;-)

Posted by: Roy Schestowitz at March 28, 2007 04:31 PM

I think the Fedora Project (Red Hat's community linux) has come up with a pretty good solution for this problem. Don't count downloads, count how many people connect to the update server. That's a much better indication of how many people are actually using it.

Posted by: ERM at March 29, 2007 05:29 AM

Ahh. The joy of Analytics.

What is a "download"? Grabbing a tarball from the home web site? apt-get'ing from a Debian mirror? Madriva rawhide? Dag's RPM respository? SunFreeware Solaris PKG?
What about the various "unknown" mirrors - it's Free software freely distributed isn't it?

Is there a 1 to 1 relationship between downloads and users? In my case with my principal project, one user was planning on providing it to multiple hundreds to thousands of end clients - who in turn could be 1 to many actual users.
1 download == Many.

Downloads is a measure. It's not a Key Performance Indicator (KPI) or it shouldn't be if it is. What is "your" KPI will vary from project to project.

The danger with any metric is when folk misinterpret what it means or tells you. They grant it more ... credit than what it actually represents.
1 download == 1 user? Rubbish. 1 download == 1 download from server ABC that wasn't cached by any intervening doodahs; No guarantee they even use the software.

Maybe they have a tarball fetish.... ;-)


FWIW, Eric T Peterson (http://blog.webanalyticsdemystified.com/weblog) , one of the leading lights in the Web Analytics industry has been filling out the concept of measuring "engagement". Which sounds exactly like what Javier is wanting. Sadly the tool he uses to do this is expensive by even the standards of the high end of the Web Analytics industry. :-)

But the concepts are worth watching, and maybe one day we will have an agreed method and tool for the FOSS world. One day.

Steve

Posted by: Steve McInerney at March 29, 2007 08:20 PM

Microsoft Mini Spotlight
  • Get Started
  • Port 25 Blogs
  • OSS News
  • Join a Project

{Open Source} Heroes Happen Here

Start today and order your own Hero Hack Pack – which includes Getting Started with Open Source, Windows Server 2008 and Visual Studio 2008 Trial. Each pack is a chance to win a free pass to OSCON 2008.







Technology White Papers

 

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert

Find out when the latest white paper is available:
 
 
» BUY A LINK NOW

Sponsored Technology Links