Measuring Community Success: Two Contrarian KPIs for Web 2.0

We've seen a shift recently in how some of the major players in Web measurement are evaluating success. Nielsen/Netratings switched from counting page views to counting time spent--to the derision of some in the analytics community.

Time spent has problems, it's true. But I haven't heard a lot of great suggestions about how you do measure success on participatory web sites, social media sites, or online communities. The truth is, the devil's in the details, there is no silver bullet, and most of all, it depends what you're trying to accomplish.

Yes, everything in web analysis should track back to your goals. Repeat that ten times.

Nonetheless, I will offer up a couple ideas about how measurement in online community is different than on brochure sites, and how standard measures, to the extent there are any, fall short in typical online communities.

My suggestion is basically that we need to think differently about success, and abandon some of the old-school, quantity-focused measures, like number of posts, number of comments, and number of registered users, for an approach that emphasizes quality. Here are two examples of what I mean:

 

Return Visitor Sign-In, Not New User Registrations

Sign-in is more important than return visit--it's even more important than registration. Participants in online communities typically register to access content they can't access otherwise, to take an action they can't take without registering, or to streamline a future process (e.g. saving credit card information).

So, participants typically register for the promise of a reward. Sign-in is different. When a registered user returns to the site and authenticates, you are a little closer to knowing you gave that person something valuable. In other words, when the cousins get up for more, you can be pretty sure they like your mashed potatoes. If you want to gauge the success of your meal, measure the number of second helpings, not the number of people at the table.

 

Qualified Content Consumption, Not Number of Posts

I've written in some detail about the value of so-called "passive" participation (I personally wouldn't call information-seeking online a "passive" activity). In short, content consumption tends to be greatly undervalued relative to content creation. While content creation introduces the potential for value, content consumption represents the realization of that value. Tracking content consumption, then, helps you learn not just how much stuff you have, but, more importantly, how good is your stuff.

Part of the problem with quantity measures in social media is that all content is not created equal. The best content, while it's created only once, creates value many times over, while the worst content can actually be detrimental to the participant experience. So the number of content objects on your site tells you much less than does the number of times those objects are consumed--in total, on average, and individually.

But how do you know content consumption when you see it? Content detail page views can be misleading, but I differ from those who'll tell you it's all about time on site in Web 2.0. Instead, count page views qualified by either a time-based or action-based threshold. 15 seconds on a detail page is a decent place to start, but it doesn't account for the effect of tabbed browsing, which can skew time-based measures upward.

Here are some examples of action-based measures for qualified content consumption:

• file downloads
• play embedded media
• copy embed code
• forward via email
• scroll to bottom of page
• print
• favorite

There are many more, and there's a lot more to say on this topic. Drop me a note in the comments with your thoughts, ideas, and questions.

Comments

[Daniel Rae]

Hi,

I'd like to track content consumption for a blog page that doesn't have a print button etc. I like the idea of scroll to bottom of page, but how is this measured? It's not something I've seen in Google Analytics.

Cheers!

[Ryan]

Hi Daniel:

I bounced your question off one the analytics folks I work with, and here's the response I got:

I think you could do this in GA but it would not be a simple implementation. What you would need to do is write your own custom javacript function that detects when your browser scrollbar reaches the bottom (max Y height limit). You would then need to put this function in your google js file so that when that event occurs, it sends the data to your google analytics account. Then, within your reporting interface, you would need to setup a custom variable so that when it receives the completed browser scrolling event, it tracks it in the variable.

So there you have it. Basically, it sounds kinda complicated and difficult (though it's doable). Before running off and digging in to that effort, though, I'd encourage you to back up and look closely at the goal you're measuring against. Consider what user actions, alone or in combination, communicate a valuation of the content you're publishing relative to that goal, and decide whether there's a simpler way to measure.

The upshot is, in social media, understand quality, and let quantity follow.

Good luck! And thanks for the great question. --Ryan