04 Mar 2010

Using Open Communities for Market Research – Caveat Emptor

0 Comment

Late last week an old colleague of mine — Tamara Barber, Data Analyst at Forrester Research — contacted me and told me that she is writing a research document on how to use social media for listening in a way that adds value to the MR role. I’m looking forward to seeing the piece because since Tamara raised the issue with me, I can’t get it out of my head.

My first reaction is a loose interpretation of the title of this blog — Buyer Beware. Open social media communities like Twitter, Facebook, and blogs (just to name a few) are great places to listen (to use a popular buzz-word), develop hypotheses, and see what some folks think about your brand. However, it’s the next step that introduces the Buyer Beware part of my thinking. At this time there is little defined research logic, order, or rules in the social media world. This makes it difficult — if not impossible at this time — to guarantee the validity or reliability of these findings.

I don’t want to dismiss what happens in the social media sphere. The rise of social media has changed how consumers communicate with companies and with each other. It’s clear that there’s a lot of important knowledge being shared through tweets and blogs. The key for the research community is to define how to interpret and harness this knowledge in a way that is meaningful for business.

In most cases it would be dangerous to make business decisions over a couple of tweets or blogs. At the same time, if a company notices a significant rise in the number of tweets about product quality, it should be seen as a red flag that warrants further investigation. Likewise, a one-question poll on Facebook can easily lead a company down the wrong path due to sample bias. However, Facebook can be a great place to recruit brand advocates for a Market Research Online Community (MROC) like Waggle.

In summary — social media can be a good starting point for research. But at the same time, researchers need to be careful how they use social media to help guide decision making. Just as researchers put caveats around the ability to project around small sample sizes or qualitative research methods, there should be a caveat put around initial findings from open social media communities. This caveat should be something like, “Findings from analysis of open social media communities, additional research is needed to validate or refute finding across a projectable base.”

Christopher Kelley
Research Director