13 Apr 2010

MROCs – Qual or Quant? Let’s let a pint decide!

0 Comment

I’ve been following an interesting discussion forum started by Ray Poynter  in the NewMR Group on Linked In.  In this forum, about forty or so of us have been debating the usage of MROCs — are they qualitative or quantitative research?  The purists among us (yes, that’s me) stomp our feet and exclaim “Qualitative!” for the polls and surveys run through MROCs have not proven their reliability.  Yet, as this user group discussion suggests, are we entering a new dimension of data collection where there is some blurring of qual/quant?  A new anthropological species?  It’s really no different than what we’ve faced with traditional focus groups (how many voted for that concept?) but on a larger scale.

We at DRI have been clear to highlight that quantitative findings generated through MROCs have inherent biases built in — and note this in our reports.  We want to make sure our clients understand that there can be a difference between the quant collected in an MROC as compared to other data collection methods.  However, does this mean that these MROC findings aren’t valuable?

Companies are clearly finding value in their Voice of the Consumer work, and the quantitative data generated by an MROC is also helpful in providing useful insight on their brands.  “Should the advertisement use image A or image B?” Surely, your community members can help you with that.  Yet caution still needs to be employed — given the potential biases of community members, particularly those who have participated in the community over time.  Therefore, it simply would not be prudent for some types of quantitative studies to be fielded through a community.

For example, market segmentation or any study that needs to be projectable to a market would not be a good idea — as community members generally don’t represent the broad market.    Longitudinal work is also generally not well suited to a community.  As community members build bias over time, this would affect results in subsequent fields.

We as researchers have an obligation to educate our clients on the proper use of quantitative data collection within a community, and how to interpret the resulting data generated.  In Ray’s discussion forum, it was brought up that we should perhaps develop a new nomenclature as well as accepted use parameters for quantitative data drawn from MROCs.  I like that idea.  So, what should we call them? QualiQuant studies?  CommQuant?  Qualtilative? Surely we can think of something better.   I really liked the idea of one forum participant — that we should get together over a pint to discuss this.  Then the truly creative ideas will emerge!

Jane Mount
Executive Vice President