Comments on: The Due Diligence of Sharing Social Media Facts

By: michaelatkinson

michaelatkinson — Fri, 30 Dec 2011 02:13:00 +0000

Tom, social media data is secondary research and has its own ontology, so just thinking like a primary researcher and applying AAPOR rules is just asking for trouble. Social web data is unfiltered, messy, difficult to interprent without context and massive. Sure everything that helps turn data into insights and ultimately useful intelligence needs a methodology and must have order. But, the social media consumer serves a different master; no one. There is a big difference between a survey and interpreting 1 million conversations centered around a keyword. Different technologies, layers of geo + NLP + Sentiment make social media intelligence a whole new industry. I am not surprised that traditional market research professionals have a hard time with this.

By: Tom Webster

Tom Webster — Sat, 14 Aug 2010 06:06:40 +0000

Thanks, John!

By: JohnZajaros

JohnZajaros — Wed, 11 Aug 2010 03:45:23 +0000

Yes! Yes! Yes!

I made this point the other day about The 100 Best This and The 100 Best That studies! And I too made the point regarding the recent Twitter headline. If I had used the slipshod methodology in my undergraduate stats classes, never mind while working on my doctorate, I never would have graduated!

This is precisely why there is such a huge credibility gap and lack of trust with Internet-related research and facts thrown around by the supposed authority sites.

Brilliant Tom!

By: Sean SEO Marketer

Sean SEO Marketer — Wed, 04 Aug 2010 03:53:29 +0000

Hey Tom, I had asked one of these questions on Jason's post 'The Quiet Social Networks' and I am waiting for his response. Soon after I got his response will share it here. Looks like my question would have inspired you writing this ;)

By: Matt Ridings - Techguerilla

Matt Ridings - Techguerilla — Sat, 31 Jul 2010 00:12:38 +0000

Quite right. My whole world at the moment is consumed with predicting behavior. The difference between that type of work and surveys is that we essentially apply the scientific method to testing our predictions. We may be right, we may be wrong, but in the end we *know*.

By: Tom Webster

Tom Webster — Sat, 31 Jul 2010 00:07:47 +0000

I do that too, Meg, but I don't systematically reject anything necessarily based on who paid for the research. For example, if a lobby group paid for some kind of political research – and that research appears to be otherwise sound and collected by a reputable firm – my assumption is that the data reported is probably sound. I also assume there is other data they aren't reporting :) But a sin of omission, while every bit as sinful, is a different dog from a sin of commission.

Thanks for reading, Meg!

By: Tom Webster

Tom Webster — Sat, 31 Jul 2010 00:04:45 +0000

Sometimes the only way to measure what a respondent *would* do in reaction to something, is to just go ahead and do that thing and see what they did :) Most people are crap at hypotheticals anyway.

By: Meg Fowler

Meg Fowler — Sat, 31 Jul 2010 00:03:32 +0000

My first check is always to see who commissioned the research, or who wrote press releases about it — then you can root out the bias and figure out whether or not you take it seriously.

By: Tom Webster

Tom Webster — Sat, 31 Jul 2010 00:02:43 +0000

We live and die by AAPOR, as you know, Jeffrey (our EVP, Joe Lenski, served as President of NYAAPOR last year). Yes, it would be great if research released for promotional purposes adhered to these guidelines, but it would also be great if, at the very least, mainstream media reporters would familiarize themselves with them. 95% of the issues I see in “print” have more to do with how the data was reported than with the study itself.

Thanks for your comment!

By: Matt Ridings - Techguerilla

Matt Ridings - Techguerilla — Sat, 31 Jul 2010 00:00:02 +0000

On a side note, using your “0% of twitter users would pay!” as an example. Survey questions like the aforementioned one are really only measuring what you the participant *want* to do, not what you *would* do.

e.g. – If I had worded the question: “If twitter went to a pay model and you could only use it if you paid $1 per year, would you?” you would find the vast majority would pay the $1. Does this mean you could say “Majority of people would pay for twitter!”? Sure. But it still would be meaningless unless you include the criteria by which the question was answered (in this case $1 per year or no twitter for you). What if it was $10? $100? You get my point. Some data, even when technically correct, means nothing. Love the piece.