Recently someone posted on Twitter that 80% of Gen-Y thinks Advertising is truthful. And when I see something that does not make sense I investigate. So I did. I came across something so grossly distortional it truly upset me. It also misleads readers and goes to the heart of something. Garbage in Garbage out.
Why do so many new product launches fail? Because of poor research. The reasons are a myriad. From biased questions, biased focus groups, biased or lying participants, and how the data get's reported.
I have railed before at poor reporting of news, like Mashable who is almost so deceptive or inept that they are not a news source for me.
But news is one thing. Actual research is another. Read the headline of the next link. Then the first line of the write up. Completely different meanings and conclusions.
In this Harris Interactive study on whether people think Advertising is Truthful presented by Marketing Charts they did the double whammy. First the questions suck. Seriously. Expand the charts. What a horrible choice of questions. Secondly the write up by Marketing Charts is 100% wrong. Not just inaccurate. But WRONG!
The data shows that most people do not trust advertising what so ever. It also shows when it comes to regulating advertising they trust NO ONE!
Yet when you read the write up it sounds like most people find Advertising Truthful and that they don't trust the Government to regulate it. Insert the sound of Kyles Mom:
Two glaring issues I have with the data since I am tossing the write up into the trash.
They should of asked Do you trust Advertising to be Truthful: Yes or No?
Then maybe a more detailed question about that. When I see the word 'Sometimes' in a question what does that mean. Sometimes being truthful would mean from 1% to 50% of the time. Most of the time could mean 51% to 99%. HUGE SPREADS. And thus the data is really a statement that people are smart and they do not trust Advertising or anyone who will be given the task of Regulating it.See the Ad Contrarian's write up of a Nielsen Study as another example of poor research: LINK
I spent several years on the research side of things before making the transition to jaded and cynical creative director, and from direct experience, everything you've written is spot on. Research is a tool. Properly conducted and compiled, it gives you the lay of the land, and with the analysis of someone who knows how to navigate the data, it can yield all kinds of juicy insights. But all to often, it's a gigantic sloppy mess, with questions written and data compiled by people who know the answers they want beforehand.
ReplyDeleteIt's not any better on the receiving end, either. I can't tell you the number of times I got requests for research showing X, even when every possible data point indicated that X was completely dead wrong.
Research, and the analysis of research, requires objectivity, intellectual honesty, and a willingness to be confronted with findings that contradict your assumptions and make you look like an idiot (those contradictions are usually where the best insights come from). Sadly, these qualities are rare, at best.