Firstly: I am not a fully trained statistician or market researcher. Although I passionately deal with numbers and also immerse myself in statistics books in my free time, the more you know, the more you know what you don’t know. I didn’t acquire this wisdom by spoon-feeding, and I’m always grateful when experts provide feedback. However, I believe that any reasonably clear-thinking person can understand without a basic statistics course when data has not been collected sensibly or false conclusions are being drawn.
Are print ads more persuasive than those on social media?
I became aware of this thesis through the postings of Andre Alpar and Karl Kratz. It comes from Best4Planning, here is the whole study “Quality is more important than likes”. Unlike the very esteemed colleagues Andre and Karl, I don’t think best4planning is a satire site, but the b4p derivation from the data is at least courageous. If you disregard the hair growth products in the classifieds of a newspaper, the typical “lose 10 kilos in 2 days” or “millionaires don’t want to see this video” advertisement on Facebook & Co does not speak for social. So at least subjectively, I would confirm the thesis. The entry barrier for advertising on the Internet is wonderfully low, in print it costs a lot of money upfront. Whether the automotive advertising for clean diesel in print is still perceived as credible today remains to be seen, but any idiot can offer a product in online media for little money. Therefore, I would not dispute the data from b4p that advertising in print is more credible.
The situation is different with the interpretation. Is credible the same as a stimulating purchase? First of all, I think this is a misinterpretation of the data. But in my opinion, the problem lies in the graphic, because the credibility feature is mentioned as the source: There is also the characteristic “stimulating to buy”:
So it makes sense. I don’t know how they got to 28.1% now, since I understand print to mean both daily newspapers and magazines etc, but maybe they simply calculated (34+22)/2 and thus came up with the number (internally they will probably have decimal places). However, the explanation in the graph is unfortunate. Because that’s how you think (at least I think) that trustworthy is the characteristic that stimulates buying.
Let’s go one step further. The data is spread across all age groups, what does it look like if we divide by age group?
Wow. The demographic factor seems to have struck well. If I haven’t done everything wrong now, then the statement is true that older people find advertising in print more trustworthy and more incisive, younger people more likely to find social media. As already commented by Karl on FB, 65% of the respondents are at least 40 years old, 49% at least 50, because under 14 is not surveyed. So we have a surplus of older people who are also more likely to use print. Now, of course, the under-14s are not so strongly represented in social and not in print, but the age development in Germany does the rest to ensure that the number may be correct. We hardcore onliners won’t like that, dear Andre, dear Karl, but outside our bubble there are actually people who don’t have a telepathic connection to the net.
If you take a closer look: 28.1% of those surveyed find advertising in print credible, according to the graphic. 28.1%. That is less than a third. In the end, you could also say, yes, advertising in print is not credible for the majority of respondents, but even less so in social advertising!
Conclusion: Numbers ok, interpretation difficult if you don’t see it differentiated. The excitement around the statement is understandable, because it is true, but an average of the total population is still an average, and it is usually suboptimal because it says nothing about the distribution.
Is best4planning legit?
Now you can also do a lot of crap with b4p. Especially if you don’t understand how the data is collected. But we also have that at Google Trends. Or at Similar Web. A fool with a tool is a fool. Point. Of course, we can make it easy for ourselves and say that all market research is a lie anyway, but that wouldn’t be fair, because there are enough market researchers who put a hell of a lot of effort into it.
But how is b4p’s data created? In fact, 30,121 people were surveyed for this study, which is many more people than those involved in the ratings, to whom I had once belonged, and it is a good basis. There were two waves of surveys, all of which can be read here. I see no reason to doubt that just because publishers are the clients. Because you can also rummage around in the data in such a way that some things don’t really look chic for print.
The technical measurement took place with 10,231 participants in the GfK Crossmedia Link Panel, who were also sent an apparently somewhat shortened form of the questionnaire. Then these data from the panel and the interviews were “superimposed”, so to speak. This is a common procedure. However, this also has consequences. Example: I build a target group in b4p of people over 50 who are self-employed in a construction company, etc. n=30,121 then quickly becomes n=12, who were surveyed. That’s difficult, but well. But if I then look at media use, it may be that it comes from the third of the respondents of the GfK panel, so I would have to calculate correctly 12:3 and would be at 4. Unfortunately, the b4p page says nothing about such cases, does not give any help. But I wouldn’t invest my advertising millions on the basis of such data if n is so small. Does this mean that b4p is dubious? No. The problem is in front of the screen, because the small number in the upper left corner, which stands for the number of cases, is very small and is therefore often overlooked. What would also be the alternative? Nothing at all?
Where I actually have a stomachache, these are the questions, at least the ones that are published, and I have only found one: “Now think about the days from Monday to Saturday. In general, how many of those 6 business days do you watch shows on TV between 6:00 a.m. and 9:00 a.m.? Please also remember that Saturday is often different from the other working days”. Because of social desirability: Who with enough brain cells voluntarily admits that they are already bombarding themselves with breakfast TV in the morning? But I hope that such questions will be backed up by further control questions.
Conclusion: b4p is a good place to go for many concerns. It is important to back up the data with other data sources and common sense.
What is the age distribution of Germans on Pinterest?
Are there alternatives to b4p? Few. Some swear by Statista, but the unreflected use of surveys kills a statistics unicorn as well as b4p (that’s the reason why you don’t see unicorns anymore). Statista is of course an extremely cool site with great data, but it’s also worth taking a look at the details of who actually did the study. For example, this study from February 2018 that tells us that 14% of people aged 60 and over are on Pinterest. Also 14% of 50-59 year olds. n for both groups around 600. Sounds great for Pinterest. Unfortunately, you have to pay for Statista if you want the source, but someone kindly did that for me. The whole study comes from the Faktenkontor. Survey by Toluna. Toluna? What is that? A website where you get something for taking part in surveys. Humph. So we didn’t randomly select people from the at least 60-year-olds we surveyed, but simply took those who are already online anyway and then also know this portal, etc. It’s just a shame that only 55% of the over-65s are on the Internet at all, according to destatis. So we surveyed an already online-savvy group whose probability of being on Pinterest is correspondingly higher. I would at least be very cautious about recommending Pinterest as a target group for pensioners and using this figure of 14% somewhere.
Conclusion: Take a close look at how the data was collected.
It’s not the data that is the problem, it’s you
The problem is not the data, as long as you understand where it comes from and how it was collected. But sometimes you don’t want to know that exactly, because you have an opinion and prove it by choosing only the data that confirms your own opinion (and I’m not exempt from that). This is called confirmation bias. I have also written about this elsewhere. Or you don’t have the time to check data. Or sometimes not the will. You only see a number, have your own opinion and then shoot off. Sometimes I wish for more differentiation. Just by the way. But simple answers are always easier to communicate.
By the way, a nice book I’m reading right now: Thomas Bausch – Random Sampling Methods in Market Research. It’s available used for less than 5€ at Amazon.