Tuesday July 22nd 2008

"The internet is great, for we can measure directly the relationship between input & output." Yes, it's an attractive current belief but marketing buyer's beware. Reality may be an illusion.

The use of the internet, not only as a marketing communication tool but also as a marketing effectiveness service holds many attractions. (I touched on these benefits in yesterday's #15 of 25 Things about new research applications).There is no doubt that the internet brings a revolution in so many things we have experienced or hitherto been unable to entertain.

But don't throw caution to the wind. Here are just a few marketing effectiveness internet-take-great-care experiences we have found worthy of industry wide attention.

First internet research can suffer from worryingly low response rates, bias from professional respondents &  inactive panels that have to be re-built therefore destroying the continuous respondent benefit

Second, the marketing effectiveness metrics regularly suffer from that technical deficiency called "crap". (If I have to suffer any more "click through" statistics I will go mad - we would never suffer such gross simplifications in other media so why do we allow common sense to go down the drain when the magic word "internet" is mentioned?)

Third, if you dig deep you find that the absolute numbers are low and hence misleading. Predictions & projections are regularly based on false premises. And worst of all confusion reigns between absolutes and relatives.

Try out this recent "click through" experience from one of our clients and see if it rings bells for you. (Even if you disregard the "crap" metric, you can still get it wrong)

Marketing version A versus version B

• 50% A & 50% B
• All determining variables controlled
• Version A delivers CTR (Click Through Rate) 25% higher than Version B
• Agency pushes to adopt Version A and cancel Version B

But we suggested we wait for more data

After one week

• 400K impressions (200K for each version)
• Version A = 100 clicks a CTR of 0.05%
• Version B = 80 clicks a CTR of 0.04%
• A beats B by 25%
• Error rate = 15% and confidence +/- 86%

After three weeks

• 1500K impressions (750K for each version)
• Version A = 300 clicks
• Version B = 350 clicks
• Now B beats A by 17%  (smaller than the 25% margin than A led after one week)
• Error rate down to only 8% and confidence +/- 95%

So whilst after three weeks B performs RELATIVELY less well than after one week,  it performs BETTER ABSOLUTELY and with far lower error rate and within much higher confidence limits.

In all forms of marketing, and in this regard the internet is no different to any other commercial communication, you can't proceed on relative measurement for you can only bank absolutes