Posts Tagged 'research'

Google Databoard

Google have recently launched Databoard, a stunning tool that lets you find all sorts of stats and data about the online world.

Alongside offering a simple way to search for specific stats, the tool also allows you to create simple yet elegant infographics that combine a variety of data points, just by clicking a few buttons:


This video below explains more, but you can try the tool out for yourself here.

Data Don’t Tell Stories

Data may not lie, but people’s selective interpretation of data can significantly change the stories they tell with data’s support.

Take this piece of analysis from the IDC, “the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets”:

Despite beating Wall Street expectations in terms of shipment volumes, Apple’s share in the worldwide smartphone operating system market posted a year-over-year decline during the second quarter of 2013 (2Q13). Meanwhile, Android and Windows Phone both managed slight increases during the same period. “The iOS decline in the second quarter aligns with the cyclicality of iPhone,” says Ramon Llamas, Research Manager with IDC’s Mobile Phone team. [from here]

Now, let’s look at the actual data:


[data source]

Of course, what the IDC notes is true – Apple’s share has declined by 340bp over the past 12 months – but the important part of the story they’ve chosen not to highlight is that Apple’s shipments still increased by 20% during the same period.

My interest here is not whether iOS is going to beat Android though; I’m merely concerned with this skewed representation of an important story.

As  with all propagandata, it’s another case of “torture numbers and they’ll tell you anything.

Always question what the numbers really say, not what the presenter chooses to highlight.

Social, Digital and Mobile in Asia

We’ve been sharing in-depth profiles of the social, digital and mobile landscapes of 24 Asian countries over on the We Are Social Singapore blog over recent weeks.

We’ve still got a few left to publish, but here’s the regional overview to whet your appetite:

Come on over to the We Are Social blog to see detailed stats on China, India, and many more countries as well.

digital, mobile and social media in india

Here’s the latest report in our BBH Data Snapshot series, this time profiling the Digital, Mobile and Social Media landscape in India.

Just like the recent China edition, this report is packed with useful data, stats, and soundbites – here are a few appetisers:

  • Mobile is more than 50% bigger than TV in India;
  • The number of people using Facebook in India is
    greater than the population of Australia;
  • 18% of India’s 12 million rural internet users walk
    more than 10km to access the web

You’ll find many more jaw-dropping stats in the full SlideShare presentation below.

As befits a report on social media, this document is designed to be shared freely, so please do pass it on to anyone you think might benefit.

And if you’d like a PDF copy, you can download one here.

All comments and feedback very much welcome!

digital, mobile, and social media in china

Here’s the latest in our BBH Asia-Pacific Data Snapshot series, with some truly stunning numbers on the digital, mobile, and social landscape in China.

You’ll find more of these snapshots, including profiles of Indonesia and The Philippines, on our SlideShare site.

facebook in south-east asia

refining vs revolution

David Armano shared a great presentation on his superb Logic + Emotion blog a few days back:

It offers plenty of food for thought – unsurprising considering that Armano “curated” it with fellow Edelman heavyweight Steve Rubel – but, ironically, the slide I found most inspiring probably wasn’t intended to be a cornerstone of the presentation:

[click to enlarge]

The source of the chart seems to be Edelman’s Trust Barometer, so I’m confident that a significant amount of quality research informed the findings.

However, instinctively, it feels like the number of times we need to repeat something depends on what we’re actually saying (or doing); some ‘messages’ will be easier to grasp than others, and some will be more easily forgotten, even if they are initially easily understood.

The whole conversation hinges on something we discussed a few months back: effective communication is about what the receiver understands, not what the sender says.

I shared the remainder of this post as a comment on David’s blog, but I’d like to extend the conversation here; it has such far-reaching implications that I believe it would benefit from as many minds as possible. That includes yours, so please do share your thoughts in the comments section below.

Social channels offer more effective ways for us to establish a shared understanding with our audiences, so they have the potential to revolutionise how we approach brand communications, not just refine the existing model.

For example, if people are struggling to grasp something, we have two options: either repeat the same message over and over again until it sticks, or keep refining the ‘message’ until people understand it.

The first option is the most practical approach in a purely broadcast communication model, because the costs involved in constantly measuring and refining a mass-media message are prohibitive. Within that model, brands often struggle to gauge whether people have understood their communications. Furthermore, by the time they find out, they’ve usually used up the entire budget and it’s infeasible to refine anything.

However, social channels offer brands a practical and cost-efficient way to share multiple messages, and determine whether people understand them – all in real-time. They allow us to communicate in a context that is more akin to face-to-face conversation: we can constantly refine and tweak our ‘message’ until we’re sure it has sunk in. More excitingly, such conversations can help us to identify whether our communications are even addressing our audience’s most important issues. When it comes to communication, what we hear is often more valuable than what we say.

In this second model, our communications challenge shifts from trying to identify an optimum number of repetitions of just one message, to identifying better ways of sharing what matters to people.

That may sound like stating the obvious, but I get the sense that many marketers still see social channels from a broadcast perspective – we are trying to adapt them to our existing model, rather than adapting the model to the take advantage of these new opportunities.

What do you think? Do social channels really allow brands to engage in dynamic conversation? Will repetition still play a key role in this new approach to brand communications? Share your thoughts here.

groundhog marketing


Jon alerted me to a disturbing report on eMarketer yesterday.

Apparently, most marketers still think that one-night stands are better than long-term relationships:

emarketer mktg success

According to the accompanying article,

“Nearly three-quarters of companies have guidelines to measure the success of their marketing programs, and for one-half such measurements are a requirement for obtaining marketing funding.”

This statement implies that more than 25% of the marketers polled don’t measure marketing success in any way.

I’m shocked.

To make matters worse, the majority of those who do measure their marketing see new acquisitions as the ‘best’ indication of success.

I don’t understand; it’s no secret that nurturing existing customers delivers better ROI than trying to attract new ones all the time.

I recognise that growing the customer base is important, but surely their retention, or even satisfaction, gives a better indication of marketing success?

Oddly, in the same article, the report’s author notes:

“Marketers have been aware of the effectiveness of building relationships and trust with content since long before the Internet…”

So why are those same marketers ignoring opportunities to build relationships, and instead resorting to transaction-based, ‘groundhog’ marketing?

Perhaps it’s not just ad measurement that needs to evolve; perhaps it’s all marketing measurement.

Then again, maybe the real problem is a lack of understanding of fundamental marketing principles.

Maybe it’s time to go back to basics…

measures of success

fixing marketing measurement

John posted a great comment in response to a recent post:

“Measurement of [TV’s] effectiveness is based on samples, not actual viewers, and often the best data you get about the audience is generalized demographic / psycho-graphic information.”

It’s no secret that I have strong opinions when it comes to measurement and research.

My concern is simple: I don’t believe we’re measuring the right things.

It’s time we changed that.

This post outlines an alternative approach to ad measurement, but it still needs some tweaking, so I’d really appreciate your suggestions on how we might improve it.

Let’s begin with some context…

The role of measurement

As early as the mid-1800s, John Wanamaker remarked:

wanamaker ad dollars quote

His fears have been echoed by marketers ever since, and we continue to invest huge sums trying to identify which of our dollars are wasted.

However, this focus on wastage means we’ve been missing the forest for the trees; in order to understand how hard investments are working, we first need to understand whether campaigns are delivering on our objectives.

Advertising success is not just about efficiency; we also need to measure its effectiveness.

To examine these factors in context, we first need to understand  the objectives we hope to address with advertising – in other words, what do we want our brand communications to achieve?

Why do we communicate?

At a fundamental level, communication serves a very simple purpose:

To create a shared understanding between two or more people

It follows that the purpose of brand communications is:

To create a shared understanding between a brand and the people it wishes to influence.

So, in order to measure advertising’s effectiveness, we simply need to determine whether the audience has understood what the brand intended.

To measure the campaign’s efficiency, we need to compare the proportion of the audience that correctly understood  the message with the different campaign elements they’ve experienced, and the cost of those different elements.

In light of the above definitions, it seems logical that both measures centre on the audience’s level of understanding.

So why do we consistently resort to metrics that have so little to do with what really matters?

Flawed measures

Reach (as media agencies currently use the term) is simply a projection of potential audience size. This is the concern that John highlighted in his comment above: reach doesn’t tell you whether anyone actually witnessed your communications, and it gives no indication of whether those who did witness them understood anything.

Meanwhile, frequency is equally limited in its value, informing us of little more than the number of opportunities each individual had to witness the campaign (but again, not telling us if they actually did see things that many times). I’ve talked about frequency’s limitations before, so I’ll avoid going into any more detail here.

The problem with these metrics is that they equate volume with success. However, the more you shout at people, the more they’ll try to ignore you.

shouting ignorance

Even ‘brand health’ metrics are compromised when it comes to determining advertising’s impact, because they tend to look at a brand’s performance in aggregate, rather than just the performance of its communications. This macro view means that we cannot determine the campaign’s influence ceteris paribus, and consequently, we cannot ascribe changes in brand health scores solely to advertising.

So what can we do to improve advertising measurement?

Start out right

A simple improvement would be to focus on objectives.

It’s vital that brands and their agency partners develop campaigns around what the brand wants its audiences to understand – although it’s a widely misused term, I’ll refer to this as ‘the message’.

Only once we’ve agreed this message can we be sure that we’re developing the most efficient and effective communications.

An example might help to put this in context.

A popular brand of face cream wants to grow revenues amongst existing users in Thailand. Traditional ‘brand health’ research shows that the brand is well liked and respected, but that its use is sporadic, with medium and light users applying the product just a few times each month.

Face-to-face conversations with these consumers reveal that they think it does a great job of rehydrating their skin, but the humid climate in Thailand means that this benefit is only relevant on a few days each month.

The brand recognises that it needs to modify people’s perceptions to help them see the brand as an everyday cosmetic, rather than simply as a functional moisturiser. In other words, it needs to tell them that it offers a bonus, and not just a remedy:

benefit baseline

Further research reveals that this audience uses moisturisers to “bring their skin back to life.” When their skin is dry, they think that it looks “flat” and “tired”, but when it’s properly hydrated, they believe it exudes a “radiant glow.” A closer look at the research indicates that this radiance is not only a critical category driver, but that the brand already scores highly on this attribute.

Brand management believes that focusing the audience’s attention on this motivating benefit will encourage them to use the brand more often. They brief a campaign that drives the perception that “using BrandX every day gives my skin a radiant glow.”

This clear statement of what the brand wants people to understand ensures that everybody works towards the same objective. Rather than simply trying to ‘raise awareness’, each agency knows what it must communicate, and can therefore optimise its approach to focus on this outcome.

Measure what matters

In order to assess the campaign’s contribution, the brand needs to measure how strongly the audience associates this statement with the brand. The research must begin well before any communication takes places, enabling the brand to identify baseline scores against which it can compare the scores obtained during and after the campaign.

It’s important to note that research should only canvass the relevant audience. Communications should always be tailored to appeal specifically to the people the brand wants to influence, so there’s little point in measuring their impact on other people.

I’ve mentioned before that I’m not a great believer in advertising pre-testing, but I recognise that many people feel more comfortable using it. If you do use it, make sure that it focuses on assessing the campaign’s ability to influence this core attitude.

Once the campaign launches, we need to measure its impact on the relevant attitude score(s).

Measuring up

The first thing we must identify is whether respondents have witnessed any of the relevant communications.

Historically, brands have placed great importance on metrics such as ‘top-of-mind awareness’ and ‘unaided recall’. However, the audience’s ability to remember the campaign doesn’t say much about whether or not they understood anything.

I’d argue that prompted research is much more indicative. For a start, significant differences between projected reach and prompted awareness suggest an area the brand will want to research further. It also provides:

A reliable indication of actual campaign reach;
A ‘control group’ of respondents who claim not to have witnessed any aspect of the campaign, whose scores we can compare to those of people who claim they have seen at least one part of it.

The research continues by showing each respondent a selection of activity from the campaign, and asking them to identify all that they’ve witnessed. This can be as simple as showing them visuals or playing them snippets of audio from different executions, and asking them to confirm whether or not they’ve seen or heard these activities before.

Your research agencies will be best placed to recommend the maximum number of examples you can show, but make sure that you include all the channels included in the campaign mix at some point in research. Don’t forget to include things like events and sponsorship if they’ve been part of the plan.

Examining scores across people who have seen a different combination of channels allows us to determine the cumulative effect of different activities. Furthermore, by comparing these results to the scores of those who have only seen a few activities, we can begin to infer the impact of specific channels and creative executions.


The next step is to assess whether the respondent has understood what the brand intended. The best way to measure this is to ask it as an open-ended question, such as “what did you understand after seeing this advert?

shared understanding (2)

However, this is often impractical due to respondents’ levels of commitment and involvement, and the need for researchers to record exactly what was said.

As an alternative, you can present respondents with a variety of statements about the brand and the campaign, and allow them to indicate the strength to which they agree with them. It’s best to offer a few variations on what the brand actually intended, along with some other statements that have little to do with the intended message, to allow for more reliable findings.

Once researchers have gathered all the responses, the final step involves interpreting the results. This begins by comparing pre-campaign scores, ‘control group’ scores, and the scores of people who’ve experienced different aspects of the campaign.

Following these steps with similar research a few months after the campaign has finished allows you to assess whether the campaign has permanently modified perceptions, or simply influenced shorter-term attitudes.

And that’s pretty much it: a simple, practical, but powerful approach to improving advertising measurement.

The barriers

So what’s holding us back?

Here are the most common oppositions I encounter when proposing this approach:

“It’s too difficult”
“It would cost too much”

The first point is moot; I’d argue that this process is a lot simpler than the one research companies already use when tracking brand health.

The second point relates to focus: yes, if we ran this process in addition to current measurement, it would add significant cost. However, this approach tells us almost everything we need to know about our advertising, so why would we continue with that other measurement?

A start, not a solution

I don’t pretend that this process is a panacea, and as with all aspects of marketing, you’ll need to adapt it to the specifics of your brand and its context.

However, provided you canvass a sufficient proportion of your audience with focused and relevant questions, this approach should deliver results that are far more informative than most current practices.

What do you think? Which parts need tweaking? What could be added or removed to make it better?

I’d really value your thoughts and comments – please feel free to share them, along with any questions, in the comments section below.

You may also find the following posts useful:

8 steps to better communications
Anjali shared some great thoughts on a similar subject in this recent post

ethics in advertising – poll results

ethics in advertising poll results

The fine line poll has been open a few weeks now, so I thought it was a good time to share some initial results.

Fascinatingly, one in five participants would refuse to promote a religion – more than any other category.

I was intrigued by this ranking.

It’s perhaps less surprising to see tobacco in second position, but the presence of political parties in third place was another finding that tickled me.

The majority of participants who voted “other” highlighted sexual exploitation of some description.

Meanwhile, one in ten participants were comfortable promoting anything for the right price – fewer than I initially expected.

So, an interesting set of results.

And while most are in line with my expectations, religion’s place at the top of the heap still fascinates me.

Being the curious person that I am, my response is: why?

What is it that advertisers find so morally repugnant about promoting a religion?

Is it because it’s beneath them?

Is it because they can’t think of anything that differentiates one religion from another?

Is there some kind of ethical barrier, and if so, why is the promotion of a brand that purports to save our souls more heinous than doing so for brands that have been shown to kill us?

I’d really like to know your perspectives – the comments section is open for your thoughts.

* The “Other” option allows participants to specify what they wouldn’t advertise. Thanks very much to everyone who has taken part so far. If you didn’t get the chance to vote, please feel free to take part in the poll now


Other Distractions