Is media analysis really just lies, damned lies and statistics? - a Cubit thought piece

Like all effective measurement systems, media-centric metrics need to be tailored to and aligned with your circumstances and your requirements by people with the skill and the tools to do so.

About a week ago I saw a comment on a LinkedIn forum, along the lines that media analysis never tells the “whole story”, and is therefore so flawed as to be useless to PR folk and the world in general.  Someone else agreed, suggesting that the most effective measure a PR professional can have is his/her “gut”, and that every other measure of success is useless.

I may be overstating their positions a little, but you get the idea.

These comments suggests several things to me:

1. These folk may have had some very bad experiences with measurement systems,
2. They have probably not spent a whole lot of time considering what the “whole story” or “complete picture” of any situation might actually entail, and how real-world measurement systems cope with that,
3. There is a need for colleges to provide those studying to become professional communicators with a more adequate introduction to the broader concepts of how measurement systems work and evolve.

_______________________________

In regard to the first point – they may have had some bad experiences with measurement systems.  That would not shock me at all. 

There are some pathetic excuses for media analysis systems out in the marketplace -- that are practically guaranteed to give you a bad experience. 

But there are also some absolutely lousy golfers out in the world as well, and their existence doesn’t necessarily mean that golf per se should be considered bad or useless.  In this case, it’s about how well something is done, rather than the thing itself. 

It’s true that there is some genuinely useless media analysis on offer around the world – but that simply does not equate to the proposition that all media analysis shares those characteristics.

Second point.  What’s a “complete picture” or a “whole story” anyway?

Let’s start this with an example. 
You’re driving along and you look at the speedometer.  It says you’re doing 95.  That’s an important measure, but will knowing your current groundspeed ever tell the “whole story” of what’s going on around you or in the car?  Of course not – and you don’t expect it to.

You know that a more complete picture would include many more data: the speed limit for the stretch of road you’re using, the prevailing weather, visibility, and traffic conditions, as well as the state of your car and its driver at that point in time.  You also know that all that information just wouldn’t fit on your dashboard, and if it did you wouldn’t be able read and absorb it without crashing your car. Which defeats the purpose of giving you helpful little bits of measurement that guide your actions - like your current speed.

The bottom line is, the “complete picture” or a “whole story” is almost never available to us.  And even if there were, a single metric wouldn’t paint it or tell it. 

The world is just too complex for that. 

Most times, measurement systems are there to present just the stuff you really need to know right now – maybe in order to do something better or safer, or in the case of a well-designed media analysis regime, to make better business decisions in respect of your media positioning.  And often, that’s enough.  It’s certainly a whole lot better than not knowing anything. 

Point three.  Here’s a short introduction to the realities of measurement.

1. No matter which human endeavor you’re talking about, no measurement system is perfect. 
2. Today’s measurement systems are often as good as we can make them.
3. There’s almost always a better measurement system under development.
4. Using well thought-out, well-designed measurement systems makes the world work.  From the power, water and gas supplies we take for granted to traffic management, food delivery, and health services … everything is facilitated by the effective implementation of measurement systems.  And PR is no exception
5. Depending on how your organization thinks about measuring performance, it may be untenable to present “what you feel in your gut” to your senior executive team as proof that as a PR professional you (a) know what you’re doing, or that you (b) have succeeded in accomplishing your allotted task, or you (c) should be taken seriously in a world where most people are held accountable via hard numbers.

Regarding media analysis systems in particular: they can answer a lot of questions regarding any organization’s competitive media landscape – which in turn can inform both strategic planning and tactical responses to situations that arise in the media. 

Like all measurement systems, media-centric metrics need to be tailored to and aligned with your circumstances and your requirements - by people with the skills and the tools to do so. The alternative may be nothing more than (expensive) lies, damned lies and statistics.