Interview with Richard Bagnall, co-managing partner of CARMA and Chairman of AMEC, the International Association for the Measurement and Evaluation of Communication.
Hi Richard, what is your background and what is included in your current roles at CARMA and AMEC?
I began my career as a PR and communications professional and fell into measurement by chance, joining the founder of a new company, Metrica, in the mid ‘90s. I was brought in to grow the business and, over a process of 15 years, it developed into the world’s largest comms measurement business of its day. In 2009, the company was bought by a private equity firm and together with Durrants, Gorkana and Metrica, we merged to form the Gorkana Group. I ran global measurement and evaluation for four years before leaving to become PRIME Research’s UK CEO and SVP of Europe.
Three years ago, I joined long time industry friend and colleague Mazen Nahawi at CARMA as the global co-managing partner and the CEO of our European and Americas business. I have P&L responsibilities for these businesses while also consulting with our clients on tailored evaluation services to provide meaningful PR measurement. Our approach is to attract the world’s most experienced team of measurement and evaluation consultants, which when combined with the great tech on which our services are built, makes CARMA what we believe to be the strongest PR measurement company in the world.
On top of this, I have been the Chair of AMEC for almost six years now. My role at AMEC focuses on industry best practice, ongoing education and ensuring a common and consistent approach all around the world. For education to succeed, it is critical to have a common and consistent voice around the globe all singing from the same hymn sheet.
How does Carma differentiate itself from other media intelligence companies?
First, we are a truly global business and work on all major continents. Second, we are not owned by a private equity firm; we are predominantly privately-owned, including by the management team.
As an industry, there has been a rush driven by the influx of venture capital to focus on software-as-a-service (Saas) and platform-based solutions. This brings with it an emphasis on technology, tools and armies of sales teams. Our view is that PR and comms pros do not need yet more nuisance sales calls, pretty charts and dashboards! They need relevant experience, critical thinking, expertise, and world class service and support to help them make sense of the changing media and comms environment.
Great technology is a critical building block in the offering, but it must be used appropriately for what it is good at: it is excellent for massive number crunching and the lightning-fast heavy lifting of significant data sets, but it is not a solution to PR’s measurement challenges in itself. It has to be the servant, not our master. By their very nature, Saas dashboards tend to count what is easy to count, not measure what actually matters. To measure what matters requires tailoring against organisational and comms objectives, and measuring beyond activity to the actual effects that a communications programme delivered.
Our culture also allows us to stand out. Our teams in each geography are some of the brightest, most hard-working and enthusiastic that I have ever had the pleasure to work with. We support, train and educate our team constantly to ensure that we are all, always learning. This approach attracts the very best talent, and the clients too.
What are the challenges for the market ahead?
The major challenge the market faces is one of education and understanding. For too long, PR and communication evaluation has focused on media content evaluation and ‘output’ metrics. These alone only measure activity, not results, and the number on their own can be pretty meaningless – often the inflated ‘vanity’ metric of which we all hear. Worse than this is the concept of a single number to measure PR and comms. For some this was the appeal of AVEs, and some PRs are hoping that AMEC might create or endorse a new single metric to replace this discredited number. But a single number can never measure all the nuances of communication, nor can it provide relevance, context nor insight.
The global pandemic has accelerated the need for PR and comms pros to professionalise their approach to evaluation. We must point to the value that we create, not just count activity. Activity without benefit is just cost. CFOs across the world are stripping unnecessary costs out of budgets as they look to save money, preserve cash and shore up finances for the uncertainties that lie ahead.
To do this, PR teams must not run the risk of being seen as ‘busy fools’ embroiled only in tactics. It begins with a proper plan, aligning with organisational objectives, setting meaningful targets and KPIs, and then measuring beyond the outputs to showing how opinions have been changed, minds informed and advocacy developed, and then ultimately the organisational impact of the work done.
What are the most important aspects that AMEC provides the industry?
AMEC is the global, single, credible voice of best practice, an organisation that the world of comms can turn to for advice, guidance, case studies and education on all aspects of media intelligence and public relations measurement. Founded 25 years ago as a UK-based media evaluation trade association, it has grown into a global professional body covering all aspects of communication evaluation with almost 200 members in 86+ countries.
Since inception, AMEC has been known for its educational initiatives. The first of these were the Barcelona Principles which we launched in 2010. They are seven broad statements defining what comprises best practice. Think of them as a 30,000-foot view, principles that explain what you should and should not be doing from a measurement perspective. They have been iterated every five years since to make sure they are up to date and reflect latest thinking and media and comms trends.
The Integrated Evaluation Framework then takes the Barcelona Principles one step further by showing the way to operationalise the principles. I led the talented AMEC team that created the Framework. It incorporated experts from all areas of comms and evaluation, PR agencies, in-house practitioners, monitoring/evaluation agencies, and academia. All gave generously of their time and worked collaboratively together, one of AMEC’s core strengths.
The framework is based on something called Process Evaluation which is a standard approach used in other business disciplines to measure effectiveness and efficiency throughout organisations. It was important for AMEC to bring proven management performance measurement approaches to the world of communications evaluation, and not invent something that would lack credibility in the C Suite. It was also important that it would be easy to understand, provide advice and guidance and work for all organisations of whatever size and with whatever budget.
The Framework has been translated into 22 languages, is taught at universities and is globally acknowledged as best practice. Latest AMEC research shows it’s been used at over 2000 organisations around the world.
Where will the Barcelona Principles and Integrated Evaluation Framework be improved upon?
The Barcelona Principles were created in 2010 and have been revised and refreshed since then every five years, the most recent time being in 2020. We believe this to be the right cadence as the comms and media industries continue to evolve to ensure that they are up to date.
The Integrated Evaluation Framework, being based on process evaluation, is an approach or a methodology, not a tool or a metric. As such, there is not a lot within it to iterate, but we are focused on making it more and more accessible and approachable to organisations across the globe. We are focused then on more how-tos, case studies and educational support. This year for example we launched a raft of planning support materials, showing how and why proper PR planning is such a critical and integral part of measurement and evaluation, and where it fits within a measurement framework. For the coming year, our educational focus will be on a free online course showing how to apply the framework to your organisation.
Privacy around the use of social data is an emerging challenge. How do you think that will affect the media intelligence industry?
There will be a further decline in relevance and meaning for many of the old school output metrics that were the primary focus of evaluation for so long. Many of them are now just the vanity metrics of today, the ever-larger numbers that mean very little. Impressions, for example, are impossible to define accurately and cannot be thought of as anything much more than an index.
We have seen some evaluation vendors try to innovate and come up with cookie-based solutions to work out more accurate impression numbers. This sounds great as a concept but relying on cookie and ad-based tracking technologies has been hampered by privacy concerns and cookie and ad blocking software. My personal view is that the search for an ‘accurate’ number of people reached across the diverse media landscape is nigh on impossible. Instead of focusing on that, professionals should be looking to link the output metrics to the outtakes and outcomes that their organisation cares about.
I also see the automation side of the media intelligence sector declining in importance while AI is not yet suitably equipped to provide the insights that the industry requires.
Do you have any final thoughts?
That the communication and PR industry in my mind stands at a crossroads. It has great opportunity but also faces a significant threat. It has to evolve its approach to measurement and evaluation, to focus on demonstrating how we support and drive organisational impact. This involves taking the time to tailor and structure a relevant measurement approach for your own organisation, one that looks across all three dimensions that we have been talking about – outputs, outtakes and outcomes. It involves planning up front and setting targets based around desired outcomes, not just activity. This takes time, and thought and can’t be outsourced to commoditised vendors. It needs relevance, critical thinking and experience.
By Peter Appleby
