Not Everything Counts Equally

Evaluation is about counting the right things

by Jak Carroll, Strategic Sport Solutions

Have you ever been to an “educational” online webinar which goes something like this?

00-05 mins         Asking attendees to type something inane in the chat box while waiting for more people to turn up.

05-20 mins         Advertising for the organisation hosting the webinar.

20-25 mins         Introducing the keynote speaker.

25-35 mins         Expanding on their background (by the keynote speaker).

35-45 mins         Explaining why the content is important.

45-50 mins         Delivering the key content (which could easily be found by a quick Google search).

50-60 mins         Requesting questions from members of the audience who are still logged on.

These webinars often have eye-catching titles like “Five Things All Managers Need to Know to Survive in a Post-Covid World” and attract healthy audience numbers.

Some organisations will run a series of such webinars and their evaluation summary at the conclusion of the series will go something like this: in 2022 we ran six webinars, with 480 attendees, representing 280 organisations, making 57 social media posts, and generating 4563 likes.

But what did they really achieve? Did their audience learn anything useful? Do they feel better disposed towards that organisation running the webinars? Are they likely to engage with that organisation again?

Many organisations confuse outputs with outcomes. Outputs are about what you do or produce, whereas outcomes are about the impact you have on people and communities.

According to a 2012 Harvard Business Review article* the difference between outputs and outcomes is fundamental and profound. While the former is a precondition for achieving the latter, it is not enough to just conduct activities without assessing their impact.

For sport and non-profit organisations, there is often a focus on outputs such as events, programs, training, and workshops. However, the key question that needs to be asked is this: How do these outputs enable us to better achieve our mission?

For example: Are people safer or more knowledgeable? Are our clients more skilled? Are communities more active? This is what needs to be assessed if the organisation is to truly evaluate its impact.

Albert Einstein once said: Not everything that counts can be counted, and not everything that can be counted counts.

But when it comes to evaluation, I believe the reality is often more like this: Not everything that counts is evaluated, and not everything that can be evaluated counts.

* It’s Not Just Semantics: Managing Outcomes Vs. Outputs by Deborah Mills-Scofield, November 26, 2012