6:3 Improving event measurement

We harness AI and voice recognition to generate transcripts, which we subsequently review and edit. However, due to conversational nuances and technical jargon, absolute accuracy cannot be guaranteed.Lee:
Welcome to the Event Engine podcast. This is your host, Lee. Hey. Today, we have back on the show as if it was yesterday. It's Dax, mate. How are you doing?

Dax:
It does feel like yesterday.

Lee:
It literally does feel like yesterday. A year ago. I feel like I'm always wearing the same clothes as well, I think. I may be wearing the same. I'm pretty sure I am, too. So, mate, you were on the show back in 2023. And you're back again. What you've been up to for the last year?

Dax:
Thank you for asking. It's been a very interesting year. I left my full-time role. I was head of strategy at SMILE. My last day was November last year, so it's been just over a year. I decided to do my own thing and go freelance. Amazing. Very scary. It took a little time. It was a little slow, but now it's rocking. Okay. I'm doing freelance strategy, work for a number of agencies and some direct to clients, almost Well, not all, but a lot of event marketing strategy, measurement strategy, and even some broader cross-channel communication strategy, which I love. It's very interesting. It's different than event strategy. But all strategy and creative thinking and copy and all content development and all the upfront of a project. Then thankfully, someone else then delivers what I set up because I'm not good the delivery part.

Lee:
You're good at the creative element and all of that.

Dax:
I can write the plan, but don't ask me to execute the plan. It's not going to work.

Lee:
Fairly sure there's some ADHD in there somewhere.

Dax:
It could be. Then I can measure the plan afterwards and be like, What went wrong? Why didn't you follow the strategy?

Lee:
That's awesome. I mean, thinking of November last year, that's just when everyone's calming down for Christmas, isn't it? It's always a scary time.

Dax:
It's not a good time to leave a full-time role right before Christmas with nothing on the horizon.

Lee:
I'm so glad to hear, though. It's rocking. Let's talk then. What have you been doing around measurements then over the last year?

Dax:
Well, I think you may know that I also am the head of a not-for-profit called the Experiential Marketing Measurement Coalition. That's a mouthful, so we'll just say EMMC from this point forward. However, if you're interested, the website is eventmeasurement.org. That's easier to remember. Maybe we should have the thing that, but anyway. That's actually why I'm here, because we have a partner. This association is focused on developing more credible measurement in our industry, standardisation of measures and metrics and methodologies so that we can give evidence to support why we invest in events. That is critical for an event person because they have stakeholders almost who are saying, Why are we spending all this money? Because events can be expensive investments. We also use data to drive improvement. We know from the data what worked well, what didn't work as well, and then we can ultimately drive continuous improvement over time. The EMMC, in its establishment of standards and benchmarks, is trying to help the industry have something to compare themselves against. So as an example, if I were to say, We have We're going to event sustainability live here next door. If I were to say to you, what is the average carbon footprint for a stand of this size?

Lee:
I would have no idea.

Dax:
Because it doesn't exist. The answer does not exist. And so those are the kinds of questions and gaps that exist that we need to fill because then you can say, All right, we've evaluated our carbon footprint. How do we compare against everyone else? That is when the data becomes more meaningful, and that's what one of the initiatives of the EMMC is to standardise and benchmark key data points for the industry.

Lee:
Yeah, that's phenomenal. I imagine event organisers are capturing a lot of data but not necessarily doing anything with it as well.

Dax:
Well, the thing about the data is there's a lot of data that is automatically flying around. Here at EventTech Live, we have a number of companies who have platforms and technologies, and they may have an app or event management or registration, all these different technologies. All of them produce data almost automatically. Event organisers may find that data of interest, but they may not know how to interpret it. There may be data points that they don't understand how to derive meaning from. Here's an example. I'm working with a company now where they did a tracking of people's movement through an environment of an exhibit. In their reporting, they say, Well, these are your percentage of engaged users. Engaged visitors. I said, Well, how do you know that they were engaged? Well, they spent more than 20 seconds in the stand. I'm like, That's not actually a true credible data point.

Lee:
They could have been instead texting like the guy behind us.

Dax:
It could have been here. They could have been waiting to talk to someone. They could have been laughing at how bad the stand was. I don't know what they really were getting out of it. That's a data point that's in their analysis within the technology. An organiser would look at that and say, Well, that looks great. We think it's good, but we don't know what it means, and we don't know if it's good.

Lee:
It's also a faulty interpretation, isn't it?

Dax:
Yeah, it's exactly right. That's why groups like mine with this EMMC are so important because we are trying to elevate the standards. We're trying to bring clarity where there's generally lack of clarity. And this is what I try to as a strategist anyway. My job is to make something clear, look at a lot of information, a lot of data, and come to some level of clarity and simplicity and actionability. That is true in the measurement space as well. We have lots of data flying around. Another example of a data point might be, that stand is busy, that stand is less busy. So visually, you're like, Well, that stand is more successful than the other one. But that may not be true because that stand that's busy. Giving away free coffee. Free coffee, exactly. Of course, they're busy. The question is quality over quantity. Yes. Some executive would look at that and say, Wow, look at how great we are. But if that same executive is trying to evaluate the impact of the leads captured and the potential revenue of that investment, the free coffee to generate traffic didn't necessarily do that. Yeah. It's getting to honesty.

Dax:
I really want us and this association, we want to get to credible truth if we can.

Lee:
It's so hard to get to the truth. But everyone's a winner in that circumstance then, aren't they? Because if people are making investment decisions based on faulty interpretations of data, then everyone loses.I.

Dax:
Would think so.In the end. The challenge is, and I remember working on a project where it was a marketing campaign, and the senior executive, we were talking about how we're going to measure impact. Yes, we talked about in-store purchase rates and digital click-throughs and all that stuff. He said, But really, I want to be on the morning show getting interviewed about this campaign. We achieved that. We got him on the morning show talking about the campaign, so he was happy. But that's not a business measure, is it? That's an ego measure.

Lee:
I was going to say an ego measure.

Dax:
I think for some people, the ego measure is almost enough, but the business reality has to connect back to that at some point. 100%. That's where I'm always focused on what are the underlying business challenges that we're trying to face, that we're trying to affect, and how do we make sure we measure to know that we did that?

Lee:
On the data then, I think you alluded to it earlier as well, it's very complicated, isn't it? They've looked at that example of the flow through the event. They tried to extrapolate from that data an engagement Great. In the world of AI, can AI help us get better interpretations of data as opposed to us trying to work things out or us sticking our finger in the air?

Dax:
I don't know if it can yet. It may be the future of data analysis and AI. Let's talk about AI for a moment. Sure. There are some technologies that will do facial analysis. That's an AI technology. As people are walking through an event or in an experience, a camera is trained on their face, and it can apparently tell what they're feeling through the experience based on facial analysis. I think that data could be very interesting if it's accurate. The question is, one, is it accurate? I haven't seen any incredible correlation studies to say, if people were smiling in the audience, they're willing to buy more product. If someone can make that connexion point, then I would say the AI becomes very helpful because it becomes an indicator through facial analysis of actual revenue-driving behaviour. Sure. That's fascinating. It's not there yet. I have talked to some of those companies and said, We should do this analysis. We should try and draw a corollary relationship between how people felt and what actually happened from a business standpoint. Where AI is useful right now is if you're doing a survey. Okay. Well, actually, let me take a step back.

Dax:
If When you're trying to do your event strategy, AI can be a very helpful shortcut to getting some insight into what's going on with the target audience, what's happening in the industry, what's happening competitively. I often use AI as an assistant to help me with those data to help inform my strategic point of view. It's not the only tool, but I find it very helpful. If I'm creating a survey for an event, I can go to an AI and I can say, Give me five survey questions connected to this topic. Perfect example is, if I'm trying to measure the perception of a brand, if I say, Well, I want to understand people's perception of this brand as a thought leader, what question might you ask around that. The AI will tell me and give me some inspiring examples for that. Ai is also very good at data visualisation. It's getting better. Where if I'm looking at the analysis of data from an event and I can say, All right, I have all this data about the survey response rate or the survey responses versus the pre-event survey. I can say to the AI, Help me visualise this data in a chart that's very simple to understand.

Dax:
That's incredible because I'm not a designer. By any stretch, I'm not good at design. I can do the words, but not the... I use AI as a tool to help me do that in my reporting of data, which I find very interesting.

Lee:
Because I've spent hours in Excel trying to do bar charts. Exactly.

Dax:
If I can load an Excel sheet into AI and say, give me a summary of this data, or if I can say, here's some open-ended responses to questions, build a word cloud analysis of this and tell me what people said the most. That's this great shortcut for my job.

Lee:
No, absolutely. I think what we're saying is AI is definitely not there on the deep strategic or analysing that data and pulling out some of that value, but it is there. It is able to do things if you inform it. If you're using AI with the knowledge you already have, AI can be very powerful. You can also use it for ideation, so for giving you ideas, but again, it's still based on the knowledge that you already have.

Dax:
It's funny because in thinking about the tool sets of measurement, you have some pretty standard tools. You have surveys, and surveys are There are very useful ways to understand what people think. They're not perfect, but I don't know really how to understand what people are thinking without asking them. Again, going back to facial analysis, I'm looking at your face. I think you're looking at me like, Who is this idiot?

Lee:
Whereas I'm like, This is amazing.

Dax:
But that's my interpretation. Exactly. That's not yours. I need you to tell me what you're thinking because that's the best known method to get to that. Surveys is a great tool. I do think tracking Tracking of people's movement through an environment gives you some clues about what's going on. I think those kinds of tracking methodologies are useful. I really like doing polling questions at the end of a session. If you're doing a session, at the very end, asking people, did you get value? That was the session? There are a range of tools, but one of the tools that I've been working on more or in bringing into my methodology is observation by an expert. I just did this for an event where I went as... I helped do all the measurement for the event, but I said, I'm also going to go to the event and I'm going to experience it as a customer, as a punter. My own observation helped to combine with the more quantitative analysis and helped me to come back and say, Here's what worked and didn't work, based on all of these data inputs, my observation being one of them.

Dax:
Now, going back to AI. Will there be a point at which AI can do what I just did? Yeah.i don't know.You never know. Question. Could it become an expert in event marketing? It could be observing what happens at an event, looking at all the data and coming back and saying, Here's what worked and what didn't work in your event. I'd be out of a job, but I don't think it'll have that same intuition for a long time to understand those things.

Lee:
I would like to think it would never have. We all would. I'd like to think that AI is always going to be there to help us with the knowledge that we have, to help spark ideas, to help us be creative, to do some of the insight. But I do think when you say mom's intuition and that, I do think that's impossible. My wife knows things about our kids before I ever know it. I don't think any amount of AI would ever know there's a problem, but my wife can tell. I think there's that human aspect, I'm sure of it.

Dax:
But do you think... This is getting philosophical.Oh, it is now. Ai, as we know, one of the challenges with AI is programmed bias, where, for example, going back to the facial analysis for a moment, If the AI can't read certain skin tones, then there's a bias automatically in the AI. Now, I think they've solved those issues or it's getting better. But there is also human bias. If you take, for example, recruitment of new employees to a job, if I have a meeting, if you're looking for a job and I sit and chat with you and I'm like, Wow, we really hit it off. I think that guy will fit in. I think he's a good fit for the business. Part of that may be my own bias.

Lee:
Because we're rucking and rolling, we're having a good chat.

Dax:
Well, not just that. We look very similar. We're not that different of age. We have maybe similar points of view on the world. I'm connecting with you by saying, Oh, yeah, this is somebody like me. Now, AI, if you take out the bias for a moment that's programmed in, potentially could be a better judge of whether someone is a good fit for the company and the job role.

Lee:
Because they might be like, he might be a nice guy, but he actually doesn't know what he's talking about. Exactly.

Dax:
He doesn't know fuck all about what he's doing, even though he's a great guy. Believe me, we've all been there. I've hired people that I thought, this is a great person. Yeah, I've done it so many times. We're not good at the job. Yes. These are the big questions, right? We have a belief of our own intuition, and the question The question is, is it honest? Going back to what we said earlier, is it the truth?

Lee:
Yeah.

Dax:
Does AI help us get to the truth? We will see.

Lee:
What we're saying here is AI could help us, apart from the programmed bias, be less biassed. It could be. It could be.

Dax:
Another example, event example. When we have an executive get up on stage and speaking to a group of employees, their reaction to that executive and that person's content is going to be more positive, probably because that person is an executive.Yeah, he pays my salary.Exactly. How do you take that out of it? You either have to cognitively do that as a human, as you're observing, or potentially the AI could be like, But if you look at his content and the way he presented it, it was rubbish. I don't know. These are all really interesting areas to explore.

Lee:
Yeah, no, that's good. Ai could be certainly there used to help us analyse the data and maybe come out of ourselves a little more and get rid of those preconceived biases. Exactly. I hadn't thought of it like that. I've only ever thought of AI predominantly for things like ideation, now helping me come up with ideas, and then I'll then take it further.

Dax:
I think it's good for that as long as you don't... I have to say on LinkedIn lately, every other article I read, I'm like, this is definitely someone's using Exactly. I feel no human voice in it.

Lee:
It usually starts in a world of in a digital world. It's like you know it's ChatGPT.

Dax:
It is glaringly obvious. I'm sure that'll get better. But as long as you've said, you use it as an assistive technology, maybe inspiration to think of some things you haven't thought of. I do use it for that, and I find it really helpful. I use Claude as my AI tool. Brilliant tool. I'll bring up Claude and I'll say, Claude, Tell me what's going on in this industry and with this company and its competitors. I'm like, Oh, I didn't know that one clue. I need to validate it, whatever, research it on my own, but it still helps me on that journey, and it shortcuts some of the research that I would use.

Lee:
I think on the shortcut as well. I use Claude, I use ChatGPT as well. Because you can train those models, you can give it a whole load of data, which would be harder for you. A 90-page document would be harder for you to consume about an industry or about a company, whereas you can put that into a large language model and boom, within minutes you're getting summaries. That, again, helps speed up some of that process. That's my laziness. I don't want to read 90 pages, but I'll definitely put it into an AI and let it tell me the-Here's an ethical question for you.

Dax:
Tell me what you think. Let's say we have a bunch of registration data about an event, and we want to analyse that data to understand who's coming, what are the job titles, what types of companies, what types of businesses, what are they said about themselves? Would you load data into AI to ask it to interpret it for you?

Lee:
No. Because I'd be too scared of putting people's personal information into it. Yeah, exactly. Because it's not much. If I had the LLM locally hosted, so if I was using LAMA, for example, and had it on my own servers, then yeah, absolutely, because I know I'm not connected to the internet, and I would say, Okay, well, maybe I can see if it will give me some insights into the data. But I don't put anything online about it.

Dax:
I have the same thing. I'm really nervous about what I tell the AI.

Lee:
Is it going to come up in someone else's answer years to come? I know. Oh, and Lee's address was. Well, on that dark note, mate.Thank you so much for your time.My pleasure. Good to chat with you. What's the best way for people to connect with you?

Dax:
Linkedin is easy. Dax Callner on LinkedIn. Just look me up, drop me a line. I'm very responsive if something comes up or someone reaches out on LinkedIn.

Lee:
Awesome. Mate, you're a legend. Thank you so much.Thank you. Take it easy. It's a good one..