
Ops Cast
Ops Cast, by MarketingOps.com, is a podcast for Marketing Operations Pros by Marketing Ops Pros. Hosted by Michael Hartmann, Mike Rizzo & Naomi Liu
Ops Cast
The Meaning of Life, the Universe, and MOPs with Andy Caron
Text us your thoughts on the episode or the show!
On todays episode, we down with Andy Caron, President of Revenue Pulse, to explore the unexpected intersections of curiosity, attribution, psychology, and the marketing operations profession. Andy shares her non-linear journey from costume design and publishing to marketing ops leadership, revealing how seemingly unrelated experiences laid the foundation for a successful career in MarTech and consulting.
We unpack the role of curiosity and "hand-raisers" in MOPS success, debate the nuances and pitfalls of attribution modeling (with a detour through The Hitchhiker’s Guide to the Galaxy), and dive deep into how understanding human psychology enhances leadership and system architecture. They also explore the evolving influence of AI in marketing operations and what the future might hold for the AI-augmented MOPS professional.
Tune in to hear:
- From Costumes to Campaigns: Andy’s unique journey from theater and publishing to MOPS shows how creative roots and adaptability foster systems thinking and leadership in tech.
- Curiosity as a Superpower: Why the best MOPS professionals are tinkerers, willing to break things and raise their hands to figure it out.
- 42 and Attribution: A humorous yet profound analogy between Douglas Adams' "42" and the complexities—and misinterpretations—of marketing attribution models.
- The Psychology of Ops: How studying human behavior helps bridge stakeholder needs, build better systems, and influence organizational dynamics.
- AI in MOPS: Insights into how AI is reshaping the profession, from task automation to agent orchestration—plus why being AI-activated (not replaced) is key to the future.
Episode Brought to You By MO Pros
The #1 Community for Marketing Operations Professionals
Visit UTM.io and tell them the Ops Cast team sent you.
Join us at MOps-Apalooza: https://mopsapalooza.com/
Save 10% with code opscast10
Hello everyone, welcome to another episode of OpsCast brought to you by MarketingOpscom. Powered by all the mo pros out there, I am your solo host, michael Hartman. Joining me today is Andy Caron to talk about curiosity, attribution psychology and how all of it connects to marketing ops in surprising ways. Andy is currently president of Revenue Pulse, where she brings nearly two decades of experience across marketing operations, sales ops, platform strategy and consulting. Her journey started in costume design and publishing before moving into the world of sales and eventually becoming a Salesforce admin. Along the way, she found her niche in the marketing tech space, blending creativity, systems thinking and human insight. So, andy, welcome. Thanks for joining us.
Andy Caron:Excited to be here.
Michael Hartmann:By the way, I try to do this and I still forget. Did I pronounce your last name correctly, caron? See, I should have asked.
Andy Caron:That's fine. Many people say Karen and I don't even correct. It's totally fine, karen. Caron, my first name is actually Androlin, which is one of the reasons I go by Andy and it frequently gets mispronounced as Adrenalin. So my last name being pronounced correctly or incorrectly is the least of my worries when you have a first name like that.
Michael Hartmann:I mean I won't take this off, but my name is spelled. It's not that hard, but it's spelled oddly enough with the two Ns that I do get misspellings. It used to really bother me and I finally am like it doesn not that hard, but it's spelled oddly enough with the two ends that I do get misspellings. It used to really bother me and I finally I'm like that's bothering me. But there was an episode at the, at the driver's license office one time that there's a story there. So anybody who's listening if I see you, see me, you know you can.
Andy Caron:You're welcome to ask me about that at some point and I'll tell you the story for Mr Her own, frequently enough that, uh, you know I just all bets are off, it's all good.
Michael Hartmann:Wow, yeah, yeah, here we go. All right, well, sorry about that, we'll carry on, but let's yeah, let's start with your background. Um, you, you, you began in costume design and publishing, and publishing, um. So I'm just curious, like, first off, what you may not know if you've listened to a lot of episodes. You're not the first person who's got some sort of connection to, like theater world. Um, that's been a guest and is in marketing ops now. So I think that's an interesting fact. But what, what drew you to that? How did how did you evolve, like, tell that story?
Andy Caron:uh, I fell bass awkward into mops is, is, is the long and short of it. But, um, the the journey, uh, in a sense, I guess, was that I started out pretty young with an interest in theater and did a lot of school plays and uh was part of, you know, audition group, choir and all that kind of stuff. And in high school I had the opportunity to work at our local repertory theaters wonderful, um, small theater up in the Sierra Nevadas. Uh did great, uh, great productions and I did everything from backstage work to sound production and back again. And my mom actually had a background in clothing design and I grew up with that.
Andy Caron:I grew up with someone who was always sewing and making things. I showed a proclivity and a talent for it early as well. You know Barbie, clothes and all the things you'd expect of a child, with scraps of fabric and a needle to hand. So I ended up on a production run of Annie Get your Gun, stitching up a very overzealous dancer's pants every night because he would tear them, and from that got recruited into the costuming division of the theater. I absolutely loved it. I've always had a passion for costume and the idea of being able to create an experienced persona through clothing Just absolutely fascinating.
Andy Caron:So after I moved on to college I started picking up local theater work and when I graduated I actually moved down to LA to pursue costuming in film. I discovered I wanted absolutely nothing to do with the movie industry, as it turns out, not my vibe, as the kids would say and so from that I actually had a referral from a friend from college into gear book publishing, of all things. It was my entree into Adobe InDesign 2.0 and also all things you know, education of technology. To those that haven't encountered it before, don't know how to use it. So, working with middle schoolers and high schoolers to help them learn Adobe, help them learn journalism.
Michael Hartmann:So it wasn't just the, it wasn't just the publishing of the yearbooks. No, it was also helping the students.
Andy Caron:Yes, the end production and delivery of the yearbooks themselves. So I got to work with about 40 schools across the Los Angeles basin for nearly four years and absolutely loved it. It was a hard industry to be in, especially with yearbook sales kind of going down at that point, particularly in that region, and ultimately I relocated out of LA and at that time also particularly in that region, and ultimately I I relocated out of LA and um at that time also exited that company. But very cool experience, um, really cool uh sort of training wheels for some of the future things that I would encounter a lot of sales, a lot of marketing in that, in that role to bring on new schools uh into the, the publishing um uh pantheon on on that side and uh, that definitely carried forward uh with me, along with the creative aspects of that, as I started to look for where I would eventually land Um land.
Andy Caron:Oddly enough for someone who was raised by a very out in the world working mom, I had somehow been raised with this idea that I would grow up to be a stay at home mom.
Andy Caron:I don't know why my mom instilled that or kind of put that in there, but she was actually the first female photographer to work for Ford Motorsports. So I don't know if maybe she just thought, instilled that or kind of put that in there, but, um, she was actually the first female photographer to work for Ford Motorsports. So I don't know if maybe she just thought, you know, wanted something less complex for me, or whatever, but that was my thinking. And so in my twenties I was, frankly, treading water with jobs, not careers. Um, because of that you know mentality and um, in, uh, in the year I turned 30, I took a job with a startup, um, and that set me on my path on unbeknownst to me, as you know, my first day on the job there, uh, to where I sit now. So, um, it's, it's definitely not, uh, a direct or even a squiggly path to get here.
Michael Hartmann:That, uh, that you hopped across a couple of lily pads to get there.
Andy Caron:It feels like several, including a very big one to Chicago.
Michael Hartmann:So yeah, yeah. So before we get to that, I'm curious, like the things you you kind of hinted at, like there are things that you learn from those experiences in the theater and with the yearbook company that you still probably use on a daily basis in your current role. Like what, what are some of the things, like lessons learned from that that you still use?
Andy Caron:things like less like lessons learned from that that you still use. I think the biggest lesson from the costuming piece is the more complicated and complex you make it, the more complicated and complex you make it for yourself later.
Michael Hartmann:Yeah.
Andy Caron:So the way that you put something together so that it can be deconstructed later is very important in costuming, especially when you're resizing, having to do a lot of things. So building things in a way where they can be reworked when someone changes their mind or the role changes, as it were, et cetera, very much applies in in mops and I think in um, the B2B world in particular. Uh, across the board, um, and from the yearbook publishing side. I think it is sort of twofold One anyone who has the right level of curiosity and willpower can learn to do anything. First, I totally agree, yeah, I totally agree, yeah. And secondly, it was that you know you can do any job to the job description, but the ones where you actually sit down and pull up a chair and scooch in and get your elbows dirty, even if it's not your job, the outcomes will always far exceed what would have happened if you just let people kind of run on their own without the assistance that they maybe could have used from your perspective inside of it.
Michael Hartmann:Yeah, I think that's great, I love it. So you hinted at like you jumped a big, big lily pad jump.
Andy Caron:You remember you moved to chicago, jobless, right in in 2008, of all years, which right before the crash happened super, super good timing. I have exceptional timing. Um, that's what that tells me. Uh, yes, I did. I I was originally from northern california, had moved to los angeles post-graduation and I was in this phase of my life where I was considering moving back to Northern California, but that felt like going backward. I came out to Chicago on a trip and it just felt like home and I thought you know, in my late twenties, it's the time to do a crazy thing. If I'm going to do it, now's the time. Crazy thing If it's. If I'm going to do it, now is the time. And so I, I picked up and I moved and, uh, it was a bumpy time for finding jobs. I, I took temp work, I did tables, I did all the things I need to do, sometimes simultaneously.
Michael Hartmann:All the things you did when you were in theater. World right you know that too.
Andy Caron:Yeah, exactly, Just going back to basic. But ultimately I land with a Chicago based startup. I was their 12th hire, was actually brought in in a sales role. The entirety of my job description was open enterprise level opportunities period, and I like that Actually, I think over.
Michael Hartmann:I think a lot of job descriptions are overengineered these days.
Andy Caron:Fair. The enablement, though, was challenging. My very first day on the job, they gave me a computer that had seen better days, with a spreadsheet on it where the columns and rows did not all match data wise, with 300 companies that they wanted to pursue, uh, and a phone, and I laughed at them and told them to go get me Salesforce. And on day two of that job, I became the de facto Salesforce admin and thus uh, thus launched me into sales ops. I ultimately um helped to build out uh, out their SDR team. They brought in email marketing and then evolved that into automation and eventually Marketo in 2012.
Andy Caron:And was with that organization for five years, during which I was both marketing and sales admin on both sides, until, I think, the final year when they, when they did bring in a Salesforce admin, uh, who, who? I off-boarded you know those, those parts to it and and on the Marketo side. But, um, there was no uh documentation anywhere online on how to do any of this. They were Marketo user groups. There were, um, you know, peers within the community and, and you know the online help me boards and that type of thing, but the the term we have today, marketing operations didn't exist. I had, you know, database, specialist sort of titles and all that kind of thing during that tenure.
Michael Hartmann:And it's e-marketing. E-marketing on your title somewhere.
Andy Caron:I didn't know. No, e-marketing, no, no, yes, yeah I. I don't think I had a marketing technology title until 2018.
Michael Hartmann:That seems interesting. I'm just imagining that, like the Glenn Gary, glenn Ross scene, right when you first walked in, right? That's what I'm imagining. Right, we've got you the leads right. The leads are good.
Andy Caron:I was there 12th higher, so I don't think it was even necessarily that coordinated um yeah, I bet it was a, it was a wonderful, um, sort of birthing ground for me to have the, the support and capacity to figure it out and try it and hand raise, and I think that was really just exceptional, as far as you know, as someone who is a hand raiser, to be able to be in a space where I could say I'll give it a shot, let's see if I can do it and have leadership, support that and then reward it when it was successful was was, really, was, really, really cool.
Michael Hartmann:Yeah, that's, that's fantastic. You mentioned that term hand raisers. So when we talked earlier you said that you think curious hand raisers tend to excel in the space space being like ops. So first did I get it right? And then like, why do you think that's the case?
Andy Caron:I have found that people who are tinkerers, people who took apart VCRs when they were kids, who were willing to break a thing, to figure out how to fix a thing, tend to excel in mobs because it's, you know, historically been kind of an uncharted space. An uncharted space, one where you have to cut your own path and sort of navigate and maybe use maps that are left, you know, from a previous, you know incarnation or role there or elsewhere in the community. But I find that hand raisers are willing to fail and when you're trying something new, the willingness to fall flat on your face and learn how not to do it five, five ways first or 50 ways first is, um, is really a boon.
Michael Hartmann:Yeah, was it Thomas Edison who said um, I've never failed Like I've, I'd be like I found 500 ways not to make a light bulb or something he did with it.
Andy Caron:I don't think he ever actually said it yeah, okay that, uh that quote, regardless of who coined it yes, yeah, right, right, right.
Michael Hartmann:Well, it's interesting you bring up tinkerers, because that's a term. I always like I'm not a tinkerer but I in some ways I think about my childhood. I, I don't even know, I always say as a tinkerer, but like I was willing to try crazy stuff and like take things up. Like literally, my neighbor and I we took apart two bicycles, recreated a new one out of parts from both, painted it, made ramps, hurt ourselves. You know the whole bit. Yeah, um, and we made go-karts and you know like we did all kinds of stuff. I don't really do much that, but but it was like purpose, like it was purposeful tinkering. What I don't like is just tinkering for the sake of tinkering, like we there we inherited and I just tinkering for the sake of tinkering Fair.
Michael Hartmann:We inherited it. Somebody gave us an old sailboat. I didn't grow up sailing and it needed lots of work. And when I went out there I was all excited. I was like I want to sail. It sounds great. And then I noticed that everybody out there was literally out there all the time. All they did was just tinker with their boats and I was like that, like I want to enjoy the boat. I'm willing to do that if I can. But they didn't seem to be doing that right. They were just messing around so that like.
Michael Hartmann:So I think on the one hand I feel like a tanker, but on the other hand like I don't. It's not just for that sake, does that?
Andy Caron:make sense. It does it. Yeah, definitely makes sense yeah, but I've.
Michael Hartmann:It's interesting, that combination of what you talked about from your early roles in that description I also think of. My first job out of college was with a consulting firm Price Warehouse and one of the things they did is they sent the consultants at that time They'd actually shortened it. It was only three months of training on their methodology and their way of doing stuff, and you had these cohorts from all over the country. Most of us were people like me with, like, an engineering or business background or something like that in terms of our education. There's this one woman who, super bright um, was a religion major and I was like how is she going to survive going through this? Because we were doing programming and stuff like that. But it turns out what I realized from that is like she was curious, she was willing to try and she knew how to learn and she, like she had a lot of other traits that I realized were just as valuable and it kind of made me go like, yeah, I can probably code better and faster than she can now, but she's gonna figure it out yes so yeah
Michael Hartmann:yeah, it's funny. Um, yeah, so totally get all that. So, yeah, another thing that we you and I talked about was, um, you studied human psychology, yeah, um, and maybe this comes from some of your work in the theater stuff too. I suspect there's a lot of like human psychology that, yeah, and maybe this comes from some of your work in the theater stuff too. I suspect there's a lot of like human psychology that comes into play there, but I'm probably overgeneralizing. So what do you think there are Like? Is that a skill set? Do you think more marketing ops folks should be developing Right? Is it underrated? What do you like? What's your take on that?
Andy Caron:I know a lot of people in mops that have psychology degrees. Actually it's it's a, it's a trend similar to the theater trend that has definitely been noted on on my part and I I think that you know, first and foremost, when a lot of us went and got our degrees, there was no mops, there was no Martech in some cases. So what could possibly prepare you for this future role and sector of the market? And I think psychology as a whole in marketing, in business operations, whether it's marketing or sales or rev ops or other areas of operations, in leadership all of those are so integral to working with people and to figuring out how to get people to want to work with you. Um, and so psychology and sales, psychology and marketing I think those are natural, but when you are architecting systems, to react and accelerate leads toward the goal that you want them to want to get to. I think there's a lot of psychology that goes into that and the, the if, then what statements all have that you know, getting into the mindset of the customer associated with them, while also getting into the mindset of the business and the stakeholders and what they want and what their initiatives and goals and requirements for data and insights and, you know, roi for these initiatives are. So I think it's all psychology.
Andy Caron:Um, you know, I think a lot of people do it without thinking about it, with or without a degree. Um, it just so happened that, you know, I actually started out thinking I would go into therapy, that I would become a therapist, and when I was getting ready to graduate undergrad, I remember having this thought of being on a sofa or on an armchair, across from someone on a sofa who was there to talk to me about, you know, his middle-aged life, with his wife not understanding him and his kids hating him and him hating his job and me having had none of those things outside of school, just being in a position where I completely was unable to help this individual, and the um, uh, irresponsibility of moving to that next phase of my education and then career without having had some exposure to it. And so I ultimately decided that maybe I'd wait until I was like an empty nester and go back to school and that would be my, my retirement plan to retire as a therapist or something. Um, ultimately, I went back to grad school when I was 31. And I actually was focused on forensic psychology at that time a deep interest in psychology and the law and sort of helping in that arena.
Andy Caron:And in the midst of all of that, that's when I started working with Marketo at the startup and I it, it, it was the right spot. I could dig in and blink and four hours later I'd missed lunch and I'd done all these cool things and I loved it. And so I I pivoted into psychology of organizational leadership at at that point and, you know, went, went forward. But I I think about that journey and I think psychology is probably a really good lead in for these, these types of of roles, even going forward, especially with with AI and how that changes the, the morals and the thinking and the fundamental sort of architecture of how people react and respond to it For sure.
Michael Hartmann:Yeah, it's interesting Cause I think I think if I, if, if I was to put myself in my shoes 20 years ago, I probably would have rolled my eyes at the idea that psychology was an important thing to understand in the business world right, because it's like this right, like the false impression that things are everyone acts rationally in the business world right, is just bullshit and I don't think I realized it till later.
Michael Hartmann:And now I agree with you Like I think the there's probably a lot of people who are listening or watching who might go like oh yeah, it really is, like it's just silly and it's not that important, and I would tell them to like rethink it. But I think for those who are interested, right, they go, like you mentioned, like some people kind of naturally are able to do it, but I'm a bit like and so they go. That's a soft skill in quotes, right and term I hate because it's a skill, right, and some people like any skill, some people are going to be more gifted naturally than others and that. But you can, like everyone can become confident, I think, if they put their mind to it.
Andy Caron:Yeah, there's a fantastic book that was done around deep research in what makes a good CEO. It's called CEO Excellence and McKinsey group had put that together. It was absolutely wonderful. But there's a chapter section in there called the soft stuff is the hard stuff, and I always think about that, right, Because working with people the soft skills right, that's the hard stuff. We're more complex than half the systems we work in, Right Um, I also also always laugh about the fact that I had thought that I might be, you know, a therapist one day and eventually found my way into um, mops and MarTech consulting and and uh, essentially became a, a, a mops or Marketo therapist, right Uh, consulting is often sort of a form of therapy.
Michael Hartmann:It is a form of therapy, for sure, yeah. Yeah, sure, yeah, yeah exactly yeah, I mean having been a consultant and hired consultants. I think sometimes, sometimes it's easier to be honest with the consultant than it is with people internal to the company, which is a hard thing to say say it straight to your folks, when maybe when you say it it doesn't quite translate, or register or hit yeah for sure.
Michael Hartmann:Yeah, so it's interesting. I think I told you that one of our early podcast episodes we had Brandy Sanders on and she mentioned psychology and chess right Understanding chess also is like that yes.
Andy Caron:Four plus years later, I still think of that on a daily basis, probably at least a weekly basis, like how important the idea of like understanding how things are connected, thinking ahead, and then the human psychology part of it yes, I think that playing forward into possible paths and actions and those, you know, checkmates from your opponents, aka systems or processes or departments on, well, this won't work because of this and this way I could get here, but if this happens then it's a failure, and that logic forward almost like a la three-dimensional chess from star trek is very much part of this role.
Michael Hartmann:Um, absolutely, both systems and also the organizational side of it yeah, totally yeah, I think I still love that, that idea, um, but let's maybe shift gears a little bit. So something you want we were going to talk about was, um, you know, the idea to talk about attribution everyone's favorite, uh, subject to love or hate, right and when, and when we were talking, I thought it was interesting that somehow we landed on the Hitchhiker's Guide to the Galaxy. Yeah, and how? The answer to the question, right is a great analogy for the trap that we could fall into about being obsessed with data. So you want to walk through what kind of the thought process there?
Andy Caron:Yeah, so I struck on this a couple of years ago and I actually got the opportunity to present on this at the first year from Upsalpalooza, which was really, really fun. I walked away with a 42 tattoo a la the tattoo booth as well sort of a fun remembrance of that, but it's interesting. So he's a favorite author of mine and and I love you know, sci-fi and and fantasy and and anything that marries that together with insights on humanity, and in particular I'm I'm always drawn to and Douglas Adams just did such a phenomenal job of that. But within his first book, in the Hitchhiker's Guide to the Galaxy and the subsequent ones, that you find out the story of this sort of super beings that wanted to know the meaning of life, the universe and everything. And they built a massive computer to get the answer to the meaning of life, the universe and everything. And this thing processed for generations and when it was finally times like a media circus and they built up so much around it within their society. And the computer powers its screen on and comes up and says the answer to the meaning of life. It screen on and comes up and says the answer to the meaning of life, the universe and everything is 42 and you know you can envision in the book kind of not just being able to hear a pin drop with like the um, and ultimately the computer indicates that 42 is the answer. But they need the question for the answer to make sense and that it's not smart enough to give them the question and they have to build a bigger, smarter, much more complex, much longer running uh computer and system to give them the question so that then the answer will make sense. And in pure British comedy, douglas Adams style, the computers destroyed moments before it's set to spit out the question, much to no one's surprise, if you, if you're expecting it, which I think most people are.
Andy Caron:But the parable of that is just is fascinating to me as someone who has, you know, implemented and run many attribution systems and processes in previous organizations and has done so, you know, as a consultant, many, many times, the politics surrounding it and sort of the thinking and process of we have to build this very complex system to get at this massive question right. And then we get an answer and it's too complex and it doesn't make sense because we're not speaking the computer's language and the computer isn't speaking our language and instead of figuring out how to translate or to ask better questions, we build bigger, more complex systems to then tell us more stuff, which then ultimately crashes, and, in the interim, no progress or optimization to the business is being done or made on a data based level. And the irony in all of this being that Douglas Adams was a huge technophile and one of the earlier coding languages the one that purportedly he favored 42, was a keystroke for an asterisk which, if you work in data, asterisk is a placeholder. It's literally, in my mind, translates to whatever you need it to mean for you. And if what the computer was saying was the meaning of life, the universe and everything is literally whatever you need it to mean for you, that is quite a deep, profound answer.
Andy Caron:Absolutely yeah, with attribution, in particular, around asking huge questions not understanding the data that comes back and then adding in more complexity instead of going backward, adding specificity or working on better understanding the data or getting the data to translate into more common language that they can understand, and it creates this feedback loop of complexity and lack of faith and more complexity and lack of faith that ultimately undermines and defeats any initiative around what ultimately should be budget optimization and allocation of spend in a more data driven way into a political credit or, you know, decrediting of the whole idea of attribution as a whole, even the inverse, to the end that you know a lot of times it's after much blood, sweat, tears and dollars it's jettisoned.
Michael Hartmann:Yeah, it's interesting to me because I think I was an early fan of the idea of attribution, because I truly believe, with all this data, we should be able to get a better insight into what's working and not working right. Where should we put our next dollar in marketing investment? I wouldn't say I've gone completely the other way. I think it still has a place, but I basically stopped interacting with the debates to get online about, like, which model to use, because I don't think it really matters that much, right? My general take is pick a model, stick with it, look for trends. Don't expect it to be the answer. Right, which is your point.
Andy Caron:I'm on the opposite end of that, which is my favorite question anyone ever asks me, is what model should I use? Because it inherently is a flawed question. That shows the psychology of the thinking, which is there are a slew of models related to attribution, because each one answers a different question and so the answer is all of them to answer the different questions and report against the different OKRs and KPIs that you are benchmarking for and in how you're allocating budget. If you're allocating budget primarily to acquire, then you're going to look at your top of funnel, you're going to look at a U U shaped model what have you and go from there. Right, I think people get stuck in building a perfect model or building one model. If there was a one model to win them all, we would just have one model.
Andy Caron:And it gets tricky where people, especially, I think, conflate building a model with the models, and so they'll think like we're just going to build a model. Especially, I think, conflate building a model with the models, and so they'll think like we're just going to build a model, as opposed to we're going to do different layers of data analysis from statistical models which is what they really are at the end of the day against the larger data set. I think people also get stuck in this idea of having to have 100% of the data instead of being at statistical significance. If you talk to someone who you know has studied and worked within math, especially higher maths, you know 5% is statistically significant, right? Yeah, of course you know if we're trying to get to 80, 90, 100% data completeness or parity, like it's a fool's errand data completeness or parity, like it's a fool's errand.
Michael Hartmann:It's totally so. I I tell I say this all the time that people are looking for they. They want to eliminate ambiguity and I think, just by the nature of this data and the lack of controls around it, like you're never going to get to that. But I think you're hitting on another thing. So my training is in operations research. My college degree is, which part of that is linear programming, and so the linear programming is guided towards.
Michael Hartmann:Give me a real life example of what people who I never practiced it, but the idea is like one of the applications right now that I know of is being used by, like professional sports leagues to come up with schedules for their teams that have a lot of constraints and requirements about. You know when a facility is available, how often can teams play, how far can they travel, you know what other events, and so there's there's not. This gets to the point like there's not an answer. Right, there are probably nearly an infinite number of potential answers, and so you can optimize for one or a handful of them or some basket of them, but you've got like this is this difference we had another guest on.
Michael Hartmann:We talked about about. He told me about the kenevan framework, um, which is kind of talks about like there's different kinds of questions and problems, but there's this um, the difference between complex and um. Um, I'm going to play anyway. But like, complex, is, uh, a modern race car, right, where, like, there's an answer. Like there's a modern race car right when, like there's an answer. Like there's a problem, there's an answer If you know enough about it.
Michael Hartmann:Complicated is right. This is the other one. Is, uh, you really don't know the answer, right, you can take a guess, you can take an action, and I think people think that they have a complex problem right where there is an answer, when really it's a complicated one where there is not a single answer. So the best you can do is try to make a step in the right direction, learn from it, adjust right. Yes, and I think so. When I think about attribution, it is one of many potential things that you could be measuring I would tend to lean towards the things that I know I have more control over. Did my email get opened and delivered? There's stuff that people are moving away from in a lot of cases To me. I'm thinking about the full journey, all these places where things can break down and you can prove them.
Andy Caron:Bot activity and other things have created such false positives around some of those things that I have seen a trend to move away from one of some of the like more almost vanity metrics, if you will. But I think the biggest thing that people also struggle with in that space and that discipline is causation versus correlation right.
Andy Caron:Like. I saw this fantastic video that was comparing the rise in ice cream sales, in a nearly perfect chart line, with shark attacks. And so there's correlation, right, because as it gets warmer, people buy more ice cream. People go more in the water. Not a shocker when you think about it. It gets warmer, people buy more ice cream, people go more in the water Not a shocker when you think about it.
Andy Caron:But someone looking at that chart could potentially if they were passing glance at it or maybe didn't stop to look at what the metrics really were create a causational. Oh well, when people eat more ice cream, sharks bite them, right, like kind of weird. But I think and that's a ridiculous example, but I think it illustrates that causation correlation that people can get stuck between, and also the fact that they want causation when sometimes the best you can get is correlation, but you have a better, um, you know, uh data metric against. Like people are more likely to get bit by sharks when they are spending the day at the beach, right, that arc of beach visitors versus shark bites Like that's. That's a much more parallel and causational metric, potentially, so um we just had a guest.
Michael Hartmann:We just had a guest on.
Andy Caron:We just talked about this like correlations, not equal causation but correlation is correlation right so there may be something you can infer or learn from it, but not necessarily and if we move this lever does this happen. Okay, so now we can actually show causation and not just correlation, and that gives us the methodology to then, you know, pump the brakes on that spend if it's not performing, or to increase, right, like. I think that's where it goes to a scientific methodology of we're going to test a hypothesis and see what happens and then use that to then go from there with the proof points to then modify or optimize, you know budget, until we get to a point of, you know, either diminishing returns or kind of if maxed out, and even if they diminish, it's still better than what we had. So let's do more, you know yeah, yeah, so it's interesting.
Michael Hartmann:So the I thought the previous guest that I talked about. One of the things that we were talking about is AI-driven models for some stuff and how they can generate things that look like they're correlated and people that just don't necessarily make sense. One example that he used was all the people who returned a product were the ones who bought right, which is, of course, right. So this gets into the potential of AI, right? So I'm curious about what you're seeing in terms of AI impacting the role, but also maybe this specific one where, like I've said for a while, I actually am really excited about the potential of AI to uncover things that don't require a hypothesis necessarily to go investigate, especially a large data set. It's just you have to narrow down what you're going to do if you're going to use that scientific method so they might be able to churn through all this data and look for things that might be related, but you need a human, like I might believe is like, until further evidence. Right, you still need a human. Like my belief is, like, until further evidence.
Andy Caron:Right, you still need a human to interpret that I for a long time now have said and I will hold to this that most LLMs that I have played with or use regularly sit somewhere in the very eager and fairly smart intern position.
Michael Hartmann:Yeah, it's not a bad analogy.
Andy Caron:They're not ready to be hired on full time to take on one of the paid roles, but they can certainly contribute and participate and assist. But oftentimes you'll ask them to do something and they'll come back and you have to correct it and then correct it and correct it to get it the direction that you, that you need it to go into. I hold and will continue to hold the line that I think this sort of pervasive, underlying fear that a lot of professionals have that they'll be replaced by AI is, in the short and midterm, not realistic for two reasons. One, we're not there yet with AI. I mean, how often do you go into GPT and ask it for something and it gives you misinformation and when you call it out on it, it goes oops, sorry, my bad, you're right. And when you call it out on it, it goes oops, sorry, my bad, you're right. So the fallibility of that and the capacity for it to to give them if we're really focusing on a deep T aspect of of that but I don't think that anyone's job is is going to be replaced by AI for the most part in the near future, other than some of what we see with like automation at checkout stands, as opposed to having cashiers and that type of thing, although we've seen a resurgence back toward cashiers again. So I think you know it will be a progress that we'll see over time. I do think people will be replaced by people that are AI activated and AI capable and have the capacity and know how to work with and activate AI within the role. I think that will happen. The other thing is just a ROI factor.
Andy Caron:There's a absolutely fantastic book called A World Without Work. It was actually written in 2019. Daniel Suskin is the author well ahead of his time fascinating mind, and what he said you know literally more than five years ago, well before the emergence of everyday AI utilization was that, yes, he does think that eventually we'll get to a place where, you know, society doesn't necessarily have to work because AI can do the bulk of it, and that comes with its own challenges and interesting philosophical contemplations. But what he said, which was really interesting, was that early postulization was that you know, the low level, kind of grunt work would be what was replaced by machines and AI, but he didn't think that that would be what would happen, because the cost efficiency isn't there. There are things that humans will do, at whatever rate that you cannot build a robot to do at the same cost efficiency.
Andy Caron:I've got an example Right, and so if that's the case, then AI becomes inefficient and therefore doesn't make sense.
Michael Hartmann:Yeah, so the example, just if you're curious. So I'd like to say I'm a fly fisherman but like I haven't been in a couple of years. But one thing I learned is that so if you've ever gone fly fishing and you had to buy flies like they're expensive for kind of what you get from them, but at least they feel like it, and I don't know if I have asked, have asked. It's like like where are they made? And like I assumed that they were, there was some sort of like factory?
Michael Hartmann:well, there is, but it's all people. So apparently the process of doing that is doing well, it's all. Yeah, they're all hand like and so it's very human like it. Just I guess they have not figured out, maybe that was. It's been probably 10 years since I asked that kind of question, but yeah, so I think you're right. I mean, there are things out there that just don't like, either don't make sense from, like you said, an ROI standpoint to automate, or that are just, you know, whatever the dexterity capabilities of the machines is not there.
Andy Caron:Yeah, exactly so, um, you know all of that said, uh, rp is has been an interesting place over the last several years. In, uh, spring of 2023, we sat down and and made the decision to do AI training and enablement across the entirety of our team everybody, hr, finance sales, all of it and, of course, across our consultative team, because we knew that was coming. And we have continued and rolled out new training and enablement over the last two years to not only keep pace but to, you know, be ahead of of that curve. And we have been doing a lot of AI automation and augmentation for for clients and you know are are coming to the market with an, an agent that you know is a mops cops, um sort of partner in crime, if you will.
Andy Caron:I always say he's not an autopilot, he's not a co-pilot, he's a wingman, because we're still not at a place where I want AI QA-ing things or, um, you know, without a layer of humanity in between, to check and make sure that everything looks good, because errors can happen with humans, errors can happen with AI, but you know the level of of error ratio we have a 99.9% confidence at this point with him um is low. It's probably less than folks.
Andy Caron:I've had working inside of my instances over the years, but, um, what I'm seeing with AI is the capacity not only to train and create agents that truly are very specialized in a particular area, but to orchestrate multiple agents into processes and to then automate those in with MCP models where, you know, this one goes and does this, this one goes and calls that API because it knows how to do that process, and so on and so forth. That scale is just super exciting and really, really cool and, um, definitely moving in the direction, um, that I think all of us kind of hoped that we could go with AI, which is there was someone, oh gosh, about a year and a half ago and I pardon me, I don't remember her name, but she, she. I recall she said she was so disappointed with AI because she thought AI would give her, ai would give her more time to write and create music and take care of the dishes and the laundry, and instead it's writing and creating music and giving her more time to do dishes. Yeah, and I think this type of thing is that scenario where it clears, you know, has the potential to, and can actively clear and clutter away the laundry and the dishes and the day to day sort of repetitive tasks that you don't want to be doing, so you can get to that corner of desk stuff, so you can get to the strategic work, so you're not like just you know, all day long a task taker that's, that's dispositioning stuff for other people, um and instead can get to the the more fun, more interesting, more strategic work.
Andy Caron:And so I think that emergent piece there is, um is really exciting and excellent and becoming part of the tech stack as a much larger piece of the overall deployments that we would see in the scale of that that he was predicting was both somewhat humbling and mind paradigm shift from, you know, homegrown tech versus going out and buying a SaaS platform. Like the SaaS platform is. Anytime someone says it's an in-house developed platform or tech, you kind of cringe a little bit. You're like, oh no, what's it going to do? What doesn't it do? But this idea of you know, developed in-house AI agents that have very specific roles and processes I think we'll see a rise of that over AI-powered enabled tech and I love that. I think it's really cool, it's really smart.
Michael Hartmann:Yeah, I'm seeing that actually already. So, yeah, I think it's going to be fascinating. I think the short version of what I think is like the people who are not learning how to use AI as a tool to help them in their jobs are the ones who are going to be in trouble. I think it's, but it is a tool right. It's just like any other new technology. It's got our phones right. They all potentially cause disruptions, and so we need to learn to adapt and take advantage of it I think in the workspace it's, it's awesome, it.
Andy Caron:It scares me a little bit on the, the social side of things, to be honest. I think it goes down a whole different rabbit hole there. But for for, from a work side of things, I think it's an accelerator. It's, it's so exciting, it's really, really cool and, um, you know, the, the, just the, the velocity of innovation and discovery around it has just been so cool.
Michael Hartmann:Yeah, it's mind blowing. Um, I wish we could go on. This has been a blast. Um, now maybe we'll just call this episode the answer is 42 and see if people can figure out what it is. That's a blast, Andy. Thank you so much. If folks want to kind of connect with you, learn more, maybe just brainstorm on this kind of stuff with you. What's the best way for them to do that?
Andy Caron:You can find me at LinkedIn. Easy peasy, just Andy Caron. You can send me a quick email, andy, at Revenuepulsecom, always looking forward to connecting with people. And, of course, I am on the Slack community as well, so feel free to look me up there.
Michael Hartmann:Fantastic. Well, thank you again, andy. Thanks to our listeners and supporters, we always appreciate you. If you have ideas for topics or guests or want to be a guest, reach out to Naomi, mike or me and we'd be happy to talk to you about that. Until next time, bye everybody.