Welcome And Why Measurement Fails

Michael Hartmann 0:01

Hello, everyone. Welcome to another episode of OpsCast, brought to you by marketingops.com, powered by all the MoPros out there. I am your host, Michael Hartman, joined by no one again. We'll get Mike and Naomi back at some point here, I'm sure. But today I am talking with Paul Shirer, founder and CEO of Infinite Ideas AI and director of AI and go-to-market technology at Bridge Partners. Paul spends his time helping organizations evaluate, implement, and scale AI initiatives. And in earlier conversations with him, we found ourselves coming back to a theme that many ops professionals are wrestling with right now. That is, despite massive investment in AI, many organizations are struggling to generate meaningful business impact. That is measurement. We'll explore why that is, where companies are getting AI adoption wrong, how leaders should be thinking, think about workflow design and decision making, and what practical steps organizations can take to move beyond experimentation and toward measurable value. So, Paul, welcome to the show. Thanks for joining.

Paul Shirer 1:01

Hi, Michael. Yeah, thanks so much for having me. Yeah, excited to talk today.

Michael Hartmann 1:05

Yeah, it's it's an all-Texas affair today, since you and I are uh what, about 50 miles apart? Is that about right? Yeah, that's about right.

Paul Shirer 1:14

You got the north side covered, I got the south side. We got our we got our territories we stay in.

Michael Hartmann 1:18

So there you go. There we go. Yeah. Um, well, again, I appreciate it. Uh so

AI Adoption Beats AI Tools

Michael Hartmann 1:25

let's just j dive right in. So when we when when you've talked and what we took sort of trade message about, you've you've said that many organizations are treating AI as a tooling problem when in reality it's a workflow and adoption problem. So break that down. What do you like what does that mean in your words? Um, and and how did you get to that that sort of point of view?

Paul Shirer 1:49

Yeah. Well, I got to that point of view by failing, uh, number one. Uh so that that helped or a little bit of a lot of people.

Michael Hartmann 1:55

That's yeah, pain pain is a good teacher, right?

Paul Shirer 1:58

That's right. Um, yeah, so I I remember, I recall, you know, back whatever a couple years ago, when maybe it was two and a half years ago now when the Chat G GP uh moment hit. And uh the first thing was like we need to get Chat GPT, right? And so we got it, everyone got everyone got a license, you know, and it was like, and then you'd go look at the usage of that tool, and it would be like, you know, very few people using it at the time. Now I'm sure that would be different if we did it today, but at the time it was very little. Yeah. And so you started looking at uh, you know, uh how are we gonna start transforming uh the way we work? And really that was the theme from the beginning. If you were getting ahead of it from a strategic standpoint, you'd be thinking this is going to change, but how long is it gonna take? What's it gonna take to do that? And what are the baby steps along the way, right? And so a tool is just a tool, it's sort of like a hammer is just a hammer, like we hear that all the time. That's right, what it is, but unless you have something to use it on and and kind of what you're trying to build, then it doesn't, it's not really all that useful. Um so I think that the key to this whole thing is for most organizations is you know, start where people are. Don't don't get way, way ahead of yourself. You know, like the idea that you're going to change away all of your workflows and processes and get people to follow along and do that all at the same time, it's a very difficult thing to do. And I think a lot of people, a lot of folks have done that in terms of how they piloted AI. Um, they're trying to rewire everything. And so I I kind of advocate for meet people where they are, how they're currently working, um, give them to buy in, get them to actually start to use these things and see the value and let them be your research, you know. And I one of the metaphors I use, and not that's a metaphor, but uh a sort of strategic uh framing I use is we're working inside organizations, but it's not much different than uh if you were going to market with a new tool. And so think about your organization as the marketplace, if you will. Yep. And what are the pain points? What are the problems they're they're they're the challenges that they have? And how can we have this new technology come in and help solve some of those problems? So meeting them where they're at, you know, getting their sort of buy-in in the early early stages of it, how can you help them create better content, make better decisions, um, you know, look at analytics in a different way, all the things that we know AI can do, but talk to them about it and start to begin to understand how you can support them with the tool itself. And uh that means, you know, obviously skilling, obviously means um, you know, onboarding and creating communities of practice, make it be their idea, so to speak. Right, how people share those ideas. Um, anyway, point being that I think a lot of folks have been trying to solve the whole thing all at once. I don't think that that's reasonable in terms of transformation within an organization, and you have to have a strategic approach to looking at the workflows, uh looking at the the adoption first, and then kind of slowly moving towards how you can kind of uh impact and operationalize uh AI with uh the specific workflows.

Michael Hartmann 5:05

Yeah, I mean, I like that idea. Uh so for the audience who's listening only, I'm sitting here nodding my head all the way along through Paul's description there, because I think uh that I like that idea of thinking about this as a little bit of an internal sales and marketing initiative, right? Or go-to-market initiative. That's an interesting one. But your point about meeting people where they are, I think is a is an often missed one with kind of all change management. Um, I can't remember if we talked about this when we last spoke, but I I did some research on change management as I was working with a client um a couple of years ago, and I came across a Harvard Business Review article from like I want to say it was the late 60s, it may have been the 70s. I don't remember exactly when it had been revised. But what struck me about it was even though it was that old, right, the principles were still there. And I think, you know, the this idea that one size fits all is not realistic. I think, and I don't know if this is true or not, I would imagine there's sort of a bell curve, right? Of people who are like on the one end, you've got people who are gonna be like jumping right in, they're hot to do this, they want to change this. On the other end, of the people who are like totally reticent, why break something that's already working? And then like over time, like you've got this whole big chunk of people who are sort of somewhere in between. Yeah, and and so thinking that they're all on the one end, right, is probably not gonna be successful if you treat everything that way. Now you could take advantage of those people. I think that you were hinting at that, right? You were talking about pockets of community. I mean, if I think about like that pocket of community is the ones that are gonna really test the boundaries and then they can help advocate and teach others.

Paul Shirer 6:44

Yeah, exactly. I mean, I think a good uh skilling program. There's different levels we actually put together kind of a different levels of skilling program uh with one of our um, as we were initially rolling out, this is about a year and a half ago or so, um, that we really spent a lot of cycles on this. And um the skilling is yeah, giving them some free courses that are on YouTube or LinkedIn learning. That was obviously obvious or low-hanging route. Yeah, and then some certificate programs, maybe pay paying for a few things. But one of the best things you could do is identify those champions and and those those folks that are the super users that and and really they can train the train, you know, they can train the folks. They're they're happy to share all the practice. In fact, they're they're very vocal about it.

Michael Hartmann 7:27

It's an ego, it's an ego-based thing for them, right?

Champions Skilling And Communities Of Practice

Paul Shirer 7:30

So yeah, and it's a career path for them in some ways, too. They're seeing it as an in a an opportunity to build not only their skills and in their position, but also maybe even leap within the comp company or who knows, go somewhere else. But they they will be the leaders of a community of practice. So we've uh I've stood up probably about um, I guess three or four uh communities of practice now within companies. And um, you know, you just get them started in Slack or Teams and have those folks that really want to be the champions come in and they're very vibrant internal communities. I was actually surprised at how vibrant they would be.

Michael Hartmann 8:06

Yeah. I mean, I've I've seen the same thing internally. I mean I'm a big fan of like the internal communities for things like that. I think I think um the only thing I've seen that has hampered them is when there's um there's too much concern about what might be said inside those, right? Um it's kind of the same with it as when you have a say customer community, but like I'm always I tend to go like let it go. Like, and if it happens, right? First off, you're I think it's like the like teaching kids, right? They they fear doing something new, and it's usually the fear is way greater than the reality, right? That's right. And and I and um I think I'm all for like let it flow, right? Don't I mean set guidelines about the way that you can communicate that's acceptable? So that's a little bit different, but if like in terms of the content itself, okay, if they say this tool sucks, right? Let them say it. Ask why, right? Yeah, um, and let what I found is that then the rest of the community who doesn't agree will counter it.

Paul Shirer 9:12

And it's kind of like social media.

Michael Hartmann 9:14

Yeah, I mean, well, yeah, I mean uh with with with hopefully with some more guardrails than what you see. So um all right, well, so another thing you and I talked about is that you and I see this too, right? I've seen companies, I've seen comments about people sort of accumulating tools, agents, um spending sort of unexpected large amounts of money on um tokens, things like that with some of these tools. So when you you you use the word agent sprawl, right? How do you how do you think about that versus maybe a more um appropriate adoption of AI and that it might be more useful for organization?

Paul Shirer 10:01

Yeah, it's a great question. And you know, not having a perfect answer by by any means, but just uh observing what I've seen in kind of the evolution. Um, you know, we first saw, we just talked about the idea of having tools was the first thing, and then prompt techniques were everywhere, right? And everybody was sharing those prompt techniques, and that was kind of a nice, fun little age where people were just playing with these things. Um and uh then, of course, agents come around. And uh what just to kind of define what agents are for this point here, I'm making is we sure co-pilot agents within Microsoft uh 365 or the ChatGPTs within OpenAI or Gemini Gems and and things like that. Um now there are some other a little more robust agents that folks were building, but by and large, the citizen developed agents that are in play uh that are out there. And I don't want to I'll take it a little further. There were folks paying for you know more production level and for development companies to build some agents too. Um, but the the problem with it all was it was all done kind of in a vacuum and uh all done in terms of um we need this now, let's build it and let's we're not gonna think about how everything connects together. Um, you know, what my uh co my uh colleague is doing, or what this other department over here is doing, not gonna think about um you know all the governance or any of these things, and we're just gonna do it. In part, that was also because leaders were saying use AIRLs. They were freeing up budgets to invest in AI and they just said go. And so what we ended up having was all these agents. Um, some of them were duplicate efforts, a lot of them were lower quality citizen development agents, um, things that were you know helping like product marketing or sales outreach or content development or analysis, and they've been doing some things. Um, and uh and but they by and large um weren't being used in the most efficient ways, the most productive ways, and there was just a lot of waste. Um, so that's what I call that agent sprawl factor.

Michael Hartmann 12:00

Yeah.

Paul Shirer 12:01

We we saw this a lot at enterprise, but I I I think it was happening pretty much everywhere, even in mid-size, um, etc. But um that that's the agent sprawl. But I'm happy to go further and talk about, you know, you know what why that happened and what we can do about it and what's the next evolution, if you like, as well.

Michael Hartmann 12:20

Yeah, I mean, I think um I I definitely see it. I don't think it's isolated to, in fact, there's part of me that thinks that it's probably less of an issue in enterprise organizations because they're so hard to move. Um there's a part of me that thinks it's like in like early stage small companies where people sort of they just figure shit out and how to get things done, right? They're gonna be adopted with less of that four-letter word governance, right? Um yeah, so I'm curious to me, I are you seeing some of that more in enterprise organizations? Are you seeing more of it in mid to smaller companies?

Paul Shirer 13:01

It's definitely both. I agree with your point on um the you know, mid-size is the kind of the one where it's the least it's the most propensity for enough people to do some things, but less governance. So that's where it's happening, probably probably the biggest problem area. Smaller is just not enough people to mess it up anyway. So you're kind of yeah, yeah.

Michael Hartmann 13:22

And uh the implications are the Yeah, the risk, the risk is lower in terms of like the scope. Yeah, I get it.

Paul Shirer 13:27

Yeah, yeah. And then but enterprise, yeah, definitely saw it now. It they have, and I work with Microsoft, Google, um, SAP, AWS, and we're talking about big cloud enterprise. Sure. There's a couple other hyperscalers we work with and that kind of thing, and they're partners. But now what I saw there, what I've we're in the midst of is there was a whole fix my agent phase. So the citizen developed, and then we came in and fixed their agent, so to speak.

Michael Hartmann 13:54

Yeah.

Paul Shirer 13:55

Now um, so they were seeing, and then RI responsible AI started really coming down the pipe, right? And so, in particular on at the Microsoft area, uh, responsible AI is a huge part of now of the overall, just getting an agent into their system now is becoming a big thing. But um, the fix my agent was the thing uh for a little while. The biggest thing that's happening now is um that's this stakeholder and this one business unit over here is talking to this stakeholder because they have teams of matrixes uh you know across all these. They're very similar in a lot of ways. My my Microsoft's maybe a little more matrix-oriented. But um, they're now going, wait, you're doing this, I'm doing this, you're doing this. Maybe we can bring these things together. So they're getting pretty proactive about being bringing things together and not just working in their silos.

Michael Hartmann 14:47

Yeah, that makes sense. I mean, that's the one that it's

Agent Sprawl Governance And Waste

Michael Hartmann 14:50

the benefit of having a larger organization. You probably if thinking about Belker, I mentioned, right, there's gonna be just more volume of people at any one particular place in terms of their propensity to use the tools. So um so a curious, you know, um, and I think you you and I when we talked about like you're thinking about evolving, you're thinking about AI and how to incorporate it and implementing it, all that has evolved over time. So if I remember right, you you felt pretty strongly that you need to wire together end-to-end kind of automated workflows, call it magents, whatever. Um, but it sounds like you're I think in our conversation, it sounds like your perspective has shifted shifted. So maybe talk about like, yeah, what was your belief at kind of early and then how has it evolved and what like what is it that's changed in in that process?

Paul Shirer 15:39

Yeah, yeah. Thanks for that question. And then that is really probably at the center of my the thinking that I'm really really focusing on now with the teams uh that I work with as well, is um the reality of the concept of an end-to-end workflow where you can, first of all, just agree on the key steps of that workflow. Say it's a sales prospecting all the way down to nurture, down to some level of conversion.

Michael Hartmann 16:06

Sure.

Paul Shirer 16:06

Um, or a content pipeline or whatever it may be, especially in the market. Um the idea that there are quote unquote key steps and micro tasks that happen that are rinse and repeatable all along that cycle is a is a little more illusion than reality. You know, like it's it's you want it to be true. Theoretically, it sounds like it can be true, but then you get in to do the work and you find out everyone's different. You find out uh there's nuance to it, and there's like customization all the time that's necessary.

Michael Hartmann 16:38

Shadow, shadow processes, shadow systems, right? Yeah.

Paul Shirer 16:41

Yeah, exactly. You got it, yeah, all these different things that you weren't aware of, and you you you always and this goes back to all the days that we we tried to be to engineer workflows, right? We've been trying to be agile for years, or not just that not scratch that word agile.

Michael Hartmann 16:55

I don't really mean I meant it in the kind of the not the general certain not the methodology. I get it. Okay.

Paul Shirer 17:00

Yeah. When we first started talking about workflows, this is our old land. But the the point is uh that we um we've been trying to do this for a long time. And so we thought AI would be the forcing function in a way to get our stuff together so that we could have the right AI doing things and human in the loop or human on the loop and we can construct anything. But the the trial struggle was not only just that can we get something that is rinse and repeatable, but then getting human beings to be in the loop.

Michael Hartmann 17:27

Yeah.

Paul Shirer 17:27

It's like rallying cats, right, to come in and do this thing and to believe and to buy buy into it and all that stuff. Now, I'm not saying I gave up on that. I I do think there's uh certain uh pieces of the of the puzzle, the pipeline where you can come in and and and get some pretty nice uh wired uh workflows. Um and you know, having AI autonomously do some pieces of the puzzle that would be called agentic workflows. I think that's real.

Michael Hartmann 17:53

Yeah.

Paul Shirer 17:53

Um, but yeah, I just I just decided that, you know, the bigger opportunity and the thing that we kind of just rushed right past is just getting people to be better at using the tools that they have and expanding out the workspaces, which I won't go into that right now. I think we're gonna probably talk about that a little bit more. But um, that is what I've seen. I know it's contrary to everything you're seeing on LinkedIn. I get it. They're all workflow first. And I was writing about workflow first a year ago. I was uh I was talking to my uh my leaders and companies, and but uh as we've tried to do it, I just it just hasn't always played out the way we expected it to.

Michael Hartmann 18:31

It's I just want to make sure I'm clear. So when I hear you talk about that, I'm thinking like a workflow, like say it's let's say you can even agree on like your point about agreeing on what the steps are, right? But let's say you've got general agreement, you've got 10 steps, and you try to automate the entire process. Um, is that what you're talking about? Like okay. Yeah.

Paul Shirer 18:53

Yeah, I think it's um when I look at workflows, it's always going to be some mix of what AI can do, what automation can be done, and then humans take a piece of the puzzle and then they're in the loop. But it requires this official formal design of a workflow. And and that's you know, this conflict. So I've seen a lot of uh another, it's again, I was talking about this over a year ago, and I'm taking back just a little bit, is that um a lot of folks have said you have to invert, and that's what I was saying, you know, back then too. It's like invert from um people first to workflow first. And so instead of hiring people, and this has implications for how you change your workforce and who you hire and all that kind of stuff, you're hiring for the workflow versus having people do it and then have them use tools. So it's just and and again, I haven't given up on it, Michael. I I do think that there are some spots where this is useful, but I think that this sort of blanket statement around workflow-first mentality is is it's not as practical as it may sounds.

Michael Hartmann 19:55

Yeah, and the reason I ask you that, I I I remember seeing LinkedIn posts when I was still sort of, I feel like I was late to dipping my toes into using tools like ChatGPT or Cloud or whatever on an individual basis and kind of thinking about it as I was working on a problem or some sort of deliverable, right? I it wasn't at my normal, it took a while for that to become a part of how I thought about getting things done. But what I remember seeing is some people who had tried to do whether it was you call it agentic workflows or whatever, that had um like lots of sort of AI components and an overall automated workflow, right? Maybe there was some human in the loop component to it. Um, and it feels like people were sharing, like, this is what I'm doing, and here's what it looks like. And then it felt like it wasn't that long after where I started to like actually what we find is if a step something goes off the rails in one step, right, it it has this cascading effect. Um yeah, it's to me, it's like I think about like okay, if I'm this is gonna maybe offend some people, like if you're shooting a gun at a target, right? And it the tar the farther the target is away, right, any little variance is gonna be at the end when you get to that target, it's gonna be way off. And it feels like that's kind of what it was happening, is like people are like, oh, I'm gonna automate this whole thing, but then something would break here. And actually, what I think I remember is like not only did it break somewhere, it was harder to figure out where something went off the rails.

Paul Shirer 21:24

Yeah, yeah, it's I don't again, we would we probably need 15 different use cases to really search this. And you'll you know, if you're doing data consumption and you're extracting data and turning it into some organized form, and then that's then manifesting an email that gets pushed to your business leader. Maybe that's the uh a useful automation that probably would work most of the time. If you're having AI make decisions for you based on some level of criteria, I think your your example comes into play more. It's like, yeah, that could go off the rails. That's not right. And um and then you have the AI. For all its promise, and I I love it. Look, I use AI every day. I'm a this is what I do for a living. Sure. You know, the the generative AI portions are always they're just not quite right yet all the time. And so we can't just have it do things for us, um uh, you know, and do it in these automated ways. But um the as you add as you add human in the loop uh components to it, um the biggest problem we have there is then, so let's say that you have an end-to-end, uh, I'll just do a content engine real quickly. We'll just do a little campaign in a box concept real quick. Got it. So a campaign in a box, you would have a creative brief of what the campaign is. Um, you'd probably marketing a specific product or service. You have target personas, you have probably a message you can position you framework that governs that brand guidelines, all these things that can help you kind of put together a content campaign. And maybe your content campaign has a uh an ebook and then maybe has a one-pager and then has nurture it has reach out emails, nurture streams, ads, all these things that are kind of in that campaign that help that.

Michael Hartmann 23:04

Sure.

Paul Shirer 23:04

Well, if you put together an end-to-end workflow that says we're gonna pipeline this thing, we're gonna say AI is gonna do this part, humans are gonna do this part, AI is gonna come next, and blah, blah, blah. You put this whole thing together and you say, got it, we got this really streamlined thing. Our efficiencies are gonna go down, we still have quality gates, all these things are great. And then you um then you go put that into practice. Well, you've got writer X over here that has to pick it up after this. Where did it go? Where's the project management tool we're gonna use? What's the status of this thing? Well, now do we have to put a project

Why End To End Automation Breaks

Paul Shirer 23:40

management tool in this thing? Okay, and and people have done that. And then it's like, what's next after that? What's next after that? And so what you thought was this theoretically easy end-to-end thing, it started got it started to get really complicated when there was new writers coming in. Oh, that person was fired. No, they're not here anymore. Oh, that it's like it's like, you know, there's just the idea that it's just gonna somehow work, you know, um, you know, that's the problem. But yeah, you try.

Michael Hartmann 24:06

Yeah. So and you so you hinted at this uh a few minutes ago about the idea or con concept of um I think you call it connected workspaces. So rather in in that as an alternative. So first off, maybe like that you did earlier, like what's the what's the definition of a connected workspace? How is it different than some of these other scenarios we talked about where things seem to be uh appear to be, you know, connected but aren't really right? They're disconnected. Um, and then what does that look like? You know, do you have an example or something where you can talk about in the practice? Maybe maybe what you just went through as a variance.

Paul Shirer 24:45

Yeah, we I think we've been evolving to this in general with different solutions of late. And I'll try to lay this out. And without having like visuals and you know, kind of get looking at these examples, I think it might be a little hard to follow, but I'll do the best I can to kind of lay this out. Um, we've talked about some of the um underlying problems, the the agent sprawl, for example, right? You think of a marketing team or a sales team and or any go-to-market team, right? And they are uh working, they they started off, they got these tools, they're starting to work in silos, they're building all these, and we got the agent sprawl issue. Um you then go, okay, well, why don't we create workflows that will, again, we just talked about the workflows, these are all part and parcel, where let's end to end, we're gonna have agent X come in here, and the loop do Y and things like that. So that was another sort of address way to address the kind of the work to be done in that area. And we've already talked about some of those problems. So now what we have is like co-workspaces that are are growing up. So popularity of Cloud and co-work and Microsoft has its own word, um, um, its own co-workspace, and we'll see more and more of these. But these are really personal workspaces still, where people are doing work and they're doing in all their one-off ways in their own idiosyncratic personal ways, which is what they should do. But what I think is uh emerging is you still have marketing teams, you still have sales teams, and you even have teams within those teams that do work together. And so I'm what I'm seeing is this concept of a connected workspace where it's built around a specific team's needs, and um, it's anchored in all the the data that they work together with, the documentation they work with, the processes too. But shared context. It's a shared context. And so one example we've been building out recently has been around um product marketing teams um that also, of course, do activation campaigns. They also do sales enablement and things like that. So you get it, you get it acrossroads and you go, well, aren't there some tools out there that we can just buy? And then you go down the buy world and you go, like, well, no, not quite right. Doesn't quite fit me. It's not exactly what it was. And then then what do we do with all of our data? Like, is that just all over there? Like, what are we doing with it? So what I'm seeing is, and we're building out a couple of these recently, they've been really resonating with clients, and so we're happy that that's happening. That's always good.

Michael Hartmann 27:14

Market validation is always good.

Paul Shirer 27:15

Yeah, yeah. So, and so this is kind of an exciting area where we're building all those agents that work on are disconnected or they were just plugged into M360 Microsec 365, or they were GPTs or whatever those. Instead, we're building one space where all the product marketing needs are there, where you can build the vision and mission statement for the product, the um the overviews, the features and benefits, the ICPs, all there, but it gets stored in a data layer. So it's not just knowledge sources that are unstructured, like all these documents that are you're putting in there. Okay. We're storing it in a day layer. And then they go to create um blogs or they create uh ebooks and things like that, and that's all getting stored in the day layer. And so are all the chat sessions. So what we're creating is the really this completely connected space, a round of business function where all the rich data is being built up over time. And so when you go to each agent that builds something, whether you're building the blog or you're building the draft of a of an email sequence and all that kind of stuff, instead I have to reground every one of those agents in the documents, it's pulling from the same data layer across everything. And uh it's creating a much more powerful way to do work and do it at scale. And I think that I don't know where this all will go. I'm just saying what I see, Michael, and what we're building.

Michael Hartmann 28:31

I swear, like I've given up trying to predict because it's like it feels like every time I turn around, there's some new thing out there. And if I go back to like we talked about like the adoption of different types of companies, I mean, I know of at least one organization that I've heard where developed their own sort of um tool for people to use, you know, uh um LLM kind of based on some of the other um models. Um, and then using IP restrictions like blocked all the other ones. Well, new ones are popping up all the time for specific use cases and people are getting around it, or they turn off a VPN or whatever, right? It's like, I mean, that's the stuff that's really happening out there.

Paul Shirer 29:11

Yeah, the shadow AI, I was I'm I've actually got a post that's gonna go out in the next few days. Um, I've been doing some a research piece around it first, and I'm gonna post around it uh on shadow AI and what to do about that. And I think that we're seeing a first, it's like, don't ask, don't tell. And then it's like, then it's like, let's be punitive. Let's like, uh well, you know, do all that. But then I think the next phase, which I believe we need to look at very closely, is let's let's let that be a tell on the market. What do people want? Because these are the people that want to use things, and so ask them. You know, embrace it, intake what they're doing, and then maybe you could put together a plan about which tools you're going to buy to support them, even though Shadow AI is still going to be there, at least you're embracing what's happening and know that that's not nefarious. It's like, I just want to do my work better, I want to move faster and to embrace it, in other words, yeah.

Michael Hartmann 30:06

Yeah, no, I mean I'm again kind of going back to this. Like, I I would rather be have that be in the open and know about it than to think it's not happening because you know, and then people feel like they can't talk about it. I think that's a a worse kind of outcome. You you you talked about a data layer, and and I don't I want to maybe dive into that a little bit because I'm not sure this is exactly what you were talking about. So if not, that's fine. We can we can kind of connect the dots here. But yeah, one of the challenges I think everyone in ops would tell you is like our data sucks and go to market, right? But there's marketing sales, whatever, especially in B complex B2B sales stuff. Um how two-part question, because I I have a strip. So, how important do you think that data quality is to enabling these kinds of tools? Um, and then one thing that I'm would like to see, I haven't heard a lot of people talking about this. I'd like to see AI being used to help improve the data quality and completeness. Like just think is is it like so in other words, don't wait for the data to be whatever. We pick the level you want, right? Um before you start using AI tools for other things, but like how can the AI tools help expose and correct the data? Like, what's your take on all that?

Paul Shirer 31:21

Yeah, that's a great question, Michael. And I think um to to just say it um and state that data's a problem is one thing, but to then unravel like what's inside all of that as you work through it. And I think um, you know, it it's a problem in so many ways, and it's always been a problem. It's just that AI is now exposing it because as you try to really leverage the power of AI, which essentially it's like it just needs data, it just loves data, it can't do anything without

Connected Workspaces With A Shared Data Layer

Paul Shirer 31:50

data, um, really to really be effective or to be differentiating or useful. So you just could go across any area of the go-to-market um space, you know, all the way to up and down the customer journey, if you will, and you're gonna find, well, you can't personalize uh outreach if you don't have data, right? You can't um follow, you can't um have on-point battle cards and sales enablement if you don't have data. I mean, you can wing it as much as you want, um, but you're gonna need that intake. And so um the the struggle we have is that data uh in an unstructured form is not all that useful. It has to be structured, and we know that. But um, so how do you go about doing that is the question. Um, and how do you pull from all these sources? Well, um, sometimes um it's just a matter of like you can't even get access to a lot of data. Like, you know, yeah, sometimes it's not collecting the right data. Think of all the just the entire trail, we're not collecting all the data properly. There's that. But sometimes even within organizations, it's just so many systems, whether it's within Salesforce or you know, different CRMs, um, or maybe it's all your ad data over here and your customer intelligence or feedback loops, the market research that you do that's over here. And um, so we're like just recently I've been working on um competitive intelligence a lot for whatever reason. I have uh two competitive intelligence solutions. Um, and it's really rearing its ugly head around like, well, we have external data sources we can grab with AI, which that's available on a lot of levels. Sure. Where are all your internal sources? Like, well, there's documents just gathered everywhere. There's there's CRM data, there's sales force casting data, there's our customer feedback data, and it's everywhere. Um, but the challenge is then um how do you get access to it, pull it together, normalize it so it can actually be used, and then you can actually have AI do all kinds of levels of rag responses and in you know ingestion of it and do things with it.

Michael Hartmann 34:01

Right.

Paul Shirer 34:02

Um, but it's just it's a long process. And it's the thing that then the the biggest thing I would say is this, and like if you go macro, sometimes we can dissect these things, but if you want to just put it as a big equation, um data is super important. Leadership doesn't want to invest in it because it's not they don't understand, it's not sexy, it doesn't make sense, right?

Michael Hartmann 34:23

Right. The the immediate immediate, there's not a there's just not an immediate, obvious value to putting in the effort and cost yet. Totally.

Paul Shirer 34:31

They don't want to invest in it, but it's super important, so crappy output. It's it's the simplest equation in the world. But um it, you know, I think that um the good thing is that AI has exposed a lot of things, and we talk about this a lot, exposed our workflow process issues, we've exposed um you know, talent and skilling issues, is exposed um data issues, and I think that's one of the best sort of value benefits we've gotten out of AI.

Michael Hartmann 34:57

Yeah. I I I think so. Like I still I think I still feel like there's a play here to use AI to help with that data quality. I just don't I don't know what it would look like. You know what I mean?

Paul Shirer 35:12

Yeah, um well if I agree. No, no, AI, it's one of its beautiful things it can do is it can organize, right? It's taxonomy, uh I it's it's so much better than us, and it's so much faster. And so I agree with you on that. Um in fact, we were doing a project, there's another project we're working on where it's a um first thing we're doing is we're just doing some data auditing. And uh we want to build some dashboards for um OKR. So you know, the um objective is in key results for a quarter. And so the first thing we're doing is we're going through like 15 different dashboards that have already been created. Just like nuts.

Michael Hartmann 35:52

And uh dashboard is a four-letter word to me.

Paul Shirer 35:55

Yeah, it's it is at this point, you know. If you do it right to some degree, but the problem is people never go, there's a lot of problems with dashboards.

Michael Hartmann 36:01

Yeah. Anyway.

Paul Shirer 36:02

Nonetheless, the we can use AI to go into these data systems, which you know, some the inconsistencies of data across all those systems, those dashboards. We are using AI to go in and extract all that and create a master taxonomy, like a sensible master taxonomy that can come all of that in order to then build the the next dashboard. Um, but then we'll also have push, you know, notification, you know, push information and things like that will come off of it. Um, but that's one way AI can just intelligently sift through it and suggest a better taxonomy. That's a simple sort of thing.

Michael Hartmann 36:35

Yeah. Well, I mean, I'm sure somebody's got to be doing this right. A very common thing for uh in the ops world, both marketing and sales ops probably is get an external list, you gotta normalize country or state or whatever, right? And I'm like, that's a or or normalize uh job levels, right, based on titles. Yeah. Um, I mean, I think that's a powerful way of using it, right? Doing that that organization for you and doing it dynamically, right? So over time, like you it adjusts based on what the real data is. Um flat as it may be.

Paul Shirer 37:10

Yeah, absolutely. I think that's that's a great point. And um the well, I'll just say that my uh the basis of the business I'm building, Infinite Ideas AI, is all based on the idea that AI can create new and unique data in the market. Um and so that's yeah, my not only agreeing with you, like it's like um to me, I I was looking at the way we um all the signals that are out there in the world, you know, not just around conversations on Twitter or Reddit and things like that, and sort of that typical sentiment analysis and things like that, but just taking the everyday sort of things that are happening out there and turning it into real signals, um, you know, what's the velocity of investment in a specific space? Um what's the what's the um if you knew what your current product was and you and you saw um something happening, some news item hacking happening in the market, um, how related is this news item to my product line and should I do something about it? And you could score that, right? Right.

Michael Hartmann 38:13

So like a like a risk assessment or an opportunity assessment. Yeah. Yeah. Yeah, I like that. It's interesting your point, like the this is the other thing that really it wasn't until I was talking to someone who was a writer at one point, and I we were talking about something else, and I asked through in the question, like, oh, what do you like? Because I'm assuming most writers detest right these LLMs for what they do. And and this person was like, Well, we do breaking news, so they can help us make our writing better,

Shadow AI As A Signal

Michael Hartmann 38:40

but like they can't break new news because there's nothing to go on.

Paul Shirer 38:43

That's right. That's right. I do love that part. And and uh when people were talking about content creation first, that was kind of an area that I was like, well, you can always do new reporting, you can always do new things. You can there's things happening everywhere and um that are unique that um yeah, that LMs are not trained on, not yet.

Michael Hartmann 39:04

Yeah, well, I mean, there's this whole thing I saw that like if you're a top performer and whatever you think you're gonna use AI for, uh, the likelihood that you're gonna stay a top performer is actually pretty low. Like it's gonna bring you down to the median, right? And if you're a poor performer, poor it's gonna help you up. Like, so is it in a lifetime? I mean, I mean, I was like, oh yeah, that's like, oh yeah. Like, so if you're like if you're a top performer, should you be using AI? Well, maybe you should be using AI differently. This like Ed like it begs a lot of interesting sort of questions that go beyond that.

Paul Shirer 39:36

Great point on that one. It's an equalizer. We used to say that when we were playing, I played a lot of basketball in my life. I still play a lot of basketball. Um, and when we played basketball in the rain, it's like, well, they want your speed advantage, that's gone.

Michael Hartmann 39:49

Yeah.

Paul Shirer 39:51

There's no first step in the rain. So uh, you know, everybody was equally the same speed in their yeah, yeah, yeah, yeah, for sure.

Michael Hartmann 39:58

Outdoor basketball, or if there's a wet lot of wind, right?

Paul Shirer 40:01

Sparkman, yeah. Football is the same, right? If it's raining or snowing, like it doesn't really matter who was better anymore.

Michael Hartmann 40:08

Yeah. Yeah. I uh yeah, like all kinds of good sports analogies there, I'm sure. Um, so I want to like I'm gonna kind of take two two ends of sort of the the the whole process of AI adoption, and I would like to get your thoughts on sort of both because it feels like there's a linkage. Um the first is for the people who are listening or watching this, right? How should they be thinking about whether so if they're leaders, how should be they thinking about how to get AI initiatives started, right? What should we be doing, maybe what should they not be doing? What are some guiding principles there? Um, should they be looking at doing things like building versus buying, especially if it's like we call it agenc stuff? I hesitate to even use that term. But then the one that keeps coming up regularly now is then how do they set things up so they know that they can measure the the the benefit from a I think really from a financial standpoint. Maybe there's non-financial benefits too, but like what's your take on that? That's big-ended, open-ended question for you there.

Paul Shirer 41:10

Yeah. Yeah, that's good, good, good questions. Um, and I think that that's what we're that we're in that era now. Like we had the fun, there was sort of blink checks written. It's like we'll figure this out later. The competitions are doing it, and so we got to do it. If we don't do it, we may be left behind, fear factors, all that stuff. And now it's like, okay, wait a minute. Where's the real um where's the real ROI and uh how should we really be deliberate and have some sort of method to our madness and how we actually make decisions around this? Uh, which I love, it's which kind of the area I focus a lot on. And so um now having an expertise in that area actually makes a difference. I don't know. It's kind of interesting because um I I always like look at like how many people are actually um Googling or what's the trends around AI strategy or something like that, you know? And we see that uptick. I kind of follow that a lot being in the strategy space, and it's like I think they're starting to get the point that we might have to think about this and that kind of thing. But um so um, you know, when you're looking at um if we start with um, I guess a buy versus build, I'll kind of kind of give you a couple of anecdotes on this uh to kind of show you the difference and in might help guide the decision. Um it's really gonna come down to, and this has always been the case in buy versus build is not a new thing, right?

Michael Hartmann 42:30

I think I know where this is going. Okay, go.

Paul Shirer 42:33

Yeah. You know, it's like um how bespoke, how how bespoke to your business does it need to be? And yeah, um, because there is going to be um a s a place, there's gonna be a time when it's it's just needs to be more bespoke. It's not like it's one size fit all will fit all. It's just we know that's not true. Um, even at the more precision like level and more nuanced and niched oriented solutions, it still can come up short. Um, I'll give you an example of a client I just met with yesterday, actually. Um and it was interesting. I was glad to hear it. Um, you know, it was like, look, I can go to compete intelligence tools right now. I know I can do that. I told you I was kind of working on a couple of compete uh solutions lately. But it doesn't help me because it's not best enough to me. It doesn't, I it doesn't know, it doesn't work with the data that I really wanted to work with. Um, it doesn't deliver it the way I need to deliver it. It doesn't have the lenses or the views or the outputs and the ways I need to see it. So there's just one thing after another. And so I think if you're going to uh do a buy versus build situation, that's the number one thing that they're looking at is um, and sometimes it's um there's nothing wrong with uh this is I remember getting this question a long time ago. It's like there there's really nothing wrong with buying right now and then building later. Right. You know, you can gap it, you know, you can get and you might learn from the buy. Experience to then maybe build later too. Um so sometimes a decision is just better than nothing at all.

Michael Hartmann 44:06

Yeah. Well, I think what uh this so what I expected you to be talking about is the trade-off, right? For getting something more spoke bespoke is that now you're taking on sport and development maintenance.

Paul Shirer 44:16

You know, that's a great point. I do want to when you're ready, I want to comment on that. Don't that's it, that's it. Okay, good. Yeah, uh, yeah. No, that's I love that point. And this is the crazy world we live in um with vibe coding, you know, and then with agentic level coding. And it's like it's like where you would have to buy in the past, you you may be able to spin something up that's reasonable. And then I mean like someone who knows what they're doing. I don't mean like straight go out to uh you know Lovable or Bolt and just like put up a prototype because there's a lot of vibe coding, it's just prototypes, they're not working. But you know, doing a lot of development uh with working with developers a lot, you know, with cursor and and now with cloud code and things like that, you can get things pretty bes well, bespoke for sure, but you can get it on point and in market with some level of MVP pretty quickly now. So your the build versus buy equation has changed because of that. I'm not saying that that's for everyone. I'm just noting that that is uh it's changed the equation.

Michael Hartmann 45:16

Fair enough, true. Yeah, yeah. No, I think that's that's a good point. But there's still like that is something you have to take on, right? It's not a zero cost.

Paul Shirer 45:22

It's technical debt. It's technical debt that you don't know where it's going. That's the the big unknown. You will never know where something you're building is going to go. And yeah, then you need good product managers. And I think that's always a way un misunderstood skill set in the world.

Michael Hartmann 45:38

Yeah.

Paul Shirer 45:38

Try managers know how to evaluate something for its for its business impact. They know how to t-shirt size things to know how big they are and how long they're gonna take and how much they're gonna cost. They're gonna know the technical debt, but most organizations don't have that um that muscle, so to speak. No, I think they think they they think they have it.

Michael Hartmann 45:58

Yeah, no, I think so too. Uh so so okay, so there's a buy versus bill. Like, how do you how are you seeing companies that are successful in measuring the impact of AI initiatives?

Paul Shirer 46:13

Yeah, another great one, right? And so you can build all the RRI calculators you want. There's a lot of them out there.

Michael Hartmann 46:20

Um all models all models are inherently flawed, right?

Paul Shirer 46:24

We can vibe code one right now.

Michael Hartmann 46:26

Yeah.

Paul Shirer 46:27

Right. No, no, but uh, but seriously, on this, you know, there are some things you can do before and after. I mean, there are clear things where like KP, like conversion rates on a sales funnel or something like that is a clear thing that can make a difference. Um, if you got uh outreach, um like you've got volume and quality equations with, say, outreach and sales, you can get a hold of did the quality versus the quality game work work? Did where was the personalization of this AI helping me in this matter? Um you can um you can do obviously time to time to be able to make decisions. Like decision time is huge when it comes to data and how you can ingest it and things like that. Those are things that I think you can get a hold of. But a lot of it's just gonna be pure qualitative. And I think a lot of people are scared of the qualitative, but I love it. I mean, I I I like the idea of go go ask your people. I mean, what's wrong with that? You know, yeah.

Michael Hartmann 47:19

I I uh the analogy I I think you and I talked about was it's sort of like attribution reporting, which is the marketing. Oh, yeah, yeah. And and and I think it's right. It's like it's it's rife with all kinds of data issues and challenges with measuring and you can't capture everything. Uh, but I'm with you, right? I'm a big believer that the the a lot of marketers, uh, and part of why I think they have their their percept the perception of them sometimes in organizations is not great. It's because they forget to use the skill they should be really good at, which is storytelling within their organizations.

Paul Shirer 47:49

Storytelling, there you go. That's awesome. I love that point a lot. I'm I'm gonna bank that one for sure. Go ahead because I mean that's the truth. And you know, have all these uh what I'm dealing with a lot is uh CEOs or leaders in orgs that have to get a board ready thing. You know, they have to it's like it's gonna be storytelling no matter what you do here, because there is no way to say anything any right now that would make everyone happy. I mean, there are investments being made, there are lots of proof of concepts, there are a lot of things that we're trying to do and understand, but you have to make the investments to get to the next step, otherwise you're gonna be left behind. And so it's not the thing that a board member wants to hear, or like someone who's paying and making the investment, um, because it's not what it's it's not how they want to work and anybody does want results, right? But the reality is this is I I've actually said it the other day, it's like I was like, wow, yeah, it's right. Like this is kind of a cool concept. We're the whole world is in in an RD phase right now.

Michael Hartmann 48:46

Yeah.

Paul Shirer 48:47

We're we're making investments and things that we just need to figure them out. And we have to kind of we're all collectively doing it together.

Michael Hartmann 48:54

Well, and I know of at least one person on LinkedIn who's been uh posting regularly in the last two months, probably. Here we are mid mid-June 2026, about like how these organizations are, yeah, the CFOs are all of a sudden seeing large

ROI Build Vs Buy And Ops Leaders

Michael Hartmann 49:10

chunks of money going out the door, you know. So say they're using uh pick the clay is the easy target here, right? Yeah. People are like, oh, look at all this great stuff. We can replace something like Zoom info and do this waterfall like um enrichment process, and then the bill comes in for what it costs to do that. And it's this hidden cost that I think are going to drive the there's gonna be the forcing function that's gonna push people to go like we've got to figure out some way of measuring the impact. Um, because otherwise there's just too much, too much money leaking out of the organization.

Paul Shirer 49:44

Yeah, totally. And or and and not only that, modify behavior because of it, you know, because token costs is really are becoming huge. Um, more than the token war or like the uh the model war, they they I felt like they all figured it out. We'll just keep charging people and keep promoting fear that they don't get the latest model.

Michael Hartmann 50:04

Yeah. Um yeah. Anyway, well, it's been great. This we could like maybe let's let's wrap up here. And and it's too bad Mike Rizzo couldn't join us because I think he would like us to ask he would want me to ask this question if if he or he would if I if he was here, which is I think he you and he, I think the two of you talked before, uh kind of he said that you had have a similar view as him and some others that like feature CMOs are gonna come out of an ops space, uh, or at least have a background in ops. Um so curious, like is that true for you? And then if so, the sense I get is you think that that like AI, like a lot of this AI activity is gonna be something that might accelerate that process.

Paul Shirer 50:49

Yeah, yeah, yeah. Completely agree with Mike on that. Um and I think it's um it's a macro thing across leadership, and but it it's particularly in this area. Um you know, leaders of the past, and I actually did some research on this because I was wondering about what's the shift in terms of technology driven or oriented leaders versus you know like the classical, like maybe financially oriented or some sales-oriented leaders, like you look at C shows and things like that. Um, and it is shifting considerably, and you see in a lot of key companies, um, you know, like the Nvidia's and the uh even Microsoft when it shifted to more of a uh tech oriented, they they got their stuff together. Microsoft was becoming a joke, right? And then seven, eight years ago they shifted and now they're they're getting track. And I see we see this everywhere. And I think it's the same within marketing ops and marketing. Um, I mean, how can you possibly lead an organization, right? But not having some felt or understanding, like you gotta feel things. I was like the the I don't know if you've seen white man can't jump, it's like a movie.

Michael Hartmann 51:58

It's been a long time, but yes, I know you're talking about, yeah.

Paul Shirer 52:01

Yeah, it's like you you can feel you can't you hear Jimmy, but you can't feel Jimmy when he was talking about Jimi Hendrix, right?

Michael Hartmann 52:06

So yeah, yeah. I don't remember the scene, but that's uh it's a true statement, I think.

Paul Shirer 52:12

Yeah, you know, that's the thing, that's the difference. Like you can't go into like just hearing it and somehow you gotta feel it. You you have to have been there. And I it's always hard for me to kind of talk to people, like explain to them why you can intuitively understand how something's built. Well, if you built something a hundred times, you just know you just can see it and you know where it's going. Kind of you're not always right, but you you got a real good sense of it.

Michael Hartmann 52:34

Yeah.

Paul Shirer 52:34

And I think if you don't have that, you're really in trouble. And it just I can't see how it plays out. Um, there's one other piece of that puzzle. I do still think you need leadership skills. You know, like leadership skills are a thing. Absolutely. You've got to know how orgs work, you gotta know how to um sell something internally. Um, you gotta know how to work with people and lay the groundwork. Um, uh all those things are, you know, everyone we know about leadership skills, but that a powerful combination of having that marketing ops background and knowing how this thing functionally works and feeling it. Um, plus the leadership skills, I think is the combination.

Michael Hartmann 53:11

Love it. Okay. Well, I'm glad we got to get get into that question a little bit. So hopefully people will stick around and listen to that. That's an important piece. Hey, so Paul, again, thanks for joining us. Um, if folks want to continue, like they're interested in continuing and going deeper on some of these topics with you, what's the best way for them to do that?

Paul Shirer 53:29

Yeah, uh, go out to LinkedIn, uh Paul Shire, P-A-U-L-S-H-I-R-E-R. I'm the only Paul Shire, I think, out there. Well, there was one other. He's got a he's got a financial planning uh company, and uh we talked. And so he's helping very well. It's a good guy. Um, but no, Paul Shire, I'm the other, you know, literally Paul Shire. And then uh Infinite Ideas AI is the uh business I've uh launched and um would love to even hear even feedback on that if you all get a chance to take a look at it.

Michael Hartmann 53:55

Fantastic. Again, Paul, thank you. Thanks to our audience out there for continuing to support us. We are always grateful for your input and and support. And if you have ideas for topics or guests or want to be a guest, you can reach out to Naomi Mike or me, and we'd be happy to get the ball rolling. Until next time, bye everybody.