Tag: Human Judgment

Why MMM Still Demands Clean Data and Human Judgment
I see marketing mix modeling (MMM) becoming easier to access, but I do not think it has become easy to get right.

After several conversations about MMM adoption, I keep hearing the same concern: “We believe in MMM, but we do not know how to get started.”

My answer is that open-source platforms have lowered the barrier to entry in a meaningful way. What they have not lowered is the level of expertise required to produce results that are trustworthy, explainable, and useful for decision-making.

Open-source MMM has changed the starting point

I am seeing MMM adoption accelerate because marketers need more durable measurement methods. Almost half of U.S. marketers expect to invest more in MMM over the next year, and many now rank it as one of the most reliable measurement approaches available.

The open-source shift is real. Three production-grade libraries now give teams a practical way to approach MMM across a wide methodological spectrum.
- Robyn (Meta, R): I see this as the most approachable starting point because it includes automated hyperparameter search through Nevergrad, Pareto frontier model selection, decomposition, and response curve plots. It is also the one I use most often because it is highly customizable.
- Meridian (Google, Python/TensorFlow): I view Meridian as a more rigorous option, especially because it uses Bayesian inference, geo-level priors, and principled uncertainty quantification. The tradeoff is a steeper learning curve.
- PyMC-Marketing (PyMC Labs, Python): I consider this the most flexible path. It offers a full probabilistic model that comes closest to academic-grade Bayesian MMM, but it also demands the most statistical fluency.
This generation of tools has removed the old $150,000 to $500,000 consulting gate that used to be the primary path into MMM. A team with R or Python expertise and reasonably clean historical data can now run a model in-house.

Open-source R and Python tools have lowered the cost of starting with marketing mix modeling, but the expertise needed to produce trustworthy, actionable MMM remains the real ceiling.

The caveat I always make explicit is this: “free tool” does not mean “free model.” The software may be free, but the domain expertise needed to configure it correctly is not. That expertise is a major part of the value.

The vendor landscape is crowded and complicated

I also see a fast-growing SaaS layer built on top of open-source MMM. To evaluate it clearly, I find it helpful to separate vendors into a few practical groups.

Data-layer-first vendors

Platforms like Rockerbox and Northbeam started with attribution and data collection, then added MMM. Their advantage is usually pipeline speed and data access, not deep modeling flexibility or customization.

Measurement-first vendors

Platforms such as Measured, Analytic Partners, Ekimetrics, and Nielsen Gracenote tend to offer more rigorous modeling and enterprise-grade capabilities, usually at a higher price point.

Google Meridian and GA360

I think Google’s decision to open-source Meridian is both a generous contribution to the field and a strategic move. When a walled garden funds and packages a measurement methodology that can be used to evaluate its own channels, I believe it is worth maintaining healthy skepticism about priors, defaults, and assumptions, even when the code is transparent.

Open-source MMM tools now span a clear trade-off: Robyn offers the most approachable starting point, Meridian adds Bayesian rigor, and PyMC-Marketing pushes deepest into statistical flexibility.

The practical vendor question I keep coming back to is simple: who owns the data layer, and does that ownership create conflicts in the modeling layer?

Challenge 1: Data access can quietly break MMM

I think data access is the most underappreciated MMM implementation blocker. A well-specified model needs more than a quick export from a dashboard.
- I usually want two to three years of weekly data as a baseline, so the model can capture at least two full seasonality cycles and enough spend variation to learn from.
- I need consistent channel-level spend granularity, not just a broad “digital” bucket. Search, social, display, video, and other channels need to be separated.
- I need offline channels such as TV, OOH, radio, events, and direct mail, even though they often live in different systems, belong to different teams, and use incompatible time periods.
- I need external covariates, including macro indicators, competitor activity, pricing data, and product launch calendars.
- For B2B, I often need even more history because longer sales cycles and lower conversion volumes make the data requirements more demanding.
In practice, I often find that the real blocker is the six-week data archaeology project that happens before modeling begins. Finance owns revenue. The brand team owns TV. The agency owns digital spend. A spreadsheet from 2021 may be the only record of trade promotions.

The model is only as good as the data archaeology behind it, and that is rarely the part anyone highlights in a vendor demo.

Challenge 2: I still have to roll up my sleeves

AI assistants have lowered the syntax barrier. They can scaffold a Robyn run, generate a Meridian configuration, or help debug a PyMC model. What they cannot reliably do yet is make the judgment calls that determine whether an MMM is credible.

Rows of illuminated data cabinets and paper files stretch into the distance, capturing the pressure on marketers to turn fragmented customer data into a smarter performance engine.
- I still have to decide where to land on a Pareto frontier across hundreds of model solutions, balancing NRMSE against DECOMP.RSSD tradeoffs.
- I still have to know whether Nevergrad’s optimizer has meaningfully converged or simply landed in a local minimum.
- I still have to configure adstock transformation parameters, including Weibull shape and scale or geometric decay, so they reflect realistic channel behavior.
- I still have to diagnose why a model gives a channel an implausible contribution and decide whether the fix is a prior, a data correction, or a variable exclusion.
In other words, if I try to vibe code my way into MMM, I may end up with a model that appears to work but is wrong in ways I will not catch. The scripting is not the hardest part. The real work is validating the output, including using channel-specific incrementality experiments to calibrate the model.

Challenge 3: Human expertise is not optional

Even if the tools mature enough for AI to run a competent default MMM, I still see human expertise as essential. The irreplaceable work is encoding business context that no model can infer from the data alone.
- Adstock and carryover context: I need to know whether a TV buy carries over for four weeks, paid search carries over for three days, or a brand awareness campaign decays over months. That knowledge usually lives with channel experts, not inside the dataset.
- Saturation curve shape: I need to recognize when a channel is probably approaching diminishing returns before the model says so, and I need to question the model when it suggests something implausible.
- Guardrails and anomaly handling: I need to explicitly model or flag COVID troughs, product launches, pricing shifts, and macro disruptions as structural breaks. AI does not automatically know a client had a pricing crisis in Q3 2022.
- Interpretation sanity checks: If a model assigns 40% of contribution to TV for a brand spending $2 million on TV, I need the experience to say, “That feels wrong,” and investigate. That intuition is earned, not computed.
- Organizational translation: A technically correct model has little value if I cannot explain why it recommends moving 15% of search budget to CTV in language a CMO and CFO will act on.
I start with the groundwork before the model

The best place to begin is not the model itself. I start by understanding what data is needed, who owns it, and who can help interpret it in the context of real marketing decisions.

None of that is quick or easy, but it is essential if I want meaningful insight from MMM, whether I choose an open-source library or a subscription-based platform.

As a practical first step, I would download Robyn’s demo script and experiment with sample data before applying MMM to my own business data. That kind of hands-on testing makes the strengths, limits, and judgment calls much clearer.

Inspired by this post on Search Engine Land.
July 9, 2026
Why I Judge AI Deliverables by Outcomes, Not Effort

When I think about AI deliverables, I keep coming back to a simple scenario: a client receives two pieces of work.

Both deliverables solve the problem they were hired to solve. Both are accurate, useful, and tied to the same business outcome. The client is happy, and from the outside, there is no meaningful difference in the results.

Then the client learns that one took 20 hours to create, while the other took 20 minutes. That is when the uncomfortable questions begin.

Was AI involved? Should the faster deliverable cost less? Is the person who completed it less skilled because they found a faster, more efficient way to reach the same result?

What I find most interesting is how differently many of us react to AI depending on which side of the transaction we are on. I love using AI when it saves me time, but I also understand why customers can feel uneasy when they discover AI helped create something they paid for.

I recently ran a LinkedIn poll asking a simple question: if the outcome is great, do we really care how it was made?

The responses reinforced something I have been thinking about for a while. Many of the strongest objections people have to AI are not really about quality at all.

The Time vs. Value Fallacy

I think part of the discomfort comes from the fact that we have spent decades tying value to effort.

Long hours feel valuable. Fast work feels suspicious. Struggle often gets mistaken for expertise.

The harder something appears to be, the easier it becomes to justify the price attached to it.

There is an old story about a ship engine that stopped working. After multiple failed attempts to repair it, the owners brought in an engineer with decades of experience. He inspected the engine, tapped it once with a small hammer, and the machine roared back to life.

His invoice was $10,000.

The owners were furious and demanded an itemized bill. The response was simple: hammer tap, $2. Knowing where to tap, $9,998.

People debate whether that story is true or just a useful tale for people like me who believe in value-based pricing. But whether it really happened almost does not matter. The lesson still holds.

People are not paying for the tap. They are paying for the expertise behind it.

That is what makes AI such an important topic for me. It forces us to confront a question many of us have avoided for years: are we paying for expertise, or are we paying for visible effort?

Those are not always the same thing.

The Objections That Actually Matter

To be clear, I do not think every objection to AI is unreasonable. I have shared plenty of my own concerns, and some of them are serious.

In fact, I think the strongest arguments against AI have very little to do with how quickly something was created.

Risk matters. Hallucinations matter. Bad recommendations matter. Compliance, privacy, and security concerns matter. Accountability matters.

Those are legitimate concerns. What stands out to me is that none of them has much to do with how long it took to create the deliverable.

They are questions of trust.

Can the output be trusted? Can the recommendation be defended? Can someone confidently stand behind the work if it is questioned six months from now?

Because when something goes wrong, nobody gets to blame the AI. The employee is accountable. The consultant is accountable. The company is accountable.

That is why I have always found the quality debate to be the least interesting part of the conversation. The more important question is not whether AI was involved. It is whether the outcome is trustworthy enough for someone to put their name behind it.

The Outcome Test

The more I think about AI, the less interested I become in whether it was used.

Instead, I find myself asking a different set of questions. Was the outcome accurate? Was it useful? Was it better than the alternative? Would I be willing to stand behind it with my name, reputation, and credentials on the line?

If the answer to all of those questions is yes, then I have a hard time arguing that the production method matters more than the result.

I suspect this is where many people become uncomfortable because it shifts the conversation away from tools and back toward results.

Ironically, this is also where humans become more important, not less.

The future is not machines versus humans. I know, "The Terminator" and "I, Robot" movies will never feel the same. The real shift is humans using AI versus humans who refuse to adapt.

The premium will not come from avoiding AI. It will come from judgment, taste, decision-making, communication, and accountability.

AI can accelerate execution, but people still decide what should be built, what should be published, and what risks are acceptable. More importantly, people are still responsible for the outcome.

The people who lose to AI will not be the ones using it. They will be the ones still evaluating effort while everyone else is measuring outcomes.

This post first appeared on the author’s website and is republished here with permission.

Inspired by this post on Search Engine Land.

July 3, 2026
Why I Stop Positioning AI as a People Replacement

I think one of the biggest mistakes in AI marketing is positioning a product as a replacement for people. That message can win attention in the short term, but I believe it quietly drains trust over time.

This is a little different from what I usually write about, but it matters. The way we talk about AI shapes how customers, employees, executives, and markets respond to it.

In this memo, I want to focus on three things: why “substitution positioning” feels powerful at first but weakens a brand later, what the data says about whether AI is actually replacing people, and how I think companies should position AI instead.

The cardinal sin of positioning in the AI era is replacement. I call it substitution positioning. It is tempting because it sounds bold, efficient, and disruptive. But over time, it creates anxiety, skepticism, and credibility problems.

We have seen this pattern already. Anthropic CEO Dario Amodei predicted that software engineering jobs could disappear within 6 to 12 months as models began doing most or all of what software engineers do end to end. Yet demand for software engineers has continued to look strong.

OpenAI CEO Sam Altman also predicted that many customer support jobs would go away because AI could handle that work better. Soon after, customer service hiring began outpacing the broader job market.

I understand why fear works as a marketing tool. The fear of being replaced gets attention fast. It got me, too. When powerful AI models gained traction, I worried about my own future. But when I still see AI companies hiring copywriters, SEOs, engineers, and support teams, I sleep better.

Fear sells because it taps into fight-or-flight. Layoffs make that story even louder. They let companies frame cost-cutting as innovation and make the replacement narrative feel more real than it may actually be.

But I do not think the facts support the clean replacement story. In New York, companies can indicate when mass layoffs are caused by technological innovation or automation. In one reported period, more than 160 companies filed mass layoffs affecting roughly 28,300 workers, and not one chose AI as the reason. That list included companies such as Amazon and Goldman Sachs.

Researchers at Yale also studied employment data from the Current Population Survey over 33 months and found no evidence of job displacement from AI. To me, the pattern looks less like instant replacement and more like the earlier waves of computers and the internet changing how work gets done.

That is why I keep coming back to this point: stop trying to make replacement happen. It is not happening in the simple, dramatic way many AI narratives suggest.

AI is powerful, but it is also inconsistent. In its current form, it can do some tasks better than humans and fail badly at others. That paradox is often called the Jagged Frontier.

The Jagged Frontier idea matters because it explains why some people see AI as transformative while others remain lukewarm. A BCG and Harvard study of 758 knowledge workers found that people get the most value from AI when they understand what it is good at and where it breaks down.

Microsoft reached a similar conclusion in its 2026 Work Trend Index Annual Report. The company found that a small group of advanced AI users, described as Frontier Professionals, were not simply using AI more often. They also knew which mode of AI use fit each task.

That distinction is important. The best AI users are not handing everything over blindly. They are applying judgment. They know when to use AI as a helper, when to use it as a collaborator, when to use agents for multi-step workflows, and when to keep a human firmly in control.

I still do not trust most AI workflows enough to leave them running with no maintenance, review, or quality assurance. The question I ask is simple: would I bet my brand, customer experience, or revenue on a fully automated workflow with no human oversight?

Klarna is a useful warning here. The company publicly promoted the idea that AI was doing the work of hundreds of agents and helping reduce headcount. Later, it reversed course and rehired humans after leadership acknowledged that aggressive cost-cutting had lowered quality and that customers still wanted a human option.

That is the tradeoff I see with substitution positioning. It creates immediate attention, but it can damage long-term credibility. The words often do not match the operational reality.

Replacement positioning could work if customers truly wanted full replacement and if the technology were consistently ready for it. I do not think either condition is true.

Cost reduction is a strong AI argument because it shows up quickly on the P&L. Productivity gains usually take longer. They build inside companies over time and often take even longer to appear across the broader economy.

But when replacement positioning goes beyond cost-cutting and becomes people-cutting, I believe it starts to antagonize the very people companies need to win over.

We have already seen backlash. Duolingo’s AI-first memo drew heavy criticism before the company reframed AI as a tool to accelerate work rather than replace contractors. Surveys have found that some workers refuse to use AI tools because they fear job loss. Pew has reported that many U.S. adults are more concerned than excited about AI in daily life. Reuters/Ipsos polling has shown widespread fear that AI will permanently displace workers.

There is also a quality problem. When employees believe the purpose of AI is to replace them, they may disengage or produce lower-quality work. In my view, that is not just an adoption issue. It is a positioning failure.

Executives often feel more excited about AI than the employees asked to use it every day. That gap matters. If leadership talks about AI as a replacement engine, employees hear a threat. If leadership talks about AI as leverage, employees have a reason to learn.

Token economics also complicate the replacement story. Some companies have bragged about massive AI usage, but token costs are still a real business variable. As those costs normalize, the math may make junior employees look interesting again, especially when human judgment, context, and accountability are part of the output.

So what should replace replacement? I think the answer is enhancement. Instead of positioning AI as a way to remove people, I would position it as a way to make capable people more effective.

AI can be used in two broad ways. A company can try to reduce the number of people, or it can grow output with the same number of people. The data I have seen suggests that productivity gains often create the stronger return.

A National Bureau of Economic Research paper surveyed 750 executives about AI’s impact on productivity and labor markets. Larger firms showed more interest in replacing labor costs, but the highest ROI came from productivity growth.

That is the lesson I take from the research: doing more with the talent you already have is often stronger than trying to remove the talent that knows what good work looks like.

Building products has become easier, but distribution has not. When supply explodes, the scarce thing is not output. The scarce thing is being the product, brand, or service that actually gets chosen.

That is why positioning matters more than ever. Product quality still matters, but the way I frame AI use can determine whether people see it as empowering or threatening.

My takeaway is simple: I would stop selling AI as a people replacement. I would sell it as judgment leverage, workflow acceleration, and creative expansion. Fear can get attention, but empowerment is a better long-term strategy.

This post first appeared on the author’s website and is republished here with permission.

Inspired by this post on Search Engine Land.

July 3, 2026
Why 40% of AI Projects Fail: The Human Element Matters Most

In exploring the world of agentic AI, I’ve come across a startling prediction from Gartner: by the end of 2027, more than 40% of these projects will have been canceled. This isn’t due to the technology being insufficient; it’s because of the human factors involved. The real issue lies not with the tech, but with our deployment strategies and the absence of essential human insights.

Gartner’s research, involving over 3,400 organizations that are currently investing in agentic AI, makes it clear that the downfall isn’t in the capabilities of AI itself. It’s in the decisions we, as humans, are making. Anushree Verma from Gartner notes that most of these AI projects are merely hype-driven experiments, lacking in strategic direction and governance.

This brings a critical reminder for those of us in marketing: agentic AI can optimize and scale tasks exponentially, yet without a knowledgeable human behind it, the technology is as good as the strategy guiding it. We need agents that can handle audience selection, content generation, and journey orchestration effectively, but we must steer these agents with insight and responsibility.

If we’re spurred by fear of missing out (FOMO), we might find ourselves hastily deploying AI solutions. This rush can lead to poorly constructed workflows and inadequate data strategies, resulting in agents implementing erroneous actions at inappropriate times. FOMO isn’t a sustainable strategy; it’s a costly oversight.

Another pitfall presented by Gartner is what’s termed ‘agent washing.’ This is where existing chatbots are disguised as agentic AI without delivering authentic autonomous functionality. As marketing teams, if we invest in these disguised solutions, we’re essentially falling for dressed-up automation without real AI benefits.

Deploying AI prematurely can be damaging. Gartner anticipates that by 2026, many companies might harm their customer relationships through misguided AI applications, leading to eroded trust and damaged brand reputations. Our role as marketers should be to prioritize strategy and judgment alongside technological advancements.

One of the gravest challenges we face is the potential erosion of critical thinking brought about by reliance on AI. Gartner predicts half of the organizations will need to reassess competencies, ensuring that our human ability to question and evaluate AI outputs remains sharp and undiminished.

In this rapidly evolving landscape, the successful marketer will be one who integrates AI while maintaining a leadership role. This encompasses being a multidisciplinary thinker who utilizes AI to transcend traditional roles, driving strategy and ensuring that AI recommendations align with our brand’s vision and values.

As we embrace the agentic era, it’s imperative that we balance technological advancements with human insights. We shouldn’t slow down but rather be deliberate—ensuring that our AI endeavors are guided by robust human judgment to harness true value, protect customer trust, and avoid costly missteps.

Inspired by this post on Search Engine Land.

April 29, 2026