The AI Attribution Lie Nobody Wants to Talk About

Something is happening in marketing right now that nobody wants to say out loud.

Onboarding surveys are showing AI or ChatGPT or AI search as a top discovery channel. Leadership is excited. The attribution is being reported to the board. Budgets are shifting. Strategies are being built on the back of it. People with doubtful skills are getting hired.

And almost nobody is asking the obvious question: do you actually trust what users say in a post-signup survey?

I don’t. And you shouldn’t as well. AI is influencing discovery. I have no doubt about that. But it does so in ways that are genuinely hard to measure, especially when companies have no real framework for it and hand the problem to whoever owns SEO because it feels adjacent. A checkbox ticked in a hurry during onboarding is not proof of anything. It is a measurement of something, but what it is measuring is far messier than most marketing teams are willing to admit.

This piece is about what is actually happening when someone answers “where did you hear about us” and why treating that answer as reliable attribution data is one of the cleanest attribution errors in marketing right now.

What People Are Doing When They Answer The Onboarding Question

Picture the moment. Someone has just signed up for your product. They are mid-flow, they want to get to the thing they just signed up for, and there is a question standing between them and the dashboard: how did you hear about us?

They are not doing memory retrieval. They are not mentally replaying their discovery journey, tracing back through the channels that first introduced them to your brand, the touchpoints that built familiarity, the moment that triggered the decision to sign up. That process, done properly, would take several minutes and require more cognitive effort than most people are willing to spend on a form field during onboarding.

What they are actually doing is pattern-matching under time pressure. They scan the options, something feels plausible or recent or obvious, and they select it. The whole thing takes about four seconds.

This is not a flaw in your users. It is behavioural economics 101. Human beings are cognitive misers, a term introduced by psychologists Susan Fiske and Shelley Taylor to describe our tendency to conserve mental effort by defaulting to shortcuts rather than deliberate reasoning. Daniel Kahneman’s later framing in Thinking, Fast and Slow calls this System 1 thinking: fast, automatic, triggered by tasks that feel routine or low-stakes.

Answering an attribution question on an onboarding form is exactly that kind of task. It is inconsequential, it is frictionless, and it sits between the user and the thing they actually signed up for. So they answer fast, with whatever comes to mind first, and move on.

The problem is that you then treat that four-second response as evidence of a multi-week discovery journey. And that gap, between what the answer represents and what you are using it to prove, is where the attribution error lives.

The Three Biases Corrupting Your Data

Before naming the biases, it’s worth understanding the structural problem underneath all of them. When you present a list of options and ask someone to select how they found you, you are not asking them to recall a memory. You are asking them to recognise something plausible from a menu. Recall and recognition are fundamentally different cognitive tasks. Recall requires retrieving a memory independently — harder, slower, more effortful. Recognition only requires matching something on a list to something that feels familiar or possible — much easier, much faster, and far less accurate as a representation of what actually happened.

An attribution survey with predefined options is a recognition task dressed up as a recall task. And recognition, under time pressure, will consistently favour whatever is most salient, most recent, or most prominent on the list — not what is most true.

The three biases below are all essentially recognition errors. Understanding that changes how you read the data.

Primacy and Recency Bias: The Order of Your Options Is Shaping the Answers

In survey design, the position of an option on a list consistently influences how often it gets selected. Options that appear first tend to be overselected: people read the first option, it seems plausible, they stop reading and select it. Options that appear last are overselected for a different reason: recency makes them easier to retrieve from short-term memory.

Options buried in the middle of a long list are chronically underselected, regardless of their actual relevance.

This means that if AI or ChatGPT or AI search is near the top of your where you hear about us options because you put it somewhere prominent, you will see higher selection rates for that option than if it were buried in the middle. The position is generating the result, not the actual discovery channel.

Run an experiment: randomise the order of your options for two weeks and watch what happens to the distribution. In most cases the channel that was first is no longer dominant. What changed is not user behaviour. What changed is which bias you are measuring.

Recency Bias: The Last Touchpoint Wins, Even When It Should Not

When people try to recall how they discovered something, they systematically overweight recent experiences and underweight earlier ones. A user who first heard about your product six months ago through a word-of-mouth recommendation, saw it mentioned in a newsletter three months ago, and then had it come up in a ChatGPT answer last week is very likely to say AI in your onboarding survey.

Not because AI was the most important discovery channel. Because it was the most recent. Recency makes memories more accessible, and accessible memories feel more true.

This is a particularly acute problem right now because AI search is genuinely new and visible. People notice it. They remember it. It stands out in a way that a Google search result or a LinkedIn post from six months ago simply does not. So even in cases where AI played a real but small role in the discovery journey, it will capture attribution credit disproportionate to its actual influence.

Social Desirability Bias: The Answer That Makes People Feel Smart

People do not just answer surveys accurately. They answer in ways that feel appropriate, intelligent, or current given the context. This is social desirability bias: the well-documented tendency to give answers that reflect well on the respondent rather than answers that are strictly true.

Saying you found something through AI search signals that you are someone who uses AI tools, who is ahead of the curve, who is engaged with how discovery is changing. It is the modern equivalent of saying you found a restaurant through a food critic rather than because you walked past it and the menu looked good.

I found your product through ChatGPT is, right now, a slightly flattering thing to say. It makes the user sound current and intentional. I googled it sounds boring. I do not really remember sounds careless. So even users who are not consciously trying to present well will gravitate toward the AI option as the answer that feels right for who they are.

Why AI Is the Most Overreported Channel in Onboarding Surveys Right Now

Every one of these biases points in the same direction for AI, at this specific moment in marketing history.

AI search is new, so it is cognitively salient: people notice it and remember it in a way they no longer notice Google. AI is prominent in media and professional conversation, so selecting it feels current and informed. And AI is increasingly being added to the top of where did you hear about us lists by companies who want to track it, which means primacy bias is actively boosting its selection rate before a single real discovery event has been recorded.

None of this means AI is not influencing discovery. It certainly is. The question is whether the signal you are getting from onboarding surveys reflects that influence accurately or whether you are measuring a confluence of three cognitive biases that all happen to point at the same option.

The honest answer is that you cannot tell from the survey alone. A high selection rate for AI in your onboarding survey is consistent with AI genuinely driving discovery. It is also entirely consistent with AI appearing first on your list, being the most recent channel users encountered, and being the answer that makes users feel current. These explanations are not mutually exclusive, and the survey cannot distinguish between them.

What You’re Really Measuring When Users Select AI in Your Onboarding Survey

When a user selects AI in your onboarding survey, you are measuring one of the following things, and you cannot easily tell which:

• AI genuinely drove discovery and the user accurately recalled and reported it
• AI was the most recent touchpoint in a multi-channel journey and recency bias drove the selection
• AI appeared first or prominently in the option list and primacy bias drove the selection
• The user felt that selecting AI reflected well on them and social desirability drove the selection
• Some combination of the above

The first scenario is the one you want to be measuring. The others are noise. And the structure of the survey itself, the order of options, the wording, the context, the moment in the user journey when it appears, determines how much noise there is relative to signal.

If you have not deliberately designed your survey to minimise these biases, you are not measuring AI discovery. You are measuring a weighted average of cognitive shortcuts, and the weight assigned to AI is as much a function of your survey design as it is of actual user behaviour.

Case Study: Onboarding Survey Showed AI as Top Channel — Here’s What the Data Revealed

A client came to me convinced their users were increasingly discovering them through ChatGPT. The onboarding survey numbers looked compelling: AI was showing up consistently as a top channel, strongly surpassing search, direct and paid.

Because I’ve seen a lot of false claims and hype during my career, I’m always very skeptical and want data before concluding something.

So I ran three checks.

First, I used Hotjar to watch session recordings of users moving through the onboarding flow and hitting the attribution question. I did this across a lot of sessions and over a sustained period (being a researcher helps when it comes to avoiding burnout and frustration). What I saw was consistent: users arriving at the question, selecting an option within seconds, barely scrolling. There was little visible deliberation or hesitation. It looked like the cognitive miser pattern playing out in real time: users choosing the quickest, most immediately available answer rather than carefully evaluating every option.

Second, I cross-referenced against traffic data for the same period to check whether the AI attribution spike in the survey had any corresponding signal in direct or AI-referred sessions. It did not.

Third, I looked at branded search volume for the same window. No movement there either.

The survey was showing AI as a top channel. The session recordings showed people not really thinking about it. The traffic data showed no movement that would explain it.

Three independent signals pointed to the same conclusion: the survey was measuring something—just not what the client believed it was measuring.

When This Kind of Attribution Data Is Still Useful

None of this means you should stop asking the question. It means you should stop treating the answer as proof of something it cannot prove. There are legitimate uses for self-reported attribution data, as long as you are honest about what it is.

Early-Stage Signal

When you are at the very beginning of building a product and have limited data infrastructure, a qualitative sense of where users are coming from is better than nothing. Self-reported attribution gives you directional signal: a rough map of the territory, not a precise measurement. Use it as a hypothesis generator, not a conclusion. If AI is consistently mentioned, that is a reason to investigate further, not a reason to report AI is our top acquisition channel.

Internal Buy-In

Sometimes organisations need a nudge before they will invest in investigating something properly. If onboarding survey data is showing AI mentions and that is what it takes to get leadership to fund a proper attribution study, the imperfect data has served a legitimate purpose. Use it to start the conversation, not to end it.

Directional Tracking Over Time

If you keep the survey design constant—the same options, order, placement, and wording—changes in the distribution over time may still provide a useful signal, even when the absolute numbers are unreliable. If AI selections double over six months while the other responses remain relatively stable, that is worth investigating. It does not prove that AI-generated leads have doubled, but it may indicate a change in awareness, user perception, discovery behaviour, or channel influence.

The trend can be more informative than the snapshot, but it still requires validation against other evidence.

The key caveat for all three: be explicit about what the data is.

“Our onboarding survey shows that AI mentions are increasing, which we are treating as a directional signal worth investigating” is honest.

“AI is driving X percent of our leads, confirmed by onboarding survey data” is not.

The difference between those two statements is the difference between using imperfect data responsibly and building a strategy on a measurement error.

What Better Attribution Actually Looks Like

This piece is not a full measurement guide. But if you want to move beyond onboarding surveys as your primary AI attribution source, here is where to start:

• Triangulate across signals: Branded search volume moving in parallel with AI activity is a stronger signal than a survey response. Traffic spikes following specific AI-visible content or community moments are more concrete. Inbound leads that mention AI unprompted in a sales conversation carry more weight than a checkbox.
• Ask open-ended questions: Where did you first hear about us as a free-text field produces messier data but more honest data. People who genuinely discovered you through AI will say so. People who are pattern-matching will either leave it blank or reveal their actual uncertainty.
• Ask at a different point in the journey: The onboarding flow is the worst possible moment for accurate recall: users are in task mode, not reflection mode. A follow-up survey sent two to three days after signup, when the user has had time to actually use the product and is no longer mid-flow, produces meaningfully different and more reliable responses.
• Compare against what you can measure directly: AI Overviews citations, Perplexity references, and direct traffic from AI-adjacent sources are all trackable to some degree. If your survey shows 40 percent AI attribution but your traffic data shows almost no AI-referred sessions, the gap is telling you something important about your survey design.

AI Isn’t the First Channel to Be Overcredited in Onboarding Surveys. It Won’t Be the Last

Every channel that becomes prominent goes through this cycle. Social media did it: in 2012, companies were reporting social as a top discovery channel on the back of onboarding surveys, while most of the selection was driven by its prominence in the cultural conversation rather than actual discovery journeys. Content marketing did it. Influencer marketing did it.

The pattern is consistent: a channel becomes culturally prominent, it gets added to attribution surveys, cognitive biases inflate its reported contribution, marketing teams build strategies on inflated numbers, the channel eventually matures and measurement improves, and the actual contribution turns out to be real but smaller than the survey data suggested.

AI is going through this cycle right now. The underlying discovery influence is real. The measurement is almost certainly inflated. And the gap between those two things is where bad strategic decisions get made.

The solution is not to dismiss AI as a discovery channel but to be honest about what your current measurement can and cannot tell you and to build better measurement before you build strategy on top of the data you have.

If you’re looking at your onboarding survey data and wondering whether it’s telling you the truth — it probably isn’t, not entirely. I work with marketing, data, and technical teams to build attribution frameworks that go beyond self-reported surveys: cross-referencing behavioural data, traffic signals, and branded search to give you a picture of discovery you can actually act on. If you want to understand what’s really driving your growth before you build strategy on top of assumptions, let’s talk.