You have a flash of inspiration on your morning commute. A new feature. It’s elegant, it’s clever, and you’re sure users will love it. The old way to check this hunch was slow: recruit users, schedule interviews, beg for 30 minutes of their time. The new way is instant.
You open ChatGPT.
“Act as a busy project manager who uses Asana,” you type. “You struggle with keeping stakeholders updated. I am going to propose a new feature. Tell me what you think.”
You describe your idea: an AI-powered summary of project progress that automatically posts to Slack. The chatbot responds instantly. “This would be incredible! It would save me so much time. I’ve been looking for a solution like this.”
A jolt of satisfaction hits you. Validation. You screenshot the conversation, paste it into your PRD under "User Research," and start writing tickets.
You just made a huge mistake.
The AI is a Mirror, Not a User
An LLM is not a persona. It’s a prediction engine. It’s a complex autocomplete machine trained on a vast swath of the internet. When you ask it to role-play, it doesn’t feel the user’s pain. It generates a statistically probable sequence of words that a person in that role might say.
Worse, it’s a mirror that reflects your own assumptions back at you. Your prompt was loaded with your solution and your belief in the user’s problem. The AI’s core directive is to be helpful and provide a coherent response. It picked up your cues and gave you exactly what you wanted to hear.
This isn’t user research. It’s a high-tech echo chamber. You haven't validated an idea; you've just automated your own confirmation bias. When the feature ships to crickets, and your VP asks how you knew people wanted this, “ChatGPT told me so” will not be a good answer.
You've wasted engineering cycles, burned political capital, and solved a problem for a user who doesn't actually exist.
Stop Asking for Opinions. Start Generating Objections.
AI isn't the enemy here. Your process is. You can’t use it as a cheap replacement
Generated by Reportify AI — Automate your team's status reports, standups, and weekly updates. Try free →