Concept testing questions: 20 to ask before you build

What a concept test is

A concept test is research that puts an idea in front of the people it is for, before it exists, to learn whether they understand it, need it and would change what they do today. The concept can be a paragraph, a sketch, a landing page, a short video or a clickable mock-up. It tests value, not usability: whether this is worth building, not whether people can find the button.

Marty Cagan at Silicon Valley Product Group sorts product risk into four kinds: value, usability, feasibility and business viability. A concept test is the cheapest way to work on the first. It sits early, next to other kinds of research that answer different questions:

Why most concept tests mislead

Most concept tests mislead because they ask people to forecast their own behavior, and people forecast badly and politely. Three findings from three different fields point the same way.

• Joel Rubinson, a market research practitioner, wrote in 2015 that only about 30% of people who say they will “definitely buy” a new product, on average, go on to buy it after launch.
• A 2005 meta-analysis by Murphy and colleagues of 28 studies found that what people said they would pay was a median 1.35 times what they actually paid, with some studies far higher.
• A 2006 meta-analysis by Webb and Sheeran found that a medium-to-large change in what people intend to do (d = 0.66) produced only a small-to-medium change in what they did (d = 0.36).

Nielsen Norman Group puts the practical version plainly: watch what people do rather than trust what they say they will do. Rob Fitzpatrick’s The Mom Test turns that into rules for conversation: talk about their life rather than your idea, ask about specific past events rather than general habits or the future, and treat compliments as noise. Alberto Savoia, who coined pretotyping, makes the same point about data: trust evidence where people have some skin in the game.

So a good concept test moves every answer up this ladder, from what people say toward what they do and what they give up:

None of this means stated reactions are useless. A first reaction tells you whether the idea is understood and what it reminds people of. It just cannot tell you whether they will pay. Keep the two apart when you read the results.

When to run a concept test

What you test depends on the decision in front of you. These are the common uses in four roles.

Product managers

Product managers use concept tests to decide what to build next.

• Prioritizing between problems: which problem hurts enough to solve next, before you design anything.
• Prioritizing between features: which of two or three planned features people understand and would reach for first.
• Validating a roadmap bet: whether a planned feature solves a problem users actually have.
• Testing a change to the product: whether users understand a new workflow or pricing change, and what it would replace for them.

Product designers

Designers use them to choose and sharpen a direction before it is polished.

• Choosing between design directions: which one people understand, before you polish it. Show each direction to different people, because the first one colors the second.
• Checking a new concept explains itself: whether people get it without help.
• Naming and labels: whether a feature’s name tells people what it does.
• Reactions to an early prototype: first impressions of a clickable mock-up. When the question becomes whether people can use it, switch to a usability test.

Marketers

Marketers use them to test how the product is described before money is spent on it.

• Messaging and positioning: which headline or value proposition people understand and care about.
• Landing pages: before a launch or before you spend on ads.
• Product and feature names: what a name suggests to people who have never seen it.
• Pricing and plan pages: whether people understand what each plan includes, and which one they would pick.

Founders

Founders use them to decide whether an idea deserves the next few months.

• A new product idea: whether it is worth building at all. If the problem is not real for these people, go back to customer discovery first.
• Choosing a customer segment: show the same concept to two groups and compare how strongly each responds.
• Testing a pivot: before you rebuild around it.
• Pre-selling: whether people will commit time or money to a pilot or waitlist, which is the strongest signal you can get.

Decide what you are testing before you write a question

Write down the one or two assumptions that would sink the idea if they were wrong, and test those rather than the whole idea. Teresa Torres makes the case in her piece on assumption testing: when a whole idea fails a test, you learn only that it failed. When one assumption fails, you learn what to change. She sorts assumptions into five kinds: desirability, viability, feasibility, usability and ethical. A concept test is strongest on the first.

A useful assumption is specific enough to be wrong. Compare:

Pick a test design: monadic or sequential monadic

If you have more than one version of the idea, decide how people see them. In a monadic test, each person sees one concept. In a sequential monadic test, each person sees two or more concepts, one after another, and answers the same questions for each. These are the definitions Qualtrics uses; some popular guides mix them up.

The 20 concept testing questions

Ask them in this order: the problem first, then the reaction, then value, alternatives, concerns and commitment. You will not use all 20 in one test. Pick six to ten that test your riskiest assumption and keep the whole thing under ten minutes. Where a common wording does damage, it is shown under the question.

Before they see the concept: the problem

These come first so the concept cannot color the answers. If the problem is not real for this person, nothing after it matters.

1. 1

   “Tell me about the last time you [had this problem]. What did you do?”

   A specific, recent story shows whether the problem is real and how it plays out. Almost everyone says yes to a hypothetical tool.

   Best spoken · Instead of “Would a tool that does X be useful to you?”

2. 2

   “What do you use today to deal with it?”

   This is your real competition, and it is often a spreadsheet, a colleague or nothing at all. “Nothing” is an answer worth hearing.

   Best spoken

3. 3

   “What have you tried that did not work?”

   People who have already searched for a fix have a problem that hurts. People who have never looked usually do not.

   Best spoken

4. 4

   “How many times did this come up in the last 30 days?”

   A number for a fixed period is a fact. “Usually” invites the idealized version of someone’s month.

   Multiple choice · Instead of “How often do you usually deal with this?”

Their first reaction and whether they understand it

Now show the concept. Say nothing to explain it; the concept has to do that on its own, as it will in the real world.

1. 5

   “Take a look at this. What is your first reaction?”

   An unprompted reaction catches confusion, excitement and doubt in the person’s own words, before they have had time to be polite.

   Best spoken · Instead of “Do you like it?”

2. 6

   “In your own words, what does this do, and who is it for?”

   If people cannot explain it back, stop and fix the concept. Their words also show you how they will describe it to others.

   Best spoken · Instead of “Is this concept clear?”

3. 7

   “What, if anything, is confusing or hard to believe?”

   Giving permission to doubt surfaces the claims people do not trust. A yes-or-no version gets a yes.

   Best spoken · Instead of “Did you understand everything?”

Value: what it would change for them

Once they understand the idea, find out what it would change for them. Tie each answer to their own work, not to how the idea sounds.

1. 8

   “Which part would matter most to you next week, and why?”

   Forcing one choice, tied to a near date, shows what carries the value. The “why” is the part to listen to.

   Best spoken

2. 9

   “What, if anything, would this replace?”

   A replacement competes with a known budget line. An addition has to justify a new one, which is a much harder sale.

   Best spoken · Instead of “Would you use this alongside your current tools?”

3. 10

   “What is missing for this to be useful to you?”

   It asks for the gap rather than a verdict, which is easier to answer honestly and more useful to hear.

   Best spoken · Instead of “What do you think of it?”

Alternatives: how it compares

Next, compare it with what they do today, including doing nothing. That is the competition the idea actually faces.

1. 11

   “How is this different from what you use now, if at all?”

   “If at all” gives people room to say it is the same, which is the answer you most need to hear early.

   Best spoken · Instead of “Is this better than [competitor]?”

2. 12

   “If this did not exist, what would you do instead?”

   It exposes the real alternative, including doing nothing. If the answer is a shrug, the problem may not be urgent.

   Best spoken

3. 13

   “Compared with what you use now, this would be: much worse, a bit worse, about the same, a bit better, much better, or not sure.”

   A comparison is easier to answer than an absolute rating, and it lets you compare concepts side by side.

   Multiple choice

Concerns: what would stop them

Then ask what would stand in the way. People rarely volunteer doubts, so give them a clear opening to name them.

1. 14

   “What would stop you from using this?”

   It assumes something would, which makes it easy to name. “Any concerns?” is easy to answer with no.

   Best spoken · Instead of “Do you have any concerns?”

2. 15

   “Who else would have a say in using this, and what would they ask?”

   In B2B the user is rarely the only decider. This maps the buyer, the budget and the security review before they surprise you.

   Best spoken

Commitment and price: what they would give

These move the answers up the evidence ladder. Ask them last, once the person has told you about their problem and reacted to the idea.

1. 16

   “What would need to be true for you to switch?”

   It turns a yes-or-no forecast into the conditions for a real decision, which you can check.

   Best spoken · Instead of “Would you buy this?”

2. 17

   “What does this problem cost you today, in hours or money?”

   Today’s cost is a fact and anchors value. Hypothetical prices tend to run high: in the Murphy meta-analysis, stated values were above what people actually paid.

   Best spoken · Instead of “How much would you pay for this?”

3. 18

   “We are opening a small pilot next month. Do you want a place?”

   A real offer asks for real time. The share who accept, and who then show up, is worth more than any rating.

   Multiple choice

4. 19

   “How likely are you to use something like this in the next three months?”

   Use this scale only to compare concepts with each other, never as a forecast. In one analysis, most people who said “definitely” did not go on to buy.

   Multiple choice

The close

End with one open question, so people can add what your questions missed.

1. 20

   “What should I have asked that I did not?”

   People often save the thing that matters most for last. This gives them a door.

   Best spoken

Check your own questions

Most weak concept testing questions break one of seven rules. Run each question you write against this list before you send the test.

• Ask about the past, not the future

  People are poor forecasters of their own behavior, and polite ones.

  Instead of: “Would you use a tool that summarizes your customer calls?”

  Ask: “Tell me about the last time you went through your customer calls.”

• Leave your idea out of the problem questions

  Once the idea is on the table, people review it instead of describing their week.

  Instead of: “What do you think of our idea for call summaries?”

  Ask: “What do you use today to keep track of what customers say?”

• Stay neutral

  A leading question hands over the answer you hope for.

  Instead of: “Don’t you find it frustrating to rewatch calls?”

  Ask: “What was hard about the last time you rewatched a call?”

• Ask about a real moment, not a habit

  “Usually” invites the idealized version of someone’s month.

  Instead of: “How often do you usually review calls?”

  Ask: “How many calls did you review in the last 30 days?”

• Keep it open

  A yes or no ends the answer before the reason arrives.

  Instead of: “Do you like this design?”

  Ask: “Take a look at this. What is your first reaction?”

• Ask one thing at a time

  People answer the easier half, and you cannot tell which.

  Instead of: “What did you try, and did it work?”

  Ask: “What have you tried so far?”

• Ask what it costs today, not what they would pay

  Hypothetical prices tend to run high. Today’s cost is a fact.

  Instead of: “How much would you pay for this?”

  Ask: “What does this problem cost you today, in hours or money?”

How many people you need

For open, spoken answers, 5 to 8 people per segment per round will show you the main problems with a concept, and about 12 per segment is the point where new themes usually stop appearing. In a widely cited study, Guest, Bunce and Johnson found thematic saturation at 12 interviews; a later re-analysis across several sites needed 20 to 40, as Bath Social and Development Research summarizes. Different kinds of customers are different segments, so count again for each.

You may have heard that five users are enough. That finding, from Nielsen Norman Group, is about finding usability problems in a design, not about judging whether an idea is wanted. And if you want to compare scores between two concepts, you need enough people that the difference is bigger than the noise, which usually means dozens per concept rather than a handful.

How to read the results

Read for understanding first, then for evidence, then for reasons, and decide per assumption. A clear order keeps a few enthusiastic answers from carrying the whole study.

1. Check understanding. Go through question 6. If more than a few people explain the concept wrongly, the rest of their answers are about a different idea. Fix the concept and test again before you read further.
2. Sort each person onto the evidence ladder. Compliment, stated intent, a real story, time, reputation or money. Count the upper rungs, not the lower ones.
3. Group the reasons. Pull the “why” out of each spoken answer and group similar reasons. A theme that three unrelated people raise without prompting deserves attention. AI summaries help here, but read a few answers in full to check them.
4. Compare, do not forecast. Use scales to compare concepts or rounds with each other. A 40% “very likely” is not a sales forecast.
5. Decide per assumption. For each assumption you wrote down, choose one: keep, change or drop. Then write the next test for whatever you changed.

How to run a concept test in Feedbaq

Feedbaq is built for exactly this: testing an idea with the people it is for, before you spend weeks building the wrong thing. You write the questions once and send a link, and people answer in their own time. What comes back is their real reactions and reasons, not polite guesses.

Why it works better than a traditional survey

• People talk instead of type. Spoken answers are about three times longer than typed ones: 44 words against 14 in a 2024 study with 1,001 people.
• An AI asks the follow-up. After an answer, it asks the “why” a form never gets, guided by a line you write.
• People can show you. With a screen share, they open your clickable prototype and talk you through what they see.
• Answers are ready to read. Every spoken answer is transcribed and labelled, and the most useful answers come first.
• Your AI agent can do the work. Connect Claude, ChatGPT or another agent, and it can build the test and read every answer for you.

Set it up in five steps

1. Start from the template. The concept test template opens with what people use today, their first reaction, whether they can explain the concept back, and what would stop them from using it.
2. Add your concept. Attach an image or a video to the first-reaction question. For a clickable prototype, use a screen-share question with a link to it, so people open it and talk you through what they see.
3. Swap in questions from this guide, keeping the problem questions first. Give each AI follow-up a line of guidance, such as “ask what they would use it for” or “ask what made them think that”.
4. Send the link to 8 to 12 people who have the problem. They need no account.
5. Read the answers. Every spoken answer is transcribed and labelled, and the most insightful answers come first. Or ask your own AI agent, connected to Feedbaq, to sort the answers by assumption and quote people exactly.

Here is what one spoken answer and its AI follow-up look like. The follow-up turns a hedged reaction into a fact about cost:

Building a test is free and you can use our concept test template to start collecting feedback today.

Run a concept test this week

A useful concept test takes about a week: one day to prepare, a few days of answers, and an afternoon to read them.

1. Monday: write the one or two assumptions that would sink the idea. Make the concept: a paragraph, a sketch, a landing page or a short video.
2. Monday afternoon: pick six to ten questions from this guide, problem questions first. Start from the concept test template if you want the order set up for you.
3. Tuesday: send the link to 8 to 12 people who have the problem. Existing users, your waitlist and warm introductions answer far more often than strangers.
4. Friday: read everything in one sitting, in the order above. Decide keep, change or drop for each assumption, and write down the next test.

Questions people ask

Sources

1. Rubinson, “The 5-point purchase intent question sucks… or does it?” Joel Rubinson, 2015.
2. Murphy, Allen, Stevens and Weatherhead, “A meta-analysis of hypothetical bias in stated preference valuation” Environmental and Resource Economics, 2005.
3. Webb and Sheeran, “Does changing behavioral intentions engender behavior change?” Psychological Bulletin, 2006.
4. Teresa Torres, “Assumption testing” Product Talk.
5. Marty Cagan, “The four big risks” Silicon Valley Product Group.
6. Marty Cagan, “Prototype testing” Silicon Valley Product Group.
7. Rob Fitzpatrick, The Mom Test (summary) Unusual Ventures.
8. Alberto Savoia, Pretotyping pretotyping.org.
9. Nielsen Norman Group, “Leading questions”
10. Nielsen Norman Group, “First rule of usability? Don’t listen to users”
11. Nielsen Norman Group, “Why you only need to test with 5 users”
12. Qualtrics, “Concept testing questions” Definitions of monadic and sequential monadic testing.
13. Lenzner, Höhne and Gavras, spoken versus typed answers to open-ended probes Journal of Survey Statistics and Methodology, 2024.
14. Bath Social and Development Research, “Sampling” Summarizes Guest, Bunce and Johnson (2006) and Hagaman and Wutich (2017) on saturation.