Sep 21 2020
The 5 Steps to Concept Testing
Whether you’re an experienced market researcher or launching your first concept test, you will want to make sure you are covering the fundamentals. This checklist will get you there.
This checklist is designed to help you:
- Frame up your concept testing goals
- Gather your samples
- Develop your survey
- Launch an effective concept test.
All in just 5 straightforward steps.
Concept Testing Checklist
- Define Your Concept Testing Objective
– Describe your concept test goals
– List your key questions and hypotheses - Target Your Sample
– Determine your target customer criteria and sample size
– Choose your sample source - Gather Your Concepts
– Gather your concept descriptions
– Collect concept visuals - Create Your Survey
– Outline your survey path
– Develop your survey questions
– Program your survey - Launch Your Concept Test!
– Run a quality assurance test across devices
– Launch your concept test
1. Define Your Concept Testing Objective
Describe Your Concept Test Goals
A successful concept test starts with clear goals. What you’re looking to learn will drive many of the decisions you make in the steps that follow. Here, it’s important to cover your overarching research needs while maintaining a test focus. While it may be enticing to gather broad feedback, broad objectives often yield broad results. This makes deeper dives more difficult. Make sure to describe your concept test goals with clarity and focus.
List Your Key Questions and Hypotheses
While survey writing will come later, it’s important to outline your concept test early on. Right away, you should list your key questions to help guide your sample selection and survey question development. Don’t worry about writing perfect survey questions yet. Simply begin to move into more detail with key questions written in a language that works for you and your team. If you’re interested in launching a concept test, you probably already have much of this work formulated.
Regarding hypotheses, it’s common to have an individual, team, or company perspective on expected outcomes. Whether you are testing different graphics or trying to better understand persona attributes, you may be expecting certain results, as well as reasons for those results. These are your concept test hypotheses. While they are not mission-critical for launching a concept test, they can provide testing guidance.
Ultimately, your brand’s internal hypotheses are technically only possibilities without enough evidence yet to be proven out. This is exactly why you’re doing your research. If you’re able to formulate your hypotheses, you can better develop survey questions that will help you test them.
2. Target Your Sample
Determine Your Target Customer Criteria and Sample Size
Targeting the right customers for your sample is critical for accurate and actionable results. Gathering your identifying criteria is key. First, you should be determining who you want to hear from. Then you will list the criteria that will help you identify those target customers. Much of this work can draw directly from early market research and product strategy. Be sure to focus on your market opportunity and persona definitions in addition to your target customers’ behaviors and attitudes.
While integrated tester communities like MESH01 Testers provide you with a wealth of target customer information including bio attributes, activity, lifestyle, profession information, and more, screening surveys that include questions designed to further define and filter testers can help you further refine your sample, making sure you hear from exactly the right people.
Once you cover the basics including demographics and activities, additional criteria will provide the depth needed for a relevant sample. Some guiding questions for these criteria include what are your target customer’s needs? How would our concept fit into their life?
Next, you will need to determine your sample size needs. If you are new to conducting surveys, this step may seem daunting – but it doesn’t need to be. From small-group feedback sessions to large-scale studies capable of supporting detailed analysis of numerous subgroups, your sample needs could range from tens to hundreds to thousands.
In order to address this, we recommend you reasses your objectives and testing goals.
Choose Your Sample Source
For this step, you have a few options, each with benefits and potential limitations. When pulling together a sample for your concept test, it’s critical to ensure you’re talking to the right people. This is most commonly done by sourcing a sample from a third party, utilizing your customer contact list or a blend of the two.
One benefit of using panel providers can include breadth and access to large numbers of respondents. It may also decrease the risk of potential undesired reactions from your actual customers (for example, sending a customer too many emails). While the targeting options and potential sample size availability may be beneficial, it can also come at a premium price. Be sure to evaluate that cost against the sample needs you’ve just identified.
Utilizing your customer list – often managed by the marketing department – can ensure precise targeting while cutting down on costs. However, drawbacks can include limitations when looking to explore new market opportunities, particularly in situations involving new products/markets, sub-branding, or test designs requiring blind studies.
3. Gather Your Concepts
Create Your Concept Descriptions
Here, it’s critical to communicate your concepts as best as possible while considering their relevance to your target customer. This step is another in which referencing your concept test objectives and goals will provide you with important guidance.
Are you looking to test a new product concept within a newly-trending activity? Make sure to cover your bases when it comes to detailed product concept positioning statements and description messaging. Doing so will ensure your testers fully understand what they are being presented with and will ultimately give you confidence in your findings.
Or, maybe you are looking to test multiple product design directions against each other. In this scenario, be sure to clearly communicate differing attributes and features. Doing so here will lead to confidence that the right direction can be identified, along with a comprehensive understanding as to why that direction is best.
Collect Your Concept Visuals
With your concepts framed up, it’s time to pull together any visuals required for your testing. These visuals may be concept sketches, technical drawings or renderings, logo ideas, or new graphics. Flexible concept testing platforms are designed to handle almost any testing use case your team can come up with.
On MESH01, you can even test your concepts in 3D. This view gives your testers an enhanced feedback experience that ensures a thorough review of your digital prototypes before moving into physical prototyping or sampling.
4. Create Your Survey
Outline Your Survey Path
With your sample defined and concept content gathered, it’s time to outline your survey path according to your test design. Some of the most common types of concept tests we see on MESH01 can be categorized as either single-concept or multi-concept tests.
Single-concept test surveys focus on the detailed attributes, positioning, and messaging around a singular concept. This may be a new product, a new market segment, etc. These surveys are often front-loaded with questions designed to better define the tester. They frame up the conversation you’re looking to have with them through questions such as “What brands do you currently use?” or “What do you like or dislike about the product you currently use?” Then, you’re able to move into single-concept test questions such as “What do you think of this concept” and “Do you believe this will meet your needs?”.
In multi-concept test surveys, the outline should include an initial focus on questions that will assist with tester definition as well but will move next into a sequence of different concepts or digital prototypes. This can be achieved through a single image depicting multiple concepts next to each other, or through a series of product concept images and descriptions with a series of questions repeated for each one.
Well-designed digital product concept testing tools match functional horsepower with use-case flexibility. Both are critical to effective testing. Flexibility allows you to conduct numerous different types tests for different research needs all on the same familiar platform. As with product testing, this flexibility is vital for concept tests where your use case may differ from test to test.
Develop Your Survey Questions
With your outline complete, begin filling it in with questions that cover all areas. It will be important to maintain a logical flow through the outline as well as the question sequences within each outline area, too.
At the question level, there are important best practices that should be adhered to in order to ensure both a good respondent experience as well as accurate feedback. The following are some of the most important guidelines to reference when developing your survey questions:
Concept Testing Survey Questions Best Practices
- Keep it conversational – write like you would talk to a friend or coworker
- Avoid jargon or terminology that everyone might not understand
- Keep your questions and responses objective and neutral, not leading
- Keep your questions and responses simple, short, and direct
- Be sure there’s only one way to interpret a question
- Be sure to account for most possibilities in responses
- Choose the right question type for your survey question
Lastly, you will want to review your questions to make sure they will result in the information needed to fulfill your test goals and effectively test your hypotheses.
Developing your survey questions can be as simple as filling out your outline in an offline document with draft questions and answer choices, but you can speed up the process by developing your questions directly within the survey programming tool.
Program Your Survey
If your question development took place offline, it’s time to move your questions into your survey builder tool.
At the beginning of your concept test, we recommend briefly introducing your survey with a conversational tone. Consider language like “We’re looking for passionate trail runners to take a look at a few of our new product concepts and tell us what you think…”
Programming your questions can be straightforward. Essentially, you will drop your question language and answer choices into the appropriate question type. The level of complexity increases when a survey design requires complex branching and/or display logic. This functionality is used when you want to route certain testers through a certain question path, ask follow-up questions when a specific response is chosen, etc. In addition to programming complexity, branching/display logic will also increase the complexity of QA testing as each path will need to be tested.
Lastly, you may find a need to tweak a question or two for them to be better presented in the survey. While you still have one more step coming up to address any last concerns, this step is important for making sure your survey questions read well and will result in the feedback you need.
5. Launch Your Concept Test
Run a Quality Assurance Test Across Devices
The last step we recommend before launching your concept test is to run a quality assurance test across devices. Typically, this test can be done safely with a small group in a parallel “QA testing” send or by filtering out QA testing responses in the data on the back end of a real test. What’s important here is to make sure your survey visuals and questions are displaying correctly and reading properly, and all question-level functionality is intact and operating as planned.
If your QA recipients (say, your coworkers) are new to this experience, instruct them to go through the survey and check each question on each of their devices, making note of anything that might need to be fixed before you launch. As mentioned above, if your survey includes different paths, it will be important for the QA testers to test each of those, too. Lastly, make sure your QA testers can cover all (or at least most) common devices as your test may be going out to thousands of testers using different devices.
Launch Your Concept Test!
You’ve defined your objectives and goals, targeted your sample, gathered your concepts, and programmed and tested your survey. You’re ready to launch a great concept test that’s sure to efficiently and effectively generate the customer insights you need to make highly informed product concept decisions.
Click that button and let the voice of the customer start rolling in!
MESH01 provides a concept testing solution that features an integrated tester community within its feedback platform, which increases sample targeting precision and recruitment efficiencies, usually without additional panel costs. The MESH01 Testers community and surveying flexibility also means that you can add your own testers (customer lists, influencers, ambassadors, etc.) easily within the same test, making it simple to create customized samples for your specific testing needs.
If you’re looking for any additional tips, recommendations, or want to go deeper on these topics, contact us to learn more about efficient and effective concept testing.
