Different types of questions & responses in online surveysDifferent types of questions & responses

Close-ended vs. open-ended questions

Sometimes you need answers that are easy to quantify. In such cases, you can easily go ahead and create a question with several predefined choices. Examples of questions with predefined answers that you can make with SurveyLegend are single selection or multiple selection. Results from such questions are really easy to illustrate, and very easy to understand. Example:
  Which one is your favorite mobile brand?
   Apple    Samsung    Sony    Microsoft   
When presenting the statistics for a question like the example above, you can easily illustrate that e.g. 70% of male participants liked brand X or Y. Such data is easier to use when making  decisions, and easier to share. However, pros recommend that in a multiple choice question, the choices should cover all possible answers. Sometimes, this will mean including an option for “Other,” or “Don’t know,” or even “Don’t wish to say” for sensitive questions. Not only will this get you more accurate data, but it builds trust. If respondents feel you’re trying to make them give you an answer they don’t agree with, they may just skip the question, or stop answering questions altogether.  
Tip:
Some respondents can’t or won’t answer certain questions because they don’t have the experience or aren’t really sure how they want to respond. For these situations, you should offer an option for them to select “Does Not Apply” or “Don’t Know.” However, if you are sure that a respondent is absolutely able to answer the question, you don’t need to offer an “out.”
  When you give the chance to participants to provide you with their own choices (which are not included in your predefined choices), you are making an open-ended question. Example:
  Which one is your favorite mobile brand?
   Apple    Samsung    Sony    Microsoft    Other     
This gives participants some freedom to express and share more deliberate answers. Example:
  Which brand is your favorite mobile brand?      
 
Tip:
Although SurveyLegend offers open-ended questions, if you have the possibility, we recommend that you use closed-ended questions. We have great analytic tools and diagrams that can help you understand the data from closed-ended questions in an insightful way.
 

Limit the respondents to single selection

Sometimes you need to give your respondents the flexibility to answer by making more than a single choice. Although SurveyLegend’s multiple selection is a very useful survey field, it can be tricky to analyze the results you collect with it. If your participant chooses more than one of the answers, does that mean they are equally important? Try to think whether you really need them to choose the answer that matters the most,, or just want to be nice and give them more choices and more flexibility? It’s not wrong to use multiple selections, but it’s harder in most cases to evaluate the results.  

Make sure answers don’t overlap

When using single selection questions, choices should be mutually exclusive and should not overlap. For example, if a question asks: “Which kind of food is your favorite?”, the answers shouldn’t include both Indian food and vegetarian food, because there might be Indian foods which are vegetarian too. If someone’s favorite food is one of those foods, which response is appropriate?  

For scaled questions, use words not numbers

People think with words, not numbers. Although numbers in the analyzed survey data (e.g. average) can make abstract meanings easier to grasp for survey creators, they are hard for survey participants to relate to, when answering. Let’s say you want to ask them: “On a scale of one to seven, with 1 being extremely likely, and 7 being not likely at all, how likely are you to pay for our product again?” First of all, you just made an unnecessarily loooong question for no reason. Then, you expect your respondents to answer while remembering all the abstract things you mentioned and calculate some number in their head that is perhaps close to what they think; and after that, answer three or four or whatever. You make so much cognitive work for them and don’t know if they will write a number that really represents what they are thinking. Instead, you can easily verbalize the answers for them, put them in a scaled order, and then quickly convert the collected data to numbers in your final calculations. Example:
  On a scale of one to five, with 1 being extremely likely, and 5 being not likely at all, how likely are you to recommend our product?
   1    2    3    4    5   
  How likely are you to recommend our product?
      Extremely likely    Very likely    Moderately likely    Slightly likely    Not at all likely   
 
Tip:
In this beautifully illustrated article you can read more in-depth information about so called Likert-type scale responses, and how to use them in your online surveys. Read more….
 

Unipolar vs. bipolar questions

Speaking of questions with scaled answers, how many choices/answers should you give the participants? Survey methodologists have come to the conclusion that: For unipolar questions When you ask a unipolar question that asks the respondents to rate only one aspect of something, a common 5-point scale gives the most reliable data. Example: Unipolar question with 5-points
  How visually appealing is our product, in your eyes?
      Extremely appealing    Very appealing    Moderately appealing    Slightly appealing    Not at all appealing   
For bipolar questions When you ask a bipolar question, you are asking the respondent to describe their opinions or feelings based on two opposite notions, for example “Liking” ↔ “Disliking” something. Therefore, it’s good to add a middle choice too, being “Neutral”. So the scale will be something like this: Like ↤ Neutral ↦ Dislike Then you can decide to include 1, 2, or 3 options on each side on the “Neutral” choice. So, your final bipolar question will have 5-points scale or 7-points scale. In this case, it makes sense to think about having odd-numbers of choices, but just keep in mind that giving too many choices to participants might make them confused. Example: Bipolar question with 5-points
  Do you like our product, neither like or dislike it, or dislike it?
      Like a lot    Like a little    Neither like or dislike    Dislike a little    Dislike a lot   
 
Tip:
SurveyLegend offers different customizable rating questions where you can choose smilies, stars, sliders, or thumbs. They work great in a digital survey!
 

Use Yes/No questions carefully

If you ask your survey participants: “Do you like to play computer games?” and give them only two choices of “Yes” and “No”, what do you expect them to say? Sometimes they might have time and interest, and during some periods they might not have the time for it. Or they might like some specific sorts of computer games, but not all… By limiting your respondents to only two choices, you ask them to see the subject as black and white. But if your aim is to analyze how people really think within the spectrum from black to white, then this is not a well thought-out question. So, sometimes it’s better to give them more choices. Example:
 Will you buy our product again?
   Yes    No   
  How likely are you to buy our product again?
      Extremely likely    Very likely    Moderately likely    Slightly likely    Not at all likely   
 
Tip:
However, yes/no questions are very handy when measuring absolute facts. For example, the answer to “Do you own a cat today?” is either yes or no.
 
Tip:
SurveyLegend offers a visual type of survey field for yes/no or like/dislike or hot/not types of questions. Check it out by choosing Rating > Thumbs..
 

Measuring quantities

When you want to measure how much your survey respondents spend time, money, or energy on something, you can use a scaled question with 5 to 7 choices. If you believe your respondents probably can’t give you exact numbers, just use words like in the example below: Example: Verbal scale
  About how much of your food budget do you spend on eco-friendly products?
      All of it    Most of it    About half it    Some of it    None of it   
However, if you think it is possible for them to answer with more exact numbers, and the numbers are important to you, go ahead and make some numeric intervals. It mostly depends on your data needs. Example: Numeric intervals
  About how much of your food budget do you spend on eco-friendly products?
      00 ↔ 20 %    21 ↔ 40 %    41 ↔ 60 %    61 ↔ 80 %    81 ↔ 100 %   
 

Beware when making ranges

Remember not to make overlapping ranges. This makes people confused when they are trying to chose the right answer. For example lets say you ask them:  “About how long do you play games in a typical day?” and give them these choices: A) 1 hour or less, B) 1 to 2 hours, C) 2 to 3 hours, D) more than 3 hours. Which one should a person choose if he/she plays 2 hours per day? Choice B or C? Example:
  In a typical week, about how many days do you exercise outdoors?
      1 to 3 days    3 to 4 days    4 to 7 days   
  In a typical week, about how many days do you exercise outdoors?
      never    1 to 2 days    3 to 4 days    5 to 6 days    every day   
Therefore, you should try to keep your ranges roughly equal in size and make sure they do not overlap one another.  

Include a way out

Let’s say you ask a question about your respondents’ favorite flavor of ice cream. You may include some flavors in the options that you provide, but will never be able to include all flavors. In such cases, it’s better not to force the participants to choose only from your choices. Try to include an “Other” option. You can even let them write down their own favorite flavors, with an open-ended question and collect the data for further analysis. There may be very interesting data in such responses. Example:
  Which one is your favorite ice cream flavor?
      Bacon ice cream    Garlic ice cream    Oyster ice cream    Hokey pokey ice cream    Salt and Straw ice cream   
  Which one is your favorite ice cream flavor?
      Bacon ice cream    Garlic ice cream    Oyster ice cream    Hokey pokey ice cream    Salt and Straw ice cream    Other     

Become a Legend Now ;)


Make your 1st free online survey today!



starMobile-ready surveys
starUnlimited responses
starLots of amazing features!