Different types of questions & responses

Close-ended vs. open-ended questions

Sometimes you need answers that are easy to quantify. In such cases, you can easily go ahead and create a question with several predefined choices. Examples of questions with predefined answers that you can make with SurveyLegend are single selection or multiple selection. Results from such questions are really easy to illustrate, and very easy to understand. Example:

Which one is your favorite mobile brand?    Apple    Samsung    Sony    Microsoft

When presenting the statistics for a question like the example above, you can easily illustrate that e.g. 70% of male participants liked brand X or Y. Such data is easier to use when making  decisions, and easier to share. However, pros recommend that in a multiple choice question, the choices should cover all possible answers. Sometimes, this will mean including an option for “Other,” or “Don’t know,” or even “Don’t wish to say” for sensitive questions. Not only will this get you more accurate data, but it builds trust. If respondents feel you’re trying to make them give you an answer they don’t agree with, they may just skip the question, or stop answering questions altogether.     When you give the chance to participants to provide you with their own choices (which are not included in your predefined choices), you are making an open-ended question. Example:

Which one is your favorite mobile brand?    Apple    Samsung    Sony    Microsoft    Other

This gives participants some freedom to express and share more deliberate answers. Example:

Which brand is your favorite mobile brand?

   

---

Limit the respondents to single selection

Sometimes you need to give your respondents the flexibility to answer by making more than a single choice. Although SurveyLegend’s multiple selection is a very useful survey field, it can be tricky to analyze the results you collect with it. If your participant chooses more than one of the answers, does that mean they are equally important? Try to think whether you really need them to choose the answer that matters the most,, or just want to be nice and give them more choices and more flexibility? It’s not wrong to use multiple selections, but it’s harder in most cases to evaluate the results.  

---

Make sure answers don’t overlap

When using single selection questions, choices should be mutually exclusive and should not overlap. For example, if a question asks: “Which kind of food is your favorite?”, the answers shouldn’t include both Indian food and vegetarian food, because there might be Indian foods which are vegetarian too. If someone’s favorite food is one of those foods, which response is appropriate?  

---

For scaled questions, use words not numbers

People think with words, not numbers. Although numbers in the analyzed survey data (e.g. average) can make abstract meanings easier to grasp for survey creators, they are hard for survey participants to relate to, when answering. Let’s say you want to ask them: “On a scale of one to seven, with 1 being extremely likely, and 7 being not likely at all, how likely are you to pay for our product again?” First of all, you just made an unnecessarily loooong question for no reason. Then, you expect your respondents to answer while remembering all the abstract things you mentioned and calculate some number in their head that is perhaps close to what they think; and after that, answer three or four or whatever. You make so much cognitive work for them and don’t know if they will write a number that really represents what they are thinking. Instead, you can easily verbalize the answers for them, put them in a scaled order, and then quickly convert the collected data to numbers in your final calculations. Example:

	On a scale of one to five, with 1 being extremely likely, and 5 being not likely at all, how likely are you to recommend our product?    1    2    3    4    5
	How likely are you to recommend our product?       Extremely likely    Very likely    Moderately likely    Slightly likely    Not at all likely

   

---

Unipolar vs. bipolar questions

Speaking of questions with scaled answers, how many choices/answers should you give the participants? Survey methodologists have come to the conclusion that: For unipolar questions When you ask a unipolar question that asks the respondents to rate only one aspect of something, a common 5-point scale gives the most reliable data. Example: Unipolar question with 5-points

How visually appealing is our product, in your eyes?       Extremely appealing    Very appealing    Moderately appealing    Slightly appealing    Not at all appealing

For bipolar questions When you ask a bipolar question, you are asking the respondent to describe their opinions or feelings based on two opposite notions, for example “Liking” ↔ “Disliking” something. Therefore, it’s good to add a middle choice too, being “Neutral”. So the scale will be something like this: Like ↤ Neutral ↦ Dislike Then you can decide to include 1, 2, or 3 options on each side on the “Neutral” choice. So, your final bipolar question will have 5-points scale or 7-points scale. In this case, it makes sense to think about having odd-numbers of choices, but just keep in mind that giving too many choices to participants might make them confused. Example: Bipolar question with 5-points

Do you like our product, neither like or dislike it, or dislike it?       Like a lot    Like a little    Neither like or dislike    Dislike a little    Dislike a lot

   

---

Use Yes/No questions carefully

If you ask your survey participants: “Do you like to play computer games?” and give them only two choices of “Yes” and “No”, what do you expect them to say? Sometimes they might have time and interest, and during some periods they might not have the time for it. Or they might like some specific sorts of computer games, but not all… By limiting your respondents to only two choices, you ask them to see the subject as black and white. But if your aim is to analyze how people really think within the spectrum from black to white, then this is not a well thought-out question. So, sometimes it’s better to give them more choices. Example:

	Will you buy our product again?    Yes    No
	How likely are you to buy our product again?       Extremely likely    Very likely    Moderately likely    Slightly likely    Not at all likely

     

---

Measuring quantities

When you want to measure how much your survey respondents spend time, money, or energy on something, you can use a scaled question with 5 to 7 choices. If you believe your respondents probably can’t give you exact numbers, just use words like in the example below: Example: Verbal scale

About how much of your food budget do you spend on eco-friendly products?       All of it    Most of it    About half it    Some of it    None of it

However, if you think it is possible for them to answer with more exact numbers, and the numbers are important to you, go ahead and make some numeric intervals. It mostly depends on your data needs. Example: Numeric intervals

About how much of your food budget do you spend on eco-friendly products?       00 ↔ 20 %    21 ↔ 40 %    41 ↔ 60 %    61 ↔ 80 %    81 ↔ 100 %

 

---

Beware when making ranges

Remember not to make overlapping ranges. This makes people confused when they are trying to chose the right answer. For example lets say you ask them:  “About how long do you play games in a typical day?” and give them these choices: A) 1 hour or less, B) 1 to 2 hours, C) 2 to 3 hours, D) more than 3 hours. Which one should a person choose if he/she plays 2 hours per day? Choice B or C? Example:

	In a typical week, about how many days do you exercise outdoors?       1 to 3 days    3 to 4 days    4 to 7 days
	In a typical week, about how many days do you exercise outdoors?       never    1 to 2 days    3 to 4 days    5 to 6 days    every day

Therefore, you should try to keep your ranges roughly equal in size and make sure they do not overlap one another.  

---

Include a way out

Let’s say you ask a question about your respondents’ favorite flavor of ice cream. You may include some flavors in the options that you provide, but will never be able to include all flavors. In such cases, it’s better not to force the participants to choose only from your choices. Try to include an “Other” option. You can even let them write down their own favorite flavors, with an open-ended question and collect the data for further analysis. There may be very interesting data in such responses. Example:

	Which one is your favorite ice cream flavor?       Bacon ice cream    Garlic ice cream    Oyster ice cream    Hokey pokey ice cream    Salt and Straw ice cream
	Which one is your favorite ice cream flavor?       Bacon ice cream    Garlic ice cream    Oyster ice cream    Hokey pokey ice cream    Salt and Straw ice cream    Other