Most of the questions used in a survey can only be answered by a closed set of options (aka “closed questions”). However, it is also possible to ask questions that allow the respondent to answer anything, without a list of possible alternatives (aka “open ended questions”).
There are two main types of open ended questions (Couper et al, 2011): ones that lead to very short answers (e.g., place of birth, or preferred beer brand) and ones that require narrative responses in the form of more elaborate and detailed responses (e.g., what do you think will be the next challenges for you in the future?).
Open ended questions to collect narrative responses have some pros and cons. On one hand, answers cannot directly be used for quantitative analysis, as coding needs to be done to be able to work with these responses. This coding can be time consuming and quite complicated if the answers given by the respondents are not directly in line with what the researcher wanted to measure (see our previous post on questionnaire design). On the other hand, they allow us to obtain a much wider range of responses, with more detailed content and information. Additionally, respondents are not guided in any particular direction, which allows them to answer in a more natural way that resembles a dialogue, giving greater freedom to the respondents to give accurate answers. Hence, these types of questions tend to better suit the researcher’s needs than closed questions.
In face-to-face or phone surveys, the respondent can give an answer of the length of their choice. But in an online survey, the researcher needs to decide the best way of presenting the open ended question. Particularly, two decisions have to be made: the text box size (the visual box in which the respondent writes the answer) and whether or not to include a counter indicating how many characters are available.
Results from different research projects on the impact of the size of the text box (Christian and Dillman, 2004; Israel, 2010; Smyth et al. 2009) agree that a bigger text box produces longer responses. There is also scientific evidence showing that the use of a counter can lead to an increase in the average number of characters in the responses (Emde y Fuchs, 2012).
Moreover, Emde and Fuchs (2013) studied the interaction between the text box size and the number of characters of the counter. The results are quite intuitive: a counter with more characters than the text box size shown at the beginning produces longer answers, whereas a counter with fewer characters leads to shorter responses. Therefore, it seems that respondents take the counter into account more (when there is only one) than the reference provided by the size of the text box.
Furthermore, Emde and Fuchs (2013) also conclude that both the text box size (150 characters or 300 or 600) and the counter presence do not affect the nonresponse rate, and neither the number average of mentioned topics. Therefore, what really changes is the level of detail in the provided answers.
In conclusion, to collect narrative responses with open ended questions, it seems advisable to use a text box in combination with a counter which indicates more characters than initially visible (up to 33% more). It is preferable to use a relatively large text box, without being too excessive, since combining it with a counter with more characters means answers will be longer and, moreover, it will not change the number of mentioned topics.
Bibliographic references: