16.2 POINTSCONSIDEREDWHENWRITINGANDINTERPRETINGQUE...
Clarity.This is probably the area that causes the greatest source of mistakes in questionnaires.Questions must be clear,succinct,and unambiguous.The goal is to eliminate the chance that the question will mean different things to different people.If the designers fail to do this,then essentially participants will be answering different questions.
To this end,it is best to phrase your questions empirically if possible and to avoid the use of necessary adjectives.For example,if asking a question about frequency,rather than supplying choices that are open to interpretation such as:
(1)Very Often
(2)Often
(3)Sometimes
(4)Rarely
(5)Never
It is better to quantify the choices,such as:
(1)Every Day or More
(2)2-6 Times a Week
(3)About Once a Week
(4)About Once a Month
(5)Never
Leading Questions.A leading question is one that forces or implies a certain type of answer.It is easy to make this mistake not in the question,but in the choice of answers.A closed format question must supply answers that not only cover the whole range of responses,but that are also equally distributed throughout the range.All answers should be equally likely.An obvious,nearly comical,example would be a question that supplies these answer choices:
(1)Superb
(2)Excellent
(3)Great
(4)Good
(5)Fair(https://www.daowen.com)
(6)Not So Great
A less blatant example would be a Yes/No question that asked:
Is this the best CAD interface you have ever used?
In this case,even if the participant loved the interface,but had a favorite that was preferred,she would be forced to answer No.Clearly,the negative response covers too wide a range of opinions.A better way would be to ask the same question but supply the following choices:
(1)Totally Agree
(2)Partially Agree
(3)Neither Agree or Disagree
(4)Partially Disagree
(5)Totally Disagree
This example is also poor in the way it asks the question.Its choice of words makes it a leading question and a good example for the next section on phrasing.
Phrasing.Most adjectives,verbs,and nouns in English have either positive or negative connotations.Two words may have equivalent meaning,yet one may be a compliment and the other an insult.Consider the two words“child-like”and“childish”,which have virtually identical meaning.Child-like is an affectionate term that can be applied to both men and women,and young and old,yet no one wishes to be thought of as childish.
A more subtle,but no less troublesome,example can be made with verbs that have neither strong negative nor positive overtones.Consider the following two questions:
—Do you agree with the Governor's plan to oppose increased development of wetlands?
—Do you agree with the Governor's plan to support curtailed development of wetlands?
They both ask the same thing,but will possibly produce different data.One asks in a positive way,and the other in a negative.It is impossible to predict how the outcomes will vary,so one method to counter this is to be aware of different ways to word questions and provide a mix in your questionnaire.If the participant pool is very large,several versions may be prepared and distributed to cancel out these effects.
Embarrassing Questions.Embarrassing questions dealing with personal or private matters should be avoided.Your data are only as good as the trust and care that your respondents give you.If you make them feel uncomfortable,you will lose their trust.Do not ask embarrassing questions.
Prestige Bias.Prestige bias is the tendency for respondents to answer in a way that makes them feel better.People may not lie directly,but may try to put a better light on themselves.For example,it is not uncommon for people to respond to a political opinion poll by saying they support Samaritan social programs,such as food stamps,but then go on to vote for candidates who oppose those very programs.Data from other questions,such as those that ask how long it takes to learn an interface,must be viewed with a little skepticism.People tend to say they are faster learners than they are.
There is little that can be done to prevent prestige bias.Sometimes there just is no way to phrase a question so that all the answers are noble.The best means to deal with prestige bias is to make the questionnaire as private as possible.Telephone interviews are better than person-to-person interviews,and written questionnaires mailed to participants are even better still.The farther away the critical eye of the researcher is,the more honest the answers.