Questionnaires that confuse and confound
Taking off from a post I have just made on my other blog – Why 70? Why not 68? Or 72? these are my reflections as I walk away from a “personality test” in the form of a true-false questionnaire. Apart from the ever-debated question of how is it possible, and how far is it posssible to quantify human emotions?, there is the issue of what kind of questions would reveal accurate answers to what a person is thinking – or feeling? And related to this, why this overwhelming need for numerical data – how do numbers make data or more importantly, insights, sacrosanct?
I do not have the questionnaire with me or remember the exact questions – but these are the kind of questions that feature in such tests (not the exact words – but somewhat) – the kind of questions that I kept analysing(!) as I got myself analysed.
1. self reporting : here is this educated trusting person handing over a long questionnaire to a person, expecting him or her to fill in each of the answers truly and honestly.
As a child, I have been set to the principal’s room for violent behavior in class.
I often have abnormal sexual urges which I cannot get rid of
Yes, indeed. While they are at it, why don’t they add to it – I am abnormally and stupidly honest
2. restricting : this is true – I am not making this question up – I loved my father
As opposed to what? I no longer love my father? I loved my dad but not my mother? I hated my father?
What if I mildly liked my father? And still do?
3. the double-barrelled question : and this happpens in more questionnaires than you would believe…
At work, I am known to be efficient and imaginative
You see it, don’t you? Only a few fortunate ones among us are both – or neither? Chances are, most of us are definitely one. And if we are not, in any case, go back and read what I have said about the abnormally honest bit. What if the respondent is only one of them?
4. iffiness – I am the researcher, I know exactly what I mean and what I want – does the respondent?
I get frequent attacks of nausea and vomitting – What is frequent? Is your idea of frequent bound to be the same as mine? And the next person’s?
5. the sweeper – what if I feel differently in different situations?
I believe it is wrong to give alms to beggars – well, with able-bodied young men, yes. With old ailing women, no. Oh, no, you can’t say all this- say true or false. Say just what you feel most of the time. Well, most of the time I feel as ambivlent about it as I just did.
6. how many negatives – sentences that twist and wind around themselves they give the respondent a crick in the neck.
I do not think it is not wrong to steal to satisfy one’s hunger – Uh, What was that again?
And these are just some of the “cardinal sins” of questionnaire design that we keep hearing about. What are your favorite ones?
This is what I came across just now on the net – The Only Thing Worse than No Information Is Bad Information
– Ron Sellers. And “bad information” is just what we land up with when the choice of research method is not indicated. And that is compounded by the kind of questions that go into the instrument.
Filed under: Research methods | 3 Comments