Questionnaires that confuse and confound

02Nov06

Taking off from a post I have just made on my other blog – Why 70? Why not 68? Or 72? these are my reflections as I walk away from a “personality test” in the form of a true-false questionnaire. Apart from the ever-debated question of how is it possible, and how far is it posssible to quantify human emotions?, there is the issue of what kind of questions would reveal accurate answers to what a person is thinking – or feeling? And related to this, why this overwhelming need for numerical data – how do numbers make data or more importantly, insights, sacrosanct?

I do not have the questionnaire with me or remember the exact questions – but these are the kind of questions that feature in such tests (not the exact words – but somewhat) – the kind of questions that I kept analysing(!) as I got myself analysed.

1. self reporting : here is this educated trusting person handing over a long questionnaire to a person, expecting him or her to fill in each of the answers truly and honestly.

As a child, I have been set to the principal’s room for violent behavior in class.
I often have abnormal sexual urges which I cannot get rid of

Yes, indeed. While they are at it, why don’t they add to it – I am abnormally and stupidly honest

2. restricting : this is true – I am not making this question up – I loved my father

As opposed to what? I no longer love my father? I loved my dad but not my mother? I hated my father?

What if I mildly liked my father? And still do?

3. the double-barrelled question : and this happpens in more questionnaires than you would believe…

At work, I am known to be efficient and imaginative

You see it, don’t you? Only a few fortunate ones among us are both – or neither? Chances are, most of us are definitely one. And if we are not, in any case, go back and read what I have said about the abnormally honest bit. What if the respondent is only one of them?

4. iffiness – I am the researcher, I know exactly what I mean and what I want – does the respondent?

I get frequent attacks of nausea and vomitting – What is frequent? Is your idea of frequent bound to be the same as mine? And the next person’s?

5. the sweeper – what if I feel differently in different situations?

I believe it is wrong to give alms to beggars – well, with able-bodied young men, yes. With old ailing women, no. Oh, no, you can’t say all this- say true or false. Say just what you feel most of the time. Well, most of the time I feel as ambivlent about it as I just did.

6. how many negatives – sentences that twist and wind around themselves they give the respondent a crick in the neck.

I do not think it is not wrong to steal to satisfy one’s hunger – Uh, What was that again?

And these are just some of the “cardinal sins” of questionnaire design that we keep hearing about. What are your favorite ones?

This is what I came across just now on the net – The Only Thing Worse than No Information Is Bad Information
– Ron Sellers. And “bad information” is just what we land up with when the choice of research method is not indicated. And that is compounded by the kind of questions that go into the instrument.

****

technorati tags: , , ,

Advertisements


3 Responses to “Questionnaires that confuse and confound”

  1. I had a chance to fill a questionnaire yesterday – and I admit it was no where near the weird crazy questions in the post. My contention was about the “Qualititavity” of the “Inherently Desirably Quantitative”. (Excuse the terms, I was terribly influenced by the questions in the post).

    After the sesion was done with, I was asked to give my feedback to the different aspects of the session, in terms of: (E) Excellent, (G) Good, (P) Poor.

    Hmmmm. That was tricky. I mean, the interpreting part of the completed feedback. How bad is poor? How good is good? And what would it take to be excellent? Should there have been an option – “(H) Horrible” or “(I) Intolerable”? Would it have conveyed more sense if it had been from 0 to 5, 6, 0r 10, perhaps?

    But then, there would always be people who would want a (-)1, (-) 2 etc till (-)10?

    That apart, I was carried away by a pair of questions in a personality assessment test:

    Question x. When I leave home for a tour out of station, I generally leave my house in a mess with things scattered around haphazardly.

    A couple of questions later:
    Question x+2. When I return from my tour, I find things messy and scattered around haphazardly in my house.

    Eh?


  1. 1 A Time To Reflect » Blog Archive » Why 70? Why not 68? Or 72?
  2. 2 More on bad survey design « MindSpace

%d bloggers like this: