Thread: Can questionnaire research ever be trusted?

A lot of behavioral research is done by asking random people to fill out questionnaires.

However i know that when i do a questionnaire although my answers may not be outright lies the truth is somewhat subverted and the answers i give are not 100% accurate. even if the questionnaire is anonymous i almost sub-consciously give imprecise answers.

Now it could be said that this kind of uncertainty is just a random uncertainty and so by collecting enough data this is cancelled out. However consider this argument.

say we take a 100 people and ask them "would you ever screw over a colleague to get a promotion."

the answers here give a distribution of 25% yes and 75% no.
now if we take into account sub-conscious/conscious lying. Then the results may be innacurate.
Now you could argue that some people who said yes actually meant no, and some who said no actually meant yes. so that a large amount of data means that a balance should be struck.

However it is clear that the people who said yes in this situation have a lower moral compass and therefore their answer is more likely to be truthful as obviously they don't believe there is a problem with screwing someone over.

therefore the "Untruthfulness" of the answer is not balanced from both sides. this means that we have a systematic uncertainty in our experiment, and that actually taking larger amounts of data increases the uncertainty. A systematic uncertainty is caused by a problem with the instrument being used, which in this case is the questionnaire.

So can the questionnaire ever really be trusted?

BTW - how can i get images from my hard disk e.g.JPG,GIF, onto a post like this?
it would make my point easier to explain.

In my opinion, questionnaire-based research is best suited for initial, exploratory level research, and the information should most definitely always be taken with a grain of salt and re-tested with objective measures before any solid conclusions are made.

You have to understand that the use of questionnaires is largely an issue of practicality. Is there an objective way to measure what you're looking for? And if so, how accurately does the measure reflect the information you think it does? And how difficult (complex, time consuming, expensive, unpleasant for the volunteers) will it be to get those objective measures?

Sometimes the results commonly used questionnaires will be compared to the results of objective measures, and in the case where they do in fact reliably match up, doing a questionnaire is probably the easiest route to your goal. Or if they differ, but differ consistently and predictably, that would still favor the use of a questionnaire.


You'll have to upload your image onto some kind of webserver, like Imageshack or something like that (there are tons of online picture sharing websites you can do this at). Then once you have the URL of your picture, you put it in between the img brackets in a post.

Sometimes you have to metagame to get at the truth. For example if you want to correlate SIDS deaths with the overall keenness of parental care, you can't just ask "Do you care for your baby?" or "How much?". So you metagame with, "Several studies have shown a decreased incidence of SIDS associated with the bright pattern of a Croatian flag hung over a crib. Would you like more information about these studies? Would you be willing to hang a (provided) flag over your baby's crib as part of this ongoing research?"

Idk but participating as a research subject for psychology grad students is an easy way to make a couple bucks.

That and donating blood, there's always a scientist around in need of human blood.

The last study I participated in was on the influence of media on opinions of race relation.

I had to answer a survey online a week before going in, in person. Then I was locked in a room alone and asked to read a paper. 1 of 2 randomly chosen, one about the troubles faced by minorities, and one which I never saw but was apparently was critical of minorities in some way. They then made me re-do the survey after reading the paper. After which they compared the two surveys to see if there was an effect. I don't think a study like this could show how people are affected in any decent way, but it could give you a general idea if people are affected.

Questionnaires are ground level research I'd say. It helps you get your foot in to see exactly the rough average data you can expect. But there are so many criticisms of questionnaires (particularly in psychology) that I don't even know where to begin with them. One instance is Yahoo!'s questionnaires. I recently saw one saying that Britains favourite free time activity was making love, sex, whatever you want to call it. I read the article and saw at the bottom that this entire publicised statement across the world was deduced by taking the questionnaire to 2,500 odd young British citizens. Hmm. The number and the age itself clearly puts it in incredible bias.

To add to what some of the others here have said, my experience is primarily from the world of market research, rather than psychological research, and in market research the well-designed questionnaire is an invaluable tool.

Of course, in market research issues like sample size (yes Chaotic, 2500 people can, if correctly chosen, be representative of the ritish population), and sampling itself (various old chestnuts like the 'left hand rule' and so on) are quite important. This does not make it perfect - see the sometimes dire results of the psephologists - but it can be a very reliable indicator of what is working and what is not.

You wouldn't use it, though, as a means of exploring the human mind - but then market researchers are happy to scratch the surface: they're looking for reliable levers, not deep explanations.

i think that they can be more accurate then guessing but yes you need to take in account of the persons mood, position, sub concious and sub-consious. Theres also another factor but i forgot the name of it. Its when a subject to a test finds out that he himself is being observed and he subconsiously is observing him self.

hawthorne effect