To test pilot our questionaire for BCM210 (as well as my own interview techniques) I recorded an interview with an old housemate and recent UOW graduate Matthew Bernard – a technique advocated for by Priest (2010).
I think this process largely worked pretty well. I probably needed to talk with more authority and clarity when asking the questions at times, but I was pleased with my ability to ask appropriate clarifying questions on the fly when it was needed. I think that had I practiced the questions a little more beforehand to familiarize myself with them I would have sounded a little more confident in my delivery (Wimmer and Dominick 2006).
I also think I did a pretty good job of making clarifying statements or questions without falling into the trap of pushing my subject into answering in a certain way. Having said that it was probably not the best to say “…I assume you don’t read newspapers or magazines” because that does discourage the subject from saying that they do if I am mistaken (although in this case having lived with Matt I already knew this to be true – so my familiarity with the subject played a part in this mistake as well). This is also an example of how easily biases can be unintentionally encoded into the questions through their wording (Priest 2010)
Evidently the final question “can you list 5 words you associate with the type of science news/information you use?” has some issues in it that need to be ironed out. The way the question is phrased currently suddenly puts a large, on-the-spot demand on the subjects ability to answer in a way that they’re satisfied with. It’s essentially a double-barrelled question with a certain prestige bias – in that the respondent may actually be assessing their own scientific understandings just as much as the scientific content they consume (De Vause 2002) We originally intended to include this question as a focus group question, thinking it would be too open ended for a written survey. But I think that it’s a lot for the subject to think about on the spot. Matt was visibly caught off-guard by this question and even after some additional prompts by me he only really got around to listing two words rather than the requested five? As Krueger and Casey (cited in Wimmer and Dominick 2006, p. 132) the questions in a focus group work best when they are conversational, short and clear; which I don’t think is true of this question. So either this question might be better for the written survey after all or it might need to be tweaked slightly to make it less intimidating in an interview or focus group scenario.
I also feel that we need to develope just a few more questions to help develope a clearer picture of how subjects engage with scientific news and how they perceive their own scientific literacy. It also dawned on me the importance of asking how interested the participants were in any aspects of science at all. Since conducting this interview I have developed around four further questions for the group to consider using in the survey.
References
De Vaus, D A 2002, ‘Constructing questionnaires’, in De Vaus, D. A. Surveys in social research, 5th ed, Allen & Unwin, St. Leonards, N.S.W, pp. 94-121
Priest, S H 2010, ‘Designing quantitative research: surveys, experiments, and quantitative content analysis’, in Doing media research : an introduction, 2nd ed., Sage, London, pp. 71-94
Wimmer and Dominick 2006, ‘Qualitative Research Methods’, in Mass Media Research (no further information available)
