Monday, November 15, 2010

Ad hoc boundaries, ethics, and other discussion inspired thoughts...

This probably won't be the most well laid out blog post, but thoughts and comments and questions from today's discussion in class keep swirling around in my head, so I figured better here where someone might expand and explain and comment than in my head where only I contribute.

We talked quite a bit about Hine's boundaries thing, and how some people are pro setting them beforehand and being relatively rigid about it, and some prefer a more negotiable approach. On either end, aren't there dangers? I'm not trying to play devil's advocate, but if you set them before you begin actual research, how can boundaries be any less than arbitrary? And if you 'renegotiate' to set them after... hello ad hoc boundary creation. Or a little more texas sharpshooting, please. How can you avoid BOTH criticisms in your work? Is being self-reflexive enough? And if not, which is the lesser of the two evils in the world of your peers?

We also talked about framing and validity. The point was raised that perhaps findings validate your method, but of course poor method = poor finding validity, and furthermore, sometimes the findings (most of the time, that is) are inconclusive, so what does that say about your method? But we moved into discussions of framing, and the suggestion was made that framing the question appropriately can make a set of findings more or less valid (this is where Brian's discussion of the trolling DIY conference presenter and his anti-hack-lab talk come in).
Can really fantastically well done framing make up for not-so-solid-and-fabulous-methodology?

Last but not least... we didn't really touch on this point, and maybe we won't ever, but Christina's discussion of stats- when they're valid, when they're not, etc. etc., certainly raised an issue I've had for a long time. Full disclosure: I love stats, and took a whack of stuff about stats in undergrad, and am still the obnoxious person in presentations who asks after people's alpha levels. But what gets me the most is that in all of my classes, as I came to better understand how much (and how very, very, very little) stats actually tell us, the more I came to realise how much faith people put into them. I'll never forget the quote written in to the front cover of my first stats class second-hand-text: "There are lies, there are damn lies, and then there are statistics." (which according to Wikipedia goes back to an article in Nature in Nov 26, 1885: :"A well-known lawyer, now a judge, once grouped witnesses into three classes: simple liars, damned liars, and experts." ).
In any case, when you're reporting your findings, and writing your lovely abstract... what are your responsibilities as an author and researcher, ethically? Do you assume no one will actually read your entire piece, and therefore make sure the abstract doesn't overstate? Do you avoid any mention of stats (specifically awkward "false stats") knowing how misleading they are to most of the population? In an abstract, when people read "88 % said ... ", I think it's fair to say many people will decontextualize (even without meaning to do so) and assume that applies to 88% of EVERYONE EVER RELATED TO THIS TOPIC. Of course people make this assumption- otherwise advertisers wouldn't use creepy true-ish "stats" like "9 out of 10 dentists" and "80% of Moms use". This may also be because of some of that discussion Mike and Rebecca had around rhetoric and numbers being "neutral".
So knowing all this-- where do your ethical responsibilities lie?
And given how many sensational news headings come from not-at-all-well-represented-studies (ridiculous things like "X Gives You Cancer!" (did you know there was conclusive proof? Me either), and "Fact: New Scientific Study Says X")... do you have a responsibility to try to minimize this potential misuse of your work?

No comments:

Post a Comment