Thursday, September 06, 2007

Anecdotal Ignorance

I apologize if anyone went to last night's debate hoping for a scandal.

What I (and most people) expected to be a heated, emotional debate, turned out to be two men having very different conversations in the same space.

Indeed, the most exciting moment of the evening was when a terrifyingly large Palmetto bug flew out from a corner and landed on Brian McCarter's arm.

But I am not here to talk about insectoid interference.

What stunned me the most, aside from Mr. Schwartz's inability to answer a question directly, was his refusal to acknowledge the relevance of an informal poll of Mr. McCarter's.

While it can be universally agreed that if you want to learn about a religion, you go to a leader of that faith. However, I feel that if you want to learn about its place in a culture, you go to the people.

Mr. McCarter went to a predominantly Muslim neighborhood in Queens, and asked 20 young Muslim men if they felt the First Amendment was good, then repeated the question with a very extreme qualifier.

Mr. Schwartz refused to acknowledge this information as "representative of moderate Muslims."

At no point did he define what a Moderate Muslim is, nor who would be described as such.

While I appreciate his desire for accurate information coming from informed sources, for the debate at hand, I (and many others) felt that "The Man on The Street" would be the better source. If you're talking about immigration, and the potential for restrictions, Average Yusuf may be a better representative than a more official individual.

If I have a question about Halacha, I'll ask a rabbi. If I have a question about what it's like to be a young, female, observant Conservative Jew in New York, I'll ask my old roommate.

If I have a question about a Mystery of Catholicism, I'll ask a priest. If I have a question about Catholic youth culture in America, I'll ask my friend Alissa.

Which brings me to a question I've been mulling for many weeks now...

There's a saying that goes, "Anecdotal evidence isn't."

Okay, fair enough. The story your friend told you about her cousin's best friend in Ohio probably isn't a good source of information. But then you are pressed for official information. From studies. From journals.

I can think of at least one recent "official study" that's totally bogus.

Meanwhile, there are hundreds (thousands?) of minority groups who are dealing with injustices and oppression, but their stories either aren't heard, or deemed irrelevant because no one's put out an official study about them.

If I can't use the personal experiences of a Female-to-Male transexual professor at MIT to demonstrate sexism in the science community, and no study has been done on it, what use is that person's complaint? Or the complaints of other women in the science community? (This happened in an email conversation I had recently.)

This feels like a dirty trick by the majority to determine what is relevant and valid, and what is just the "uppity" minority whining about something or other.

We can't cherry pick. At what point do situations become relevant/interesting/exotic/dire enough to warrant studying? At what point do anecdotes become data?

I really am curious about this. Why, in so many instances, does something have to be sanctioned for its information to be "real?"


northern_ophelia said...

Part of it has to with an overall insistence on the idea of "scientific fact" being the highest criterion for EVERYTHING. Unfortunately, the social sciences are quite a bit behind the "hard sciences" in actually being able to predict the outcome of situations OR to establish actual causal relationships. It isn't their fault; they haven't had as much time to develop.
A study that comes out and says that a virus has proven to cause syndrome A by causing symptoms 1, 2 & 3 is reported in the same matter as one that says women are as talkative as men or one that says obesity is transferred through social networks. And all of them are often treated as if they are as factual as a study saying water is made of hydrogen & oxygen.
They aren't, but its easier to treat all of them as equally representative of the "truth."
And so if you can't manage to come up with at least 1 study that "proves" your assertion, obviously you can't be representing what's really going on in the world.

Joshua Logan said...

While there are a number of "soft sciences" that are new, they are as old, and in some cases, even older than the "hard sciences." The study of communication is one such example, with the last century giving way to countless breakthroughs of understanding while Aristotle was one of the first people to define rhetoric and give it rules that are still applied today.

The problem lies not in the age of their development, but an issue inherent to the social sciences: they are not based on solid consistently verifiable data, but are affected by the people involved at the time it occurs. There is no exception to gravity, only variables that can affect it. But communication (as an example that I know far too well) changes under a million and one different circumstances.

The studies, however "true" or not, have to be published and reported in the same fashion as the hard sciences. To begin, it is out of respect for researchers who spend their time devoted to this. Additionally, by reporting, it allows people to learn more and should cause people to question it and thus, produce more research and eventually find answers.

"Anecdotal evidence" is very useful in the social sciences. However, it is very costly, thus they often choose quantitative data over qualitative data. The kicker is that you can not use anecdotal evidence to explain a problem because we do not know all the variables. If all variables are accounted for, then yes, we can say there is an issue, at least more so than if we just go by a few anecdotes.

Issues become enough to study when 1) there is enough social response to the issue, or 2) someone takes it upon themselves to study it because it impacts them/makes them curious. It has to be sanctioned because information has to be verified by a group of peers to be accepted by the (academic) community. Flukes occur, and it's the job of researchers to find the consistently recurrent problems.