Anecdotes and Anecdotal Evidence.

[danydandan]

14 minutes ago, eight bits said:

This is probably more serious than the OP intends this topic to be, but here goes.

My estimate is that the median working scientist is "Humean" in their approach to thinking about evidence. As that relates to "anecdotal evidence," there is what you could call "David Hume's Membrane," easily adapted from his analysis of miraculous stories to the assessment of uncorroborated testimony generally:

Upon hearing a report, which would you be LESS surprised to learn:

- that the report is true?
- that the person making the report is mistaken, for whatever reason?

If the former, then the report sails through the membrane, and is provisionally accepted as true (always subject to possible refutation by other evidence). If the latter, then the report is caught on the membrane and dismissed with prejudice (subject to corroboration rehabilitating it, reopening the question of its truth).

Simple as it is, that "rule" makes for a fairly efficient way of life.

One complication that has emerged in the thread (as it does in real life) is whether a second or more similar report can serve as corroboration for a first report.

It would seem so intuitively. The American founders, wishing treason to be difficult to prosecute, adopted a refinement of the Biblical standard for conviction based on testimony: two witnesses to the same overt act (or confession in open court, Article III, Section.3 paragraph 1, US Constitution).

Actually, though, the form of the Membrane can be retained when adapted for multiple reports. Which is less surprising to learn, that the reports are true OR that every person making such a report is mistaken, for whatever reason(s)?

However, the implementation of that is complicated enough that you begin to appreciate what Laplace brought to the field when he introduced probability as a mathematical model of rational confidence. That parenthesis around the "s" reveals the difficulty: if there are distinct reports, how independent are they from one another?

That is, if they're wrong, are they each wrong for very different reasons peculiar to each report(er), or for the "same reason," or for closely related reasons, or...?  PLUS that question must be answered knowing that if they aren't wrong, then simply being observers of the same thing must introduce some dependency between the reports.

The quality of "independence" among the reports that we're looking for is pretty simple to frame up in mathematics, but not necessarily intutitive to articulate and apply without some practice using worked examples - as in doing calculations with realistic data.

Well, that's a lot to digest for a single post, but that's what I think is going on with "anecdotal evidence," and why it's sometimes accepted and sometimes not, why the plural of anecdote actually is sometimes data, but not in general, etc. Skeptics are people with a very sticky Hume's Membrane; scientists typically train theirs to be just sticky enough, and believers hardly have one at all .

As your very good post highlighted, subjective observations without controls are 'prickly', thus is why I believe that the scientific community cannot accept anecdotes as evidence. They can only truly be a precursor or a reason to start an investigation, scientifically speaking. In my opinion the same is true for corroborating anecdotes, all they can certainly say is an event occurred, it might indicate what may have occurred but these anecdotes shouldn't be seen as evidence to what actually occurred, that should be left up to independently gained evidence gathered from controlled conditions set by experiments carried out via the scientific method. 

Again I'm speaking from a purely scientific perspective, the law and other things use anecdotes as evidence all the time. Which is probably also incorrect application. 

[eight bits]

56 minutes ago, danydandan said:

Again I'm speaking from a purely scientific perspective, the law and other things use anecdotes as evidence all the time. Which is probably also incorrect application. 

I probably like what lawyers do with evidence better than you do . Remember, though, they have a different problem: the purpose of the trial is not only to establish the truth, but to proceed while respecting all participants' rights and achieving in the end justice, rather than admiration for one's accuracy.

56 minutes ago, danydandan said:

They can only truly be a precursor or a reason to start an investigation, scientifically speaking.

Yes, but that's irrelevant to the probative value of anecdotes. Assuming Richard Feynman is typical (and he is exemplary), you read his memoirs and notice that whenever a scientist gets interested in some factual question (not necessarily in their own specialty),  almost reflexively, they immediately begin thinking about systematic investigation. Systematic = opposite of anecdotal.

Poor anecdotes never get a chance to carry the day with scientists. As soon as anecdotes are taken seriously, they are overwhelmed by their opposite.

That's also how the median scientist (if my earlier estimate is good) gets away with staying stuck at a Humean level of evidentiary performance. Systematic investigation often gives categorical answers confidently backed by copious evidence. All that "manging the dependency of multiple sources" stuff? Your lot hires a consulting statistician to sort that out .

 

Edited February 9, 2019 by eight bits

[Copasetic]

8 hours ago, eight bits said:

This is probably more serious than the OP intends this topic to be, but here goes.

My estimate is that the median working scientist is "Humean" in their approach to thinking about evidence. As that relates to "anecdotal evidence," there is what you could call "David Hume's Membrane," easily adapted from his analysis of miraculous stories to the assessment of uncorroborated testimony generally:

Upon hearing a report, which would you be LESS surprised to learn:

- that the report is true?
- that the person making the report is mistaken, for whatever reason?

If the former, then the report sails through the membrane, and is provisionally accepted as true (always subject to possible refutation by other evidence). If the latter, then the report is caught on the membrane and dismissed with prejudice (subject to corroboration rehabilitating it, reopening the question of its truth).

Simple as it is, that "rule" makes for a fairly efficient way of life.

One complication that has emerged in the thread (as it does in real life) is whether a second or more similar report can serve as corroboration for a first report.

It would seem so intuitively. The American founders, wishing treason to be difficult to prosecute, adopted a refinement of the Biblical standard for conviction based on testimony: two witnesses to the same overt act (or confession in open court, Article III, Section.3 paragraph 1, US Constitution).

Actually, though, the form of the Membrane can be retained when adapted for multiple reports. Which is less surprising to learn, that the reports are true OR that every person making such a report is mistaken, for whatever reason(s)?

However, the implementation of that is complicated enough that you begin to appreciate what Laplace brought to the field when he introduced probability as a mathematical model of rational confidence. That parenthesis around the "s" reveals the difficulty: if there are distinct reports, how independent are they from one another?

That is, if they're wrong, are they each wrong for very different reasons peculiar to each report(er), or for the "same reason," or for closely related reasons, or...?  PLUS that question must be answered knowing that if they aren't wrong, then simply being observers of the same thing must introduce some dependency between the reports.

The quality of "independence" among the reports that we're looking for is pretty simple to frame up in mathematics, but not necessarily intutitive to articulate and apply without some practice using worked examples - as in doing calculations with realistic data.

Well, that's a lot to digest for a single post, but that's what I think is going on with "anecdotal evidence," and why it's sometimes accepted and sometimes not, why the plural of anecdote actually is sometimes data, but not in general, etc. Skeptics are people with a very sticky Hume's Membrane; scientists typically train theirs to be just sticky enough, and believers hardly have one at all .

 

7 hours ago, danydandan said:

As your very good post highlighted, subjective observations without controls are 'prickly', thus is why I believe that the scientific community cannot accept anecdotes as evidence. They can only truly be a precursor or a reason to start an investigation, scientifically speaking. In my opinion the same is true for corroborating anecdotes, all they can certainly say is an event occurred, it might indicate what may have occurred but these anecdotes shouldn't be seen as evidence to what actually occurred, that should be left up to independently gained evidence gathered from controlled conditions set by experiments carried out via the scientific method. 

Again I'm speaking from a purely scientific perspective, the law and other things use anecdotes as evidence all the time. Which is probably also incorrect application. 

 

6 hours ago, eight bits said:

I probably like what lawyers do with evidence better than you do . Remember, though, they have a different problem: the purpose of the trial is not only to establish the truth, but to proceed while respecting all participants' rights and achieving in the end justice, rather than admiration for one's accuracy.

Yes, but that's irrelevant to the probative value of anecdotes. Assuming Richard Feynman is typical (and he is exemplary), you read his memoirs and notice that whenever a scientist gets interested in some factual question (not necessarily in their own specialty),  almost reflexively, they immediately begin thinking about systematic investigation. Systematic = opposite of anecdotal.

Poor anecdotes never get a chance to carry the day with scientists. As soon as anecdotes are taken seriously, they are overwhelmed by their opposite.

That's also how the median scientist (if my earlier estimate is good) gets away with staying stuck at a Humean level of evidentiary performance. Systematic investigation often gives categorical answers confidently backed by copious evidence. All that "manging the dependency of multiple sources" stuff? Your lot hires a consulting statistician to sort that out .

 

 

Ehh, its not that we don't use anecdotal evidence in science and medicine, its just that we weight it differently. In the hierarchy of evidence, its the short, balding, fat guy at the end of the bar who doesn't pick up the ladies. It also can be subject to peer review. For instance, in medicine we publish anecdote regularly, we just call it sumfin' fancy: case-reports. In this way, anecdote in science and medicine can serve as novelty generators, but not much else. 

The explicit fault of anecdote in science is laid bare in its definition, that it is collection of data in a causal manner. There is no inherent flaw in causal collection of data within the limitations of its application. Not something many of us "Humeans" twist our panties in knots over. The implicit fault in anecdote is in the bias(es) they bring to the party, which are as many and varied as Haagen-Dazs ice cream flavors (really, Peach-blue cheese-ghost pepper? get out of here with that brah). Copious flavors notwithstanding, systematizing anecdote does nothing to address those implicit faults. Again, more novelty generation, but short on the probative prowess--no matter how many case-series you make of those reports. 

Hired statisticians aside (just one? nah 2 or 3, the departments paying so what the hell ), the kind of systematically produced, categorical answering, copious evidence we are looking for comes by way of hypothesis testing. And good little scientific hypotheses must be falsifiable.

So what's the recipe for a Humean membrane? Maybe 2 parts Hume: 1 part Wason: 1 part Bacon and 3 parts Popper. Mmmm, sounds delish 

[onlookerofmayhem]

Interesting article on why believers still believe:

https://www-sciencealert-com.cdn.ampproject.org/v/s/www.sciencealert.com/new-study-sheds-light-on-why-so-many-people-believe-in-psychic-powers/amp?amp_js_v=0.1&amp_lite#webview=1&cap=swipe

[Habitat]

4 hours ago, onlookerofmayhem said:

Interesting article on why believers still believe:

https://www-sciencealert-com.cdn.ampproject.org/v/s/www.sciencealert.com/new-study-sheds-light-on-why-so-many-people-believe-in-psychic-powers/amp?amp_js_v=0.1&amp_lite#webview=1&cap=swipe

Should they not have any such belief ?

[onlookerofmayhem]

14 minutes ago, Habitat said:

Should they not have any such belief ?

In my opinion, there is no reason to believe in any psychic phenomena. 

Obviously people who believe in it have a reason and are entitled to their belief.

 

[Habitat]

1 minute ago, onlookerofmayhem said:

In my opinion, there is no reason to believe in any psychic phenomena. 

Obviously people who believe in it have a reason and are entitled to their belief.

 

I like the way you worded that, certainly isn't the full-on rejection I see from quite a few here.

[papageorge1]

On 2/6/2019 at 5:51 AM, danydandan said:

 I would like this thread to discuss why proponents of anecdotes and anecdotal evidence, based on the terms I defined at the start, think we as a scientific community should just accept your anecdotes as evidence? And why you think your anecdotal evidence is valid in the first place?

 

Whatever happened to common sense and middle ground? We should CONSIDER anecdotal evidence which means neither outright acceptance nor outright dismissal.

Our human capacity for rational analysis is our great tool in a world of neither all black or white.

[danydandan]

16 minutes ago, papageorge1 said:

Whatever happened to common sense and middle ground? We should CONSIDER anecdotal evidence which means neither outright acceptance nor outright dismissal.

Our human capacity for rational analysis is our great tool in a world of neither all black or white.

All I'm saying is that anecdotes are a useful tool to determine if something is worth investigating. They should not be used to determine the validity of their own claims. 

By that I mean, it's sort of similar circular reasoning, if we do use anecdotes as evidence. 

[papageorge1]

29 minutes ago, danydandan said:

All I'm saying is that anecdotes are a useful tool to determine if something is worth investigating. They should not be used to determine the validity of their own claims. 

By that I mean, it's sort of similar circular reasoning, if we do use anecdotes as evidence. 

I would say then I agree as far as mainstream science is concerned.

Even as one who is pro-science, I am also interested personally in the analysis of things outside of what science can directly study at this time with their tools. I also consider anecdotes in that endeavor.

[Golden Duck]

On 11/02/2019 at 4:38 AM, papageorge1 said:

Whatever happened to common sense and middle ground? We should CONSIDER anecdotal evidence which means neither outright acceptance nor outright dismissal.

Our human capacity for rational analysis is our great tool in a world of neither all black or white.

Common sense is not all it's made out to be...

Quote

Common sense, in other words, is extremely good at making the world seem sensible, quickly classifying believable information as old news, rejecting explanations that don’t coincide with experience, and ignoring counterfactuals. Viewed this way, common sense starts to seem less like a way to understand the world, than a way to survive without having to understand it.

http://freakonomics.com/2011/09/29/the-myth-of-common-sense-why-the-social-world-is-less-obvious-than-it-seems/

[+DieChecker]

On 2/6/2019 at 2:51 AM, danydandan said:

In my opinion, and most other scientifically minded people, anecdotes should be used only to indicate the possibility of a phenomenon, (notice that I said possibility not probability), that may need further research or exploration. Anecdotes can be used to, I suppose, generate an hypothesis, but not to confirm one.

I would tend to agree. A eyewitness story of some encountering an angel, an alien, or bigfoot, are all not evidence of those beings, but can be used to determine, depending on the situation, if some phenomena may be occurring. 

If 2000 people say the saw a UFO in Arizona, that would be more worth following up on then a single person seeing the UFO.

Quote

I would like this thread to discuss why proponents of anecdotes and anecdotal evidence, based on the terms I defined at the start, think we as a scientific community should just accept your anecdotes as evidence? And why you think your anecdotal evidence is valid in the first place?

As a Christian, I think people telling anecdotes, or "witnessing", to people can be a powerful influencer. Especially when surrounded by other believers who have had similar experiences. 

I do think that such anecdotes should not be considered evidence. Certainly not evidence of God/Jesus. But I do think they allow people to get to a point of being prepared, where they are primed to have that same experience. Which will seem very real to them.

Humans operate on "feelings"... impulses... intuition... way more then a scientific, educated society would be thought to. I see it every day. Humans and logic are almost foreign to each other. People think they are good at making decisions, yet the automobile accident rates, and diverse rates, and poverty rates, and many other indicators put that to the lie.

So... I don't think (religious) anecdotes are evidence, but I think they can be used to give examples for social constructs, or influence others to join those constructs/religions.

On 2/6/2019 at 3:24 AM, ExpandMyMind said:

I see it as this: a single case of anecdotal evidence is evidence of nothing much at all, but multiple cases of aligning anecdotal evidence substantial enough to warrant further research.

It also depends on the type of anecdotal evidence. There is a clear difference between something observable and definable, such as the actions of other people, and something that is open to interpretation, such as that one time you saw a ghost that could also have just been a shadow.

Does it matter if 1 person saw Bigfoot, or that 2000 people saw Bigfoot? What if 2 billion people thought they saw Bigfoot? 

What about things like historical accounts? Did Hanno the Navigator, really discover Gorillas on the African coast over 2000 years ago, or is it just a report (anecdote/story)?

Edited February 15, 2019 by DieChecker

[Jodie.Lynne]

Question:

How many anecdotes, or anecdotal stories equals one datum?

[Mr Walker]

43 minutes ago, Jodie.Lynne said:

Question:

How many anecdotes, or anecdotal stories equals one datum?

One; if it is a factually correct anecdote. After all, data is just anecdotal evidence written down, where it then becomes NON anecdotal, by definition. 

 

[Jodie.Lynne]

30 minutes ago, Mr Walker said:

One; if it is a factually correct anecdote. After all, data is just anecdotal evidence written down, where it then becomes NON anecdotal, by definition. 

 

I disagree.

Data is information that can be repeated, tested, verified.

Anecdotes, without evidence, are merely stories.

My Aunt's brother's sister claimed that she was molested by the Gentle folk, and that was how she became an unwed mother.

Data point, or fairy tale? YOU decide.

[Mr Walker]

4 minutes ago, Jodie.Lynne said:

I disagree.

Data is information that can be repeated, tested, verified.

Anecdotes, without evidence, are merely stories.

My Aunt's brother's sister claimed that she was molested by the Gentle folk, and that was how she became an unwed mother.

Data point, or fairy tale? YOU decide.

No data is just data.

Under the right conditions it can be tested etc but does not have to be testable  

I once did traffic counts of moving and parked  cars and other vehicles  on several major roads and car parks.

What i collected was data and it could never be verified or repeated, exactly.  It was used for town planning and was quite effective 

I would need more details to decide about your   relative.  

I just picked 27 peaches and 42 apples from  our fruit trees to give to our neighbours.  

I also gave 2 cucumbers, and a dozen apples, to our gardener 

Data or anecdote?  

Ps they (apart from  the ones i gave to the gardener)  are sitting on my kitchen table and, if i had the skills, I could post a photo of them   

 

 

[Habitat]

3 minutes ago, Jodie.Lynne said:

Anecdotes, without evidence, are merely stories.

"There's many a slip twixt lip and cup". The idea that the word of someone has no inherent decision making value, is an absurdity, but it suits what is obviously your underlying agenda, which is to conscript the "rigour" of science to support your pre-conceptions, never mind if real science is quite mute about what you want to believe, in the negative.

[Jodie.Lynne]

17 minutes ago, Mr Walker said:

Under the right conditions it can be tested etc but does not have to be testable  

If a story cannot be confirmed, by neutral observers, or by repeatability, then it is not necessarily true. This is a point you seem unable to grasp. 

 

17 minutes ago, Mr Walker said:

I just picked 27 peaches and 42 apples from  our fruit trees to give to our neighbours.  

I also gave 2 cucumbers, and a dozen apples, to our gardener 

This is not an outrageous claim. From the data, I have gleaned from your many (many, many, many) posts, it is not inconceivable that this occurred as you related it.

 

17 minutes ago, Mr Walker said:

I once did traffic counts of moving and parked  cars and other vehicles  on several major roads and car parks.

What i collected was data and it could never be verified or repeated, exactly.  It was used for town planning and was quite effective 

Assuming that you did not manipulate the data, this would certainly count as factual data. What the information was used for, and whether you knew which way to slant the data (Not saying you did!) is another matter.

Let's say the data you collected was for the purpose of allowing, or denying, a permit for a corporation to construct a mall in the area. YOU are opposed to the idea (play with me here, OK?), so you omit a few datums to show that a mall would not be in the best interests of the community. No one can go back and verify your data, yeah?

 

My point being that outrageous claims (in the form of anecdotes), does NOT equal data, if it cannot be repeated or verified by outside researchers. People exaggerate. People lie, for a variety of reasons. YOU, Mr Walker, may trust people at first glance, but my background & history, cause me to look much closer at peoples motivations.

Edited February 21, 2019 by Jodie.Lynne
&#!&## typos!

[Habitat]

5 minutes ago, Jodie.Lynne said:

If a story cannot be confirmed, by neutral observers, or by repeatability, then it is not necessarily true

Nor is it necessarily untrue, and when you have a considerable body of such stories, all more or less consistent, it becomes less likely to be untrue.

[Jodie.Lynne]

2 minutes ago, Habitat said:

Nor is it necessarily untrue, and when you have a considerable body of such stories, all more or less consistent, it becomes less likely to be untrue.

Again, how many anecdotes equals one datum?

By your reasoning, we must assume that all tales of alien abduction are true; that all tales of bigfoot encounters are true; that all tales tales told of godly visitations are true, merely because there are many tales told of such.

Does one unsubstantiated tale equal data?

Does ten such tales?

A hundred?

A Thousand?

There are countless tales told of the 'little people' in Ireland. Does the number of stories mean that there is truth to these tall tales?

 

[Mr Walker]

23 minutes ago, Jodie.Lynne said:

If a story cannot be confirmed, by neutral observers, or by repeatability, then it is not necessarily true. This is a point you seem unable to grasp. 

 

This is not an outrageous claim. From the data, I have gleaned from your many (many, many, many) posts, it is not inconceivable that this occurred as you related it.

 

Assuming that you did not manipulate the data, this would certainly count as factual data. What the information was used for, and whether you knew which way to slant the data (Not saying you did!) is another matter.

Let's say the data you collected was for the purpose of allowing, or denying, a permit for a corporation to construct a mall in the area. YOU are opposed to the idea (play with me here, OK?), so you omit a few datums to show that a mall would not be in the best interests of the community. No one can go back and verify your data, yeah?

 

My point being that outrageous claims (in the form of anecdotes), does NOT equal data, if it cannot be repeated or verified by outside researchers. People exaggerate. People lie, for a variety of reasons. YOU, Mr Walker, may trust people at first glance, but my background & history, cause me to look much closer at peoples motivations.

Of course it is not necessarly true.

It is still data.

Not all data is correct 

Eg I wouldn't have a clue how many apples and peaches I picked. I just made the numbers up (although the rest was true) 

What has outrageous got to do with truthfulness ?  It is easier to  successfully lie about   believable things than non believable ones because people become trusting 

So  yes my traffic count and the  number of pieces of fruit i picked are both data, but unverifiable or repeatable  data- counts 

The y have to be taken on trust (or rejected) 

your point is valid yet insurmountable (one way of making this less liekly is to have two counters,and make sure the other one is not known to me (but i could still bribe them)   

Outrageous has nothing to do with it 

An anecdote is an oral account of an event etc 

Usually it is unverifiable But yes it can count as data, and yes it can be either true or false .

A t uni as students we did  LOT of data collection around the state on many geographical issues  it was all used for actual purposes ie it was accepted on faith as accurate and usable  I know mine was accurate and carefully collected, but its possible some was made up after an afternoon spent down the pub.

its true I trust people (although i am fairly perceptive to liars)  but that has nothing to do with it either.

It is just true that most data is collected first by individuals and cannot be checked or verified except by comparing and contrasting with other, similar, data.

   All data collection by humans  begins as anecdotal and is converted to non anecdotal when it is recorded and written  down for analysis 

To put it simply.  If data cannot be tested it cannot be validated; BUT, an inability  to test it does not INVALIDATE it 

 

Edited February 21, 2019 by Mr Walker

[Mr Walker]

4 minutes ago, Jodie.Lynne said:

Again, how many anecdotes equals one datum?

By your reasoning, we must assume that all tales of alien abduction are true; that all tales of bigfoot encounters are true; that all tales tales told of godly visitations are true, merely because there are many tales told of such.

Does one unsubstantiated tale equal data?

Does ten such tales?

A hundred?

A Thousand?

There are countless tales told of the 'little people' in Ireland. Does the number of stories mean that there is truth to these tall tales?

 

Maybe.

Maybe not.

It is logical fallacy to conclude,  without evidences, that ALL such stories are untrue, just as it would be to conclude they are all true.

 

 

[Habitat]

1 minute ago, Jodie.Lynne said:

Again, how many anecdotes equals one datum?

By your reasoning, we must assume that all tales of alien abduction are true; that all tales of bigfoot encounters are true; that all tales tales told of godly visitations are true, merely because there are many tales told of such.

Does one unsubstantiated tale equal data?

Does ten such tales?

A hundred?

A Thousand?

There are countless tales told of the 'little people' in Ireland. Does the number of stories mean that there is truth to these tall tales?

 

Your real agenda, which you are currently unsuccessful at achieving, is that ALL such anecdotes be rendered as having zero value, are untrue, and that materialism reigns supreme. You speak of wanting to know what it feels like to have faith in such things, when in reality you want to feel 100% satisfied that all "woo" is just fantasy or deceit. It sticks out a mile. Thus far, you have not managed to convince yourself, which shows that the inner dialogue continues, whereas the external one here is purely that of strident opposition to what you call "woo" !

[Jodie.Lynne]

7 minutes ago, Mr Walker said:

An anecdote is an oral account of an event etc 

Usually it is unverifiable But yes it can count as data, and yes it can be either true or false .

An anecdote is a recital of an event that may or may not be true or truthful.

Period. Full stop,

"While at the beach last summer, a little boy came up to me to show me the mermaid he had caught. Looking at the wee little baby mermaid, we both decided that she needed to be with her Momma, so I swam out beyond the breakers with the poor creature and released her. Soon, both baby and Momma mermaid were seen happily swimming away. This event was witnessed by several hundred people on shore."

 

Factual data? Or made-up story? You have a 50/50 chance of guessing correctly.

[Jodie.Lynne]

11 minutes ago, Mr Walker said:

It is logical fallacy to conclude,  without evidences, that ALL such stories are untrue, just as it would be to conclude they are all true.

The logical, default position, should be to disbelieve until E-V-I-D-E-N-C-E is presented.