DRUNKEN DOWSING: cherry picking
I want to talk a little bit about a logical fallacy called “Cherry Picking,” but first I should bring you up to date on the dowsing test that my friend and I are working on.
I was really pleased with what my dowsing friend Tom (not his real name) said at the last party. He suggested for the dowsing test I hide rebar in one of several PVC pipes. I went to Home Depot and purchased some 2 ½“ PVC and 4 pieces of rebar. I plan to cluster the 4 pieces of rebar together and place the cluster in one of five PVC pipes for him to pick.
I was thinking that the day of the test Tom could designate 5 spots in the backyard that he feels are clear of any interference. We could put towels down to mark those spots. I would then place the PVC pipes on those spots. He would use his dowsing ability to pick which PVC pipe contained the cluster of rebar. We would do this a number of times. Each time I would switch out which piece of PVC contained the rebar.
Tom thought each round should have 10 choices (10 pieces of PVC). I liked that idea also but thought it requires a big reset for each round. One person can’t reset 10 pieces of PVC easily and because I am horribly cynical, there is no way I’m letting anyone else touch the PVC. Knowing my friends they would quickly become bored between rounds and start wandering away perhaps into the nearby lake. That would be tragic. I thought 5 pieces of PVC would be a better number because we can just double it to figure out the percentage. If we do the test ten times, and Tom has only chance on his side, he should be able to choose the correct PVC twice. 5 choices for each round also make it easy for me to randomize my PVC choices because I can use dice. I will just ignore the number 6 whenever I roll it.
The idea of videotaping all of this (which I love) also came up. I have an award winning filmmaker friend who said he is ready to film. I am required by him to mention the award any time I talk about him. So, Youtube, here we come.
This brings me to “Cherry Picking.” Tom and I both agreed that we shouldn’t let anyone else try dowsing at the party until after the test is finished and the dowsing claim is confirmed (or not). Letting everyone try dowsing first off leads to a false positive on tests called “cherry picking the results.” Basically cherry picking goes like this: During the test, Tom might not be able to show a greater then chance ability to dowse but (given enough people trying) someone will. That is just how averages work. Some people do below average. Some people do above average. If you get ten people each to flip a coin, say four times, the overall average is going to be around 50%. But, one of those people might get four heads in a row (just as one of them might get four tails in a row). If the person who got four heads in a row didn’t know any better he or she would walk away from the test thinking that every time they flipped a coin it will come up heads. That is what would happen if we let everyone take a swipe at guessing were the pipe is. By sheer chance alone one of those people (say Mary) has to do better than chance, just as one of those people (say Sally) has to do worse than chance. And then Mary will walk away thinking she has some dowsing ability. Everyone will forget about the original claim Tom made about dowsing and instead focus on Mary and her amazing skill. The belief in dowsing would continue and I would be left alone drunk and naked passed out in the back yard. I’m not sure how the last part would happen but trust me it always does.
I was really pleased with what my dowsing friend Tom (not his real name) said at the last party. He suggested for the dowsing test I hide rebar in one of several PVC pipes. I went to Home Depot and purchased some 2 ½“ PVC and 4 pieces of rebar. I plan to cluster the 4 pieces of rebar together and place the cluster in one of five PVC pipes for him to pick.
I was thinking that the day of the test Tom could designate 5 spots in the backyard that he feels are clear of any interference. We could put towels down to mark those spots. I would then place the PVC pipes on those spots. He would use his dowsing ability to pick which PVC pipe contained the cluster of rebar. We would do this a number of times. Each time I would switch out which piece of PVC contained the rebar.
Tom thought each round should have 10 choices (10 pieces of PVC). I liked that idea also but thought it requires a big reset for each round. One person can’t reset 10 pieces of PVC easily and because I am horribly cynical, there is no way I’m letting anyone else touch the PVC. Knowing my friends they would quickly become bored between rounds and start wandering away perhaps into the nearby lake. That would be tragic. I thought 5 pieces of PVC would be a better number because we can just double it to figure out the percentage. If we do the test ten times, and Tom has only chance on his side, he should be able to choose the correct PVC twice. 5 choices for each round also make it easy for me to randomize my PVC choices because I can use dice. I will just ignore the number 6 whenever I roll it.
The idea of videotaping all of this (which I love) also came up. I have an award winning filmmaker friend who said he is ready to film. I am required by him to mention the award any time I talk about him. So, Youtube, here we come.
This brings me to “Cherry Picking.” Tom and I both agreed that we shouldn’t let anyone else try dowsing at the party until after the test is finished and the dowsing claim is confirmed (or not). Letting everyone try dowsing first off leads to a false positive on tests called “cherry picking the results.” Basically cherry picking goes like this: During the test, Tom might not be able to show a greater then chance ability to dowse but (given enough people trying) someone will. That is just how averages work. Some people do below average. Some people do above average. If you get ten people each to flip a coin, say four times, the overall average is going to be around 50%. But, one of those people might get four heads in a row (just as one of them might get four tails in a row). If the person who got four heads in a row didn’t know any better he or she would walk away from the test thinking that every time they flipped a coin it will come up heads. That is what would happen if we let everyone take a swipe at guessing were the pipe is. By sheer chance alone one of those people (say Mary) has to do better than chance, just as one of those people (say Sally) has to do worse than chance. And then Mary will walk away thinking she has some dowsing ability. Everyone will forget about the original claim Tom made about dowsing and instead focus on Mary and her amazing skill. The belief in dowsing would continue and I would be left alone drunk and naked passed out in the back yard. I’m not sure how the last part would happen but trust me it always does.
Comments
http://footballexperttips.blogspot.com/