Blog - Authors

You can find EVIDENCE to support ANY view?


Posted on January 24th, 2012 by Jeff Thiboutot M.S.

There seems to be a fairly common view that virtually any idea or concept can be supported with “evidence”. I have had a couple of recent comments that have highlight this view.

One of the recent comments was;

“All scientific research is based on controlled studies, all of which can be arranged to “prove” any outcome that you want. I can setup a viable study to have it demonstrate that protein is bad and whole grains is good, even if that is not the case.”

I also think that these types of statements are used by people who feel that their view on a subject is challenged. They use this position as some type of argument.

Is this true, can we prove anything we want and find research to back up anything? On the surface there may be some truth to these statements. However, when you look behind the curtain you really CAN’T prove everything or find research to back up anything, IF, and this is a big IF, you play by the “rules”. Let me explain.

When it comes to research and doing studies there are many variables that will determine the internal and external validity of it. Basically, the type of research and how the studies are done will determine how well it can demonstrate a CAUSE and effect relationship. There are many aspects that help us determine the “quality” of the research and therefore, how much emphasis we can put on it. Some of the variables include (this is not an exhaustive list);

  • Sample size
  • Sample type
  • Randomization
  • Length of intervention
  • Blinded or not
  • Type of statistical analysis
  • How well variables were controlled (i.e., metabolic ward versus free-living)
  • Type of study (randomized controlled trials, cohort, case-controlled, cross-section survey, etc)

These variables will determine if the study actually demonstrates some kind of effect and if we can determine WHAT actually CAUSED the effect.

Before making any conclusive judgments, it is generally recommended that a similar effect has been produced more than once.

These would be the “rules” of research and if they are followed it is NOT likely that anything can be supported. I think anyone familiar with the many aspect of research would agree. You just CAN’T prove anything, just like you CAN’T say that left is right, up is down, or that researchers always follow the rules.

This last point also relates to the view that “anything” can be supported or there is more uncertainty than there is on a topic. Researchers are human, therefore, like us mortals, they to do NOT always follow the rules. This means that the conclusions from a study may actually have little or no validity. This can be due to weaknesses in the above “rules”. However, the problem can actually be with how the authors state the results in the conclusion of the paper which is the part that is included in the abstract, which often translates into a misleading title of a paper (due to the the peer-review process this should really not happen, but it does). These latter points are what are often publicized and contribute to the confusion.  However, if someone were to look at the meat of the paper, the boring information explaining the “rules” of the study and the actual results, a different conclusion may be more likely or at least a less “definitive” conclusion would be warranted. I am not saying that this happens all the time, but it happens enough to muddy the waters and leads to the impression that “anything” can be supported with evidence.

This problem also stems from biases (often we are not conscious of) and a lack or lapse of critical thinking (it is very demanding) that happens to researchers, writers, and practitioners. We WANT to believe certain things, therefore we can easily focus on what seems to support our current beliefs and miss the information that may challenge our current beliefs.

On a superficial level, there is certainly an impression that most anything can be supported with evidence. But, when a deeper perspective is taken, this view loses its validity. I am NOT saying that things are black and white, there is definitely a lot of grey. But, the grey area is often more like nuances of a topic rather than a complete divergence from what the “quality” evidence actually demonstrates. With that said, I don’t think that the argument that anything can be supported with evidence or any study can be manipulated to show any result are valid IF the rules are followed. When the rules are followed and we become aware of our biases and implement critical thinking skills there is often a rather small range of possible conclusions that can be supported by evidence. 

 

 

Please share with friends...

7 Responses to “You can find EVIDENCE to support ANY view?”

  1. Sizzlechest says:

    “I also think that these types of statements are used by people who feel that their view on a subject is challenged. They use this position as some type of argument.”

    The problem is what passes as proof. Most of the health related news articles that claim something helps (or harms) are really based on the results of an observational study. Even if the results of the data from “The China Study” showed a statistically significant correlation between animal protein and health problems, it doesn’t really prove anything. That’s not going to stop people from using it as evidence. What’s worse is that many of the people in the upper echelons of the medical world think that’s just fine and dandy.

    This describes the issue quite succinctly: http://www.phdcomics.com/comics/archive.php?comicid=1174

  2. Matt Schoeneberger says:

    The first thing that strikes me when someone uses this argument is “That just means you’re not checking the references.” Jeff and I, as well as many others, put our references behind our work so that readers can check them. I’ve often put completely fake and ludicrous references in the list, just to see if anyone is reading them. I haven’t been called on one yet.

    I can read an author’s work and be convinced solely by the fact that he has references, assuming he’s done his homework. Or I could actually review the references myself and use my own judgement.

    The responsibility is that of the audience. If an audience member is not willing to put in the work and call BULLSHIT! when it’s necessary, they’ll be convinced of anything or nothing at all.

  3. Matt Schoeneberger says:

    Sizzlechest, that comic is perfect!

  4. Susan says:

    I do believe that one may find evidence to support just about any view”….they may not correct but, it can be done…

    example..I can say that left is left and right is right but, if you are facing me, your left is different and your right is different…..

    The point being..it is a matter of perspective!

  5. Chris says:

    Matt, for those of us without scientific background I can attest to the fact that although I have repeatedly tried to check a variety of references and research papers, it is worse than difficult to try to trudge through one and understand what is being said. To also be able to read it critically to find flaws, is just short of impossible.

    Unfortunately, that can put me at the mercy of researchers and their biases.

    However in the particular instance of the SAD, personal experience let me know that something was not right. That recognition led me to read, read, and read.

    Slowly, I am getting better at reading these papers, but it is work. We can’t all be experts in all fields, in the end I have to trust someone’s interpretation of the research. Hopefully, I find the ones that understand that a well planned protocol and cleanly collected data is not really open to feelings or perspective.

  6. Matt Schoeneberger says:

    Cris, I understand what you’re saying. I think this is where it’s important for an author to explain the pertinent parts of the research enough so the audience can understand it. We’ve been giving him a lot of shouts lately, but this is an area in which I think Anthony Colpo shines. He explains the studies’ design and results thoroughly. We try our best to do the same.

    Keep reading and you’ll be a critical eye in no time! :) You can always ask for our help as well.

    As I write this from my mechanic’s waiting area, he sat down to tell me what I need. Here I am trusting a professional…

  7. Lori says:

    Hi Matt & Jeff,
    I absolutely love your book! I just want to say, I really appreciate your dedication and hard work to ‘get it right’. But after being in this line of work (health & health care) for a very long time, I’ve seen the re-hashing of good and bad info and studies since before the days of Jack La Lanne.
    It is just not realistic to think that every person, even an educated person who researches something can check and re-check and write a book when they just want to look up something for a client. (I agree with Chris and that’s why I recommend your book :)

    Anthony Colpo’s book was one of the first books I read, outside of traditional ones, after my my mother dropped dead on a treadmill from a heart attack back in 2007. I have been following him since. I’ve found that the best research I learned came from my grandmother when it comes to eating right and being healthy. And I use this reference often. People love it! ‘Cause they can relate. (My grandmother lived 20 years longer than my mother did, there goes the hereditary theory, that’s all the proof I need)
    Thanks for all you do.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>