Wednesday, October 31, 2007

The Problem with Experts

If we all worked on the assumption that what is accepted as true is really true, there would be little hope of advance. - Orville Wright

Education can be a wonderful thing as long as it is not just pepetuating false assumptions.

Dr Ioannidis began his study by reviewing 49 research articles printed in widely read medical journals between 1990 and 2003. Each of these articles had been cited by other scientists in their own papers 1,000 times or more. However, 14 of them—almost a third—were later refuted by other work. Having established the reality of his point, he then designed a mathematical model that tried to take into account and quantify sources of error. Again, these are well known in the field.

One is an unsophisticated reliance on “statistical significance”. To qualify a result as statistically significant, the probability that it is the result of pure coincidence should be smaler than 1:20. But, as Dr Ioannidis points out, adhering to this standard means that simply examining 20 different hypotheses at random is likely to give you one statistically significant result. In fields where thousands of possibilities have to be examined, such as the search for genes that contribute to a particular disease, many seemingly meaningful results are bound to be wrong just by chance.

In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.

When Dr Ioannidis ran the numbers through a simulation, his model predicted that even a large, well-designed study with little researcher bias has only an 85% chance of being right. An underpowered, poorly performed drug trial with researcher bias has but a 17% chance of producing true conclusions. Overall, the model predicts that more than 50% of all published research is probably wrong.
- Read whole article here.

Because the educated professionals have invested a lot of time and money in studying current thinking, they are reluctant to discard any of it. This is despite the fact that modern medicine really has very few accomplishments to take credit for and diseases that have been around for centuries will probably be attacking future generations.

This is an incomplete list of incurable diseases:
Acute Lymphocytic Leukemia
Acute Myeloid Leukemia
Acquired Immune Defficiency Syndrome (AIDS) see also HIV
Adrenoleukodystrophy (ALD)
Alzheimer disease
Amyotrophic Lateral Sclerosis (ALS,Lou Gehrig's disease)
Attention Deficit Disorder (ADD)
Avian Influenza
Batten disease (Juvenile Neuronal Ceroid Lipofuscinosis)
Bovine Spongiform Encephalopathy (BSE, "Mad Cow" disease)
Bipolar disorder (Manic-depression)
Chronic lymphocytic leukemia (CLL)
Chronic myeloid leukemia (CML)
Creutzfeldt-Jakob Disease (CJD) see also Bovine Spongiform Encephalopathy and Kuru
Common Cold
chicken pox aka Herpes Zoster aka varicella-zoster aka Shingles
Currarino Triad
Cystic Fibrosis
Dysmyelogenic leukodystrophy (DMD a.k.a. Alexander disease)
Emphysema (C.O.P.D.)
Farber disease
Fatal Familial Insomnia
Fibrodysplasia ossificans progressiva
Galactosialidosis (Goldberg syndrome)
Gaucher disease
GM1 Gangliodsidosis
Hairy cell leukemia
Herpes Zoster aka varicella-zoster aka chicken pox
Hopeless Astrocytoma (Brain Cancer)
Hurler syndrome (includes Hurler-Scheie)
Hunter syndrome
Infantile Neuronal Ceroid Lipofuscinosis
Krabbe disease
Kuru see Creutzfeldt-Jakob Disease
Lymphocytic Lymphomas
Hodgkin Lymphoma
Non-Hodgkin lymphoma
Small lymphocytic lymphoma
Metachromatic Leukodystrophy (MLD)
Morquio A
Mucolipidosis II (I-Cell Disease)
Mucolipidosis IV
Multiple sclerosis (MS)
Niemann-Pick disease, Types A and B
Pompe disease
Pseudomyxoma peritonei
Salla disease
Sandhoff disease
Sanfilippo A
Scheie syndrome
Schindler disease
Sialidosis (Mucolipidosis I)
Sly syndrome
Spreading Adenocarcinoma
Spreading Melanoma
Takayasu's arteritis (Pulseless Disease)
Tay-Sachs disease
Wolman disease


At 6:17 AM, Blogger Mark said...

"Orville Wright" the wright brothers were true scientists as they did not accidentally find the method to fly.
vulcanization(of rubber) dynamite penicillin are reportedly accidental discoveries.

At 6:37 AM, Blogger Flimsy Sanity said...

I read a kids book on Goodyear and I thought he discovered how to make rubber (but his partner tried to steal the idea from him). Of course, the real current bonanza is Viagra which was a side effect of the drug they developed for high blood pressure.

The doctor that discovered that ulcers were the result of a bacteria struggled for ten years to try and get people to listen to him.

At 7:09 AM, Blogger liberated psych said...

What a mixed message. "Think Outside the Box" and no one will take you seriously unless your work is loaded with references.


Post a Comment

Links to this post:

Create a Link

<< Home