Next is an example of how reading the original article can help assess the validity of an essay in a non-medical journal. This method will also be helpful to read essays on scientific subjects in any journal or on the internet. Many journalists do not have the training or time to evaluate the accuracy and validity of scientific papers they review. They go for headlines, breakthroughs, press briefings and capsule summaries.
The essay I was assessing for its validity was on
meditation. The conclusion of that essay was that meditation may not be helpful
to many and may lead a few into more selfishness. This was written by a reporter
based on an article published in a scientific journal.
It is easy to just accept that report and keep passing on
that information, because it is interesting and may be true. If you are already
inclined towards that point of view, you are more likely to accept it without
any verification. But, before I accepted the reporter’s conclusion, I wanted to
go to the original scientific report and read it for myself. But why?
First, most reporters are not trained in reading and
understanding scientific reports. Most of them are not familiar with
statistical concepts. They report on the conclusions, most of the time without
knowing for certain whether the conclusion was valid.
Fortunately, most journals give us a reference to the source
and several on-line publications even give a link to the source article. This
is what I did and found the original using the link provided.
The questions I wanted to get answers for were: What was the
hypothesis these authors were testing? How rigorous was their methods and
procedures? Given the methods they used, do I think they could have tested
their hypothesis correctly/ What was the data? What was their analytical
methods? What were the limitations of the study such as selection of subjects,
accounting for the biological and methodological variables? Is their conclusion
valid given the data they got? If it is, how generally can this conclusion be
applied? Is it applicable to just a limited group of individuals similar to the
subjects in the study or to a wider group?
With these questions in mind, when I looked at the methods
the authors used, here is what I found. (All the italicized sections within
parenthesis are “cut-and-paste” from the original article)
The author's question (hypothesis) was: "Does mindfulness makes m people more generous and cooperative, or is it possible that it can actually make people more selfish?"
First, in biology and behavioral sciences, definition of words such as mindfulness, cooperative attitude and selfishness are essential. When you then try to quantitate these qualities which we cannot even define consistently, another layer of difficulty creeps in. In addition, there are several scales to measure behaviors such as cooperation mindfulness and selfishness. Which one of these scales did the authors choose and why?
Their hypothesis was that mindfulness which developed in societies in the orient which emphasize interdependence and cooperation may not work to the same level in the western societies which emphasize individuality and autonomy. The authors said that “Individuals with independent self-construals tend to act in ways that are consistent with goals of autonomy, separateness, and self-maximization; whereas those with interdependent self-construals tend to value the well-being of other group members, relationships, and interpersonal harmony (Gardner et al., 1999; Holland et al., 2004)”.
In addition to centuries of observation, recent studies have
documented that mindfulness helps us learn to avoid distractions and develop
sustained attention on the task at hand. Sustained attention helps develop and
hone one’s skills. This is one reason, mindfulness is taught, to help develop
sustained attention to the task at hand. One can become great researcher
by being good at focusing on a research question. One can get good at prayers
without letting the mind wander. So can a thief who can get good at being a
thief, if that is how he wants to use his mindfulness training.
That is the problem when meditation is practiced out of
context and without its inherent spiritual component.
In the Method section, we find that the authors used 366 undergraduate students from one University who were part of another study on the pro-social behavior of students. Therefore, the results of the study may or may not be applicable to other age-groups. Fortunately, the sample included males and females and also a mixture of whites, Afro-Americans, Asians and those of mixed race. Therefore, the sample population is a good one.
In
describing the procedure, the authors use the word “meditation manipulation”
and use this designation several times throughout the article. This
suggests that the authors were biased against it even at the start of the
research.
All the participants “completed measures of personality, trust, and prosocial tendencies that were not the focus of this investigation and then were randomly assigned to a meditation condition: either mindfulness meditation or a meditation control (mind wandering)”. The test subjects were given instructions on mindful breathing. “Both the mindful breathing and meditation control instructions were presented over the course of a 15-minute meditation period”. Members of the control group were asked to “Use the time to let your mind wander and think freely without needing to focus hard on anything in particular.”
This
is the biggest weakness of this study. How can anyone learn mindful meditation
in a single short session?
The authors measured: Self-Construal using the Relational Interdependent Self Construal scale; Compassion using the Cameron and Payne’s compassion scale and Prosocial Behavior using the number of envelopes participants stuffed. The last one is a strange way to study pro-social behavior.
The
participants were asked to stuff envelopes of donation for a worthy cause soon
after they finish reading an article about that cause in a newspaper article.
The number of envelopes they stuffed was considered a measure of whether the
participants became “caring” and more altruistic following the meditation. In
the words of the authors: “Following the meditation manipulation,
participants read an article from a local newspaper, ostensibly randomly chosen
to assess how meditation affects information processing”.
How can anyone measure a behavioral change immediately after a 15-minute meditation and how valid is it even if it shows changes in measurement? It takes many years of practice, if at all, to develop an open mind or to become compassionate.
The authors conclude by saying that the comparison between the test group and the control group showed a difference. They also said this score correlated with the score they obtained in the scale for selfishness and selflessness.
How
can anyone measure a behavioral change immediately after a 15-minute meditation
and how valid is it even if it shows changes in measurement? It takes many
years of practice, if at all, to develop an open mind or to become
compassionate.
Simple logic tells me that this study was full of flaws in methodology. Therefore, the conclusions of this study are not valid.
I
hope this exercise gives a glimpse of how I read scientific essays whether they
are published in non-medical journals or in medical journals.